Commit Graph
23230 Commits
Author SHA1 Message Date
Ishaan JaffandGitHub 9241fca2f5 Fix: Adds support for choosing the default region based on where the model is available (#11566)
* fix: vtx default region for global only models

* track gemini-2.5-pro-preview-05-06

* fix is_global_only_vertex_model

* test_is_global_only_vertex_model

* test_get_vertex_region_global_only_model

* fix json format

* fix get_supported_regions
2025-06-09 18:29:44 -07:00
Krrish Dholakia 230dd70604 docs(data_security.md): data_security.md
update to indicate litellm does have soc 2 type2
2025-06-09 17:53:11 -07:00
Marc AbramowitzandGitHub 3bd36238dc Simplify management_cli.md CLI docs (#10799)
Offer just 1 easy option for installing with `uv tool` so people can get
started quickly
2025-06-09 17:38:06 -07:00
Marc AbramowitzandGitHub ad9b79259f Make all commands show server URL (#10801)
* Make all commands show server URL

so users know which server they're accessing

* Write msg to stderr & only if it's a tty
2025-06-09 17:37:40 -07:00
cd8ec4556f [Feat] Add reasoning_effort support for perplexity models (#11562)
* fix: add reasoning_effort for pplx

* docs pplx reasoning

* [tests] add mock tests for pplx reasoning (#11564)

* Add tests for Perplexity reasoning models and effort parameter

* tests perplexity reasoning effort

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>

* test pplx reasoning effort

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-06-09 17:07:31 -07:00
9b87fde633 [Bug fix]: aiohttp fixes for transfer encoding error on aiohttp transport (#11561)
* fix: fixes for transfer encoding error on aiohttp transport

* Update tests/test_litellm/llms/custom_httpx/test_aiohttp_transport.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-06-09 16:14:34 -07:00
Cole McIntoshandGitHub 322aefb97d Update documentation for configuring web search options in config.yaml (#11537)
* Update web_search.md to include new supported providers and models, enhance web search options, and improve documentation for using web search with various AI models.

* Update LiteLLM version in web_search.md to reflect the latest stable release.

* Fix formatting in web_search.md for model declaration consistency.

* docs(web_search.md): add configuration options for web search in config.yaml

This update introduces sections for setting default and custom web search options in the proxy config file, including examples for different models and context sizes. A note clarifies that users can override these settings in API requests.
2025-06-09 15:19:57 -07:00
fengbohelloandGitHub 400acd4297 docs: fix database_url config description (#11547) 2025-06-09 15:18:37 -07:00
Cole McIntoshandGitHub abe4c8fe4c feat: add gpt-4o-audio-preview model configuration to model_prices_and_context_window.json (#11560) 2025-06-09 14:56:36 -07:00
Cole McIntoshandGitHub 7f44d1a2f8 Merge pull request #11538 from colesmcintosh/feat/add-cursor-to-gitignore
feat: add .cursor to .gitignore
2025-06-08 15:50:09 -06:00
Cole McIntosh f1e3609296 feat: add .cursor to .gitignore 2025-06-08 14:35:50 -06:00
LaurienandGitHub 0c50f8bcc9 Update enduser spend and budget reset date based on budget duration (#8460) 2025-06-08 08:39:14 -07:00
Krrish Dholakia 9f15907697 bump: version 1.72.2 → 1.72.3 2025-06-07 20:50:33 -07:00
Krish DholakiaandGitHub 8dd8615a54 Ensure consistent 'created' across all chunks + set tool call id for ollama streaming calls (#11528)
* fix(streaming_handler.py): maintain same 'created' across all chunks

Fixes https://github.com/BerriAI/litellm/issues/11437

* test: add unit test to ensure created is always the same across all chunks

* fix(types/utils.py): set a tool call id, if missing in delta tool call

Ensures stream chunk builder can reconstruct tool calls correctly

Fixes https://github.com/BerriAI/litellm/issues/11262

* fix(responses/transformation.py): support passing mcp server tool call to anthropic

allows switching between openai and anthropic for mcp tool calling

* fix(ollama/chat/transformation.py): set tool call id's when missing
2025-06-07 20:50:07 -07:00
Krish DholakiaandGitHub 2654d3b0b1 Support env var vertex credentials for passthrough + ignore space id on watsonx deployment (throws Json validation errors) (#11527)
* fix(router.py): support vertex credentials set in env var for passthrough

Closes https://github.com/BerriAI/litellm/issues/11245

* fix(watsonx/common_utils.py): do not pass space_id on watsonx deployment requests - raises Json validation error

Fixes https://github.com/BerriAI/litellm/issues/10941

* test: update unit test
2025-06-07 20:31:05 -07:00
Krrish Dholakia 7acaa08ad4 refactor(parallel_request_limiter_v3.py): remove redundant code - make it easier for users to understand what's happening 2025-06-07 19:52:36 -07:00
Krrish Dholakia 35bc2d52a3 docs(index.md): document rate limiting improvements 2025-06-07 19:51:26 -07:00
Krrish Dholakia ff5a52fe3f docs(index.md): reference anthropic mcp docs on release notes 2025-06-07 19:27:54 -07:00
Krrish Dholakia 483f835c65 docs(anthropic.md): document anthropic mcp tool calling support 2025-06-07 19:21:14 -07:00
Ishaan Jaff bb8a5e752e docs 1.72.2 notes 2025-06-07 18:32:51 -07:00
Ishaan Jaff 3f41b84408 docs add audit logs to release note 2025-06-07 18:27:27 -07:00
Ishaan Jaff 6f6f9bf58e docs cleanup 2025-06-07 17:22:40 -07:00
Ishaan JaffandGitHub 4fc92244b5 [Docs] v1.72.2.rc (#11519)
* v1-72-2.rc

* docs v1.72.2.rc

* docs 1.72.2.rc

* docs update

* docs Bug Fixes

* add TLDR section

* add table

* docs release notes

* docs v1-72-2-stable

* docs /v1/messages

* docs hf rerank
2025-06-07 17:20:15 -07:00
Ishaan Jaff 49a7833861 (ui) new build v1.72.2.rc 2025-06-07 16:15:05 -07:00
Ishaan Jaff a134842b49 ui fix order of cols on audit logs 2025-06-07 16:14:15 -07:00
Ishaan Jaff 38a3933385 ui new build 2025-06-07 15:48:08 -07:00
Ishaan Jaff 6e711756ac UI: add AuditLogsInfoMessage 2025-06-07 15:46:19 -07:00
Ishaan Jaff c79f7a3ad9 (ui) fix display of model table 2025-06-07 15:35:01 -07:00
Ishaan Jaff 4199060b18 (ui) fix, use chevron icon for expand 2025-06-07 15:33:34 -07:00
Krish DholakiaandGitHub 5d049eb742 fix(add_credentials_tab.tsx): filter for null values when adding credentials (#11525) 2025-06-07 15:32:19 -07:00
Ishaan Jaff 8b60b703f7 UI - fix col headers 2025-06-07 15:28:11 -07:00
Ishaan Jaff d8687b5d43 (ui) fix color on audit logs table 2025-06-07 15:28:11 -07:00
3351f0513e Update Anthropic unified docs with multi-provider examples and proxy usage (#11523)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-06-07 15:24:14 -07:00
Krish DholakiaandGitHub 17d2711cf4 UI - fix invitation link + ensure team models returned when team has 'all-proxy-models' + team only models (#11524)
* fix(onboarding_link.tsx): fix adding ui/invitation id

* fix(onboarding_link.tsx): update invitation link function to handle w/ and w/out custom server path cases

* fix(model_checks.py): ensure team only models returned when all proxy models set for team
2025-06-07 15:19:52 -07:00
Krish DholakiaandGitHub e3c66418b8 Litellm anthropic mcp support (#11474)
* feat(anthropic/): initial commit adding working mcp tool call support

pass in mcp tool via `tools` and litellm will handle translating it to the right anthropic param

* feat(anthropic/): map openai mcp tool to anthropic mcp tool

allows usage within responses api

* fix(databricks/transformation.py): fix databricks linting error

* test(test_anthropic_chat_transformation.py): fix test

* test: update test

* fix(anthropic/chat/transformation.py): add dummy tool call
2025-06-07 15:11:55 -07:00
Ishaan Jaff 440a6aa3d2 ui qa - allow copying on 0.0.0.0 urls 2025-06-07 13:56:39 -07:00
Krrish Dholakia 8554106a8c build(ui/): new ui build 2025-06-07 12:51:53 -07:00
Krish DholakiaandGitHub c42740a4b9 Simplify experimental multi-instance rate limiter - more accurate (#11424)
* refactor: comment out circuit breaker

causes incorrect rate limiting in high traffic

* fix(base_routing_strategy.py): don't reset value if redis val is lower than current in-memory value

Fixes issue where redis might be trailing in-memory value

* fix(parallel_request_limiter_v2.py): if in-memory higher than redis, don't reset value; add previous slot keys to redis increment to correctly 'get' them

* fix(parallel_request_limiter_v3.py): v3 implementation of parallel request limiter

does not use background redis syncing - increments redis in call

 simplify rate limiting logic, to improve accuracy

* fix: fix ruff errors

* fix(parallel_request_limiter_v3.py): don't decrement limit on post call success - causes double decrements

* fix(parallel_request_limiter_v3.py): working accurate multi-instance logic

ensured just 100 requests allowed on 100 users, 10 ramp up, 100 rpm limit key, 2 instances

* fix(parallel_request_limiter_v3.py): working accurate rate limiting with time window resets

allows rate limiting to work across multiple windows

* test: add unit tests for v3 rate limiter

* fix(parallel_request_limiter_v3.py): return window value into in-memory cache

allows in-memory cache checks to be used correctly

* refactor(parallel_request_limiter_v3.py): refactor rate limiting to work for multiple window/counter key pairs

enables using for user/team/model rate limiting

* feat(parallel_request_limiter_v3.py): working rate limiting, across key/user/team/end-user

* fix(parallel_request_limiter_v3.py): add model specific rate limiting

* fix(parallel_request_limiter_v3.py): ignore if no rate limits set

skip unecessary rate limit checks - if no limits set

* fix(parallel_request_limiter_v3.py): initial commit bringing token rate limits back

* fix(parallel_request_limiter_v3.py): increment by value in list + update assertions to handle tokens + max parallel requests

* test(parallel_request_limiter_v3.py): more testing

* fix(parallel_request_limiter.py): working in-memory cache limiter

* fix(redis_cache.py): ignore linting error - use safe hasattr

* fix(parallel_request_limiter_v3.py): fix linting error

* refactor: remove redundant parallel_Request_limiter_v2.py

old / inaccurate implementation

* test: update tests

* style: cleanup

* test: update test

* docs(config_settings.md): document new env var

* test(test_base_routing_strategy.py): update test
2025-06-07 11:10:55 -07:00
Cole McIntoshandGitHub 1d86fc84fe Update web search documentation for new provider support (xAI, VertexAI, Google AI Studio) (#11515)
* Update web_search.md to include new supported providers and models, enhance web search options, and improve documentation for using web search with various AI models.

* Update LiteLLM version in web_search.md to reflect the latest stable release.

* Fix formatting in web_search.md for model declaration consistency.
2025-06-07 09:13:03 -07:00
Krish DholakiaandGitHub bc7dd9fa6d Litellm dev 06 06 2025 p1 (#11496)
* fix(proxy/_types.py): add key masking to audit logs - prevent leaking sk- keys

* fix(navbar.tsx): fix getting image url when proxy base url is null
2025-06-07 09:12:16 -07:00
Tu VuandGitHub 3b7746a13b Update the correct test directory in contributing_code.md (#11511) 2025-06-07 07:35:01 -07:00
Ishaan Jaff 5299c4bb6e docs - stable release v1.72.0 2025-06-06 20:58:02 -07:00
Ishaan Jaff 18081cf250 test fix test_aaauser_personal_budgets v1.72.2-nightly 2025-06-06 20:55:27 -07:00
Ishaan Jaff bc835c6044 test_lm_studio_completion 2025-06-06 20:41:00 -07:00
Ishaan JaffandGitHub 362e358a77 [Feat] Allow using litellm.completion with /v1/messages API Spec (use gpt-4, gemini etc with claude code) (#11502)
* feat: add anthropic stream wrapper

* feat: add AnthropicExperimentalPassThroughConfig

* feat: working non streaming anthropic

* feat: working streaming anthropic-litellm bridge

* test - anthropic OpenAI bridge tests

* fix: add sync support for anthropic_messages

* fix: using is async check

* fix: ensure streams are SSE

* fix: imports

* fix code qa check

* fix: linting errors

* test_sync_openai_messages

* cleanup remove stash file
2025-06-06 20:35:53 -07:00
Tu VuandGitHub bb45844ad8 Update model version in deploy.md (#11506) 2025-06-06 20:35:14 -07:00
Tu VuandGitHub bc0e93e8a1 Remove retired version gpt-3.5 from configs.md (#11508) 2025-06-06 20:34:45 -07:00
Ishaan Jaff 08d6f3e142 Revert "Enhance proxy CLI with Rich formatting and improved user experience (#11420)"
This reverts commit 3b911ba1b2.
2025-06-06 17:55:45 -07:00
Ishaan Jaff 3be1dab1e1 Revert "fix don't require rich for litellm python SDK"
This reverts commit b76e7acbb8.
2025-06-06 17:55:36 -07:00
Ishaan Jaff b76e7acbb8 fix don't require rich for litellm python SDK 2025-06-06 17:47:17 -07:00