Commit Graph
22978 Commits
Author SHA1 Message Date
Jugal Bhatt e2cb701fd9 added changes to mcp url wrapping
add mcp server url wrapping

Add mcp url wrapping
2025-07-01 11:10:45 -07:00
raz-alonandGitHub 0dd50ea336 Fix: correct user_id validation logic in Anthropic… (#11432)
* fix(transformation.py): correct user_id validation logic in AnthropicConfig

* fix conflict
2025-07-01 08:17:14 -07:00
Krrish Dholakia 7ec1def253 bump: version 1.73.6 → 1.73.7 2025-06-30 22:00:25 -07:00
Krish DholakiaandGitHub 49ed3065f6 VertexAI Anthropic - streaming cost tracking w/ prompt caching fixes (#12188)
* fix(rebuild-usage-object---ensure-cache_tokens-is-set): Ensures cache tokens is correctly set

Fixes https://github.com/BerriAI/litellm/issues/12149

* test(test_stream_chunk_builder_utils.py): add unit test to ensure cached tokens is part of stream chunk builder

Ensures standardized values are used
2025-06-30 22:00:10 -07:00
Ishaan Jaff 847c4514c4 test fix - test_anthropic_messages_passthrough.py 2025-06-30 21:56:31 -07:00
Krish DholakiaandGitHub ee9dd158dd Fix - handle empty config.yaml + Fix gemini /models - replace models/ as expected, instead of using 'strip' (#12189)
* fix(proxy_server.py): handle empty config yaml

Fixes https://github.com/BerriAI/litellm/issues/12163

* fix(gemini/common_utils.py): replace models/ as expected, instead of using 'strip'

Fixes https://github.com/BerriAI/litellm/issues/12160

* fix(anthropic/experimental_pass_through/messages/transformation.py): check for env var when selecting api key

* docs(config_settings.md): add api key to docs
2025-06-30 21:56:03 -07:00
Krrish Dholakia 5188c67c6c docs: update docs 2025-06-30 21:39:42 -07:00
Guanghua DingandGitHub e6111f032b fix - support anthropic_messages call types and add max tokens validation (#12162) 2025-06-30 20:19:34 -07:00
Young HanandGitHub 042f6b187e [Bug Fix] Fix Error code: 307 for LlamaAPI Streaming Chat (#11946)
* fix: add follow_redirects to avoid 307 error code

* feat: add error 307 test case

* feat: add follow_redirects=True to AsyncClient
2025-06-30 16:52:42 -07:00
Ishaan JaffandGitHub d65a9fdcc7 [Bug Fix] Using /messages with lowest latency routing (#12180)
* add validate_anthropic_api_metadata

* fixes for lowest latency deployment

* add _select_metadata_field

* test_anthropic_messages_litellm_router_latency_metadata_tracking
2025-06-30 15:57:19 -07:00
Krzysztof SzafranekandGitHub 2e467a6c99 Use the -d flag in docs instead of -D (#12179) 2025-06-30 15:25:42 -07:00
Ishaan JaffandGitHub c1d495f09e [Bug Fix] Allow passing litellm_params when using generateContent API endpoint (#12177)
* add _add_generic_litellm_params_to_request

* fix type

* fix: setup_result.litellm_params

* fix passing litellm params

* test_api_base_and_api_key_passthrough

* fixes for passing litellm params

* fix linting errors
2025-06-30 15:17:46 -07:00
Ishaan JaffandGitHub d727d63a81 [Feat] Add new AWS SQS Logging Integration (#12176)
* add aws_sqs

* add sqs controls

* add SQS to registry

* fix url lib parse

* fixes AWS SQS

* test_async_sqs_logger_flush

* fix test

* fix SQS logger auth

* add AWS SQS

* add aws sqs

* docs logging

* test_async_sqs_logger_flush

* test_async_sqs_logger_flush

* add SQS logger

* update SQS logging

* use constants for SQS
2025-06-30 14:02:49 -07:00
Krrish Dholakia 4978d7ad00 docs(lakera_ai.md): detail supported params 2025-06-30 13:18:49 -07:00
Cole McIntoshandGitHub a7594196cd fix: support Cursor IDE tool_choice format {"type": "auto"} (#12168)
* fix: support Cursor IDE tool_choice format {"type": "auto"}

- Update validate_chat_completion_tool_choice to normalize {"type": "auto"} to "auto"
- Handles Cursor IDE sending non-standard tool_choice format
- Add comprehensive tests for tool choice validation

Fixes #12098

* fix: return full tool_choice object for Cursor IDE format

Based on PR feedback, updated validate_chat_completion_tool_choice to return
the full tool_choice dictionary instead of just extracting the type string.
This maintains consistency with downstream code that expects the full object
structure.

- Changed behavior: {"type": "auto"} now returns {"type": "auto"} instead of "auto"
- Updated tests to reflect the new expected behavior
- Ensures compatibility with code that passes tool_choice to optional_params

Addresses feedback from PR #12168
2025-06-30 12:39:58 -07:00
Ishaan Jaff 4bb732d30f docs slack support 2025-06-30 10:57:10 -07:00
Kobi KadoshandGitHub cff0030d05 Update management_cli.md (#12157)
add quotes to the uv tool install command the current will fail.
2025-06-30 10:54:16 -07:00
Ishaan Jaff b7cb05114c docs add slack support 2025-06-30 10:45:37 -07:00
Ishaan Jaff 6b464248c7 chore - add slack support link 2025-06-30 10:23:53 -07:00
Krrish Dholakia f76375b651 docs(index.md): add batch api cost tracking to docs v1.73.6.rc.1 2025-06-28 22:25:15 -07:00
Krrish Dholakia 1df5c160c3 docs(index.md): add more hyperlinks to docs 2025-06-28 22:22:10 -07:00
Krrish Dholakia 7cc2ce0cf8 docs: index.md
release note cleanup
2025-06-28 22:22:10 -07:00
Ishaan Jaff 79ffb8a89e docs gemini cli x litellm 2025-06-28 22:06:42 -07:00
Krish DholakiaandGitHub cec7e4941a UI QA Fixes - prevent team model reset on model add + return team-only models on /v2/model/info + render team member budget correctly (#12144)
* fix(team_endpoints.py): prevent overwriting current list of team models on new model add

* fix(networking.tsx): fix default proxy base url

* fix(proxy_server.py): include team only models when retrieving all deployments on `/v2/model/info` helper util

ensures team only models are shown to user

* fix(router.py): check model name by team public model name when team id given

Fixes issue where team member could not see team only models when clicking into that team on `Models + Endpoints`

* fix(team_member_view.tsx): fix rendering team member budget, when budget is set

* test: update tests

* test: update unit test
2025-06-28 22:04:49 -07:00
Ishaan Jaff 7da4e2197d Benefits of using gemini-cli with LiteLLM 2025-06-28 21:59:07 -07:00
Ishaan Jaff 50dd9980dc docs - update release notes 2025-06-28 21:56:58 -07:00
Krish DholakiaandGitHub f7af8902b0 /v1/messages - Remove hardcoded model name on streaming + Tags - enable setting custom header tags (#12131)
* fix(anthropic/experimental_pass_through): use given model name when returning streaming chunks

don't harcode model name on streaming

confusing for user

* fix(anthropic/streaming_iterator.py): remove scope of import

* feat(litellm_logging.py): allow admin to specify additional headers for using as spend tags

Closes https://github.com/BerriAI/litellm/issues/12129

* test(test_litellm_logging.py): add unit tests

* feat(openweb_ui.md): add custom tag tutorial to docs

* docs(cost_tracking.md): add tag based usage UI screenshot

* test: update test

* fix: fix import
2025-06-28 21:49:35 -07:00
Krrish Dholakia 123631c93d docs(index.md): update release note with cleaner table for updated models 2025-06-28 21:48:24 -07:00
Ishaan Jaff a021e7cb9d bump poetry 2025-06-28 21:16:52 -07:00
Ishaan Jaff 973520255b fix for o-series param checks 2025-06-28 15:13:54 -07:00
Ishaan Jaff 04f5ccfee9 fix - revert list team changes 2025-06-28 15:09:36 -07:00
Ishaan Jaff ca8910496e fix imports 2025-06-28 15:01:47 -07:00
Ishaan JaffandGitHub 0c19414b36 [️ Python SDK import] - reduce python sdk import time by .3s (#12140)
* use 1 file for KeyManagementSystem

* move key management settings

* fix import locs

* test_proxy_types_not_imported

* test the import loc

* fix import item

* fix imports

* fix import loc

* fix imports
2025-06-28 14:57:10 -07:00
Ishaan JaffandGitHub 79a8b1a953 Revert "🧹 Refactor init.py to use a model registry (#12138)" (#12141)
This reverts commit f93326a214.
2025-06-28 14:56:39 -07:00
Ishaan Jaff 6b623f9c98 test whitelisted models 2025-06-28 14:46:16 -07:00
Ishaan Jaff 092e3591f5 fix import loc 2025-06-28 14:27:12 -07:00
Ishaan JaffandGitHub f93326a214 🧹 Refactor init.py to use a model registry (#12138)
* fix - refactor init to use a registry

* # noqa: PLR0915
2025-06-28 13:54:54 -07:00
Krrish Dholakia e77eb82cd6 docs(index.md): initial pre-release note 2025-06-28 13:24:19 -07:00
Ishaan JaffandGitHub 9b4f26faff [️ Python SDK Import] - 2 second faster import times (#12135)
* speedup - move wb logger to conditional

* fix import path
2025-06-28 12:40:04 -07:00
Ishaan Jaff a83f8f3de6 fix - using on python 3.9 v1.73.6-nightly 2025-06-28 12:03:35 -07:00
Guanghui LiuandGitHub 27849359a9 fix example config.yaml in claude code tutorial (#12133) v1.73.6.rc-draft 2025-06-28 11:07:48 -07:00
Ishaan Jaff 8b73716a66 add docs 2025-06-28 09:26:12 -07:00
Ishaan Jaff 59aaef07ef fix code qa checks 2025-06-28 09:23:02 -07:00
Ishaan Jaff 1897abb147 bump: version 1.73.5 → 1.73.6 2025-06-28 09:20:23 -07:00
Ishaan Jaff fe8250be18 bump: version 1.73.4 → 1.73.5 2025-06-28 09:19:55 -07:00
Ishaan JaffandGitHub cfc07f361b [Bug Fix] Invite links email should contain the correct invite id (#12130)
* use common helper create_invitation_for_user

* use common util in proxy

* fix create_invitation_for_user

* refactor base email

* test_get_invitation_link_creates_new_when_none_exist

* fix code QA checks
2025-06-28 09:19:32 -07:00
tanjiroandGitHub edc84b6b6d Update conditional_public_model_name.tsx (#11713) 2025-06-28 08:06:01 -07:00
Krish DholakiaandGitHub d895ae827d Litellm batch api background cost calc (#12125)
* feat(check_batch_cost.py): emit spend log on successful request

ensures cost tracked for batch requests

* feat(proxy_server.py): add background job to poll completed batch jobs

used for calculating cost for batch jobs

* fix(proxy_server.py): run batch cost tracking job every hour

batch jobs take time to complete, no need to run every few seconds

* feat(proxy_server.py): run batch cost tracking job every hour
2025-06-27 21:46:25 -07:00
codeugarandGitHub 06c86d6130 Update model_prices_and_context_window.json (#11972)
add
--
"deepseek/deepseek-r1": {
        "max_tokens": 8192,
        "max_input_tokens": 65536,
        "max_output_tokens": 8192,
        "input_cost_per_token": 5.5e-07,
        "input_cost_per_token_cache_hit": 1.4e-07,
        "output_cost_per_token": 2.19e-06,
        "litellm_provider": "deepseek",
        "mode": "chat",
        "supports_function_calling": true,
        "supports_assistant_prefill": true,
        "supports_tool_choice": true,
        "supports_reasoning": true,
        "supports_prompt_caching": true
    },
    "deepseek/deepseek-v3": {
        "max_tokens": 8192,
        "max_input_tokens": 65536,
        "max_output_tokens": 8192,
        "input_cost_per_token": 2.7e-07,
        "input_cost_per_token_cache_hit": 7e-08,
        "cache_read_input_token_cost": 7e-08,
        "cache_creation_input_token_cost": 0.0,
        "output_cost_per_token": 1.1e-06,
        "litellm_provider": "deepseek",
        "mode": "chat",
        "supports_function_calling": true,
        "supports_assistant_prefill": true,
        "supports_tool_choice": true,
        "supports_prompt_caching": true
    },
--
tencent custom deploy deepseek named "deepseek-r1" and "deepseek-v3".
Thanks very much !
2025-06-27 21:36:59 -07:00
Daniele ScasciafratteandGitHub 24816fe470 Remove duplicated entry in logs on key cache update (#12032)
As per title and you can see the code is a duplicate
2025-06-27 21:33:11 -07:00