* fix(streaming_handler.py): maintain same 'created' across all chunks
Fixes https://github.com/BerriAI/litellm/issues/11437
* test: add unit test to ensure created is always the same across all chunks
* fix(types/utils.py): set a tool call id, if missing in delta tool call
Ensures stream chunk builder can reconstruct tool calls correctly
Fixes https://github.com/BerriAI/litellm/issues/11262
* fix(responses/transformation.py): support passing mcp server tool call to anthropic
allows switching between openai and anthropic for mcp tool calling
* fix(ollama/chat/transformation.py): set tool call id's when missing
* fix(onboarding_link.tsx): fix adding ui/invitation id
* fix(onboarding_link.tsx): update invitation link function to handle w/ and w/out custom server path cases
* fix(model_checks.py): ensure team only models returned when all proxy models set for team
* feat(anthropic/): initial commit adding working mcp tool call support
pass in mcp tool via `tools` and litellm will handle translating it to the right anthropic param
* feat(anthropic/): map openai mcp tool to anthropic mcp tool
allows usage within responses api
* fix(databricks/transformation.py): fix databricks linting error
* test(test_anthropic_chat_transformation.py): fix test
* test: update test
* fix(anthropic/chat/transformation.py): add dummy tool call
* refactor: comment out circuit breaker
causes incorrect rate limiting in high traffic
* fix(base_routing_strategy.py): don't reset value if redis val is lower than current in-memory value
Fixes issue where redis might be trailing in-memory value
* fix(parallel_request_limiter_v2.py): if in-memory higher than redis, don't reset value; add previous slot keys to redis increment to correctly 'get' them
* fix(parallel_request_limiter_v3.py): v3 implementation of parallel request limiter
does not use background redis syncing - increments redis in call
simplify rate limiting logic, to improve accuracy
* fix: fix ruff errors
* fix(parallel_request_limiter_v3.py): don't decrement limit on post call success - causes double decrements
* fix(parallel_request_limiter_v3.py): working accurate multi-instance logic
ensured just 100 requests allowed on 100 users, 10 ramp up, 100 rpm limit key, 2 instances
* fix(parallel_request_limiter_v3.py): working accurate rate limiting with time window resets
allows rate limiting to work across multiple windows
* test: add unit tests for v3 rate limiter
* fix(parallel_request_limiter_v3.py): return window value into in-memory cache
allows in-memory cache checks to be used correctly
* refactor(parallel_request_limiter_v3.py): refactor rate limiting to work for multiple window/counter key pairs
enables using for user/team/model rate limiting
* feat(parallel_request_limiter_v3.py): working rate limiting, across key/user/team/end-user
* fix(parallel_request_limiter_v3.py): add model specific rate limiting
* fix(parallel_request_limiter_v3.py): ignore if no rate limits set
skip unecessary rate limit checks - if no limits set
* fix(parallel_request_limiter_v3.py): initial commit bringing token rate limits back
* fix(parallel_request_limiter_v3.py): increment by value in list + update assertions to handle tokens + max parallel requests
* test(parallel_request_limiter_v3.py): more testing
* fix(parallel_request_limiter.py): working in-memory cache limiter
* fix(redis_cache.py): ignore linting error - use safe hasattr
* fix(parallel_request_limiter_v3.py): fix linting error
* refactor: remove redundant parallel_Request_limiter_v2.py
old / inaccurate implementation
* test: update tests
* style: cleanup
* test: update test
* docs(config_settings.md): document new env var
* test(test_base_routing_strategy.py): update test
* Enhance proxy CLI with Rich formatting and improved user experience
- Integrated Rich library for better console output in `proxy_cli.py`, including version display, health check results, and test completion responses.
- Updated health check and test completion methods to provide progress indicators and formatted tables.
- Refactored feedback display in `proxy_server.py` to use Rich for a more visually appealing user interface.
- Adjusted tests in `test_proxy_cli.py` to mock console output instead of using print statements, ensuring compatibility with Rich formatting.
* fix linting error
* refactor(proxy_cli.py): simplify DB setup logging
- Removed progress indicators for IAM token generation and environment variable decryption to simplify the code.
- Consolidated the logic for generating the database URL and setting environment variables.
- Enhanced error handling for configuration loading and database setup, ensuring clearer feedback
* Update test-linting workflow to include proxy-dev dependencies in Poetry installation
* Enhance proxy server initialization with Rich console for improved model display. Added support for loading model parameters from environment variables and refined provider identification logic. Fallback to original print formatting if Rich is not available.
* Refactor feedback handling: Moved feedback message generation and custom warning display to utils.py. Enhanced feedback box with rich formatting and fallback to ASCII for environments without rich. Cleaned up proxy_server.py by removing obsolete code.
* fix linting error
* Refactor model initialization display: Moved model initialization logic to a new utility function `display_model_initialization` for improved readability and maintainability. Enhanced model provider extraction with a dedicated function. Fallback to basic logging if Rich console is unavailable.
* Refactor model provider extraction: Replace the `_extract_provider_from_model` function with a more robust approach using `get_llm_provider`. Implement fallback logic for provider identification and improve error handling. Ensure compatibility with Rich console for model initialization display.
* Refactor get_end_user_id_from_request_body to support user ID retrieval from custom headers and multiple request body formats. Enhance tests to cover various scenarios including header precedence and fallback mechanisms.
* Refactor get_end_user_id_from_request_body function to accept request_body as the first parameter, improving clarity and flexibility. Update tests for compatibility and add new cases to ensure correct functionality across various request body formats.
* Update _user_api_key_auth_builder and user_api_key_auth to pass request object to get_end_user_id_from_request_body, enhancing user ID retrieval from request data.
* refactor(auth_utils.py): update get_end_user_id_from_request_body to accept request_headers instead of request, and adjust related function calls in user_api_key_auth and tests
* refactor(tests): update mock request handling in LLM pass-through endpoint tests
- Replaced the Request object with a Mock for better flexibility in testing.
- Enhanced mock setup to include user API key handling and virtual key retrieval.
- Updated test calls to reflect changes in mock request structure and added necessary patches for new dependencies.
* refactor(vertex_and_google_ai_studio_gemini.py): remove redundant variable declaration for url_context_metadata, linting error
* Handle file content type transformation in responses api (#11310)
* Handle file content type transformation in responses api
* change to use input_file
* -
* TestLiteLLMCompletionResponsesConfig
* test: TestLiteLLMCompletionResponsesConfig
* fix: fix linting
---------
Co-authored-by: Jayme Gordon <jayme_gordon@icloud.com>
* fix(vertex_and_google_ai_studio_gemini.py): add web search request tracking
Enables cost calculation for google web search
* fix(vertex_and_gemini): use common processing logic across stream / non-stream calls
* fix(vertex_And_google_ai_studio_Gemini.py): fix initial choice
* fix: fix linting error
* fix: add initial support for google search cost tracking
* fix(tool_call_cost_tracking.py): working tool cost tracking for gemini
* fix(vertex_ai/gemini/cost_calculator.py): add google web search tool cost tracking for vertex ai
Closes LIT-210
* fix: fix check
* build(model_prices_and_context_window.json): fix amazon nova max output tokens
Closes https://github.com/BerriAI/litellm/issues/11441
* fix: fix ruff check
* Add tests for function calling support in LiteLLM proxy models
- Introduced a new test script `test_proxy_function_calling.py` to validate function calling capabilities for both direct and proxied models.
- Created a comprehensive test suite in `tests/litellm_utils_tests/test_proxy_function_calling.py` using pytest, covering various model configurations and edge cases.
- Implemented parameterized tests to ensure consistency between direct and proxied model function calling support.
- Added tests for specific proxy models, edge cases, and import verification for the `supports_function_calling` function.
- Included a demonstration test to highlight the current issue with proxy model resolution.
* feat: add fallback handling for litellm_proxy models in model info retrieval
* feat: enhance proxy function calling tests with custom model name handling and documentation
* fix: add type ignore comments for custom logger callback initialization
* fix: remove styling diff
* fix: style
* fix(utils.py): remove outdated comment regarding litellm_proxy models
* feat(utils.py): add proxy model handling for underlying model extraction
* feat(utils.py): enhance model name handling for litellm_proxy integration
* refactor(utils.py): remove unused _handle_proxy_model_names function
* fix: using litellm with claude code bedrock
* fix: usage for bedrock with /messages
* fix: bedrock_sse_wrapper
* tests: test for test_chunk_parser_usage_transformation
* test fix
Introduce the ExceptionCheckers class to encapsulate methods for checking error conditions in exception strings, specifically for identifying rate limit errors. Update the Fireworks AI exception mapping tests to cover various scenarios, including standard 429 errors and text-based detection, ensuring accurate mapping to RateLimitError. Enhance test coverage for both positive and negative cases of rate limit detection.
* fix(huggingface): use get() instead of pop() for input_type parameter
Fixes embedding generation for HuggingFace models where input_type override
is required (e.g. BAAI/bge-m3). The pop() method was mutating optional_params
and removing input_type before downstream functions could access it.
* Add unit tests to catch regression
* Move tests around
* fix(convert_dict_to_response.py): handle None values in usage field for gpt-image-1
* test: add tests for handling None and partial values in usage fields for gpt-image-1 responses
- Renamed SSOSettingsResponse to inherit from a new base class SettingsResponse for better structure.
- Introduced InternalUserSettingsResponse and DefaultTeamSettingsResponse models for internal user and default team settings.
- Updated endpoint responses to use field_schema instead of schema for consistency.
- Enhanced test cases to validate the new response structure and ensure proper functionality of SSO settings.
* fix(anthropic/): fix regression when passing file url's to the 'file_id' parameter
add test and ensure anthropic file url's are correctly sent as 'document' blocks
* fix(vertex_and_google_ai_studio.py): Use same usage calculation function as non-streaming
Closes https://github.com/BerriAI/litellm/issues/10667
* test(test_vertex_and_google_ai_studio_gemini.py): update test
* fix(ui_sso.py): update user as proxy admin in db table, when checking for proxy_admin_id
Fixes issue where existing internal user, unable to make calls when set as proxy admin id
* fix(utils.py): fix custom base path
* Fixes issue with team_endpoints on member budget update
* refactored location of budget membership fix
* added test for _upsert_budget_membership func
