Commit Graph
2737 Commits
Author SHA1 Message Date
Krrish Dholakia b23e35f75e test: update tests 2025-08-23 12:09:19 -07:00
Krrish Dholakia 34ad646101 fix(auth_checks.py): handle both flows + add unit tests 2025-08-20 16:42:08 -07:00
Krrish Dholakia e5c3e006c2 fix(proxy_server.py): fix merge config + db router settings logic 2025-08-20 16:33:31 -07:00
Ishaan Jaff 12bae8fdda test_partner_models_httpx_streaming 2025-08-20 09:44:47 -07:00
Ishaan Jaff e69a895884 test_apply_default_settings 2025-08-20 08:53:38 -07:00
Krish DholakiaandGitHub 04feaf33c0 Merge pull request #13748 from hakasecurity/migrate-to-aim-new-firewall-api
Migrate to aim new firewall api
2025-08-19 22:32:22 -07:00
Krish DholakiaandGitHub be30bc68ae Merge pull request #13759 from kankute-sameer/litellm_feat_correct_cost_calculations
Add long context support for claude-4-sonnet
2025-08-19 22:30:25 -07:00
Krrish Dholakia 7d09375d52 fix: fix gpt-5-chat mappings 2025-08-19 22:21:00 -07:00
Ishaan JaffandGitHub 56ac778316 [Bug Fix] Bedrock KB - Using LiteLLM Managed Credentials for Query (#13787)
* fix: add get_credentials_for_vector_store

* test_search_uses_registry_credentials

* test_bedrock_search_with_credentials_managed_registry
2025-08-19 15:39:36 -07:00
Sameer Kankute 48622a4ee7 add cache above 200k keys in INTENDED_SCHEMA 2025-08-20 01:47:34 +05:30
Sameer KankuteandGitHub fa81b6c639 Update test_cost_calculator.py 2025-08-19 23:14:22 +05:30
Ishaan JaffandGitHub 195ea6515e [Feat] Datadog LLM Observability - Add support for tracing guardrail input/output (#13767)
* add guardrail information on DD LLM Obs

* test_guardrail_information_in_metadata
2025-08-19 10:26:25 -07:00
Sameer KankuteandGitHub b5f0a7b49b Merge branch 'main' into litellm_feat_correct_cost_calculations 2025-08-19 18:55:36 +05:30
Sameer Kankute d39b2e8888 Add test for long context cost calculation 2025-08-19 16:56:13 +05:30
drorbaron 6b78ade918 migrate to use new aim FW API 2025-08-19 12:20:19 +03:00
Krish DholakiaandGitHub 88e52c55d0 Merge pull request #13675 from colesmcintosh/fix/groq-streaming-encoding
Fix Groq streaming ASCII encoding issue
2025-08-18 23:00:00 -07:00
Krish DholakiaandGitHub 9dadd279a4 Merge pull request #13741 from BerriAI/litellm_dev_08_18_2025_p1
Refactor - forward model group headers - reuse same logic as global header forwarding
2025-08-18 22:58:39 -07:00
Krrish Dholakia 2c0520635d test: cleanup old tests 2025-08-18 22:58:29 -07:00
Krish DholakiaandGitHub 048f22b7ec Merge pull request #13742 from BerriAI/litellm_dev_08_18_2025_p2
Fix - gemini prompt caching cost calculation
2025-08-18 22:54:28 -07:00
Krish DholakiaandGitHub 3713c926c0 Merge pull request #13704 from michal-otmianowski/use-namespace-as-prefix-for-s3-cache
Use namespace as prefix for s3 cache
2025-08-18 22:37:18 -07:00
Krrish Dholakia f7f1a0d0b7 test: add unit test 2025-08-18 22:32:36 -07:00
Krrish Dholakia 2e16f2cb13 test: add unit tests 2025-08-18 21:19:43 -07:00
Krish DholakiaandGitHub 422447b7f1 Responses API - add default api version for openai responses api calls + Openrouter - fix claude-sonnet-4 on openrouter + Azure - Handle openai/v1/responses
Responses API - add default api version for openai responses api calls + Openrouter - fix claude-sonnet-4 on openrouter + Azure - Handle `openai/v1/responses`
2025-08-18 18:59:28 -07:00
Krish DholakiaandGitHub 3b52545db3 Merge pull request #13529 from BerriAI/litellm_dev_08_11_2025_p1
[Fix] Cooldowns - don't return raw Azure Exceptions to client
2025-08-18 18:54:19 -07:00
Ishaan JaffandGitHub ba0881d728 [Bug Fix] image_edit() function returns APIConnectionError with litellm_proxy - Support for both image edits and image generations (#13735)
* add image edits litellm proxy on SDK

* add image gen provider

* add IMG Gen support for litellm_proxy provider
2025-08-18 18:26:32 -07:00
Ishaan JaffandGitHub 76f1064229 [Bug Fix] litellm incompatible with newest release of openAI v1.100.0 (#13728)
* fix imports OpenAI SDK

* ResponseText fixes

* fixes ResponseText

* fix imports

* catch AttributeError

* fix import

* use openai==1.100.1

* fix build from PIP

* fix lint test

* Print OpenAI version

* fix Install dependencies
2025-08-18 18:26:17 -07:00
Ishaan JaffandGitHub ba1d2e8749 [Feat] DD LLM Observability - Add time to first token, litellm overhead, guardrail overhead latency metrics (#13734)
* fixes for DDLLMObsLatencyMetrics

* use _get_latency_metrics

* DD LLM Obs - track latency metrics

* fixes for bedrock guardrails

* DD unit tests

* test DD
2025-08-18 17:38:04 -07:00
Ishaan JaffandGitHub ef08e18c66 [Feat] Datadog LLM Observability - Add support for Failure Logging (#13726)
* add async_log_failure_event for DD LLM Obs

* update types

* DataDogLLMObsLogger  add failure logging support

* test_async_log_failure_event

* dd test failure
2025-08-18 15:19:48 -07:00
Thiago SalvatoreandGitHub 169a17400f fix(vertexai-batch): fix vertexai batch file format (#13576)
* fix(access group): allow access group on mcp tool retrieval

* fix(test): fix broken tests and add test case for access group

* fix(mypy): fix typing issues

* fix(memory file): add content type to in memory file
2025-08-18 10:19:23 -07:00
Michal Otmianowski 405e74ec16 use namespace as prefix in s3 2025-08-18 13:43:51 +02:00
Krrish Dholakia 6322aef0e3 fix(streaming_handler.py): fix streaming chunk calculation 2025-08-16 14:25:29 -07:00
Krrish Dholakia eb66daeef7 test: update test
we now return correct token usage on clientside
2025-08-16 14:14:35 -07:00
Ishaan Jaff d96df5e9be Revert "test_stream_token_counting_anthropic_with_include_usage"
This reverts commit c3aee1194b.
2025-08-16 13:07:00 -07:00
Ishaan Jaff c3aee1194b test_stream_token_counting_anthropic_with_include_usage 2025-08-16 13:06:31 -07:00
Krrish Dholakia 8bb5ee2ba5 test: update unit test 2025-08-16 12:39:03 -07:00
Krrish Dholakia 1ec0808713 test: update test - anthropic returns input tokens in final usage chunk now 2025-08-16 12:31:52 -07:00
Ishaan Jaff f522f40228 test_passing_tool_result_as_list 2025-08-16 08:08:25 -07:00
Cole McIntosh 000ecad4e2 Fix Groq streaming ASCII encoding issue
Replace iter_lines()/aiter_lines() with iter_text()/aiter_text() using explicit
UTF-8 encoding to handle non-ASCII characters like µ in streaming responses.

- Added utf8_iter_lines() and utf8_aiter_lines() helper functions
- Ensures proper UTF-8 decoding of streaming response content
- Added comprehensive tests for Unicode character handling

Fixes #12660
2025-08-16 08:32:22 -05:00
Krish DholakiaandGitHub 1b2ec16eee Merge pull request #13640 from yytdfc/fix_bedrock_epc
[Bug Fix] Add cachePoint support for assistant and tool messages in Bedrock
2025-08-16 01:20:28 -07:00
Ishaan JaffandGitHub 40395598da [Feat] UI - Allow editing team member rpm/tpm limits (#13669)
* show team member tpm/rpm limits

* ui - allow setting team settings

* fix better debugging

* fix types: TeamMemberUpdateRequest

* add _upsert_budget_and_membership

* allow updating team member RPM/TPM in teamMemberUpdateCall

* editing team member rpm/tpm

* UI - fixes for team member component

* fix info

* test_upsert_rpm_only_creates_new_budget
2025-08-15 17:29:44 -07:00
0x-fangandGitHub 9f17bedb48 bump up boto3 and aioboto3 versions (#13665) 2025-08-15 17:06:08 -07:00
Ishaan Jaff f5a1c14929 Revert "update boto3 version to 1.37.38 (#13656)"
This reverts commit 6f2264a13d.
2025-08-15 11:24:59 -07:00
Daniel BarkerandGitHub 47edecd5bc Fixed incorrect key info endpoint (#13633) 2025-08-15 11:10:06 -07:00
0x-fangandGitHub 6f2264a13d update boto3 version to 1.37.38 (#13656) 2025-08-15 11:06:54 -07:00
Ishaan JaffandGitHub c3608adb0f [Bug Fix] Using stream=True + background=True with Responses API (#13654)
* test_update_responses_api_response_id_with_model_id_handles_dict

* fix linting
2025-08-15 08:58:28 -07:00
FuChen 511363d3a7 test: Add comprehensive test cases for cachePoint support
- test_assistant_message_cache_control: Tests assistant messages with string content and cache_control
- test_assistant_message_list_content_cache_control: Tests assistant messages with list content and cache_control
- test_tool_message_cache_control: Tests tool messages with list content and cache_control
- test_tool_message_string_content_cache_control: Tests tool messages with string content and cache_control
- test_assistant_tool_calls_cache_control: Tests assistant tool_calls with cache_control
- test_multiple_tool_calls_with_mixed_cache_control: Tests multiple tool calls with mixed cache_control
- test_no_cache_control_no_cache_point: Tests that messages without cache_control don't generate cachePoint blocks

These tests ensure that cachePoint blocks are correctly generated for all message types when cache_control is present.
2025-08-15 11:56:34 +08:00
Ishaan JaffandGitHub 48c89812c4 [Bug Fix] /messages endpoint - ensure tool use arguments are returned for non-anthropic models (#13638)
* bug fix _translate_streaming_openai_chunk_to_anthropic

* test  test_translate_streaming_openai_chunk_to_anthropic_with_partial_json
2025-08-14 17:29:42 -07:00
Ishaan JaffandGitHub b78495d398 [Fix] Ensure /messages works when using `bedrock/converse/<model> with LiteLLM (#13627)
* get_bedrock_provider_config_for_messages_api

* fixes for get_bedrock_provider_config_for_messages_api

* test_anthropic_messages_litellm_router_bedrock

* fix merge conflicts

* fix - refactor based on jugal's comment
2025-08-14 16:50:05 -07:00
Jugal D. BhattandGitHub b8fe5f7b17 [MCP Gateway] LiteLLM Fix MCP gateway key auth (#13630)
* Fix - add safe divide by 0 for most places to prevent crash

* Enhance MCPRequestHandler to support permission inheritance and intersection logic for access groups. Added integration tests to verify behavior when keys have no permissions and when both keys and teams have overlapping permissions.

* Remove redundant assertions for permission checks in test_user_api_key_auth_mcp.py to streamline test logic.

* Refactor integration tests for MCPRequestHandler to simplify mocking. Replace complex database mocks with direct function mocks for permission inheritance and intersection scenarios, improving test clarity and maintainability.

* Revert "Fix - add safe divide by 0 for most places to prevent crash"

This reverts commit 265d40e39051e148996b9fb7f354730c57ff23ac.
2025-08-14 16:32:18 -07:00
aea0605eed [LLM Translation] Fix Realtime API endpoint for no intent (#13476)
* fix intent params

* Add responses

* fix unrelated test

* test fix - fireworks API endpoint is down

* test fix fireworks ai is having an active outage

* test_completion_cost_databricks

* dbrx fix test API currently not responding

* Update OpenAI Realtime handler to use the correct endpoint and include all query parameters. Adjusted error messages for missing API base and key. Updated health check URL construction to pass model as a query parameter.

* Enhance OpenAI Realtime handler tests to ensure model parameter inclusion in WebSocket URL. Added new tests to verify correct URL construction with model and additional parameters, preventing 'missing_model' errors. Updated existing tests for consistency.

* Remove debug print statements for API base and key in OpenAIRealtime handler to clean up the code.

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
2025-08-14 16:24:14 -07:00