Commit Graph
24024 Commits
Author SHA1 Message Date
Ishaan JaffandGitHub 4a10eeb75e [Feat] Team Member Rate Limits - show team member tpm/rpm limits (#13662)
* show team member tpm/rpm limits

* ui - allow setting team settings
2025-08-15 12:18:13 -07:00
Ishaan Jaff f5a1c14929 Revert "update boto3 version to 1.37.38 (#13656)"
This reverts commit 6f2264a13d.
2025-08-15 11:24:59 -07:00
Ishaan Jaff c331473b75 Revert "fix aioboto3"
This reverts commit 82d1cb48b7.
2025-08-15 11:24:50 -07:00
Ishaan Jaff 49bef09a3a Revert "fix aioboto3"
This reverts commit 92c638ec59.
2025-08-15 11:24:41 -07:00
Ishaan Jaff 92c638ec59 fix aioboto3 2025-08-15 11:19:21 -07:00
Ishaan Jaff 82d1cb48b7 fix aioboto3 2025-08-15 11:19:14 -07:00
James PageandGitHub 46e8e50b47 trivy/secrets false positives (#13631)
* alert_to_webhook_url: avoid secret false positive

Trivy detects the example as a potential secret; tweak the hostname
a bit to make it obviously not hooks.slack.com to avoid this.

* Remove log file

Drop log file checked into the git tree; this contains strings that
trivy detects as potential secrets.
2025-08-15 11:10:54 -07:00
Daniel BarkerandGitHub 47edecd5bc Fixed incorrect key info endpoint (#13633) 2025-08-15 11:10:06 -07:00
William FinniganandGitHub 99f30fe5b7 Fix LangfuseOtelSpanAttributes constants to match expected values at https://python.reference.langfuse.com/langfuse#LangfuseOtelSpanAttributes (#13659)
- Update TRACE_USER_ID from 'langfuse.trace.user_id' to 'user.id'
- Update SESSION_ID from 'langfuse.trace.session_id' to 'session.id'
2025-08-15 11:08:32 -07:00
0x-fangandGitHub 6f2264a13d update boto3 version to 1.37.38 (#13656) 2025-08-15 11:06:54 -07:00
Ishaan JaffandGitHub c3608adb0f [Bug Fix] Using stream=True + background=True with Responses API (#13654)
* test_update_responses_api_response_id_with_model_id_handles_dict

* fix linting
2025-08-15 08:58:28 -07:00
d6fa6b60d7 [Feat] UI - Add Confirmation Modal Before Deleting Keys (#13655)
* Enhance key deletion with confirmation input and improved modal UI

Co-authored-by: ishaan <ishaan@berri.ai>

* remove file

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: ishaan <ishaan@berri.ai>
2025-08-15 08:51:32 -07:00
Ishaan Jaff d29bc4255b bump: version 1.75.6 → 1.75.7 v1.75.7-nightly 2025-08-14 21:13:21 -07:00
Ishaan JaffandGitHub 48c89812c4 [Bug Fix] /messages endpoint - ensure tool use arguments are returned for non-anthropic models (#13638)
* bug fix _translate_streaming_openai_chunk_to_anthropic

* test  test_translate_streaming_openai_chunk_to_anthropic_with_partial_json
2025-08-14 17:29:42 -07:00
17db9edd85 UI - Fix image overflow in LiteLLM model (#13639)
* Improve LiteLLM model name display with better styling and overflow handling

Co-authored-by: ishaan <ishaan@berri.ai>

* Add Tooltip to LiteLLM model name for improved text display

Co-authored-by: ishaan <ishaan@berri.ai>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: ishaan <ishaan@berri.ai>
2025-08-14 17:29:23 -07:00
Krrish Dholakia 5631d97964 docs(readme.md): cleanup 2025-08-14 17:16:05 -07:00
Krrish Dholakia 41f7901cfa docs(readme.md): add note, saying poetry is required 2025-08-14 17:15:29 -07:00
Krrish Dholakia 817b8408fa docs(readme.md): fix readme 2025-08-14 17:13:04 -07:00
Ishaan JaffandGitHub b78495d398 [Fix] Ensure /messages works when using `bedrock/converse/<model> with LiteLLM (#13627)
* get_bedrock_provider_config_for_messages_api

* fixes for get_bedrock_provider_config_for_messages_api

* test_anthropic_messages_litellm_router_bedrock

* fix merge conflicts

* fix - refactor based on jugal's comment
2025-08-14 16:50:05 -07:00
Jugal D. BhattandGitHub b8fe5f7b17 [MCP Gateway] LiteLLM Fix MCP gateway key auth (#13630)
* Fix - add safe divide by 0 for most places to prevent crash

* Enhance MCPRequestHandler to support permission inheritance and intersection logic for access groups. Added integration tests to verify behavior when keys have no permissions and when both keys and teams have overlapping permissions.

* Remove redundant assertions for permission checks in test_user_api_key_auth_mcp.py to streamline test logic.

* Refactor integration tests for MCPRequestHandler to simplify mocking. Replace complex database mocks with direct function mocks for permission inheritance and intersection scenarios, improving test clarity and maintainability.

* Revert "Fix - add safe divide by 0 for most places to prevent crash"

This reverts commit 265d40e39051e148996b9fb7f354730c57ff23ac.
2025-08-14 16:32:18 -07:00
aea0605eed [LLM Translation] Fix Realtime API endpoint for no intent (#13476)
* fix intent params

* Add responses

* fix unrelated test

* test fix - fireworks API endpoint is down

* test fix fireworks ai is having an active outage

* test_completion_cost_databricks

* dbrx fix test API currently not responding

* Update OpenAI Realtime handler to use the correct endpoint and include all query parameters. Adjusted error messages for missing API base and key. Updated health check URL construction to pass model as a query parameter.

* Enhance OpenAI Realtime handler tests to ensure model parameter inclusion in WebSocket URL. Added new tests to verify correct URL construction with model and additional parameters, preventing 'missing_model' errors. Updated existing tests for consistency.

* Remove debug print statements for API base and key in OpenAIRealtime handler to clean up the code.

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
2025-08-14 16:24:14 -07:00
Jugal D. BhattandGitHub 40550e5b88 [Proxy] Litellm add DB metrics to prometheus (#13626)
* Fix - add safe divide by 0 for most places to prevent crash

* feat(prometheus): add new metrics for monitoring pod lock manager and spend update queues

* fix(prometheus): specify type for buffer monitoring metrics in PrometheusMetricLabels

* Revert "Fix - add safe divide by 0 for most places to prevent crash"

This reverts commit 265d40e39051e148996b9fb7f354730c57ff23ac.
2025-08-14 16:16:48 -07:00
Krish DholakiaandGitHub ab8bd70c5e Merge pull request #13635 from BerriAI/litellm_mock_test_fix
[Proxy] LiteLLM mock test fix
2025-08-14 15:47:41 -07:00
Jugal Bhatt 936c36bd5f Increase timeout for test-litellm workflow from 20 to 25 minutes to accommodate longer test execution times. 2025-08-14 15:41:58 -07:00
Ishaan Jaff 025ce17564 bump: version 1.75.5 → 1.75.6 v1.75.6-nightly 2025-08-14 15:30:45 -07:00
TomuHirataandGitHub f6e53deacd Update mlflow logger usage span attributes (#13561)
* test: sync mlflow request tags

* fix test
2025-08-14 15:20:50 -07:00
Jugal Bhatt f53599825d Merge branch 'main' into litellm_mock_test_fix 2025-08-14 15:17:31 -07:00
Jugal Bhatt 45f188b041 Add mock user API key authentication in tag management tests
This update integrates mock user API key authentication into the tag management endpoint tests, ensuring accurate simulation of user roles for creating, updating, and deleting tags. The changes enhance the reliability of the tests by properly setting up user authentication before executing test cases.
2025-08-14 15:03:21 -07:00
Krrish Dholakia aaf9c38a10 test: skip test - ran out of credits 2025-08-14 15:01:26 -07:00
Jugal Bhatt d21f467264 Revert "Add mock user API key authentication in tag management tests"
This reverts commit 5fc0803b94.
2025-08-14 14:58:49 -07:00
Jugal Bhatt 5fc0803b94 Add mock user API key authentication in tag management tests
This update introduces a helper function to create a mock user API key authentication object, which is utilized in the tag management endpoint tests. The mock authentication is integrated into the test cases for creating, updating, and deleting tags, enhancing the reliability of the tests by simulating user roles accurately.
2025-08-14 14:40:49 -07:00
Jugal Bhatt a6e55c0447 Revert "Revert "Fix - add safe divide by 0 for most places to prevent crash""
This reverts commit 5ad698f2cc.
2025-08-14 14:23:06 -07:00
Jugal Bhatt bfb0a3854e Enhance logging in cost calculation tests to ensure DEBUG level captures are accurate. Updated tests to set logger level before assertions and restored original logger level after execution. This improves reliability of log level checks in both cost and batch cost calculation tests. 2025-08-14 14:21:22 -07:00
Jugal Bhatt 5ad698f2cc Revert "Fix - add safe divide by 0 for most places to prevent crash"
This reverts commit 265d40e39051e148996b9fb7f354730c57ff23ac.
2025-08-14 14:21:07 -07:00
Jugal D. BhattandGitHub 3a38912bce [Proxy] Litellm fix mapped tests (#13634)
* Fix - add safe divide by 0 for most places to prevent crash

* mock prisma client

* Revert "Fix - add safe divide by 0 for most places to prevent crash"

This reverts commit 265d40e39051e148996b9fb7f354730c57ff23ac.
2025-08-14 14:19:36 -07:00
Krish DholakiaandGitHub cedba81f4c Merge pull request #13588 from BerriAI/litellm_admin_viewer_updates
[Management/UI]Allow routes for admin viewer
2025-08-14 12:03:00 -07:00
Krish DholakiaandGitHub f360e0ead2 Merge pull request #13590 from BerriAI/litellm_bedrock_api_header
[LLM translation] Refactor Anthropic Configurations and Add Support for `anthropic_beta` Headers
2025-08-14 11:32:18 -07:00
Jugal D. BhattandGitHub 1beba93cc8 Fix - add safe divide by 0 for most places to prevent crash (#13624) 2025-08-14 11:17:49 -07:00
Krish DholakiaandGitHub 8d76935457 Merge pull request #13607 from huangyafei/update_price
Add deepseek-chat-v3-0324 to OpenRouter cost map
2025-08-14 11:05:53 -07:00
Krish DholakiaandGitHub fe2833817e Merge pull request #13625 from BerriAI/litellm_dev_08_13_2025_p1
perf(main.py): new 'EXPERIMENTAL_OPENAI_BASE_LLM_HTTP_HANDLER' flag
2025-08-14 11:04:35 -07:00
Ishaan JaffandGitHub 5bb96af818 [Feat] Add reasoning_effort param for hosted_vllm provider (#13620)
* add reasoning_effort to hosted_vllm

* test_hosted_vllm_supports_reasoning_effort

* Reasoning Effort
2025-08-14 10:10:30 -07:00
Cole McIntoshandGitHub dea98a315b fix(volcengine): handle thinking disabled parameter properly (#13598)
* fix(volcengine): handle thinking disabled parameter properly

When thinking is set to {"type": "disabled"} in volcengine provider config,
skip adding the parameter to extra_body entirely instead of passing it through.
This prevents "thinking: undefined" from appearing in request logs.

Fixes #13039

* test(volcengine): fix and enhance thinking parameter tests

- Fixed existing test that expected broken behavior
- Added comprehensive test coverage for all thinking parameter scenarios:
  * thinking disabled → omitted from extra_body
  * thinking enabled → included in extra_body
  * thinking None → included in extra_body as None
  * custom thinking values → included in extra_body
  * no thinking parameter → empty result

All tests passing, verifying the fix for issue #13039
2025-08-14 09:10:06 -07:00
tanjiroandGitHub b5d0a7eb95 adding missing imports + removing unused imports (#13610) 2025-08-14 08:26:15 -07:00
Krrish Dholakia 0288ed35da test: update tests 2025-08-13 23:33:32 -07:00
Krrish Dholakia b53962dee2 test: update test 2025-08-13 23:09:18 -07:00
huangyafei 2b89f95e37 Add deepseek-chat-v3-0324 to OpenRouter model list 2025-08-14 10:44:15 +08:00
Krrish Dholakia 5a7a889d93 perf(main.py): new 'EXPERIMENTAL_OPENAI_BASE_LLM_HTTP_HANDLER' flag
improves RPS for openai calls by 100 (100 users, 10 start-up)

 Moves to using litellm's asynchttphandler vs. openais's sdk for llm calling
2025-08-13 19:12:39 -07:00
Ishaan Jaff cd893134b7 test team endpoints 2025-08-13 18:43:50 -07:00
Ishaan Jaff ce4210a17a bump litellm proxy extras 2025-08-13 17:50:10 -07:00
Ishaan Jaff 086621e3d3 test_handle_jwt.py 2025-08-13 17:27:50 -07:00