Commit Graph
23118 Commits
Author SHA1 Message Date
Cole McIntosh 9c481e3ba8 Add SSO configuration endpoints and UI integration
- Introduced new SSOConfig class to manage SSO settings.
- Added endpoints for fetching and updating SSO settings in proxy_setting_endpoints.py.
- Created a new __init__.py file to expose the SSO router.
- Updated AdminPanel and SSOModals components to handle SSO settings retrieval and updates.
- Removed deprecated SSO update logic from AdminPanel.
- Enhanced error handling and logging for SSO operations.
2025-06-04 14:09:19 -06:00
Cole McIntosh 7d4a70bfe3 Enhance Admin Panel UI: Adjust button styles and layout for better accessibility and user experience 2025-06-04 13:32:43 -06:00
Lik WeiandGitHub a24077fb4e fix: add enterprise feature gating to RegenerateKeyModal in KeyInfoView (#11400)
Pass premiumUser prop to RegenerateKeyModal to control access
Disable Regenerate Key button and show tooltip for non-premium users
Ensure consistent enterprise feature messaging across components
2025-06-04 12:05:56 -07:00
Krrish Dholakia 26891c23c5 docs: update docs 2025-06-04 11:42:53 -07:00
Walt WellsandGitHub f782cc7e4d update GCSBucketBase to handle GSM project ID if passed (#11409) 2025-06-04 10:02:26 -07:00
Krish DholakiaandGitHub e74ff23b45 Litellm dev 06 03 2025 p3 (#11388)
* fix(vertex_ai/common_utils.py): Close https://github.com/BerriAI/litellm/issues/11383

* feat(anthropic/batches): transformation.py

new transformation config for anthropic batches

* feat(anthropic/batches): working token tracking for anthropic batch calls via `/anthropic` passthrough route

* fix(anthropic_passthrough_logging_handler.py): ruff check fixes
v1.72.1-nightly
2025-06-03 22:35:51 -07:00
Krrish Dholakia b21efd435a test(test_sso_sign_in.py): update test 2025-06-03 21:46:34 -07:00
Krish DholakiaandGitHub 3bd128630d Anthropic - pass file url's as Document content type + Gemini - cache token tracking on streaming calls (#11387)
* fix(anthropic/): fix regression when passing file url's to the 'file_id' parameter

add test and ensure anthropic file url's are correctly sent as 'document' blocks

* fix(vertex_and_google_ai_studio.py): Use same usage calculation function as non-streaming

Closes https://github.com/BerriAI/litellm/issues/10667

* test(test_vertex_and_google_ai_studio_gemini.py): update test
2025-06-03 21:36:49 -07:00
Krrish Dholakia 0273b72b68 build: bump poetry 2025-06-03 21:25:53 -07:00
Krish DholakiaandGitHub e0fa33f099 UI / SSO - Update proxy admin id role in DB + Handle SSO redirects with custom root path (#11384)
* fix(ui_sso.py): update user as proxy admin in db table, when checking for proxy_admin_id

Fixes issue where existing internal user, unable to make calls when set as proxy admin id

* fix(utils.py): fix custom base path
2025-06-03 21:16:55 -07:00
Ishaan JaffandGitHub a1f3a1c5dc [Feat] Performance - Don't create 1 task for every hanging request alert (#11385)
* feat: add async_get_oldest_n_keys in memory cache

* fix: add add_request_to_hanging_request_check

* test: alerting

* feat: v2 hanging request check

* fix: HangingRequestData

* fix: AlertingHangingRequestCheck

* fix: check_for_hanging_requests

* fix: use correct metadata location for hanging requests

* fix: formatting alert

* test hanging request check

* fix: add guard flags for background tasks alerting
2025-06-03 21:12:54 -07:00
Krrish Dholakia affc4524e1 build: cascade budget changes 2025-06-03 21:02:19 -07:00
Krrish Dholakia 2fe0a2750b test: ensure aws region correctly set 2025-06-03 20:58:21 -07:00
Cole McIntoshandGitHub 7bbd8262ed Add Claude 4 Sonnet & Opus, DeepSeek R1, and fix Llama Vision model pricing configurations (#11339)
* fix: update model path for llama-v3p2-90b-vision-instruct in pricing configuration (missing fireworks_ai/ prefix)

* feat: add deepseek-r1-0528 model configuration to pricing JSON

* feat: add configurations for new Claude 4 model alias to pricing JSON

* undo prefix change

* fix: update supports_response_schema to false in pricing JSON for litellm_provider

* update supports_tool_choice and supports_response_schema

* Update model configuration to disable function calling and tool choice for multiple models in fireworks_ai. Adjusted supported parameters in FireworksAIConfig to conditionally include tools and tool_choice based on model compatibility.

* Refactor FireworksAIConfig to use supports_function_calling from utils

* Enhance FireworksAIConfig to conditionally support tool_choice based on model capabilities
2025-06-03 20:39:47 -07:00
c0c04b0e63 [Performance]: Add debugging endpoint to track active /asyncio-tasks (#11382)
* feat: add debug/asyncio-tasks

* Update litellm/proxy/common_utils/debug_utils.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-06-03 19:04:30 -07:00
Marty SullivanandGitHub d247a390bd add gemini-embeddings-001 model prices and context window (#11332)
* add gemini-embeddings-001 model prices and context window

* use scientific notation
2025-06-03 15:59:30 -07:00
haganandGitHub 0f449bf038 [Bug Fix] Create/Update team member api 500 errror (#10479)
* Fixes issue with team_endpoints on member budget update

* refactored location of budget membership fix

* added test for _upsert_budget_membership func
2025-06-03 15:57:33 -07:00
Ishaan JaffandGitHub 7c1e1790e3 fix: health liveliness check auth (#11378) 2025-06-03 15:16:12 -07:00
Ishaan Jaff 209362664f add Dockerfile.dev 2025-06-03 12:03:52 -07:00
Ishaan JaffandGitHub 99c91fe41f [Feat]: Performance add DD profiler to monitor python profile of LiteLLM CPU% (#11375)
* feat: add DD profile

* fix: test_should_use_dd_profiler

* docs dd profiler

* docs DD profiler
2025-06-03 12:03:08 -07:00
Ishaan Jaff 41a2a62511 fix: bedrock kb test 2025-06-03 11:55:41 -07:00
Cole McIntoshandGitHub 50d5ee9138 Fix linting error (#11374) v1.72.1.dev1 2025-06-03 11:32:39 -07:00
Cole McIntoshandGitHub 621d609879 feat: add cerebras/qwen-3-32b model pricing and capabilities to model_prices_and_context_window.json (#11373) 2025-06-03 11:32:13 -07:00
Krrish Dholakia d5842edf09 docs(vllm.md): add vllm - model list loadbalancing tutorial to docs 2025-06-03 09:38:33 -07:00
Krrish Dholakia cff8c3971c fix: fix import 2025-06-03 09:22:37 -07:00
AnilArenandGitHub 2486743904 Doc : Nvidia embedding models (#11352)
* fix: bedrock ai21 jamba models will work now

* Update supported_embedding.md

* Update supported_embedding.md
2025-06-03 09:17:07 -07:00
Krrish Dholakia 5ff6db5062 fix: fix merge conflicts 2025-06-03 08:26:33 -07:00
Krish DholakiaandGitHub 4611b821ec Support returning virtual key in custom auth + Handle provider-specific optional params for embedding calls (#11346)
* feat(custom_auth_auto.py): support returning a litellm virtual key from custom auth

allows admin to remap old keys to litellm virtual keys

* fix(utils.py): correctly handle optional params for openai sdk calls

Fixes https://github.com/BerriAI/litellm/issues/11126

* test: update test

* fix(utils.py): handle edge cases
2025-06-03 07:24:13 -07:00
Krrish Dholakia 3f87b8611b bump: version 1.72.0 → 1.72.1 2025-06-02 23:44:22 -07:00
Krish DholakiaandGitHub ccc085faee Merge in - Gemini streaming - thinking content parsing - return in reasoning_content (#11298)
* fix(base_routing_strategy.py): compress increments to redis - reduces write ops

* fix(base_routing_strategy.py): make get and reset in memory keys atomic

* fix(base_routing_strategy.py): don't reset keys - causes discrepency on subsequent requests to instance

* fix(parallel_request_limiter.py): retrieve values of previous slots from cache

more accurate rate limiting with sliding window

* fix: fix test

* fix: fix linting error

* fix(gemini/): fix streaming handler for function calling

Closes https://github.com/BerriAI/litellm/pull/11294

* fix: fix linting error

* test: update test

* fix(vertex_and_google_ai_studio_gemini.py): return none on skipped chunk

* fix(streaming_handler.py): skip none chunks on async streaming
v1.72.0.dev3
2025-06-02 23:14:38 -07:00
Ishaan Jaff a366f9247a docs update s3 logger 2025-06-02 21:53:47 -07:00
Ishaan JaffandGitHub a73cf185ac fixes: expose flag to disable token counter (#11344)
* fixes: expose flag to disable token counter

* fix add disable_token_counter
2025-06-02 21:53:08 -07:00
Ishaan JaffandGitHub 3db272b6d2 [Perf] - Add Async + Batched S3 Logging (#11340)
* fix: add s3 v2 async

* fix: add s3 v2 async

* fix: add s3 v2 async

* test: s3 v2 logging

* fixes: s3 logging

* fixes: s3 logging use max upload batch size

* fixes: s3 logging tests

* fixes: s3 logging tests

* fixes: s3 logging tests
2025-06-02 21:52:34 -07:00
Krrish Dholakia d408814978 build(ui/): new ui build 2025-06-02 17:55:14 -07:00
Krish DholakiaandGitHub 00be76abf4 UI - Custom Server Root Path (Multiple Fixes) (#11337)
* fix(proxy_server.py): working swagger on custom base

removes the swagger monkey patch - this seems to render the swagger on custom base paths

* fix(ui/): working custom auth uptil login success event

* fix(ui/): working custom server root path for login

* fix(proxy_server.py): create typed dict for ui returned token

allows better documentation of expected params

* refactor(proxy_server.py): refactor all ui login endpoints to use same returned ui token object

* feat(ui_sso.py): add server root path to ui token

* feat(ui_sso.py): allows ui to call correct endpoint

* fix(networking.tsx): update proxy base url with custom root path

* fix(networking.tsx): handle updating proxy base url for non-local instances

* refactor: remove uneccessary references to proxybaseurl in ui code - reduce potential for errors

* fix: fix linting error

* fix(onboarding_link.tsx): fix onboarding link when custom server path is set

* feat(ui_discovery_endpoints.py): add new public .well-known/ route for litellm ui config

returns the server root path and proxy base url for constructing api calls

* feat(_types.py): add litellm well known config as public route

allows ui to query it

* fix(/_types.py): add .well-known config to as public route

* fix(page.tsx): create pattern for loading in ui config before making network requests

ensures requests are formatted correctly

* fix(page.tsx): call credential endpoint once ui config is loaded

* fix(page.tsx): route correctly to litellm dashboard from new user login

* fix(page.tsx): remove hardcoded `/litellm` for /sso/key/generate request

* fix(proxy_server.py): re-add moderations endpoint

* fix(proxy_server.py): mount __next__ at / and /litellm

allows it to work when proxy is mounted on root

* docs(contributing.md): remove /ui on ui doc - it will now run on root

* docs(custom_root_ui.md): add docs on custom root path
2025-06-02 17:48:03 -07:00
Ishaan JaffandGitHub 8a4b163453 [Feat] DD Trace - Add instrumentation for streaming chunks (#11338)
* fix: add tracing for litellm.completion

* fix: NULL span add trace

* fix: add tracing for litellm.completion streaming

* fix: add tracing for litellm.completion streaming

* fix: use a constant for str
2025-06-02 16:48:39 -07:00
Cole McIntoshandGitHub ba89d4f00f refactor: update model handling in Azure and OpenAI audio transcription classes (#11333)
- Changed hardcoded model "whisper-1" to dynamic model extraction in AzureAudioTranscription and OpenAIAudioTranscription classes.
- Added tests to ensure correct model mapping for various transcription models, including GPT-4o and Azure whisper-1.
2025-06-02 16:25:51 -07:00
Ishaan Jaff 75dbb86d34 fix: use handle exception on proxy v1.72.0.dev1 2025-06-02 13:58:22 -07:00
Ishaan JaffandGitHub f247f1031b [Fix] Fix SCIM running patch operation case sensitivity (#11335)
* fix: fix SCIM patch op

* test: test SCIM patch op
2025-06-02 13:53:26 -07:00
Cole McIntoshandGitHub 702e399d94 Add presidio_language yaml configuration support for guardrails (#11331)
* feat: Add presidio_language parameter for PII analysis configuration via yaml config files

- Introduced presidio_language parameter in guardrail_initializers and guardrail_hooks to allow language specification for Presidio PII analysis via yaml config files
- Updated PresidioPresidioConfigModelUserInterface to include presidio_language with a default value of "en".
- Added tests to verify language configuration functionality, ensuring correct language usage in analyze requests.

* docs: Enhance PII masking documentation with language support details

- Added information on configuring the default language for PII analysis using the `presidio_language` parameter in both the UI and YAML configuration.
- Included supported language codes (English, Spanish, German) and their precedence in language settings.
- Provided examples for default language configuration and per-request language overrides to clarify usage.
2025-06-02 13:13:17 -07:00
Cole McIntoshandGitHub 94650c10fe feat: Add support for Cohere Embed v4.0 model (#11329)
- Updated model_prices_and_context_window.json to include embed-v4.0 with relevant pricing and metadata.
- Added embed-v4.0 to cohere_embedding_models in constants.py.
- Implemented comprehensive tests for Cohere Embed v4.0 in test_cohere.py, covering basic functionality, input types, error handling, and optional parameters.
2025-06-02 11:25:29 -07:00
Tyler WagnerandGitHub 63d7f99e27 fix: remove dupe server_id MCP config servers (#11327) 2025-06-02 09:04:55 -07:00
Matt NitzkenandGitHub dd17661b13 Add support for DataRobot as a provider in LiteLLM (#10385)
* Add support for DataRobot as a router in LiteLLM

* Updates to logic

* Changes to make things work better

* Capitalize bearer

* Revert change

* Undo and simplify things

* Add basic testing

* Add some extra handling

* More tests

* Lowercase

* Fix

* Comment

* Add local test_completion for datarobot

* Sync repo to main

* Update get_complete_url to accept deployments

* Migrate to OpenAILike

* Mock datarobot in test

* Migrate everything to OpenAI LIke

* Apply fixes and changes from review

* Update docs

* Update env vars

* Move tests
2025-06-02 08:37:21 -07:00
tanjiroandGitHub 7155e157da converted action buttons to sticky footer action buttons (#11293) 2025-06-02 07:50:12 -07:00
Krrish Dholakia 9630386f2b docs: add release candidate notice 2025-06-01 22:39:57 -07:00
Krish DholakiaandGitHub 83becdbc11 Litellm doc fixes 05 31 2025 (#11305)
* docs: cleanup

* docs: add anthropic file tutorial

* docs: add to sidebar
2025-06-01 00:53:56 -07:00
Ishaan Jaff bdfa24be23 update doc v1.72.0.rc 2025-05-31 20:57:48 -07:00
Krish DholakiaandGitHub a40b81cd6b Rate Limiting: Check all slots on redis, Reduce number of cache writes (#11299)
* fix(base_routing_strategy.py): compress increments to redis - reduces write ops

* fix(base_routing_strategy.py): make get and reset in memory keys atomic

* fix(base_routing_strategy.py): don't reset keys - causes discrepency on subsequent requests to instance

* fix(parallel_request_limiter.py): retrieve values of previous slots from cache

more accurate rate limiting with sliding window

* fix: fix test

* fix: fix linting error
v1.72.0.rc
2025-05-31 18:32:13 -07:00
Ishaan Jaff 10fa45d987 docs fix 2025-05-31 16:29:19 -07:00
AyrennCandGitHub 8ae79178ae feat: Add audio parameter support to gemini tts models (#11287)
* feat: Add Gemini TTS audio parameter support

- Add is_model_gemini_audio_model() method to detect TTS models
- Include 'audio' parameter in supported params for TTS models
- Map OpenAI audio parameter to Gemini speechConfig format
- Add _extract_audio_response_from_parts() method to transform audio
  output to openai format

* updated unit-test to use pcm16

* - created typedict for speechconfig
- simplified gemini tts model detection
- moved gemini_tts test to test_litellm

* simplified is_model_gemini_audio_model more
v1.72.0-nightly
2025-05-31 16:20:19 -07:00