Commit Graph
22399 Commits
Author SHA1 Message Date
Ishaan JaffandGitHub c99daef689 [Fix]: /v1/messages - return streaming usage statistics when using litellm with bedrock models (#11469)
* fix: using litellm with claude code bedrock

* fix: usage for bedrock with /messages

* fix: bedrock_sse_wrapper

* tests: test for test_chunk_parser_usage_transformation

* test fix
v1.72.1.dev8
2025-06-05 21:18:19 -07:00
Ishaan JaffandGitHub f0cb80ec50 [Feat] Return response_id == upstream response ID for VertexAI + Google AI studio (Stream+Non stream) (#11456)
* fix: vertexAI return responseID

* fix: vertexAI return responseID

* test_vertex_ai_response_id

* test: test_vertex_ai_streaming_response_id

* test_vertex_ai_streaming_response_id
2025-06-05 20:18:55 -07:00
Ishaan JaffandGitHub 23627d6a26 [Fix] [Bug]: Knowledge Base Call returning error (#11467)
* fix:get_and_pop_recognised_vector_store_tools

* test: tools wwith vector stores

* test - bedrock kb tools

* fix: add clear comment

* fix: vector store tools
2025-06-05 18:24:36 -07:00
RMeansandGitHub 742405f6cf Add pangea to guardrails sidebar (#11464) 2025-06-05 18:11:52 -07:00
Ishaan JaffandGitHub 18ea65218b [Feat] Make batch size for maximum retention in spend logs a controllable parameter (#11459)
* feat: add SPEND_LOG_CLEANUP_BATCH_SIZE

* docs update

* test: test_cleanup_batch_size_env_var
2025-06-05 17:11:51 -07:00
Krish DholakiaandGitHub d05eda0311 Custom Root Path Improvements: don't require reserving /litellm route (#11460)
* fix(proxy_server.py): initial commit with asset prefix rewriting for custom base path

Closes https://github.com/BerriAI/litellm/issues/11451

* docs(litellm_proxy.md): clarify version requirement

* fix(proxy_server.py): replace litellm well known route with custom server root path

Ensures UI calls correct endpoint

* build(ui/): update ui build
2025-06-05 16:36:47 -07:00
Cole McIntoshandGitHub a3da7f1876 Add AGENTS.md (#11461) 2025-06-05 16:29:28 -07:00
Sean WalkerandGitHub 29dc4e51f9 Fix HuggingFace embeddings using non-default input_type (#11452)
* fix(huggingface): use get() instead of pop() for input_type parameter

Fixes embedding generation for HuggingFace models where input_type override
is required (e.g. BAAI/bge-m3). The pop() method was mutating optional_params
and removing input_type before downstream functions could access it.

* Add unit tests to catch regression

* Move tests around
2025-06-05 15:48:55 -07:00
Krrish Dholakia ab9d09a464 fix(ui/): fix linting errors 2025-06-05 15:17:30 -07:00
Krrish Dholakia 30f1c5e852 docs: clarify pre-release 2025-06-05 15:00:04 -07:00
Cole McIntoshandGitHub 7c513856dc Fix None values in usage field for gpt-image-1 model responses (#11448)
* fix(convert_dict_to_response.py): handle None values in usage field for gpt-image-1

* test: add tests for handling None and partial values in usage fields for gpt-image-1 responses
2025-06-05 13:19:18 -07:00
Krish DholakiaandGitHub 69c9d75f20 fix(prometheus.py): pass custom metadata labels in litellm_total_toke… (#11414)
* fix(prometheus.py): pass custom metadata labels in litellm_total_tokens metric

* fix(handler.py): handle /v1 for openai realtime translation

Closes https://github.com/BerriAI/litellm/pull/11398

* fix(prometheus.py): fix incrementing total tokens metric
2025-06-05 00:15:23 -07:00
Low Jian ShengandGitHub a3e5bc4856 Support no reasoning option for gemini models (#11393)
* support no reasoning for gemini models

* change none to disable

* remove print statements

* update docs
2025-06-05 00:11:45 -07:00
Krrish Dholakia 505d2fe0c7 build: bump 2025-06-05 00:08:53 -07:00
Krish DholakiaandGitHub db23016536 fix(redis_cache.py): support pipeline redis lpop for older redis vers… (#11425)
* fix(redis_cache.py): support pipeline redis lpop for older redis versions

Fixes https://github.com/BerriAI/litellm/issues/10379

* test: add mock host
2025-06-05 00:05:54 -07:00
ShaandGitHub a301ef873e added gemini url context support (#11351)
* added gemini url context support

* lint issue fix
2025-06-04 23:56:21 -07:00
Tom BocklischandGitHub d7982bb0af Use proper attribute for sagemaker request (#11362) 2025-06-04 22:47:43 -07:00
Jimmy TsaiandGitHub 4019f79808 feat: add deepseek-r1 family model configuration to pricing JSON (#11394) 2025-06-04 22:39:06 -07:00
Ishaan JaffandGitHub 02a34d319a bump to ddtrace==3.8.0 (#11426) 2025-06-04 22:18:07 -07:00
Ishaan Jaff f0e0007eaf fix: gemini-2.0-flash-preview-image-generation test 2025-06-04 21:21:28 -07:00
Cole McIntoshandGitHub 049e65a84e Merge pull request #11417 from colesmcintosh/sso-config-ui 2025-06-04 20:51:59 -06:00
Ishaan JaffandGitHub de306cfcb3 [Performance] Performance improvements for /v1/messages route (#11421)
* fix: perf anthropic /v1/messages

* fix: perf anthropic /v1/messages

* fix: linting checks

* fix: linting checks
2025-06-04 18:47:53 -07:00
raz-alonandGitHub fada9c79be Add User ID validation to ensure it is not an email or phone number (#10102) 2025-06-04 18:38:02 -07:00
Cole McIntosh c1a324c2fb Merge remote-tracking branch 'origin/main' into sso-config-ui 2025-06-04 18:42:43 -06:00
Cole McIntosh 65b28826a6 Add uiAuditLogsCall function 2025-06-04 18:19:53 -06:00
9da32d9e14 Litellm audit log staging (#11418)
* Audit logs added (#11226)

* audit logs added

* audit logs populated

* adding json response

* collapsible json columns

* add created at column

* added changed field

* added premiumUser description

* added paginated filtered logs

* convert table names

* remove test file

* added new ui for audit logs

* only show the difference in before value and updated value

* fix: add lucide-react to package json

---------

Co-authored-by: tanjiro <56165694+NANDINI-star@users.noreply.github.com>
2025-06-04 14:34:17 -07:00
Cole McIntosh 3a946933ee Refactor settings response models in proxy_setting_endpoints.py
- Renamed SSOSettingsResponse to inherit from a new base class SettingsResponse for better structure.
- Introduced InternalUserSettingsResponse and DefaultTeamSettingsResponse models for internal user and default team settings.
- Updated endpoint responses to use field_schema instead of schema for consistency.
- Enhanced test cases to validate the new response structure and ensure proper functionality of SSO settings.
2025-06-04 15:08:05 -06:00
Cole McIntosh 3cc9460922 Add SSO settings response model in proxy_setting_endpoints.py
- Introduced SSOSettingsResponse model to encapsulate SSO configuration values and schema information.
- Updated the get_sso_settings endpoint to utilize the new response model, enhancing API clarity and usability.
2025-06-04 14:55:29 -06:00
Cole McIntosh c41b14e27b Add clear SSO settings functionality in SSOModals component
- Introduced a confirmation modal for clearing SSO settings.
- Implemented handleClearSSO function to reset SSO settings and provide user feedback.
- Updated UI to include a 'Clear' button for SSO settings, enhancing user experience.
- Added state management for the confirmation modal visibility.
2025-06-04 14:39:26 -06:00
Cole McIntosh 72c7fd63bf Implement SSO configuration check in AdminPanel and update SSOModals to reflect SSO status
- Added logic to check SSO configuration and set state in AdminPanel.
- Introduced a new function to handle SSO configuration checks.
- Updated UI to conditionally render SSO button text based on configuration status.
- Passed SSO configuration status as a prop to SSOModals for better integration.
2025-06-04 14:34:18 -06:00
Cole McIntosh 58b1f78ff0 Remove console log from handleShowInstructions in AdminPanel component 2025-06-04 14:25:57 -06:00
Cole McIntosh 9c481e3ba8 Add SSO configuration endpoints and UI integration
- Introduced new SSOConfig class to manage SSO settings.
- Added endpoints for fetching and updating SSO settings in proxy_setting_endpoints.py.
- Created a new __init__.py file to expose the SSO router.
- Updated AdminPanel and SSOModals components to handle SSO settings retrieval and updates.
- Removed deprecated SSO update logic from AdminPanel.
- Enhanced error handling and logging for SSO operations.
2025-06-04 14:09:19 -06:00
Cole McIntosh 7d4a70bfe3 Enhance Admin Panel UI: Adjust button styles and layout for better accessibility and user experience 2025-06-04 13:32:43 -06:00
Lik WeiandGitHub a24077fb4e fix: add enterprise feature gating to RegenerateKeyModal in KeyInfoView (#11400)
Pass premiumUser prop to RegenerateKeyModal to control access
Disable Regenerate Key button and show tooltip for non-premium users
Ensure consistent enterprise feature messaging across components
2025-06-04 12:05:56 -07:00
Krrish Dholakia 26891c23c5 docs: update docs 2025-06-04 11:42:53 -07:00
Walt WellsandGitHub f782cc7e4d update GCSBucketBase to handle GSM project ID if passed (#11409) 2025-06-04 10:02:26 -07:00
Krish DholakiaandGitHub e74ff23b45 Litellm dev 06 03 2025 p3 (#11388)
* fix(vertex_ai/common_utils.py): Close https://github.com/BerriAI/litellm/issues/11383

* feat(anthropic/batches): transformation.py

new transformation config for anthropic batches

* feat(anthropic/batches): working token tracking for anthropic batch calls via `/anthropic` passthrough route

* fix(anthropic_passthrough_logging_handler.py): ruff check fixes
v1.72.1-nightly
2025-06-03 22:35:51 -07:00
Krrish Dholakia b21efd435a test(test_sso_sign_in.py): update test 2025-06-03 21:46:34 -07:00
Krish DholakiaandGitHub 3bd128630d Anthropic - pass file url's as Document content type + Gemini - cache token tracking on streaming calls (#11387)
* fix(anthropic/): fix regression when passing file url's to the 'file_id' parameter

add test and ensure anthropic file url's are correctly sent as 'document' blocks

* fix(vertex_and_google_ai_studio.py): Use same usage calculation function as non-streaming

Closes https://github.com/BerriAI/litellm/issues/10667

* test(test_vertex_and_google_ai_studio_gemini.py): update test
2025-06-03 21:36:49 -07:00
Krrish Dholakia 0273b72b68 build: bump poetry 2025-06-03 21:25:53 -07:00
Krish DholakiaandGitHub e0fa33f099 UI / SSO - Update proxy admin id role in DB + Handle SSO redirects with custom root path (#11384)
* fix(ui_sso.py): update user as proxy admin in db table, when checking for proxy_admin_id

Fixes issue where existing internal user, unable to make calls when set as proxy admin id

* fix(utils.py): fix custom base path
2025-06-03 21:16:55 -07:00
Ishaan JaffandGitHub a1f3a1c5dc [Feat] Performance - Don't create 1 task for every hanging request alert (#11385)
* feat: add async_get_oldest_n_keys in memory cache

* fix: add add_request_to_hanging_request_check

* test: alerting

* feat: v2 hanging request check

* fix: HangingRequestData

* fix: AlertingHangingRequestCheck

* fix: check_for_hanging_requests

* fix: use correct metadata location for hanging requests

* fix: formatting alert

* test hanging request check

* fix: add guard flags for background tasks alerting
2025-06-03 21:12:54 -07:00
Krrish Dholakia affc4524e1 build: cascade budget changes 2025-06-03 21:02:19 -07:00
Krrish Dholakia 2fe0a2750b test: ensure aws region correctly set 2025-06-03 20:58:21 -07:00
Cole McIntoshandGitHub 7bbd8262ed Add Claude 4 Sonnet & Opus, DeepSeek R1, and fix Llama Vision model pricing configurations (#11339)
* fix: update model path for llama-v3p2-90b-vision-instruct in pricing configuration (missing fireworks_ai/ prefix)

* feat: add deepseek-r1-0528 model configuration to pricing JSON

* feat: add configurations for new Claude 4 model alias to pricing JSON

* undo prefix change

* fix: update supports_response_schema to false in pricing JSON for litellm_provider

* update supports_tool_choice and supports_response_schema

* Update model configuration to disable function calling and tool choice for multiple models in fireworks_ai. Adjusted supported parameters in FireworksAIConfig to conditionally include tools and tool_choice based on model compatibility.

* Refactor FireworksAIConfig to use supports_function_calling from utils

* Enhance FireworksAIConfig to conditionally support tool_choice based on model capabilities
2025-06-03 20:39:47 -07:00
c0c04b0e63 [Performance]: Add debugging endpoint to track active /asyncio-tasks (#11382)
* feat: add debug/asyncio-tasks

* Update litellm/proxy/common_utils/debug_utils.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-06-03 19:04:30 -07:00
Marty SullivanandGitHub d247a390bd add gemini-embeddings-001 model prices and context window (#11332)
* add gemini-embeddings-001 model prices and context window

* use scientific notation
2025-06-03 15:59:30 -07:00
haganandGitHub 0f449bf038 [Bug Fix] Create/Update team member api 500 errror (#10479)
* Fixes issue with team_endpoints on member budget update

* refactored location of budget membership fix

* added test for _upsert_budget_membership func
2025-06-03 15:57:33 -07:00
Ishaan JaffandGitHub 7c1e1790e3 fix: health liveliness check auth (#11378) 2025-06-03 15:16:12 -07:00
Ishaan Jaff 209362664f add Dockerfile.dev 2025-06-03 12:03:52 -07:00