Commit Graph
22139 Commits
Author SHA1 Message Date
Krish DholakiaandGitHub 2efaa3cf36 Expose /list and /info endpoints for Audit Log events (#11102)
* feat(audit_logging_endpoints.py): expose list endpoint to show all audit logs

make it easier for user to retrieve individual endpoints

* feat(enterprise/): add audit logging endpoint

* feat(audit_logging_endpoints.py): expose new GET `/audit/{id}` endpoint

make it easier to retrieve view individual audit logs

* feat(key_management_event_hooks.py): correctly show the key of the user who initiated the change

* fix(key_management_event_hooks.py): add key rotations as an audit log event

'

* test(test_audit_logging_endpoints.py): add simple unit testing for audit log endpoint

* fix: testing fixes

* fix: fix ruff check
2025-05-23 22:54:59 -07:00
Ishaan JaffandGitHub 8d7e234efd [Feat] Add content policy violation error mapping for image editd (#11113)
* feat: add image edit mapping for content policy violations

* test fix
2025-05-23 21:57:43 -07:00
Krrish Dholakia af930f22a3 docs: add missing docs 2025-05-23 21:55:30 -07:00
Krrish Dholakia a4f809b06a fix: fix linting error 2025-05-23 21:41:17 -07:00
Krrish Dholakia 247c364451 ci(config.yml): run enterprise and litellm tests separately 2025-05-23 21:37:06 -07:00
Krrish Dholakia 42c1d932f9 bump: version 0.1.21 → 0.2.0 2025-05-23 21:32:38 -07:00
Krrish Dholakia 046c6c7149 fix(user_api_key_auth.py): handle user custom auth set with no custom settings 2025-05-23 21:30:44 -07:00
JuHyun BaeandGitHub 4d2048e208 Fix/background health check (#10887)
* fix: improve health check logic by deep copying model list on each iteration

* test: add async test for background health check reflecting model list changes

* fix: validate health check interval before executing background health check

* fix: specify type for health check results dictionary
2025-05-23 20:52:35 -07:00
Ishaan JaffandGitHub 716d7c21f0 use n 4 for mapped tests (#11109) 2025-05-23 18:49:58 -07:00
Ishaan Jaff ba4148100f test: test_openai_image_edit_litellm_sdk 2025-05-23 17:34:03 -07:00
8c0054a77e Add devstral-small-2505 model to pricing and context window configuration (#11103)
- Added mistral/devstral-small-2505 with 128K context window
- Pricing: bash.1/M input tokens, bash.3/M output tokens (same as Mistral Small 3.1)
- Supports function calling, assistant prefill, and tool choice
- Source: https://mistral.ai/news/devstral

Co-authored-by: openhands <openhands@all-hands.dev>
2025-05-23 14:57:36 -07:00
Ishaan Jaff 13bbf11ab0 test: fix failing deepeval test 2025-05-23 14:40:39 -07:00
KreatoandGitHub 2e0dcedac0 Proper github images (#10927)
* feat: add seperate image URLs to distinguish types of release

* feat: remove new nightly/dev image URLs, only keep stable
2025-05-23 12:38:00 -07:00
Krrish Dholakia 856ed51ec1 build(pyproject.toml): add langfuse dev dependency for tests 2025-05-23 12:04:27 -07:00
MayankandGitHub 8da898c55e [feature] ConfidentAI logging enabled for proxy and sdk (#10649)
* async success implemented

* fail async event

* sync events added

* docs added

* docs added

* test added

* style

* test

* .

* lock file genrated due to tenacity change

* mypy errors

* resolved comments

* resolved comments

* resolved comments

* resolved comments

* style

* style

* resolved comments
2025-05-23 11:40:48 -07:00
Krish DholakiaandGitHub e9b7059af4 Litellm add file validation (#11081)
* fix: cleanup print statement

* feat(managed_files.py): add auth check on managed files

Implemented for file retrieve + delete calls

* feat(files_endpoints.py): support returning files by model name

enables managed file support

* feat(managed_files/): filter list of files by the ones created by user

prevents user from seeing another file

* test: update test

* fix(files_endpoints.py): list_files - always default to provider based routing

* build: add new table to prisma schema
2025-05-22 23:05:45 -07:00
Krish DholakiaandGitHub 5f6928bd50 Enable switching between custom auth and litellm api key auth + Fix /customer/update for max budgets (#11070)
* feat(user_api_key_auth.py): (enterprise) allow user to enable custom auth + litellm api key auth

makes it easy to migrate to proxy

* fix(proxy/_types.py): allow setting 'spend' for new customer

* fix(customer_endpoints.py): fix updating max budget on `/customer/update`

Fixes https://github.com/BerriAI/litellm/issues/6920

* test(test_customer_endpoints.py): add unit tests for customer update endpoint

* fix: fix linting error

* fix(custom_auth_auto.py): fix ruff check

* fix(customer_endpoints.py): fix documentation
2025-05-22 23:04:37 -07:00
Tornike GurgenidzeandGitHub db4183715a feat: add embeddings to CustomLLM (#10980)
* feat: add embeddings to CustomLLM

* feat: add aembedding to custom llm
2025-05-22 22:55:46 -07:00
daarko10andGitHub e2d147102d Fix/openrouter stream usage id 8913 (#11004)
* Add handling and verification for 'usage' field in OpenRouter chat transformations and streaming responses.

* Ensure consistent response ID by using valid ID from any chunk.

* Remove redundant comments from OpenRouter chat transformation tests and logic.

* Remove this from here as I'm opening a new pr

* Reverting space

* Remove redundant assertions from OpenRouter chat transformation test
2025-05-22 22:54:56 -07:00
c1a4d3a704 fix:Databricks Claude 3.7 Sonnet output token cost: $17.85/M instead of (#11007)
$178.5/M

Co-authored-by: Tommy PLANEL <tommy.planel2@ubisoft.com>
2025-05-22 22:54:15 -07:00
Emerson GomesandGitHub b350cd306a Add Azure Mistral Medium 25.05 (#11063)
* Add Azure Mistral Medium 25.05

* fix provider
2025-05-22 22:45:50 -07:00
Gunjan SolankiandGitHub f1cc2d544e feat(helm): Add loadBalancerClass support for LoadBalancer services (#11064)
* feat(helm): Add loadBalancerClass support for LoadBalancer services

Adds the ability to specify a loadBalancerClass when using LoadBalancer service type.
This enables integration with custom load balancer implementations like Tailscale.

* fixup! feat(helm): Add loadBalancerClass support for LoadBalancer services
2025-05-22 22:45:14 -07:00
Martin LiuandGitHub d4eec9558b Fix proxy_cli.py: avoid overriding DATABASE_URL when it’s already provided. (#11076) 2025-05-22 22:40:21 -07:00
64f325b92e adds tzdata (#10796) (#11052)
With tzdata installed, the environment variable `TZ` will be respected by Python's datetime module. This means that users can specify the timezone they want LiteLLM to use.

Co-authored-by: Simon Stone <sipreuss@gmail.com>
2025-05-22 22:36:19 -07:00
Ishaan JaffandGitHub 329e69f610 Update feature_request.yml (#11078) 2025-05-22 21:42:12 -07:00
Ishaan JaffandGitHub ae33113908 Update feature_request.yml 2025-05-22 21:39:13 -07:00
Ishaan JaffandGitHub a7a5b22393 [Feat] Add Azure AD certificate-based authentication (#11069)
* feat: add cert based auth for Azure get_azure_ad_token_provider

* test: tests azure cert auth

* fix update poetry

* fix: fix linting
2025-05-22 18:05:28 -07:00
Ishaan JaffandGitHub c8a0088970 [Fix] Reliability Fix - Removing code that was creating threads on errors (#11066)
* fix: only init langfuse if active

* fix: only init langfuse if active

* fix: add initialized_langfuse_clients count

* fix: add MAX_LANGFUSE_INITIALIZED_CLIENTS

* fix: use safe init langfuse

* test: init langfuse clients

* test: test_langfuse_not_initialized_returns_none_early

* docs MAX_LANGFUSE_INITIALIZED_CLIENTS

* fix: use correct langfuse callback

* fix: code qa
2025-05-22 18:04:15 -07:00
Ishaan Jaff 5c90e51ad4 (build) fix context window for claude 4 model family 2025-05-22 17:55:29 -07:00
Krish DholakiaandGitHub 70f32154c5 Litellm managed file updates combined (#11040)
* Add LiteLLM Managed file support for `retrieve`, `list` and `cancel` finetuning jobs (#11033)

* feat: initial commit adding managed file support to fine tuning endpoints

* feat(fine_tuning/endpoints.py): working call to openai finetuning route

Uses litellm managed files for finetuning api support

* feat(fine-tuning/main.py): refactor to use LiteLLMFineTuningJob pydantic object

includes 'hidden_params'

* fix: initial commit adding unified finetuning id support

return a unified finetuning id we can use to understand which deployment to route the ft request to

* test: fix test

* feat(managed_files.py): return unified finetuning job id on create finetuning job

enables retrieve, delete to work with litellm managed files

* feat(managed_files.py): support managed files for cancel ft job endpoint

* feat(managed_files.py): support managed files for cancel ft job endpoint

* feat(fine_tuning_endpoints/endpoints.py): add managed files support to list finetuning jobs

* feat(finetuning_endpoints/main): add managed files support for retrieving ft job

Makes it easier to control permissions for ft endpoint

* LiteLLM Managed Files - Enforce validation check if user can access finetuning job (#11034)

* feat: initial commit adding managed file support to fine tuning endpoints

* feat(fine_tuning/endpoints.py): working call to openai finetuning route

Uses litellm managed files for finetuning api support

* feat(fine-tuning/main.py): refactor to use LiteLLMFineTuningJob pydantic object

includes 'hidden_params'

* fix: initial commit adding unified finetuning id support

return a unified finetuning id we can use to understand which deployment to route the ft request to

* test: fix test

* feat(managed_files.py): return unified finetuning job id on create finetuning job

enables retrieve, delete to work with litellm managed files

* feat(managed_files.py): support managed files for cancel ft job endpoint

* feat(managed_files.py): support managed files for cancel ft job endpoint

* feat(fine_tuning_endpoints/endpoints.py): add managed files support to list finetuning jobs

* feat(finetuning_endpoints/main): add managed files support for retrieving ft job

Makes it easier to control permissions for ft endpoint

* feat(managed_files.py): store create fine-tune / batch response object in db

storing this allows us to filter files returned on list based on what user created

* feat(managed_files.py): Ensures users can't retrieve / modify each others jobs

* fix: fix check

* fix: fix ruff check errors

* test: update to handle testing

* fix: suppress linting warning - openai 'seed' is none on azure

* test: update tests

* test: update test
2025-05-22 17:20:41 -07:00
Krrish Dholakia 469d395177 test: update groq test - change on their end v1.70.4-nightly 2025-05-22 15:02:01 -07:00
Krrish Dholakia 889f0093e0 fix: fix checking optional params from logging object for function call 2025-05-22 14:46:42 -07:00
Ishaan Jaff 754a94db97 Revert "Revert "Support passing prompt_label to langfuse (#11018)""
This reverts commit 0be7e7d088.
2025-05-22 14:14:39 -07:00
Ishaan Jaff 0be7e7d088 Revert "Support passing prompt_label to langfuse (#11018)"
This reverts commit 2b50b43ae2.
2025-05-22 14:11:19 -07:00
Ishaan Jaff 89daa1dbad docs add claude-4 models 2025-05-22 13:41:09 -07:00
Ishaan JaffandGitHub 197c608078 [Feat] Add claude-4 model family (#11060)
* add new claude-sonnet-4-2025051

* feat: add bedrock claude-4 models

* add bedrock claude-4 models

* add vertx_ai/claude-sonnet-4

* fix provider=bedrock_converse

* feat: ensure thinking is supported for claude-4 model family
v1.70.2.dev5
2025-05-22 13:33:54 -07:00
Ishaan Jaff 2c90ca0189 docs fix example 2025-05-22 09:33:22 -07:00
Ishaan Jaff d95c3a16f3 docs fix ad hoc recognizer 2025-05-22 08:55:07 -07:00
Ishaan Jaff d595c4ef2d bump: version 1.70.3 → 1.70.4 2025-05-22 07:32:35 -07:00
tanjiroandGitHub 1c652b67b6 Model filter on logs (#11048)
* add model filter

* remove calling all models
2025-05-22 07:31:43 -07:00
Ishaan JaffandGitHub dd4a65b83a Feat: add MCP to Responses API and bump openai python sdk (#11029)
* feat: add MCP to responses API

* feat: bump openai version to 1.75.0

* docs MCP + responses API

* fixes: type checking

* fixes: type checking

* build: use latest openai 1.81.0

* fix: linting error

* fix: linting error

* fix: test

* fix: linting errors

* fix: test

* fix: test

* fix: linting

* Revert "fix: linting"

This reverts commit ebb19ff8cb1f8fcc3e224390e351676daccb33de.

* fix: linting
2025-05-22 07:24:10 -07:00
tanjiroandGitHub bfb04d3e75 added cloding tags for </TabGroup> </Col> </Grid> + indentation changes (#11046) 2025-05-22 07:23:26 -07:00
Krish DholakiaandGitHub 2b50b43ae2 Support passing prompt_label to langfuse (#11018)
* fix: add prompt label support to prompt management hook

* feat: support 'prompt_label' parameter for langfuse prompt management

Closes https://github.com/BerriAI/litellm/discussions/9003#discussioncomment-13221555

* fix(litellm_logging.py): deep copy optional params to avoid mutation while logging

* fix(log-consistent-optional-param-values-across-providers): ensures params can be used for finetuning from providers

* fix: fix linting error

* test: update test

* test: update langfuse tests

* fix(litellm_logging.py): avoid deepcopying optional params

might contain thread object
2025-05-21 22:27:36 -07:00
Krrish Dholakia cd496fee2e fix: fix linting error 2025-05-21 22:04:41 -07:00
jmorenoc-oandGitHub 1cd2595006 Fixes the InvitationLink Prisma find_many query (#11031)
Related: https://github.com/BerriAI/litellm/commit/3b6c6d05dd8f8bcd83f776cdc1c8fc64d3d85d13#r157675103

We should use "order", according to the prisma python docs https://prisma-client-py.readthedocs.io/en/stable/reference/limitations/#order-argument 

Also we are using "order" in other files of the project: 
https://github.com/search?q=repo%3ABerriAI%2Flitellm%20order%3D%7B&type=code
2025-05-21 21:42:57 -07:00
Krish DholakiaandGitHub 58f958f30a Litellm dev 05 21 2025 p2 (#11039)
* feat: initial commit adding managed file support to fine tuning endpoints

* feat(fine_tuning/endpoints.py): working call to openai finetuning route

Uses litellm managed files for finetuning api support

* feat(fine-tuning/main.py): refactor to use LiteLLMFineTuningJob pydantic object

includes 'hidden_params'

* fix: initial commit adding unified finetuning id support

return a unified finetuning id we can use to understand which deployment to route the ft request to

* test: fix test

* feat(managed_files.py): return unified finetuning job id on create finetuning job

enables retrieve, delete to work with litellm managed files

* test: update test

* fix: fix linting error

* fix: fix ruff linting error

* test: fix check
2025-05-21 21:40:53 -07:00
Krrish Dholakia 546a508c8c test: mark flaky test 2025-05-21 21:36:46 -07:00
e487f1e17d feat: add Databricks Llama 4 Maverick model cost (#11008)
Co-authored-by: Tommy PLANEL <tommy.planel2@ubisoft.com>
2025-05-21 20:59:18 -07:00
Jay GowdyandGitHub 85d577c8e6 Fix: Handle dict objects in Anthropic streaming response (#11032)
* fix: handle dict objects in Anthropic streaming response

Fix issue where dictionary objects in Anthropic streaming responses
were not properly converted to SSE format strings before being yielded,
causing AttributeError: 'dict' object has no attribute 'encode'

* fix: refactor Anthropic streaming response handling

- Added STREAM_SSE_DATA_PREFIX constant in constants.py
- Created return_anthropic_chunk helper function for better maintainability
- Using safe_dumps from safe_json_dumps.py for improved JSON serialization
- Added unit test for dictionary object handling in streaming response

* fix: correct patch path in anthropic_endpoints test
2025-05-21 20:58:11 -07:00
tanjiroandGitHub 0cde73ffb7 Spend rounded to 4 for Organizations and Users page (#11023)
* spend rounded to 4

* fixed for organization and users table
2025-05-21 20:41:54 -07:00