Commit Graph
21844 Commits
Author SHA1 Message Date
tanjiroandGitHub 3fafe37eb9 Fix log table bugs (after filtering logic was added) (#10712)
* fix status filter

* fix refresh button

* minor

* fix refresh and auto refresh
2025-05-09 23:46:05 -07:00
Mattias MichauxandGitHub a7550c0b3d Update model_prices_and_context_window.json (#10690)
set correct context window length for all gemini 2.5 variants
2025-05-09 23:38:43 -07:00
6084d6c437 Azure LLM: fix passing through of azure_ad_token_provider parameter (#10694)
* Azure LLM: fix passing through of azure_ad_token_provider parameter

* add test

---------

Co-authored-by: Clara Luise Pohland <clara-luise.pohland@telekom.de>
2025-05-09 23:38:13 -07:00
Krish DholakiaandGitHub 6f32189093 fix(caching_handler.py): fix embedding str caching result (#10700)
* fix(caching_handler.py): fix embedding str caching result

Fixes issue where str caching results were not being correctly assembled on str input

* feat(azure/image_generation): Support dropping response_format for azure gpt-image-1

Fixes LIT-118

* test(test_utils.py): add unit testing

* test: rename file to avoid testing conflict
2025-05-09 23:37:02 -07:00
Marc AbramowitzandGitHub 3787aa27b0 Add management client docs (#10703)
* Add management client docs

* Docs enhancements

* Add "proxy/management_cli" to sidebar

* Small formatting changes

* Add overview table

* Wrap long line

* Add "Quick Start" section

* Document uv tool upgrade and uninstall

* Add blank lines to eliminate yellow squigglies in VS Code

* For each section link to section on swagger used

* Wrap some long command examples

* Break long http request command on to multiple lines
2025-05-09 18:59:24 -07:00
Marc AbramowitzandGitHub 3d87da555a Add --version flag to litellm-proxy CLI (#10704)
* Add --version flag to litellm-proxy CLI

```shell
$ litellm-proxy --version
litellm-proxy version: 1.68.1
```

* Return both client and server version

* Update docs

* Add a test for the version command

* Add litellm/proxy/client/health.py
2025-05-09 18:58:37 -07:00
Ishaan JaffandGitHub e5a08a5ae1 [Feat] Add streaming support for using bedrock invoke models with /v1/messages (#10710)
* add basic bedrock transform

* test_anthropic_messages_streaming_bedrock_invoke

* fix: typing ant

* fix: get async response iterator

* fix: code quality check
2025-05-09 18:56:23 -07:00
Ishaan JaffandGitHub 3731ee436a [Refactor] Use pip package for enterprise/ folder (#10709)
* init enterprise pip

* init enterprise pip

* init enterprise pip

* test: enterprise pip

* add litellm-enterprise to pip

* litellm ent check

* litellm ent check

* fix import email router

* fix setup_litellm_enterprise_pip

* fix local testing with enterprise pip
2025-05-09 17:18:48 -07:00
Ishaan Jaff c50abd3254 bump litellm proxy extras 2025-05-09 14:18:21 -07:00
Ishaan Jaff a8e7b01608 build: use big int for daily spend tables 2025-05-09 14:16:18 -07:00
Muhammad HusnainandGitHub 0675dc109e [FIX] Update token fields in schema.prisma to use BigInt for improved… (#10697)
* [FIX] Update token fields in schema.prisma to use BigInt for improved handling of large values across LiteLLM models.

* [FIX] revert SpendLogs back to Int
2025-05-09 14:14:39 -07:00
479c6e4017 fix: support for python 3.11- (re datetime UTC) (#10471) (#10701)
* fix support for python 3.11-

3.11 introduced datetime.UTC, this provides a fallback for 3.11-

* use litellm.utils.get_utc_datetime

* remove unused timezone import

Co-authored-by: Matthew Farrellee <matt@cs.wisc.edu>
2025-05-09 14:13:57 -07:00
Krish DholakiaandGitHub 9c32525c17 build: update model in test (#10706) 2025-05-09 13:33:11 -07:00
Ishaan Jaff 02efffcbcb fix: test_team_update_sc_2 v1.68.2-nightly 2025-05-09 11:59:13 -07:00
Tom MatthewsandGitHub e2a9cd0c75 Add nscale support for streaming (#10698) 2025-05-09 11:39:23 -07:00
Ishaan Jaff 1a42245da6 test: fix test_team_update_sc_2 2025-05-09 09:38:03 -07:00
Krrish Dholakia 584facebf1 test: temporarily skip test due to change testing model change - need to update test for new model 2025-05-09 09:02:08 -07:00
GPTLocalhost (Word Add-in)andGitHub 86f19d2fae Add GPTLocalhost to "docs/my-website/docs/projects" (#10687)
* Create GPTLocalhost.md

* Update sidebars.js
2025-05-09 07:44:48 -07:00
Emerson GomesandGitHub 7372978c1a New azure models (#9956)
* Adding Azure Deepseek-v3-0324

* Adding Azure Llama4 models

* add function call support to azure deepseek-v3-0324
2025-05-09 07:20:59 -07:00
Krish DholakiaandGitHub d6bdd0473a Extract 'thinking' from nova response + Add 'drop_params' support for gpt-image-1 (#10680)
* test(base_llm_unit_tests.py): return '<thinking>' tag in response content

* fix(converse_transformation.py): extract `<thinking>` block from nova tool use response

Fixes https://github.com/BerriAI/litellm/issues/9063

* fix(factory.py): handle non-signature reasoning blocks to bedrock

pass as text input - bedrock raises ""User messages cannot contain reasoning content. Please remove the r
easoning content and try again." otherwise

* fix(main.py): Add drop params support for gpt

Fixes https://github.com/BerriAI/litellm/issues/10501

* fix(converse_transformation.py): fix linting error

* fix(utils.py): fix linting error

* test: cleanup test

* test: skip test until we have bedrock prompt caching permission
2025-05-08 23:01:38 -07:00
Krish DholakiaandGitHub a1964eab18 Realtime API - Set 'headers' in scope for websocket auth requests + reliability fix infinite loop when model_name not found for realtime models (#10679)
* fix(user_api_key_auth.py): add 'headers' to constructed request for websocket

Fix issue on some datastructure versions which require a headers field in scope

* test(test_user_api_key_auth.py): add unit testing for headers in scope change

* fix(router.py): migrate `_arealtime` to generic router endpoint

Fix infinite loop on model name missing for realtime api calls

* test(test_router_helper_utils.py): cleanup test post refactor
2025-05-08 22:50:09 -07:00
Krish DholakiaandGitHub 5325ee4382 fix(factory.py): Handle system only message to anthropic (#10678)
Fixes error where passing system only message to anthropic causes requests to fail
2025-05-08 22:44:39 -07:00
Krrish Dholakia 96e31edad3 build(proxy_server_config.yaml): move to model with higher quota 2025-05-08 22:18:27 -07:00
a0ee31edf8 [Feat] Add support for using Bedrock Invoke models in /v1/messages format (#10681)
* fix: add transform_anthropic_messages_request

* fix: add get_requested_response_api_optional_param

* fix: use base llm http handler for anthropic messages

* fix: add anthropic transform response

* fix: transform_anthropic_messages_response

* fix: fixes for anthropic messages

* fix: code qa fixes

* fix: pass thinking to anthropic

* fix: linting

* fixes

* feat: add folder for bedrock invoke messages

* feat: init bedrock invoke messages for anthropic claude family

* test: add bedrock invoke test for us anthropic

* test: test_anthropic_messages_non_streaming_bedrock_invokec

* feat: update anthropic messages transforms

* feat: update anthropic messages transforms

* Update litellm/utils.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: test_anthropic_messages_non_streaming

* fix: linting override

* fix: linting error

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-05-08 21:16:47 -07:00
Krrish Dholakia 49deea0df9 test: update test 2025-05-08 21:12:14 -07:00
Ishaan Jaff 42b8be526d ui new build 2025-05-08 20:21:14 -07:00
Ishaan JaffandGitHub 9d8f570f14 [Refactor] Anthropic /v1/messages endpoint - Refactor to use base llm http handler and transformations (#10677)
* fix: add transform_anthropic_messages_request

* fix: add get_requested_response_api_optional_param

* fix: use base llm http handler for anthropic messages

* fix: add anthropic transform response

* fix: transform_anthropic_messages_response

* fix: fixes for anthropic messages

* fix: code qa fixes

* fix: pass thinking to anthropic

* fix: linting

* fixes
2025-05-08 17:56:50 -07:00
Ishaan Jaff 5886b96135 bump litellm proxy extras 2025-05-08 16:23:45 -07:00
tanjiroandGitHub d4f7403cb9 Filter logs on status and model (#10670)
* added status filtering on logs

* added model filter

* fix linter

* fix model filtering

* modified status filter to use status column from LiteLLM_SpendLogs

* remove json import
2025-05-08 13:50:17 -07:00
Ishaan Jaff fcd51a8f4b fix: nscale models 2025-05-08 13:49:10 -07:00
Ishaan Jaff c2ce9c537b fix: model cost map 2025-05-08 12:46:13 -07:00
Ishaan Jaff e85323e8dd fix: linting 2025-05-08 12:42:42 -07:00
Ishaan JaffandGitHub d9f2415f65 [Refactor - Filtering Spend Logs] Add status to root of SpendLogs table (#10661)
* add status to spendlogs

* add status to spendlogs

* add status to spendlogs

* add status to spendlogs

* fix tests with spendlogs
2025-05-08 11:29:25 -07:00
Bryan LowandGitHub 416429e9f4 Add New Perplexity Models (#10652)
* add new perplexity models

* update backup json

* updated deep research

* updated backup json
v1.68.1.dev4
2025-05-08 08:56:16 -07:00
Tom MatthewsandGitHub ff279faf35 Add support for Nscale (EU-Sovereign) Provider (#10638)
* Add support for nscale provider

* Add image generation support and fix unit tests

* Add docs for nscale

* Fix unit test import issues

* Minor doc improvement

* Remove redundant null tokens from model cost map

* Address PR review comments for doc updates

* Revert changes to large text
2025-05-08 08:50:15 -07:00
Gary CaldwellandGitHub 78c264d7a5 Fix Slack alerting not working if using a DB (#10370)
* Try to add prints

* more print

* fix print

* batch debugging

* Change batch size

* Revert "Change batch size"

This reverts commit af16d8635f17e3928da9000bb9fec5b0af920816.

* Look into periodic task problems

* Fix missing periodic in slack init

* Initialize periodic flush on startup

* Log update_values

* more logging

* Fix startup

* Cleanup change

* Add a unit test for the change

* Renamed and moved to standard

* Merging in with new test

* comment change

* Extend the timeout because normal runs are over 5 min
2025-05-08 08:40:05 -07:00
Krish DholakiaandGitHub b8b78f1fde Support unified file id (managed files) for batches (#10650)
* refactor(managed_files.py): move enterprise feature into enterprise folder

prevent unexpected surprises

* refactor: safely handle enterprise hooks

* fix: fix ruff check errors

* fix(files_endpoints.py): cleanup enterprise code from OSS

* refactor: complete cleanup

* fix(managed_files.py): complete cleanup

* fix(managed_files.py): instrument to be able to update deployment values post-router selection and just before making llm call

* fix(managed_files.py): instrument to be able to update deployment values post-router selection and just before making llm call

* fix: fix linting error

* fix: fix linting error
2025-05-07 23:39:40 -07:00
Marc AbramowitzandGitHub fcaa4a9f30 Add user management functionality to Python client library & CLI (#10627)
* Add user mgmt functionality to client & CLI

* Add user mgmt client & CLI tests

* Add user mgmt client & CLI info to README.md's

* lint fixes: litellm/proxy/client/users.py

* Fix mypy errors
2025-05-07 23:26:26 -07:00
CasparandGitHub d680feb5dc Fix cache miss for gemini models with response_format (#10635)
* Fix unwanted mutation of kwargs in apply_response_schema_transformation

* add assertion the original dict is not mutated
2025-05-07 23:13:37 -07:00
xsg22andGitHub 2361bd98b0 Update crebras llama-3.1-70b model_prices_and_context_window (#10648) 2025-05-07 23:10:41 -07:00
Krish DholakiaandGitHub b178b08d63 Ollama - fix custom price cost tracking + add 'max_completion_token' support (#10636)
* fix(utils.py): check if ollama model has custom attributes

Fixes https://github.com/BerriAI/litellm/discussions/9630

* fix(ollama/completion): add 'max_completion_token' support for ollama
2025-05-07 22:53:47 -07:00
Krrish Dholakia b899e7d44c Squashed commit of the following:
commit 122f037990da9c008130cde0ec5bd54259b3d0b0
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Wed May 7 14:19:35 2025 -0700

    fix(ollama/completion): add 'max_completion_token' support for ollama

commit 314b68cfd4da36be8fbbc0c30f86da15e36438b7
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Wed May 7 13:27:56 2025 -0700

    fix(utils.py): check if ollama model has custom attributes

    Fixes https://github.com/BerriAI/litellm/discussions/9630
2025-05-07 22:53:28 -07:00
Ishaan Jaff 88f5f9b7f8 fix ai21 test 2025-05-07 21:45:57 -07:00
Ishaan Jaff 580e221000 fix ai21 test 2025-05-07 21:26:35 -07:00
Ishaan Jaff 764e9a1934 bump litellm proxy extras 2025-05-07 21:23:05 -07:00
Ishaan Jaff 24acee3596 new ui build 2025-05-07 21:10:29 -07:00
f524eb4bd5 Contributor PR - MCP Server DB Schema (#10634) (#10641)
* MCP Server DB Schema (#10634)

* feat: initial mcp db schema

* user -> team table for mcp_servers

* remove mcp_servers from team and verificationtoken tables

---------

Co-authored-by: wagnerjt <wagnerjt@github.com>

* add mcp table

* Team and Key Object Permission table for MCP Servers (#10643)

* feat: initial mcp db schema

* user -> team table for mcp_servers

* remove mcp_servers from team and verificationtoken tables

* adding ObjectPermissionTable relationship for mcpservers

* fix: object_permission_id and added relationship to User and Organization tables

---------

Co-authored-by: wagnerjt <wagnerjt@github.com>

* migration file - add_mcp_key_team_permission_mgmt

---------

Co-authored-by: Tyler Wagner <wagnerjt@users.noreply.github.com>
Co-authored-by: wagnerjt <wagnerjt@github.com>
2025-05-07 21:09:51 -07:00
Ishaan JaffandGitHub ad970ca41d [Feat] Add endpoints to manage email settings (#10646)
* add email notifs endpoints

* ui refactor email settings

* allow controlling email settings / notifs through ui

* add email settings to enterprise router

* change order of email events

* manage email settings in general_settings

* fix location of email settings types

* testing for email settings
2025-05-07 18:56:17 -07:00
Ishaan JaffandGitHub 6a3ea83b23 [Feat] Bedrock Guardrails - Add support for PII Masking with bedrock guardrails (#10642)
* allow defining mask_request_content for guardrails

* allow pii masking with bedrock

* implement bedrock pre call hook

* docs bedrock pii masking

* fix linting error

* fix code quality checks
2025-05-07 18:30:57 -07:00
tanjiroandGitHub 959ba21585 Filter on logs table (#10644)
* remove existing filter

* added team id based filtering
2025-05-07 17:51:03 -07:00