Commit Graph

3630 Commits

Author SHA1 Message Date
Ishaan Jaff a366f9247a docs update s3 logger 2025-06-02 21:53:47 -07:00
Ishaan Jaff 3db272b6d2 [Perf] - Add Async + Batched S3 Logging (#11340)
* fix: add s3 v2 async

* fix: add s3 v2 async

* fix: add s3 v2 async

* test: s3 v2 logging

* fixes: s3 logging

* fixes: s3 logging use max upload batch size

* fixes: s3 logging tests

* fixes: s3 logging tests

* fixes: s3 logging tests
2025-06-02 21:52:34 -07:00
Krish Dholakia 00be76abf4 UI - Custom Server Root Path (Multiple Fixes) (#11337)
* fix(proxy_server.py): working swagger on custom base

removes the swagger monkey patch - this seems to render the swagger on custom base paths

* fix(ui/): working custom auth uptil login success event

* fix(ui/): working custom server root path for login

* fix(proxy_server.py): create typed dict for ui returned token

allows better documentation of expected params

* refactor(proxy_server.py): refactor all ui login endpoints to use same returned ui token object

* feat(ui_sso.py): add server root path to ui token

* feat(ui_sso.py): allows ui to call correct endpoint

* fix(networking.tsx): update proxy base url with custom root path

* fix(networking.tsx): handle updating proxy base url for non-local instances

* refactor: remove uneccessary references to proxybaseurl in ui code - reduce potential for errors

* fix: fix linting error

* fix(onboarding_link.tsx): fix onboarding link when custom server path is set

* feat(ui_discovery_endpoints.py): add new public .well-known/ route for litellm ui config

returns the server root path and proxy base url for constructing api calls

* feat(_types.py): add litellm well known config as public route

allows ui to query it

* fix(/_types.py): add .well-known config to as public route

* fix(page.tsx): create pattern for loading in ui config before making network requests

ensures requests are formatted correctly

* fix(page.tsx): call credential endpoint once ui config is loaded

* fix(page.tsx): route correctly to litellm dashboard from new user login

* fix(page.tsx): remove hardcoded `/litellm` for /sso/key/generate request

* fix(proxy_server.py): re-add moderations endpoint

* fix(proxy_server.py): mount __next__ at / and /litellm

allows it to work when proxy is mounted on root

* docs(contributing.md): remove /ui on ui doc - it will now run on root

* docs(custom_root_ui.md): add docs on custom root path
2025-06-02 17:48:03 -07:00
Ishaan Jaff 8a4b163453 [Feat] DD Trace - Add instrumentation for streaming chunks (#11338)
* fix: add tracing for litellm.completion

* fix: NULL span add trace

* fix: add tracing for litellm.completion streaming

* fix: add tracing for litellm.completion streaming

* fix: use a constant for str
2025-06-02 16:48:39 -07:00
Cole McIntosh 702e399d94 Add presidio_language yaml configuration support for guardrails (#11331)
* feat: Add presidio_language parameter for PII analysis configuration via yaml config files

- Introduced presidio_language parameter in guardrail_initializers and guardrail_hooks to allow language specification for Presidio PII analysis via yaml config files
- Updated PresidioPresidioConfigModelUserInterface to include presidio_language with a default value of "en".
- Added tests to verify language configuration functionality, ensuring correct language usage in analyze requests.

* docs: Enhance PII masking documentation with language support details

- Added information on configuring the default language for PII analysis using the `presidio_language` parameter in both the UI and YAML configuration.
- Included supported language codes (English, Spanish, German) and their precedence in language settings.
- Provided examples for default language configuration and per-request language overrides to clarify usage.
2025-06-02 13:13:17 -07:00
Krrish Dholakia 9630386f2b docs: add release candidate notice 2025-06-01 22:39:57 -07:00
Krish Dholakia 83becdbc11 Litellm doc fixes 05 31 2025 (#11305)
* docs: cleanup

* docs: add anthropic file tutorial

* docs: add to sidebar
2025-06-01 00:53:56 -07:00
Ishaan Jaff bdfa24be23 update doc v1.72.0.rc 2025-05-31 20:57:48 -07:00
Ishaan Jaff 10fa45d987 docs fix 2025-05-31 16:29:19 -07:00
Ishaan Jaff 3f616423a4 docs fixes 2025-05-31 15:30:53 -07:00
Ishaan Jaff ab2f066df8 docs prometheus 2025-05-31 14:26:42 -07:00
Ishaan Jaff cdfb6b8c37 docs prometheus end user tracking 2025-05-31 14:21:37 -07:00
Ishaan Jaff 170af8f2c8 [Docs] 1.72.0-stable release note (#11295)
* draft 1.72.0 stable

* docs - note on aiohttp transport

* docs - section for guardrails

* clean up key highlights

* docs aiohttp transport

* docs cleanup

* docs organize logging/guardrail section

* docs logging+guardrails

* docs add prometheus note

* docs fixes release note

* docs 1.72.0-stable

* docs vector store permissions
2025-05-31 14:15:16 -07:00
Ishaan Jaff e011167317 docs DISABLE_AIOHTTP_TRANSPORT 2025-05-31 12:30:52 -07:00
Ishaan Jaff 5e6f6ddc52 [Feat]: Add Bedrock InvokeAgents as a /chat/completions route on LiteLLM (#11239)
* feat: init structure for bedrock AGENTs

* feat: add basic  routing for bedrock AGENTs

* feat: add basic transforms for bedrock AGENTs

* fix: url for bedrock agent runtime

* fix: working agents request

* feat: working agents non-streaming request

* feat: bedrock agents

* feat: add streaming for bedrock agents

* feat: add cost tracking for bedrock agents

* docs litellm with bedrock agents

* fix: linting errors

* test: invoke agents tests
2025-05-29 16:48:55 -07:00
Fadil Rahman dd8d23be62 Removed syntax error from docs (#11242) 2025-05-29 15:34:16 -07:00
Krrish Dholakia 368e59181d docs(entity_usage.tsx): fix bad link 2025-05-29 11:00:18 -07:00
VigneshwarRajasekaran 974c928672 changed model name is request and updating to latest response format in proxy quick start guide as 3.5 turbo is retired and leads to error when a call is made using liteLLM to respective Azure-openAI endpoints. (#11223) 2025-05-29 08:27:56 -07:00
Emmanuel Ferdman 7dcfde2a1c fix the litellm CLA reference (#11230)
Signed-off-by: Emmanuel Ferdman <emmanuelferdman@gmail.com>
2025-05-29 08:27:14 -07:00
Krrish Dholakia dececb84c1 docs(image_edits.md): clarify proxy version support for image edits 2025-05-29 07:50:41 -07:00
Krish Dholakia ba39f9e360 Helicone base url support + fix for embedding cache hits on str input (#11211)
* fix(helicone.py): add helicone api base support

Fixes https://github.com/BerriAI/litellm/issues/10825

* test: add unit test for cache hit response on embedding calls

* fix(caching_handler.py): fix handling cache hit on embedding when input is string

Fixes LIT-197

* docs(helicone_integration.md): document new helicone api base param
2025-05-28 22:02:55 -07:00
Krrish Dholakia 75f591a6e2 docs(cli.md): add new skip_server_startup flag 2025-05-28 21:47:08 -07:00
Vinnie-Singleton-NN 178a614d4a Add sentry sample rate (#10283)
* Add SENTRY_API_SAMPLE_RATE configuration option for Sentry SDK

* removed print line

* Update Sentry documentation with sample rate information

---------

Co-authored-by: Vinnie <vinnie@Vinnies-MacBook-Pro.local>
2025-05-28 16:44:10 -07:00
Ishaan Jaff 9a6d5c119e feat: Allow Adding MCP Servers Through LiteLLM UI (#11208)
* feat: MCP Servers with CRUD operations (#10699)

* feat: mcp CRUD operations with authn/authz

* feat: mcp server UI

* mcp server page with overview, mcp tools, and settings page

* Adding MCP Server flow

* prisma generate before test

* UI callbacks add/remove with api server refetch

* test fix: poetry run prisma

* feat: mcp server db and config connection

* fix: MCPTool filter on description when not present

* feat: mcp on UI and integrated with list tools

* feat: Update mcp server endpoint

* tests: Unit and integration tests for mcp management endpoints

* fix: docs and ensuring global_mcp_manage up to date

* ui: remove the mcp tools view

* fix: ruff lint

* fix: unit -> integration test area

* fix(ui): remove left nav menu of previous tools

---------

Co-authored-by: wagnerjt <wagnerjt@github.com>
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>

* fix: sync DB MCP tools with in memory

* fix: sync DB MCP tools with in memory

* fix: stop using prisma.models

* fix: code qa check

* fix: import MCP

* fix: code QA checks

* fix: code QA checks

* fixes - only list tools for the specific MCP server

* fix: only list MCP tools for selected server

* fix linting error

---------

Co-authored-by: Tyler Wagner <wagnerjt@users.noreply.github.com>
Co-authored-by: wagnerjt <wagnerjt@github.com>
2025-05-28 16:29:27 -07:00
Niko Izsak 85525c5c6b Azure OIDC provider improvements + OIDC audience bug fix (#10054)
* added support for custom scope in get_azure_ad_token_provider

* if AZURE_FEDERATED_TOKEN_FILE not set, use azure_token_provider to retrive token with the oidc audiances as scope

* fix bug where oidc audience that contains "/" won't be extract correctly

* added tests for get_secret with oidc

* moved tests to litellm tests folder

* tes file naming aligned with source code

* renamed test_main because it caused issue in the test in github workflow

* updated docs

* moved docs to the end of file

* fix aws region in example config

* renamed test file

* added support for custom scope in get_azure_ad_token_provider

* if AZURE_FEDERATED_TOKEN_FILE not set, use azure_token_provider to retrive token with the oidc audiances as scope

* fix bug where oidc audience that contains "/" won't be extract correctly

* added tests for get_secret with oidc

* moved tests to litellm tests folder

* tes file naming aligned with source code

* renamed test_main because it caused issue in the test in github workflow

* updated docs

* moved docs to the end of file

* fix aws region in example config

* renamed test file

* fix merge conflict resolution error

* added support for custom scope in get_azure_ad_token_provider

* if AZURE_FEDERATED_TOKEN_FILE not set, use azure_token_provider to retrive token with the oidc audiances as scope

* fix bug where oidc audience that contains "/" won't be extract correctly

* added tests for get_secret with oidc

* moved tests to litellm tests folder

* tes file naming aligned with source code

* renamed test_main because it caused issue in the test in github workflow

* updated docs

* moved docs to the end of file

* fix aws region in example config

* renamed test file

* added tests for get_secret with oidc

* moved tests to litellm tests folder

* tes file naming aligned with source code

* renamed test_main because it caused issue in the test in github workflow

* updated docs

* moved docs to the end of file

* renamed test file

* fix merge conflict resolution error
2025-05-28 09:33:13 -07:00
RMeans d181138ad5 Add Pangea provider to Guardrails hook (#10775)
* added Pangea as a guardrail vendor

* Adding output recipe, cleaning up some imports

* Add Pangea guardrails tests

* Add docs and sidebar

* Move to use async_precall_hook instead of moderation hook

* Update to "new" format (accept mode)

Add response for redaction, support transforming request / response
based off the original type of call (/v1/completions,
/v1/chat/completions)

* Fix tests

* Fix unused imports

* Fix .md

---------

Co-authored-by: michael weinberger <michael.lee.weinberger@gmail.com>
2025-05-27 22:01:28 -07:00
Krish Dholakia 7072466775 VertexAI - codeExecution tool support + anyOf handling (#11195)
* fix(vertex_and_google_ai_studio_gemini.py): handle both camel case and underscores in the tool for vertex ai code execution

support vertex ai code execution

* docs(vertex.md): add code execution example to vertex ai

* fix(vertex_ai/common_utils.py): when anyof in field, just select anyof - don't include other k,v pairs - vertex throws error

Fixes https://github.com/BerriAI/litellm/issues/11164

* fix(common_utils.py): add title field inside anyof - to retain some description

Addresses https://github.com/BerriAI/litellm/issues/11164#issuecomment-2914728385
2025-05-27 21:23:14 -07:00
Akim Tsvigun acaa80294c Integration with Nebius AI Studio added (#11143)
* integration with Nebius AI Studio added

* Merged with main

* Reviewer's comments resolved

* spelling error fixed

* accidental change reverted
2025-05-27 11:05:22 -07:00
k10 9a35c41462 added logprobs (#11161) 2025-05-26 11:21:47 -07:00
Ishaan Jaff 77d2615185 docs release notes 2025-05-24 21:24:44 -07:00
Ishaan Jaff 66c44bb9f3 docs notes 2025-05-24 21:00:01 -07:00
Ishaan Jaff 39c85c22de docs document all config settings 2025-05-24 20:48:09 -07:00
Krrish Dholakia bfbec06214 docs(index.md): cleanup 2025-05-24 19:53:59 -07:00
Ishaan Jaff 95b59cbf4d docs image edits 2025-05-24 19:46:10 -07:00
Ishaan Jaff 7fae80b4c6 docs image edits 2025-05-24 19:44:54 -07:00
Ishaan Jaff cadd86af95 docs image edit 2025-05-24 19:44:20 -07:00
Ishaan Jaff 1637413677 docs image edits 2025-05-24 19:43:48 -07:00
Krrish Dholakia cb58fc7f6d docs: doc improvements 2025-05-24 19:06:00 -07:00
Krrish Dholakia 061a2a730e docs(managed_finetuning.md): cleanup 2025-05-24 18:21:06 -07:00
Ishaan Jaff 7775e8471c docs benchmarks 2025-05-24 17:43:38 -07:00
Ishaan Jaff 39feb742cd docs aiohttp benchmarks 2025-05-24 17:41:10 -07:00
Ishaan Jaff 85bd3cfca1 fixes 2025-05-24 17:31:57 -07:00
Ishaan Jaff c73ac784d6 docs latest release 2025-05-24 17:31:19 -07:00
Ishaan Jaff c761c0ca2f docs release notes 2025-05-24 17:23:31 -07:00
Ishaan Jaff e957fbf38c docs release notes 2025-05-24 17:17:56 -07:00
Ishaan Jaff db96c8ca53 docs release notes 2025-05-24 17:17:44 -07:00
Ishaan Jaff e7ede28189 notes update 2025-05-24 17:17:30 -07:00
Ishaan Jaff c3f74b47f8 v1.71.1-stable - notes (#11133)
* draft notes

* fixes anthropic

* fixes release notes
2025-05-24 16:56:25 -07:00
Krrish Dholakia c725cbbde6 docs: add managed finetuning docs 2025-05-24 16:14:35 -07:00
Krrish Dholakia 76107bfb6b docs(litellm_managed_files.md): add tutorial on managing file permissions 2025-05-24 14:11:15 -07:00