Commit Graph

4385 Commits

Author SHA1 Message Date
Ishaan Jaff efa782d6d2 [Feat] Add Nvidia NIM Rerank Support (#15152)
* feat: add NvidiaNimRerankConfig

* fix: NvidiaNimRerankConfig

* fix: NvidiaNimRerankConfig

* fix routing to nvidia nim

* docs nvidia nim rerank

* TestNvidiaNim

* nvidia nim rerank fixes

* fix rerank

* transform_rerank_response

* Usage with LiteLLM Proxy

* fixes linting

* NvidiaNimRerankConfig.DEFAULT_NIM_RERANK_API_BASE

* fix Custom API Base URL

* fix rerank base

* fix main.py

* fix transform

* fix linting

* map_cohere_rerank_params

* ruff fix

* linting fixes

* ruff fix
2025-10-02 18:58:52 -07:00
Sameer Kankute 544db8d140 (feat)Litellm x twelvelabs bedrock[Async Invoke Support] (#14871)
* Add async invoke support

* Add docs and correct embedding response

* fix cicd erros

* fix cicd erros

* fix mypy error

* Add litellm param input_type

* Update the docs
2025-10-02 18:52:33 -07:00
Krish Dholakia 7b8426af53 Merge pull request #15130 from deepanshululla/feature/logging_payload_spec_update
Add provider name to payload specification
2025-10-02 17:29:24 -07:00
Amir Refaee 8991657d67 added railtracks to projects using litellm (#15144) 2025-10-02 14:21:38 -07:00
Krrish Dholakia ebba9e0b2a docs: cleanup docs 2025-10-02 09:51:16 -07:00
deepanshu 2512d89872 Add provider name to payload specification 2025-10-02 10:23:26 -04:00
Krish Dholakia cb39a1bb60 Merge pull request #15024 from kowyo/main
docs: use docker compose instead of docker-compose
2025-10-01 21:34:42 -07:00
Deepanshu Lulla 68adca04c8 Gitlab based Prompt manager (#14988)
* add prompt

* add prompt

* add prompt

* add prompt

* add prompt management via gitlab

* gitlab client

* gitlab client

* gitlab client

* fix lint issues

* fix lint issues

* remove router changes

---------

Co-authored-by: deepanshu <deepanshu.lulla@hq.bill.com>
2025-10-01 18:13:11 -07:00
Ishaan Jaff e73d053de3 [Fix] Proxy Auth - Ensure LLM_API_KEYs can access pass through routes (#15115)
* test_virtual_key_llm_api_routes_allows_registered_pass_through_endpoints

* fix: is_registered_pass_through_route

* docs fix
2025-10-01 14:09:01 -07:00
Krish Dholakia 64083111d3 (Feat) Add Vertex AI Live API WebSocket Passthrough with Cost Tracking
(Feat) Add Vertex AI Live API WebSocket Passthrough with Cost Tracking
2025-09-30 21:14:16 -07:00
Ishaan Jaff 0ca11eefde [Feat] Guardrails - add logging for important status fields (#15090)
* add StandardLoggingPayloadStatusFields

* add status_fields

* add StandardLoggingPayloadStatusFields

* noma guard: add_standard_logging_guardrail_information_to_request_data

* fix: StandardLoggingPayloadStatusFields

* fix tests

* fix StandardLoggingPayloadStatus

* get_standard_logging_object_payload

* test_bedrock_guardrail_status_failure

* fix: _get_status_fields

* fixes new guardrail tracing

* fix ruff
2025-09-30 18:38:07 -07:00
Ishaan Jaffer f46f9d3fd9 docs azure passthrough api fixes 2025-09-30 15:54:09 -07:00
Krish Dholakia c475723c67 Merge pull request #15044 from BerriAI/litellm_dev_09_29_2025_p1
MCP - enforce server permissions on call tools + Teams - add model specific tpm/rpm limits to teams on LiteLLM
2025-09-30 15:32:53 -07:00
Krish Dholakia 39d1d2a379 Merge pull request #14840 from eddierichter-amd/lemonade-integration
Add AMD Lemonade provider support
2025-09-30 12:34:30 -07:00
Ishaan Jaffer 45bfd2599c docs fix 2025-09-30 11:35:15 -07:00
Eddie Richter abaf77da43 Small updates to documentation 2025-09-30 12:12:24 -06:00
Eddie Richter 929510ef5d Adding unit tests and documentation 2025-09-30 12:12:24 -06:00
Krish Dholakia 551a171243 Merge pull request #15043 from cedarm/fix-remove-vertex-latest
fix: remove invalid vertex -latest models
2025-09-29 21:54:53 -07:00
Yuta Saito def4afedd7 doc: add missing api_key parameter 2025-09-30 12:52:17 +09:00
Krrish Dholakia 91f420160f docs(mcp.md): document oauth support 2025-09-29 13:47:21 -07:00
Ishaan Jaff 038863a1fe [Feat] Add new claude-sonnet-4-5 model family (#15041)
* add new claude-sonnet-4-5

* docs fix

* fix tool_use_system_prompt_tokens

* add anthropic.claude-sonnet-4-5-20250929 to bedrock converse models
2025-09-29 13:09:00 -07:00
Cedar Myers 5357b1e102 fix: remove invalid vertex -latest models 2025-09-29 16:04:33 -04:00
Kowyo 858f557bce docs: use docker compose instead of docker-compose 2025-09-29 11:59:53 +00:00
Krish Dholakia e7939b0521 Merge pull request #14983 from abhijitjavelin/main
Feat: Add Javelin standalone guardrails integration for LiteLLM Proxy
2025-09-28 17:59:07 -07:00
Cursor Agent c684a18e38 feat: Add OAuth2 auth and scheduled key rotation
Co-authored-by: teddy <teddy@berri.ai>
2025-09-28 16:39:35 +00:00
Krish Dholakia c328b4a036 Merge pull request #15002 from BerriAI/litellm_dev_09_27_2025_p3
MCP - specify forwardable headers, specify allowed/disallowed tools for MCP servers
2025-09-28 09:21:52 -07:00
Krrish Dholakia e6810b5f87 docs(mcp.md): add allow/disallowed tools to docs 2025-09-28 08:50:40 -07:00
eycjur 4f1e409082 use extra_query for get request 2025-09-28 18:03:32 +09:00
Sameer Kankute f510ac15f8 Merge branch 'main' into litellm_live_api_passthrough 2025-09-28 09:45:57 +05:30
abhijitjavelin aedcedb2b6 Merge branch 'BerriAI:main' into main 2025-09-28 07:53:06 +05:30
Krrish Dholakia db27600ce9 docs(mcp.md): update docs with new 'extra_headers' tutorial 2025-09-27 18:13:19 -07:00
Ishaan Jaff 6299f07e54 [Draft] v1.77.5 Release note (#14985)
* docs draft

* docs fix

* docs

* docs fix

* doc fix

* docs

* docs fix
2025-09-27 16:33:58 -07:00
Ishaan Jaffer c366036312 docs fix 2025-09-27 13:51:55 -07:00
Ishaan Jaffer 329e494094 docs notes 2025-09-27 13:50:44 -07:00
Ishaan Jaffer e270e0a797 docs fix 2025-09-27 13:42:15 -07:00
abhijitjavelin adea1bfbc4 Merge branch 'BerriAI:main' into main 2025-09-28 01:08:16 +05:30
Abhijit L 7bba372ac0 feat: add javelin guardrails 2025-09-28 01:07:08 +05:30
Krish Dholakia 3e3e3837a1 Merge branch 'main' into litellm_dev_09_24_2025_p1 2025-09-27 11:21:07 -07:00
Krrish Dholakia 5c6a863e40 docs: cleanup 2025-09-27 11:10:44 -07:00
Teddy Amkie fdfff39222 Add Azure passthrough documentation and sidebar entry (#14958)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-09-27 10:13:16 -07:00
Sameer Kankute 94edbd1b35 Merge pull request #14957 from BerriAI/main
merge main
2025-09-27 01:54:44 +05:30
Ishaan Jaff ea8d0bb7d5 fix: re-add scheduled rotations 2025-09-26 11:40:46 -07:00
Ishaan Jaff d04c6d4eea [Feat] Add new anthropic web fetch tool support (#14951)
* add web_fetch tool for ANTHROPIC_HOSTED_TOOLS

* add ANTHROPIC_BETA_HEADER_VALUES, ANTHROPIC_HOSTED_TOOLS

* feat: add web fetch tool anthropic

* test_anthropic_tool_use

* docs web fetch

* docs fix

* docs fix
2025-09-26 11:04:11 -07:00
Olivier Cornelis 6fe8c33448 docs: add documentation for additional cost-related keys in custom pricing (#14949)
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
2025-09-26 10:32:30 -07:00
Ishaan Jaff 360befa216 [Feat] Add support for Gemini 2.5 Flash and Flash-lite preview models (09-2025 release) (#14948)
* add gemini-2.5-flash-preview-09-2025

* docs add gemini-2.5-flash-preview-09-2025 model family
2025-09-26 09:51:38 -07:00
Krish Dholakia 79ebb2c95e Merge pull request #14888 from mrFranklin/feat/improve-opik
feat: improve opik integration code
2025-09-25 23:40:41 -07:00
Teddy Amkie 0e8f60fe67 Remove aiohttp_ prefix from config (#14920)
* Fix: Correct model name in load test advanced docs

Co-authored-by: teddy <teddy@berri.ai>

* Fix: Update load testing provider to openai/fake

Co-authored-by: teddy <teddy@berri.ai>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-09-25 16:09:41 -07:00
Sameer Kankute 1681bf7175 (Feat) Add BitBucket Integration for Prompt Management (#14882)
* Add bitbucket integration for prompt management

* remove not needed file

* remove not needed file

* Add the correct readme

* Add the correct readme

* fix test for bitbucket

* fix test for bitbucket
2025-09-25 15:51:00 -07:00
Teddy Amkie dcbccd1fea Corrected docs updates sept 2025 (#14916)
* docs: Corrected documentation updates from Sept 2025

This PR contains the actual intended documentation changes, properly synced with main:

 Real changes applied:
- Added AWS authentication link to bedrock guardrails documentation
- Updated Vertex AI with Gemini API alternative configuration
- Added async_post_call_success_hook code snippet to custom callback docs
- Added SSO free for up to 5 users information to enterprise and custom_sso docs
- Added SSO free information block to security.md
- Added cancel response API usage and curl example to response_api.md
- Added image for modifying default user budget via admin UI
- Re-ordered sidebars in documentation

 Sync issues resolved:
- Kept all upstream changes that were added to main after branch diverged
- Preserved Provider-Specific Metadata Parameters section that was added upstream
- Maintained proper curl parameter formatting (-d instead of -D)

This corrects the sync issues from the original PR #14769.

* docs: Restore missing files from original PR

Added back ~16 missing documentation files that were part of the original PR:

 Restored files:
- docs/my-website/docs/completion/usage.md
- docs/my-website/docs/fine_tuning.md
- docs/my-website/docs/getting_started.md
- docs/my-website/docs/image_edits.md
- docs/my-website/docs/image_generation.md
- docs/my-website/docs/index.md
- docs/my-website/docs/moderation.md
- docs/my-website/docs/observability/callbacks.md
- docs/my-website/docs/providers/bedrock.md
- docs/my-website/docs/proxy/caching.md
- docs/my-website/docs/proxy/config_settings.md
- docs/my-website/docs/proxy/db_deadlocks.md
- docs/my-website/docs/proxy/load_balancing.md
- docs/my-website/docs/proxy_api.md
- docs/my-website/docs/rerank.md

 Fixed context-caching issue:
- Restored provider_specific_params.md to main version (preserving Provider-Specific Metadata Parameters section)
- Your original PR didn't intend to modify this file - it was just a sync issue

Now includes all ~26 documentation files from the original PR #14769.

* docs: Remove files that were deleted in original PR

- Removed docs/my-website/docs/providers/azure_ai_img_edit.md (was deleted in original PR)
- sdk/headers.md was already not present

Now matches the complete intended changes from original PR #14769.

* docs: Restore azure_ai_img_edit.md from main

- Restored docs/my-website/docs/providers/azure_ai_img_edit.md from main branch
- This file should not have been deleted as it was a newer commit
- SDK headers file doesn't exist in main (was reverted) and wasn't part of your original changes

Fixes the file restoration issues.

* docs: Fix vertex.md - preserve context caching from newer commit

- Restored vertex.md to main version to preserve context caching content (lines 817-887)
- Added back only your intended change: alternative gemini config example
- Context caching content from newer commit is now preserved

Fixes the vertex.md sync issue where newer content was incorrectly deleted.

* docs: Fix providers/bedrock.md - restore deleted content from newer commit

- Restored providers/bedrock.md to main version
- Preserves 'Usage - Request Metadata' section that was added in newer commit
- Your actual intended change was to proxy/guardrails/bedrock.md (authentication tip) which is preserved
- Now only has additions, no subtractions as intended

Fixes the bedrock.md sync issue.

* docs: Restore missing IAM policy section in bedrock.md

Added back your intended IAM policy documentation that was lost when restoring main version:

 Added IAM AssumeRole Policy section:
- Explains requirement for sts:AssumeRole permission
- Shows error message example when permission missing
- Provides complete IAM policy JSON example
- Links to AWS AssumeRole documentation
- Clarifies trust policy requirements

Now bedrock.md has both:
- All newer content preserved (Request Metadata section)
- Your intended IAM policy addition restored

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-09-25 15:49:19 -07:00
Ishaan Jaff 2dd38420a7 [Feat] - Cost Tracking - show input, output, tool call cost breakdown in StandardLoggingPayload (#14921)
* add new CostBreakdown typed dict

* fix CostBreakdown type

* fix fix _store_cost_breakdown_in_logging_obj

* fix CostBreakdown

* test_cost_breakdown_in_standard_logging_payload
2025-09-25 15:48:22 -07:00