Commit Graph

4843 Commits

Author SHA1 Message Date
Ali Saleh 6b5ad5d5a6 docs: Update Instructions For Phoenix Integration (#17373) 2025-12-02 22:03:54 -08:00
Ishaan Jaff 427074ac6e Fix: Datadog callback regression when ddtrace is installed (#17393)
* fix DD agent host logging

* docs fix

* test_datadog_agent_configuration

* test_datadog_ignores_ddtrace_agent_host
2025-12-02 17:27:50 -08:00
Ishaan Jaff 6c188c5ae2 [Feat] New model/provider - Adds support for Google Cloud Chirp3 HD on /speech (#17391)
* docs vertex tts

* place vertex ai types in file

* use VertexAITextToSpeechConfig

* use vertex_voice_dict

* refactor docs

* docs vertex ai chirp

* TestVertexAITextToSpeechConfig

* new provider vertex ai chirp3

* test_litellm_speech_vertex_ai_chirp

* add vertex_ai/chirp cost trackign
2025-12-02 15:36:23 -08:00
Ishaan Jaff db6c6eea89 [Docs] Add guide on how to debug gateway error vs provider error (#17387)
* add error diagnosis

* docs error diagnosis
2025-12-02 14:10:00 -08:00
Cesar Garcia 81f4d863ca docs: add Azure AI Foundry documentation for Claude models (#17104)
* docs: add Azure AI Foundry documentation for Claude models

Add documentation explaining how to use Claude models (Sonnet 4.5,
Haiku 4.5, Opus 4.1) deployed on Azure AI Foundry with LiteLLM.

Azure exposes Claude using Anthropic's native API, so users can use
the existing anthropic/ provider with their Azure endpoint.

Closes #17066

* docs: Add alternative method for Azure AI Foundry using anthropic/ provider

Document that users can use anthropic/ provider with Azure endpoint
as an alternative to the dedicated azure_ai/ provider.
2025-12-02 09:08:10 -08:00
Sameer Kankute 4ac9e4c81c Merge pull request #17345 from BerriAI/litellm_fix_jwt_auth_route_issue
Add other routes in jwt auth
2025-12-02 22:21:04 +05:30
Alexsander Hamir 9ff2ecc16d Fix: update default proxy_batch_write_at number (#17355)
The default is 10 seconds, not 30.
2025-12-02 05:52:00 -08:00
AlexsanderHamir 7324905c95 fix: update default database connection number 2025-12-02 05:29:03 -08:00
Sameer Kankute 6d296b1d25 Add other routes in jwt auth 2025-12-02 14:00:34 +05:30
Krish Dholakia 4c7a988454 Guardrail API V2 - user api key metadata, session id, specify input type (request/response), image support (#17338)
* refactor(generic_guardrail_api.py): refactor to update to new guardrail api logic

* refactor: refactor llm api integrations to support passing in text as a list[str] instead of one at a time

* refactor: fix linting errors

* refactor: pass request type to guardrail api

allows request vs. response processing to occur

* feat: pass user api key dict information to the guardrail api

* fix: pass user api key dict information to the guardrail api

* feat: pass litellm call id + trace id, if present

* docs: update docs
2025-12-01 20:11:58 -08:00
YutaSaito da5b81c1ff feat: add experimental latest-user filtering for Bedrock (#17282)
* feat: add experimental latest-user filtering for Bedrock

* doc: add experimental bedrock latest-message flag
2025-12-01 20:02:28 -08:00
Cesar Garcia 965406c643 feat(provider): add Z.AI (Zhipu AI) as built-in provider (#17307)
* feat(provider): add Z.AI (Zhipu AI) as built-in provider

Add support for Z.AI GLM models as a native OpenAI-compatible provider.

- Add "zai" to openai_compatible_providers list
- Add ZAI enum to LlmProviders
- Add provider URL resolution for https://api.z.ai/api/paas/v4
- Add 8 GLM models with pricing to model cost maps:
  - glm-4.6 (200K context, $0.6/$2.2 per 1M tokens)
  - glm-4.5, glm-4.5v, glm-4.5-x, glm-4.5-air, glm-4.5-airx
  - glm-4-32b-0414-128k
  - glm-4.5-flash (free tier)
- Add unit tests for provider integration

Closes #17289

* docs: add Z.AI provider documentation

- Add zai.md with usage examples, model list, and pricing
- Add to sidebars.js navigation
2025-12-01 19:56:47 -08:00
Elias 37ecb03d4f Add support of audio transcription for OVHcloud (#17305) 2025-12-01 18:26:39 -08:00
Krish Dholakia b6d6f834e0 (feat) Generic Guardrail API - allows guardrail providers to add INSTANT support for LiteLLM w/out PR to repo (#17175)
* feat(generic_guardrail_api.py): new generic api for guardrails

Allows guardrail providers to work with litellm for guardrails without needing to make a PR to LiteLLM

* docs(generic_guardrail_api.md): document new generic guardrail api

* Fix: Improve PII detection and guardrail API integration

Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>

* feat: correctly extract raw request from guardrail api

* docs(generic_guardrail_api.md): document this is a beta feature

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-12-01 14:29:52 -08:00
Krrish Dholakia c9afb86994 docs(azure_ai.md): document anthropic model usage on azure ai 2025-12-01 14:06:31 -08:00
Krrish Dholakia 7a46f3a083 docs: document azure ai provider for anthropic 2025-12-01 14:05:54 -08:00
Ishaan Jaff 24f847b84c [Feat] JWT Auth - AI Gateway, allow using regular OIDC flow with user info endpoints (#17324)
* feat: allow fetching OIDC user info

* test: use test_auth_builder_with_oidc_userinfo_enabled gets user info when enabled

* fix tool permission doc

* docs fix diagram
2025-12-01 13:59:00 -08:00
Ishaan Jaff ce0dc0c8b9 [Feat] WatsonX - allow passing zen_api_key dynamically (#16655)
* test_watsonx_zen_api_key_from_client

* zen api key

* docs using zen api key
2025-12-01 12:55:47 -08:00
YutaSaito a73bd751fc doc: add images for tool permission guardrail (#17322) 2025-12-01 12:38:49 -08:00
orgersh92 7808a610f8 Fix session consistency, move Lasso API version away from source code (#17316)
* store and fetch lasso-conversation id from cache

* include gateway/v# in the baseUrl to allow simpler version migrations in the future

* add tests for cached conversation ID
2025-12-01 10:03:51 -08:00
Sameer Kankute 7f42b9b987 Merge pull request #17193 from BerriAI/litellm_twelvelabs_int
Added support for twelvelabs pegasus
2025-11-28 22:09:00 +05:30
Sameer Kankute 9d058398df Fix pegasus response and add doc 2025-11-28 21:41:25 +05:30
Sameer Kankute bcc35a6069 Merge pull request #17253 from BerriAI/litellm_nova_embedding_support
Add nova embedding support
2025-11-28 21:16:06 +05:30
Sameer Kankute 6c326ceb87 Merge pull request #17142 from BerriAI/litellm_anthropic_update_new_feat
Update new anthropic feats as reviewed
2025-11-28 21:06:34 +05:30
Sameer Kankute 8700c5ced6 Add nova embedding support 2025-11-28 14:56:46 +05:30
YutaSaito 334d09b3b2 feat: add regex-based tool_name/tool_type matching for tool-permission (#17164)
* feat: add regex-based tool_name/tool_type matching for tool-permission

* docs: update tool permission quick start for UI workflow
2025-11-27 21:26:27 -08:00
Sameer Kankute 71f41352fa Merge pull request #17202 from BerriAI/litellm_azure_ai_anthropic_support
(Bug)Migrate Anthropic provider to azure ai
2025-11-28 08:14:14 +05:30
Ishaan Jaff edfc35ddac [Feature]: Add Provider publicai.co (#17230)
* init PublicAIChatConfig

* add publicai

* init public ai

* add publicai

* add publicai/swiss-ai models etc
2025-11-27 13:08:03 -08:00
Ishaan Jaff ffb75b04fd [Feat] UI - allow adding pass through guardrails through UI (#17226)
* add PassThroughGuardrailsConfig

* init JsonPathExtractor

* feat PassthroughGuardrailHandler

* feat pt guardrails

* pt guardrails

* add Pass-Through Endpoint Guardrail Translation

* add PassThroughEndpointHandler

* execute simple guardrail config and dict settings

* TestPassthroughGuardrailHandlerNormalizeConfig

* add passthrough_guardrails_config on litellm logging obj

* add LiteLLMLoggingObj to base trasaltino

* cleaner _get_guardrail_settings

* update guardrails settings

* docs pt guardrail

* docs Guardrails on Pass-Through Endpoints

* fix typing

* fix typing

* test_no_fields_set_sends_full_body

* fix typing

* init add pass through guardrails

* ui allow setting target fields on gd

* docs ui settings guardrails
2025-11-27 12:27:16 -08:00
Ishaan Jaff d612d71ef4 [Feat] Add guardrails for pass through endpoints (#17221)
* add PassThroughGuardrailsConfig

* init JsonPathExtractor

* feat PassthroughGuardrailHandler

* feat pt guardrails

* pt guardrails

* add Pass-Through Endpoint Guardrail Translation

* add PassThroughEndpointHandler

* execute simple guardrail config and dict settings

* TestPassthroughGuardrailHandlerNormalizeConfig

* add passthrough_guardrails_config on litellm logging obj

* add LiteLLMLoggingObj to base trasaltino

* cleaner _get_guardrail_settings

* update guardrails settings

* docs pt guardrail

* docs Guardrails on Pass-Through Endpoints

* fix typing

* fix typing

* test_no_fields_set_sends_full_body

* fix typing

* Potential fix for code scanning alert no. 3834: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-11-27 12:06:53 -08:00
Cesar Garcia 5197380dd2 docs: add OpenAI Agents SDK to projects (#17203)
* docs: add OpenAI Agents SDK to projects

Add documentation for OpenAI Agents SDK which has an official
LiteLLM extension enabling 100+ LLM providers in multi-agent workflows.

* Update project items in sidebars.js
2025-11-27 09:47:26 -08:00
Sameer Kankute 784c13ae84 Add docs for microsoft foundry 2025-11-27 22:26:38 +05:30
Ishaan Jaffer 3b330c3f0f docs config settings 2025-11-27 08:15:51 -08:00
Sameer Kankute 5fc950ec05 migrate anthropic provider to azure ai provider 2025-11-27 19:47:54 +05:30
Krish Dholakia 1cb5fcddba make generic api OSS + support multiple generic API's (#17152)
* feat(generic_api_callback.py): make generic api OSS + support multiple generic API's

Enables https://github.com/BerriAI/litellm/pull/17094#discussion_r2562832967

* feat(callback_utils.py): support custom generic api callbacks

* feat(generic_api_callback.py): support specifying which event types to run the generic api for

* fix(litellm_logging.py): log system prompt for anthropic messages

* feat(generic_api_callback.py): support generic api compatible api's - e.g. rubrik agent cloud

* docs(sidebars.js): document new OSS generic api

* docs(generic_api.md): document new OSS Generic API

* docs(custom_webhook_api.md): document custom webhook api integration tutorial

* docs(custom_webhook_api.md): cleanup

* docs(custom_webhook_api.md): document what get's logged to custom webhook api

* Refactor: Pass callback config to GenericAPILogger

Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>

* Fix: Handle empty messages list in logging payload

Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>

* Checkpoint before follow-up message

Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>

* feat: Cache GenericAPILogger instances to improve performance

Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-11-26 18:38:38 -08:00
Ishaan Jaffer b487e67dec sec fix 2025-11-26 18:23:18 -08:00
Ishaan Jaff 379655e16b [Feat] LiteLLM RAG API - Add support for Vertex RAG engine (#17117)
* add VertexAIVectorStoreOptions

* Revert "add VertexAIVectorStoreOptions"

This reverts commit b086adf10b122a72abc7071dce5a7061f456f575.

* add VertexAIVectorStoreOptions

* add get_rag_ingestion_class

* add VertexAIRAGTransformation

* test ingestion

* docs vertex ai rag engine
2025-11-26 15:49:04 -08:00
Ishaan Jaff d987593855 [Feat] Add audio transcriptions for WatsonX (#17160)
* feat: add IBMWatsonXAudioTranscriptionConfig

* init IBMWatsonXAudioTranscriptionConfig

* add inits

* test_watsonx_transcription_url_and_headers

* docs watsox audio

* docs watsonx audi

* docs
2025-11-26 14:15:49 -08:00
Ishaan Jaffer 42376330ba add DEFAULT_CHUNK_OVERLAP, DEFAULT_CHUNK_SIZE 2025-11-26 11:39:24 -08:00
Sameer Kankute d4e80c6132 Merge pull request #17124 from BerriAI/litellm_gemini_file_search
Add gemini file search support
2025-11-26 22:07:11 +05:30
Sameer Kankute c7ef668d78 Update documentation for azure 4 feats 2025-11-26 21:18:47 +05:30
Sameer Kankute 241ad27843 Add gemini file search support 2025-11-26 11:17:30 +05:30
ripepersimmon be970735de feat: Add gemini-3-pro-image-preview model support for imageSize parameters (#17019)
- Add model identifier to FLASH_IMAGE_PREVIEW_MODEL_IDENTIFIERS
- Add imageSize parameter support (1K, 2K, 4K) with GeminiImageSize type
- Add tests for imageSize parameter transformation
- Update documentation with new model
2025-11-25 19:38:29 -08:00
Igal Boxerman e6e1e8fca4 feat(pillar): add automatic LiteLLM context headers (#17076)
- Automatically pass LiteLLM virtual key context as X-LiteLLM-* headers
- Includes key_alias, user_id, team_id, org_id, and user_email
- No configuration required - always enabled for application/user tracking
- Excludes sensitive data (metadata, API tokens) for security
- Add comprehensive tests (30 tests, all passing)
- Update documentation with header details
2025-11-25 19:35:39 -08:00
Ishaan Jaff 5c192a23c3 [Feat] Add new RAG API on LiteLLM AI Gateway (#17109)
* init RAG api types

* add RAG endpoints

* init main.py for RAG ingest API

* init RecursiveCharacterTextSplitter

* add BaseRAGIngestion

* fix OpenAIRAGIngestion

* fix img handler

* init OpenAIRAGIngestion

* init BedrockRAGIngestion

* init BedrockRAGIngestion

* init rag tests

* init BedrockVectorStoreOptions

* implement BedrockRAGIngestion

* add BaseRAGAPI

* add endpoint for RAG ingest

* add ingest RAG endpoints

* add test doc

* add parse_rag_ingest_request

* update endpoints

* docs add docs for new RAG API

* fix qa check

* fix linting

* docs ficx

* docs

* add max depth checks

* docs anthropic
2025-11-25 17:54:29 -08:00
Krrish Dholakia 5cb5c2a7b7 docs: more doc cleanup 2025-11-25 16:04:27 -08:00
Krrish Dholakia 8ee6812edf docs: cleanup launch post 2025-11-25 15:58:51 -08:00
Krrish Dholakia 70a1325847 docs: more doc cleanup 2025-11-25 15:01:22 -08:00
Krrish Dholakia f3d5775920 fix: fix doc load issue 2025-11-25 14:40:26 -08:00
Ishaan Jaff be712908a3 [Feat] Add OpenAI compatible bedrock imported models. - qwen etc (#17097)
* test_bedrock_openai_imported_model

* AmazonBedrockOpenAIConfig

* add openai route for bedrock

* docs fix

* fix code qa check
2025-11-25 12:20:39 -08:00