tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-07-11 17:05:43 +00:00
Code Issues Packages Projects Releases Wiki Activity
30,935 Commits 2,072 Branches 1,425 Tags
5b341ee842e2e2b2d074060985dab55ea3a009e4
Commit Graph

739 Commits

Author SHA1 Message Date
milan-berri 37b7dff194 add spend-queue-troubleshooting docs (#19659) 
* add spend-queue-troubleshooting docs

* adjust spend-queue-troubleshooting docs
 2026-01-23 13:44:41 -08:00
Ishaan Jaff c23e4b87dc [Feat] New LiteLLM Policy engine - create policies to manage guardrails, conditions - permissions per Key, Team (#19612) 
* init PolicyMatcher

* TestPolicyMatcherGetMatchingPolicies

* TestPolicyMatcherGetMatchingPolicies

* feat: init PolicyResolver

* init resolver types

* init policy from config

* inint PolicyValidator

* validate policy

* init Architecture Diagram

* test_add_guardrails_from_policy_engine

* init _init_policy_engine

* test updates

* test fixws

* new attachment config

* simplify types

* TestPolicyResolverInheritance

* fix policy resolver

* fix policies

* fix applied policy

* docs fix

* docs fix

* fix linting + QA checks

* fix linting + QA fixes

* test fixes
 2026-01-22 19:49:53 -08:00
milan-berri c9516d68d4 add opencode tutorial (#19602) 2026-01-22 15:29:19 -08:00
Sameer Kankute ad1edd38d5 Merge branch 'main' into litellm_staging_01_21_2026 2026-01-22 17:56:40 +05:30
Ishaan Jaff ab606c9a73 [Feat] Add Structured output for /v1/messages with Anthropic API, Azure Anthropic API, Bedrock Converse (#19545) 
* fix: add AnthropicMessagesRequestOptionalParams

* add _update_headers_with_anthropic_beta

* fix output format tests

* test_structured_output_e2e

* TestAnthropicAPIStructuredOutput

* test_structured_output_e2e

* fix BASE

* TestAzureAnthropicStructuredOutput

* fix: Bedrock Converse

* add nthropic Messages Pass-Through Architecture

* fix: bedrock invoke output_format

* fix: transform_anthropic_messages_request for vertex anthropic

* TestBedrockInvokeStructuredOutput

* docs anthropic vertex

* docs fix

* docs fix
 2026-01-21 20:09:18 -08:00
Cesar Garcia 4106d24215 feat: add GMI Cloud provider support (#19376) 
* feat: add GMI Cloud provider support

Add GMI Cloud as an OpenAI-compatible provider with:
- Provider configuration in providers.json
- Documentation page with usage examples
- Model pricing for 16 models (Claude, GPT, DeepSeek, Gemini, etc.)
- Sidebar entry for docs navigation

* Add gmi_cloud to provider_endpoints_support.json

Add provider entry to pass CI validation check that ensures all
providers in openai_like/providers.json are documented.

* Fix provider key: gmi_cloud -> gmi

Match the provider key with providers.json

---------

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
 2026-01-21 15:48:15 -08:00
Ishaan Jaff a02c43d300 Litellm cc docs max (#19466) 
* docs claude code max

* docs fix

* docs

* docs fix

* docs fix
 2026-01-20 20:08:36 -08:00
Sampson 09941dd1d1 add search provider for brave search api (#19433) 
* add search provider for brave search api

Introduces a minimal implementation of the Brave Search API as a search provider. Additionally, this PR introduces a test file to ensure the provider works properly, and numerous other smaller changes (e.g., changes to docs to mention the new option).

* Update transformation.py
 2026-01-20 19:23:29 -08:00
Sameer Kankute deb9142117 Merge pull request #19400 from BerriAI/main 
merge main iin 19/1 staging
 2026-01-20 16:45:01 +05:30
Ishaan Jaffer f865f92bec docs plugin marketplaces 2026-01-19 19:42:15 -08:00
Manuel Schweigert 29adf34313 Add ChatGPT subscription support and responses bridge (#19030) 
* Add ChatGPT subscription support and responses bridge

* Fix typing import for responses bridge

* Guard device code timestamp parsing

* add /v1/messages endpoint to chatgpt model
 2026-01-19 05:37:45 -08:00
Ishaan Jaff 1417b002a3 [Feat] Claude Code x LiteLLM WebSearch - QA Fixes to work with Claude Code (#19294) 
* fix websearch_interception_converted_stream

* test_websearch_interception_no_tool_call_streaming

* FakeAnthropicMessagesStreamIterator

* LITELLM_WEB_SEARCH_TOOL_NAME

* fixes tools def for litellm web search

* fixes FakeAnthropicMessagesStreamIterator

* test_litellm_standard_websearch_tool

* use new hook for modfying before any transfroms from litellm

* init WebSearchInterceptionLogger + ARCHITECTURE

* fix config.yaml

* init doc for claude code web search

* docs fix

* doc fix

* fix mypy linting
 2026-01-17 16:30:31 -08:00
yuneng-jiang 19a69a89f0 deleted keys and teams docs 2026-01-17 13:19:48 -08:00
Ishaan Jaffer eb26ebc926 docs ui usage 2026-01-17 12:15:37 -08:00
YutaSaito 66d67ae356 Revert "Add sanititzation for anthropic messages" 2026-01-17 06:01:12 +09:00
Sameer Kankute fb3b4e6b33 Merge pull request #19196 from BerriAI/litellm_sanitise_anthropic_mesages 
Add sanititzation for anthropic messages
 2026-01-16 17:48:55 +05:30
Sameer Kankute 8be3712e82 Add docs for message sanitisation 2026-01-16 12:52:13 +05:30
Sameer Kankute d585b760c9 Add fallback endpoints support 2026-01-16 10:51:33 +05:30
Ishaan Jaff b5b9c39beb [Docs Guide] Litellm claude code end user tracking (#19176) 
* add to sidebar

* v1 guide

* guide claude granular cost tracking

* docs fix
 2026-01-15 18:32:58 -08:00
Krish Dholakia 664ee27ef5 Litellm dev 01 15 2026 p1 (#19153) 
* fix: safely handle unmapped call type

* docs: cleanup links for ai coding tools

* docs(claude_non_anthropic_models.md): add tutorial showing non anthropic model connection to claude code

* docs: link to non-anthropic model tutorial for claude code
 2026-01-16 00:48:41 +05:30
Krrish Dholakia c7ca2dd4d8 docs(claude_mcp.md): separate claude mcp tutorial into a separate doc 
easier to surface
 2026-01-15 20:52:31 +05:30
YutaSaito e3e6fc2806 Merge pull request #19122 from BerriAI/litellm_docs_mcp_troubleshooting 
[doc] add MCP troubleshooting guide
 2026-01-15 09:34:35 +09:00
Alexsander Hamir f442b57848 docs: Add structured issue reporting guides for CPU and memory issues (#19117) 2026-01-14 15:28:20 -08:00
Yuta Saito 0b04145c11 doc: add MCP troubleshooting guide 2026-01-15 08:01:59 +09:00
Ishaan Jaff 1b00576711 [Feat] New Model - Azure Model Router on LiteLLM AI Gateway (#19054) 
* fix - azure model router integration

* fix:_check_provider_match

* fix:_get_response_model

* tests azure model router

* test_azure_ai_model_router_streaming_model_in_chunk

* fix LlmProviders.AZURE.value

* test_azure_ai_model_router_streaming_cost_with_stream_options

* def test_get_model_from_chunks_azure_model_router():

* _get_model_from_chunks

* docs azure model router

* azure model router
 2026-01-13 18:31:43 -08:00
Sameer Kankute 844c766c65 Merge pull request #18763 from BerriAI/litellm_staging_01_07_2026 
Staging - 01/07/2026
 2026-01-09 17:01:58 +05:30
Ishaan Jaff cbac70a4ec MANUS docs (#18817) 2026-01-08 18:58:10 +05:30
Abliteration AI dc4ce7c5a2 feat: Add abliteration.ai provider (#18678) 
* feat: Add abliteration.ai provider

* adding signoz integration to observability docs

* Fixing build

* Adding timeout for flaky test

* Fixing e2e

* add team member budget duration in team/update

* Reusable Duration Select and update team member budget UI

---------

Co-authored-by: Goutham Karthi <goutham@signoz.io>
Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com>
Co-authored-by: YutaSaito <36355491+uc4w6c@users.noreply.github.com>
 2026-01-07 21:46:54 +05:30
Krish Dholakia 80ead21c3a Litellm improve endpoint discovery (#18762) 
* docs: document all endpoints in .json and add consistency checks against docs + providers.json

* docs: add more tests + improve coverage
 2026-01-07 17:35:01 +05:30
drorIvry 000913fa12 Hotfix - docs qualifire (#18724) 
* Hotfix - docs qualifire

* Hotfix - docs qualifire

* Hotfix - docs qualifire

* Hotfix - docs qualifire

* Hotfix - docs qualifire

* Hotfix - docs qualifire

* Hotfix - docs qualifire
 2026-01-07 17:23:12 +05:30
Lundin Matthews 762345172c Add LlamaGate as a new provider (#18673) 
Adds LlamaGate (https://llamagate.dev) as an OpenAI-compatible provider with:
- Provider configuration in providers.json
- Documentation page with usage examples
- Model pricing for 17 models across categories:
  - General purpose (Llama 3.1/3.2, Mistral, Qwen, Dolphin)
  - Reasoning (DeepSeek R1, OpenThinker)
  - Code (Qwen Coder, DeepSeek Coder, CodeLlama)
  - Vision (Qwen VL, LLaVA, Gemma 3)
  - Embeddings (Nomic, Qwen3 Embedding)

Provider details:
- Base URL: https://api.llamagate.dev/v1
- Auth: Bearer token via LLAMAGATE_API_KEY
- Pricing: $0.02-$0.55 per 1M tokens
- All models are open-weights
 2026-01-07 00:00:30 +05:30
Ishaan Jaff 76eda472be [Feat] New API Endpoint - Responses API (v1/responses/compact) (#18697) 
* init transform_compact_response_api_request

* init acompact_responses

* init async_compact_response_api_handler in llm http handler

* init transform_compact_response_api_request for openai

* init acompact_responses

* fix acompact_responses

* add OAI Compact API

* docs responses API Compact

* code qa checks

* test_openai_compact_responses_api

* fix mypy linting
 2026-01-06 16:24:04 +05:30
Ishaan Jaff bf1c5bef59 [UI] Add view for estimating costs across requests (#18645) 
* add estimate_cost endpoint

* TestCostEstimateEndpoint

* fix estimate_cost

* add /cost/estimate to spend tracking routes

* fix code QA checks

* fixes endpoint

* v0 cost estimator

* v0 cost estimator

* formatNumberWithCommas

* fix admin view

* docs

* docs fix + export PDF/CSV

* fixes for export
 2026-01-05 19:19:58 +05:30
Matt Cowger 9f4a4fd271 Add docs as follow on to #18362 (#18388) 2025-12-25 11:08:47 +05:30
Ishaan Jaff 1123cfa928 [Feat] AI Gateway - Add support for Platform Fee / Margins (#18427) 
* init cost_margin_config

* feat: add cost margin

* init types

* LITELLM_SETTINGS_SAFE_DB_OVERRIDES

* feat _apply_cost_margin

* ui endpoint

* ui provider margins

* add margin

* refactored ui

* test cost margins

* refactored ui

* provider discounts

* add cost_breakdown to spendLogs

* add CostBreakdownViewer

* fix cost breakdown

* docs fix

* doc margins

* docs margins
 2025-12-25 11:07:27 +05:30
Sameer Kankute 42121ad13b Update minimax docs to be in proper format 2025-12-24 09:25:36 +05:30
Ishaan Jaff 4f7682f670 [Feat] RAG query endpoint - Add RAG Search / Query endpoint (#18376) 
* add rag query to llm api endpoints

* add rag query as a new endpoint

* docs - new endpoint

* ingest+query

* docs add rag query
 2025-12-23 22:30:37 +05:30
Sameer Kankute c7e0366925 Add support for minmax tts 2025-12-23 21:53:53 +05:30
Sameer Kankute 403875256c Add minmax documentation 2025-12-23 16:11:26 +05:30
Ishaan Jaff 2677d9d30d [Feat] New provider TTS - Add AWS polly API for TTS (#18326) 
* add aws_polly as new provider

* init AWSPollyTextToSpeechConfig

* test_aws_polly_tts_with_native_voice

* init aws_polly + AWS polly dispatch

* init AWSPollyTextToSpeechConfig

* fix transform

* add aws_polly as a new provider for TTS API

* add to sidebar

* docs aws polly

* code qa fix

* add AWS Polly Text-to-Speech

* add cost tracking for AWS polly

* docs fix
 2025-12-22 18:19:34 +05:30
Farhan Aulianda 63c3a6e228 fix: enable JSON-only provider routing + add xiaomi_mimo provider (#18291) 
* fix: enable JSON-only provider routing + add xiaomi_mimo provider

* docs: add xiaomi_mimo provider documentation
 2025-12-22 12:05:22 +05:30
Anil Kodali afba676b2e Add Amazon Nova to sidebar and under supported models in README (#18220) 2025-12-19 19:07:34 +05:30
Ishaan Jaff 5ea0854eda [Feat] Guardrails Load Balancing - Allow Platform admins to load balance between guardrails (#18181) 
* add _aguardrail_helper for LB

* add _aguardrail_helper on router.py

* test_proxy_logging_pre_call_hook_load_balancing

* add _execute_guardrail_with_load_balancing

* add LB TEsting

* docs guard lb

* fix linting

* fix lint
 2025-12-19 00:08:03 +05:30
Krish Dholakia 26cd2c4473 Guardrails - add built in guardrails for harmful content, bias, etc. (#18029) 
* feat(litellm_content_filter.py): add support for content filtering categories

make it easy for proxy admin to prevent messages about violence, self harm or illegal weapons going through litellm

* feat: initial commit adding bias detection

allows admin to block inappropriate content about sexual orientation, etc.

* refactor: simplify content_filter.py

use a more exhaustive set of keywords, instead of guessing at potential phrases user can use

* feat(content_filter.py): add new denied topics for in-built content filter guardrails

allow user to automatically block content relating to certain categories from being sent to the LLML

* refactor(content-filter): document new params to litellm content filter

* feat(ui/): litellm content filter - select content categories on ui

* docs: update documentation

* docs(litellm_content_filter.md): document new content filters
 2025-12-18 16:28:58 +05:30
Ishaan Jaff 274d996a87 [Feat] New Search API Provider - LinkUp Search (#18174) 
* add linkup search provider

* add Linkup Search docs

* add get_provider_search_config

* get_provider_search_config

* add linkup/search provider

* fix mypy linting
 2025-12-18 14:27:36 +05:30
Ishaan Jaffer f80628502d init docs 2025-12-17 02:27:54 +04:00
Ishaan Jaff 244d83ff47 [Docs] Litellm add docs vertex ai engine (#18027) 
* new provider doc

* add to sidebar

* stash docs

* docs fix

* docs vertex agent engine
 2025-12-15 20:11:40 -08:00
Ishaan Jaff ce113f4e4b [Docs] Add docs on using pydantic ai agents with LiteLLM A2a gateway (#18026) 
* init A2AProviderConfigManager

* move file

* move file

* add pydnatic ai folder

* init providers

* test_pydantic_ai_non_streaming

* fix import

* INIT pydantic

* use_a2a_form_fields

* test_vertex_agent_engine_streaming

* add agent_engine

* init transform for agent engine

* init agent engine

* VertexAgentEngineSSEStreamIterator

* sample

* ui add new fields

* fix vertex_credentials

* working SSE iterator

* TestVertexAgentEngineTransformRequest

* fix code QA check

* stash docs

* docs fix

* fix logo

* docs fix

* doc pydantic ai

* docs pydantic ai

* new provider

* docs fix
 2025-12-15 19:43:12 -08:00
Ishaan Jaffer 87ba4fabd7 docs fix 2025-12-14 14:20:47 -08:00
Ishaan Jaffer 3cac6b0a40 vllm batch 2025-12-14 12:36:07 -08:00