tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-07-20 22:21:21 +00:00
Code Issues Packages Projects Releases Wiki Activity
26,119 Commits 2,072 Branches 1,425 Tags
ff4eda0ea67730da3363f2d151decb9ee3ce4e9c
Commit Graph
935 Commits
Author SHA1 Message Date
Ishaan JaffandGitHub cb385e080f Fix "azure_ai/grok-4-fast-reasoning" entry in "model_prices_and_context_window.json" (#15204) 
* fix: azure_ai/grok-4-fast-non-reasoning

* fix pricing
 2025-10-04 14:50:44 -07:00
Ishaan Jaffer 2dc11316f2 fix failing deepseek-ai/DeepSeek-V3.1 2025-10-04 11:50:30 -07:00
Krish DholakiaandGitHub c83a3ac9b4 Merge pull request #14939 from Toy-97/patch-1 
update: DeepInfra model data refresh [2025-09-26]
 2025-10-03 21:48:31 -07:00
YutaSaitoandGitHub 81a8766b84 feat: add JP Cross-Region Inference (#15188) 2025-10-03 21:20:04 -07:00
Ishaan JaffandGitHub 4415b195d1 Add "eu.anthropic.claude-sonnet-4-5-20250929-v1:0" in "model_prices_and_context_window.json" (#15181) 
* feat: add eu.anthropic.claude-sonnet-4-5-20250929-v1:0

* fix: nvidia_nim_models
 2025-10-03 15:52:18 -07:00
Ishaan JaffandGitHub efa782d6d2 [Feat] Add Nvidia NIM Rerank Support (#15152) 
* feat: add NvidiaNimRerankConfig

* fix: NvidiaNimRerankConfig

* fix: NvidiaNimRerankConfig

* fix routing to nvidia nim

* docs nvidia nim rerank

* TestNvidiaNim

* nvidia nim rerank fixes

* fix rerank

* transform_rerank_response

* Usage with LiteLLM Proxy

* fixes linting

* NvidiaNimRerankConfig.DEFAULT_NIM_RERANK_API_BASE

* fix Custom API Base URL

* fix rerank base

* fix main.py

* fix transform

* fix linting

* map_cohere_rerank_params

* ruff fix

* linting fixes

* ruff fix
 2025-10-02 18:58:52 -07:00
Krish DholakiaandGitHub 6b3db2fd49 Merge pull request #15140 from niharm/fix-sonnet-4-5-200k-pricing 
Price Fix: Add 200K prices for Sonnet 4.5
 2025-10-02 17:28:47 -07:00
Mubashir OsmaniandGitHub f2107a189d add azure_ai grok-4 model family (#15137) 
* added oauth mcp to docs

* added azure ai/grok-4 model family

* Revert "added oauth mcp to docs"

This reverts commit 950b7cef44f14b2db1429f6fbd32548a7c95d325.
 2025-10-02 14:21:12 -07:00
nihar cd7acd2eb2 Add 200K prices for Sonnet 4.5 2025-10-02 12:39:44 -07:00
Ishaan JaffandGitHub d9664a3ee4 fix gpt-5-chat-latest on model cost map (#15116) 2025-10-01 14:35:57 -07:00
Uzair AliandGitHub fcfe856e10 Add support for GPT 5 codex models (#14841) 
* Add support for GPT 5 codex models

* lint

* fixes
 2025-09-30 18:44:35 -07:00
Ishaan JaffandGitHub 862736e74b feat: add groq/moonshotai/kimi-k2-instruct-0905 (#15079) 2025-09-30 12:51:21 -07:00
Eddie Richter f9e98f75a6 Adding max_input_tokens and max_output_tokens 2025-09-30 12:12:24 -06:00
Eddie Richter ae92404d05 Initial addition of Lemonade provider. 2025-09-30 12:12:24 -06:00
Krish DholakiaandGitHub 551a171243 Merge pull request #15043 from cedarm/fix-remove-vertex-latest 
fix: remove invalid vertex -latest models
 2025-09-29 21:54:53 -07:00
Ishaan Jaffer 3e474b9e81 fix claude-sonnet-4-5 model cost map 2025-09-29 18:26:29 -07:00
Ishaan JaffGitHubCursor Agentishaan
 		05955042d5 Add model pricing and context window for claude-sonnet-4-5 (#15049) 
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: ishaan <ishaan@berri.ai>
 2025-09-29 15:09:40 -07:00
Ishaan JaffandGitHub 038863a1fe [Feat] Add new claude-sonnet-4-5 model family (#15041) 
* add new claude-sonnet-4-5

* docs fix

* fix tool_use_system_prompt_tokens

* add anthropic.claude-sonnet-4-5-20250929 to bedrock converse models
 2025-09-29 13:09:00 -07:00
Cedar Myers 5357b1e102 fix: remove invalid vertex -latest models 2025-09-29 16:04:33 -04:00
Wenxi Onyx ad29d2f944 add models served from ollama cloud 2025-09-28 14:06:08 -07:00
Wenxi Onyx 4adb854e51 add models served from ollama cloud 2025-09-28 13:20:35 -07:00
Krish DholakiaandGitHub f8a64345b7 Merge pull request #14942 from danielmklein/main 
Fix inconsistent token configs for gpt-5 models
 2025-09-27 07:49:19 -07:00
Nicolas HermentandGitHub 50f625433d Revert incorrect changes to sonnet-4 max output tokens (#14933) 2025-09-26 11:19:52 -07:00
Ishaan JaffandGitHub 360befa216 [Feat] Add support for Gemini 2.5 Flash and Flash-lite preview models (09-2025 release) (#14948) 
* add gemini-2.5-flash-preview-09-2025

* docs add gemini-2.5-flash-preview-09-2025 model family
 2025-09-26 09:51:38 -07:00
Daniel Klein de795a4531 Fix inconsistent token configs for gpt-5 models 2025-09-26 09:43:26 -04:00
Toy-97andGitHub 6c95bd926f update: DeepInfra model data refresh [2025-09-26] 
Added models:
deepinfra/deepseek-ai/DeepSeek-V3.1-Terminus

Removed models:
deepinfra/zai-org/GLM-4.5-Air

Modified models:
deepinfra/NousResearch/Hermes-3-Llama-3.1-70B:
   - input_cost_per_token: 1.2e-07 → 3e-07

deepinfra/Qwen/Qwen3-32B:
   - output_cost_per_token: 3e-07 → 2.8e-07

deepinfra/Qwen/Qwen3-Next-80B-A3B-Instruct:
   - max_tokens: 4096 → 262144
   - max_output_tokens: 4096 → 262144
   - max_input_tokens: 4096 → 262144

deepinfra/Qwen/Qwen3-Next-80B-A3B-Thinking:
   - max_tokens: 4096 → 262144
   - max_output_tokens: 4096 → 262144
   - max_input_tokens: 4096 → 262144

deepinfra/Qwen/Qwen3-235B-A22B-Instruct-2507:
   - input_cost_per_token: 1.3e-07 → 9e-08

deepinfra/meta-llama/Meta-Llama-3.1-70B-Instruct:
   - input_cost_per_token: 2.3e-07 → 4e-07

deepinfra/google/gemini-2.5-flash:
   - output_cost_per_token: 1.75e-06 → 2.5e-06
   - input_cost_per_token: 2.1e-07 → 3e-07

deepinfra/meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo:
   - output_cost_per_token: 2e-08 → 3e-08
   - input_cost_per_token: 1.5e-08 → 2e-08

deepinfra/meta-llama/Llama-3.2-3B-Instruct:
   - output_cost_per_token: 2.4e-08 → 2e-08
   - input_cost_per_token: 1.2e-08 → 2e-08

deepinfra/Sao10K/L3-8B-Lunaris-v1-Turbo:
   - input_cost_per_token: 2e-08 → 4e-08

deepinfra/openai/gpt-oss-120b:
   - input_cost_per_token: 9e-08 → 5e-08

deepinfra/google/gemini-2.5-pro:
   - output_cost_per_token: 7e-06 → 1e-05
   - input_cost_per_token: 8.75e-07 → 1.25e-06

deepinfra/NousResearch/Hermes-3-Llama-3.1-405B:
   - output_cost_per_token: 8e-07 → 1e-06
   - input_cost_per_token: 7e-07 → 1e-06

deepinfra/Qwen/Qwen3-235B-A22B:
   - output_cost_per_token: 6e-07 → 5.4e-07
   - input_cost_per_token: 1.3e-07 → 1.8e-07

deepinfra/nvidia/Llama-3.1-Nemotron-70B-Instruct:
   - output_cost_per_token: 3e-07 → 6e-07
   - input_cost_per_token: 1.2e-07 → 6e-07

deepinfra/meta-llama/Llama-3.3-70B-Instruct-Turbo:
   - output_cost_per_token: 1.2e-07 → 3.9e-07
   - input_cost_per_token: 3.8e-08 → 1.3e-07

deepinfra/deepseek-ai/DeepSeek-V3-0324:
   - input_cost_per_token: 2.8e-07 → 2.5e-07
   - cache_read_input_token_cost: 2.24e-07 → None

deepinfra/mistralai/Mistral-Small-3.2-24B-Instruct-2506:
   - output_cost_per_token: 1e-07 → 2e-07
   - input_cost_per_token: 5e-08 → 7.5e-08

deepinfra/Qwen/Qwen3-235B-A22B-Thinking-2507:
   - output_cost_per_token: 6e-07 → 2.9e-06
   - input_cost_per_token: 1.3e-07 → 3e-07

deepinfra/zai-org/GLM-4.5:
   - output_cost_per_token: 2e-06 → 1.6e-06
   - input_cost_per_token: 5.5e-07 → 4e-07

deepinfra/mistralai/Mixtral-8x7B-Instruct-v0.1:
   - output_cost_per_token: 2.4e-07 → 4e-07
   - input_cost_per_token: 8e-08 → 4e-07

deepinfra/openai/gpt-oss-20b:
   - output_cost_per_token: 1.6e-07 → 1.5e-07

deepinfra/google/gemma-3-27b-it:
   - output_cost_per_token: 1.7e-07 → 1.6e-07
 2025-09-26 20:11:26 +08:00
Krish DholakiaandGitHub 2f3155c2ee Merge pull request #14858 from oytunkutrup1/litellm_fix_gpt3.5_price_fix 
GPT-3.5-Turbo price updated.
 2025-09-25 23:47:43 -07:00
Krish DholakiaandGitHub 6a1be4722e Merge pull request #14879 from huangyafei/update_price 
Add gpt-5 and gpt-5-codex to OpenRouter cost map
 2025-09-25 23:46:35 -07:00
Ishaan Jaffer 1585750c1b test fix 2025-09-24 21:45:42 -07:00
huangyafei 107c45da9d Add gpt-5-codex to OpenRouter cost map 2025-09-25 11:05:56 +08:00
huangyafei 1b95940d18 Add gpt-5 to OpenRouter cost map 2025-09-25 11:02:54 +08:00
Luis Felipe Salazar UcrosandGitHub c6cb36186c Add sambanova deepseek v3.1 and gpt-oss-120b (#14866) 
* add sambanova deepseek v3.1 and gpt-oss-120b

* add sambanova deepseek v3.1 and gpt-oss-120b
 2025-09-24 10:54:22 -07:00
Mubashir OsmaniandGitHub 0cd91a82d2 Added vertex_ai/qwen models and azure/gpt-5-codex (#14844) 
* added qwen models and gpt-5-codex

* fix flaky test

* fix failing test
 2025-09-24 10:40:00 -07:00
onlylonlyandGitHub 22eef373eb feat: New model - Add support for Qwen models family & Deepseek 3.1 to Amazon Bedrock (#14845) 
* New model - Add Bedrock deepseek v3.1 model - "deepseek.v3-v1:0"

* New model - Add Bedrock Qwen models - "qwen.qwen3-coder-480b-a35b-v1:0", "qwen.qwen3-235b-a22b-2507-v1:0", "qwen.qwen3-coder-30b-a3b-v1:0", "qwen.qwen3-32b-v1:0"

* fix: add "deepseek.v3-v1:0" in  litellm/model_prices_and_context_window_backup.json
 2025-09-24 09:58:23 -07:00
oytun.kutrup 410e4956c4 GPT-3.5-Turbo price updated. 2025-09-24 15:32:23 +03:00
Krish DholakiaandGitHub d5ced7e068 Merge pull request #14639 from danielmklein/main 
fix: update sonnet 4 configs to reflect million-token context window pricing
 2025-09-23 18:06:15 -07:00
Ishaan JaffGitHubCursor Agentishaan
 		fff1d1a9f9 Add Vertex AI Qwen3 models to pricing and context window (#14828) 
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: ishaan <ishaan@berri.ai>
 2025-09-23 16:16:57 -07:00
Ishaan JaffandGitHub 8443eff4a2 feat: add xai/grok-4-fast models (#14833) 2025-09-23 16:01:44 -07:00
Krish DholakiaandGitHub 88f9cad886 Merge pull request #14796 from BerriAI/litellm_anthopic_token_count_issue 
Add service_tier based pricing support for openai[ BOTH Service & Priority Support]
 2025-09-22 22:38:59 -07:00
Sameerlite d2208023d2 Add service_tier based pricing support for openai 2025-09-23 10:48:27 +05:30
Gagan Raj ChinkaandGitHub a5ed3446cd opnerouter/x-ai/grok-4-fast:free" to model_prices_and_context_window.json (#14779) 
* Update model_prices_and_context_window_backup.json

* Update model_prices_and_context_window.json
 2025-09-22 21:29:15 -07:00
Anubhav SinghandGitHub fde4dcb0b8 Merge branch 'main' into wandb-inference 2025-09-23 00:14:11 +05:30
Tim Elfrink 282617f2bc Fix gemini-2.5-flash-image-preview model routing 
- Update mode from 'chat' to 'image_generation' for both model variants
- Ensures correct routing to image generation endpoints
- Resolves 400 'request not supported' error for image generation
 2025-09-19 07:58:58 +02:00
Ishaan Jaffer 114d077cc9 fix: model cost map check 2025-09-18 17:52:56 -07:00
Ishaan Jaffer c1a967992f fix: model cost map check 2025-09-18 17:37:09 -07:00
Ishaan Jaffer 0626affa11 fix: model cost map check 2025-09-18 17:25:28 -07:00
Ishaan JaffandGitHub 4c983f985a [Feat] Add Bedrock Twelve Labs embedding provider support (#14697) 
* fix: add 12 labs to bedrock embedding

* fix: get_bedrock_embedding_provider

* test: test_text_embedding

* fix: 12 labs embedding transform

* fix: refactor 12 labs transform logic

* fix: test_e2e_bedrock_embedding

* fix: test_e2e_bedrock_embedding

* feat: add bedrock twelvelabs pricing

* DOCS: docs bedrock embedding

* DOCS: 12 labs bedrock overview

* fix: bedrock embeddings 12 labs
 2025-09-18 17:16:45 -07:00
Sameer KankuteandGitHub 36bedc69ff Add TwelveLabs marengo model (#14674) 2025-09-18 11:21:35 -07:00
Daniel Klein b8b30775a4 fix: update sonnet 4 configs to reflect million-context-window pricing 2025-09-17 11:33:23 -04:00
Krrish Dholakia 97cc5f55d6 build(model_prices_and_context_window.json): add claude-3-5 haiku, claude-3-5 sonnet, claude-opus 3, claude-haiku 3 "cache_creation_input_token_cost_above_1hr" pricing 2025-09-16 18:58:49 -07:00