tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-08-13 22:23:04 +00:00
Code Issues Packages Projects Releases Wiki Activity
25,256 Commits 2,072 Branches 1,425 Tags
dfdba527e2ed3fc51dd236cf0b5ea7c8a6528d1d
Commit Graph
888 Commits
Author SHA1 Message Date
Tim Elfrink 282617f2bc Fix gemini-2.5-flash-image-preview model routing 
- Update mode from 'chat' to 'image_generation' for both model variants
- Ensures correct routing to image generation endpoints
- Resolves 400 'request not supported' error for image generation
 2025-09-19 07:58:58 +02:00
Ishaan Jaffer 114d077cc9 fix: model cost map check 2025-09-18 17:52:56 -07:00
Ishaan Jaffer c1a967992f fix: model cost map check 2025-09-18 17:37:09 -07:00
Ishaan Jaffer 0626affa11 fix: model cost map check 2025-09-18 17:25:28 -07:00
Ishaan JaffandGitHub 4c983f985a [Feat] Add Bedrock Twelve Labs embedding provider support (#14697) 
* fix: add 12 labs to bedrock embedding

* fix: get_bedrock_embedding_provider

* test: test_text_embedding

* fix: 12 labs embedding transform

* fix: refactor 12 labs transform logic

* fix: test_e2e_bedrock_embedding

* fix: test_e2e_bedrock_embedding

* feat: add bedrock twelvelabs pricing

* DOCS: docs bedrock embedding

* DOCS: 12 labs bedrock overview

* fix: bedrock embeddings 12 labs
 2025-09-18 17:16:45 -07:00
Sameer KankuteandGitHub 36bedc69ff Add TwelveLabs marengo model (#14674) 2025-09-18 11:21:35 -07:00
Krrish Dholakia 97cc5f55d6 build(model_prices_and_context_window.json): add claude-3-5 haiku, claude-3-5 sonnet, claude-opus 3, claude-haiku 3 "cache_creation_input_token_cost_above_1hr" pricing 2025-09-16 18:58:49 -07:00
Krrish Dholakia 3188ae9281 build(model_prices_and_context_window.json): add claude-opus-4, claude-3-7-sonnet, claude-sonnet-4 "cache_creation_input_token_cost_above_1hr" pricing 2025-09-16 18:56:24 -07:00
Krrish Dholakia a2bfd3e476 build(model_prices_and_context_window.json): add claude-opus-4-1 "cache_creation_input_token_cost_above_1hr" pricing 2025-09-16 18:53:56 -07:00
Krrish Dholakia 0fa11c3c25 build(model_prices_and_context_window.json): add "cache_creation_input_token_cost_above_1hr" to all claude-3-5-sonnet models 2025-09-16 18:52:37 -07:00
Krrish Dholakia 1c855385c9 build(model_cost): add cache_creation_input_token_cost_above_1hr pricing 2025-09-16 18:43:57 -07:00
Tim ElfrinkandGitHub 30c3e7b3d3 Fix: Bedrock cross-region inference profile cost calculation (#14566) 
* Add tests for Bedrock cross-region inference profile mapping

- Test model mapping lookup works correctly
- Test proxy cost calculation scenario reproduces original issue
- Verify cost calculation returns expected values
- Ensure compatibility with existing test patterns

* Fix Bedrock cross-region inference profile cost calculation

- Add mapping for bedrock/us.anthropic.claude-3-5-haiku-20241022-v1:0
- Sync backup file for local testing consistency
- Resolve proxy spend tracking failures for cross-region profiles
- Maintain identical configuration with standalone profile

Fixes #14458
 2025-09-15 07:10:20 -07:00
Elias TOURNEUX ef9d1ddc40 feat: Add OVHCloud AI Endpoints as a provider 2025-09-12 13:37:03 +02:00
Ishaan Jaff 69ef062f55 fix tiered_pricing test 2025-09-11 19:56:44 -07:00
Ishaan JaffandGitHub dda115cc6d [Feat] Cost Tracking - Add support for Tiered Cost Tracking for Qwen API (Dashscope) (#14471) 
* add dashscope logo

* docs fix

* docs fix

* fix supports_batch_calling

* fix naming

* fix input_cost_per_audio_token

* use output_cost_per_reasoning_token

* add tiered_pricing in get_model_info

* test fixes

* fix cost calc

* ruff fix
 2025-09-11 18:14:39 -07:00
Ishaan Jaff 258b674dbb fix deepinfra test 2025-09-10 19:39:23 -07:00
Krish DholakiaandGitHub e57a05b2dc Merge pull request #14324 from Toy-97/patch-1 
update: DeepInfra model data refresh [2025-09-08]
 2025-09-09 22:31:44 -07:00
Krrish Dholakia 076b46c806 build: remove end of life bedrock model 2025-09-09 19:49:22 -07:00
Pedro AzevedoandGitHub 44dc6d3aef Fix: Add supports_function_calling for GPT OSS in Bedrock provider (#14375) 
* Refactor JSON formatting and remove unnecessary whitespace in model prices and context window

* Fix formatting inconsistencies and remove unnecessary whitespace in model prices JSON
 2025-09-09 13:49:32 -07:00
Krrish Dholakia 3dba103e2c fix: fix typo 2025-09-08 18:13:34 -07:00
Toy-97andGitHub 890ee1abfa update: DeepInfra model data refresh [2025-09-08] 
Removed models:
deepinfra/Qwen/Qwen2.5-Coder-32B-Instruct
deepinfra/deepseek-ai/DeepSeek-V3-0324-Turbo
deepinfra/meta-llama/Llama-3.2-90B-Vision-Instruct
deepinfra/meta-llama/Llama-4-Maverick-17B-128E-Instruct-Turbo
deepinfra/meta-llama/Meta-Llama-3-70B-Instruct
deepinfra/mistralai/Devstral-Small-2507
deepinfra/mistralai/Mistral-7B-Instruct-v0.3
deepinfra/mistralai/Mistral-Small-3.1-24B-Instruct-2503

Modified models:
deepinfra/deepseek-ai/DeepSeek-R1-0528:
   - cache_read_input_token_cost: None → 4e-07

deepinfra/google/gemma-3-4b-it:
   - input_cost_per_token: 2e-08 → 4e-08
   - output_cost_per_token: 4e-08 → 8e-08

deepinfra/Qwen/QwQ-32B:
   - input_cost_per_token: 7.5e-08 → 1.5e-07
   - output_cost_per_token: 1.5e-07 → 4e-07

deepinfra/deepseek-ai/DeepSeek-V3-0324:
   - cache_read_input_token_cost: None → 2.24e-07

deepinfra/deepseek-ai/DeepSeek-V3.1:
   - input_cost_per_token: 3e-07 → 2.7e-07
   - cache_read_input_token_cost: None → 2.16e-07

deepinfra/Qwen/Qwen3-Coder-480B-A35B-Instruct-Turbo:
   - cache_read_input_token_cost: 1.5e-07 → 2.4e-07

deepinfra/NousResearch/Hermes-3-Llama-3.1-70B:
   - supports_tool_choice: True → False

deepinfra/deepseek-ai/DeepSeek-R1-Turbo:
   - max_output_tokens: 163840 → 40960
   - max_input_tokens: 163840 → 40960
   - max_tokens: 163840 → 40960
 2025-09-08 19:17:58 +08:00
Krish DholakiaandGitHub ba10173ec7 Merge branch 'main' into heroku-llms 2025-09-06 22:10:20 -07:00
Thomas Rehn d88771ca49 fix: correct output pricing for gemini-2.5-flash-image-preview https://ai.google.dev/gemini-api/docs/pricing#gemini-2.5-flash-image-preview 2025-09-05 16:22:07 +02:00
Krish DholakiaandGitHub f67339a86c Merge pull request #14028 from onlylhf/volcengine-embedding-support 
Add Volcengine embedding module with handler and transformation logic
 2025-09-04 21:01:25 -07:00
Ishaan JaffandGitHub 23ae7170d1 [Feat] Allow using Veo Video Generation through LiteLLM Pass through routes (#14228) 
* fix: add follow_redirects=True,

* test_pass_through_with_httpbin_redirect

* cook book veo video

* docs Veo Video Generation with Google AI Studio

* add veo-3.0-generate-preview cost tracking details

* track vertex_video_models
 2025-09-03 18:25:43 -07:00
Harshavardhan KandGitHub bc84b411f4 Fix/remove deprecated cerebras gpt oss 20b (#14213) 
* fix: remove deprecated cerebras/openai/gpt-oss-20b model from pricing files

- Remove cerebras/openai/gpt-oss-20b entry from model_prices_and_context_window.json
- Remove cerebras/openai/gpt-oss-20b entry from model_prices_and_context_window_backup.json
- Fixes GitHub issue #13972 where the deprecated model was still being treated as valid
- Only the 120b version is now supported by Cerebras

* fix: remove deprecated cerebras gpt-oss-20b
 2025-09-03 09:46:08 -07:00
Ishaan JaffandGitHub c821f1ddf1 [Feature]: Support GPT-OSS models on vertex ai (#14184) 
* add VertexAIGPTOSSTransformation

* fix: optional_params

* fix: is_vertex_partner_model

* test_partner_models_httpx

* docs GPT oss docs

* test_vertex_ai_gpt_oss_reasoning_effort

* add vertex ai models
 2025-09-02 14:15:26 -07:00
Ishaan Jaff 94cac6fff2 fix: groq health check 2025-09-01 20:08:36 -07:00
Mubashir OsmaniandGitHub 58ac3ecf84 openrouter: added gpt 4.1 model family (#14101) 2025-09-01 15:36:17 -07:00
Krish DholakiaandGitHub 9e947fce19 Merge pull request #14087 from zainhas/main 
update together models
 2025-09-01 15:07:58 -07:00
Ishaan Jaff fa4c833e58 fix: supports reasoning 2025-08-30 10:29:08 -07:00
Krish DholakiaandGitHub 8b11452d9e Merge branch 'main' into add-vercel-ai-gateway-provider 2025-08-29 22:27:08 -07:00
Zain HasanandGitHub 8a5df2eb74 update together models 
- add tool choice for models
- add deepseek v3.1
 2025-08-30 00:44:42 -04:00
Ishaan JaffandGitHub be86861eb8 feat: add gpt-realtime models - gpt-realtime (#14082) 
* feat: add gpt-realtime models

* add input_cost_per_image
 2025-08-29 17:49:47 -07:00
Ishaan JaffandGitHub 607f425cc6 [Bug]: Fix Can't set reasoning_effort for DeepSeek-V3.1 on DeepInfra by default (#14053) 
* fix DeepInfraConfig

* track supports reasoning

* fix: get_supported_openai_params

* test_deepseek_supported_openai_params
 2025-08-29 13:15:43 -07:00
Ishaan JaffandGitHub 81db36c06c add xai/grok-code-fast (#14054) 2025-08-28 18:10:04 -07:00
李海峰 2d0a57a719 Add Volcengine embedding module with handler and transformation logic 
- Implemented VolcEngineEmbeddingHandler for synchronous and asynchronous embedding requests.
- Created VolcEngineEmbeddingConfig for transforming requests and responses to/from Volcengine format.
- Added integration tests for embedding functionality, covering various scenarios including error handling and parameter validation.
- Established test structure for Volcengine embedding, ensuring compliance with LiteLLM testing patterns.
- Included comprehensive tests for parameter mapping, request transformation, and response handling.
 2025-08-28 15:05:11 +08:00
Krish DholakiaandGitHub c43706061f Merge branch 'main' into heroku-llms 2025-08-27 22:21:36 -07:00
Ifta Khairul Alam Adil d9f8eb27c9 fix: enable tool choice support for model prices and context window 
- Updated the "supports_tool_choice" field to true in the model_prices_and_context_window.json file, allowing for tool choice functionality in the specified model.

Signed-off-by: Ifta Khairul Alam Adil <ikaadil007@gmail.com>
 2025-08-27 19:46:53 +02:00
Ifta Khairul Alam Adil 8808f78ba6 Merge branch 'main' into fix/gemini-imagen-model-name-validation 2025-08-27 15:45:59 +02:00
Krish DholakiaandGitHub 4613f7821d Merge pull request #13536 from edwardsamuel/feat/openrouter-gpt-5-pricing 
Add openrouter gpt-5 family models pricing
 2025-08-26 23:16:53 -07:00
Krish DholakiaandGitHub 201d96b2d2 Merge pull request #13917 from Toy-97/patch-1 
Deepinfra Metadata Update 24082025
 2025-08-26 23:13:46 -07:00
Ifta Khairul Alam Adil 0f5b31fd78 fix: resolve invalid model name error for Gemini Imagen models (#13851) 
- Fix URL construction in Gemini image generation to strip 'gemini/' prefix
- Google AI API expects base model name without the prefix
- Update model references and pricing information for consistency
- Remove outdated image generation pricing entries

Fixes issue where models like 'gemini/imagen-4.0-fast-generate-preview-06-06'
were being rejected by the Google AI API due to incorrect URL formatting.
 2025-08-27 04:17:35 +02:00
Ishaan JaffandGitHub 5647757ab3 [Feat] New model gemini-2.5-flash-image-preview (#13979) 
* add gemini-2.5-flash-image-preview

* add gemini-2.5-flash-image-preview
 2025-08-26 15:04:02 -07:00
Krish DholakiaandGitHub 55c3fd62ac Add Claude Opus 4 and 4.1 via OpenRouter and Fix Claude Sonnet 4 and 3.7 via OpenRouter in Prices and Context Window Config 
Update model prices and context window
 2025-08-25 23:28:39 -07:00
Krish DholakiaandGitHub 3803af5ff5 Merge pull request #13566 from kankute-sameer/fix/azure-gpt4-web-search-support 
fix: remove incorrect web search support for azure/gpt-4.1 family
 2025-08-25 23:28:15 -07:00
Teddy Amkie 97e9502f4a Add DeepSeek-v3.1 pricing for Fireworks AI provider 
- Add fireworks_ai/accounts/fireworks/models/deepseek-v3p1 model configuration
- Set context window: 128K input, 8K output tokens
- Pricing: /bin/zsh.56/1M input tokens, .68/1M output tokens
- Supports response schema and tool choice
- Based on DeepSeek API unified pricing effective Sept 2025
 2025-08-25 18:41:49 -07:00
Toy-97andGitHub f2bb1ce31e Deepinfra Metadata Update 24082025 
Deepseek v3.1 Price Dropped
 2025-08-24 08:30:25 +08:00
Ishaan JaffandGitHub 9750374081 [Feat] New LLM API - AI/ML API for Image Gen (#13893) 
* add LlmProviders.AIML

* add AIMLChatConfig

* add aiml

* fix AimlImageGenerationConfig

* add AimlImageGenerationConfig

* add cost_calculator

* fixes for AI ML API

* add known AIML Flux image models

* add AIML Cost Calc

* add AI/ML API

* add aiml_models
 2025-08-23 13:12:44 -07:00
Krish DholakiaandGitHub 49cb9bdbca Merge pull request #13843 from BerriAI/litellm_dev_08_29_2025_p3 
SSO - Free SSO usage for up to 5 users + remove deprecated dbrx models (dbrx-instruct, llama 3.1)
 2025-08-23 12:06:01 -07:00