tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-08-11 14:22:48 +00:00
Code Issues Packages Projects Releases Wiki Activity
26,173 Commits 2,072 Branches 1,425 Tags
ff893cce42f980a2f55f358fa3cb4154ccfc8cac
Commit Graph
26173 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Krrish Dholakia ff893cce42 feat(mcp_server_manager.py): working call tools for openapi to mcp servers 2025-10-08 15:31:52 -07:00
Krrish Dholakia 386468ad77 feat: initial commit for openapi to mcp conversion 
allows users to give openapi spec for a server, and litellm to serve it as an mcp
 2025-10-08 15:27:48 -07:00
Krrish Dholakia 45ec1ffdaa fix(ui/): set is sso modal visible to true for all users 
allow backend to handle more complex state management
 2025-10-08 13:13:21 -07:00
Sameer KankuteGitHubWeijie
 		0b6b69cd5b fix issue with parsing assistant messages (#15320) 
* Update factory.py

* reinclude the test cases

* fixed test

---------

Co-authored-by: Weijie <weijie-tan3@github.com>
 2025-10-08 09:49:02 -07:00
Krish DholakiaandGitHub 13703f289b Merge pull request #15292 from timelfrink/fix/bedrock-prompt-caching-cost-calculation 
fix(bedrock): include cacheWriteInputTokens in prompt_tokens calculation
 2025-10-07 19:11:45 -07:00
Krish DholakiaandGitHub 3f4d2c6ada Add Cohere Embed v4 support for AWS Bedrock 
Add Cohere Embed v4 support for AWS Bedrock
 2025-10-07 19:10:22 -07:00
Ishaan Jaffer 5d6cd3ffed UI new build 2025-10-07 18:59:11 -07:00
Ishaan Jaffer a8243079d3 doc fix 2025-10-07 18:51:27 -07:00
Ishaan JaffandGitHub 8a8cc5a1d3 [QA/Fixes] - Dynamic Rate Limiter v3 - final QA (#15311) 
* fix: PriorityReservationSettings

* fix: use correct casting

* fix type casting
 2025-10-07 18:42:17 -07:00
Krish DholakiaandGitHub 1238464e20 Merge pull request #15303 from BerriAI/litellm_tenacity_upgrade 
Upgrades tenacity version to 8.5.0
 2025-10-07 18:19:52 -07:00
Krish DholakiaandGitHub 60fb8cdb87 Merge pull request #15308 from BerriAI/litellm_models_page_crash_fix 
fix: model + endpoints page crash when config file contains router_settings.model_group_alias
 2025-10-07 18:19:15 -07:00
Ishaan Jaffer 7b82473bfb fix gpt-image-1-mini 2025-10-07 17:56:51 -07:00
Ishaan Jaffer e1ab3620ee fix: mapped tests 2025-10-07 17:55:52 -07:00
Ishaan Jaffer 8590646b84 fix code QA check 2025-10-07 17:49:57 -07:00
Ishaan Jaffer f2a96e6830 fix: e2eUI testing 2025-10-07 17:45:30 -07:00
Achintya Rajan 9fa8c6099e Update model_group_alias_settings.tsx 2025-10-07 17:42:04 -07:00
Ishaan JaffandGitHub 36c971a6fd [MCP Gateway] QA/Fixes - Ensure Team/Key level enforcement works for MCPs (#15305) 
* fix: _set_object_permission

* fix: _set_object_permission on teams

* fix: _set_object_permission

* fixes for team/key permissions

* statsh: object permission view

* fix: MCPServerPermissions

* fix: _get_team_object_permission

* test mcp checks for permissions

* fix server checks with prefix names

* test_list_tools_strips_prefix_when_matching_permissions

* ruff fix

* docs - refactor MCP

* docs update MCP docs

* docs allowed tools
 2025-10-07 17:34:48 -07:00
Ishaan JaffandGitHub 7b56ba240e [MCP Gateway] Litellm mcp fixes team control (#15304) 
* fix: _set_object_permission

* fix: _set_object_permission on teams

* fix: _set_object_permission

* fixes for team/key permissions

* statsh: object permission view

* fix: MCPServerPermissions
 2025-10-07 16:48:00 -07:00
Alexsander HamirandGitHub 49e04e0217 [Fix] Networking: remove limitations (#15302) 
* fix: remove limitations

* fix: linter issues
 2025-10-07 16:45:42 -07:00
Achintya Rajan f72f21b261 Update requirements.txt 2025-10-07 16:18:48 -07:00
Ishaan JaffandGitHub bc26eff98f Fix: Make PATCH /model/{model_id}/update handle team_id consistently with POST /model/new (#15297) 
* fix: _update_team_model_in_db

* test_patch_model_with_team_id_creates_proper_setup
 2025-10-07 14:04:08 -07:00
Tim Elfrink d71d801e4d Add Cohere Embed v4 support for AWS Bedrock 
- Add cohere.embed-v4:0 to model pricing configs
- Update bedrock_embedding_models constant
- Update documentation with v4 model support

Fixes #15272
 2025-10-07 22:08:12 +02:00
Ishaan JaffandGitHub 07a17d6d6b [Feat] Proxy CLI - dont store existing key in the URL, store it in the state param (#15290) 
* Feat: CLI Auth fixes for UI SSO

* fix auth.py

* fix test ui sso.py
 2025-10-07 12:36:17 -07:00
Tim Elfrink c5eb22381d fix(bedrock): include cacheWriteInputTokens in prompt_tokens calculation 
Fixes #15263

This PR fixes the cost calculation for Bedrock Anthropic models with prompt caching.

**Root Cause:**
PR #9838 incorrectly removed adding `cacheWriteInputTokens` to `prompt_tokens`
for Bedrock, based on the assumption that it would cause double counting (similar
to an Anthropic API issue). However, Bedrock's token structure is different:

- **Bedrock API**: `inputTokens`, `cacheReadInputTokens`, and `cacheWriteInputTokens`
  are ALL separate values that should be summed for total input tokens
- **Anthropic API**: Same structure - all three token types are separate

The fix in #9838 was later reverted for Anthropic (correctly re-adding
`cache_creation_input_tokens` to `prompt_tokens`), but Bedrock was never fixed.

**Changes:**
1. Re-add `cacheWriteInputTokens` to `input_tokens` in Bedrock transformation
2. Update test assertions to reflect correct behavior
3. Add regression test for prompt caching cost calculation
4. Fix typo in Anthropic transformation where `cache_creation_tokens` was
   incorrectly set to `cache_read_input_tokens`

**Testing:**
- All existing Bedrock transformation tests pass
- New test validates correct cost calculation with prompt caching
- Verified costs are non-negative and accurate
 2025-10-07 20:28:46 +02:00
Sameer KankuteandGitHub 51971f4750 Add gpt-realtime-mini support (#15283) 2025-10-07 11:27:04 -07:00
Sameer KankuteandGitHub e4892735f0 fix gemini cli by actually streaming the response (#15264) 
* fix gemini cli by actually streaming the response

* fix cost tracking

* fix test
 2025-10-07 11:26:39 -07:00
Sameer KankuteandGitHub 73f96712f5 fix the reasoningresponse id (#15265) 2025-10-07 11:24:29 -07:00
Krrish Dholakia 421d38c94a build(ui/): build new ui v1.77.7.dev.3 2025-10-07 10:48:19 -07:00
Krish DholakiaandGitHub f044eb80de Merge pull request #15285 from BerriAI/litellm_infinity_new_provider_ui 
feature: adds Infinity as a provider in the UI
 2025-10-07 10:46:05 -07:00
Achintya Rajan e2f21beb7f added Infinity as a provider in the UI 2025-10-07 10:21:18 -07:00
Sameer KankuteandGitHub c0d0424eb8 Added streaming support for response api streaming image generation (#15269) 2025-10-07 08:15:57 -07:00
Krrish Dholakia faeb7484db docs(vertex.md): fix doc 2025-10-06 20:45:58 -07:00
Krish DholakiaandGitHub 12cbac74b1 Merge pull request #15210 from uc4w6c/feat/add_global_cross_region 
feat: add Global Cross-Region Inference
 2025-10-06 20:21:18 -07:00
Krish DholakiaandGitHub 077b5e105f Merge pull request #15240 from BerriAI/litellm_dev_10_06_2025_p1 
Azure - passthrough support with router models
 2025-10-06 20:06:43 -07:00
Krish DholakiaandGitHub 94a34dd53a Merge branch 'main' into litellm_dev_10_06_2025_p1 2025-10-06 20:06:30 -07:00
Krish DholakiaandGitHub 8b357c24c2 Merge pull request #15226 from otaviofbrito/chore/vertex-ai-context-caching 
Chore/vertex ai context caching
 2025-10-06 20:05:24 -07:00
Krish DholakiaandGitHub 6b4415684e Merge pull request #15253 from BerriAI/litellm_dev_10_06_2025_p2 
fix(azure/responses): remove invalid status param from azure call + MCP - support setting CA_BUNDLE_PATH
 2025-10-06 20:01:39 -07:00
Ishaan JaffandGitHub d1d411ca2c feat: add litellm_add_gpt_image1_mini_models (#15259) 2025-10-06 19:58:55 -07:00
Teddy AmkieandGitHub e5ca61d026 feat: add stop parameter to non-supported params for GPT-5 (#15244) 
- Add 'stop' parameter to the non_supported_params list in OpenAIGPT5Config
- This ensures the stop parameter is automatically dropped when calling GPT-5 models
- Resolves issue where stop parameter was not being stripped for GPT-5
 2025-10-06 19:57:21 -07:00
Sameer KankuteandGitHub 8d7f39798c Removed stop param from unsupported azure models (#15229) 
* Removed stop param from unsupported model

* Use better handling for stop method

* Use better handling for stop method
 2025-10-06 19:56:18 -07:00
Krish DholakiaandGitHub 6e538033ed Merge pull request #15249 from BerriAI/litellm_router_settings_update_fix 
fix: Router settings do not update despite success message
 2025-10-06 19:53:08 -07:00
Krish DholakiaandGitHub 6bb34e5168 Merge pull request #15252 from BerriAI/litellm_view_key_pagination_calls_fix 
(perf) fix: Replaces bloated key list calls with lean key aliases endpoint
 2025-10-06 19:51:26 -07:00
Krish DholakiaandGitHub 3f6717bfd8 Merge pull request #15250 from BerriAI/litellm_next_turbopack_for_dev 
Adds turbopack to the npm run dev command in UI to build faster during development
 2025-10-06 19:50:41 -07:00
Teddy AmkieandGitHub 1ad4ef267d Add GPT-5 Pro model configuration and documentation (#15258) 
- Added gpt-5-pro entry to model_prices_and_context_window.json with complete specifications
- Mode: responses (Responses API only)
- Pricing: $15.00 input / $120.00 output (Standard), $7.50 input / $60.00 output (Batch)
- Context: 400,000 input tokens, 272,000 output tokens
- Endpoints: /v1/batch, /v1/responses only
- Features: Web Search, Function Calling, Vision, PDF Input, Prompt Caching, Reasoning
- No streaming support, no Code Interpreter, no Computer Use

- Added gpt-5-pro to supported models list in docs/my-website/src/pages/completion/supported.md
- Added gpt-5-pro to OpenAI provider documentation in docs/my-website/docs/providers/openai.md
- Added comprehensive GPT-5 Pro Special Notes section with usage examples and limitations
 2025-10-06 19:43:40 -07:00
Ishaan JaffandGitHub 1d468a59f3 [MCP Gateway] Fine-grained Database Object Storage Control (#15255) 
* add supported_db_objects

* add _should_load_db_object

* add docs on storing MCP objects in DB

* test_should_load_db_object_with_supported_db_objects

* type fix
 2025-10-06 19:31:44 -07:00
Ishaan JaffandGitHub 930f88078e [MCP Gateway] Backend - Allow storing allowed tools by team/key (#15243) 
* add mcp_tool_permissions column to LiteLLM_ObjectPermissionTable

* add mcp_tool_permissions

* add LiteLLM_ObjectPermissionTable

* test key/team mgmt

* fix code QA check

* fix check_tool_permission_for_key_team

* test_key_tool_permission_allows_permitted_tool

* test_list_tools_filters_by_key_team_permissions

* fix _get_tools_from_mcp_servers

* fix logger.debug

* fix check_tool_permission_for_key_team

* fix: filter_tools_by_key_team_permissions

* fix _get_team_object_permission

* fix _set_object_permission

* fix handle_update_object_permission_common

* fix add allowed tools to keys

* test add key/team level tool permissions for MCPs
 2025-10-06 18:49:32 -07:00
Krrish Dholakia 7f88a3f9c6 fix(mcp/): add ssl certificate settings for mcp clients 
respect ca bundle path for mcp calls
 2025-10-06 18:36:05 -07:00
Alexsander HamirandGitHub 9aa2dc2e8e [Refactor] Utils: extract inner function from client (#15234) 
* fix: remove func definition from inside client

It makes the function bigger and harder to understand, I left just the wrappers.

* fix: test_arouter_test_team_model failure

- Added fallback to the model_name to index functionality.
 2025-10-06 18:30:01 -07:00
Achintya RajanandGitHub 264f1cded1 Merge branch 'main' into litellm_view_key_pagination_calls_fix 2025-10-06 18:10:57 -07:00
Krrish Dholakia 5336fcc000 fix(azure/responses): always remove status 
unsupported parameter
 2025-10-06 18:08:57 -07:00