Commit Graph
244 Commits
Author SHA1 Message Date
Krish DholakiaandGitHub fbc0ec98b6 Merge pull request #12438 from jasonpnnl/fix-customer-update
Fix 500 error in `/customer/update` endpoint when updating with `budget_id`
2025-09-06 22:12:30 -07:00
c9ab104445 [Fix] Perf fix: Heavy RAM Usage over time when using Passthrough Routes (#14305)
* Implement pass-through route registration and removal to prevent duplicates and memory leaks

* fix MYPY linting

* fix InitPassThroughEndpointHelpers

* fix test_aaparallel_function_call

---------

Co-authored-by: Jugal Bhatt <jugalbhatt3@gmail.com>
2025-09-06 19:22:25 -07:00
Ishaan Jaff 7f5460465c unit test oai pass through 2025-09-06 14:54:34 -07:00
Sameer KankuteandGitHub 07ba3ff036 [Feat] Add pass through image gen and image editing on OpenAI (#14292)
* add pass through image gen and image editing on OpenAI

* fix lint
2025-09-05 12:25:49 -07:00
Ishaan JaffandGitHub 379b0dbf14 [Fix] Ensure team_id is a required field for generating service account keys (#14270)
* generate_service_account_key_fn

* fix validate_team_id_used_in_service_account_request

* fix types

* test_validate_team_id_used_in_service_account_request_requires_team_id
2025-09-04 18:14:38 -07:00
Sameer KankuteandGitHub 5f79e8aac6 Litellm passthrough cost tracking chat completion (#14256)
* feat: add structured output for sdk

* Add support for cost tracking for chat completion in passthrough

* remove not required changes
2025-09-04 09:57:48 -07:00
Krish DholakiaandGitHub a8cce5fbf3 Merge pull request #14237 from yeahyung/fix/tpm_limit_bug
Fixes #14204 TPM Rate Limit Bug
2025-09-04 07:29:47 -07:00
22mSqRi 25afa67616 fix: Key Budget not resets at expectable times 2025-09-04 09:04:01 +00:00
yeahyung 7829de2948 (#14204) add test code 2025-09-04 16:41:29 +09:00
Ishaan JaffandGitHub 23ae7170d1 [Feat] Allow using Veo Video Generation through LiteLLM Pass through routes (#14228)
* fix: add follow_redirects=True,

* test_pass_through_with_httpbin_redirect

* cook book veo video

* docs Veo Video Generation with Google AI Studio

* add veo-3.0-generate-preview cost tracking details

* track vertex_video_models
2025-09-03 18:25:43 -07:00
Ishaan JaffandGitHub beb300abae [Fix] SCIM - Bug fixes for handling SCIM Group Memberships (#14226)
* Feat: add better SCIM debugging

* fix _get_scim_member_display

* fix patch_group

* test_update_group_e2e

* test_get_scim_member_value
2025-09-03 16:15:51 -07:00
kayoch1n 76555cad81 Format code 2025-09-03 11:20:52 +08:00
kayoch1n ffbe5cd899 Add a testcase 2025-09-03 11:08:23 +08:00
Ishaan JaffandGitHub 128d9a3488 [Feature]: Add header support for spend_logs_metadata (#14186)
* fix: allow settings spend_logs_metadata

* fix add_litellm_data_for_backend_llm_call

* fix: add add_litellm_metadata_from_request_headers

* fix add_litellm_metadata_from_request_headers

* test_add_litellm_metadata_from_request_headers

* add_litellm_metadata_from_request_headers

* docs Tracking Spend with custom metadata

* add_litellm_metadata_from_request_headers

* add_litellm_metadata_from_request_headers
2025-09-02 15:13:15 -07:00
Krish DholakiaandGitHub d2c519f005 Merge pull request #14133 from retanoj/fix/gemini-token-count
Fix token count error for gemini cli
2025-09-01 22:39:51 -07:00
Ishaan JaffandGitHub 98d57b5d27 [Feat] Allow using x-litellm-stream-timeout header for stream timeout in requests (#14147)
* fix: allow passing stream_timeout header

* fix: _get_stream_timeout_from_request

* test_add_litellm_data_to_request_with_stream_timeout_header

* docs: LiteLLM Headers

* test_add_litellm_data_to_request_with_stream_timeout_header
2025-09-01 15:59:14 -07:00
retanoj 002c2f16dc fix token count error when proxy gemini cli to openai like model 2025-09-01 20:02:53 +08:00
Ishaan JaffandGitHub 37a599932d [Bug Fix] Virtual keys with llm_api type cause Internal Server Error when using /anthropic/* and other llm passthrough routes (#14046)
* fix route checks passthrough_routes_wildcard

* refactor check_route_access

* test_virtual_key_llm_api_route_includes_passthrough_prefix
2025-08-28 15:14:09 -07:00
Krish DholakiaandGitHub 10816a44d4 Merge pull request #13160 from pangeacyber/aidr-update
Update Pangea Guardrail to support new AIDR endpoint
2025-08-27 22:15:32 -07:00
9acf80b8ad Fix issue #13995: Handle None metadata in batch requests (#13996)
* Fix issue #13995: Handle None metadata in batch requests

- Added null check in add_key_level_controls method to prevent NoneType error
- Updated type hint to Optional[dict] for better type safety
- Added comprehensive test suite to verify the fix works correctly
- All existing tests pass, confirming no regression

Fixes #13995

Co-authored-by: openhands <openhands@all-hands.dev>

* Move test file to tests/test_litellm/proxy/ directory

- Moved test_batch_metadata_none_fix.py from tests/ to tests/test_litellm/proxy/
- Updated import structure to match existing test patterns
- This ensures the test runs in GitHub Actions as requested by @krrishdholakia

Co-authored-by: openhands <openhands@all-hands.dev>

---------

Co-authored-by: openhands <openhands@all-hands.dev>
2025-08-27 14:51:09 -07:00
Krish DholakiaandGitHub 45f465d84f Merge pull request #13572 from Noma-Security/noma-security-guardrails-support
Add Noma Security guardrail support
2025-08-26 23:15:17 -07:00
Krish DholakiaandGitHub 9df6f4ef08 Merge branch 'main' into litellm_dev_08_20_2025_p1 2025-08-23 12:09:53 -07:00
Krrish Dholakia b23e35f75e test: update tests 2025-08-23 12:09:19 -07:00
Ishaan Jaff 825d6b08f0 test fix location of encode bedrock runtime modelid arn 2025-08-22 17:34:09 -07:00
Jugal D. BhattandGitHub 3c1dcb64cc fix by adding init lock (#13666) 2025-08-22 11:42:43 -07:00
Krish DholakiaandGitHub 6214d22126 Merge pull request #13863 from luizrennocosta/fix/pll-req-lim-v3-api-key
fix: fixing descriptor/response size mismatch on parallel_request_limiter_v3
2025-08-21 23:14:04 -07:00
Krish DholakiaandGitHub a61275f6e2 Merge pull request #13796 from 0x-fang/fix_aip_pass_through_08_20
Fix(Bedrock): fix application inference profile for pass-through endpoints for bedrock
2025-08-21 19:41:23 -07:00
Ishaan JaffandGitHub 77a5de37b1 [Fix] Ensure Service Account Keys require team_id field on API Endpoints (#13873)
* fixes for generating service accounts through the API

* fixes for test_update_service_account_works_with_team_id
2025-08-21 14:55:06 -07:00
Luiz Rennó Costa f8711200d0 fix: fixing descriptor/response size difference when indexing descriptor 2025-08-21 13:43:23 -03:00
Fang Gong 91040b6b56 fix the api key support for bedrock guardrail in proxy 2025-08-20 18:46:00 -07:00
Krrish Dholakia 34ad646101 fix(auth_checks.py): handle both flows + add unit tests 2025-08-20 16:42:08 -07:00
Krrish Dholakia e5c3e006c2 fix(proxy_server.py): fix merge config + db router settings logic 2025-08-20 16:33:31 -07:00
Fang Gong caad9b3ca7 fix application inference profile for pass-through endpoints for bedrock 2025-08-20 00:40:43 -07:00
Krrish Dholakia 2e16f2cb13 test: add unit tests 2025-08-18 21:19:43 -07:00
Ryan Means a05330fcd8 Fix unit tests 2025-08-18 12:00:03 -07:00
Ishaan JaffandGitHub 40395598da [Feat] UI - Allow editing team member rpm/tpm limits (#13669)
* show team member tpm/rpm limits

* ui - allow setting team settings

* fix better debugging

* fix types: TeamMemberUpdateRequest

* add _upsert_budget_and_membership

* allow updating team member RPM/TPM in teamMemberUpdateCall

* editing team member rpm/tpm

* UI - fixes for team member component

* fix info

* test_upsert_rpm_only_creates_new_budget
2025-08-15 17:29:44 -07:00
Daniel BarkerandGitHub 47edecd5bc Fixed incorrect key info endpoint (#13633) 2025-08-15 11:10:06 -07:00
Jugal D. BhattandGitHub b8fe5f7b17 [MCP Gateway] LiteLLM Fix MCP gateway key auth (#13630)
* Fix - add safe divide by 0 for most places to prevent crash

* Enhance MCPRequestHandler to support permission inheritance and intersection logic for access groups. Added integration tests to verify behavior when keys have no permissions and when both keys and teams have overlapping permissions.

* Remove redundant assertions for permission checks in test_user_api_key_auth_mcp.py to streamline test logic.

* Refactor integration tests for MCPRequestHandler to simplify mocking. Replace complex database mocks with direct function mocks for permission inheritance and intersection scenarios, improving test clarity and maintainability.

* Revert "Fix - add safe divide by 0 for most places to prevent crash"

This reverts commit 265d40e39051e148996b9fb7f354730c57ff23ac.
2025-08-14 16:32:18 -07:00
Jugal Bhatt 45f188b041 Add mock user API key authentication in tag management tests
This update integrates mock user API key authentication into the tag management endpoint tests, ensuring accurate simulation of user roles for creating, updating, and deleting tags. The changes enhance the reliability of the tests by properly setting up user authentication before executing test cases.
2025-08-14 15:03:21 -07:00
Jugal Bhatt d21f467264 Revert "Add mock user API key authentication in tag management tests"
This reverts commit 5fc0803b94.
2025-08-14 14:58:49 -07:00
Jugal Bhatt 5fc0803b94 Add mock user API key authentication in tag management tests
This update introduces a helper function to create a mock user API key authentication object, which is utilized in the tag management endpoint tests. The mock authentication is integrated into the test cases for creating, updating, and deleting tags, enhancing the reliability of the tests by simulating user roles accurately.
2025-08-14 14:40:49 -07:00
Ishaan Jaff cd893134b7 test team endpoints 2025-08-13 18:43:50 -07:00
Ishaan Jaff 086621e3d3 test_handle_jwt.py 2025-08-13 17:27:50 -07:00
Ishaan JaffandGitHub 8e76f8e7d0 [Feat] Team Member Rate Limits + Support for using with JWT Auth (#13601)
* fix - assign tpm/rpm limit onJWT

* add team member rpm/tpm limits

* update - rate limiter v3 with team member rate limits

* update utils

* fixes for LiteLLM_BudgetTable

* undo change

* add TeamMemberBudgetHandler

* add _process_team_member_budget_data

* add get_team_membership

* add safe_get_team_member_rpm_limit and safe_get_team_member_tpm_limit

* LiteLLM_TeamMembership

* add LiteLLM_TeamMembership rate limit for JWTs

* fix

* tests
2025-08-13 17:21:36 -07:00
Krzysztof GąsiorowskiandGitHub 0b28930ad4 [Fix] Hide sensitive data in /model/info - azure entra client_secret (#13577)
* Remove litellm_params.client_secret from /model/info

Added pop of client_secret (Azure provider secret) from litellm_params in remove_sensitive_info_from_deployment used by /model/info endpoints

* Added test for litellm.proxy.common_utils.openai_endpoint_utils.remove_sensitive_info_from_deployment
2025-08-13 07:35:53 -07:00
Dor Zion 6a1f5bdc2f Add Noma Security guardrail support 2025-08-13 13:39:21 +03:00
Krish DholakiaandGitHub f00e891004 LiteLLM SDK <-> Proxy: support user param + Prisma - remove use_prisma_migrate flag - redundant as this is now default (#13555)
* fix(litellm_proxy/chat/transformation.py): support 'user' and all other openai chat completion params

Fixes issue where 'user' was not being sent in request to litellm proxy via sdk

* fix(prisma_migration.py): remove 'use_prisma_migrate' flag, is now default

* docs: cleanup docs

* fix(proxy_cli.py): remove --use_prisma_migrate flag

* refactor: remove references to use_prisma_migrate env var

This is now the default flow for db migrations
2025-08-12 22:03:39 -07:00
Ishaan JaffandGitHub 74ddd21067 [Bug fix] SCIM Team Memberships - handle metadata (#13553)
* test_update_group_metadata_serialization_issue

* fixes for update_group

* test_update_group_metadata_serialization_issue
2025-08-12 20:53:12 -07:00
Ishaan JaffandGitHub f60a9cf908 [Bug]: Fix JWTs access not working with model groups (#13474)
* fix can_team_access_model

* test_find_team_with_model_access_model_group
2025-08-09 16:14:51 -07:00
Jugal D. BhattandGitHub 67833590d6 [Proxy changes] Litellm add model price reload schedule for multi-pod (#13470)
* added mcp guardrails doc in mcp.md

* add button to reload models

* Added button changes

* added button for scheduling reload

* add multi pod support to reloading the model price json

* fix ruff
2025-08-09 16:12:13 -07:00