Commit Graph

253 Commits

Author SHA1 Message Date
Ishaan Jaff 18372f9ebe Revert "fix vertex ai file upload" (#14501) 2025-09-12 12:02:24 -07:00
Boopesh Shanmugam 8b338a4d8c User Headers X LiteLLM Users Mapping feature (#14485)
* Draft commit.

* user header mapping feature with backward compatibility with user_header_name field.

* user header mapping feature with backward compatibility with user_header_name field optimizations.

* Added unit tests.
2025-09-12 11:49:37 -07:00
Krish Dholakia 113d2a8c5a Merge pull request #14459 from holzman/fix-provider-budget
Fix provider budgets
2025-09-12 00:34:52 -07:00
Burt Holzman e9e548d797 Fix provider budgets 2025-09-11 12:08:43 -05:00
Sameer Kankute 090e0fddf4 fix vertex ai file upload 2025-09-11 22:20:54 +05:30
Krrish Dholakia bdd7255bab test: remove redundant tests - moved to parallel_request_limiter_v3.py 2025-09-09 21:35:50 -07:00
pinkgu 285b14bd69 Fix/gcp model armor (#14381)
* chore: fix model armor always success

Signed-off-by: bjorn <bjornjee95@gmail.com>

* chore: fix sanitisation field extraction

Signed-off-by: bjorn <bjornjee95@gmail.com>

---------

Signed-off-by: bjorn <bjornjee95@gmail.com>
2025-09-09 09:18:37 -07:00
Krish Dholakia 54c3ae15c8 Merge pull request #14312 from mubashir1osmani/main
fix: spend tags endpoint returns 500
2025-09-08 19:05:13 -07:00
mubashir1osmani d9bbbc10a6 added tests 2025-09-07 10:17:28 -04:00
Krish Dholakia fbc0ec98b6 Merge pull request #12438 from jasonpnnl/fix-customer-update
Fix 500 error in `/customer/update` endpoint when updating with `budget_id`
2025-09-06 22:12:30 -07:00
Ishaan Jaff c9ab104445 [Fix] Perf fix: Heavy RAM Usage over time when using Passthrough Routes (#14305)
* Implement pass-through route registration and removal to prevent duplicates and memory leaks

* fix MYPY linting

* fix InitPassThroughEndpointHelpers

* fix test_aaparallel_function_call

---------

Co-authored-by: Jugal Bhatt <jugalbhatt3@gmail.com>
2025-09-06 19:22:25 -07:00
Ishaan Jaff 7f5460465c unit test oai pass through 2025-09-06 14:54:34 -07:00
Sameer Kankute 07ba3ff036 [Feat] Add pass through image gen and image editing on OpenAI (#14292)
* add pass through image gen and image editing on OpenAI

* fix lint
2025-09-05 12:25:49 -07:00
Ishaan Jaff 379b0dbf14 [Fix] Ensure team_id is a required field for generating service account keys (#14270)
* generate_service_account_key_fn

* fix validate_team_id_used_in_service_account_request

* fix types

* test_validate_team_id_used_in_service_account_request_requires_team_id
2025-09-04 18:14:38 -07:00
Sameer Kankute 5f79e8aac6 Litellm passthrough cost tracking chat completion (#14256)
* feat: add structured output for sdk

* Add support for cost tracking for chat completion in passthrough

* remove not required changes
2025-09-04 09:57:48 -07:00
Krish Dholakia a8cce5fbf3 Merge pull request #14237 from yeahyung/fix/tpm_limit_bug
Fixes #14204 TPM Rate Limit Bug
2025-09-04 07:29:47 -07:00
22mSqRi 25afa67616 fix: Key Budget not resets at expectable times 2025-09-04 09:04:01 +00:00
yeahyung 7829de2948 (#14204) add test code 2025-09-04 16:41:29 +09:00
Ishaan Jaff 23ae7170d1 [Feat] Allow using Veo Video Generation through LiteLLM Pass through routes (#14228)
* fix: add follow_redirects=True,

* test_pass_through_with_httpbin_redirect

* cook book veo video

* docs Veo Video Generation with Google AI Studio

* add veo-3.0-generate-preview cost tracking details

* track vertex_video_models
2025-09-03 18:25:43 -07:00
Ishaan Jaff beb300abae [Fix] SCIM - Bug fixes for handling SCIM Group Memberships (#14226)
* Feat: add better SCIM debugging

* fix _get_scim_member_display

* fix patch_group

* test_update_group_e2e

* test_get_scim_member_value
2025-09-03 16:15:51 -07:00
kayoch1n 76555cad81 Format code 2025-09-03 11:20:52 +08:00
kayoch1n ffbe5cd899 Add a testcase 2025-09-03 11:08:23 +08:00
Ishaan Jaff 128d9a3488 [Feature]: Add header support for spend_logs_metadata (#14186)
* fix: allow settings spend_logs_metadata

* fix add_litellm_data_for_backend_llm_call

* fix: add add_litellm_metadata_from_request_headers

* fix add_litellm_metadata_from_request_headers

* test_add_litellm_metadata_from_request_headers

* add_litellm_metadata_from_request_headers

* docs Tracking Spend with custom metadata

* add_litellm_metadata_from_request_headers

* add_litellm_metadata_from_request_headers
2025-09-02 15:13:15 -07:00
Krish Dholakia d2c519f005 Merge pull request #14133 from retanoj/fix/gemini-token-count
Fix token count error for gemini cli
2025-09-01 22:39:51 -07:00
Ishaan Jaff 98d57b5d27 [Feat] Allow using x-litellm-stream-timeout header for stream timeout in requests (#14147)
* fix: allow passing stream_timeout header

* fix: _get_stream_timeout_from_request

* test_add_litellm_data_to_request_with_stream_timeout_header

* docs: LiteLLM Headers

* test_add_litellm_data_to_request_with_stream_timeout_header
2025-09-01 15:59:14 -07:00
retanoj 002c2f16dc fix token count error when proxy gemini cli to openai like model 2025-09-01 20:02:53 +08:00
Ishaan Jaff 37a599932d [Bug Fix] Virtual keys with llm_api type cause Internal Server Error when using /anthropic/* and other llm passthrough routes (#14046)
* fix route checks passthrough_routes_wildcard

* refactor check_route_access

* test_virtual_key_llm_api_route_includes_passthrough_prefix
2025-08-28 15:14:09 -07:00
Krish Dholakia 10816a44d4 Merge pull request #13160 from pangeacyber/aidr-update
Update Pangea Guardrail to support new AIDR endpoint
2025-08-27 22:15:32 -07:00
Xingyao Wang 9acf80b8ad Fix issue #13995: Handle None metadata in batch requests (#13996)
* Fix issue #13995: Handle None metadata in batch requests

- Added null check in add_key_level_controls method to prevent NoneType error
- Updated type hint to Optional[dict] for better type safety
- Added comprehensive test suite to verify the fix works correctly
- All existing tests pass, confirming no regression

Fixes #13995

Co-authored-by: openhands <openhands@all-hands.dev>

* Move test file to tests/test_litellm/proxy/ directory

- Moved test_batch_metadata_none_fix.py from tests/ to tests/test_litellm/proxy/
- Updated import structure to match existing test patterns
- This ensures the test runs in GitHub Actions as requested by @krrishdholakia

Co-authored-by: openhands <openhands@all-hands.dev>

---------

Co-authored-by: openhands <openhands@all-hands.dev>
2025-08-27 14:51:09 -07:00
Krish Dholakia 45f465d84f Merge pull request #13572 from Noma-Security/noma-security-guardrails-support
Add Noma Security guardrail support
2025-08-26 23:15:17 -07:00
Krish Dholakia 9df6f4ef08 Merge branch 'main' into litellm_dev_08_20_2025_p1 2025-08-23 12:09:53 -07:00
Krrish Dholakia b23e35f75e test: update tests 2025-08-23 12:09:19 -07:00
Ishaan Jaff 825d6b08f0 test fix location of encode bedrock runtime modelid arn 2025-08-22 17:34:09 -07:00
Jugal D. Bhatt 3c1dcb64cc fix by adding init lock (#13666) 2025-08-22 11:42:43 -07:00
Krish Dholakia 6214d22126 Merge pull request #13863 from luizrennocosta/fix/pll-req-lim-v3-api-key
fix: fixing descriptor/response size mismatch on parallel_request_limiter_v3
2025-08-21 23:14:04 -07:00
Krish Dholakia a61275f6e2 Merge pull request #13796 from 0x-fang/fix_aip_pass_through_08_20
Fix(Bedrock): fix application inference profile for pass-through endpoints for bedrock
2025-08-21 19:41:23 -07:00
Ishaan Jaff 77a5de37b1 [Fix] Ensure Service Account Keys require team_id field on API Endpoints (#13873)
* fixes for generating service accounts through the API

* fixes for test_update_service_account_works_with_team_id
2025-08-21 14:55:06 -07:00
Luiz Rennó Costa f8711200d0 fix: fixing descriptor/response size difference when indexing descriptor 2025-08-21 13:43:23 -03:00
Fang Gong 91040b6b56 fix the api key support for bedrock guardrail in proxy 2025-08-20 18:46:00 -07:00
Krrish Dholakia 34ad646101 fix(auth_checks.py): handle both flows + add unit tests 2025-08-20 16:42:08 -07:00
Krrish Dholakia e5c3e006c2 fix(proxy_server.py): fix merge config + db router settings logic 2025-08-20 16:33:31 -07:00
Fang Gong caad9b3ca7 fix application inference profile for pass-through endpoints for bedrock 2025-08-20 00:40:43 -07:00
Krrish Dholakia 2e16f2cb13 test: add unit tests 2025-08-18 21:19:43 -07:00
Ryan Means a05330fcd8 Fix unit tests 2025-08-18 12:00:03 -07:00
Ishaan Jaff 40395598da [Feat] UI - Allow editing team member rpm/tpm limits (#13669)
* show team member tpm/rpm limits

* ui - allow setting team settings

* fix better debugging

* fix types: TeamMemberUpdateRequest

* add _upsert_budget_and_membership

* allow updating team member RPM/TPM in teamMemberUpdateCall

* editing team member rpm/tpm

* UI - fixes for team member component

* fix info

* test_upsert_rpm_only_creates_new_budget
2025-08-15 17:29:44 -07:00
Daniel Barker 47edecd5bc Fixed incorrect key info endpoint (#13633) 2025-08-15 11:10:06 -07:00
Jugal D. Bhatt b8fe5f7b17 [MCP Gateway] LiteLLM Fix MCP gateway key auth (#13630)
* Fix - add safe divide by 0 for most places to prevent crash

* Enhance MCPRequestHandler to support permission inheritance and intersection logic for access groups. Added integration tests to verify behavior when keys have no permissions and when both keys and teams have overlapping permissions.

* Remove redundant assertions for permission checks in test_user_api_key_auth_mcp.py to streamline test logic.

* Refactor integration tests for MCPRequestHandler to simplify mocking. Replace complex database mocks with direct function mocks for permission inheritance and intersection scenarios, improving test clarity and maintainability.

* Revert "Fix - add safe divide by 0 for most places to prevent crash"

This reverts commit 265d40e39051e148996b9fb7f354730c57ff23ac.
2025-08-14 16:32:18 -07:00
Jugal Bhatt 45f188b041 Add mock user API key authentication in tag management tests
This update integrates mock user API key authentication into the tag management endpoint tests, ensuring accurate simulation of user roles for creating, updating, and deleting tags. The changes enhance the reliability of the tests by properly setting up user authentication before executing test cases.
2025-08-14 15:03:21 -07:00
Jugal Bhatt d21f467264 Revert "Add mock user API key authentication in tag management tests"
This reverts commit 5fc0803b94.
2025-08-14 14:58:49 -07:00
Jugal Bhatt 5fc0803b94 Add mock user API key authentication in tag management tests
This update introduces a helper function to create a mock user API key authentication object, which is utilized in the tag management endpoint tests. The mock authentication is integrated into the test cases for creating, updating, and deleting tags, enhancing the reliability of the tests by simulating user roles accurately.
2025-08-14 14:40:49 -07:00