Commit Graph
27 Commits
Author SHA1 Message Date
Ishaan JaffandGitHub ec9cdf783a Revert "Add streamGenerateContent cost tracking in passthrough (#15199)" (#15202)
This reverts commit 8095de506a.
2025-10-04 14:37:02 -07:00
Sameer KankuteandGitHub 8095de506a Add streamGenerateContent cost tracking in passthrough (#15199)
* Add streamgenerate cost tracking for gemini provider

* add cost tracking test
2025-10-04 12:31:25 -07:00
Sameer KankuteandGitHub 7ec7e5332c Add generateContent cost tracking (#15014) 2025-10-01 09:03:45 -07:00
Ishaan JaffandGitHub 0476a33d9f [Bug Fix] Passthrough API Endpoints - Ensure query params are forwarded from origin url to downstream request (#15087)
* test_pass_through_request_query_params_forwarding

* fix: pass_through_request

* test_azure_openai_assistants_e2e_operations_stream

* test_azure_openai_assistants_e2e_operations_stream
2025-09-30 15:01:38 -07:00
Sameer KankuteandGitHub 3ab1c31e4e (Feat) Add cost tracking for Vertex AI Passthrough /predict endpoint (#15019)
* Add cost tracking for passthrough for predict endpoint

* restore file
2025-09-29 13:19:04 -07:00
Ishaan Jaffer 35c14f7178 Revert "Merge pull request #14821 from hazyone/fix-13380"
This reverts commit 37ea89eff0, reversing
changes made to aede742db5.
2025-09-27 18:52:59 -07:00
hazyone 96aed6a4d4 test added 2025-09-23 20:16:05 +02:00
Otavio Brito ffd9117357 fix failing test 2025-09-22 11:06:11 -03:00
Otavio Brito 33a11c4b07 fix vllm passthrough 2025-09-22 10:09:02 -03:00
c9ab104445 [Fix] Perf fix: Heavy RAM Usage over time when using Passthrough Routes (#14305)
* Implement pass-through route registration and removal to prevent duplicates and memory leaks

* fix MYPY linting

* fix InitPassThroughEndpointHelpers

* fix test_aaparallel_function_call

---------

Co-authored-by: Jugal Bhatt <jugalbhatt3@gmail.com>
2025-09-06 19:22:25 -07:00
Ishaan Jaff 7f5460465c unit test oai pass through 2025-09-06 14:54:34 -07:00
Sameer KankuteandGitHub 07ba3ff036 [Feat] Add pass through image gen and image editing on OpenAI (#14292)
* add pass through image gen and image editing on OpenAI

* fix lint
2025-09-05 12:25:49 -07:00
Sameer KankuteandGitHub 5f79e8aac6 Litellm passthrough cost tracking chat completion (#14256)
* feat: add structured output for sdk

* Add support for cost tracking for chat completion in passthrough

* remove not required changes
2025-09-04 09:57:48 -07:00
Ishaan JaffandGitHub 23ae7170d1 [Feat] Allow using Veo Video Generation through LiteLLM Pass through routes (#14228)
* fix: add follow_redirects=True,

* test_pass_through_with_httpbin_redirect

* cook book veo video

* docs Veo Video Generation with Google AI Studio

* add veo-3.0-generate-preview cost tracking details

* track vertex_video_models
2025-09-03 18:25:43 -07:00
Ishaan Jaff 825d6b08f0 test fix location of encode bedrock runtime modelid arn 2025-08-22 17:34:09 -07:00
Fang Gong caad9b3ca7 fix application inference profile for pass-through endpoints for bedrock 2025-08-20 00:40:43 -07:00
039c8a922c Azure api_version="preview" support + Bedrock cost tracking via Anthropic /v1/messages (#13072)
* fix(azure/chat/gpt_transformation.py): support api_version="preview"

Fixes https://github.com/BerriAI/litellm/issues/12945

* Fix anthropic passthrough logging handler model fallback for streaming requests (#13022)

* fix: anthropic passthrough logging handler model fallback for streaming requests

- Add fallback logic to retrieve model from logging_obj.model_call_details when request_body.model is empty
- Fixes issue #12933 where streaming requests to anthropic passthrough endpoints would crash due to missing model field
- Ensures downstream logging and cost calculation work correctly for all streaming scenarios
- Maintains backwards compatibility with existing non-streaming requests

* test: add minimal tests for anthropic passthrough logging handler model fallback

- Add unit tests for the model fallback logic in _handle_logging_anthropic_collected_chunks
- Test existing behavior when request_body.model is present
- Test fallback logic when request_body.model is empty but logging_obj.model_call_details has model
- Test edge cases where both sources are empty or missing
- Ensure backwards compatibility and graceful degradation

* fix(anthropic_passthrough_logging_handler.py): add provider to model name (accurate cost tracking)

* fix(anthropic_passthrough_logging_handler.py): don't reset custom llm provider, if already set

* fix: fix check

---------

Co-authored-by: Haggai Shachar <haggai.shachar@backline.ai>
2025-07-29 08:13:55 -07:00
Ishaan JaffandGitHub deb1efa8ce [Bug Fix] Pass through logging handler VertexAI - ensure multimodal embedding responses are logged (#13050)
* fix _is_multimodal_embedding_response

* test_vertex_passthrough_handler_multimodal_embedding_response
2025-07-28 16:30:19 -07:00
Ishaan JaffandGitHub dc2c1122bf [Feat] UI QA: Pass through endpoints (#11939)
* use ID for pass through management

* use id for pass through

* fix columns

* fix PassThroughInfoView

* cleanup

* working edit and delete pass through

* fix rendering id for pt row

* fixes for pt info view

* working delete pass through

* fix use NumericalInput

* fix alignment

* qa - creating pt

* show route preview

* fix show just 1 msg

* test_create_pass_through_endpoint

* fix ui linting
2025-06-20 18:35:38 -07:00
Ishaan JaffandGitHub 3a1a8f5a78 [Bug Fix] - Ensure "Request" is tracked for pass through requests on LiteLLM Proxy (#11873)
* fixes for pt tracing

* test_pass_through_request_contains_proxy_server_request_in_kwargs
2025-06-18 21:15:09 -07:00
dfdbfdd71c [Feat] Passthrough - Add support for setting custom cost per pass through request (#11870)
* use cost_per_request

* fix cost_per_request

* fixes cost_per_request

* fixes for cost per request for pass through

* ui fix param name

* fixes for _set_cost_per_request

* test cost per request pass through endpoints

* Update tests/test_litellm/proxy/pass_through_endpoints/test_pass_through_endpoints.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* tests pass through endpoints

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-06-18 16:24:38 -07:00
Ishaan JaffandGitHub 0eb8a3de10 [Feat] v2 Pass through endpoints - Add support for subroutes for pass through endpoints + Cleaned up UI (#11827)
* fix: add construct_target_url_with_subpath

* add InitPassThroughEndpointHelpers

* added debugging for pass through routes

* add PassThroughGenericEndpoint to include subpath and input_cost_per_request

* polish page

* Add Pass-Through Endpoint

* polish pass through ui

* fixes for initialize_pass_through_endpoints

* PassThroughGenericEndpoint

* test_add_subpath_route

* test_initialize_pass_through_endpoints_with_include_subpath

* fix code QA check
2025-06-17 20:52:28 -07:00
35d74af5a0 [Fix] Pass through - Langfuse don't log request to Langfuse passthrough on Langfuse (#11768)
* dont log lf pass through request

* test_is_langfuse_route

* Update litellm/proxy/pass_through_endpoints/success_handler.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-06-16 17:30:23 -07:00
Alvaro SevillaandGitHub 196a7269f6 fix(vertex_ai): support global location in vertex ai passthrough (#11661)
* Support global location in vertex ai passthrough

* refactor
2025-06-13 09:51:04 -07:00
Cole McIntoshandGitHub e191e72746 Fix: Respect user_header_name property for budget selection and user identification (#11419)
* Refactor get_end_user_id_from_request_body to support user ID retrieval from custom headers and multiple request body formats. Enhance tests to cover various scenarios including header precedence and fallback mechanisms.

* Refactor get_end_user_id_from_request_body function to accept request_body as the first parameter, improving clarity and flexibility. Update tests for compatibility and add new cases to ensure correct functionality across various request body formats.

* Update _user_api_key_auth_builder and user_api_key_auth to pass request object to get_end_user_id_from_request_body, enhancing user ID retrieval from request data.

* refactor(auth_utils.py): update get_end_user_id_from_request_body to accept request_headers instead of request, and adjust related function calls in user_api_key_auth and tests

* refactor(tests): update mock request handling in LLM pass-through endpoint tests

- Replaced the Request object with a Mock for better flexibility in testing.
- Enhanced mock setup to include user API key handling and virtual key retrieval.
- Updated test calls to reflect changes in mock request structure and added necessary patches for new dependencies.

* refactor(vertex_and_google_ai_studio_gemini.py): remove redundant variable declaration for url_context_metadata, linting error
2025-06-06 14:21:02 -07:00
44a69421ea Anthropic - Files API w/ form-data support on passthrough + File ID support on /chat/completion (#11256)
* fix(anthropic/chat): support passing 'file_id' param to anthropic

Partial fix for LIT-200

* feat(anthropic/chat): use correct anthropic content block based on file object

* fix(anthropic/chat): fix file id for container_upload message type

* fix(anthropic/chat/transformation.py): fix check for adding code execution to tool calls - needed for 'container_upload' message type

* fix(llm_passthrough_endpoints.py): support reading form data for anthropic passthrough

* refactor(llm_passthrough_endpoints.py): refactor block into function for easier testing

* test: add unit test

* fix: don't pass in empty tools list

* [Fix] Responses API - Session management  (#11254)

* fix: import session handling

* fix: imports for session handler

* tests: tests for session handler

* Update enterprise/litellm_enterprise/enterprise_callbacks/session_handler.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* bump: bump litellm enterprise

* fixes: test_create_user_default_budget

* fix: fix linting error

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-05-29 23:54:24 -07:00
Krish DholakiaandGitHub ef42461c1e Litellm fix GitHub action testing (#11163)
* test: add __init__.py files

* refactor: rename test folder to avoid naming conflict

* test: update workflows

* test: update tests

* test: update imports

* test: update tests

* test: remove unused import

* ci(test-litellm.yml): add pytest retry to github workflow

* test: fix test
2025-05-26 14:41:42 -07:00