Commit Graph

31298 Commits

Author SHA1 Message Date
Ishaan Jaffer f07ef8af00 refactored code 2026-01-30 15:53:05 -08:00
Ishaan Jaffer 8b7a9250ce v0 - looks decen view 2026-01-30 15:53:05 -08:00
Alexsander Hamir a11b043f33 fix(proxy): resolve high CPU when router_settings in DB by avoiding REGISTRY.collect() in PrometheusServicesLogger (#20087) 2026-01-30 14:01:45 -08:00
Ishaan Jaff 6897d5f59e [Feat] Add async_post_call_response_headers_hook to CustomLogger (#20083)
* Add async_post_call_response_headers_hook to CustomLogger (#20070)

Allow CustomLogger callbacks to inject custom HTTP response headers
into streaming, non-streaming, and failure responses via a new
async_post_call_response_headers_hook method.

* async_post_call_response_headers_hook

---------

Co-authored-by: michelligabriele <gabriele.michelli@icloud.com>
2026-01-30 12:44:44 -08:00
Ishaan Jaff 974837c4e1 fix: working claude code with agent SDKs (#20081) 2026-01-30 11:58:05 -08:00
Harshit Jain 481bb4b6ce fixes: ci pipeline router coverage failure (#20065) 2026-01-30 11:55:49 -08:00
Ishaan Jaff 3910161a02 Realtime API benchmarks (#20074)
* Add /realtime API benchmarks to Benchmarks documentation

- Added new section showing performance improvements for /realtime endpoint
- Included before/after metrics showing 182× faster p99 latency
- Added test setup specifications and key optimizations
- Referenced from v1.80.5-stable release notes

Co-authored-by: ishaan <ishaan@berri.ai>

* Update /realtime benchmarks to show current performance only

- Removed before/after comparison, showing only current metrics
- Clarified that benchmarks are e2e latency against fake realtime endpoint
- Simplified table format for better readability

Co-authored-by: ishaan <ishaan@berri.ai>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: ishaan <ishaan@berri.ai>
2026-01-30 11:55:29 -08:00
Alexsander Hamir 0acfcb494b Add mock client factory pattern and mock support for PostHog, Helicone, and Braintrust integrations (#19707)
* Add LangSmith mock client support

- Create langsmith_mock_client.py following GCS and Langfuse patterns
- Add mock mode detection via LANGSMITH_MOCK environment variable
- Intercept LangSmith API calls via AsyncHTTPHandler.post patching
- Add verbose logging throughout mock implementation
- Update LangsmithLogger to initialize mock client when mock mode enabled
- Supports configurable mock latency via LANGSMITH_MOCK_LATENCY_MS

* Add Datadog mock client support

- Create datadog_mock_client.py following GCS, Langfuse, and LangSmith patterns
- Add mock mode detection via DATADOG_MOCK environment variable
- Intercept Datadog API calls via AsyncHTTPHandler.post and httpx.Client.post patching
- Add verbose logging throughout mock implementation
- Update DataDogLogger and DataDogLLMObsLogger to initialize mock client when mock mode enabled
- Supports both async and sync logging paths
- Supports configurable mock latency via DATADOG_MOCK_LATENCY_MS

* refactor: consolidate mock client logic into factory pattern

- Create mock_client_factory.py to centralize common mock HTTP client logic
- Refactor GCS, Langfuse, LangSmith, and Datadog mock clients to use factory
- Improve GET/DELETE mock accuracy for GCS (return valid StandardLoggingPayload)
- Fix DELETE mock to return empty body (204 No Content) instead of JSON
- Reduce code duplication across integration mock clients

* feat: add PostHog mock client support

- Create posthog_mock_client.py using factory pattern
- Integrate mock client into PostHogLogger with mock mode detection
- Add verbose logging for mock mode initialization and batch operations
- Enable mock mode via POSTHOG_MOCK environment variable

* Add Helicone mock client support

- Created helicone_mock_client.py using factory pattern (similar to GCS)
- Integrated mock mode detection and initialization in HeliconeLogger
- Mock client patches HTTPHandler.post to intercept Helicone API calls
- Uses factory pattern for should_use_mock and MockResponse utilities
- Custom HTTPHandler.post patching required since HTTPHandler uses self.client.send()

* Add mock support for Braintrust integration and extend mock client factory

- Add braintrust_mock_client.py with mock HTTP client for Braintrust integration testing
- Integrate mock client into BraintrustLogger with mock mode detection
- Refactor Helicone mock client to fully utilize factory's HTTPHandler.post patching
- Extend mock_client_factory to support patching HTTPHandler.post for sync calls
- Enable endpoint-specific mock responses for Braintrust (/project vs /project_logs)
- All mock clients now properly handle both async (AsyncHTTPHandler) and sync (HTTPHandler) calls

* Fix linter errors: remove unused imports and suppress complexity warning

- Remove unused imports from gcs_bucket_mock_client.py (httpx, json, timedelta, Dict, Optional)
- Remove unused Callable import from mock_client_factory.py
- Add noqa comment to suppress PLR0915 complexity warning for create_mock_client_factory function

* Document mock environment variables for PostHog, Helicone, Braintrust, Datadog, and Langsmith integrations

- Add POSTHOG_MOCK and POSTHOG_MOCK_LATENCY_MS documentation
- Add HELICONE_MOCK and HELICONE_MOCK_LATENCY_MS documentation
- Add BRAINTRUST_MOCK and BRAINTRUST_MOCK_LATENCY_MS documentation
- Add DATADOG_MOCK and DATADOG_MOCK_LATENCY_MS documentation
- Add LANGSMITH_MOCK and LANGSMITH_MOCK_LATENCY_MS documentation

All mock env vars follow the same pattern: enable mock mode for integration testing by intercepting API calls and returning mock responses without making actual network calls.

* Fix security issue
2026-01-30 09:52:53 -08:00
yuneng-jiang c301339412 Merge pull request #20039 from BerriAI/litellm_ui_key_model_bd
[Feature] UI - Usage: Model Breakdown Per Key
2026-01-30 09:16:23 -08:00
Sameer Kankute 6e9210381c Merge pull request #20058 from BerriAI/litellm_vertex_ai_prompt-caching-scope-2026-01-05,
Fix: remove unsupported prompt-caching-scope-2026-01-05 header for vertex ai
2026-01-30 20:32:00 +05:30
Sameer Kankute 8363a26d2e Fix: remove unsupported prompt-caching-scope-2026-01-05 header for vertex ai 2026-01-30 17:33:40 +05:30
Sameer Kankute 1877483deb Merge pull request #20056 from BerriAI/litellm_vllm_embedding
Fix: vllm embedding format
2026-01-30 17:01:57 +05:30
Sameer Kankute 5277a0cf82 Merge pull request #20053 from BerriAI/litellm_gemini_edit_jan_30
fix aspectRatio mapping in image edit
2026-01-30 17:01:46 +05:30
Sameer Kankute 8b9d5018b6 Merge pull request #20052 from BerriAI/litllm_disable_cache_jan_29
Add disable flag for anthropic gemini cache translation
2026-01-30 17:01:34 +05:30
Sameer Kankute 404c33eca6 Merge pull request #20051 from BerriAI/litellm_xai_web_search_routing
Add routing of xai chat completions to responses when web search options is present
2026-01-30 17:01:19 +05:30
Sameer Kankute 8d485f2403 Merge pull request #19986 from BerriAI/litellm_batch_cost_tracking_jan29
[Feat]Add cost tracking and usage object in aretrieve_batch call type
2026-01-30 17:00:42 +05:30
Sameer Kankute cfc9c77347 Merge pull request #19981 from BerriAI/litellm_fix_batches_permissions
Fix: Batch and File user level permissions
2026-01-30 17:00:10 +05:30
Sameer Kankute 51747ee534 Merge pull request #20055 from BerriAI/litellm_files_gemini_endpoints_support
feat: add /delete endpoint support for gemini
2026-01-30 16:58:59 +05:30
Sameer Kankute a8054264ae Merge pull request #19975 from BerriAI/litellm_oss_staging_01_29_2026
Litellm oss staging 01 29 2026
2026-01-30 16:58:28 +05:30
Sameer Kankute 55348dd9c5 Fix: vllm embedding format 2026-01-30 16:42:59 +05:30
Sameer Kankute fdb4b54add Merge pull request #20009 from genga6/fix/#20006-update-max-input-tokens-for-gpt-5.2-codex
Fix `max_input_tokens` for `gpt-5.2-codex`
2026-01-30 16:18:53 +05:30
Sameer Kankute ab8b459dfc Merge pull request #20046 from nht1206/bugfix/input-tokens-details-not-correctly-extracted
fix(ResponseAPILoggingUtils): extract input tokens details as dict
2026-01-30 16:13:35 +05:30
Sameer Kankute ed4c71ecf5 Merge pull request #20018 from varunsripad123/fix/google-files-retrieval
fix(gemini): support file retrieval in GoogleAIStudioFilesHandle
2026-01-30 16:09:19 +05:30
Sameer Kankute 78dd4b2b2f feat: add /delete endpoint support for gemini 2026-01-30 16:06:07 +05:30
Sameer Kankute 3c451e945a fix aspectRatio mapping 2026-01-30 15:25:18 +05:30
Sameer Kankute 5ac3f75996 Add disable flahg for anthropic gemini cache translation 2026-01-30 14:58:10 +05:30
Sameer Kankute 11c2160379 Add web search tests 2026-01-30 14:56:11 +05:30
Sameer Kankute dbc8006190 Add routing of xai chat completions to responses when web search options is present 2026-01-30 14:15:35 +05:30
Sameer Kankute b0b4362dd7 Merge pull request #19989 from BerriAI/litellm_add_passthrough_openai_route
Add /openai_passthrough route for openai passthrough requests:
2026-01-30 11:10:52 +05:30
Rhys e95b9b8015 fix(ResponseAPILoggingUtils): extract input tokens details as dict 2026-01-30 10:44:56 +07:00
Sameer Kankute eb50c780e9 Merge branch 'main' into litellm_oss_staging_01_29_2026 2026-01-30 09:03:05 +05:30
Sameer Kankute c509ffcea8 Merge pull request #20042 from BerriAI/main
merge main in passthrough
2026-01-30 08:44:00 +05:30
milan-berri 8fcdf6105f fix: run prisma generate as nobody user in non-root container (#20000)
Fixes permission error where prisma generate fails with 'Permission denied'
when trying to write schema.prisma in non-root containers.

The issue was that prisma generate was running as root before switching
to nobody user, causing generated files to be owned by root:root.
Moving prisma generate after USER nobody ensures files are owned by
nobody:nobody and can be written to during runtime.

Fixes #19859
2026-01-29 19:04:59 -08:00
yuneng-jiang 19b6c446a8 adding tests 2026-01-29 18:29:15 -08:00
yuneng-jiang aa75bec100 Model Usage per key 2026-01-29 18:27:10 -08:00
Ishaan Jaff c9658f877e [Docs] Claude Agents SDK x LiteLLM Guide (#20036)
* docs claude agent SDK

* docs fix

* docs

* docs
2026-01-29 18:04:54 -08:00
Ishaan Jaff 476f0b29d2 [Feat] LiteLLM x Claude Agent SDK Integration (#20035)
* fix: bedrock invoke - does not support prompt-caching-scope

* fix: UNSUPPORTED_BEDROCK_INVOKE_BETA_PATTERNS

* init requirements.txt

* init README for claude Agent SDK

* fix: using converse models with UNSUPPORTED_BEDROCK_CONVERSE_BETA_PATTERNS

* fix main.py

* init: proxy_e2e_anthropic_messages_tests
2026-01-29 17:48:38 -08:00
Ishaan Jaff f7e1a22947 [Feat] New Model - amazon.nova-2-pro-preview-20251202-v1:0 (#20033)
* init: amazon.nova-2-pro-preview-20251202-v1:0

* init: nova amazon.nova-2-pro

* add s3_vectors
2026-01-29 16:55:55 -08:00
Alexsander Hamir 12f58247ef Add event-driven coordination for global spend query to prevent cache stampede (#20030) 2026-01-29 16:07:20 -08:00
yuneng-jiang f782a2a8a1 Merge pull request #20024 from BerriAI/litellm_new_badge_dot
[Feature] UI - New Badge Dot Render
2026-01-29 14:25:17 -08:00
yuneng-jiang 5cd482cd05 Adding tests 2026-01-29 14:17:59 -08:00
yuneng-jiang ca8056f74f Merge remote-tracking branch 'origin' into litellm_new_badge_dot 2026-01-29 13:37:37 -08:00
yuneng-jiang 6b77060bbd Adjusting new badges 2026-01-29 13:36:38 -08:00
yuneng-jiang f3515b71eb Merge pull request #20017 from BerriAI/litellm_ui_spendlogs_setting_hook
[Feature] UI - Spend Logs: Show Current Store and Retention Status
2026-01-29 13:25:31 -08:00
yuneng-jiang 3c02abb47f Merge pull request #20015 from BerriAI/litellm_logs_error_code
[Fix] error_code in Spend Logs metadata
2026-01-29 13:24:54 -08:00
yuneng-jiang 96cb2efedb Adding proxy_server 2026-01-29 13:05:38 -08:00
yuneng-jiang e080f92b7f Adding tests 2026-01-29 13:05:07 -08:00
yuneng-jiang d081e01ed0 Show spend logs settings + allow delete of rentention period 2026-01-29 12:59:42 -08:00
yuneng-jiang 158e1e32d1 error_code in spend logs error metadata 2026-01-29 11:43:18 -08:00
yuneng-jiang 42081a57db Merge pull request #19886 from BerriAI/litellm_bulk_edit_keys
[Feature] Bulk Update Keys Endpoint
2026-01-29 09:07:58 -08:00