Commit Graph
22925 Commits
Author SHA1 Message Date
Bougou NisouandGitHub 58dda44fda feat: enhance redaction functionality for EmbeddingResponse (#12088) 2025-06-27 21:30:26 -07:00
Cole McIntoshandGitHub 6578133bb7 Fix user-team association issues in LiteLLM proxy (#12082)
* Fix user-team association issues in LiteLLM proxy

- Update list_team function to properly filter teams using user's teams array instead of only checking members_with_roles field
- Add Field descriptions and docstring to TeamMemberAddRequest and related models for better Swagger/OpenAPI documentation
- Maintain backward compatibility with fallback to members_with_roles if user lookup fails

This ensures users created with teams parameter appear correctly in team views and improves API documentation.

* Fix duplicate member checking in team_member_add endpoint

- Enhanced team_member_add_duplication_check to check both user_id and user_email
- Added additional duplicate prevention logic after user creation/lookup
- Fixed issue where users added by email could be duplicated in teams
- Added logging for debugging duplicate detection

This addresses the bug where adding the same user by email multiple times would create duplicate entries in the team's members_with_roles array.

* Improve duplicate member prevention in team_member_add

- Enhanced early duplicate check to handle both user_id and user_email
- Added late-stage duplicate prevention after user lookup/creation
- Fixed issue where users could be added multiple times by email
- Cleaned up debug logging

Note: There's still an edge case where the duplicate prevention may not work
correctly in all scenarios. This needs further investigation and testing.

* Refactor team_member_add endpoint for improved member management

- Split team_member_add functionality into smaller, dedicated functions for permission validation, member processing, and team member list updates.
- Enhanced permission checks to ensure only authorized users can add members.
- Streamlined member addition logic to reduce redundancy and improve readability.
- Maintained existing functionality while improving code structure and maintainability.

* Add tests for team_member_add helper functions

- Add test for _validate_team_member_add_permissions with admin user
- Add test for _validate_team_member_add_permissions with non-admin user
- Add test for _process_team_members with single member
- Add test for _process_team_members with multiple members
- Add test for _update_team_members_list with new member
- Add test for _update_team_members_list duplicate prevention

These tests ensure the refactored helper functions work correctly
after fixing the PLR0915 linting error.
2025-06-27 21:29:49 -07:00
Youfu ZhangandGitHub 1c68c24358 introduce new environment variable NO_REDOC to opt-out Redoc (#12092)
Signed-off-by: Youfu Zhang <zhangyoufu@gmail.com>
2025-06-27 21:26:37 -07:00
Cole McIntoshandGitHub e91802da39 feat: add local LLM translation testing with artifact generation (#12120)
- Move from CircleCI dependency to direct pytest execution
- Add Python script to generate beautiful markdown reports
- Update GitHub workflow to run tests directly
- Update Makefile to use the new test runner script
- Generate both JUnit XML and markdown artifacts
- Group test results by provider with detailed statistics
2025-06-27 21:24:19 -07:00
Ishaan Jaff 7465f062c0 add panw guard docs 2025-06-27 21:13:04 -07:00
Ishaan Jaff bb98f68451 docs fix 2025-06-27 21:09:33 -07:00
Ishaan Jaff 1fc590c0d1 fix - anthropic messages exception 2025-06-27 21:06:15 -07:00
Krish DholakiaandGitHub ee6e76e1f9 Bedrock Passthrough cost tracking (/invoke + /converse routes - streaming + non-streaming) (#12123)
* refactor(passthrough_endpoints-success-handler): refactor llm passthrough logging logic

isolate the llm translation work to enable cost tracking on sdk

* feat: initial implementation of passthrough SDK cost calculation

enables bedrock passthrough cost tracking to work

* feat(cost_calculator.py): working cost calculation for bedrock passthrough

* feat(litellm_logging.py): consider allm_passthrough in cost tracking

allows async calls (e.g. via proxy) to work

* feat(bedrock/passthrough): working event stream decoding for bedrock passthrough calls + logging instrumentation for passthrough sdk calls (log on stream completion)

Enables bedrock streaming cost calculation

* feat(litellm_logging.py): support streaming passthrough cost tracking

* feat(passthrough/main.py): working async streaming cost calculation

Closes https://github.com/BerriAI/litellm/issues/11359

* feat(proxy_server.py): fix passthrough routing when llm router enabled

* feat: further fixes

* feat(bedrock/): working bedrock passthrough cost tracking (non-streaming)

* feat(litellm_logging.py): working usage tracking for bedrock passthrough calls

ensures tokens are logged

* feat(bedrock/passthrough): add converse passthrough cost tracking support

* feat(base_llm/passthrough): remove redundant function

* refactor(litellm_logging.py): refactor function to be below 50 LOC

* test: update test

* test: remove redundant test
2025-06-27 20:01:12 -07:00
tanjiroandGitHub db2a55a3d2 Add "Get Code" Feature (#11629)
* added get code

* modify template

* fixed code template

* put the codesnippets in a different file

* minor
2025-06-27 18:18:56 -07:00
Alessandro MarrellaandGitHub 0e10ae1cf1 change Message init type annotation to support other roles (#11942) 2025-06-27 18:12:13 -07:00
ohmeowandGitHub 9ab7d97b85 adds model to metadata to estimated cost is calculated; corrects how to pass metadata in braintrust in the docs (#12022) 2025-06-27 18:11:07 -07:00
Ishaan Jaff 723f24ef17 ci/cd new release 2025-06-27 18:07:44 -07:00
Ishaan Jaff 080372399f test_mock_router_testing_params_str_to_bool_conversion 2025-06-27 18:07:33 -07:00
Ishaan Jaff 0db60009b5 ci/cd new release 2025-06-27 18:06:06 -07:00
Ishaan Jaff d8dc84ee96 test fix - router utils 2025-06-27 18:05:37 -07:00
Ishaan Jaff c76f9be729 ui - new build 2025-06-27 18:00:59 -07:00
Ishaan JaffandGitHub 01b9e6efb1 Revert "fix: set logger levels based on LITELLM_LOG environment variable" (#12122) 2025-06-27 17:55:44 -07:00
Ishaan JaffandGitHub ebf6395bc1 [Feat] Add Eleven Labs - Speech To Text Support on LiteLLM (#12119)
* add ELEVENLABS as a provider

* add deepgram to main.py

* add ElevenLabsException

* add ElevenLabsAudioTranscriptionConfig

* add transform_audio_transcription_response

* TestElevenLabsAudioTranscription

* add elevenlabs/scribe_v1 to model cost map

* add ElevenLabsAudioTranscriptionConfig

* add AudioTranscriptionRequestData

* add ElevenLabs transform

* use AudioTranscriptionRequestData

* refactoring fixes

* add ProcessedAudioFile util for reading audio files

* test_elevenlabs_diarize_parameter_passthrough

* docs eleven labs

* docs fixes

* fix code qa checks

* fixes - audio transcription

* ui - add ElevenLabs logo

* add elevenlabs logo

* docs - ElevenLabs

* test fix elevenlabs
2025-06-27 17:50:49 -07:00
041db0268c [Bug fix] Router - handle cooldown_time = 0 for deployments (#12108)
* fix get cooldown time

* fixes for _should_run_cooldown_logic

* test_cooldown_time_zero_uses_zero_not_default

* Update litellm/router_utils/cooldown_cache.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update litellm/router_utils/cooldown_handlers.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-06-27 17:50:35 -07:00
Cole McIntoshandGitHub cdb5c6fb6b Merge pull request #12035 from colesmcintosh/add-gemini-md
docs(GEMINI.md): add development guidelines and architecture overview
2025-06-27 17:30:52 -06:00
Ishaan JaffandGitHub f04808e293 [Bug Fix] Exception mapping for context window exceeded - should catch anthropic exceptions (#12113)
* fix is is_error_str_context_window_exceeded

* test_is_error_str_context_window_exceeded

* fix is_error_str_context_window_exceeded
2025-06-27 15:49:07 -07:00
Jason RobertsandGitHub b57cfb8bff Adding Feature: Palo Alto Networks Prisma AIRS Guardrail (#12116)
* feat: Add PANW Prisma AIRS guardrail integration

- Add PANW_PRISMA_AIRS to SupportedGuardrailIntegrations enum
- Update guardrail registry and initializers
- Add complete documentation with curl examples and response formats
- Support pre_call, post_call, and during_call modes
- Include fail-safe error handling and comprehensive logging
- Integration with official PANW Prisma AIRS API

* feat: Add PANW Prisma AIRS guardrail integration

- Update to test file
- Fail-closed security on API errors

* update fail closed behavior

* fix: Update PANW Prisma AIRS guardrail integration pattern

* fix: Remove unused import

* fix: addressed MyPy error
2025-06-27 15:17:36 -07:00
Ishaan JaffandGitHub a42a058b0c [Bug Fix] Fix handling str, bool types formock_testing_fallbacks on router using /audio endpoints (#12117)
* use a dataclass for managing mock params

* fixes for mock testing
2025-06-27 15:11:09 -07:00
Ishaan JaffandGitHub f85cd9c54a [Docs] - Show how to use fallbacks with audio transcriptions endpoints (#12115)
* add overview

* docs audio transcriptions

* docs on fallbacks with audio endpoints
2025-06-27 12:43:55 -07:00
Ishaan Jaff 7a5e7f454d bump: version 1.73.3 → 1.73.4 2025-06-27 11:10:14 -07:00
Ishaan JaffandGitHub f1c7024e70 [Feat] Add Bridge from generateContent <> /chat/completions (#12081)
* add GenerateContentToCompletionHandler

* working - non streaming bridge

* add GoogleGenAIAdapter

* add google gen ai adapter

* working streaming bridge

* working streaming usage for adapter

* tool calling transform for generate content to openai

* fixes for accumulating tool calls

* fix code qa checks

* Best Practices for Production

* fix code qa checks

* test_streaming_partial_tool_calls_accumulation

* linting fixes

* add supported_openai_chat_completion_params

* fix translate_generate_content_to_completion

* test_google_genai_adapter.py
2025-06-27 11:08:55 -07:00
Cole McIntoshandGitHub 575d25ab16 Merge pull request #12111 from colesmcintosh/fix/logger-level-initialization
fix: set logger levels based on LITELLM_LOG environment variable
2025-06-27 11:58:05 -06:00
Cole McIntoshandGitHub cce2656376 Merge pull request #12112 from colesmcintosh/fix/cost-calculation-log-level
fix: change cost calculation logs from INFO to DEBUG level
2025-06-27 11:57:55 -06:00
Cole McIntosh bc41cee0a1 fix: change cost calculation logs from INFO to DEBUG level
- Change verbose_logger.info() to verbose_logger.debug() for cost calculation messages
- Ensures cost calculation details are only shown when DEBUG logging is enabled
- Reduces log noise in production environments

Part 2/2 of fix for #9815
2025-06-27 11:36:58 -06:00
Cole McIntosh 9d58d872ac fix: set logger levels based on LITELLM_LOG environment variable
- Initialize verbose_logger, verbose_proxy_logger, and verbose_router_logger with the correct log level
- Ensures loggers respect the LITELLM_LOG environment variable setting
- Fixes part of issue #9815 where logs were shown regardless of log level setting

Part 1/2 of fix for #9815
2025-06-27 11:34:38 -06:00
KishanandGitHub fc17da0aef [Bug Fix] Anthropic - Token Usage Null Handling in calculate_usage (#12068)
* [Bug Fix] Anthropic - Token Usage Null Handling in calculate_usage (BerriAI/litellm#11920)

* [Fix] Missed a null check and used a cast instead by error
2025-06-27 10:00:23 -07:00
Krish DholakiaandGitHub 8bd1f8f6ab Add o3 and o4-mini deep research models (#12109)
* build(model_prices_and_context_window.json): add o3-deep-research models

* build(model_prices_and_context_window.json): add o4-deep-research model

* build(model_prices_and_context_window.json): add o4-mini-deep-research versioned model
2025-06-27 09:58:53 -07:00
Cole McIntoshandGitHub a56c28ab2f fix: handle provider_config type error in passthrough error handler (#12101)
- Fix type error where _handle_error was called with provider_config=None
- Add proper provider config retrieval logic similar to llm_passthrough_route
- Handle cases where provider config may not be available
- Ensure type safety while maintaining backward compatibility
2025-06-27 09:11:28 -07:00
Ishaan JaffandGitHub e3c9318bd4 [Fix] MCP - Ensure internal users can access /mcp and /mcp/ routes (#12106)
* fixes for MCP route checks

* test is is_llm_api_route for MCP
2025-06-27 09:05:38 -07:00
Cole McIntoshandGitHub 60bf3c36bf fix(docs): Remove unused dotenv dependency from docusaurus config (#12102)
* fix(docs): Remove unused dotenv dependency from docusaurus config

The dotenv package was being required in docusaurus.config.js but was listed as
a devDependency, causing build failures. Since no environment variables are
actually used in the config, removed the unnecessary import.

* fix(docs): Remove reference to non-existent spending_monitoring doc

The sidebars.js file was referencing proxy/spending_monitoring which was deleted
in commit ba7463b9c. This was causing the documentation build to fail with missing
document errors.
2025-06-27 08:20:44 -07:00
Ishaan JaffandGitHub c994c22c13 [Bug Fix] Bedrock Guardrails - Ensure PII Masking is applied on response streaming or non streaming content when using post call (#12086)
* test_bedrock_guardrail_response_pii_masking_streaming

* fix - apply masking for response on bedrock
2025-06-27 07:41:32 -07:00
Davis FeatherstoneandGitHub 51074ffcae Fix Azure-OpenAI Vision API Compliance (#12075)
* Fix Azure-OpenAI Vision API Compliance

* Linting Fix
2025-06-26 22:54:15 -07:00
Krish DholakiaandGitHub dd92b1a0ac Refactor: bedrock passthrough fixes - migrate to Passthrough SDK (#12089)
* feat: initial commit adding bedrock support via the new sdk passthrough logic

ensures correct sequencing of tasks (pre call checks etc. can run before signing request)

* fix(route_llm_requests.py): passthrough to allm_passthrough_route if no model found

* feat(bedrock/passthrough): working bedrock passthrough via sdk support

* fix(passthrough/main.py): re-add data and json

* feat(passthrough/main): support async passthrough calls to bedrock

* feat(passthrough/main.py): async streaming + completion support

* feat(llm_passthrough_endpoints.py): migrate bedrock passthrough calls to to new bedrock passthrough sdk

Enables calls to work correctly

* fix: fix linting errors

* test: update test
2025-06-26 22:51:35 -07:00
Krish DholakiaandGitHub e8d7537b57 Raise clearer error on Anthropic Unified route + allow setting new_key on /key/regenerate
* fix(llm_http_handler.py): raise clearer error on anthropic unified route

Fixes https://github.com/BerriAI/litellm/issues/12063

* fix(key_management_endpoints.py): add new param `new_key` for setting the regenerated key value

user request

* test: add unit tests

* fix(pass_through_endpoints.py): use data instead of json for passthrough requests

fixes bedrock latency issue

* Revert "fix(pass_through_endpoints.py): use data instead of json for passthrough requests"

This reverts commit 021dfd9165f837d37e6aad247ccbd0d2e8ca6043.
2025-06-26 21:56:03 -07:00
Ishaan Jaff ba7463b9c6 docs - spend monitoring 2025-06-26 17:39:13 -07:00
Ishaan Jaff edf416c24f docs - Use LiteLLM with Gemini CLI 2025-06-26 17:31:19 -07:00
Ishaan Jaff 72ecac8962 docs - user agent 2025-06-26 16:23:59 -07:00
Ishaan Jaff 16126f0d6b docs Gemini Text-to-Speech 2025-06-26 15:59:48 -07:00
Ishaan Jaff 80acf6bc97 docs */speech to /chat/completions Bridge** 2025-06-26 15:53:25 -07:00
Ishaan Jaff fc39343599 docs update 2025-06-26 15:32:33 -07:00
Ryan CastnerandGitHub ac11bfabcb Responses API - Add reasoning content support for non-OpenAI providers (#12055)
Add reasoning content support when Responses API falls
back to completions API, enabling reasoning content for
all LLM providers (Anthropic, Vertex AI, etc.) since
OpenAI is currently the only native Responses API
provider.

* Add ReasoningSummaryTextDeltaEvent for streaming
  reasoning deltas
* Update streaming iterator to detect and transform
  reasoning content
* Extract reasoning content as separate output items in
  transformations
* Support reasoning content alongside regular message
  content

Closes https://github.com/BerriAI/litellm/issues/11302
2025-06-26 14:21:49 -07:00
Cole McIntoshandGitHub 98ba5c8ffe Merge pull request #12042 from colesmcintosh/fix-today-selector-date-mutation-bug
Fix today selector date mutation bug in dashboard components
2025-06-26 14:43:02 -06:00
Ishaan Jaff e556071544 docs - Use LiteLLM with Gemini CLI 2025-06-26 13:35:33 -07:00
Cole McIntosh 8f376a60fa fix(ui): fix Today filter not showing usage data in dashboard
- Fix date formatting in API calls from ISO format to YYYY-MM-DD
- Update userDailyActivityCall, teamDailyActivityCall, and tagDailyActivityCall
- Prevent date mutation by creating new Date objects before API calls
- Set proper time boundaries (00:00:00 to 23:59:59) for same-day selections

The API expects dates in YYYY-MM-DD format but the UI was sending
full ISO timestamps, causing the Today filter to return empty results.
2025-06-26 13:59:40 -06:00
Cole McIntoshandGitHub 92948b33b1 Merge branch 'BerriAI:main' into fix-today-selector-date-mutation-bug 2025-06-26 12:29:44 -06:00