- Fixed 'missing finish_reason for choice 1' error with reasoning_effort
- Anthropic sends multiple content blocks with different indices
- OpenAI expects all content in a single choice at index=0
- Added comprehensive tests for text-only, text+tool, and multiple tools
* Fix security vulnerability in list_team_v2 endpoint
- Add missing allowed_route_check_inside_route security check to list_team_v2
- Add @management_endpoint_wrapper decorator for consistency with list_team
- Add comprehensive tests to verify security checks work correctly
- Ensure non-admin users can only query their own teams
- Ensure admin users can query all teams
This fixes a security bug where non-admin users could potentially access
team information they shouldn't have access to through the list_team_v2
endpoint, which was missing the authorization check present in list_team.
* Fix test
* Test fixes
* Fixed test
* Restored invalid delete
* Revert
---------
Co-authored-by: openhands <openhands@all-hands.dev>
* fix(azure/chat/gpt_transformation.py): support api_version="preview"
Fixes https://github.com/BerriAI/litellm/issues/12945
* Fix anthropic passthrough logging handler model fallback for streaming requests (#13022)
* fix: anthropic passthrough logging handler model fallback for streaming requests
- Add fallback logic to retrieve model from logging_obj.model_call_details when request_body.model is empty
- Fixes issue #12933 where streaming requests to anthropic passthrough endpoints would crash due to missing model field
- Ensures downstream logging and cost calculation work correctly for all streaming scenarios
- Maintains backwards compatibility with existing non-streaming requests
* test: add minimal tests for anthropic passthrough logging handler model fallback
- Add unit tests for the model fallback logic in _handle_logging_anthropic_collected_chunks
- Test existing behavior when request_body.model is present
- Test fallback logic when request_body.model is empty but logging_obj.model_call_details has model
- Test edge cases where both sources are empty or missing
- Ensure backwards compatibility and graceful degradation
* fix(anthropic_passthrough_logging_handler.py): add provider to model name (accurate cost tracking)
* fix(anthropic_passthrough_logging_handler.py): don't reset custom llm provider, if already set
* fix: fix check
---------
Co-authored-by: Haggai Shachar <haggai.shachar@backline.ai>
* feat(langfuse-otel): Add comprehensive metadata support to Langfuse OpenTelemetry integration
This commit brings the langfuse_otel integration to feature parity with the vanilla Langfuse integration by adding support for all metadata fields.
Changes:
- Extended LangfuseSpanAttributes enum with all supported metadata fields:
- Generation-level: generation_name, generation_id, parent_observation_id, version, mask_input/output
- Trace-level: trace_user_id, session_id, tags, trace_name, trace_id, trace_metadata, trace_version, trace_release, existing_trace_id, update_trace_keys
- Debug: debug_langfuse
- Implemented metadata extraction and mapping in langfuse_otel.py:
- Added _extract_langfuse_metadata() helper to extract metadata from kwargs
- Support for header-based metadata (langfuse_* headers) via proxy
- Enhanced _set_langfuse_specific_attributes() to map all metadata to OTEL attributes
- JSON serialization for complex types (lists, dicts) for OTEL compatibility
- Updated documentation:
- Added 'Metadata Support' section explaining all fields are now supported
- Provided usage example showing how to pass metadata
- Clarified that traces are viewed in Langfuse UI (not generic OTEL backends)
- Added opentelemetry-exporter-otlp to required dependencies
This allows users to pass metadata like:
metadata={
'generation_name': 'my-generation',
'trace_id': 'trace-123',
'session_id': 'session-456',
'tags': ['prod', 'v1'],
'trace_metadata': {'user_type': 'premium'}
}
All metadata is exported as OpenTelemetry span attributes with 'langfuse.*' prefix for easy filtering and analysis in the Langfuse UI.
* Fix ruff linting error
* test(langfuse-otel): Fix failing test and add comprehensive metadata tests
- Fix test_set_langfuse_environment_attribute to use positional arguments
instead of keyword arguments when asserting safe_set_attribute calls
- Add test_extract_langfuse_metadata_basic to verify metadata extraction
from litellm_params
- Add test_extract_langfuse_metadata_with_header_enrichment to test
integration with header-based metadata using a stubbed LangFuseLogger
- Add test_set_langfuse_specific_attributes_full_mapping to comprehensively
test all metadata field mappings and JSON serialization of complex types
These tests ensure full coverage of the langfuse_otel metadata features
added in commit ab1dbe355 and fix the CI test failure.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
* fix: correct CompletionRequest messages type to match OpenAI API spec
- Changed messages field type from List[str] to List[ChatCompletionMessageParam]
- This ensures proper OpenAI API compatibility where messages should be objects with role and content fields
- Fixes type inconsistency in completion request handling
* feat(tests): Add comprehensive tests for CompletionRequest model
- Add test_completion.py for litellm.types.completion module
- Test ChatCompletionMessageParam type validation
- Test tool message format compatibility
- Test function message format (deprecated)
- Test multimodal content (text + image)
- Test default empty messages list
- Test all optional parameters
- Validate OpenAI ChatCompletion API message format compatibility
- Implement X-Initiator header logic in GithubCopilotConfig.validate_environment()
- Set header to "agent" when messages contain agent or tool roles, "user" otherwise
- Reduces unnecessary premium Copilot API usage for non-user calls
Fixes#12859
* feat(bulk_user_update/): support updating all users on proxy
* fix(bulk_edit_user.tsx): persist user settings when 'add to team' clicked
* fix(team_endpoints.py): bulk add all proxy users to team
supports flow from UI to add all existing users to a team
* fix: minor fixes
* feat(user_edit_view.tsx): support setting no default model on user edit
allows preventing users from calling models outside team scope
* fix(user_edit_view.tsx): prevent triggering submit when 'cancel' is clicked
* refactor(internal_user_endpoints.py): refactor to reduce function size
* fix(ui_sso.py): fix form action on login when sso is enabled
* fix: multiple fixes - fix resetting env var in proxy config + add key to exception message on key decryption
fixes issue where env vars would be reset
* refactor(proxy_server.py): cleanup redundant decryption line
* fix(proxy_setting_endpoints.py): show saved ui access mode
allows admin to know what they'd previously stored in db
* feat(proxy_server.py): working guardrails on streaming output
ensures guardrail actually raises an error if flagged during streaming output
* test: add unit tests
* feat(advanced_settings.tsx): add guardrails option as ui component on model add
enables setting guardrails on model add
* feat(add_model_tab.tsx): fix add model form
* feat(model_info_view.tsx): support adding guardrails on model update
* fix(add_model_tab.tsx/): working health check when guardrails selected
* fix(proxy_server.py): fix yield
* fix(custom_guardrail.py): initial logic for model level guardrails
* feat(custom_guardrail.py): working pre call guardrails
* fix(custom_guardrails.py): check if custom guardrails set before running event hook
* test(test_custom_guardrail.py): add unit tests for async pre call deployment hook on custom guardrail
* feat(custom_guardrail.py): add post call processing support for guardrails
allows model based guardrails to run on the post call event for that model only
* fix(utils.py): only run if call type is in enum
* test: update unit tests to work
If the user specified in the configuration e.g. "user_header_name:
X-OpenWebUI-User-Email", here we were looking for a dict key
"X-OpenWebUI-User-Email" when the dict actually contained
"x-openwebui-user-email".
Switch to iteration and case insensitive string comparison instead to
fix this.
This fixes customer budget enforcement when the customer ID is passed
in as a header rather than as a "user" value in the body.
* EditAutoRouterTabProps
* Revert "EditAutoRouterTabProps"
This reverts commit 2835d3a3743e6411b9914a0b01381050e2273ad7.
* add EditAutoRouterTab
* delete edit
* fixes for edit auto-router
* fix accessing model edit
* working edit auto router
* fix - edit remove custom model name
* fixes for edit auto router settings
* qa for adding a model router
* test fix
* feat(key_management_endpoints.py): Support new 'key_type' field
allow user to specify if key should be 'management' or 'llm api' key
Security fix
* test(test_route_checks.py): add unit tests
* fix(create_key_button.tsx): add ui component to select key type
allows specifying if key can call llm api vs. management routes
* feat(create_key_button.tsx): add specifying key type to ui
* fix(route_checks.py): add sensitive data masker for user id on not allowed error message
prevent leaking sensitive information
* feat: Add Pillar Security guardrail integration
Implements comprehensive LLM security guardrails using Pillar Security API with support for prompt injection detection, PII/secret detection, content moderation, and multi-mode execution (pre_call, during_call, post_call). Includes complete documentation, testing, and configurable actions on flagged content.
* fix: Resolve MyPy type error in Pillar guardrail config
Restructure PillarGuardrailConfigModel to properly inherit from GuardrailConfigModel[T]
and resolve return type compatibility issue in get_config_model method.
* fix: Resolve MyPy type error in Pillar guardrail config
Restructure PillarGuardrailConfigModel to properly inherit from GuardrailConfigModel[T]
and resolve return type compatibility issue in get_config_model method.
* fix docs
* fix docs
* improved docs
* fix examples, READY
* feat(litellm_pre_call_utils.py): add num_retries to litellm data for backend call
allow user to pass in num retries via request headers
* test(test_litellm_pre_call_utils.py): add unit test
* docs(request_headers.md): document new request header
* fix(common_daily_activity.py): show spend breakdown by model group
Partial fix for https://github.com/BerriAI/litellm/issues/12887
* feat(new_usage.tsx): new tab switcher for viewing usage by model group vs. received model
Closes https://github.com/BerriAI/litellm/issues/12887
