Addresses 4 critical OpenTelemetry span issues in LiteLLM:
Issue #3: Remove redundant attributes from raw_gen_ai_request spans
- Removed self.set_attributes() call that was duplicating all parent span
attributes (gen_ai.*, metadata.*) onto the raw span
- Raw span now only contains provider-specific llm.{provider}.* attributes
- Reduces storage and eliminates search confusion from duplicate data
Issue #4: Prevent attribute duplication on litellm_proxy_request parent span
- When litellm_request child span exists, removed redundant
set_attributes() call on the parent proxy span
- Child span already carries all attributes; parent duplication doubles
storage and complicates search
Issue #5: Fix orphaned guardrail traces
- Guardrail spans were created with context=None when no parent proxy span
existed, resulting in orphaned root spans (separate trace_id)
- Added _resolve_guardrail_context() helper to ensure guardrails always
have a valid parent (litellm_request or proxy span)
- Applied fix to both _handle_success and _handle_failure paths
Issue #8: Add gen_ai.response.id for embeddings and image generation
- EmbeddingResponse and ImageResponse types don't have provider response IDs
- Added fallback to standard_logging_payload["id"] (litellm call ID) for
correlation across LiteLLM UI, Phoenix traces, and provider logs
- Completions still use provider ID (e.g. "chatcmpl-xxx") when available
Tests added:
- TestRawSpanAttributeIsolation: Verify raw span has no gen_ai/metadata attrs
- TestNoParentSpanDuplication: Verify parent span doesn't get duplicated attrs
- TestGuardrailSpanParenting: Verify guardrails are children (not orphaned)
- TestResponseIdFallback: Verify response ID set for all call types
All existing OTEL tests pass (73 passed, 14 pre-existing protocol failures).
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Same bug as team budget: _assemble_user_object fetched user info from DB
but only used budget_reset_at, discarding max_budget. When the key cache
has a stale None for user_max_budget, _safe_get_remaining_budget returns
+Inf. Now falls back to DB max_budget when metadata value is None.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- ArizePhoenixLogger now creates spans on its own dedicated TracerProvider
instead of trying to reuse parent spans from the global otel TracerProvider
(which were invisible in Phoenix since they go to a different exporter)
- Auto-initialize ArizePhoenixLogger when otel callback is configured and
Phoenix env vars (PHOENIX_API_KEY, PHOENIX_COLLECTOR_*) are detected
- Use exact type check in get_custom_logger_compatible_class to prevent
ArizePhoenixLogger (subclass) from being returned when looking up otel
- Fix tool_permission guardrail to check non-function tools like
code_interpreter (previously skipped with `type != "function"`)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Set prometheus_emit_stream_label: true in litellm_settings to emit a
stream label (True/False/None) on litellm_proxy_total_requests_metric.
Opt-in to avoid breaking cardinality on existing deployments.
* fix: feat: add litellm_system_prompt support
* feat: support new 'litellm_agent' model provider
* feat: ui/ - new agent builder ui
* fix(anthropic/chat/transformation.py): normalize max_tokens if decimal
* feat(agentbuilderview.tsx): run compliance datasets against litellm agent
Adds per-alert-type digest mode that aggregates duplicate alerts
within a configurable time window and emits a single summary message
with count, start/end timestamps.
Configuration via general_settings.alert_type_config:
alert_type_config:
llm_requests_hanging:
digest: true
digest_interval: 86400
Digest key: (alert_type, request_model, api_base)
Default interval: 24 hours
Window type: fixed interval
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When extended thinking is enabled, the websearch interception agentic loop
builds a follow-up assistant message with only tool_use blocks. Anthropic's
API requires assistant messages to start with thinking/redacted_thinking
blocks when thinking is enabled, causing a 400 Bad Request.
Extract thinking blocks from the model's initial response, thread them
through the agentic loop, and prepend them to the follow-up assistant
message — matching the pattern used by anthropic_messages_pt in factory.py.
Fixes the error: "Expected 'thinking' or 'redacted_thinking', but found
'tool_use'"
Save and restore litellm.initialized_langfuse_clients around
test_max_langfuse_clients_limit to prevent ordering-dependent failures
in other tests that rely on the counter's value.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- test_litellm_pre_call_utils.py: wrap test body in try/finally so
litellm.callbacks is always restored even when an assertion fails,
addressing greptile review comment
- test_langfuse_otel.py: resolve trivial merge conflict in comment
("unpatched" vs "unpatch-ed"), keeping correct spelling
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: guard against None metadata in prometheus metrics
Use get_litellm_metadata_from_kwargs and get_metadata_variable_name_from_kwargs
helpers to properly resolve metadata from both 'metadata' and 'litellm_metadata'
keys, with None safety.
* test: add test for None metadata in prometheus metrics
Three test failures caused by the real langfuse SDK import being triggered
at test time:
1. test_langfuse_prompt_management.py: Both tests create LangfusePromptManagement()
which calls `import langfuse`. Since earlier TestLangfuseUsageDetails tests
remove sys.modules["langfuse"] via patch.dict teardown, the real langfuse
import runs and fails on Python 3.14 (pydantic v1 incompatibility).
Fix: add setup_method/teardown_method to mock sys.modules["langfuse"].
2. test_langfuse.py::test_max_langfuse_clients_limit: Same root cause — creates
LangFuseLogger() without mocking sys.modules["langfuse"].
Fix: wrap test body with patch.dict("sys.modules", {"langfuse": mock}).
3. test_langfuse_otel.py::test_extract_langfuse_metadata_with_header_enrichment:
Replaces sys.modules["litellm.integrations.langfuse.langfuse"] with a stub
without restoring it, causing patch() in later tests to target the stub
instead of the real module.
Fix: use monkeypatch.setitem() which auto-restores after the test.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Three test failures caused by the real langfuse SDK import being triggered
at test time:
1. test_langfuse_prompt_management.py: Both tests create LangfusePromptManagement()
which calls `import langfuse`. Since earlier TestLangfuseUsageDetails tests
remove sys.modules["langfuse"] via patch.dict teardown, the real langfuse
import runs and fails on Python 3.14 (pydantic v1 incompatibility).
Fix: add setup_method/teardown_method to mock sys.modules["langfuse"].
2. test_langfuse.py::test_max_langfuse_clients_limit: Same root cause — creates
LangFuseLogger() without mocking sys.modules["langfuse"].
Fix: wrap test body with patch.dict("sys.modules", {"langfuse": mock}).
3. test_langfuse_otel.py::test_extract_langfuse_metadata_with_header_enrichment:
Replaces sys.modules["litellm.integrations.langfuse.langfuse"] with a stub
without restoring it, causing patch() in later tests to target the stub
instead of the real module.
Fix: use monkeypatch.setitem() which auto-restores after the test.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
test_extract_langfuse_metadata_with_header_enrichment replaced
sys.modules["litellm.integrations.langfuse.langfuse"] with a stub
module but never restored it. This caused subsequent tests using
patch("litellm.integrations.langfuse.langfuse._add_prompt_to_generation_params")
to patch the stub instead of the real module, while _log_langfuse_v2
executed from the real module's globals (unpatched), triggering
ModuleNotFoundError and assertion failures.
Fix: use monkeypatch.setitem() so pytest automatically restores the
original module after the test completes.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This test has failed repeatedly in CI with:
'Expected _add_prompt_to_generation_params to have been called once. Called 0 times.'
Root cause: _add_prompt_to_generation_params is only called when _supports_prompt()
returns True. Under cross-test state contamination in CI (parallel workers),
langfuse_sdk_version can be in an unexpected state, causing _supports_prompt() to
return False and silently skip the call (exception swallowed by the outer try/except).
Fixes:
- Use reset_mock(side_effect=True) so setUp's trace side_effect is cleared and the
explicit return_value assignment actually takes effect
- Patch _supports_prompt on the logger instance to always return True, making the
_add_prompt_to_generation_params assertion independent of SDK version state
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Add 6 new EU PII patterns for GDPR compliance
- fr_nir: French Social Security Number (NIR/INSEE) with validation
- eu_iban_enhanced: Enhanced IBAN detection with specific format
- fr_phone: French phone numbers (+33, 0033, 0 formats)
- eu_vat: EU VAT identification numbers (all 27 member states)
- eu_passport_generic: Generic EU passport format
- fr_postal_code: French postal codes with contextual keywords
* Add GDPR Art. 32 EU PII Protection policy template
- Comprehensive GDPR Article 32 compliance policy
- 4 guardrail groups: National IDs, Financial, Contact Info, Business IDs
- Masks French NIR/INSEE, EU IBANs, French phones, EU VAT numbers
- Includes EU passport numbers and email addresses
- Medium complexity template with indigo icon
* Add comprehensive tests for EU PII patterns
- Test French NIR validation (sex digit, month range)
- Test enhanced IBAN detection (French, German)
- Test French phone number formats
- Test EU VAT numbers
- Test generic EU passport format
- Test French postal code pattern
* Add EU pattern loading and category validation tests
- Verify all 6 EU PII patterns are loaded correctly
- Verify patterns are categorized as 'EU PII Patterns'
- Ensure pattern loading consistency
* Add end-to-end tests for GDPR policy template
- 4 tests for PII that should be masked (NIR, IBAN, phone, VAT)
- 4 tests for text that should pass through (invalid patterns, no PII)
- 1 bonus test for multiple PII types in same message
- All tests verify correct masking behavior
* Add region field to policy templates
- Added region field to all 6 templates (EU, AU, Global)
- Updated both main and backup JSON files
- Enables region-based filtering in UI
* Add region filter to policy templates UI
- Added Radio.Group filter for regions (All, AU, EU, Global)
- Efficient filtering with useMemo hooks
- Clean button-based UI matching existing design
- Defaults missing regions to Global
* Apply suggestion from @greptile-apps[bot]
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Address Greptile review: add contextual guards and negative tests
- Added keyword_pattern to eu_vat (VAT, tax number, fiscal code, etc.)
- Added keyword_pattern to eu_passport_generic (passport, travel document, etc.)
- Added 3 negative unit tests for false positive prevention
- Added 2 E2E tests verifying no masking without keyword context
- All patterns now require contextual keywords to prevent false positives
* address greptile review feedback (greploop iteration 1)
- Remove unused HTTPException import from test file
- Add keyword_pattern to eu_vat for contextual VAT matching
- Add allow_word_numbers: false to eu_passport_generic
- Add negative test cases for EU VAT false positives
- All 5 Greptile comments addressed
* Address Greptile feedback: fix patterns and sync backup
- Fix fr_phone pattern: use negative lookbehind (?<!\d) to prevent false matches in longer digit strings
- Add keyword_pattern to eu_passport_generic to reduce false positives on version strings/SKUs
- Sync policy_templates_backup.json with main file (add GDPR template)
- Add keyword_pattern to eu_vat (was auto-added by formatter)
All pattern tests passing
* address greptile review feedback (greploop iteration 2)
- Update test to document that eu_vat raw pattern is intentionally broad
- Test verifies pattern DOES match common words (by design)
- Documents that keyword_pattern guard prevents false positives in production
- Addresses Greptile's false positive risk concern
* address greptile review feedback (greploop iteration 3)
- Fix test_eu_vat_masked: change gap from 2 words to 1 word (VAT number: FR...)
- This ensures keyword matching works within MAX_KEYWORD_VALUE_GAP_WORDS=1 limit
- fr_phone pattern already works correctly (verified with tests)
- test_pattern_requires_keyword_context already updated in iteration 2
Addresses final issues from Greptile 2/5 review
* fix: remove country-specific passport patterns from GDPR template
- Remove passport_france, passport_germany, passport_netherlands from template
- These patterns lack keyword guards and cause false positives
- Only eu_passport_generic remains (has keyword_pattern guard)
- Sync policy_templates_backup.json with main file
- Update test setup to match template
All 11 E2E tests now passing ✅
* Update tests/test_litellm/proxy/guardrails/guardrail_hooks/content_filter/test_gdpr_policy_e2e.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Update litellm/proxy/guardrails/guardrail_hooks/litellm_content_filter/content_filter.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Add s3_use_virtual_hosted_style parameter to support AWS S3 virtual-hosted-style URL format (bucket.endpoint/key) alongside the existing path-style format (endpoint/bucket/key).
This enables compatibility with S3-compatible services like MinIO and aligns with AWS S3 official terminology.
The test was creating fresh mocks but not fully isolating from setUp state,
causing intermittent CI failures with 'Expected generation to be called once.
Called 0 times.'
Instead of creating fresh mocks, properly reset the existing setUp mocks to
ensure clean state while maintaining proper mock chain configuration.
Fixes persistent test isolation issue in TestLangfuseUsageDetails by
saving and restoring the global litellm.initialized_langfuse_clients
counter.
Changes:
- Save litellm.initialized_langfuse_clients in setUp
- Restore original value in tearDown
- Prevents counter accumulation across tests
Root Cause:
PR #21248 added logger cleanup but missed the global client counter.
Each test increments litellm.initialized_langfuse_clients when creating
a LangFuseLogger, but the counter was never reset. This caused state
accumulation that could affect test behavior when tests run in certain
orders, leading to "Expected 'generation' to have been called once.
Called 0 times" failures.
Impact:
- test_log_langfuse_v2_handles_null_usage_values was still flaky
- Counter would accumulate: 1, 2, 3... across all tests
- While unlikely to hit MAX (50), accumulated state affected behavior
This completes the test isolation fix started in PR #21248.
Related: #21248
Fixes: Remaining test isolation issues in Langfuse tests
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Follow-up to PR #21248 addressing greptile code review feedback.
Removes hasattr checks for non-existent attributes that were identified
as dead code by greptile automated code review.
Changes:
- Remove hasattr check for LangFuseLogger._langfuse_clients (class attribute doesn't exist)
- Remove hasattr check for self.logger._langfuse_client_cache (instance attribute doesn't exist)
- Update comments to be more accurate about what cleanup is being done
The core fix from PR #21248 (nulling Langfuse reference and deleting
logger instance) remains unchanged and effective. This just removes
misleading dead code that serves no purpose.
Context:
These checks were added defensively but reference attributes that don't
actually exist on the LangFuseLogger class, making them always no-ops.
Greptile correctly identified these as dead code in PR #21248 review,
but the PR was merged before the cleanup could be applied.
Related: #21248
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Enhances test isolation in TestLangfuseUsageDetails by ensuring the
logger instance is completely fresh for each test and properly cleaned
up afterward.
Changes:
- Clear any class-level cached Langfuse clients before creating logger
- Reset logger's cached client instances in setUp
- Properly clean up logger instance and its state in tearDown
Root Cause:
The test_log_langfuse_v2_handles_null_usage_values test was failing
when run after other tests due to lingering state in the logger instance.
While the test passes in isolation, test ordering issues caused it to
fail with "Expected 'generation' to have been called once. Called 0 times."
This builds on PR #21214 which added sys.modules cleanup, but that wasn't
sufficient to prevent all state leakage between tests.
Fixes: Test isolation issues in test_log_langfuse_v2_handles_null_usage_values
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
1. test_acompletion_with_mcp_streaming_metadata_in_correct_chunks:
- Moved stream consumption inside patch context to avoid real API calls
- The previous implementation had assertions outside the `with patch(...)`
block, causing real OpenAI API calls when consuming the stream
2. TestCheckResponsesCost tests:
- Added skip condition when litellm_enterprise module is not available
- These tests import from litellm_enterprise.proxy.common_utils.check_responses_cost
which is only available in the enterprise version
The manual sys.modules restoration code was redundant because
patch.dict.stop() automatically handles the cleanup. This simplifies
the tearDown method and removes the now-unused _original_langfuse_module
instance variable.
Addresses review comment: https://github.com/BerriAI/litellm/pull/21214#pullrequestreview-3802348462
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Fixes test_log_langfuse_v2_handles_null_usage_values flaky test failure
by properly cleaning up sys.modules['langfuse'] in tearDown.
Changes:
- Store original langfuse module in setUp before mocking
- Restore original or remove mock in tearDown to prevent state pollution
- Remove invalid print_verbose parameter from log_event_on_langfuse
Root Cause:
The tearDown method was not cleaning up sys.modules['langfuse'] after
each test, causing mock state to leak between tests. This caused
intermittent failures in CI, especially when tests run in parallel or
in different orders.
Impact:
This test has a long history of flakiness with multiple attempted fixes
(#20475, #17599, #17594, #17591, #17588). The missing sys.modules cleanup
was the underlying issue causing continued failures despite those patches.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Arize Phoenix and Arize loggers now create dedicated TracerProviders
instead of fighting over the global singleton, and the otel callback
dedup check no longer incorrectly matches Arize subclasses.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
