* Fix: Map Gemini cached_tokens to Langfuse cache_read_input_tokens
Fixes#18520
## Problem
Langfuse integration was not capturing cached tokens from Gemini models.
Gemini returns cached tokens in `usage.prompt_tokens_details.cached_tokens`,
but Langfuse only read from top-level `usage.cache_read_input_tokens`
(which only Anthropic populates).
## Solution
Updated langfuse.py to check both locations:
1. First check top-level cache_read_input_tokens (for Anthropic)
2. Then check prompt_tokens_details.cached_tokens (for Gemini, OpenAI, others)
This ensures all providers' cached tokens are properly reported to Langfuse.
## Changes
- Modified litellm/integrations/langfuse/langfuse.py (lines 742-761)
- Added 3 unit tests in tests/test_litellm/integrations/langfuse/test_gemini_cached_tokens.py
- All existing Langfuse tests still pass (11/11)
## Testing
- test_cached_tokens_extraction: Verifies Gemini cached_tokens extraction
- test_cached_tokens_not_present: Backward compatibility (no cached_tokens)
- test_cached_tokens_is_zero: Edge case when cached_tokens = 0
* Refactor: Extract cache token logic into helper function
Address review feedback from @officer47p
- Created _extract_cache_read_input_tokens() helper function
- Reduces code bloat in _log_langfuse_v2 method
- Improves testability and reusability
- All tests still passing (11/11)
Fix metric name inconsistency for litellm_remaining_requests_metric
and litellm_remaining_tokens_metric. The factory received names
without the _metric suffix, causing _is_metric_enabled to fail when
users configured these metrics in prometheus_metrics_config.
Fixes#18221
Signed-off-by: majiayu000 <1835304752@qq.com>
* docs(prompt_management.md): document how to onboard prompts to litellm
* feat(arize_phoenix_prompt_manager.py): support new prompt management integration
allows users to connect arize phoenix prompt manager to litellm
* fix(proxy/utils.py): remove prompt variables to avoid re-processing prompt
* docs(arize_phoenix_prompts.md): document new prompt management integration
- Reset mock call counts at start of test to ensure clean state
- Add span method to mock trace to handle log_provider_specific_information_as_span calls
- Re-establish mock chain before test call to ensure fresh state
- Add exception handling to catch and report errors during test execution
- Add verification that trace was called before checking generation
This should fix the flaky test that was failing intermittently with
'Expected generation to have been called once. Called 0 times.'
Fixed three flaky tests that were intermittently failing in CI:
1. test_no_duplicate_spend_logs (test_litellm/responses/test_no_duplicate_spend_logs.py)
Problem: Used await asyncio.sleep(1) to wait for async logging completion,
which created race conditions. The async logging worker queues tasks
in the background, and sleep() doesn't guarantee completion.
Fix: Replaced sleep() with GLOBAL_LOGGING_WORKER.flush() which properly waits
for the logging queue to empty, ensuring all async logging tasks complete
before assertions run.
2. test_log_langfuse_v2_handles_null_usage_values (test_litellm/integrations/test_langfuse.py)
Problem: Used datetime.datetime.now() twice for start_time and end_time, which
could cause timing inconsistencies between test runs, especially in
CI environments with variable execution speeds.
Fix: Use fixed timestamps instead of datetime.now() to ensure consistent timing
across all test runs, eliminating timing-related flakiness.
3. test_watsonx_gpt_oss_prompt_transformation (test_litellm/llms/watsonx/test_watsonx.py)
Problem: Directly accessed mock_post.call_args without checking if it exists,
which could be None if the mock wasn't called or if an exception
occurred before the POST request. The test catches exceptions and
continues, making this a potential failure point.
Fix: Added proper assertions and use call_args_list[0] for safer access:
- Assert that call_args_list has at least one call
- Assert that call_args is not None
- Assert that 'data' key exists in kwargs
This ensures the test fails with clear error messages rather than
intermittent AttributeError exceptions.
All fixes maintain the original test intent while making them deterministic
and reliable in CI environments.
Reapplies the fix from commit a885e21543 that was
reverted in 6c9556be67.
The original revert was done because the test was flaky and giving false
negatives. This fix properly mocks the Langfuse client to ensure the test
can correctly verify that _log_langfuse_v2 converts None usage values to 0.
Changes:
- Add mock_langfuse_client.client attribute to prevent errors during init
- Add trace_id to mock_langfuse_generation for proper return value handling
- Remove redundant mock setup code
- Explicitly set logger.Langfuse to mock client after initialization
- Set logger.langfuse_sdk_version to ensure _supports_* methods work correctly
* Fix test_log_langfuse_v2_handles_null_usage_values test failure
The test was failing because the logger's Langfuse client wasn't properly
mocked. Even though sys.modules was mocked, the logger's __init__ method
creates its own Langfuse client instance that wasn't using the test's mock.
Changes:
- Explicitly set logger.Langfuse to the mock client after initialization
- Set logger.langfuse_sdk_version to ensure _supports_* methods work correctly
- Added mock_langfuse_client.client attribute to prevent errors during init
- Added trace_id to mock_langfuse_generation for proper return value handling
- Removed redundant mock setup code
This ensures the test can properly verify that _log_langfuse_v2 correctly
converts None usage values to 0 by allowing the mock's generation method
to be called and asserted.
Fixes: AssertionError: Expected 'generation' to have been called once. Called 0 times.
* feat(teams.py): param for disabling guardrails by team
allows use-case where you don't run global guardrails for team - only run team-specific guardrails
* feat(custom_guardrail.py): add support for disabling global guardrails
only run guardrails requested for in the request/key/team
* feat: support adding disable_global_guardrails to metadata if present in key/team metadata
* feat(create_key_button.tsx): new disable global guardrails field
* feat(key_edit_view.tsx): support disabling global guardrails on key edit
* feat(teams.tsx): add disable global guardrails on create team on UI
* feat(team_info.tsx): allow disabling global guardrails on team update
Ishaan Jaffer