The old test assumed ArizePhoenixLogger reused the global TracerProvider.
With the nested traces fix, Phoenix now creates its own dedicated provider
and produces litellm_proxy_request + litellm_request + raw_gen_ai_request
spans independently.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(tests): read CI_CD_DEFAULT_ANTHROPIC_MODEL env var in bedrock KB tests
* fix(tests): read CI_CD_DEFAULT_ANTHROPIC_MODEL env var in test_router
* fix(tests): read CI_CD_DEFAULT_ANTHROPIC_MODEL env var in test_router_retries
* fix(tests): read CI_CD_DEFAULT_ANTHROPIC_MODEL env var in test_router_timeout
SpendLogsMetadata added new fields (user_api_key, status, error_information,
etc.) that weren't in the expected spend_logs_payload.json fixture, causing
test_async_gcs_pub_sub_v1 to fail.
Adds a new config option to exclude specific fields from StandardLoggingPayload
before any callback receives it. This provides a general approach to control
what data is logged across ALL integrations (S3, GCS, Datadog, etc.).
## Changes
1. **litellm/__init__.py**: Added new global setting
`standard_logging_payload_excluded_fields: Optional[List[str]] = None`
2. **litellm/integrations/custom_logger.py**: Modified
`redact_standard_logging_payload_from_model_call_details()` to:
- Remove specified fields entirely from the StandardLoggingPayload
- Works alongside existing `turn_off_message_logging` feature
- Excluded fields take precedence (removed rather than redacted)
3. **tests/**: Added comprehensive test suite with 17 tests covering:
- Single/multiple field exclusion
- Interaction with turn_off_message_logging
- Original payload immutability
- Config loading via setattr (proxy pattern)
- Edge cases (empty list, non-existent fields, None standard_logging_object)
## Usage
```yaml
litellm_settings:
success_callback: ["s3"]
standard_logging_payload_excluded_fields: ["response", "messages"]
```
This removes the `response` and `messages` fields from logs before any
callback processes them, reducing log size and improving privacy compliance.
## Available Fields
The fields match StandardLoggingPayload TypedDict keys including:
- messages, response (large payload fields)
- metadata, hidden_params, model_parameters
- error_str, error_information
- And all other StandardLoggingPayload fields
Closes the need for per-integration flags like `s3_log_response`.
* perf: Optimize get_standard_logging_metadata with set intersection
- Cache StandardLoggingMetadata.__annotations__.keys() as module-level frozenset
- Use set intersection to iterate only keys present in both metadata and supported keys
- Single lookup for user_api_key instead of 3 separate .get() calls
Results:
- get_standard_logging_metadata: 1.55s → 1.41s (9.2% faster)
* test: add unit tests for get_standard_logging_metadata non-string user_api_key handling
Adds log_format parameter supporting json_array (default), ndjson, and single formats. NDJSON format enables webhook integrations like Sumo Logic to parse individual log records at ingest time. Defaults to json_array for backward compatibility.
* fix: enforce team member budget check in common_checks
- Add missing team member budget validation in common_checks() function
- Checks team membership budget when team key is used
- Raises BudgetExceededError when team member spend exceeds max_budget_in_team
- Follows same pattern as other budget checks (team, user, end_user)
- Uses cached get_team_membership() for performance
- Fix AttributeError in lowest_tpm_rpm.py
- Add null check for model_info before accessing .get() method
- Prevents 'NoneType' object has no attribute 'get' error
- Add unit tests for team member budget enforcement
- Test budget exceeded scenario
- Test within budget scenario
- Test edge cases (no budget, no membership, personal keys)
- Tests run without requiring proxy server
Fixes failing test: test_users_in_team_budget
* fix: mock get_async_httpx_client in test_langsmith_key_based_logging
- Mock get_async_httpx_client to return a mock AsyncHTTPHandler instance
- Fixes test failure where mock_post was never called
- LangsmithLogger creates its own httpx client instance via get_async_httpx_client,
so we need to mock the factory function rather than the class method
- Use MagicMock for response.raise_for_status (sync method) instead of AsyncMock
* fix: resolve linting errors (PLR0915, F401)
- Remove unused imports (datetime, ServiceLoggerPayload) from arize_phoenix.py
- Extract health ping setup logic from RedisCache.__init__ to reduce statement count
- Extract team member budget check from common_checks to reduce statement count
* fix: resolve type errors in ChatCompletionToolCallChunk construction
- Cast type field to Literal['function'] to satisfy TypedDict requirements
- Ensure arguments field is explicitly str type to match TypedDict signature
- Fixes pyright errors for incompatible types in transformation.py
- Add agent_id: null to expected JSON to match actual payload structure
- Fixes test_async_gcs_pub_sub_v1 test failure
- agent_id is an optional field in SpendLogsPayload that is always included (as null when not provided)
* feat(langfuse): Add support for custom masking function
Allow users to pass a custom masking function via metadata to selectively
redact sensitive data (credit cards, emails, PII) before sending to Langfuse.
Usage:
```python
def mask_pii(data):
if isinstance(data, str):
data = re.sub(r'\b\d{4}[\s-]?\d{4}[\s-]?\d{4}[\s-]?\d{4}\b', '[CARD]', data)
return data
litellm.completion(
model="gpt-4",
messages=[...],
metadata={"langfuse_masking_function": mask_pii}
)
```
* fix(langfuse): Isolate masking function from other logging integrations
Extract langfuse_masking_function from metadata early in the flow and store
it in a dedicated key (_langfuse_masking_function) that only the Langfuse
logger knows to look for. This prevents the callable from leaking to other
logging integrations (Datadog, S3, etc.) which would serialize it as
"<function at 0x...>".
Changes:
- scrub_sensitive_keys_in_metadata() now extracts and stores the function
- Langfuse logger looks in the dedicated key first, falls back to metadata
- Added tests to verify isolation works correctly
* test: add failing tests for organization budget enforcement bug
Add comprehensive tests exposing that organization-level budgets are
retrieved but never enforced during request authentication. Tests verify:
1. Basic org budget exceeded scenario (team under budget, org over)
2. Multiple teams collectively exceeding org budget
3. Organization budget fields exist but are never checked
4. Inconsistency between team budget enforcement (works) and org (doesn't)
Tests intentionally fail to document the bug. Will be fixed in next commit.
Related to organization_max_budget not being enforced in auth_checks.py
* fix: enforce organization budget in auth checks
Add organization budget enforcement to common_checks() in auth_checks.py.
Previously, organization_max_budget was retrieved from DB but never checked,
allowing teams to collectively exceed their organization's budget limit.
Changes:
- Add _organization_max_budget_check() function following team budget pattern
- Call org budget check after team budget check in common_checks()
- Add "organization_budget" to budget_alerts type literals
- Update tests to verify org budget is enforced
Budget hierarchy is now properly enforced:
Organization Budget (hard ceiling)
└─ Team Budget (sub-allocation)
└─ Team Member Budget (per-user within team)
└─ Key Budget (per-key)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix: add organization_id to budget alerts, fix enum comparison and linting of newly added code
- Add organization_id field to CallInfo class for better alert context
- Include organization_id in budget alerts (token, soft, team, org)
- Fix event_group enum comparison (was comparing enum to string)
- Add OrganizationBudgetAlert class for organization budget alerting
- Add organization_budget to test parameterizations
- Apply Black formatting to slack_alerting.py
---------
Co-authored-by: Claude <noreply@anthropic.com>
* feat(generic_api_callback.py): make generic api OSS + support multiple generic API's
Enables https://github.com/BerriAI/litellm/pull/17094#discussion_r2562832967
* feat(callback_utils.py): support custom generic api callbacks
* feat(generic_api_callback.py): support specifying which event types to run the generic api for
* fix(litellm_logging.py): log system prompt for anthropic messages
* feat(generic_api_callback.py): support generic api compatible api's - e.g. rubrik agent cloud
* docs(sidebars.js): document new OSS generic api
* docs(generic_api.md): document new OSS Generic API
* docs(custom_webhook_api.md): document custom webhook api integration tutorial
* docs(custom_webhook_api.md): cleanup
* docs(custom_webhook_api.md): document what get's logged to custom webhook api
* Refactor: Pass callback config to GenericAPILogger
Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>
* Fix: Handle empty messages list in logging payload
Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>
* Checkpoint before follow-up message
Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>
* feat: Cache GenericAPILogger instances to improve performance
Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Sameer Kankute