* fix(anthropic/experimental_pass_through): use given model name when returning streaming chunks
don't harcode model name on streaming
confusing for user
* fix(anthropic/streaming_iterator.py): remove scope of import
* feat(litellm_logging.py): allow admin to specify additional headers for using as spend tags
Closes https://github.com/BerriAI/litellm/issues/12129
* test(test_litellm_logging.py): add unit tests
* feat(openweb_ui.md): add custom tag tutorial to docs
* docs(cost_tracking.md): add tag based usage UI screenshot
* test: update test
* fix: fix import
* use common helper create_invitation_for_user
* use common util in proxy
* fix create_invitation_for_user
* refactor base email
* test_get_invitation_link_creates_new_when_none_exist
* fix code QA checks
* feat(check_batch_cost.py): emit spend log on successful request
ensures cost tracked for batch requests
* feat(proxy_server.py): add background job to poll completed batch jobs
used for calculating cost for batch jobs
* fix(proxy_server.py): run batch cost tracking job every hour
batch jobs take time to complete, no need to run every few seconds
* feat(proxy_server.py): run batch cost tracking job every hour
* Add Azure OpenAI assistant features cost tracking
Implements cost tracking for Azure's new assistant features:
- File Search: $0.1 USD per 1 GB/Day (storage-based pricing)
- Code Interpreter: $0.03 USD per session
- Computer Use: $0.003 input + $0.012 output per 1K tokens
Features:
- Provider-specific pricing (Azure vs OpenAI)
- Model-specific pricing overrides via JSON config
- Environment variable configuration
- Backwards compatible with existing OpenAI pricing
* Add comprehensive tests for Azure assistant features cost tracking
- Unit tests for file search, code interpreter, computer use, vector store
- Integration tests for combined cost calculation
- Provider-specific pricing tests (Azure vs OpenAI)
- Model-specific pricing override tests
- Edge case handling (None inputs, zero values)
- All 17 tests passing
* Fix test and ensure all Azure assistant cost tracking tests pass
- Fixed integration test approach
- All 17 tests now passing
- Comprehensive coverage of Azure assistant features cost tracking
* Enhance cost tracking for Azure assistant features
- Safely convert and extract parameters for file search, computer use, and code interpreter sessions.
- Ensure model_info is consistently converted to a dictionary format.
- Improve error handling for input values to prevent type-related issues.
- Maintain compatibility with existing cost calculation methods.
* Refactor cost tracking for Azure assistant features
- Introduced separate methods for handling costs related to web search, file search, vector store, computer use, and code interpreter.
- Enhanced parameter extraction and conversion for file search and computer use.
- Improved error handling and type safety throughout the cost calculation process.
- Maintained compatibility with existing cost calculation methods while streamlining the overall structure.
* Fix user-team association issues in LiteLLM proxy
- Update list_team function to properly filter teams using user's teams array instead of only checking members_with_roles field
- Add Field descriptions and docstring to TeamMemberAddRequest and related models for better Swagger/OpenAPI documentation
- Maintain backward compatibility with fallback to members_with_roles if user lookup fails
This ensures users created with teams parameter appear correctly in team views and improves API documentation.
* Fix duplicate member checking in team_member_add endpoint
- Enhanced team_member_add_duplication_check to check both user_id and user_email
- Added additional duplicate prevention logic after user creation/lookup
- Fixed issue where users added by email could be duplicated in teams
- Added logging for debugging duplicate detection
This addresses the bug where adding the same user by email multiple times would create duplicate entries in the team's members_with_roles array.
* Improve duplicate member prevention in team_member_add
- Enhanced early duplicate check to handle both user_id and user_email
- Added late-stage duplicate prevention after user lookup/creation
- Fixed issue where users could be added multiple times by email
- Cleaned up debug logging
Note: There's still an edge case where the duplicate prevention may not work
correctly in all scenarios. This needs further investigation and testing.
* Refactor team_member_add endpoint for improved member management
- Split team_member_add functionality into smaller, dedicated functions for permission validation, member processing, and team member list updates.
- Enhanced permission checks to ensure only authorized users can add members.
- Streamlined member addition logic to reduce redundancy and improve readability.
- Maintained existing functionality while improving code structure and maintainability.
* Add tests for team_member_add helper functions
- Add test for _validate_team_member_add_permissions with admin user
- Add test for _validate_team_member_add_permissions with non-admin user
- Add test for _process_team_members with single member
- Add test for _process_team_members with multiple members
- Add test for _update_team_members_list with new member
- Add test for _update_team_members_list duplicate prevention
These tests ensure the refactored helper functions work correctly
after fixing the PLR0915 linting error.
- Move from CircleCI dependency to direct pytest execution
- Add Python script to generate beautiful markdown reports
- Update GitHub workflow to run tests directly
- Update Makefile to use the new test runner script
- Generate both JUnit XML and markdown artifacts
- Group test results by provider with detailed statistics
* refactor(passthrough_endpoints-success-handler): refactor llm passthrough logging logic
isolate the llm translation work to enable cost tracking on sdk
* feat: initial implementation of passthrough SDK cost calculation
enables bedrock passthrough cost tracking to work
* feat(cost_calculator.py): working cost calculation for bedrock passthrough
* feat(litellm_logging.py): consider allm_passthrough in cost tracking
allows async calls (e.g. via proxy) to work
* feat(bedrock/passthrough): working event stream decoding for bedrock passthrough calls + logging instrumentation for passthrough sdk calls (log on stream completion)
Enables bedrock streaming cost calculation
* feat(litellm_logging.py): support streaming passthrough cost tracking
* feat(passthrough/main.py): working async streaming cost calculation
Closes https://github.com/BerriAI/litellm/issues/11359
* feat(proxy_server.py): fix passthrough routing when llm router enabled
* feat: further fixes
* feat(bedrock/): working bedrock passthrough cost tracking (non-streaming)
* feat(litellm_logging.py): working usage tracking for bedrock passthrough calls
ensures tokens are logged
* feat(bedrock/passthrough): add converse passthrough cost tracking support
* feat(base_llm/passthrough): remove redundant function
* refactor(litellm_logging.py): refactor function to be below 50 LOC
* test: update test
* test: remove redundant test
Ishaan Jaff