mirror of
https://github.com/tiennm99/litellm.git
synced 2026-07-17 00:17:16 +00:00
Alexsander HamirandGitHub
2e7b554747
* fix: enforce team member budget check in common_checks - Add missing team member budget validation in common_checks() function - Checks team membership budget when team key is used - Raises BudgetExceededError when team member spend exceeds max_budget_in_team - Follows same pattern as other budget checks (team, user, end_user) - Uses cached get_team_membership() for performance - Fix AttributeError in lowest_tpm_rpm.py - Add null check for model_info before accessing .get() method - Prevents 'NoneType' object has no attribute 'get' error - Add unit tests for team member budget enforcement - Test budget exceeded scenario - Test within budget scenario - Test edge cases (no budget, no membership, personal keys) - Tests run without requiring proxy server Fixes failing test: test_users_in_team_budget * fix: mock get_async_httpx_client in test_langsmith_key_based_logging - Mock get_async_httpx_client to return a mock AsyncHTTPHandler instance - Fixes test failure where mock_post was never called - LangsmithLogger creates its own httpx client instance via get_async_httpx_client, so we need to mock the factory function rather than the class method - Use MagicMock for response.raise_for_status (sync method) instead of AsyncMock * fix: resolve linting errors (PLR0915, F401) - Remove unused imports (datetime, ServiceLoggerPayload) from arize_phoenix.py - Extract health ping setup logic from RedisCache.__init__ to reduce statement count - Extract team member budget check from common_checks to reduce statement count * fix: resolve type errors in ChatCompletionToolCallChunk construction - Cast type field to Literal['function'] to satisfy TypedDict requirements - Ensure arguments field is explicitly str type to match TypedDict signature - Fixes pyright errors for incompatible types in transformation.py
Caching on LiteLLM
LiteLLM supports multiple caching mechanisms. This allows users to choose the most suitable caching solution for their use case.
The following caching mechanisms are supported:
- RedisCache
- RedisSemanticCache
- QdrantSemanticCache
- InMemoryCache
- DiskCache
- S3Cache
- AzureBlobCache
- DualCache (updates both Redis and an in-memory cache simultaneously)
Folder Structure
litellm/caching/
├── base_cache.py
├── caching.py
├── caching_handler.py
├── disk_cache.py
├── dual_cache.py
├── in_memory_cache.py
├── qdrant_semantic_cache.py
├── redis_cache.py
├── redis_semantic_cache.py
├── s3_cache.py