- Add specific rate limit type (requests/tokens/max_parallel_requests) to error message
- Include current limit value for better context
- Display reset time in human-readable format
- Handle negative remaining values gracefully by showing 0 instead
- Add reset_at header with timestamp for programmatic use
Fixes issue where rate limit errors were ambiguous about which type of limit
was exceeded and when the limit would reset.
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
* feat: add Vertex AI support for file content retrieval
- Extended `custom_llm_provider` to include "vertex_ai" in `afile_content` function.
- Implemented file content retrieval logic for Vertex AI in `VertexAIFilesHandler`.
- Added helper method to extract bucket and object from URL-encoded file_id.
- Created comprehensive unit and integration tests for Vertex AI file handling.
- Updated transformation logic to ensure compatibility with Vertex AI file responses.
* fix: update Vertex AI file transformation logic
- Modified the transformation logic in `VertexAIFilesConfig` to return a newline-separated JSON string for batch JSONL files instead of a array if JSON strings.
* fix: enhance Vertex AI output handling in transformation logic
- Updated the transformation logic in `VertexAIBatchTransformation` to utilize the new `OutputInfo` TypedDict for retrieving the GCS output directory.
- Added `OutputInfo` class to type definitions for better structure and clarity in Vertex AI responses.
* fix: avoid NoneType AttributeError when extracting tags
I've been running into this error:
```
21:47:08 - LiteLLM:ERROR: litellm_logging.py:2396 - LiteLLM.LoggingError: [Non-Blocking] Exception occurred while success logging Traceback (most recent call last):
File "/usr/lib/python3.13/site-packages/litellm/litellm_core_utils/litellm_logging.py", line 2312, in async_success_handler
await callback.async_log_success_event(
...<6 lines>...
)
File "/usr/lib/python3.13/site-packages/litellm/router_strategy/budget_limiter.py", line 396, in async_log_success_event
request_tags = _get_tags_from_request_kwargs(kwargs)
File "/usr/lib/python3.13/site-packages/litellm/router_strategy/tag_based_routing.py", line 144, in _get_tags_from_request_kwargs
return _metadata.get("tags", [])
^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'get'
```
This makes the function more resilient without resorting to try catch.
* add tests
Signed-off-by: Max Falk <gmdfalk@gmail.com>
---------
Signed-off-by: Max Falk <gmdfalk@gmail.com>
- Add response_modalities configuration to request format
- Fix response parsing to use camelCase 'inlineData' instead of snake_case 'inline_data'
- Update test to validate proper request format and response parsing
- All existing Gemini image generation tests pass
- Reverted GEMINI_2_5_FLASH_IMAGE_PREVIEW_MODEL constant usage
- Made endpoint selection conditional for gemini-2.5-flash-image-preview only
- Preserved existing Imagen models functionality with :predict endpoint
- Fixed potential breaking change that would affect 6 other Gemini image models
Update Bedrock documentation for Titan V2 encoding_format support + Anthropic - account for 1h vs. 5m cache creation token cost difference + UI - add langsmith_sampling_rate as a dynamic param
- Update endpoint from :predict to :generateContent
- Add Gemini format support for 2.5-flash-image-preview model
- Maintain backward compatibility with existing Imagen models
- Handle response parsing for candidates format
- Update mode from 'chat' to 'image_generation' for both model variants
- Ensures correct routing to image generation endpoints
- Resolves 400 'request not supported' error for image generation
Krish Dholakia