Add documentation explaining the difference between model formats:
- `gemini/model` → Gemini API (simple API key)
- `vertex_ai/model` → Vertex AI (GCP credentials)
- `model` (no prefix) → defaults to Vertex AI
This addresses user confusion when models without prefix require
GCP authentication instead of simple API key auth.
Ref #8424
* feat: add gemini video metadata and detail support
Implement support for video_metadata and enhanced detail parameter
for Gemini 3.0+ models:
- Add video_metadata field to ChatCompletionFileObjectFile type
- Supports fps, start_offset, and end_offset parameters
- Properly converts snake_case to camelCase for Gemini API
- Extend detail parameter to support medium and ultra_high levels
- Maps to MEDIA_RESOLUTION_MEDIUM and MEDIA_RESOLUTION_ULTRA_HIGH
- Update _process_gemini_image to handle video metadata transformation
- Add version gating to only apply features for Gemini 3+ models
- Add comprehensive test coverage (6 new test cases)
- Test detail parameter with file objects
- Test video_metadata fields (fps, start_offset, end_offset)
- Test combined detail + video_metadata usage
- Test new detail levels (medium, ultra_high)
- Test version gating (Gemini 1.5 vs 3.0)
Note: video_metadata is only supported for video files but error
handling is delegated to Vertex AI for other media types.
* refactor: rename _process_gemini_image to _process_gemini_media
The function handles multiple media types (images, audio, video, PDF),
not just images. Renamed to better reflect its actual purpose.
- Update function name in transformation.py
- Update all function calls and references
- Update test names and imports to match
- Improve docstring to clarify it handles all media types
* docs: add video metadata and media resolution control documentation
Add comprehensive documentation for Gemini 3+ video processing features:
- Document media resolution control (detail parameter) for images and videos
- Add video_metadata field documentation (fps, start_offset, end_offset)
- Include usage examples with tabs for basic, combined, and proxy scenarios
- Update both Gemini and Vertex AI provider documentation
- Clarify snake_case to camelCase field conversion for Gemini API
Signed-off-by: Kris Xia <xiajiayi0506@gmail.com>
* refactor(gemini): extract metadata application into helper function
Extract duplicated Gemini 3+ media_resolution and video_metadata
application logic from _process_gemini_media into a dedicated
_apply_gemini_3_metadata helper function to improve code maintainability.
---------
Signed-off-by: Kris Xia <xiajiayi0506@gmail.com>
Update Pillar Security integration to use the generic_guardrail_api
instead of the dedicated pillar guardrail type. This aligns with
the Generic Guardrail API specification introduced in previous PRs.
Changes:
- Rewrite pillar_security.md with new generic_guardrail_api config
- Add Pillar Security example to generic_guardrail_api.md
- Add Pillar Security to quick_start.md guardrails examples
Related PRs: #17175, #18647, #18932, #19023
The documentation incorrectly used `vertex_region` as the parameter name,
but the actual parameter expected by LiteLLM is `vertex_location` as defined
in VertexPassThroughCredentials and other type definitions.
Co-authored-by: Claude <noreply@anthropic.com>
* fix: Avoid attaching tool calls when a call_id already exists
* fix: Prevent MCP responses from reviving past tool calls via previous_response_id
* test: Parametrize MCP streaming test to cover OpenAI and Anthropic models
* test: Fail MCP streaming test when LiteLLM logs errors during follow-up calls
* test: Let MCP tool-execution mock accept new kwargs for streaming tests
* chore: fix lint error
* docs: Add Google Workload Identity Federation (WIF) documentation to Vertex AI (#19320)
- Added new section documenting WIF support for Vertex AI authentication
- Included SDK and Proxy configuration examples
- Added sample WIF credentials file format for AWS federation
- Mentioned LLM Credentials UI as an alternative for credential management
- Added link to Google Cloud WIF documentation
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
* fix(bedrock): deduplicate tool calls in assistant history (#15178)
* fix(types): add missing Set import to factory.py
---------
Co-authored-by: Yuta Saito <uc4w6c@bma.biglobe.ne.jp>
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: YutaSaito <36355491+uc4w6c@users.noreply.github.com>
* feat(gemini): add opt-in support for responseJsonSchema
Add support for Gemini's native responseJsonSchema parameter which uses
standard JSON Schema format instead of OpenAPI-style responseSchema.
Benefits of responseJsonSchema (Gemini 2.0+ only):
- Standard JSON Schema format (lowercase types)
- Supports additionalProperties for stricter validation
- Better compatibility with Pydantic's model_json_schema()
- No propertyOrdering required
Usage:
```python
response_format={
"type": "json_schema",
"json_schema": {"schema": {...}},
"use_json_schema": True # opt-in
}
```
This is backwards compatible - existing code continues to use
responseSchema by default.
Closes#16340
* docs: add documentation for use_json_schema parameter
Document the new use_json_schema option for Gemini 2.0+ models
in the JSON Mode documentation.
* refactor(gemini): use responseJsonSchema by default for Gemini 2.0+
Remove opt-in flag `use_json_schema` and automatically detect model version:
- Gemini 2.0+: uses responseJsonSchema (standard JSON Schema, supports additionalProperties)
- Gemini 1.5: uses responseSchema (OpenAPI format, legacy)
This follows LiteLLM's philosophy of abstracting provider differences -
users write the same code regardless of model version.
* test(vertex): update json_schema tests to accept both responseSchema formats
Gemini 2.x+ uses responseJsonSchema while Gemini 1.x uses responseSchema.
Update tests to accept both formats since litellm now auto-selects based
on model version.
* docs: update UI contributing guide with correct commands
- Replace outdated proxy_cli.py command with poetry run litellm
- Add config.yaml example with required settings
- Clarify that UI comes pre-built in the repo
- Add two development options: Build Mode and Dev Mode (hot reload)
- Note about redirect issues in Dev Mode
* docs: add hot reload login flow and PR submission section
- Document the 3000 -> 4000 -> 3000 login flow for hot reload
- Reorder: Hot Reload as Option A, Build Mode as Option B
- Add section 4 on submitting PRs
- Add note that UI changes don't require tests
* Update login flow navigation URL in contributing.md
- Added new section documenting WIF support for Vertex AI authentication
- Included SDK and Proxy configuration examples
- Added sample WIF credentials file format for AWS federation
- Mentioned LLM Credentials UI as an alternative for credential management
- Added link to Google Cloud WIF documentation
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
