* feat: add gemini video metadata and detail support
Implement support for video_metadata and enhanced detail parameter
for Gemini 3.0+ models:
- Add video_metadata field to ChatCompletionFileObjectFile type
- Supports fps, start_offset, and end_offset parameters
- Properly converts snake_case to camelCase for Gemini API
- Extend detail parameter to support medium and ultra_high levels
- Maps to MEDIA_RESOLUTION_MEDIUM and MEDIA_RESOLUTION_ULTRA_HIGH
- Update _process_gemini_image to handle video metadata transformation
- Add version gating to only apply features for Gemini 3+ models
- Add comprehensive test coverage (6 new test cases)
- Test detail parameter with file objects
- Test video_metadata fields (fps, start_offset, end_offset)
- Test combined detail + video_metadata usage
- Test new detail levels (medium, ultra_high)
- Test version gating (Gemini 1.5 vs 3.0)
Note: video_metadata is only supported for video files but error
handling is delegated to Vertex AI for other media types.
* refactor: rename _process_gemini_image to _process_gemini_media
The function handles multiple media types (images, audio, video, PDF),
not just images. Renamed to better reflect its actual purpose.
- Update function name in transformation.py
- Update all function calls and references
- Update test names and imports to match
- Improve docstring to clarify it handles all media types
* docs: add video metadata and media resolution control documentation
Add comprehensive documentation for Gemini 3+ video processing features:
- Document media resolution control (detail parameter) for images and videos
- Add video_metadata field documentation (fps, start_offset, end_offset)
- Include usage examples with tabs for basic, combined, and proxy scenarios
- Update both Gemini and Vertex AI provider documentation
- Clarify snake_case to camelCase field conversion for Gemini API
Signed-off-by: Kris Xia <xiajiayi0506@gmail.com>
* refactor(gemini): extract metadata application into helper function
Extract duplicated Gemini 3+ media_resolution and video_metadata
application logic from _process_gemini_media into a dedicated
_apply_gemini_3_metadata helper function to improve code maintainability.
---------
Signed-off-by: Kris Xia <xiajiayi0506@gmail.com>
* fix: preserve tool output ordering for gemini in responses bridge
- Keep function_call_output adjacent to its function_call when building chat messages
- Normalize function_call_output.output lists (input_* parts) into tool message content
* fix test
* small improvements
Update Pillar Security integration to use the generic_guardrail_api
instead of the dedicated pillar guardrail type. This aligns with
the Generic Guardrail API specification introduced in previous PRs.
Changes:
- Rewrite pillar_security.md with new generic_guardrail_api config
- Add Pillar Security example to generic_guardrail_api.md
- Add Pillar Security to quick_start.md guardrails examples
Related PRs: #17175, #18647, #18932, #19023
Emit Responses API streaming events for tool calls when the underlying chat stream contains tool_call deltas, and recover tool calls into the stream when they only appear in the final response.
