Extract and preserve provider-specific headers (llm_provider-*) when
handling error responses from LLM providers. This ensures that useful
debugging information from providers is available even when requests
fail with BadRequestError or similar exceptions.
Gemini API rejects JSON schemas with $defs/$ref references anywhere in
the conversation, including in function_response content. This causes
errors when tools return JSON containing schemas (e.g., toolbelt_inspect_tool
returning tool definitions).
The fix:
1. Apply unpack_defs() to all JSON tool responses before sending to Gemini
2. Recursively remove $defs sections after inlining references
3. Replace any remaining $ref (circular refs, external URLs) with placeholders
Edge cases handled:
- Circular $ref (self-referential types like TreeNode.left -> TreeNode)
- External $ref (URL-based like https://...)
- Deeply nested $ref in anyOf/oneOf/allOf
This ensures function_response content is clean before being sent to the
Gemini API.
Related issues:
- https://github.com/google-gemini/gemini-cli/issues/13326
- FastMCP #1372, #1426
* fix(vertex_ai): improve passthrough endpoint url parsing and construction (#17402)
* test(proxy): add test for vertex passthrough load balancing
Add a test that verifies _base_vertex_proxy_route uses
get_available_deployment for proper load balancing instead of
get_model_list. This ensures the correct deployment is selected
from the router and vertex credentials are properly fetched.
Also refactor the implementation to:
- Use get_available_deployment instead of get_model_list
- Add error handling for deployment retrieval
- Improve code structure with try-except block
* feat(proxy): add pass-through deployment filtering methods
Add dedicated methods to filter and select deployments for pass-through endpoints:
- Implement get_available_deployment_for_pass_through() to ensure only deployments with use_in_pass_through=True are considered
- Implement async_get_available_deployment_for_pass_through() for async operations
- Add _filter_pass_through_deployments() helper method to filter by use_in_pass_through flag
- Update vertex pass-through route to use the new dedicated method
This ensures pass-through endpoints respect the use_in_pass_through configuration and apply proper load balancing strategy only to configured deployments.
Add comprehensive tests to verify filtering and load balancing behavior.
OpenAI's 400k context window is split between input and output:
- GPT-5/5.1/5.2 models: 272k input + 128k output = 400k context
- GPT-5-pro models: 128k input + 272k output = 400k context
Reference: https://openai.com/index/introducing-gpt-5-for-developers/
"In the API, all GPT-5 models can accept a maximum of 272,000 input
tokens and emit a maximum of 128,000 reasoning & output tokens"
Fixes incorrect 400k max_input_tokens values across 32 models.
- Add cerebras/zai-glm-4.7 with same specs as 4.6 (128K context, $2.25/M input, $2.75/M output)
- Mark cerebras/zai-glm-4.6 with deprecation_date: 2026-01-20
- Both models support function calling, reasoning, and tool choice
