- Skip test_apply_patch_tool_call_converted_to_chat_completion_tool_call
when openai.types.responses.response_apply_patch_tool_call is unavailable
(CI uses openai==1.100.1 which doesn't have this module)
- Skip MCP M2M tests (test_m2m_credentials_forwarded_to_server_model,
test_m2m_drops_incoming_oauth2_headers) that fail because PR #23187
changed has_client_credentials to require explicit oauth2_flow opt-in
but _execute_with_mcp_client was not updated to pass it through
- Revert source code change to rest_endpoints.py that auto-inferred
oauth2_flow (regression risk: this changes MCP OAuth behavior)
Co-authored-by: yuneng-jiang <yuneng-jiang@users.noreply.github.com>
When reasoning_effort is passed as a dict with additional fields like 'summary' or 'generate_summary', preserve the full dict format instead of normalizing it to a string. This ensures that when requests are routed to the OpenAI Responses API, all reasoning parameters are correctly included.
The normalization to string format now only happens for simple dicts with just the 'effort' key, which is appropriate for the Chat Completions API.
Fixes issue where summary field was being dropped when routing gpt-5.4+ requests with tools + reasoning to Responses API.
Made-with: Cursor
Add test_parallel_tool_calls_comprehensive_streaming_integration which
synthesizes the full 10-event Responses API SSE sequence with split
argument deltas and asserts all fix invariants together:
1. output_item.done emits no finish_reason (no premature stream end)
2. Each call_id appears exactly once (no duplicate tool_call chunks)
3. Split argument deltas assemble to correct final JSON
4. Exactly one finish event, at the terminal response.completed chunk
5. Parallel tool calls have distinct indices (output_index 0 and 1)
All 24 unit tests pass.
The response.completed handler in the completion→responses streaming
bridge was discarding the usage object, causing prompt_tokens_details
(and cached_tokens) to always be None when streaming with models that
use the Responses API (e.g. gpt-5.2-codex, gpt-5.3-codex).
Extract usage from the response.completed event and translate it via
the existing _transform_response_api_usage_to_chat_usage helper.
Fixes#22192
The response.output_item.done handler for function_call type was emitting
finish_reason='tool_calls' and a duplicate tool_call delta. This caused
premature stream termination after the first tool call in multi-tool
scenarios — downstream wrappers (e.g. AnthropicStreamWrapper) would close
the stream before subsequent tool calls arrived.
The response.completed event already inspects the response output list and
emits finish_reason='tool_calls' when function_call items are present, so
output_item.done does not need to (and must not) do so.
This mirrors the existing fix for message-type output_item.done (#17246).
Updated test_function_call_done_emits_is_finished (renamed) to assert
finish_reason=None and no duplicate delta. Updated test_text_plus_tool_calls_sequence
to match. Added test_multi_tool_call_stream_no_premature_finish which exercises
a synthetic 2-tool-call stream and verifies no premature termination.
* fix handling of ResponseApplyPatchToolCall in completion bridge
* refactor
* style: fix black formatting
* fix: clean up lint errors in test file (unused imports, print statements, formatting)
* refactor: extract _map_optional_params_to_responses_api to fix PLR0915
* what
* this linter cannot be me
* revert cause idk what's going on
* weird
* idk why this got removed
* revert more stuff
* revert pt 3
Fixes#21331 — the Responses API streaming bridge hardcoded index=0 for
all tool call chunks, making parallel tool calls indistinguishable.
Now reads output_index from the Responses API chunk instead.
When using the Responses API (e.g., Azure gpt-5.1-codex-mini), the response.completed
event was always returning finish_reason='stop', even when the response contained
function_call items in its output. This caused agents like OpenCode to incorrectly
conclude the stream ended without tools to execute, breaking tool/function calling
workflows.
The fix inspects the response.output field in the response.completed event to determine
the correct finish_reason:
- 'tool_calls' when output contains function_call items
- 'stop' otherwise (text-only responses)
Added tests to verify:
- response.completed with function_call output returns finish_reason='tool_calls'
- response.completed with message-only output returns finish_reason='stop'
- response.completed with empty output returns finish_reason='stop' (backward compat)
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
The Responses API expects tool results to use input_text/input_image types,
not output_text. This fix ensures consistent list format for all tool results:
- String content → [{"type": "input_text", "text": "..."}]
- Image content → [{"type": "input_image", "image_url": "..."}]
This resolves the conflict between tests that expected different formats
and aligns with OpenAI's Responses API requirements.
Fixes the regression introduced in #18226.
- test_tool_message_output_is_string_not_list: verifies function_call_output.output is a string
- test_multiple_tool_calls_in_single_choice: verifies multiple tool calls are grouped in one choice
* fix(unified_guardrails.py): send all chunks on completion of final stream
* feat(generic_guardrail_api.py): handle tool call response on streaming LLM responses
* fix(anthropic/chat/guardrail_translation): initial commit adding anthropic tool response streaming guardrails
enables guardrail checks on tool response from llm's to work via `/v1/messages`
* feat(anthropic/): working guardrail checks on tool response from LLMs
ensures guardrail checks on anthropic /v1/messages works as expected
* feat(responses/guardrail_translation): support tool call response guardrails on streaming for /v1/responses
ensures complete coverage of tool call responses
* refactor(openai.py): refactor to use consistent pydantic model for responses api tool response on streaming
enables non-openai model tool call response to work correctly with guardrail checks on /v1/responses
* test: update tests
* fix: fix linting error
* fix: fix failing tests
* fix: fix import errors
* fix(openai/chat/guardrail_transformation): fix final chunk returned on streaming
When using litellm.completion() with model="openai/responses/...", images
in tool message content were not being transformed from Chat Completion
format to Responses API format.
Chat Completion format: {"type": "image_url", "image_url": {"url": "..."}}
Responses API format: {"type": "input_image", "image_url": "..."}
This caused OpenAI to reject the request with error 400 since "image_url"
is not a valid type for function_call_output content.
When OpenAI Responses API returns both text AND tool_calls, the bridge
transformation was emitting is_finished=True after the text message completed,
causing subsequent tool_call chunks to be dropped.
The fix:
- response.output_item.done for messages no longer emits is_finished=True
- Added handler for response.completed to properly signal stream end
- Fix blank function name in completions response when using native function calling
- Fix Enum name being used instead of Enum value for comparison in chunk conversion
- Added additional tests to cover changes
Thanks to @mcowger for the invaluable assitance with figuring this issue out!
Fixed#16863
Fixes#16810
## Problem
When using completion() with models that have mode: "responses" (like o3-pro,
gpt-5-codex), the response_format parameter with JSON schemas was being ignored
or incorrectly handled, causing:
- Large schemas (>512 chars) to fail with "metadata.schema_dict_json: string too long" error
- Structured outputs to be silently dropped
- Users' code to break unexpectedly
## Root Cause
The completion -> responses bridge in
litellm/completion_extras/litellm_responses_transformation/transformation.py
was missing the conversion of response_format (Chat Completion format) to
text.format (Responses API format).
The inverse bridge (responses -> completion) already had this conversion
implemented in commit 29f0ed223a, but the completion -> responses direction
was incomplete.
## Solution
Added _transform_response_format_to_text_format() method that converts:
- response_format with json_schema → text.format with json_schema
- response_format with json_object → text.format with json_object
- response_format with text → text.format with text
Updated transform_request() to detect and convert response_format parameter
before sending to litellm.responses().
## Changes
- Added _transform_response_format_to_text_format() method (lines 592-647)
- Modified transform_request() to handle response_format (lines 199-203)
- Added comprehensive tests to validate the conversion
## Testing
- 5 new unit tests covering all conversion scenarios
- Real API test with OpenAI confirming large schemas (>512 chars) work
- No more metadata.schema_dict_json errors
## Impact
Users can now use completion() with models that have mode: "responses" and:
- Use large JSON schemas without hitting metadata 512 char limit
- Get proper structured outputs
- Have their existing code continue working
This aligns the proxy experience with other models that think
automatically (e.g. Deepseek R1 and grok3). It does so by setting
the necessary request input to return thinking, but not specifying
a budget or effort (thus defaulting to the internal automatic level).
* fix: handle reasoning parameters and response in responses bridge
Updates the OpenAI completions/responses bridge to map
reasoning_effort to reasoning parameters, and the chunk parser
to return reasoning_content.
ref: 12432
* fix: using type checked objects in responses bridge transform
ref: 12432
Sameer Kankute