mirror of
https://github.com/tiennm99/litellm.git
synced 2026-07-12 19:04:59 +00:00
Sameer Kankute
ed073d382d
* fix(gemini-realtime): use GA event names for Pipecat 1.3.x compatibility Pipecat v1.3.0 adopted the OpenAI Realtime API GA event naming: response.audio.delta -> response.output_audio.delta response.text.delta -> response.output_text.delta response.audio.done -> response.output_audio.done response.text.done -> response.output_text.done The proxy was still emitting the old beta names; Pipecat's `parse_server_event` raises "Unimplemented server event type" for any unknown type, which killed the receive task handler and broke audio playback and tool-call delivery. Also: - conversation.item.created -> conversation.item.added (already handled) - client audio is buffered until backend setupComplete in deferred mode - call_id fallback UUID when Gemini returns empty id - status_details / token detail fields added to Pydantic-strict events The _GA_TO_BETA_EVENT_TYPES map in RealTimeStreaming already translates GA names back to beta for clients that opt in with the openai-beta header, so legacy clients are unaffected. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(gemini-realtime): address greptile review comments - emit outputTranscription as response.output_audio_transcript.delta instead of suppressing it; GA_TO_BETA map handles translation for legacy clients - cap pre-setup audio buffer at 200 frames to prevent memory exhaustion; log a warning when the limit is hit and additional frames are dropped - log remaining dropped message count on flush error Co-authored-by: Cursor <cursoragent@cursor.com> * fix(gemini-realtime): address veria review comments - remove unused OpenAIRealtimeConversationItemCreated import - fix guardrail bypass: semantic_vad early-return now preserves create_response when set so a guardrail-injected create_response:false is not silently dropped - add per-connection 10 MB byte cap alongside the 200-frame count cap for the pre-setup audio buffer to prevent memory exhaustion Co-authored-by: Cursor <cursoragent@cursor.com> * fix(gemini-realtime): fix mypy arg-type on _finalize_gemini_live_setup setup parameter typed as BidiGenerateContentSetup to match the TypedDict passed at both call sites; was dict which mypy rejected. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(gemini-realtime): widen _finalize_gemini_live_setup to Dict[str, Any] BidiGenerateContentSetup (TypedDict) is a subtype of Dict[str,Any] so both call sites (one passing a plain dict, one passing the TypedDict) satisfy mypy. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(gemini-realtime): cast BidiGenerateContentSetup to Dict at _finalize call site mypy rejects TypedDict as dict[str, Any] argument; cast at the call site where follow_up_setup is BidiGenerateContentSetup to satisfy the checker. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix Gemini realtime beta compatibility * Fix deferred Gemini setup audio ordering * fix: preserve Gemini audio transcript ids * fix(realtime): cap pre-setup client buffer on all append paths Route every append to the deferred-setup pending buffer through the per-connection message/byte caps. Previously only the audio-buffer fast path enforced the caps; once one frame was buffered, a client that withheld session.update could stream arbitrary frames into _pending_messages_until_setup unbounded and exhaust proxy memory. * style(gemini-realtime): apply black formatting to transformation.py * fix(gemini-realtime): log beta-translation fallback and name native-audio marker Surface the previously swallowed exception in _send_event_to_client so a failed GA->beta translation is observable instead of silently forwarding the untranslated event. Extract the native-audio model substring used by _finalize_gemini_live_setup into a named constant documenting why speechConfig is dropped on those setups. --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/test_litellm
This folder can only run mock tests.