12 Commits

Include renames

Author	SHA1	Message	Date
4 people	48c9fabb26	Fix : a2a bugs 030626 (#29566) ... * Fix error code and context id injection bug * Add support for all A2A methods * Add logging * address greptile review: relay upstream JSON-RPC errors, move _PASCAL_TO_WIRE to module level, add error path tests * fix(a2a): run pre_call_hook for tasks/resubscribe SSE path to enforce guardrails tasks/resubscribe was returning the raw SSE stream without calling proxy_logging_obj.pre_call_hook, silently bypassing any guardrails configured on the agent. This patch calls pre_call_hook before streaming begins and wires post_call_failure_hook into the SSE generator so errors are logged. Adds a regression test verifying the hook is called. * fix(a2a): use get_async_httpx_client instead of creating httpx clients per request Creating httpx.AsyncClient instances per-request adds ~500ms latency. Switch _forward_jsonrpc and _forward_jsonrpc_sse to use the shared client from get_async_httpx_client(httpxSpecialProvider.A2A). * fix(a2a): forward caller identity headers on task ops; validate push notification URL Two security fixes for task management methods: 1. All task operations (tasks/get, tasks/list, tasks/cancel, tasks/resubscribe, push notification config methods) now forward X-LiteLLM-User-Id and X-LiteLLM-Team-Id headers to the upstream agent, so the agent can scope task access to the authenticated caller. 2. tasks/pushNotificationConfig/set validates the callback URL before forwarding: requires HTTPS and rejects private/loopback/reserved IP ranges and localhost hostnames to prevent SSRF. * Fix A2A task hook and push URL handling * fix(a2a): fix mypy type errors for request_id and header_name dict key types * Fix A2A request id and params forwarding * Forward trace IDs for A2A task calls * fix(a2a): strip client-forwarded X-LiteLLM-* headers before applying authenticated identity A client could send x-a2a-<agent>-x-litellm-user-id in their request and have it forwarded to the upstream agent as an authenticated identity header. Fix: sanitize any X-LiteLLM-* headers from agent_extra_headers before merging, then apply the authenticated identity headers last so they always override client-supplied values. * Fix A2A SSE fallback JSON-RPC error code * Fix A2A SSE error id backfill * fix(a2a): validate both push notification url fields to close SSRF bypass * fix(a2a): widen request_id annotation to match JSON-RPC id call sites * fix(a2a): run post-call streaming hook for tasks/resubscribe so agent guardrails apply tasks/resubscribe returned the raw upstream SSE stream without routing events through the post-call streaming hook, so output guardrails configured on the agent were silently skipped for streaming task subscriptions while every other task method and message/stream applied them. Parse upstream JSON-RPC SSE events and feed them through async_streaming_data_generator, matching message/stream, so guardrails inspect the streamed task content. Adds a regression test that fails when the streamed events bypass the guardrail hook. --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>	2026-06-03 11:14:15 -07:00
4 people	c8bcfbb20c	feat(a2a): watsonx Orchestrate agent provider (#29410) ... * feat(a2a): add watsonx Orchestrate agent provider Bridge A2A message/send to WXO runs API (CP4D and IBM Cloud IAM auth), with dashboard agent type metadata and unit tests for transformations. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(a2a): use shared httpx client and cache WXO auth tokens Route WXO streaming through get_async_httpx_client (TLS verification enabled). Cache bearer tokens with TTL buffer. Extract A2A reply text via a dedicated helper instead of hard-coded JSON paths. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(a2a): treat CP4D token expiration as absolute Unix time CP4D /authorize returns expiration as epoch seconds, not TTL. Compute remaining lifetime against wall clock so cached tokens refresh before expiry. Co-authored-by: Cursor <cursoragent@cursor.com> * style(a2a): black-format watsonx orchestrate handler for CI py312 Co-authored-by: Cursor <cursoragent@cursor.com> * Fix watsonx orchestrate edge cases * Fix WXO streaming fallback error handling * Fix watsonx orchestrate run completion handling * fix(a2a): make WXO username optional in agent create UI Username is only required for cp4d auth; ibm_cloud uses api_key alone. Backend still validates username when auth_mode is cp4d. Co-authored-by: Cursor <cursoragent@cursor.com> * test(a2a): align WXO dashboard field test with optional username Username is not required in agent_create_fields.json; backend validates for cp4d auth_mode only. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(a2a): send Accept header on WXO streaming run request * fix(a2a/wxo): scope streaming transport fallback to initial POST only Narrow the httpx.TransportError fallback in handle_streaming so it only covers the initial POST /runs/stream. Errors during polling or SSE consumption now propagate instead of triggering handle_non_streaming, which would have submitted a duplicate WXO run for the same request. * refactor(a2a/wxo): type run-param extraction and use text response_type Return a typed WXORequestParams NamedTuple from _extract_litellm_params instead of a positional tuple so call sites read params by name, and send the user message with response_type 'text' so the run body is valid across all WXO agent configurations rather than the search-specific type. * fix(a2a/wxo): evict expired token cache entries and raise asyncio.TimeoutError on poll timeout --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>	2026-06-02 18:41:10 +05:30
5 people	af17400c38	feat(a2a): well-known agent-card discovery + LangGraph Platform mode (#28860) ... * feat(a2a): well-known agent-card discovery + LangGraph Platform mode Adds a registration-time discovery flow so admins can paste an upstream agent URL, see its skills/capabilities, pick what to expose, and have the proxy front it with a LiteLLM-shaped agent card. Backend (new litellm/proxy/a2a/ module): - fetch_well_known_card walks /.well-known/agent-card.json, /.well-known/agent.json, /agent.json by default. langgraph_platform mode hits the canonical path with ?assistant_id=<id> (LangGraph serves one shared endpoint per deployment). - merge_agent_card overlays LiteLLM overrides on the upstream card: drops upstream url, forces protocolVersion=1.0, replaces securitySchemes with LiteLLMKey bearer, emits supportedInterfaces pointing at the proxy, filters capabilities to a small allowlist, strips non-v1.0 fields. - POST /v1/a2a/discover returns the raw upstream card (admin-only) so the UI can render skills/capabilities for selection. - create/update/patch agent endpoints pre-generate the agent_id and run merge_agent_card before storing, so DB.agent_card_params already embeds the proxy-fronted URL. UI (ui/litellm-dashboard): - New AgentCardDiscovery component with a parent-driven plan: discovery_mode + params + display URL. For LangGraph the parent composes (api_base, assistant_id); for pure A2A it uses the url field. Component hides the manual URL input when the parent drives. - add_agent_form wires discovery for every non-custom agent type and overlays the user's selections onto agent_card_params at submit, fixing the bug where dynamic agent forms ignored discovery picks. Completion-bridge fixes (paired): - Add kind: "message" to A2A response messages and unwrap result so it's a Message directly per spec (matches a2a SDK SendMessageResponse validation). - Forward A2A metadata to LangGraph runs via extra_body.metadata. * fix(a2a): preserve agent url, fix streaming chunk envelope, and protect forwarded metadata - Streaming chunk: move final out of the message object into the result envelope per the A2A spec. - Agent card merge: keep upstream url on the stored card so the runtime invocation path can locate the upstream backend; the public well-known endpoint already rewrites this field to the proxy URL before exposing it to clients. - Completion bridge: apply A2A forward metadata after merging litellm_params so an agent-configured extra_body cannot overwrite the forwarded metadata. Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(a2a): fix legacy streaming chunk, agent card test, and metadata merge - providers/litellm_completion: move 'final' out of the message object into the result envelope per the A2A spec (matches the bridge fix). - agent endpoints test: the runtime invocation path now preserves the top-level 'url' on the stored card, so update the assertion to match. - completion bridge metadata: when forwarding A2A metadata via extra_body.metadata, merge into any existing extra_body.metadata instead of replacing it, so an agent-configured metadata block is preserved (forward metadata still wins on key conflicts). Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(a2a): remove dead duplicate transformation dir; drop SSRF-prone headers field from /v1/a2a/discover Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(a2a): revert accidental html→index.html rename from afc8b10f The commit afc8b10f bundled real A2A fixes alongside an unintended re-introduction of the */index.html layout that 8513d7fc had already reverted. Restore all 35 static-export pages back to the flat *.html structure that matches the upstream main branch. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(a2a): address PR review comments UI: - Auto-trigger discovery when connection details are filled; remove the "Use these selections" button (selection syncs live to parent, user just clicks Next). - Edit Settings: auto-discover upstream card on open; cross-check with DB-stored card so only already-saved skills/capabilities are pre-ticked. - Extract shared buildDiscoveryRequest + selectionsFromSavedAgentCard helpers into agent_discovery_utils.ts so both add and edit flows share the same logic. Backend: - agent_card.py: rename the proxy security requirements field from the non-standard ``securityRequirements`` to the spec-correct ``security`` key (matches AgentCard TypedDict and A2A/OpenAPI convention). - agent_card.py: remove ``securityRequirements`` from _ALLOWED_TOP_LEVEL_KEYS. - endpoints.py: _build_merged_agent_card now forwards agent_name and description from the request so the stored card reflects the admin- supplied name, not just whatever the upstream card advertised. - utils.py: remove overly-broad ``or "parts" in result`` fallback; use ``kind == "message"`` check only to avoid false matches on future result types that happen to include a ``parts`` field. - test_agent_card.py: update assertions to expect ``security`` key. Co-authored-by: Cursor <cursoragent@cursor.com> * fix: restore Next.js metadata directories to match upstream main The previous revert removed __next.* metadata subdirectories from git tracking entirely, but these directories exist on origin/main alongside the flat .html files. Restore them via checkout from origin/main so the PR diff only reflects actual code changes. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(a2a): drop dead headers option from discoverAgentCardCall The backend /v1/a2a/discover endpoint no longer accepts a headers field (removed in 78591b2 for SSRF safety), so any headers passed through DiscoverAgentCardOptions were silently discarded by the API request body. Remove the field and the conditional that copies it onto the request body. * fix(a2a): skip merge for non-A2A agents and align pydantic-ai result shape The agent create/update/patch handlers ran the LiteLLM-fronting merge unconditionally, so registrations that did not provide agent_card_params still ended up with a synthesised card carrying supportedInterfaces, securitySchemes, and default skills. Gate the merge on a non-empty agent_card_params so plain chat/LLM agents stay non-A2A in the registry. Also move kind: 'message' inside the a2a_message dict in the Pydantic AI non-streaming response so its construction matches the completion bridge rather than spreading kind on top of a separate dict. * Fix three bugs in A2A discovery flow 1. UI: Stabilize discoveryRequest deps to avoid redundant /v1/a2a/discover API calls. The parent rebuilds the discoveryRequest object on every form keystroke, so depend on primitive proxies (discovery_mode + serialized params) rather than the object identity. Read the actual object via a ref inside handleDiscover. 2. Backend: Route the well-known card fetch through async_safe_get so the admin /v1/a2a/discover endpoint can't be used to probe private/loopback addresses or cloud metadata endpoints. SSRFError is a separate handled case so it surfaces a clear AgentCardDiscoveryError. 3. Streaming: Make openai_chunk_to_a2a_chunk emit the same flat result shape as the non-streaming response (kind/role/parts/messageId at the result level), with envelope-level 'final' added. Matches the existing create_artifact_update_event pattern and lets consumers read a uniform result shape across streaming and non-streaming. Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(a2a/ui): include savedAgentCard in handleDiscover deps The previous deps list omitted savedAgentCard, so handleDiscover (and the resetSelections it calls) kept the closure's saved-card value even after the parent refetched the agent. Clicking 'Re-discover' would then pre-select skills against stale data. Adding savedAgentCard to the deps array forces the callback to refresh whenever the saved card changes. Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(a2a): align pydantic-ai test + docstring with direct-Message result shape The non-streaming A2A response was changed so that `result` is the Message itself (kind="message"), per spec / SendMessageResponse. Update the PydanticAITransformation._transform_to_a2a_response test and docstring that still described the old `result.message` envelope so internal consumers match the producer. * fix(a2a): strip additionalInterfaces and let configured metadata win over A2A request - merge_agent_card no longer carries upstream additionalInterfaces through; storing those alternate URLs would let authenticated agent callers reach the backend directly and bypass proxy auth/budget/logging. - apply_forward_metadata_to_completion_params now layers client-supplied A2A metadata UNDER any agent-owner-configured extra_body.metadata, so server-set run metadata stays authoritative on key conflicts. * fix(agents): merge agent card even when agent_card_params is an empty dict Treat an explicitly provided empty agent_card_params ({}) as 'card provided but empty' instead of 'no card', so the LiteLLM-fronting merge still injects securitySchemes, supportedInterfaces, and protocolVersion. Without this, the well-known endpoint could serve a bare card with only a rewritten url, advertising no authentication to A2A clients. Co-authored-by: Yassin Kortam <yassin@berri.ai> * refactor(a2a): drop dead openai_chunk_to_a2a_chunk helper The deprecated single-chunk helper has no callers anywhere in the codebase — the streaming path emits proper A2A events via create_task_event / create_status_update_event / create_artifact_update_event in handler.py. Removing the dead method also eliminates the inconsistency where the unused chunk inlined the envelope-level final flag inside the Message result. * fix(a2a): scope a2a lazy-feature so it doesn't subsume /v1/a2a/discover - _lazy_features.py: use /a2a prefix + /message/send suffix for the a2a feature so a request to /v1/a2a/discover no longer triggers the a2a_endpoints module to load alongside a2a_registration. - agent_endpoints/endpoints.py: drop the no-op description override kwarg from _build_merged_agent_card and its three call sites. The upstream card's description is already preserved by merge_agent_card's deepcopy, so passing it explicitly did nothing. * style: black-format litellm/a2a_protocol/litellm_completion_bridge/transformation.py * fix: address PR bugfix review for a2a discovery + metadata forwarding - agent create form (add_agent_form.tsx): drop the skills.length > 0 guard so an admin can clear all discovered skills during creation, matching the edit form's overlay behavior (consistency between create and edit flows). - agent_card_discovery.tsx: stop including savedAgentCard in the handleDiscover useCallback deps. Read it via a ref inside resetSelections instead, so a parent-driven re-render that hands us a new savedAgentCard object reference (e.g. a background refresh of the agent record) does not recreate handleDiscover and re-fire the auto-discover effect, which would otherwise overwrite in-progress user edits in parent-driven mode (debounceMs = 0). - a2a_endpoints.invoke_agent_a2a: skip 'metadata' when moving litellm params off of A2A MessageSendParams into body. The A2A protocol defines params.metadata as a first-class request-level field, and the completion bridge's get_forward_metadata is supposed to merge it with message.metadata. Previously the proxy always stripped params.metadata before constructing MessageSendParams, so the params-level branch in get_forward_metadata was dead code in the proxy flow. Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(a2a): return 404 from get_agent_card when agent has no card * fix(agents): apply discovery overlay uniformly on create and dedupe ALLOWED_CAPABILITY_KEYS - buildAgentData now applies overlayDiscoveredCardParams after every non-custom branch (a2a, use_a2a_form_fields, dynamic) so types with credential_fields no longer silently drop discovered skills, capabilities, input/output modes, provider, and icon/doc URLs on submit. Mirrors the edit flow in agent_info.tsx. - Export ALLOWED_CAPABILITY_KEYS from agent_discovery_utils and import it in agent_card_discovery so the rendering and selection-filtering logic share a single source of truth. Co-authored-by: Yassin Kortam <yassin@berri.ai> * ci(proxy-endpoints): wire tests/test_litellm/proxy/a2a into the shard The two new test files (test_discovery.py, test_agent_card.py) were not picked up by any pytest path, so their coverage never reached codecov and patch coverage fell below the auto target. * fix(ui): overlay discovered name/description in create flow for dynamic agents Mirror the edit-form overlay in agent_info.tsx so dynamic agent types (e.g. LangGraph) whose forms don't register name/description as Form.Items don't silently lose those discovery-panel edits on save. Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(a2a): default merged agent card version, null-guard runtime URL lookup, scope discovery auto-fire to A2A types - merge_agent_card now defaults version to 1.0.0 when upstream omits it (A2A v1.0 schema requires the field). - invoke_agent_a2a guards against agent_card_params being None so plain chat agents routed via the A2A path return a JSON-RPC error instead of AttributeError. - buildDiscoveryRequest no longer falls back to any URL-shaped credential field for non-A2A agent types (Azure AI Foundry, Bedrock AgentCore, Vertex). Discovery only auto-fires for pure A2A and use_a2a_form_fields runtimes; the manual URL input remains available as an escape hatch. * fix(ui): extract overlayDiscoveredCardParams + debounce parent-driven discovery Two findings from greptile review: 1. `overlayDiscoveredCardParams` was copy-pasted between `add_agent_form.tsx` and `agent_info.tsx`. Move it to `agent_discovery_utils.ts` so the create and edit flows share the same overlay logic and there's only one place to update when discovered fields change. 2. `agent_card_discovery.tsx` used a zero-debounce path for parent-driven mode, which fires one discovery HTTP request per keystroke when an admin types into the parent form's URL / api_base / assistant_id fields (the parent rebuilds the plan from watched form values every render). Apply the same 400ms debounce uniformly. * fix(a2a): preserve discovery name edit, default discovery headers, sync url on re-discover - _build_merged_agent_card: prefer card-supplied name over agent_name so the discovery panel's editable 'Name (shown to API clients)' value is not silently overwritten by the internal identifier. - async_safe_get call in fetch_well_known_card: pass headers or {} to avoid TypeError({**None, 'Host': ...}) when URL validation is enabled in production (default). - agent_info handleApplyDiscoveredCard: set url: selection.upstream_url in fieldsToSet so re-discovery during edit refreshes the form's URL field for pure A2A agents (matches add_agent_form). Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(a2a): scrub upstream url from /public/agent_hub cards Public agent_hub returned agent_card_params verbatim, exposing the retained upstream backend url to unauthenticated callers. Rewrite the url to the proxy /a2a/{agent_id} entrypoint on response, matching the behavior of the authenticated well-known agent-card endpoint, so the backend cannot be reached outside LiteLLM's auth, budget, and logging path. * fix(a2a): include suffix-matched routes in lazy warm openapi fragment --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Yassin Kortam <yassin@berri.ai> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>	2026-05-29 20:50:42 -07:00
Ishaan Jaffer	e8461b5b97	style: run black formatter on files from main merge	2026-04-17 13:02:59 -07:00
michelligabrieleandGitHub	a292add9bd	fix(a2a): preserve JSON-RPC envelope for AgentCore A2A-native agents (#25092)	2026-04-03 20:25:32 -07:00
Ishaan JaffandGitHub	b78f4c924c	[Fix] A2a Agent Gateway Fixes - A2A agents deployed with localhost/internal URLs in their agent cards (e.g., http://0.0.0.0:8001/) (#20604) ... * v1 card resolver fix * fix: is_localhost_or_internal_url * fix code * test_fix_agent_card_url_replaces_localhost * test restruct * test_a2a_non_streaming * test agnts * add exception handling * init errors * add localhost retry * add agent_testing * test_a2a_non_streaming * _build_streaming_logging_obj * code qa fixes * test_card_resolver_fallback_from_new_to_old_path * fix linting	2026-02-06 15:02:34 -08:00
Ishaan JaffandGitHub	3ef475b70e	[Fix] A2a Gateway - Allow supporting old A2a card formats (#19949) ... * fix: LiteLLMA2ACardResolver * fix: LiteLLMA2ACardResolver * feat: .well-known/agent.json * test_card_resolver_fallback_from_new_to_old_path	2026-01-28 15:02:08 -08:00
Ishaan JaffandGitHub	3054b6ea60	[Feat] A2A Gateway - allow adding Azure Foundry Agents on UI (#17909) ... * add CostConfigFields * add CostConfigFields * add output_cost_per_token * refactor table * add agent cost view * add azure foundry fields * add foundry logo * fix: clean error * fix utils * fix agent edi * add easter egg * fix order * test_handle_streaming_forwards_api_key * fix forward api key down * fix a2a send msg * add A2a comparison on compare playground * fix chat ui * fix bedrock agentcore stream	2025-12-12 16:38:04 -08:00
Ishaan JaffandGitHub	4a7437ba5f	[Feat] Agent Gateway - allow adding langgraph, bedrock agent core agents (#17802) ... * fix: langgraph bridge streaming * add public/agents/fields * test_a2a_completion_bridge_non_streaming * TestA2AStreamingTransformation * AgentCredentialFieldMetadata * add new logo * refactor add agent * fix add dynamic fields * feat allow adding langgraph agent * add langgraph provider * stash * add AgentCreateInfo * agent_create_fields * fix fields * test_a2a_completion_bridge_bedrock_agentcore * test_a2a_completion_bridge_bedrock_agentcore * add public endpoints * fix a2a endpoints * fix dynamic fields	2025-12-10 19:13:50 -08:00
Ishaan JaffandGitHub	059fedbed5	[Feat] Agent Gateway - Track agent_id in SpendLogs (#17795) ... * add agent_id in metadata in spend logs * add agent_id in SpendLogsPayload * add agent_id in SpendLogsPayload * add _set_agent_id_on_logging_obj * add agent id tracking in SpendLogs * add agent id in spend logs * fix create_a2a_client * test_asend_message_passes_agent_id_to_callback * test_get_logging_payload_includes_agent_id_from_kwargs * test_asend_message_streaming_triggers_callbacks * fix asend_message_streaming * asend_message_streaming * A2AStreamingIterator * _handle_stream_message * test_asend_message_streaming_propagates_metadata	2025-12-10 16:09:56 -08:00
Ishaan JaffandGitHub	49b91c4a35	[Feat] A2a gateway - Add cost per token pricing (#17780) ... * fix calculate_a2a_cost * add cost_per_query * add test_asend_message_uses_cost_per_query * fix: _initialize_slack_alerting_jobs * feat: add token tracking for agents invoke * add A2ARequestUtils * add _set_usage_on_logging_obj * test_asend_message_token_tracking * add _handle_a2a_response_logging * test_asend_message_streaming_token_tracking * add A2AStreamingIterator * add cost calculator for agents * test_asend_message_uses_input_output_cost_per_token * docs gix	2025-12-10 13:08:15 -08:00
Ishaan JaffandGitHub	7a33579af6	[Feat] Agent Gateway - Add cost per query for agent invocations (#17774) ... * fix calculate_a2a_cost * add cost_per_query * add test_asend_message_uses_cost_per_query * fix: _initialize_slack_alerting_jobs	2025-12-10 10:43:52 -08:00