The self-exclusion filter compared raw key strings against SHA-256
hashed tokens from the DB, so keys were never excluded and
double-counting persisted. Now hash data.key before comparison.
Also add tpm_limit_type/rpm_limit_type to _throughput_fields_changed
guard, fall back to existing_key_row.team_id for team limit checks
(matching the org pattern), and add team self-exclusion test.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Only run org validation (get_org_object + _check_org_key_limits) when
the update actually touches throughput-related fields (tpm_limit,
rpm_limit, or organization_id). Previously, any update to a key
belonging to an org would trigger the check, which would fail with a
400 if the org had been deleted — blocking unrelated field changes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When updating a key, _check_org_key_limits and _check_team_key_limits
would include the key being updated in the find_many results, causing
its current limits to be counted twice (once from the DB query, once
from the new requested limits). This caused false 400 errors on valid
limit adjustments.
Fix: exclude the key being updated (by matching token) from the
allocated totals before checking limits.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The previous fix (124b44ec) only updated 3 tests but missed 10 more
that still patched the old `ui_sso.httpx.AsyncClient` path. Also
updated credential assertions to check Authorization header instead
of httpx.BasicAuth kwargs, matching the production code change.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The /key/update endpoint was missing support for organization_id, which
was already available on /key/generate. This adds the field to
UpdateKeyRequest and validates org key limits during updates.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The test_chat_completion_low_budget test was flaky because async spend
tracking couldn't reliably catch up within 50 calls with 0.5s sleeps.
Increased to 200 calls with 0.1s sleeps (same total time budget) to
give more opportunities for budget enforcement to trigger.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The recent commit 2a997993d4 replaced httpx.AsyncClient() with
get_async_httpx_client() in ui_sso.py, but the PKCE tests still
patched the old httpx.AsyncClient path. Updated all 10 affected
tests to mock get_async_httpx_client and removed unnecessary
context manager setup since AsyncHTTPHandler is returned directly.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Use dependency_overrides for user_api_key_auth instead of relying on
uninitialized proxy globals. The auth dependency was crashing with 500
(instead of 401) and returning MagicMock user_id/team_id values that
broke json.dumps in _encode_realtime_token_payload.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The existing AttributeError detection in proxy error handling only
checked one level deep in the exception chain (__cause__, __context__,
original_exception). In practice, the AttributeError from malformed
messages gets wrapped in multiple layers (AttributeError ->
OpenAIException -> APIConnectionError), so the check never found it.
Extracted the check into _has_attribute_error_in_chain() which walks
the full exception chain recursively (depth-capped at 10 to prevent
infinite loops from circular references).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Skip test_apply_patch_tool_call_converted_to_chat_completion_tool_call
when openai.types.responses.response_apply_patch_tool_call is unavailable
(CI uses openai==1.100.1 which doesn't have this module)
- Skip MCP M2M tests (test_m2m_credentials_forwarded_to_server_model,
test_m2m_drops_incoming_oauth2_headers) that fail because PR #23187
changed has_client_credentials to require explicit oauth2_flow opt-in
but _execute_with_mcp_client was not updated to pass it through
- Revert source code change to rest_endpoints.py that auto-inferred
oauth2_flow (regression risk: this changes MCP OAuth behavior)
Co-authored-by: yuneng-jiang <yuneng-jiang@users.noreply.github.com>
- Vertex AI batch cost tests: replace removed gemini-1.5-flash-001 model
with gemini-2.0-flash-001 in pricing lookups
- MCP test_executes_tool_when_allowed: add server_id and auth_type attrs
to StubServer to match new _resolve_allowed_mcp_servers_with_ip_filter
- MCP M2M tests: infer oauth2_flow='client_credentials' in
_execute_with_mcp_client when client_id/client_secret/token_url present
(NewMCPServerRequest lacks oauth2_flow field)
- Team list test: update mock find_many to filter by team_id per the
current per-team query pattern in list_team
- Azure DALL-E 3 health check: skip test due to 410 ModelDeprecated
Co-authored-by: yuneng-jiang <yuneng-jiang@users.noreply.github.com>
Apply the same fix: move litellm.public_model_groups_links assignment
to after get_config()/save_config() so it is not overwritten by the
stale DB value read.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
get_config() internally calls _update_config_from_db which overwrites
litellm.public_model_groups with the stale DB value. Moving the
in-memory assignment to after get_config()/save_config() ensures the
new value persists.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(sso): add direct PKCE token exchange and Redis cache wiring for multi-instance SSO
When PKCE is enabled, bypass fastapi-sso and perform direct token exchange so
code_verifier is correctly included. Store PKCE verifiers as dict in cache
for proper JSON serialization in Redis. Wire user_api_key_cache to Redis when
available so PKCE verifiers are shared across ECS tasks/pods.
Also adds clearer error messages when PKCE is required but not configured.
* refactor(sso): extract PKCE token exchange into SSOAuthenticationHandler methods
- Move import httpx/jwt to module level (top of file, not inside function)
- Extract inline PKCE token exchange + userinfo logic into two static methods:
_pkce_token_exchange() and _get_pkce_userinfo()
- get_generic_sso_response PKCE path is now a single method call
- Fix double-logging in except block for non-PKCE errors
- Use %-style log formatting (no f-strings in log calls)
* fix: address greptile review feedback
- Fix access_token missing in PKCE path: read from combined_response directly
instead of generic_sso.access_token (which is only set by verify_and_process)
- Fix PKCE error hint firing when PKCE is already enabled: only show
'set GENERIC_CLIENT_USE_PKCE=true' advice when code_verifier was absent
- Fix unguarded KeyError on access_token: check for error field in HTTP 200
responses before accessing token_response['access_token']
- Fix silent empty userinfo: raise ProxyException when both userinfo endpoint
and id_token fallback produce no user data
- Fix backward-incompatible Redis wiring: only attach Redis to user_api_key_cache
when GENERIC_CLIENT_USE_PKCE=true, preserving existing in-memory behaviour
* fix: address second round of greptile review feedback
- Fix PKCE error hint: check env var directly (not code_verifier presence) to
distinguish 'PKCE not configured' from 'PKCE enabled but cache miss'
- Fix misleading Redis TTL comment in proxy_server.py
* fix: address third round of greptile review feedback
- Fix CRITICAL log firing on every non-PKCE callback: only log when PKCE is enabled
- Remove unused pkce_env_value intermediate variable
- Prefer reusing redis_usage_cache over creating separate RedisCache instance
(avoids losing advanced connection options like SSL, timeouts, db)
* fix: address fourth round of greptile review feedback
- Strip OAuth token credentials from response_convertor input to prevent
access_token/id_token appearing in restricted-group error messages
- Reuse single httpx.AsyncClient for both token exchange and userinfo requests
to avoid a second TCP/TLS handshake per SSO callback
- Revert Redis wiring to user_api_key_cache: PKCE code already uses
redis_usage_cache directly; wiring would route all API-key lookups through
Redis unnecessarily. Add startup warning instead when PKCE+Redis mismatch.
- Move _OAUTH_TOKEN_FIELDS to module level
* fix remaining PKCE test assertion for dict-format verifier storage
* sanitize PKCE cache log to not expose verifier content
* address greptile review feedback (greploop iteration 3)
* address greptile review feedback (greploop iteration 4)
* address greptile review feedback (greploop iteration 5)
* simplify _get_pkce_userinfo: remove shared-client complexity, use async with directly
* address greptile review feedback (greploop iteration 6)
* address greptile review feedback (greploop iteration 7)
* address greptile review feedback (greploop iteration 8)
* address greptile review feedback (greploop iteration 9)
* address greptile review feedback (greploop iteration 10)
* address greptile review feedback (greploop iteration 11)
* address greptile review feedback (greploop iteration 12)
* fix misleading comment on user_api_key_cache TTL line
* address greptile review feedback (greploop iteration 13)
* address greptile review feedback (greploop iteration 14)
* address greptile review feedback (greploop iteration 15)
* address greptile review feedback (greploop iteration 16)
* address greptile review feedback (greploop iteration 17)
* address greptile review feedback (greploop iteration 18)
* address greptile review feedback (greploop iteration 19)
* address greptile review feedback (greploop iteration 20)
* address greptile review feedback (greploop iteration 21)
* address greptile review feedback (greploop iteration 22)
* address greptile review feedback (greploop iteration 23)
* address greptile review feedback (greploop iteration 24)
* address greptile review feedback (greploop iteration 25)
* address greptile review feedback (greploop iteration 26)
* address greptile review feedback (greploop iteration 27)
* address greptile review feedback (greploop iteration 28)
* address greptile review feedback (greploop iteration 29)
* address greptile review feedback (greploop iteration 30)
* address greptile review feedback (greploop iteration 31)
* address greptile review feedback (greploop iteration 32)
* address greptile review feedback (greploop iteration 33)
* address greptile review feedback (greploop iteration 34)
* address greptile review feedback (greploop iteration 35)
* address greptile review feedback (greploop iteration 37)
- read GENERIC_CLIENT_USE_PKCE env var once in prepare_token_exchange_parameters
- include actual decode error in jwt.decode failure exception message
- add GENERIC_CLIENT_USE_PKCE=true to no-state regression test
* defer PKCE verifier deletion until after all downstream processing
Move _delete_pkce_verifier to after response_convertor and
process_sso_jwt_access_token complete. If JWT processing raises,
the verifier stays in cache so the user can retry without restarting
the full OAuth flow.
* address greptile review feedback (greploop iteration 38)
- fix strict-mode cache miss error message to differentiate
cross-instance routing failures (Redis configured) from single-instance
issues (TTL expiry, pod restart) when only in-memory cache is available
- add comment above _get_pkce_userinfo call explaining that bearer
credentials are always sourced from token_response in the merge step
* fix null JSON response body in _pkce_token_exchange
- Guard against HTTP 200 with body null: response.json() returns None
for JSON null, and calling .get() on None raises AttributeError.
Now raises a clean ProxyException with a clear error message.
- Fix misleading userinfo warning: was always saying "empty dict" but
also fires for JSON null responses; updated to say "empty or null".
- Add HTTP status code assertion to cache miss test.
* address greptile review feedback (greploop iteration 39)
- fix credential leakage: directly assign received_response from
combined_response instead of relying on nonlocal mutation; Pyright
was flagging the old guard as unreachable, meaning credential stripping
might not execute — now it always runs unconditionally
- add test for legacy plain-string cache format backward compat branch
- add test for HTTP 200 with no error field and no access_token (else branch)
- add test for HTTP 200 with JSON null body (new AttributeError guard)
* fix _OAUTH_TOKEN_FIELDS merge loop to preserve userinfo values on absent fields
When the token endpoint omits a bearer-credential field entirely (field
absent from token_response), the previous code deleted it from merged even
if userinfo provided a valid value. Now:
- non-null in token_response → restore authoritative token endpoint value
- explicit null in token_response → remove key from merged (clean absence)
- field absent from token_response → leave userinfo value unchanged
* use HTTP 401 for PKCE missing config errors
GENERIC_CLIENT_ID and GENERIC_TOKEN_ENDPOINT missing when PKCE is
enabled are auth-flow failures, not server errors. Use 401 instead
of 500 to avoid triggering false-positive server error alerts in
monitoring systems.
* address greptile review feedback (greploop iteration 40)
- fix duplicate error logging: demote first format-error log to DEBUG
so the detailed ERROR in strict-mode branch is not duplicated
- add HTTP status code assertions to all PKCE ProxyException tests
for better regression protection against accidental code changes
* add credential absence assertions to test_pkce_token_exchange_basic_auth
Verify that client_id and client_secret are NOT double-sent in the POST
body when Basic Auth is used (include_client_id=False with client_secret).
Catches regressions where credentials leak into both Auth header and body.
* address greptile review feedback (greploop iteration 41)
- add Bearer token header assertion to test_pkce_token_exchange_credentials_in_body
- add cache query assertions to both non-strict mode tests to confirm
the cache was accessed before the warning path triggers
* address greptile review feedback (greploop iteration 42)
- assert null id_token is absent from merged result in basic auth test
- add test for HTTP 200 empty/null userinfo body with no id_token fallback
* use caplog to verify warning logs in non-strict cache miss tests
The two non-strict mode tests now use pytest's caplog fixture to assert
that a warning is actually emitted, not just that the code continues
without raising. This catches regressions where the warning silently
disappears.
* remove dead-code response=None guard in _pkce_token_exchange
* clean up stale pkce verifier cache entries in non-strict mode
* fix test: configure async_delete_cache as AsyncMock and assert cleanup called
* add sentinel guard so pkce-no-redis warning only fires once across hot-reloads
* fix misleading comments: code_verifier init and bearer-credential merge docs
* add best-effort cleanup in strict-mode for corrupt/empty cache entries
* add redirect_uri assertion, userinfo body in non-200 log, sentinel comment
Keep both sets of tests: upstream's OAuth2 token injection test and
our case-insensitive tool matching tests. Use upstream's version of
the bedrock output_config test (more comprehensive).
* fix(proxy): make async_post_call_response_headers_hook consistent across all endpoints
The response headers hook had 5 gaps that prevented callbacks from
reliably extracting routing metadata across endpoint types:
1. Hook never fired for /audio/transcriptions (endpoint bypasses
base_process_llm_request)
2. custom_llm_provider not accessible in hook data for any endpoint
3. custom_llm_provider not stamped in ResponsesAPIResponse._hidden_params
(unlike chat completions)
4. model_info under inconsistent keys (metadata vs litellm_metadata)
5. request_headers always None at all call sites
This adds a litellm_call_info parameter to the hook that normalizes
routing metadata (custom_llm_provider, model_info, api_base, model_id)
regardless of endpoint type. Also stamps custom_llm_provider on
Responses API responses, adds the hook call to the transcription
handler, and passes request_headers at all call sites.
Supersedes PR #21385.
* fix(proxy): address review feedback — safer backwards compat and None guards
- Replace try/except TypeError with inspect.signature() check for
litellm_call_info backwards compatibility. This avoids masking real
TypeErrors inside callback implementations and prevents double
invocation with inconsistent parameters.
- Use (data.get("key") or {}) instead of data.get("key", {}) to guard
against keys that exist with an explicit None value, which would
cause AttributeError on the subsequent .get() call.
* fix(proxy): cache inspect.signature result for callback compat check
Move the inspect.signature() call into a module-level helper with a
dict cache keyed by callback identity. Avoids repeated introspection
per request per callback in the hot path.
* fix(proxy): use class identity for signature cache key
Key the _CALLBACK_ACCEPTS_CALL_INFO cache by id(type(cb)) instead of
id(cb) to avoid stale entries from Python address reuse after GC.
All instances of the same callback class share the same method
signature, so class identity is both safer and more cache-efficient.
- Thread api_version through HTTP handlers to Azure realtime endpoints
- Make expires_at optional in RealtimeClientSecretResponse
- Fix test token expiry times to be in the future
- Populate user_id and team_id in minimal_auth for spend tracking
Made-with: Cursor
- Azure Model Router transform_response: let parent extract actual model from raw response
- common_request_processing: skip model override for Azure Model Router requests
- proxy_server: skip streaming chunk model restamp for Azure Model Router
- Add _is_azure_model_router_request helper
- Add tests for non-streaming and streaming
Made-with: Cursor
* fix(mcp): OAuth2 chat connect - tools fetch, auth flow, and status fixes
- schema.prisma: add missing MCP table fields (approval_status, submitted_by, submitted_at, reviewed_at, review_notes) to prevent destructive migrations
- rest_endpoints.py: inject user OAuth token via extra_headers for OAuth2 servers so tools list is populated; add server name->UUID resolution so MCPConnectPicker name lookups work
- mcp_registry.json: fix Atlassian defaults (transport: http, url: .../v1/mcp)
- ChatPage.tsx: read mcpOauthReturn param to init sidebarView="apps" on OAuth return, clean up param after mount
- MCPAppsPanel.tsx: auto-add OAuth2 servers to selectedServers when credential detected; onConnect also enables server for chat; disconnect removes from selectedServers
- mcp_servers.tsx: sort servers by created_at DESC
- useUserMcpOAuthFlow.tsx: append mcpOauthReturn=apps to return URL so Apps panel is mounted on return
* address greptile review feedback (greploop iteration 1)
* fix(mcp): inject stored OAuth2 token when fetching tools via /responses API
When a user has connected an OAuth2 MCP server (e.g. Atlassian) and then
uses the /responses endpoint with that server, tool listing was failing
because the stored per-user OAuth token was never injected.
Two fixes:
1. server.py: add _get_user_oauth_extra_headers_from_db() helper; call it
in _get_tools_from_mcp_servers when oauth2_headers is None for an OAuth2
server, falling back to the user's stored token in LiteLLM_MCPUserCredentials
2. litellm_proxy_mcp_handler.py: also intercept MCP tools whose server_url
matches */mcp/<server_name> (e.g. http://localhost:4000/mcp/atlassian_test)
by rewriting them to litellm_proxy/mcp/<server_name> so they go through
the internal handler (and get the OAuth token injected) instead of being
forwarded to OpenAI raw where localhost is unreachable
* address greptile review feedback (greploop iteration 2)
* test(mcp): add unit test for OAuth2 token injection in _get_tools_from_mcp_servers
Verifies that when _get_tools_from_mcp_servers is called for an OAuth2 MCP
server without oauth2_headers in the request, the implementation:
- calls _prefetch_oauth_creds_for_user once (not per-server) to avoid N+1 queries
- passes the stored token as extra_headers={"Authorization": "Bearer ..."} to
_get_tools_from_server so the upstream OAuth2 MCP server authenticates correctly
* address greptile review feedback (greploop iteration 3)
* address greptile review feedback (greploop iteration 4)
* address greptile review feedback (greploop iteration 5)
* redesign credentials table to use Tremor table layout matching Keys page
* fix: /server/oauth authorize 422 - make client_id optional, fall back to real DB server
* fix: mcp_token client_id optional, resolve from server record
* fix: look up real server by UUID (get_mcp_server_by_id) before falling back to name
* Update litellm/responses/mcp/litellm_proxy_mcp_handler.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix: address greptile feedback - client_id guards, dict spread, helper refactor, tests
- mcp_management_endpoints: raise 400 when resolved_client_id is empty in
mcp_authorize and mcp_token instead of forwarding "" to upstream
- litellm_proxy_mcp_handler: use {**tool, "server_url": ...} spread instead
of dict(tool) + mutation for shallow copy safety
- rest_endpoints: extract _oauth2_server_ids set comprehension to a named
_get_oauth2_server_ids() helper for clarity; add Set to typing imports
- test_rest_endpoints: add tests for name→UUID resolution path,
access-denied when resolved UUID not in allowed list, and OAuth2 user
token injection for single-server requests; fix fake_get_tools signature
to accept extra_headers kwarg
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
- Mock litellm.completion_cost in test_pass_through_success_handler_gemini_routing
to decouple it from model_prices_and_context_window.json; prevents the same
breakage if gemini-2.0-flash is ever removed from the pricing map
- Revert _create_passthrough_logging_payload URL back to gemini-1.5-flash to
eliminate inconsistency with the other tests that use gemini-1.5-flash explicitly
* fix(test): null AWS SigV4 fields on MagicMock in test_inherit_credentials_from_existing_server
* fix(test): null AWS SigV4 fields on MagicMock in test_add_session_mcp_server_caches_and_redacts_credentials
* fixed mcp api
* added non-admin test
* resolved greptile comemnt
* fix: add IP filtering to get_mcp_server_by_id path in fetch_mcp_server
Apply _is_server_accessible_from_ip check after get_mcp_server_by_id lookup
to prevent external callers from accessing MCP servers configured with
available_on_public_internet=False when they know the server_id.
Made-with: Cursor
* fix(mcp): fix OpenAPI OAuth flow — transport mapping, error messages, and discovery bypass
Three bugs fixed to make the end-to-end OAuth flow work for OpenAPI MCP servers:
1. **Transport mapping in getTemporaryPayload**: `TRANSPORT.OPENAPI` is a UI-only concept;
the backend only accepts `"http"`, `"sse"`, or `"stdio"`. The pre-OAuth temp-session
call was sending `transport: "openapi"` and getting a 422. Fixed by mapping to `"http"`.
2. **deriveErrorMessage handles FastAPI 422 arrays**: FastAPI validation errors return
`detail` as an array of `{loc, msg, type}` objects. The shared error extractor was
returning the array directly, causing `Error: [object Object]`. Fixed to map each
item to its `.msg` field.
3. **Skip OAuth discovery when authorization_url already provided**: `build_mcp_server_from_table`
was unconditionally calling `_descovery_metadata(server_url)` for OAuth servers. For
OpenAPI servers the url is the spec JSON file, not the API base — this caused a timeout
fetching e.g. the GitHub spec (2 MB). Fixed by skipping discovery when `authorization_url`
is already set.
Also: collapsible auth section in MCP server form, "Create OAuth App →" link next to
Client ID when a docs URL is available (e.g. GitHub OAuth App creation page), and
`extractErrorMessage` helper in `useMcpOAuthFlow` for cleaner error display.
* refactor(mcp): extract needs_discovery flag and reduceStaticHeaders helper
* feat(mcp): user OAuth connect flow — OAuthConnectModal, MCPCredentialsTab, useUserMcpOAuthFlow
Adds the user-facing MCP OAuth2 PKCE connect flow:
- OAuthConnectModal: modal that launches the PKCE flow for a user to connect to an MCP server
- MCPCredentialsTab: credentials management tab in the MCP apps panel
- useUserMcpOAuthFlow: hook that handles the full PKCE auth code exchange for user-level connections
- MCPAppsPanel: wires up the new credentials tab and connect modal
- ChatPage: further cleanup after responses-API revert
- db.py / mcp_management_endpoints.py / _types.py: backend support for storing user MCP credentials
* fix(mcp): make client_id optional in /authorize — use server's stored client_id when not provided
* address greptile review feedback
* fix(mcp): narrow bare except to RecordNotFoundError in BYOK credential delete
* refactor(mcp): move inline imports to module level in db.py
* docs(claude): add MCP OAuth, transport mapping, and browser storage patterns
* fix(security): remove accessToken from sessionStorage in OAuth flow state
The LiteLLM API key was being serialised into sessionStorage as part of
StoredFlowState. After the OAuth redirect the component re-mounts with the
same accessToken prop, so it never needed to be stored. Read it from props
in resumeOAuthFlow instead.
* fix(ui): remove duplicate extractErrorMessage, sessionStorage-only in admin OAuth hook, call delete API on disconnect
* fix(ui): guard resumeOAuthFlow against wrong hook instance consuming OAuth result
* fix(ui): separate OAuth result keys per flow, sessionStorage-only, surface revoke errors
* fix(ui): remove dead OAuthConnectModal, revert tsconfig jsx mode to preserve
* fix(mcp): guard BYOK overwrite in oauth credential store, raise clear error when client_id absent
* fix: forward OAuth error params in callback, fix BYOK guard exception handling in db.py
