mirror of
https://github.com/tiennm99/litellm.git
synced 2026-08-06 10:21:32 +00:00
8f41fdbd218341fa4fec33e6df2d6a4e8cd65a8f
3618
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 tin-berriandGitHub
|
21d2c3aa83 | fix(ui): stop MCP playground tool calls from sending twice (#29821) | ||
|
ryan-crabbe-berriandGitHub
|
001bda37d9 | refactor(ui): route query-building networking calls through apiClient (#29815) | ||
|
milan-berriandGitHub
|
1f171ee018 | fix(ui): require new expiration when regenerating an expired key (#29838) | ||
|
tin-berriandGitHub
|
22186f457a | fix(ui): persist Tools-tab MCP OAuth token to DB (#29809) | ||
|
ryan-crabbe-berriandGitHub
|
6955e6f2c2 |
refactor(ui): route behavior-preserving networking calls through apiClient (#29806)
* refactor(ui): route callbacks/nudges calls through apiClient * refactor(ui): route alerting + key/user/team delete calls through apiClient * fix(ui): late-bind fetch in apiClient so global.fetch swaps take effect createApiClient captured fetch at construction time, so reassigning global.fetch (as tests do) had no effect and a real network call leaked. Resolve fetch per request instead; harmless in production where fetch is never swapped, and required for apiClient-based calls to be testable. * refactor(ui): route behavior-preserving networking calls through apiClient Collapse ~61 hand-rolled fetch() calls whose semantics already match the shared apiClient (auth header, JSON body, json-error + deriveErrorMessage + handleError) into apiClient.get/post/etc. Query-string builders and the divergent error-handling functions (no-check, custom messages, text-error) are intentionally left for a follow-up normalization pass, since converting them changes wire encoding or error behavior. Prunes the now-stale no-restricted-syntax suppressions for the removed fetch calls. * refactor(ui): convert remaining admin/guardrail GETs, guard late-bind fetch Routes adminspendByProvider, adminGlobalActivity, and the three guardrail submission calls (list/approve/reject) through apiClient so they match their already-converted siblings instead of staying on raw fetch. Adds a client.test.ts case that swaps globalThis.fetch after createApiClient() and asserts the swap takes effect, which fails on the pre-fix captured-fetch line and locks in the per-call resolution |
||
|
Mateo WangandGitHub
|
4ec4ab99d0 | feat(mcp): per-server env vars with global + per-user scopes (#28917) | ||
|
ryan-crabbe-berriandGitHub
|
e53bd7cbd1 |
feat(ui): generate dashboard API types from the proxy OpenAPI spec (#29816)
* feat(ui): generate dashboard API types from the proxy OpenAPI spec Introduces the shared type foundation for the dashboard without touching any runtime code. The proxy's FastAPI app is the source of truth; app.openapi() emits the spec and openapi-typescript turns it into src/lib/http/schema.d.ts. Adds an npm run gen:api script (a Python spec dump piped into openapi-typescript) and a Check UI API Types Sync CI job that regenerates the file from the live spec and fails if it drifts, so the committed types can never silently fall out of step with the backend. The generated file is pinned to openapi-typescript 7.13.0 and excluded from prettier, eslint, and knip, and marked linguist-generated so it collapses in diffs. No openapi-fetch and no call-site changes yet; this only makes the types exist. * chore(ui): tidy gen-api-types script per review Write the spec dump inside a with-block and clean up the temp dir in a finally, so repeated local runs don't leave stray ~MB JSON files behind. |
||
|
+9
|
d671a09c20 |
Litellm oss staging 050626 (#29774)
* Mark xAI models retiring on 2026-05-15 (#28788) Per https://docs.x.ai/developers/migration/may-15-retirement, xAI is retiring the following slugs on 2026-05-15 (auto-redirect to grok-4.3 with various reasoning efforts; callers continuing to use the old slugs will be billed at grok-4.3 pricing): grok-4-1-fast-reasoning{,-latest} -> grok-4.3 (low effort) grok-4-1-fast-non-reasoning{,-latest} -> grok-4.3 (none) grok-4-fast-reasoning -> grok-4.3 (low effort) grok-4-fast-non-reasoning -> grok-4.3 (none) grok-4-0709 -> grok-4.3 (low effort) grok-code-fast-1{,-0825} -> grok-build-0.1 grok-3 -> grok-4.3 (none) Only the direct xai/ slugs are tagged; third-party hosts (azure_ai, oci, vercel_ai_gateway, perplexity/xai) run their own schedules. The grok-3 retirement list explicitly names only the base grok-3 slug — the -mini / -fast / -beta / -latest variants are not listed, so they remain untouched. * feat(moonshot): advertise json_schema response support on live models (#29683) litellm.responses() already routes Moonshot through the responses->chat-completions bridge, and Moonshot honors response_format json_schema on chat completions. The cost-map entries left supports_response_schema unset, so discovery layers that gate on that flag dropped Moonshot from structured-output / responses listings even though the capability works end to end. Set supports_response_schema on the nine models currently live on api.moonshot.ai: kimi-k2.5, kimi-k2.6, the moonshot-v1 8k/32k/128k text and vision-preview variants, and moonshot-v1-auto. Verified against the live API that each honors json_schema and that litellm.responses() returns schema-valid structured output through the bridge. * chore(moonshot): mark models retired from api.moonshot.ai as deprecated (#29685) Thirteen Moonshot/Kimi models in the cost map no longer resolve on api.moonshot.ai (all return 404). Stamp each with its deprecation_date from platform.kimi.ai/docs/models rather than deleting the entries, so historical cost calculation keeps resolving the names while tooling can surface the retirement. Dates: kimi-thinking-preview 2025-11-11; kimi-latest and its 8k/32k/128k context variants 2026-01-28; the kimi-k2 preview/turbo/thinking series 2026-05-25; the moonshot-v1 -0430 snapshots use their own 2024-04-30 snapshot date (Moonshot publishes no discontinuation date for them). * fix(moonshot): drop temperature for reasoning models (kimi-k2.5/k2.6) (#29687) Kimi reasoning models reject every temperature except 1; a request with temperature=0.2 returns "invalid temperature: only 1 is allowed for this model". litellm only clamped temperature into [0.3, 1], so any value below 1 still 400'd. Drop the temperature param entirely for reasoning models (gated on supports_reasoning, the same signal transform_request already uses) so the model default is used; the non-reasoning moonshot-v1 models keep the existing clamp. Co-authored-by: Sameer Kankute <sameer@berri.ai> * feat(mcp): add per-server timeout configuration (#29672) * feat(mcp): add per-server timeout configuration * fix(mcp): address timeout field review comments - use is not None guard instead of or for 0.0 edge case - copy timeout in both LiteLLM_MCPServerTable constructions (health check path + _build_mcp_server_table) - add timeout Float? column to all three schema.prisma files - extend round-trip test to cover _build_mcp_server_table direction - add test for zero timeout not treated as falsy * fix(mcp): forward timeout in _build_temporary_mcp_server_record * fix(mcp): return 504 instead of 500 when per-server timeout fires * test(mcp): add 504 timeout regression test; fix black formatting * Add jp. Bedrock cross-region inference profile for claude-opus-4-7 (#28567) * fix(thinking): handle None thinking param in is_thinking_enabled (#28598) Squash-merged by litellm-agent from Terrajlz's PR. * feat(helm): support tpl rendering in podAnnotations (#28609) Squash-merged by litellm-agent from devauxbr's PR. * Forward custom_llm_provider through the Responses API bridge (Fixes #28505) (#28575) * Forward custom_llm_provider through the Responses API bridge (Fixes #28505) When a Chat Completions request to a GPT-5.4+ model contains both `tools` and `reasoning_effort`, `completion()` auto-routes through `responses_api_bridge`. The bridge handler called `litellm.responses()` / `litellm.aresponses()` without forwarding the already-resolved `custom_llm_provider`, so the downstream call re-invoked `get_llm_provider()` with `custom_llm_provider=None` and stripped a second provider prefix from a `provider/provider/model` deployment string. For a deployment configured as `openai/openai/openai/gpt-5.5`, the bridge flow sent `openai/gpt-5.5` to the upstream API instead of the correct `openai/openai/gpt-5.5`. Upstream APIs that enforce model-name allow-lists rejected this as `key_model_access_denied`. Fix: pass the locally-resolved `custom_llm_provider` into both the sync `responses()` and async `aresponses()` calls so the downstream `_resolve_model_provider_for_responses` sees an explicit provider and skips the second prefix-strip. New regression test `tests/test_litellm/completion_extras/test_responses_bridge_provider_propagation.py` pins both call sites: each must forward `custom_llm_provider`. * fix(28505): set custom_llm_provider on request_data instead of as duplicate kwarg Greptile flagged that the previous patch passed custom_llm_provider as an explicit kwarg to responses()/aresponses() while request_data already carried it via the spread of sanitized_litellm_params, which would raise TypeError: got multiple values for keyword argument on every real bridge call. Switches to assigning request_data['custom_llm_provider'] before the call so the resolved provider wins over whatever sanitized_litellm_params spread in, without duplicating the kwarg. Updates the regression test to seed request_data with a sentinel custom_llm_provider so it actually exercises the overwrite path (the previous test mocked transform_request with a minimal dict and never hit the conflict). * chore: trigger shin-agent re-eval on retargeted staging base * chore: trigger shin-agent re-eval against updated Greptile state * Add jp. Bedrock cross-region inference profile for claude-opus-4-7 AWS Bedrock documents jp.anthropic.claude-opus-4-7 alongside the existing us./eu./au./global. profiles for Claude Opus 4.7 (ap-northeast-1 Tokyo / ap-northeast-3 Osaka), but the entry is missing from model_prices_and_context_window.json. Tokyo-region users currently get an "unknown model" error when routing through the JP geo profile. Adds the entry to both the canonical file and the bundled backup, mirroring the recent pattern for sonnet-4-6 (#27831). Pricing matches the other regional profiles (10% premium over base/global). Regression test pins all six documented profiles (base, global, us, eu, au, jp) and asserts pricing parity between jp. and au. variants. Source: https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-anthropic-claude-opus-4-7.html --------- Co-authored-by: Terrajlz <info@jouleselectrictech.com> Co-authored-by: Bruno Devaux <devaux.br@gmail.com> Co-authored-by: Sameer Kankute <sameer@berri.ai> * feat(soniox): add soniox audio transcription integration (#29508) * feat(openmeter): add OPENMETER_TRUST_REQUEST_USER to prevent forged attribution (#29650) The OpenMeter callback resolves the CloudEvent subject from kwargs["user"] first, then falls back to the key-bound user_api_key_user_id. For multi-tenant proxy deployments, a client can set `"user": "..."` in the request body and cause their usage to be attributed to that arbitrary string — a billing-attribution forgery risk. Adds OPENMETER_TRUST_REQUEST_USER env var (default "true" for backward compatibility). When set to "false", the request-supplied `user` field is ignored and the subject is resolved solely from user_api_key_user_id. Matches the existing env-var-driven config pattern in this file (OPENMETER_API_KEY, OPENMETER_API_ENDPOINT, OPENMETER_EVENT_TYPE). * feat(search): add you_com as a search provider (#28370) * feat(search): add you_com as a search provider Registers You.com Search API as a first-class `search_provider` in the `search_tools` registry, alongside Tavily, Exa, Perplexity, etc. - New adapter: litellm/llms/you_com/search/transformation.py - POSTs to https://ydc-index.io/v1/search - Auth: X-API-Key from YOUCOM_API_KEY (or explicit api_key) - Maps Perplexity unified spec: max_results -> count, search_domain_filter -> include_domains, country -> country - Flattens results.web + results.news into a single SearchResult list; snippet prefers snippets[0], falls back to description; page_age -> date - Registry: SearchProviders.YOU_COM in litellm/types/utils.py and wired into ProviderConfigManager.get_provider_search_config() - Pricing entry: model_prices_and_context_window.json (placeholder $0.0; happy to adjust to maintainers' preferred public number) - Docs: example router config snippet and example proxy yaml updated - Tests: tests/search_tests/test_you_com_search.py - 5 mocked tests (payload shape, domain filter mapping, snippet fallback, news flattening, missing-api-key error) Refs upstream expansion signal: #15942 * review fixups: normalize api_base, lowercase country, scope env-var to test Addresses Greptile inline review comments on #28370: - get_complete_url: strip trailing slashes from api_base *before* the endswith("/v1/search") check, so a custom base like ".../v1/search/" doesn't become ".../v1/search/v1/search". - transform_search_request: .lower() country before sending, matching Tavily's convention so callers using the unified spec form ("US") get consistent behavior across providers. - Tests: replace direct os.environ writes with an autouse monkeypatch fixture so YOUCOM_API_KEY is set per-test and removed afterwards. The missing-key test now uses monkeypatch.delenv. New test asserts the trailing-slash normalization above. Reverts the ARCHITECTURE.md / example yaml edits per the reviewer note that documentation changes belong in the litellm-docs repo. * support keyless free tier (api.you.com/v1/agents/search) as default You.com offers an IP-throttled keyless endpoint that returns the same response shape as the keyed one (~100 queries/day, no signup). This is a significant onboarding lever - mirrors the keyless DuckDuckGo/SearXNG providers already in the search_tools registry. Behavior: - YOUCOM_API_KEY set -> keyed: POST https://ydc-index.io/v1/search (X-API-Key header) - no key -> free: POST https://api.you.com/v1/agents/search (no auth) - YOUCOM_API_BASE override -> honored as-is Tests: - New: test_you_com_search_keyless_free_tier - asserts URL + absence of X-API-Key when no key is configured. - New: test_you_com_search_validate_environment_keyless - asserts the config no longer raises when the key is absent. - Removed: test_you_com_search_raises_without_api_key (the precondition no longer holds). - Existing payload/domain-filter/etc tests still cover keyed mode via the autouse YOUCOM_API_KEY fixture. Verified both endpoints accept POST + return identical JSON shape: results.web[] / results.news[] with title, url, snippets, description, page_age. * register you_com in provider_endpoints_support.json Adding `litellm/llms/you_com/` requires a corresponding entry in provider_endpoints_support.json or the code-quality/check_provider_folders_documented CI check fails. Follows the compact tavily/serper pattern - endpoints: { search: true }. Local run of the check now reports "All 114 provider folders are documented". * move tests under tests/test_litellm/llms/ so CI exercises them The litellm CI workflows scope unit tests to `tests/test_litellm/...` (see test-unit-llm-providers.yml: `tests/test_litellm/llms` path), so tests living under `tests/search_tests/` are never run in CI - which is why codecov reports 0% patch coverage for the new adapter even though the unit tests exist and pass locally. Move test_you_com_search.py into `tests/test_litellm/llms/you_com/` so the test-unit-llm-providers job picks it up. 7/7 tests still pass at the new location. (Sibling search-only providers - tavily, exa_ai, brave, etc. - still live only in `tests/search_tests/` and would benefit from the same move, but that is out of scope for this PR.) * fix(you_com): pin Accept-Encoding: identity to dodge keyless gzip bug The keyless free-tier endpoint (api.you.com/v1/agents/search) advertises Content-Encoding: gzip but returns a body that httpx's decoder rejects with `zlib.error: Error -3 while decompressing data: incorrect header check`, surfacing as litellm.APIConnectionError in user code. curl works because it doesn't request compression by default. Pin Accept-Encoding: identity in validate_environment so the upstream server skips compression entirely. Harmless on the keyed endpoint (ydc-index.io/v1/search) which negotiates content-encoding correctly. The header uses setdefault so a caller-supplied Accept-Encoding still takes precedence. (Server-side bug has been flagged to the You.com team separately - once fixed there, this workaround can be removed.) New unit test: test_you_com_search_pins_identity_accept_encoding. --------- Co-authored-by: Sameer Kankute <sameer@berri.ai> * docs: fix README typo (#29419) Correct clear spelling mistakes in documentation without changing behavior. Confidence: high Scope-risk: narrow Tested: git diff --check; uvx codespell on changed files Not-tested: Full docs build not run; text-only changes * Fix(langfuse): pass httpx_client to Langfuse in langfuse_prompt_management to respect SSL_VERIFY (#29480) * fix(langfuse): pass ssl_verify to Langfuse httpx client * fix_langfuse_ * add unit tests * addressed comments --------- Co-authored-by: shin-berri <shin-laptop@berri.ai> Co-authored-by: yuneng-jiang <yuneng@berri.ai> * feat(models): add minimax/MiniMax-M3 to model cost map (#29412) Add MiniMax's new flagship MiniMax-M3 to the native minimax provider: 512K context, 128K max output, native multimodal (supports_vision), reasoning, prompt caching. Pricing (USD/M tokens): input 0.6 / output 2.4 / cache read 0.12. M3 has no active prompt-cache-write tier, so cache_creation_input_token_cost is omitted. Updated both the root model_prices_and_context_window.json (remote source) and the bundled litellm/model_prices_and_context_window_backup.json (local fallback), keeping them in sync. * fix(logging): handle ResponseCompletedEvent in anthropic_messages streaming spend log (#29394) * fix(logging): handle ResponseCompletedEvent in anthropic_messages streaming spend log * fix(logging): extend terminal event handling to ResponseIncompleteEvent and ResponseFailedEvent; fix return type annotation * feat(provider): Add Neosantara provider as OpenAI Compatible (#29646) * Add Neosantara provider * Register Neosantara provider enum * Address Neosantara provider review feedback * Add Neosantara packaged endpoint support --------- Co-authored-by: shin-berri <shin-laptop@berri.ai> Co-authored-by: yuneng-jiang <yuneng@berri.ai> * fix: address greptile and veria review feedback - langfuse: guard httpx_client injection behind version check (>= 2.7.3) - soniox: propagate audio_transcription_duration in _hidden_params for spend tracking - soniox: give SONIOX_API_BASE env var priority over caller-supplied api_base - mcp: replace CancelledError catch with asyncio.wait_for + TimeoutError * chore(mcp): add migration for per-server timeout column * fix(test): add tool_use_system_prompt_tokens to model prices schema validator * fix: mcp timeout test uses real asyncio.wait_for timeout; you_com get_complete_url respects resolved api_key * fix: forward resolved api_key into you_com endpoint selection and apply timeout to soniox polling GETs The search flow resolves api_key in validate_environment but never passed it into get_complete_url, so a programmatic api_key (with no YOUCOM_API_KEY in the env) set the X-API-Key header yet still selected the keyless free-tier endpoint. Forward api_key through both the search entrypoint and the http handler so the keyed endpoint is chosen. HTTPHandler.get/AsyncHTTPHandler.get had no timeout parameter, so the Soniox poll and transcript-fetch GETs silently used the client global default instead of the caller timeout. Add a per-request timeout to get() and forward the configured timeout from the Soniox handler. * fix(soniox): price stt-async-v4 per second so transcriptions are billed The handler stores audio_transcription_duration in _hidden_params, but the model carried only token cost fields and the response has no token usage, so the transcription cost path fell through to cost_per_second and returned $0. An authenticated caller could transcribe Soniox audio without decrementing their budget. Switch the entry to output_cost_per_second at Soniox's published $0.10/hour async rate so the stored duration produces a real charge. * fix(langfuse): use a dedicated httpx client for the SDK injection The httpx_client handed to the Langfuse SDK came from _get_httpx_client(), which returns LiteLLM's globally cached HTTPHandler. If Langfuse closed that client on teardown it would invalidate the shared client used by every other LiteLLM HTTP call. Build a dedicated httpx.Client instead, still resolving SSL verification and client certificate from LiteLLM's configuration. * fix(soniox): prefer caller-supplied api_base over SONIOX_API_BASE env var * fix(cohere): support max_completion_tokens on cohere v2 chat (default route) (#29779) * fix(cohere): support max_completion_tokens on cohere v2 chat The default cohere_chat route resolves to CohereV2ChatConfig, which did not list or map max_completion_tokens, so get_optional_params raised UnsupportedParamsError for the standard OpenAI parameter (the modern replacement for the deprecated max_tokens). The v1 config already maps it to cohere's max_tokens; mirror that in v2 and add v2 regression tests. * fix(cohere): make max_completion_tokens take precedence over max_tokens on v2 When both max_tokens and max_completion_tokens are supplied, prefer max_completion_tokens explicitly rather than relying on dict iteration order, and cover both orderings with a regression test. --------- Co-authored-by: Daniel Yudelevich <4537920+yudelevi@users.noreply.github.com> Co-authored-by: hectorc98 <hector.chamorroalvarez@adyen.com> Co-authored-by: Filippo Menghi <113345637+Cyberfilo@users.noreply.github.com> Co-authored-by: Terrajlz <info@jouleselectrictech.com> Co-authored-by: Bruno Devaux <devaux.br@gmail.com> Co-authored-by: Dan Lemon <dan@danlemon.com> Co-authored-by: Saswat <saswatds@users.noreply.github.com> Co-authored-by: Brian Sparker <brainsparker@users.noreply.github.com> Co-authored-by: Zhao73 <156770117+Zhao73@users.noreply.github.com> Co-authored-by: Urain Ahmad Shah <60431964+urainshah@users.noreply.github.com> Co-authored-by: shin-berri <shin-laptop@berri.ai> Co-authored-by: yuneng-jiang <yuneng@berri.ai> Co-authored-by: kape <168134658+kapelame@users.noreply.github.com> Co-authored-by: danisalvaa <159898202+danisalvaa@users.noreply.github.com> Co-authored-by: Just R <remixingmagelang@gmail.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> Co-authored-by: abhay23-AI <abhaytrivedi22@gmail.com> |
||
|
ryan-crabbe-berriandGitHub
|
4a5644d51e |
refactor(ui): centralize proxy base URL resolution into tested resolver (#29793)
* refactor(ui): centralize proxy base URL resolution into tested resolver The API base URL join logic was hand-rolled inside networking.tsx and re-derived inline at hundreds of call sites, with no test coverage and a latent double-slash bug when the base carried a trailing slash. This pulls the join into a single pure resolveApiBase() with full unit coverage and routes the existing resolution through it, also de-duplicating the env precedence ladder that was copied in two places. * test(ui): assert root-path redirect joins prefix exactly once The existing toContain check accepts a doubled separator; tighten it to a strict prefix match plus a no-double-slash assertion so a regression in the resolveApiBase origin+SERVER_ROOT_PATH join is caught end-to-end. |
||
|
tin-berriandGitHub
|
a4f57032e0 |
fix(ui): route MCP playground auth by oauth2 mode instead of token_url (#29714)
Interactive PKCE and OBO servers were mislabeled as M2M, so passthrough never showed the Authorize gate; classify by oauth2_flow + delegate_auth_to_upstream instead. |
||
|
|
89f177b7b6 |
fix(galileo): use ingest traces API and standard logging payload (#29651)
* fix(galileo): use ingest traces API and standard logging payload Switch hosted Galileo logging to /ingest/traces with nested trace/span payloads, read metrics from standard_logging_object, and include cost and total tokens on trace metrics. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(galileo): route username/password auth to v2 traces ingest Hosted Galileo no longer serves /observe/ingest; JWT login should post the same trace payload to /v2/projects/{project_id}/traces. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(galileo): address Greptile review on logging and timestamps Use debug-level logs for per-request Galileo callback messages and fall back to start_time/end_time when standard_logging_object omits startTime/endTime. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(galileo): add Galileo to proxy UI callback configuration Expose Galileo in the admin callback selector and config APIs so credentials can be configured through the dashboard instead of YAML only. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(galileo): align response type logging with Langfuse Mirror Langfuse input/output handling for rerank, speech, transcription, realtime, pass-through, and other response types so Galileo ingest no longer skips supported call types. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(galileo): redact trace payload in debug logs and format with black Avoid logging prompts and model responses in flush debug output while keeping structural metadata for troubleshooting. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(galileo): stop logging full trace payload in debug output Log only flush URL and trace count so prompts and model responses are not written to application logs when debug logging is enabled. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix Galileo token totals and prompt messages --------- Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
|
56aa55b991 |
fix(proxy): stop team BYOK model name corruption on model edit (#29731)
* fix(proxy): stop team model name corruption on edit (#28382) (#29001) Team-scoped ("Team-BYOK") models store an internal routing key model_name_{team_id}_{uuid} in the model_name column and the user-facing name in model_info.team_public_model_name. The internal name leaked into /v1, /v2, and /model/info responses; the dashboard bound its edit form to it, so any non-rename save (e.g. a TPM tweak) PATCHed the internal name back. The update path then treated it as a rename, overwriting team_public_model_name and rewriting the team's models[] ACL with the mangled string -- breaking team key calls with team_model_access_denied. Two-layer fix: - Read path (root cause): add _translate_model_name_for_response and apply it in model_info_v2 and _get_proxy_model_info so /v1, /v2, and /model/info surface the public name for team-scoped rows. The DB column and router index keep the internal name as the routing key; this is a presentation-layer swap on a shallow copy (never mutates input). - Write path (defense in depth): harden _get_public_model_name so a value matching the internal shape, or a no-op against the current DB column, is never treated as a rename -- for both the top-level model_name and an explicit model_info.team_public_model_name. Tests: regression for the reported scenario, full branch coverage of _get_public_model_name, two internal-shape guard cases, an end-to-end PATCH through _update_team_model_in_db (asserts the team ACL is untouched), and four response-translation cases. 60 passed (model management), 181 passed (proxy server). * fix(ui): key Agent Builder agent selection on model_info.id (#29729) * fix(ui): key Agent Builder agent selection on model_info.id Once team-scoped BYOK models can share a public name (the backend now returns the public name on /model/info instead of the internal routing key), selecting agents by model_name collides. Key selection, create, update and delete on the stable model_info.id instead, falling back to model_name only for config-defined agents that have no id. * fix(ui): add name-match fallback to post-create agent selection If the just-created agent's id is not yet present in the re-fetched list, try matching by name before falling back to the first agent. Addresses greptile review on #29729. --------- Co-authored-by: tushar8408 <32977767+tushar8408@users.noreply.github.com> |
||
|
ryan-crabbe-berriandGitHub
|
f3811ce63b |
refactor(ui): shared HTTP client + location-pinned fetch() lint rule (#29723)
* refactor(ui): add shared HTTP client and pin raw fetch() to one file Introduce src/lib/http/client.ts, a single typed wrapper that owns the only fetch() in the dashboard. It centralizes the base URL, the auth header, error parsing (deriveErrorMessage), non-2xx -> thrown ApiError, and JSON parsing, and is framework-agnostic (no React) so it can run from client and, later, server components. The base URL, auth header name and the logout side effect are injected through createApiClient. networking.tsx builds one configured apiClient and the 29 functions whose boilerplate maps exactly to the client's default behavior (canonical deriveErrorMessage + handleError + res.json() template) now call it instead of hand-rolling fetch. Names, signatures, return types and error behavior are unchanged; this is a pure refactor that drops ~440 lines. The no-restricted-syntax fetch rule now points at the client and a files: ["src/lib/http/**"] override makes that the only place fetch() is allowed. Re-baselined eslint-suppressions.json: networking.tsx fetch suppressions drop 270 -> 241; no other rule's counts change. The remaining networking.tsx fetches and the ~61 scattered component/hook fetches diverge from the default client behavior (text() error bodies, no res.ok check, no handleError side effect) and stay grandfathered for a follow-up burndown. * fix(ui): make the HTTP client tolerate non-JSON error bodies The non-2xx branch parsed the error body with response.json(), so a gateway returning HTML (502/503 from a reverse proxy) threw a SyntaxError before onError fired or ApiError was built, dropping the user-facing notification. This matched the old per-function behavior, but the client is now the single error path so it is the right place to harden. Read the body as text once, try JSON.parse for the existing deriveErrorMessage path, and fall back to the raw text (or the HTTP status) otherwise. The success path stays strict json() so return types are unchanged. * fix(ui): await the returned apiClient promise in 6 migrated functions The codemod rendered the `return response.json()` tail as `return apiClient.x()` without `await`. Inside the surrounding try/catch that returns an unawaited promise, so the catch never runs and its console.error log is dropped on failure; 4 of the 6 were `return await response.json()` originally, so this restores their exact behavior. Use `return await apiClient.x()` in all six. * refactor(ui): widen onError type and handle empty success bodies Address review notes on the shared client. Type onError as (message: string) => void | Promise<void> so the fire-and-forget async contract (networking passes the async handleError) is explicit rather than silently discarded by void. On the success path, read the body as text and return undefined for an empty body (e.g. a 204 No Content) instead of throwing a SyntaxError, while still parsing non-empty bodies strictly so a malformed JSON response surfaces rather than being masked. Add tests for the 204 case. |
||
|
ryan-crabbe-berriandGitHub
|
41e90a6ada |
chore(ui): remove the bare-fetch lint rule (#29712)
* fix(ui): only flag bare fetch() outside React Query queryFn/mutationFn The frontend lint rule banned every fetch() call by static AST name match, so a fetch wrapped in a React Query queryFn/mutationFn tripped it just like a loose fetch in a component. esquery (no-restricted-syntax) can't express "has ancestor", so this replaces that selector with a small custom rule (local/no-bare-fetch) that exempts a fetch lexically inside a queryFn or mutationFn and reports everything else. Re-baselined eslint-suppressions.json under the new rule id (same 44 files / 331 violations) so existing code keeps its grandfathered suppressions. Adds a RuleTester suite covering wrapped (valid) vs unwrapped, the standalone *Api.ts function pattern, queryKey, and computed-key cases. * chore(ui): remove the bare-fetch lint rule Drop the fetch lint gate (and its 331 grandfathered suppressions) ahead of the networking refactor. The plan is to centralize all fetching in a single shared http client and enforce that with a location-based rule, so keeping a fetch rule in place now would only block CI while functions are routed through the new client. Removing it unblocks that work; the location-based rule lands with the client in a follow-up. |
||
|
ryan-crabbe-berriandGitHub
|
7edf3a9cb5 |
style(ui): run prettier --write across the dashboard (#29622)
Formatting-only pass; no logic changes. Brings the UI into compliance with .prettierrc so the new format-check CI job passes |
||
|
ryan-crabbe-berriandGitHub
|
443f0ca4cd |
ci(ui): frontend-lint job enforcing prettier + eslint on changed files (#29633)
* ci(ui): add frontend-lint job enforcing prettier and eslint on changed files
Lints only the files a PR adds or modifies under ui/litellm-dashboard,
so new and touched code must be prettier-clean and eslint-clean while the
existing tree is grandfathered. Skips cleanly when a PR touches no
lintable UI files. This lets us adopt the formatters incrementally
without a repo-wide reformat
* ci(ui): write frontend-lint file lists to $RUNNER_TEMP
Keep the prettier/eslint changed-file lists out of the checkout dir so
they cannot collide with a future source file of the same name
* lint(ui): baseline existing eslint findings so only new ones block
Capture the current error-level eslint findings (318 across 183 files)
in a committed suppressions baseline via eslint --suppress-all. Every
rule stays at its error severity, so any newly introduced violation
fails the frontend-lint gate, while the existing tree is grandfathered;
touching a legacy file never forces fixing its pre-existing issues. CI
runs eslint with --pass-on-unpruned-suppressions so that fixing a
baselined issue does not fail on a now-stale suppression, and the
generated baseline is prettier-ignored since eslint owns its format.
Burn the baseline down over time with eslint --prune-suppressions
* lint(ui): enforce a count budget for explicit any
Make @typescript-eslint/no-explicit-any a warning and cap the total
instead of hard-blocking each new one. A frontend-lint step counts the
repo-wide explicit any and fails only when it exceeds the committed
budget in eslint-any-budget.json. max starts at 2031, ten above the
current 2021, so the next ten land as warnings and the build fails once
that headroom is gone. Lower max over time toward target to ratchet the
count down. New anys still surface as warnings on changed files via the
normal eslint step
* lint(ui): enable zero-cost rules no-var, no-self-assign, react/no-danger
These have no existing violations, so they need no baseline; turning them
on purely blocks new instances. react/no-danger guards against new
dangerouslySetInnerHTML (XSS), no-var enforces let/const, and
no-self-assign catches self-assignment typos. no-debugger is already
enforced by the recommended preset
* lint(ui): add baselined complexity rules
Enable complexity:20, max-depth:4, max-params:4, max-nested-callbacks:4,
with thresholds set near the codebase p99 so only genuine outliers are
flagged. The 272 existing over-threshold functions are grandfathered in
the suppressions baseline; new over-threshold functions block. Lower the
thresholds over time to ratchet complexity down. max-lines-per-function
is intentionally left off since React components are legitimately long
* lint(ui): ban new raw fetch, standardize on React Query
Add a no-restricted-syntax rule flagging bare fetch() calls, pointing
contributors at React Query (@tanstack/react-query). The rule is not
exempted anywhere, including the already-bloated networking.tsx, so all
331 existing fetch calls are grandfathered but no new ones can be added
there or elsewhere. New data access goes through React Query, and the
networking layer can be migrated out and pruned from the baseline over
time
* lint(ui): ban new @tremor/react imports
Add a no-restricted-imports rule flagging imports from @tremor/react so
tremor is phased out rather than spread further. The 232 existing tremor
imports are grandfathered in the baseline; new ones block and point at
antd. Migrate components off tremor and prune the baseline over time
* lint(ui): widen explicit-any budget headroom to 2040
Raise max from 2031 to 2040, giving ~19 of slack over the current 2021
instead of 10
* style(ui): prettier-format eslint.config.mjs
The frontend-lint gate flagged its own config file. Format it so the
prettier check on this PR's changed files passes
* lint(ui): soften complexity and max-depth to warnings
These two are smell metrics with arbitrary thresholds where a legit new
function can trip them, so make them advisory rather than hard-blocking.
They drop out of the baseline (now 963). max-params, max-nested-callbacks,
and the react-hooks rules stay strict since those are clear-cut
* lint(ui): move complexity and max-depth to the count-budget pattern
Generalize the explicit-any budget into a shared lint-budget mechanism:
eslint-budgets.json maps a rule to {max, target} and check-lint-budgets.mjs
counts each across the repo and fails when a count exceeds its max.
complexity (129, max 140) and max-depth (61, max 70) now use the same
slack-plus-counter model as explicit-any (2021, max 2040): they warn
per-file and the build only fails if the repo-wide total crosses the
ceiling. Lower each max toward its target over time
* docs(ui): note pruning the eslint suppressions baseline when fixing lint debt
|
||
|
ryan-crabbe-berriandGitHub
|
c7f1bcfd0d |
build(ui): migrate eslint to flat config and bump eslint-config-next to 16 (#29626)
ESLint 9 defaults to flat config and eslint-config-next was pinned at 15 while Next is on 16, so eslint only ran with ESLINT_USE_FLAT_CONFIG=false and next lint is gone on Next 16. Replace .eslintrc.json with a native flat eslint.config.mjs (config-next 16 ships flat configs, so no FlatCompat shim is needed), bump eslint-config-next to 16.2.6, add @eslint/js and typescript-eslint as explicit devDeps for the recommended rule sets, and point the lint script at eslint directly. This only makes eslint runnable on modern tooling; it does not wire it into CI. The same rules carry over (next/core-web-vitals, eslint and typescript-eslint recommended, prettier, unused-imports) |
||
|
+11
|
c7ab9adde5 |
Litellm oss staging 030626 (#29578)
* Fix incorrect agent API request example payload structure (#29556) * fix(otel): add litellm_metadata fallback in _get_span_context and _end_proxy_span_from_kwargs (#29427) * fix(otel): add litellm_metadata fallback in _get_span_context and _end_proxy_span_from_kwargs On /v1/messages and other LITELLM_METADATA_ROUTES, the parent OTel span is stored in litellm_params['litellm_metadata'] instead of litellm_params['metadata']. When the request body contains a native 'metadata' field (e.g. Anthropic's {"user_id": "..."}), litellm_params['metadata'] gets overwritten and the parent span is lost, producing orphan root spans with a different trace_id. Add fallback checks to litellm_metadata in: - _get_span_context(): so child spans find the correct parent - _end_proxy_span_from_kwargs(): so the proxy span gets closed Fixes: https://github.com/BerriAI/litellm/issues/27934 * test(otel): tighten assertions per Greptile review - test_span_context_metadata_takes_priority: assert litellm_metadata span is never accessed, proving metadata takes priority - test_span_context_no_parent_when_neither_has_span: assert both ctx and detected_span are None --------- Co-authored-by: shin-berri <shin-laptop@berri.ai> Co-authored-by: yuneng-jiang <yuneng@berri.ai> Co-authored-by: Aneesh-Fiddler <aneeshfiddler@gmail.com> Co-authored-by: Sameer Kankute <sameer@berri.ai> * fix: remove premature end-user budget check from get_end_user_object (#29420) * fix(proxy): remove premature end-user budget check from get_end_user_object Problem: - `_check_end_user_budget()` was called inside `get_end_user_object()` - This caused budget checks to run BEFORE `skip_budget_checks` could be evaluated - Zero-cost models (e.g., local vLLM) were incorrectly blocked when end-users exceeded their budget, even though they should bypass budget checks Solution: - Remove `_check_end_user_budget()` calls from `get_end_user_object()` - Budget enforcement now happens exclusively in `common_checks()` where `skip_budget_checks` context is available - `get_end_user_object()` keeps `route` as optional in function parameter for backwards compatibility and future implementation. * refactor(tests): update budget enforcement tests to reflect changes in get_end_user_object - test_get_end_user_object() verifies data fetching - test_check_end_user_budget() verifies enforcement - test_budget_enforcement_blocks_over_budget_users() integrates _check_end_user_budget() - test_resolve_end_user_reraises_budget_exceeded() is now test_resolve_end_user since no budget exceeded is thrown in get_end_user_object() * Gemini /images/generate and /images/edits billing fixes + add support for size and aspect ratio params (#29534) * Fix Gemini image config mapping * Address Gemini image config review * Format Gemini image generation transform * Fix Gemini image token usage logging * Share Gemini image request helpers * Fix Gemini Imagen model routing * Fixes as per self code review * Fixes per internal code review * Stop gating Imagen imageSize forwarding * Document Gemini image size mapping source * chore: retrigger lint * Clarify Gemini candidate count precedence * Add Inception provider (#29522) * add inception as provider (chat, fim) * linting * seperate test suite for chat and fim * fix test coverage * fix: model hub custom pricing model info (#29293) * Opik user auth key metadata extractors (#28397) * fix: enhance Opik metadata extraction to include user API key auth context fixed after refactoring to extractor logic * test: add unit tests for OPik metadata extraction logic * fix: enhance extract_opik_metadata function to prioritize metadata sources for improved accuracy * fix(ci): clarified comments and edited unit tests * test: add unit tests for OPik metadata extraction with auth and requester overrides * fix(ui): replace fixed favicon.ico with current api get /get_favicon (#29532) Signed-off-by: José Luis Di Biase <josx@interorganic.com.ar> * fix(vertex/gemini): keep tool_call reference when a text-only assistant message follows (#29561) `_gemini_convert_messages_with_history` tracks `last_message_with_tool_calls` so a following tool result can be matched back to its tool call. The assignment was inside a branch guarded by `assistant_msg.get("tool_calls", []) is not None`, which is also True for a text-only assistant message (an empty list is not None). As a result, an assistant message with no tool calls that appears between a tool call and its tool result overwrote the reference, and conversion failed with: Exception: Missing corresponding tool call for tool response message. This shape is common: a model emits a short narration/assistant message after a tool call before the tool result is appended. Only update `last_message_with_tool_calls` when the assistant message actually carries tool_calls (or a function_call). Adds a regression test. Co-authored-by: shin-berri <shin-laptop@berri.ai> Co-authored-by: yuneng-jiang <yuneng@berri.ai> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * Add 1-hour cache write pricing for EU/AU/JP Bedrock Anthropic models (#28572) * fix(thinking): handle None thinking param in is_thinking_enabled (#28598) Squash-merged by litellm-agent from Terrajlz's PR. * feat(helm): support tpl rendering in podAnnotations (#28609) Squash-merged by litellm-agent from devauxbr's PR. * Forward custom_llm_provider through the Responses API bridge (Fixes #28505) (#28575) * Forward custom_llm_provider through the Responses API bridge (Fixes #28505) When a Chat Completions request to a GPT-5.4+ model contains both `tools` and `reasoning_effort`, `completion()` auto-routes through `responses_api_bridge`. The bridge handler called `litellm.responses()` / `litellm.aresponses()` without forwarding the already-resolved `custom_llm_provider`, so the downstream call re-invoked `get_llm_provider()` with `custom_llm_provider=None` and stripped a second provider prefix from a `provider/provider/model` deployment string. For a deployment configured as `openai/openai/openai/gpt-5.5`, the bridge flow sent `openai/gpt-5.5` to the upstream API instead of the correct `openai/openai/gpt-5.5`. Upstream APIs that enforce model-name allow-lists rejected this as `key_model_access_denied`. Fix: pass the locally-resolved `custom_llm_provider` into both the sync `responses()` and async `aresponses()` calls so the downstream `_resolve_model_provider_for_responses` sees an explicit provider and skips the second prefix-strip. New regression test `tests/test_litellm/completion_extras/test_responses_bridge_provider_propagation.py` pins both call sites: each must forward `custom_llm_provider`. * fix(28505): set custom_llm_provider on request_data instead of as duplicate kwarg Greptile flagged that the previous patch passed custom_llm_provider as an explicit kwarg to responses()/aresponses() while request_data already carried it via the spread of sanitized_litellm_params, which would raise TypeError: got multiple values for keyword argument on every real bridge call. Switches to assigning request_data['custom_llm_provider'] before the call so the resolved provider wins over whatever sanitized_litellm_params spread in, without duplicating the kwarg. Updates the regression test to seed request_data with a sentinel custom_llm_provider so it actually exercises the overwrite path (the previous test mocked transform_request with a minimal dict and never hit the conflict). * chore: trigger shin-agent re-eval on retargeted staging base * chore: trigger shin-agent re-eval against updated Greptile state * Add 1-hour cache write pricing for EU/AU/JP Bedrock Anthropic models The 1-hour prompt-cache write tier (`cache_creation_input_token_cost_above_1hr`) was added to the us./global. variants of the Claude 4.5/4.6/4.7 family on Bedrock, but the eu./au./jp. cross-region inference profiles were left without it. AWS Bedrock pricing applies the same +10% regional premium across all geo profiles, so eu./au./jp. should carry the same 1-hour rates as us. (1.6x the 5-minute regional rate). Without these fields, cost tracking on EU/AU/JP Bedrock 1-hour-TTL prompt caching falls back to the 5-minute write rate and undercounts spend by ~60% for European, Australian, and Japanese tenants. Adds the 1-hour tier (and Sonnet 4.5's long-context >200K tier where AWS publishes one) to 14 regional Bedrock entries in both `model_prices_and_context_window.json` and the bundled `model_prices_and_context_window_backup.json`: - eu./au. Opus 4.6 ($11.00 / MTok) - eu./au. Opus 4.7 ($11.00 / MTok) - eu./au./jp. Sonnet 4.6 ($6.60 / MTok) - eu./au./jp. Sonnet 4.5 ($6.60 / MTok regular, $13.20 / MTok LC) - eu./au./jp. Haiku 4.5 ($2.20 / MTok) Also extends `tests/test_litellm/test_bedrock_anthropic_1hr_cache_pricing.py` with a `REGIONAL_EXPECTED` parametrized block covering all 13 new entries plus the existing 1.6x ratio invariant. Note: `eu.anthropic.claude-opus-4-5-20251101-v1:0` carries the wrong 5m rate today (base 6.25e-06 instead of regional 6.875e-06), which would break the 1.6x ratio check. It is intentionally left out of this PR so the scope stays "1-hour cache tier addition" — a separate follow-up should correct the EU 5m rates for Opus 4.5. --------- Co-authored-by: Terrajlz <info@jouleselectrictech.com> Co-authored-by: Bruno Devaux <devaux.br@gmail.com> Co-authored-by: Sameer Kankute <sameer@berri.ai> * Add 1-hour cache write pricing tier for Vertex AI Anthropic models (#28569) * fix(thinking): handle None thinking param in is_thinking_enabled (#28598) Squash-merged by litellm-agent from Terrajlz's PR. * feat(helm): support tpl rendering in podAnnotations (#28609) Squash-merged by litellm-agent from devauxbr's PR. * Forward custom_llm_provider through the Responses API bridge (Fixes #28505) (#28575) * Forward custom_llm_provider through the Responses API bridge (Fixes #28505) When a Chat Completions request to a GPT-5.4+ model contains both `tools` and `reasoning_effort`, `completion()` auto-routes through `responses_api_bridge`. The bridge handler called `litellm.responses()` / `litellm.aresponses()` without forwarding the already-resolved `custom_llm_provider`, so the downstream call re-invoked `get_llm_provider()` with `custom_llm_provider=None` and stripped a second provider prefix from a `provider/provider/model` deployment string. For a deployment configured as `openai/openai/openai/gpt-5.5`, the bridge flow sent `openai/gpt-5.5` to the upstream API instead of the correct `openai/openai/gpt-5.5`. Upstream APIs that enforce model-name allow-lists rejected this as `key_model_access_denied`. Fix: pass the locally-resolved `custom_llm_provider` into both the sync `responses()` and async `aresponses()` calls so the downstream `_resolve_model_provider_for_responses` sees an explicit provider and skips the second prefix-strip. New regression test `tests/test_litellm/completion_extras/test_responses_bridge_provider_propagation.py` pins both call sites: each must forward `custom_llm_provider`. * fix(28505): set custom_llm_provider on request_data instead of as duplicate kwarg Greptile flagged that the previous patch passed custom_llm_provider as an explicit kwarg to responses()/aresponses() while request_data already carried it via the spread of sanitized_litellm_params, which would raise TypeError: got multiple values for keyword argument on every real bridge call. Switches to assigning request_data['custom_llm_provider'] before the call so the resolved provider wins over whatever sanitized_litellm_params spread in, without duplicating the kwarg. Updates the regression test to seed request_data with a sentinel custom_llm_provider so it actually exercises the overwrite path (the previous test mocked transform_request with a minimal dict and never hit the conflict). * chore: trigger shin-agent re-eval on retargeted staging base * chore: trigger shin-agent re-eval against updated Greptile state * Add 1-hour cache write pricing tier for Vertex AI Anthropic models GCP Vertex AI publishes a separate 1-hour cache write column for the Claude family (1.6x the 5-minute write rate, matching the documented Bedrock ratio). LiteLLM's Vertex AI Anthropic entries only carry the 5-minute tier, so any request that uses `cache_control: {"ttl": "1h"}` on Vertex AI Claude is undercounted in cost tracking by ~60%. The runtime side already supports the 1-hour tier — `VertexAIAnthropicConfig` extends `AnthropicConfig`, populating `ephemeral_1h_input_tokens`, and `_calculate_cache_creation_cost` reads `cache_creation_input_token_cost_above_1hr`. Only the price registry was missing data. Adds the field to 19 vertex_ai/claude-* entries across both `model_prices_and_context_window.json` and the bundled `model_prices_and_context_window_backup.json`: - Haiku 4.5 ($1.25 -> $2.00 / MTok) - Sonnet 3.7 / 4 / 4.5 / 4.6 ($3.75 -> $6.00 / MTok) - Opus 4.5 / 4.6 / 4.7 ($6.25 -> $10.00 / MTok) - Opus 4 / 4.1 ($18.75 -> $30.00 / MTok) Adds `tests/test_litellm/test_vertex_anthropic_1hr_cache_pricing.py` mirroring the Bedrock equivalent — pins each (5m, 1h) pair per model and asserts the 1.6x ratio across the family. Fixes #27781. --------- Co-authored-by: Terrajlz <info@jouleselectrictech.com> Co-authored-by: Bruno Devaux <devaux.br@gmail.com> Co-authored-by: Sameer Kankute <sameer@berri.ai> * Fix Gemini multimodal function responses (#29325) Co-authored-by: shin-berri <shin-laptop@berri.ai> Co-authored-by: yuneng-jiang <yuneng@berri.ai> * address greptile review: add _transform_image_usage method and model-map supports_image_size flag - Add _transform_image_usage instance method to GoogleImageGenConfig that delegates to transform_gemini_image_usage, fixing the regression test - Replace hardcoded "2.5-flash" string check in supports_gemini_image_size with a get_model_info lookup on supports_image_size (default true) - Add supports_image_size: false to all gemini-2.5-flash model entries in model_prices_and_context_window.json so capability is controlled via the model map rather than embedded in code * fix test failures: schema validation, mypy type, model info plumbing, pricing test - Add supports_image_size to ModelInfoBase TypedDict so get_model_info surfaces it - Pass supports_image_size through _get_model_info_helper constructor call - Fix supports_gemini_image_size to use value is not False (None means unset, defaults to True) - Add supports_image_size to JSON schema in test_aaamodel_prices_and_context_window_json_is_valid - Correct gemini-3.1-flash-lite pricing assertions in test to match JSON values * Add Azure AI Kimi K2.6 metadata (#27052) * Add Azure AI Kimi K2.6 metadata * Scope Kimi metadata test cost map setup * fall back to substring check for models not in model_prices_and_context_window.json Models like gemini-2.5-flash-image-preview are not in the pricing JSON, so get_model_info raises. Fall back to "2.5-flash" not in model when the JSON has no explicit supports_image_size entry for the model. * fix(inception): don't forward global litellm.api_key to Inception FIM Match the Inception chat config: resolve only an Inception-specific key (param, litellm.inception_key, or INCEPTION_API_KEY) for the text-completion FIM path. The global litellm.api_key (often an OpenAI key) was both leaking to api.inceptionlabs.ai and taking precedence over the configured Inception key when set. * fix(auth): enforce end-user budget on custom-auth path that skips common_checks get_end_user_object() no longer raises BudgetExceededError, so custom-auth deployments with custom_auth_run_common_checks unset (which skip the centralized common_checks gate) stopped enforcing the end-user budget, letting an over-budget end user keep making requests. Re-enforce the budget in _run_post_custom_auth_checks on that path. --------- Signed-off-by: José Luis Di Biase <josx@interorganic.com.ar> Co-authored-by: Isha <72744901+IshaMeera@users.noreply.github.com> Co-authored-by: aneeshsangvikar <aneeshsangvikar@fiddler.ai> Co-authored-by: shin-berri <shin-laptop@berri.ai> Co-authored-by: yuneng-jiang <yuneng@berri.ai> Co-authored-by: Aneesh-Fiddler <aneeshfiddler@gmail.com> Co-authored-by: Suleiman Elkhoury <108065141+suleimanelkhoury@users.noreply.github.com> Co-authored-by: Dmitriy Alergant <93501479+DmitriyAlergant@users.noreply.github.com> Co-authored-by: Yanis Miraoui <yanis.miraoui19@imperial.ac.uk> Co-authored-by: Lovro Seder <vrovro@gmail.com> Co-authored-by: Thomas Mildner <12685945+Thomas-Mildner@users.noreply.github.com> Co-authored-by: José Luis Di Biase <josx@interorganic.com.ar> Co-authored-by: Lai Quang Huy <64073540+1qh@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Filippo Menghi <113345637+Cyberfilo@users.noreply.github.com> Co-authored-by: Terrajlz <info@jouleselectrictech.com> Co-authored-by: Bruno Devaux <devaux.br@gmail.com> Co-authored-by: ZHONG Ziwen <67355585+zzw-math@users.noreply.github.com> Co-authored-by: Emerson Gomes <emerson.gomes@thalesgroup.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|
ryan-crabbe-berriandGitHub
|
d991c47018 |
fix(ui/agents): make A2A skill tags enterable and validated (#29512)
* fix(ui/agents): make A2A skill tags enterable and validated
Skill tags were marked required but rendered as a comma-split text input
that couldn't surface validation and let empty values save. Switch tags
and examples to Select tag inputs, drop the misleading "Required" skills
label (the API allows zero skills), and validate the full configure step
so an added skill must be complete before advancing.
Resolves LIT-3153
* fix(ui/agents): allow Enter to create skill tags/examples
Drop open={false} from the tags and examples Select inputs. With the
dropdown forced closed, AntD suppresses the "create from input" option,
so pressing Enter (as the placeholder instructs) did nothing. Matches the
existing extra_headers Select.
|
||
|
|
ae7ac72331 |
feat(agents): add LangFlow agent provider with A2A session bridging (#28963)
* feat(agents): add LangFlow agent provider with A2A session bridging Register LangFlow as a completion provider and agent type (UI + /api/v1/run), and map A2A contextId to LangFlow session_id for multi-turn conversations. Co-authored-by: Cursor <cursoragent@cursor.com> * docs(providers): document langflow in provider_endpoints_support.json Co-authored-by: Cursor <cursoragent@cursor.com> * fix(agents): address Greptile review for LangFlow integration Move A2A contextId→session_id mapping into LangFlow A2A provider config, add langflow.svg logo, remove live integration test, use model for token count. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(langflow): prevent flow_id override via request optional_params Derive flow_id only from the authorized model name and reject flow_id kwargs so callers cannot invoke a different LangFlow run endpoint. Co-authored-by: Cursor <cursoragent@cursor.com> * refactor(langflow): remove redundant flow_id branch in _get_flow_id * fix(langflow): surface an error when the run response has no extractable message Previously the response parser returned the raw JSON blob as the assistant message when it could not find message text, silently presenting an unparseable payload as a valid answer. It now returns None and the caller raises a LangFlowError so the failure is visible to the client. * fix(langflow): URL-encode flow_id path segment to prevent path injection flow_id is taken from the model suffix and interpolated into /api/v1/run/{flow_id}. Without path-segment encoding a model such as langflow/../../x (or one containing ?) could move the request off the run endpoint to another path on the configured LangFlow server using the operator x-api-key. Encode the segment with quote(safe="") so it always stays a single path segment. * fix(langflow): reject empty flow_id from model name * fix(langflow): return stripped flow_id so validation matches URL path * fix(langflow): reject caller-supplied tweaks to prevent flow component override * fix(langflow): reject caller-supplied tweaks injected via extra_body The transform_request guard only inspected optional_params, but extra_body is popped before transform_request runs and merged into the request body afterward, letting a caller reintroduce tweaks and override the operator-configured LangFlow flow components. Validate the final request body in sign_request so tweaks cannot reach LangFlow through extra_body. * test(langflow): move provider tests into mirrored coverage path The langflow tests lived under tests/llm_translation/, whose CircleCI job runs without --cov and uploads nothing to Codecov, so none of the new langflow code counted toward patch coverage (codecov/patch reported 9.78% of the diff hit against a 70.83% target). Relocate them to tests/test_litellm/llms/langflow/, which the GitHub Actions provider job runs with --cov=./litellm and uploads, and add regression tests for the previously untested happy paths (transform_response building the ModelResponse with usage, non-JSON body handling, last-user message extraction, outputs-dict response shape, sign_request pass-through, error class and stream flags). Patch coverage on the diff is now ~88%. * fix(langflow): require litellm_params in A2A config instead of silent empty fallback * fix(langflow): scope A2A session_id to the authenticated key The LangFlow A2A bridge used the LangFlow session_id verbatim from the client-controlled A2A contextId, so two distinct virtual keys authorized for the same agent could read or append to each other's LangFlow conversation memory by reusing a contextId. Hand the authenticated key hash to the completion bridge through litellm_params and namespace the forwarded session_id with it. The same key keeps a stable session across turns, while different keys can no longer collide on a shared contextId. The principal is hashed before it is embedded in the session_id, so the stored token is never sent to the LangFlow backend; the original contextId is preserved as a suffix for operator-side correlation. * fix(langflow): wire authenticated key hash through A2A bridge and tests Define A2A_USER_API_KEY_HASH_PARAM in the completion bridge handler, strip it before litellm.acompletion, inject the authenticated key hash at the proxy A2A endpoint, and add regression tests for per-key LangFlow session scoping. --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|
+5Mateo WangGitHubYuneng JiangMilanCursorMateo WangKrrish Dholakiaryan-crabbe-berristuxfClaude Sonnet 4.6veria-ai[bot] <224490171+veria-ai[bot]@users.noreply.github.com>gym-cmdArtem DudarevClaudeYassin Kortam |
6d6eda8101 |
[internal copy of #28008] Support MCP OAuth passthrough and issuer-scoped JWT auth (#28356)
* fix(proxy): point /metrics 401 at the opt-out flag Operators upgrading past |
||
|
Sameer KankuteGitHubCursorveria-ai[bot] <224490171+veria-ai[bot]@users.noreply.github.com>mateo-berri |
b7bbddbd4d |
fix(mcp): clear allowed_tools and tool overrides on MCP server edit (#29411)
* fix(mcp): clear allowed_tools and tool overrides on MCP server edit Send empty arrays/objects from the dashboard instead of null, coerce legacy null payloads before Prisma, and stop auto-selecting all tools when the stored allowlist is empty. Co-authored-by: Cursor <cursoragent@cursor.com> * style(mcp): simplify CRUD panel value ternary per review Co-authored-by: Cursor <cursoragent@cursor.com> * fix(mcp): enforce empty tool allowlist when cleared in dashboard Set mcp_info.tool_allowlist_enforced on UI save so [] blocks all tools while legacy servers with default [] remain unrestricted. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix legacy MCP tool allowlist edit state * test(mcp): pin allowlist fields on mock server in tools test MagicMock auto-attributes are truthy and trigger server_applies_tool_allowlist after the empty-allowlist enforcement change. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(mcp): avoid locking legacy servers on quick edit save Only set tool_allowlist_enforced when already enforced or the user selected tools; skip allowlist fields on save for unrestricted servers; do not auto-select all tools when editing legacy servers before load. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(mcp): type mcp_info base for allowlist flag read Co-authored-by: Cursor <cursoragent@cursor.com> * fix(mcp): use MCPInfo type for tool_allowlist_enforced in edit save Co-authored-by: Cursor <cursoragent@cursor.com> * Update ui/litellm-dashboard/src/components/mcp_tools/mcp_server_edit.tsx Co-authored-by: veria-ai[bot] <224490171+veria-ai[bot]@users.noreply.github.com> * Remove unused MCP allowlist variable * Fix MCP legacy tool state display * Fix legacy MCP tool allowlist saves * fix(mcp): enforce allowlist when create flow deselects all tools Track explicit allowlist interaction in the create form so deselecting every tool persists tool_allowlist_enforced=true. Previously an empty selection sent the flag as false with allowed_tools=[], which the proxy treats as allow-all, contradicting the UI's 0 tools enabled state. This mirrors the existing edit-flow handling. * fix(mcp): enforce disallowed_tools on REST listing and keep restored tool selection on legacy edit --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: veria-ai[bot] <224490171+veria-ai[bot]@users.noreply.github.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|
|
e8fcb01215 |
Litellm OSS Staging (#29161)
* Cato Networks guardrail, based on Aim (#26597) * Aim was acquired by Cato Networks, creating Cato Networks guardrail based on Aim * Add more tests * Move test so they are reached by codecov coverage * base URL trailing slashes * Support Lemonade runtime context metadata (#28135) * Support Lemonade runtime context metadata * Add provider hook for runtime model metadata * Address provider model info review feedback Keep the runtime model info hook duck-typed instead of extending the base model-info class, and avoid importing ModelInfoBase from Ollama common utilities to reduce CodeQL cyclic-import noise. Co-authored-by: openhands <openhands@all-hands.dev> * Fix CI after staging rebase Relax the Ollama runtime metadata return annotation to match the provider-hook dict response and update the Google Interactions OpenAPI status expectation for the current live spec. Co-authored-by: openhands <openhands@all-hands.dev> * Normalize Lemonade runtime model metadata * Avoid leaking Ollama metadata auth * Avoid leaking Lemonade metadata auth --------- Co-authored-by: Graham Neubig <398875+neubig@users.noreply.github.com> Co-authored-by: openhands <openhands@all-hands.dev> * fix(cato): address guardrail review feedback Use proxy-authenticated user identity, forward moderation hook return values, and ensure streaming sender tasks are cancelled and awaited on exit. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(vertex_ai): route google/gemma-*-maas through partner-models OpenAI path - clone of #28010 (#28846) * fix(vertex_ai): route google/gemma-*-maas through partner-models OpenAI path Fixes #26083 vertex_ai/google/gemma-4-26b-a4b-it-maas previously fell through to the NON_GEMINI route. Per owtaylor's plan on #26083: add the google/gemma- prefix to PartnerModelPrefixes so is_vertex_partner_model picks it up and should_use_openai_handler routes it to the OpenAI-compatible /endpoints/openapi/chat/completions URL. No gemma-detection exclusion needed (the "gemma/" check uses a slash, which google/gemma-... doesn't match). No OpenAIGPTConfig subclass needed — works with the base handler. * fix(vertex_ai): mark gemma-4-26b-a4b-it-maas as vision-capable (empirically verified) * fix(vertex_ai): address greptile feedback — provider category, canonical URL, sync backup * test(vertex_ai): add function-calling and vision pass-through tests for Gemma MaaS Addresses oss-pr-review-agent-shin feedback on PR #28010: supports_function_calling, supports_tool_choice, and supports_vision were marked true but had no tests proving the payloads actually reached the OpenAI-compatible endpoint. Added: - test_gemma_maas_supports_function_calling — verifies the utility returns True when the model_cost entry carries supports_function_calling=true - test_gemma_maas_supports_vision — same for supports_vision - test_vertex_ai_gemma_function_calling_passthrough — verifies tools + tool_choice appear in the JSON body POSTed to /endpoints/openapi/chat/completions - test_vertex_ai_gemma_vision_passthrough — verifies image_url content parts survive transformation and reach the global endpoint URL * fix: Delete uv.lock * test(vertex_ai): add function-calling and vision pass-through tests for Gemma MaaS Addresses oss-pr-review-agent-shin feedback on PR #28010: P1 (patch target): Added a comment explaining why patching litellm.llms.custom_httpx.http_handler.AsyncHTTPHandler is correct — get_async_httpx_client() (defined in http_handler.py) instantiates AsyncHTTPHandler within that module's scope, so the definition-site patch intercepts it. Without the mock the test raises AuthenticationError, confirming it never silently passes. P2 (partner-provider regression guard): Added test_gemma_routes_through_openai_handler() which calls VertexAIPartnerModels.should_use_openai_handler() directly, so if Gemma's routing to VertexPartnerProvider.llama ever changes the URL-shape tests below it become a real regression guard rather than an unanchored unit test. Also added: - test_gemma_maas_supports_function_calling / supports_vision — capability flag checks via patch.dict(litellm.model_cost) - test_vertex_ai_gemma_function_calling_passthrough — tools + tool_choice forwarded in the request body - test_vertex_ai_gemma_vision_passthrough — image_url part survives transformation to the global endpoint Added: - test_gemma_maas_supports_function_calling — verifies the utility returns True when the model_cost entry carries supports_function_calling=true - test_gemma_maas_supports_vision — same for supports_vision - test_vertex_ai_gemma_function_calling_passthrough — verifies tools + tool_choice appear in the JSON body POSTed to /endpoints/openapi/chat/completions - test_vertex_ai_gemma_vision_passthrough — verifies image_url content parts survive transformation and reach the global endpoint URL * fix: proper patch for unit tests --------- Co-authored-by: Iana <iana@Shivakumars-MacBook-Pro.local> * fix(cato): guardrail all completion choices on output When n > 1, only choices[0] was analyzed and redacted. Iterate every Choices entry so block and anonymize actions apply to all completions. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix review * fix(cato_networks): harden output anonymize handling and restructure nested UI routes Guard against empty redacted_output and empty all_redacted_messages from Cato. Restructure nested admin UI HTML exports to index.html so extensionless routes work. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix mypy * fix(cato): guard missing policy_drill_down and all_redacted_messages keys * fix(cato): avoid KeyError bypassing block action on missing analysis_result * fix(cato): preserve non-text message fields during anonymize Rebuild redacted messages from the original messages, overwriting only content, so tool_calls, tool_call_id, name and multimodal fields survive the anonymize action. * fix(cato): preserve trailing messages when fewer redacted messages returned Avoid silently truncating the conversation in _anonymize_request when Cato returns fewer redacted messages than were sent, and isolate the no-api-key config test from a pre-existing CATO_API_KEY environment variable. * fix(cato,model-info): preserve stream block signal on sender teardown; forward api_key in dynamic model-info lookup Suppress ConnectionClosed (alongside CancelledError) when tearing down the Cato streaming sender task so a backend ConnectionClosed cannot mask the original StreamingCallbackError (e.g. a guardrail block) raised by the receive loop. Thread api_key through get_model_info -> _get_model_info_helper so an explicit key reaches a provider's dynamic get_model_info for a caller-supplied api_base. Previously only api_base was forwarded, so authenticated Ollama and Lemonade servers at a custom base could only be queried unauthenticated. * fix(cato): surface mid-stream forwarding errors instead of blocking on recv If the upstream LLM stream errors mid-flight, the sender task dies before sending the terminal done frame, so the consumer would block on websocket.recv() until Cato closes the connection. Race recv against the sender task and raise the stored sender exception promptly as a StreamingCallbackError. * fix(cato): drop spoofable end_user_id from guardrail user identity Only the key/JWT-bound user_email is a trusted identity. end_user_id is resolved from caller-supplied request fields (OpenAI user param, headers, metadata), so an authenticated caller with no bound user_email could set it to another user's email and have LiteLLM forward x-cato-user-email for that victim, poisoning Cato audit and policy attribution. Forward only user_email and omit the header otherwise. * fix(cato): harden output anonymize path against missing content key * fix(cato): fall back to original message when redacted content key is missing * refactor(model-info): drop unused api_key from cached model-info helper _cached_get_model_info_helper is only called by the cost-tracking hot path, which never authenticates, so the api_key parameter was never populated. Keeping it in the lru_cache key offered no benefit and risked fragmenting the high-RPS cache and retaining credential strings per entry. * fix(cato): preserve None content on tool-call-only choices in output hook * fix(ollama): respect static-model guard in OllamaConfig.get_model_info Delegate to OllamaModelInfo.get_model_info so statically-priced Ollama models short-circuit before the /api/show network call instead of hitting the server unconditionally. * fix(lemonade,ollama): treat empty api_key as unset to avoid leaking server creds An empty-string api_key was treated as an explicit key, so it passed the guard meant to keep server-side credentials off caller-supplied bases and then fell back through the env/global key chain. A caller could point api_base at a server they control and send api_key="" to receive the configured provider key in the Authorization header. Gate the credential fallback on the api_key being truthy instead of merely not-None. * fix(cato): inspect and redact Responses-API input, not just messages The guardrail only read data["messages"], so /v1/responses requests, which carry their text in data["input"], reached Cato as an empty message list and bypassed inspection entirely. Send build_inspection_messages(data) so both shapes are analyzed, and write anonymized results back with apply_redacted_messages_back when the request used input. * perf(utils): keep api_key out of get_model_info lru_cache key * fix(cato): propagate ssl_verify to streaming WebSocket connection The streaming hook applied ssl_verify only to the HTTP handler; the websockets.connect() call used default verification, so a custom Cato instance behind TLS with a self-signed cert worked for non-streaming calls but failed every streaming request. Resolve the ssl_verify setting into the connect() ssl argument, mirroring the HTTP handler. * refactor(utils): rename shadowing local in _get_model_info_helper * fix(cato): flatten multimodal chat content before inspection Chat Completions requests whose message content is a multimodal parts array were posted to Cato as the raw OpenAI parts, so text inside content: [{"type":"text", ...}] reached the model without Cato ever inspecting the string. Flatten each message's list content to plain text while keeping the list 1:1 with the request so the index-based redaction write-back stays valid; Responses-API input requests still go through build_inspection_messages. * test(lemonade): clear get_model_info cache around api_base test * fix(cato): inspect and redact Responses-API input even when messages present _inspection_messages returned early once messages was non-empty, so a /v1/responses caller could place benign text in messages and disallowed text in input and have only messages reach Cato while the model used input. Inspect both fields and write anonymize redactions back to input as well as the index-aligned messages. * test(log_db_metrics): assert table_name event_metadata contract log_db_metrics now emits minimal event_metadata via _safe_db_event_metadata (table_name only, function_name/function_kwargs/function_args dropped as redundant with call_type and unsafe to stamp on a span). The success-path test still asserted function_name membership and crashed with TypeError on the None metadata returned when no table_name is passed. Pass a table_name and assert the surfaced contract instead. * fix(cato): inspect and redact completion prompt and Responses-API instructions The Cato guardrail only inspected chat messages and the Responses-API input field, so blocked text placed in the legacy /v1/completions prompt or the /v1/responses instructions field reached the model without ever being sent to Cato. Both fields are now appended as synthetic inspection messages, and the anonymize path slices Cato's redactions back to the field they came from. * fix(cato): serialize non-str/bytes websocket chunks before forwarding * fix(cato): inspect tool descriptions and tool-call arguments * fix(cato): map redacted output by assistant index; restore get_model_info.cache_info * fix(cato): block output even when detection_message is null/empty A block_action returned by Cato on the output hook whose detection_message was null or empty was let through to the caller: the truthiness guard on detection_message skipped the HTTPException and the unblocked response was returned. Raise the HTTPException directly in _handle_block_action_on_output so the output path blocks unconditionally, mirroring the input path. * fix(cato): inspect and redact nested tool param and legacy function descriptions Tool/function parameter descriptions and the legacy functions[] array are forwarded to the model but were not seen by Cato, so blocked text hidden there bypassed inspection and anonymization. Recursively walk every description string in tools[].function and functions[] schemas for both the analyze payload and the anonymize write-back. * fix(cato): traverse schema descriptions iteratively to satisfy recursive detector The nested walk() generator recursed over tool/function JSON schemas with no depth bound, which the recursive_detector code-quality gate rejects. Replace it with an explicit-stack DFS that yields the same (container, key) refs in the same pre-order, so schema description redaction is unchanged. * fix(cato): inspect and redact response_format JSON schema descriptions response_format json_schema descriptions are forwarded to the model, so blocked text hidden in nested schema descriptions could bypass Cato inspection and redaction. Extend the schema-description walk to cover response_format alongside tools and legacy functions. * fix(cato): skip output rewrite when Cato returns no redaction Return None from call_cato_guardrail_on_output on monitor/no-action so the post-call hook only mutates the message when there is an actual redaction, instead of redundantly re-writing the original content. * refactor(utils): resolve explicit api_key model info without the cache Move the model-info build into a non-cached _build_model_info helper and drop api_key from the lru-cached _cached_get_model_info signature. Both cached helpers now take the same (model, provider, api_base) key and never forward api_key, while explicit per-caller keys are resolved through the builder directly instead of reaching into the cache wrapper's __wrapped__. * fix(cato): inspect and redact non-description schema string values Tool, function and response_format JSON schemas forward more than just description text to the model. enum, const, default, examples and title values are sent verbatim, so blocked content hidden in any of them bypassed Cato inspection and redaction. Walk those schema string values alongside descriptions on both the inspection and anonymize paths. * fix(model-info): surface swallowed dynamic model-info errors The provider-specific get_model_info dispatch falls back to the static cost map when a provider's dynamic lookup raises, which is intentional graceful degradation. Previously the exception was discarded with a bare debug line, so a real failure (e.g. a provider whose get_model_info signature does not accept api_key) was invisible. Log the exception at warning level with the model and provider context so the fallback is diagnosable. * fix(cato): inspect and redact Responses API output in post-call hook The post-call success hook only handled ModelResponse, so /v1/responses (which returns a ResponsesAPIResponse) bypassed the Cato output guardrail. Extract and inspect/redact every output_text content block and function-call arguments string, blocking on a block action, so generated text cannot escape inspection by using the Responses API. * chore: reset _experimental/out folder * chore(ui): remove orphaned prebuilt dashboard chunk files The _experimental/out manifests are byte-identical to the base branch, so the served dashboard already matches base. 436 unreferenced Next.js chunk files had accumulated in the directory and are not loaded by any manifest; removing them restores the committed UI artifacts to the base build and drops the artifact churn from this PR's diff. * fix(guardrails,ollama): forward ssl_verify to Cato init and raise_for_status on /api/show --------- Co-authored-by: Alex Yaroslavsky <trexinc@gmail.com> Co-authored-by: Graham Neubig <neubig@gmail.com> Co-authored-by: Graham Neubig <398875+neubig@users.noreply.github.com> Co-authored-by: openhands <openhands@all-hands.dev> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Piotr Placzko <piotr@icep-design.com> Co-authored-by: Iana <iana@Shivakumars-MacBook-Pro.local> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|
ryan-crabbe-berriandGitHub
|
609e1e9763 |
fix(ui): render caller-supplied filter options in caller order (LIT-3151) (#29462)
FilterComponent iterated a hardcoded orderedFilters whitelist instead of the options prop, so any consumer whose filter names were not on that list rendered nothing. The Tool Policies page passes "Input Policy", "Output Policy", "Team Name" and "Key Name", none of which were whitelisted, so its Filters panel opened to an empty area. Drop the whitelist and render the options the caller passes, in the order they pass them, so each page owns its own filter set and ordering. The Logs page array is reordered to match its prior on-screen order; VirtualKeys and TeamVirtualKeys already matched the old whitelist order and are unaffected. |
||
|
ryan-crabbe-berriandGitHub
|
1cce49b9d0 |
fix(vector-stores): support engines URL for Vertex AI Search (#27885)
Adds optional vertex_engine_id field to vertex_ai/search_api so users can route through a Discovery Engine search app instead of the data store directly. Required for website, healthcare, and connector-based data stores that return FAILED_PRECONDITION on the existing dataStores URL. Existing data-store-direct callers are unaffected. Resolves LIT-3036 |
||
|
yuneng-jiangandGitHub
|
54ed5a4eb5 |
fix(e2e): tolerate trailing slash in SERVER_ROOT_PATH login redirect (#29369)
The Next.js admin UI is exported with trailingSlash: true, so the proxy
serves /ui/login at /ui/login/index.html and 308s /ui/login → /ui/login/.
The waitForURL predicate used endsWith("/ui/login"), which never matched
the canonicalized URL and timed out after 15s.
This was masked until the build artifacts were regenerated against the
AuthContext fix: the prior bundles still hit the racy redirect path that
fired before proxyBaseUrl was populated, producing /ui/login (no prefix,
no proxy round-trip, no trailing slash) which fortuitously satisfied the
predicate. The first PR to ship the corrected bundle exposed the
assertion bug.
Switch the predicate to includes("/ui/login"); the prefix assertion below
still validates the SERVER_ROOT_PATH preservation that is the actual
contract under test.
|
||
|
yuneng-jiangandGitHub
|
f0ebfb2a1b |
fix(ui): break logout redirect loop across origins (#29360)
When the user has visited both the dev UI (e.g. localhost:3000) and the
proxy UI (e.g. localhost:4000) in the same tab, logging out from the dev
origin produced an infinite logout/login redirect.
The proxy-side LoginPage's "is the user still authenticated?" check
was reading getCookie("token"), which falls back to sessionStorage when
document.cookie has no token. The cross-origin clearTokenCookies() call
from the dev origin can clear cookies on the shared hostname, but cannot
reach sessionStorage on the proxy origin (sessionStorage is per-origin),
so the fallback returned a stale token and LoginPage interpreted the
user as logged in, redirecting back to the dev origin. Dev origin then
saw no cookie and redirected to LoginPage, repeating ~20x per second.
This change introduces getCookieFromDocument(), a cookie-only read with
no sessionStorage fallback, and uses it in LoginPage's already-logged-in
check. The HttpOnly-reverse-proxy defense from PR #23532 is unaffected:
storeLoginToken still writes both the JS cookie at /ui and the
sessionStorage backup, and getCookie still falls back for callers that
want the full read path.
|
||
|
ryan-crabbe-berriandGitHub
|
a9cc6ed68c |
test(e2e): cover PROXY_LOGOUT_URL redirect on Logout (#29080)
* test(e2e): cover PROXY_LOGOUT_URL redirect on Logout Env-gated spec mirroring the existing serverRootPathRedirect pattern: when the proxy is booted with PROXY_LOGOUT_URL set, clicking Logout in the navbar must navigate to that external URL. The standard run_e2e.sh exports an empty value so the rest of the suite is unaffected; this spec self-skips unless the env var is populated. * test(e2e): run PROXY_LOGOUT_URL spec in the suite + harden logout assertions Boot the e2e proxy with PROXY_LOGOUT_URL set (job-level env in CircleCI and run_e2e.sh) so proxyLogoutUrl.spec.ts actually runs instead of self-skipping. Nothing else in the suite performs a logout, so this only affects the behavior under test. Harden the spec to verify the logout flow rather than a URL substring: - wait for /sso/get/ui_settings before clicking so logoutUrl is populated (otherwise window.location.href = "" silently reloads same-origin) - assert a token cookie exists first, and is cleared after logout - locate the dropdown via getByRole instead of internal antd CSS classes - stub the external destination and assert on URL origin + path prefix * test(e2e): assert exact PROXY_LOGOUT_URL on logout redirect Replace the origin + startsWith(pathname) checks with a single normalized href comparison. With PROXY_LOGOUT_URL=https://www.example.com the path was "/", so startsWith("/") matched any path and left path/query/hash unchecked. Comparing normalized hrefs pins scheme, host, port, path, query and hash while still tolerating the browser's trailing-slash/default-port normalization. |
||
|
ryan-crabbe-berriandGitHub
|
90b5104475 |
feat(mcp/auth): additive key access-group grants + opt-in member assignment (#29313)
* fix(mcp): make key.access_group_ids grants additive over team ceiling A key whose unified access_group_ids grant a private MCP server was having that grant intersected against its team's MCP ceiling, so a key in a team scoped to other servers (or with no own scope) lost the granted server entirely. Resolve access_group_ids once as ungated additive grants and union them on top of the key/team ceiling instead of folding them into the key scope that gets intersected. * test(mcp): align key access-group tests with additive-grant model The previous commit moved key.access_group_ids resolution out of the intersected key ceiling (_get_allowed_mcp_servers_for_key) and into the ungated additive grant path (_get_key_access_group_mcp_server_extras), unioned on top of the team ceiling. Five tests from #28890/#29195 still asserted the old gated / in-key-scope contract and failed: - _get_allowed_mcp_servers_for_key now returns the object_permission ceiling only and never resolves access_group_ids; two tests now assert the group resolver is not called from that path (with and without an object_permission present). - The extras path is ungated, so a group whose assigned_team_ids / assigned_key_ids exclude the caller still contributes its servers. - The end-to-end test asserts the grant surfaces via the extras path rather than the base key path. - Dropped test_key_access_group_ids_empty_returns_no_extras; the empty case is already covered by the extras family's no-groups test. * feat(auth): gate member access-group assignment on keys behind opt-in Non-admin team members could attach access_group_ids to keys they create or update, letting them self-grant resources (MCP servers/models) the team admin never intended. Add an opt-in KEY_ACCESS_GROUP_ASSIGNMENT team-member permission (default-deny) enforced at /key/generate and /key/update; proxy and team admins bypass. Surfaces automatically as a checkbox in the team Member Permissions UI. * fix(auth): gate access-group assignment on /key/regenerate too RegenerateKeyRequest inherits access_group_ids and prepare_key_update_data persists it, so a non-admin key owner could self-grant access groups by regenerating. Apply the same opt-in member gate using the existing key's team. * test(auth): cover member access-group gate and additive MCP grants Add unit tests for enforce_member_can_assign_access_groups (deny without opt-in, allow with opt-in, and proxy-admin / team-admin / non-team-key bypasses) and for _get_key_access_group_mcp_server_extras (no-auth and no-resolved-servers return empty, resolved ids are expanded, errors degrade to no grants). |
||
|
ryan-crabbe-berriandGitHub
|
37e6e2da1c |
test(e2e): assert internal-user navbar identity is scoped to that user (#29077)
* test(e2e): assert internal-user navbar identity is scoped to that user The existing login.spec.ts only checks the admin's navbar identity. This adds the symmetric check for the internal user — verifying the account button + dropdown surface the internal user's email, id, and role, and that no admin-scoped values leak through. * test(e2e): harden navbar identity test per review feedback Locate the user dropdown panel by a data-testid on the popupRender div instead of Ant Design internal + Tailwind class names, so styling refactors no longer risk breaking the identity-scoping assertions. Source the seeded user emails/ids from shared constants (match seed.sql) instead of hardcoding them inline. |
||
|
ryan-crabbe-berriandGitHub
|
892838963c |
test(e2e): cover Internal User create-key flow when in no teams (#29083)
* test(e2e): cover Internal User create-key flow when in no teams The seeded e2e-internal-user is in two teams, so the "no team" branch of the Create Key modal — where the team dropdown must render empty — was unreachable. Seeds a noteam@test.local user and adds a spec that logs in fresh, opens the modal, and asserts the dropdown has zero options. * test(e2e): harden no-team dropdown assertion + add with-teams counterpart Replace the one-shot count() check with a settled-empty assertion: wait for the dropdown's loaded "No teams found" state before asserting zero options, so the test can't pass on a transient empty frame while the team-options request is still in flight. Add internalUserWithTeams.spec.ts as the differential partner; it logs in as the seeded e2e-internal-user (two team memberships) and asserts the dropdown lists exactly those teams. Without it, the no-team spec's zero-options assertion would still pass against a regression that empties the dropdown for every user. |
||
|
|
af17400c38 |
feat(a2a): well-known agent-card discovery + LangGraph Platform mode (#28860)
* feat(a2a): well-known agent-card discovery + LangGraph Platform mode Adds a registration-time discovery flow so admins can paste an upstream agent URL, see its skills/capabilities, pick what to expose, and have the proxy front it with a LiteLLM-shaped agent card. Backend (new litellm/proxy/a2a/ module): - fetch_well_known_card walks /.well-known/agent-card.json, /.well-known/agent.json, /agent.json by default. langgraph_platform mode hits the canonical path with ?assistant_id=<id> (LangGraph serves one shared endpoint per deployment). - merge_agent_card overlays LiteLLM overrides on the upstream card: drops upstream url, forces protocolVersion=1.0, replaces securitySchemes with LiteLLMKey bearer, emits supportedInterfaces pointing at the proxy, filters capabilities to a small allowlist, strips non-v1.0 fields. - POST /v1/a2a/discover returns the raw upstream card (admin-only) so the UI can render skills/capabilities for selection. - create/update/patch agent endpoints pre-generate the agent_id and run merge_agent_card before storing, so DB.agent_card_params already embeds the proxy-fronted URL. UI (ui/litellm-dashboard): - New AgentCardDiscovery component with a parent-driven plan: discovery_mode + params + display URL. For LangGraph the parent composes (api_base, assistant_id); for pure A2A it uses the url field. Component hides the manual URL input when the parent drives. - add_agent_form wires discovery for every non-custom agent type and overlays the user's selections onto agent_card_params at submit, fixing the bug where dynamic agent forms ignored discovery picks. Completion-bridge fixes (paired): - Add kind: "message" to A2A response messages and unwrap result so it's a Message directly per spec (matches a2a SDK SendMessageResponse validation). - Forward A2A metadata to LangGraph runs via extra_body.metadata. * fix(a2a): preserve agent url, fix streaming chunk envelope, and protect forwarded metadata - Streaming chunk: move final out of the message object into the result envelope per the A2A spec. - Agent card merge: keep upstream url on the stored card so the runtime invocation path can locate the upstream backend; the public well-known endpoint already rewrites this field to the proxy URL before exposing it to clients. - Completion bridge: apply A2A forward metadata after merging litellm_params so an agent-configured extra_body cannot overwrite the forwarded metadata. Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(a2a): fix legacy streaming chunk, agent card test, and metadata merge - providers/litellm_completion: move 'final' out of the message object into the result envelope per the A2A spec (matches the bridge fix). - agent endpoints test: the runtime invocation path now preserves the top-level 'url' on the stored card, so update the assertion to match. - completion bridge metadata: when forwarding A2A metadata via extra_body.metadata, merge into any existing extra_body.metadata instead of replacing it, so an agent-configured metadata block is preserved (forward metadata still wins on key conflicts). Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(a2a): remove dead duplicate transformation dir; drop SSRF-prone headers field from /v1/a2a/discover Co-authored-by: Yassin Kortam <yassin@berri.ai> * fix(a2a): revert accidental html→index.html rename from |
||
|
ryan-crabbe-berriandGitHub
|
5eafe1c1fc |
test(e2e): cover navbar Logout flow as proxy admin (#29076)
* test(e2e): cover navbar Logout flow as proxy admin
The Logout button under the navbar User dropdown was an uncovered
manual-QA step. This test signs in as admin, opens the dropdown,
clicks Logout, then navigates to a protected page and asserts the
redirect to /ui/login — proving the session was cleared.
* test(e2e): fix logout dropdown trigger and account-menu selector
The button never rendered the literal text "User" (it shows initials +
display name), and the antd Dropdown uses trigger={["click"]}, so the
synthetic mouseover/mouseenter never opened the popup. Open it with a real
click on the button's aria-label ("Account menu — ...").
|
||
|
ryan-crabbe-berriandGitHub
|
fcd5760891 |
test(e2e): cover Internal User key modal, team info, key page (#29074)
* test(e2e): cover Internal User key modal, team info, key page Three previously-uncovered manual-QA paths for the Internal User role: - Create Key modal — confirm the team dropdown is populated with the user's teams (verifying the role-scoped UI flow exists). - Team info page — confirm the Settings/Members tabs are hidden for a regular team member; only the read-only tabs render. - Virtual Keys page — confirm the proxy's internal litellm-dashboard team keys never leak into an internal user's table. * test(e2e): share clickTeamId helper, strengthen key-filter assertion Address review feedback on the Internal User e2e spec: - Extract clickTeamId into helpers/navigation.ts; import in both internalUser and teams specs instead of duplicating it. - Anchor the litellm-dashboard absence check on the user's own seeded key so it cannot pass vacuously against an empty table. - Drop redundant dismissFeedbackPopup calls (navigateToPage already dismisses internally). |
||
|
ryan-crabbe-berriandGitHub
|
10bda4456a |
test(e2e): cover Internal Viewer nav, key, and team-info gating (#29075)
* test(e2e): cover Internal Viewer nav, key, and team-info gating Three previously-uncovered manual-QA paths for the Internal Viewer role: - Nav only renders the read-only sections; admin-only items (Internal Users, Organizations, Models + Endpoints) stay hidden. - Virtual Keys page hides Create New Key, and the key detail view hides Regenerate / Reset Spend / Delete actions. - Team info page hides Members and Settings tabs for the viewer. * test(e2e): scope viewer nav to sidebar, strengthen tab assertions Address review feedback on the Internal Viewer e2e spec: - Scope the nav test to the sidebar complementary landmark and match items by link role + accessible name. The prior CSS nav, aside selector grabbed the top bar (the sidebar is a complementary landmark, not a <nav> tag), so the assertions never hit the real nav links. - Land via navigateToPage so the networkidle wait settles the role-gated nav before asserting. - Assert the Virtual Keys tab is visible (was only commented). - Use toHaveCount(0) for hidden team tabs to match the nav block; tabs are conditionally rendered, not CSS-hidden. - Drop redundant dismissFeedbackPopup calls (navigateToPage already dismisses internally). |
||
|
michelligabrieleandGitHub
|
68852ef165 | fix(teams): expose keys_count on /v2/team/list and wire UI Resources badge (#28502) | ||
|
|
f27df8d516 |
docs: hand-written CLAUDE.md; point GEMINI.md and AGENTS.md at it (#29252)
* docs: replace generated CLAUDE.md with hand-written guidance, remove AGENTS.md Swap the auto-generated CLAUDE.md for a concise hand-written version that captures how we actually want agents to work in this repo: minimal comments, simplicity first, meaningful tests with a high mutation kill rate, PRs based off litellm_internal_staging rather than main, and curl against a live proxy as proof of fix instead of pasted pytest output. Remove AGENTS.md so there is one source of truth for agent guidance. The customer and company name confidentiality policy, along with the MCP available_on_public_internet note, are carried over from the previous CLAUDE.md. * fix: further clarify communication guidelines * docs: point GEMINI.md at CLAUDE.md instead of duplicating guidance Replace the standalone GEMINI.md copy, which had already drifted from the new CLAUDE.md, with a one-line pointer so Gemini reads the same single source of truth. * docs: simplify PR template test checklist item Replace the rigid "at least 1 test is a hard requirement" checklist line with "I have added meaningful tests", which matches the testing guidance in CLAUDE.md, and tidy a comma into a semicolon in the scope-isolation item. * docs: point AGENTS.md at CLAUDE.md instead of deleting it Keep AGENTS.md so tools that read it still resolve guidance, but collapse it to the same one-line pointer to CLAUDE.md used by GEMINI.md, keeping a single source of truth. * fix: make AI-generated rules more concise * fix: spelling Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * fix: make the .env usage more careful * docs: restore MCP available_on_public_internet note to CLAUDE.md The PR description states this note was carried over verbatim from the previous CLAUDE.md, but it was dropped in the rewrite. Restore it so the file matches the description and the team guidance is not lost. * docs: restore browser storage and CI supply-chain safety notes to CLAUDE.md These security-relevant rules were dropped in the rewrite. Restore the sessionStorage-over-localStorage (XSS) guidance and the CI supply-chain rules (no curl|bash, pin versions, verify checksums) so agents editing UI or CI code are still steered away from those pitfalls. * docs: move area-specific guidance into nested CLAUDE.md files The MCP, browser-storage, and CI supply-chain notes are scoped to particular parts of the tree, so move each into a nested CLAUDE.md that Claude Code loads on demand when those files are touched: the MCP note under the mcp_server gateway, the browser-storage rule under the UI dashboard, and the CI supply-chain rules under .circleci. Keeps the root CLAUDE.md focused on general guidance while the area notes surface where they are relevant. * docs: keep CI supply-chain note in root CLAUDE.md CI guidance applies beyond .circleci (it also covers downloads in GitHub workflows and any CI script), and CI work does not reliably touch a single subtree, so a nested file under .circleci would not surface it dependably. Keep it in the always-loaded root instead. The MCP and browser-storage notes stay nested where they map cleanly to one area of the tree. * fix: make it clear we prefer httpOnly * chore: make ci rule more concise * chore: make concise Fix formatting and punctuation in MCP note. * fix: don't include Claude attribution --------- Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> |
||
|
ryan-crabbe-berriandGitHub
|
2bfbf14882 |
test(e2e): cover Team Admin view + member + key flows (#29072)
* test(e2e): cover Team Admin view + member + key flows
Adds a new spec exercising the previously-uncovered team-admin manual-QA
items: viewing all team keys (including other members'), adding a member,
removing a member, and creating a team key with All Team Models. Also
seeds a dedicated invitee user so the add-member test can run in parallel
with the proxy-admin invite test without colliding on the team roster.
* test(e2e): harden team-admin member specs per review feedback
Address Greptile feedback on the Team Admin spec:
- locate the delete action via getByTestId("delete-member") instead of
the fragile svg/img .last() selector
- match the seeded removable member by user_id (members_with_roles stores
no email, so the roster renders user_id)
- assert exact success-toast strings rather than broad regexes that could
match unrelated "success" text
|
||
|
ryan-crabbe-berriandGitHub
|
9918a9c78c |
fix(guardrails): persist disable_global_guardrails on keys (#29233)
* fix(guardrails): restore disable_global_guardrails persistence for keys The per-key/team "Disable Global Guardrails" toggle silently stopped working after #17042, which removed `disable_global_guardrails` from the key/team request models and from the premium metadata allowlist. Without those, the UI's top-level field was dropped by pydantic and never folded into key `metadata`, so the runtime gate always read False and global default_on guardrails kept running. Restore the request-model fields (KeyRequestBase, NewTeamRequest, UpdateTeamRequest) and the `LiteLLM_ManagementEndpoint_MetadataFields_Premium` entry so the flag is promoted into metadata again. Because the key edit form always submits the flag (false by default), guard the UI so it is only sent when it actually changed (edit) or is enabled (create) — this keeps the premium gate on enabling intact while not 403-ing non-premium users who edit unrelated key fields, mirroring how guardrails/tags are already stripped. * test(guardrails): cover disable_global_guardrails toggle-off + clarify premium field comment Add a prepare_metadata_fields case asserting `disable_global_guardrails: False` overwrites an existing `True`, and rewrite the PREMIUM_METADATA_FIELDS comment to explain why boolean premium fields are excluded from the empty-value strip loop. |
||
|
ryan-crabbe-berriandGitHub
|
5699a06413 |
test(e2e): cover AI Hub make-public flow and public model_hub_table (#29071)
* test(e2e): cover AI Hub make-public flow and public model_hub_table Three previously-uncovered manual-QA paths land in one spec: - Admin opens "Select Models to Make Public", advances through the multi-step modal, and verifies the success toast. - AI Hub tab strip exposes Model Hub / Agent Hub / MCP Hub / Skill Hub — note the manual-QA "Claude Code Plugin Marketplace" label was renamed to Skill Hub; the test pins the current name. - Anonymous /ui/model_hub_table loads with the master key as `?key=` and renders the Model Hub tab. Agent Hub / MCP Hub tabs are conditional on public data and are not asserted here. * test(e2e): harden AI Hub make-public + public hub assertions Address Greptile review: - Make-public test now asserts "Select All (N)" with N>=1 before clicking, so a missing-seed-data run surfaces immediately instead of timing out on the disabled Next button or the success toast. - Public model_hub_table test dismisses the feedback popup before the tab visibility assertion, matching the ordering used by navigateToPage so a popup race can't mask the tab mid-evaluation. * docs(e2e): explain admin vs public AI Hub tab asymmetry Greptile flagged the all-4-tabs assertion as a potential CI flake, inferring from the public-page comment that Agent Hub / MCP Hub might be data-conditional in the admin view too. They aren't — ModelHubTable renders all four tabs unconditionally for admins. Document the asymmetry inline so future readers (and future review passes) don't re-derive it. |
||
|
ryan-crabbe-berriandGitHub
|
bc31c570f0 |
test(e2e): cover add-MCP-server flow via discovery → custom form (#29070)
* test(e2e): cover add-MCP-server flow via discovery → custom form
The "Add MCP server" manual-QA step was uncovered. This adds a test
that opens the discovery modal, jumps into the custom-server form,
fills name + Streamable HTTP transport + a placeholder URL + None
auth, submits, and verifies both the success toast and the new row.
* test(e2e): apply greptile fixes to MCP add-server test
- Anchor the auth-type Select via its enclosing Collapse panel
("Authentication") instead of the placeholder text. The Form.Item has
no label prop, so the previous `hasText: /auth type/i` filter was
matching via "Select auth type" placeholder copy — fragile.
- Document the intentional lack of teardown, matching the pattern used
in addModel.spec.ts: the e2e runner discards the DB per invocation.
Addresses Greptile P2s on PR #29070.
* test(e2e): scope MCP row assertion to the servers table
Scope the post-create row lookup to `table tbody` so the form modal's
`server_name` input — which still holds the timestamped value during
its close animation — can't satisfy the assertion before the server
actually lands in the list.
* docs(e2e): note MCP coverage scope and link to tracker
This spec only smoke-tests the happy-path Streamable HTTP + None auth
flow. Add a top-of-file comment pointing at E2E_COVERAGE.md so future
contributors can see what's still uncovered (other transports, all
auth types, edit/delete, BYOK, tool list/call, access groups).
|
||
|
ryan-crabbe-berriandGitHub
|
9cac0471ae |
test(e2e): cover Team-BYOK add-model flow as proxy admin (#29068)
* test(e2e): cover Team-BYOK add-model flow as proxy admin The team-only model + team assignment was an uncovered manual-QA path. This adds a premium-gated test that toggles Team-BYOK, picks the seeded E2E Team CRUD, submits, and verifies the model lands in All Models with the team alias attached. * test(e2e): apply greptile fixes to Team-BYOK test - Add the 2s networkidle settle that the sibling addModel tests use — networkidle fires before the All Models table finishes re-rendering, so the search input was racing with the render. - Assert on `models-results-count` before inspecting the table body so an empty search result fails with a clear "expected results count" message instead of timing out on a missing row. Addresses Greptile P2s on PR #29068. * test(e2e): harden Team-BYOK test against flake and stale state - Add before/after cleanup that deletes any Cohere model already scoped to e2e-team-crud via /v2/model/info + /model/delete, so Playwright retries and local reruns don't accumulate rows. - Pick the team from the dropdown by role/option name instead of a global getByText match — avoids matching a previously-rendered tag elsewhere in the form. - Scope the "created successfully" assertion to .ant-notification so a stale toast from an earlier test in the same browser context can't vacuously satisfy it. - Tighten the All Models assertion: require a single row that contains BOTH the cohere model name AND the e2e-team-crud alias, so the team-less wildcard from the sibling "Add wildcard route" test can't satisfy the check. |
||
|
ryan-crabbe-berriandGitHub
|
b0ea013042 |
test(e2e): cover add-fallback flow in Router Settings (#29069)
* test(e2e): cover add-fallback flow in Router Settings as proxy admin The Router Settings → Fallbacks → Add Fallbacks flow was an uncovered manual-QA path. This adds a test that opens the modal, picks a primary + fallback from the seeded mock models, saves, and verifies both render in the fallback table. * fix(e2e): make router-fallback test idempotent and pick antd options by text - Match `.ant-select-item-option` by text instead of `getByTitle(...)` — FallbackGroupConfig uses `options=` (not <Select.Option> children), so no `title` attribute is emitted and the title-based selector hangs. - Add before/after hooks that wipe any fallback for fake-openai-gpt-4 via /config/update so retries and local reruns don't trip on leftover state. - Tighten the success assertion to a single tbody row containing BOTH the primary and the fallback names — pre-existing rows can no longer vacuously satisfy the check. - Fix the stale "Three tabs" comment to "Four tabs". Addresses Greptile P2s on PR #29069. * fix(e2e): keyboard-select fallback models + correct cleanup endpoint - Replace mouse-based option clicks with click-to-focus + type + Enter. FallbackGroupConfig's Selects use `options=` and a custom getPopupContainer, so locating options via `.ant-select-dropdown` hit several races: DOM-clicks left antd's popup state stale (the primary popup then intercepted the fallback click), `getByRole` matched always-mounted hidden options, and pointer stability fought the open animation. Typing into the showSearch input narrows the listbox to one option and Enter selects it cleanly. - Assert on dialog-side state changes (the active tab adopts the primary model name; the chain helper shows "1/10 used") instead of popup contents — these reflect the actual selection landing. - Cleanup helper now hits /get/config/callbacks (the real endpoint; /get/callbacks returns 404), so the before/after reset actually clears prior router_settings.fallbacks state. |
||
|
ryan-crabbe-berriandGitHub
|
7cae5dc08a |
test(ui): e2e cover team model edit + admin identity in navbar (#28652)
* test(ui): e2e cover team model edit + admin identity in navbar
Adds two Playwright tests as part of the manual-QA → e2e migration:
"Edit team model selection" exercises the Settings tab Models multi-select
+ Save Changes flow on a seeded team, and the existing login test now
opens the User dropdown and asserts the role and User ID render — guarding
against regressions where login succeeds but the auth context is empty.
Resolves LIT-3093
* test(ui): restore seeded models in team-edit test so retries don't fail
The 'Edit team model selection' test removed fake-anthropic-claude from
E2E_TEAM_CRUD_ID without restoring it. CI runs with retries: 2 and the seed
script runs once before the suite, so a flake on this test would fail the
retry at the "tag is visible" assertion. Wrap the test in try/finally and
restore the seeded models via /team/update before and after.
* test(e2e): fail loudly if team/update restore call fails
Surfaces the real cause when the master key is wrong or the proxy is
unreachable, instead of silently leaving the team in a stale state and
failing later on the visibility assertion.
* fix(e2e): match navbar account button by aria-label, not non-existent "User" text
The previous trigger filter (hasText: /^User$/) didn't match the rendered
UserDropdown button — its text is the displayName ("Account" for the
master-key admin, an email for SSO users), never "User". The evaluate
call then timed out after 15s in CI. Use the stable aria-label prefix
the component always emits, and click directly since the dropdown is
configured trigger=["click"] (the synthetic hover was unnecessary).
|
||
|
ryan-crabbe-berriandGitHub
|
73e9071311 |
refactor(ui): extract auth state into AuthContext (#28910)
* refactor(ui): extract auth state into AuthContext
Move auth state (token, userID, userRole, accessToken, premiumUser, userEmail,
disabledPersonalKeyCreation, showSSOBanner) out of src/app/page.tsx into a
new AuthProvider at src/contexts/AuthContext.tsx. Wrapped at the root layout
so login/onboarding/dashboard routes all have access via useAuth().
Day 1 foundation for the App Router migration: migrated (dashboard)/X/page.tsx
route entry points won't have a parent passing props, so shared auth state
must live in a context they can read from.
Sub-components are unchanged — they still receive accessToken/userID/userRole
as props from page.tsx (which now reads them from useAuth()). Only the
page.tsx → top-level-page-component handoff is de-drilled; deeper prop
drilling is left for the per-page migration to address.
Net change: -86 lines from page.tsx (state + two effects moved), +5 in
layout.tsx (provider wrap), new AuthContext.tsx (~140 lines), test update
to wrap CreateKeyPage in AuthProvider.
Fixes LIT-3366
Part of LIT-3128
* fix(ui): await getUiConfig before clearing authLoading
The AuthContext refactor flipped authLoading to false synchronously on mount
while letting getUiConfig() run fire-and-forget. On SERVER_ROOT_PATH deployments
this races the unauthenticated login-redirect effect: the redirect fires with
proxyBaseUrl still at its module-init value, sending users to /ui/login instead
of {SERVER_ROOT_PATH}/ui/login.
Restores the original sequencing inside AuthProvider's mount effect and adds a
Playwright spec wired into the existing SERVER_ROOT_PATH workflow matrix. The
spec delays the config endpoint via page.route() to make the race deterministic
across CI runners.
|
||
|
ryan-crabbe-berriandGitHub
|
13512e7abd |
refactor(ui): remove dead App Router scaffolding in (dashboard)/* (#28891)
Deletes 39 files across 12 unused subdirs under src/app/(dashboard)/. None of these routes are reachable: LEGACY_REDIRECTS in src/app/page.tsx is empty, the live UI renders everything via the legacy ?page=X switch, and no other code statically imports from these dirs. Kept api-reference, models-and-endpoints, organizations, playground, virtual-keys because they ARE imported by the legacy switch in src/app/page.tsx — they will be migrated properly in the App Router migration. Kept shared infra: layout.tsx, hooks/, components/, networking.ts, README.md — these are imported by live code. Clean baseline before the App Router migration (LIT-3128). |
||
|
ryan-crabbe-berriandGitHub
|
f75a7c6b22 |
fix(model-edit): allow clearing custom pricing on wildcard models (#28719)
* fix(model-edit): allow clearing custom input/output cost on wildcard deployments A user-set pricing override on a `/model/*` wildcard deployment could not be removed: clearing the Input/Output Cost fields in the UI succeeded visually, but the next read still showed the old values because both `litellm_params` and `model_info` (mirrored via `SPECIAL_MODEL_INFO_PARAMS`) retained the original rates. UI: when the pricing field is touched but left empty, send `null` instead of dropping it from the payload so the backend sees the clear intent. The cache-read-cost fallback now guards against `null` as well as `undefined` so a cleared input cost cannot silently wipe the cache-read override. Backend: `update_db_model` honors explicit-null clears, but ONLY for `SPECIAL_MODEL_INFO_PARAMS` (the 4 pricing fields). Restricting the null-clear path prevents a team-scoped caller from using this codepath to null out privileged fields like `team_id` or access groups. Tests cover both clear paths (`litellm_params` and `model_info`), the SPECIAL_MODEL_INFO_PARAMS mirror, PATCH semantics for omitted fields, and the security guard that non-pricing nulls don't reach the merged dict. Resolves LIT-3250 * fix(model-edit): run null-clears after both merges, not interleaved The previous version cleared `model_info` from inside the litellm_params merge block, but the subsequent `model_info.update(...)` re-injected the old pricing because the UI's PATCH carries the full model_info blob with the stale values still in it. Move the explicit-null clear pass to after both merges so a model_info passthrough cannot resurrect cleared fields. Adds a regression test for the realistic UI submit shape (both blobs in the patch, model_info still holding the old pricing). * test(e2e): clear-custom-pricing flow with create/delete cleanup Covers the dashboard model edit form's pricing-clear flow end-to-end: seeds a deployment with custom input/output pricing, drives the UI to clear both fields, asserts the outgoing PATCH sends explicit nulls, and confirms via /v2/model/info that the override is gone from both litellm_params and model_info. The dashboard DB persists across this suite, so beforeEach creates a uniquely-named deployment and afterEach POSTs /model/delete to leave the DB clean regardless of test outcome. * fix(model-edit): extend pricing clear to cache_read and cache_write costs Pre-existing parallel of the wildcard input/output cost bug: cleared cache_read_input_token_cost and cache_creation_input_token_cost overrides silently persisted because the UI omitted the key (delete or fallback) and the backend null-clear allowlist did not cover them. - types/router.py: add cache_read_input_token_cost and cache_creation_input_token_cost to SPECIAL_MODEL_INFO_PARAMS, so they are mirrored between litellm_params and model_info by Deployment.__init__ and honoured by the null-clear loop in update_db_model. - model_info_view.tsx: emit explicit null for touched-but-empty cache_read and cache_write fields. Preserve the input_cost->cache_read mirror only when cache_read itself was not touched. - model_management_endpoints.py: update the allowlist comment. - Tests: three new unit tests for cache clear paths and a preserve check; the e2e spec now seeds, clears, and asserts null PATCH + key-absence for all four pricing fields. |
||
|
ryan-crabbe-berriandGitHub
|
a8263cbc88 |
fix(ui): route API Reference back to query-param page (#28726)
* fix(ui): route API Reference back to query-param page The path-based /ui/api-reference route was broken in practice — the page-local useProxySettings hook didn't match what the root page passes down. Remove api_ref from the migration maps (LEGACY_REDIRECTS in app/page.tsx, MIGRATED_PAGES in leftnav.tsx and (dashboard)/layout.tsx), point the leftnav item back at page="api_ref", and restore the api_ref render branch in the root page. The path-based page.tsx and the useProxySettings hook stay in place unchanged; only api_ref is moved back to query-param routing while the migration infrastructure is preserved for future page moves. * fix(ui): alias ?page=api-reference to api_ref branch Handles bookmarks of the hyphen-form query param that was live during the brief path-based migration window, so they render the working APIReferenceView instead of falling through to the default page. |
||
|
Mateo WangandGitHub
|
48d7e15b83 |
chore(admin-ui): regenerate static export with trailingSlash: true (#28112)
* chore(admin-ui): regenerate static export with trailingSlash: true Rebuilds litellm/proxy/_experimental/out/ from ui/litellm-dashboard with `trailingSlash: true` enabled in next.config.mjs. Next.js now emits every route as <dir>/index.html (e.g. mcp/oauth/callback/index.html) instead of <dir>.html with a sibling metadata-only directory, which fixes the 404 on extensionless URLs served through FastAPI's StaticFiles(html=True) mount. This is the build artifact half of the fix; the config change, Dockerfile cleanup, and regression test live in the follow-up source PR that stacks on top of this branch. * fix(admin-ui): emit nested routes as <dir>/index.html (#28106) Linear and other OAuth providers redirect the user back to /ui/mcp/oauth/callback?code=...&state=... after the consent step. The packaged Next.js static export only produced /ui/mcp/oauth/callback.html, so FastAPI's StaticFiles served a 404 on the extensionless URL and the OAuth handshake never completed. The Dockerfile.non_root build step tried to paper over this at image-build time with `for html_file in *.html; do ...`, but that shell glob does not recurse, so nested routes like mcp/oauth/callback.html were left stranded next to an empty mcp/oauth/callback/ directory containing only Next.js metadata. The runtime restructure step in proxy_server.py was then skipped because the .litellm_ui_ready marker had already been dropped. Set trailingSlash: true in the dashboard's Next.js config so the export emits every nested route as <dir>/index.html natively. The Dockerfile loop is now a no-op for the bundled UI and has been removed; the .litellm_ui_ready marker is still written so the proxy keeps skipping the redundant Python restructure step at startup. Stacks on top of the static export regeneration in the parent branch. * chore: restore origin/litellm_internal_staging out files |
||
|
ryan-crabbe-berriandGitHub
|
c127968dfb |
fix(ui): show 2-decimal precision for max_budget on key overview (#28809)
The Key Info Overview tab's Spend card truncated sub-dollar budgets to "$0" because formatNumberWithCommas defaults to 0 decimals. The Settings tab passes 2; align the overview so a $0.10 budget renders as "$0.10". Resolves LIT-2845 |