Complements the stubbed-out live integration test by verifying the
outgoing Bedrock Converse request body for GPT-OSS is well-formed when
the caller supplies a tool schema with OpenAI-style metadata
($id, $schema, additionalProperties, strict):
- correct converse URL for bedrock/converse/openai.gpt-oss-20b-1:0
- toolConfig.tools[0].toolSpec has the expected name/description
- inputSchema.json keeps type/properties/required and strips fields
Bedrock does not accept
GPT-OSS on Bedrock intermittently emits truncated toolUse.input deltas
(e.g. accumulated args of '{"":"'), causing
test_function_calling_with_tool_response to hard-fail on json.loads.
The model flakiness is not a litellm regression: the same base test
passes for Anthropic in the same CI run, and the streaming delta path
at invoke_handler.py has not changed recently.
Follow the existing override pattern in TestBedrockGPTOSS
(test_prompt_caching, test_completion_cost, test_tool_call_no_arguments)
and stub the test to pass. The underlying bedrock converse streaming
tool-call path is already covered by Claude/Nova/Llama Converse suites
in test_bedrock_completion.py and test_bedrock_llama.py, so removing
the live GPT-OSS check loses no unique litellm-side signal.
Bedrock GPT-OSS occasionally emits truncated toolUse.input deltas
(e.g. accumulated args of '{"":"'), which causes
test_function_calling_with_tool_response to hard-fail on json.loads.
Other overrides in TestBedrockGPTOSS already handle similar
model-side flakiness; apply retries=6 delay=5 scoped to this subclass
so other providers keep strict behavior.
Mixtral-8x7B-Instruct-v0.1 is no longer on Together AI's serverless tier
and now requires a dedicated endpoint, causing multiple tests to fail in CI:
- test_together_ai.py::TestTogetherAI::test_empty_tools
- test_completion.py::test_completion_together_ai_stream
- test_completion.py::test_customprompt_together_ai
- test_completion.py::test_completion_custom_provider_model_name
- test_text_completion.py::test_async_text_completion_together_ai
Qwen/Qwen3.5-9B is currently serverless on Together AI and supports
function calling, satisfying BaseLLMChatTest capability requirements.
The previous v1.83.3 changelog was generated against v1.83.0-nightly and
missed ~3 weeks of work. This regenerates it against the previous stable
release and restructures the LLM API Endpoints section to group by API
type (Responses, Batch, Count Tokens, Video Generation, Pass-Through,
etc.) matching the convention used in v1.82.3, v1.82.0, and v1.81.14.
Adds ~25 previously uncited PRs, cross-section duplications for
cross-cutting changes, and a verified first-time-contributors list.
- Add 8 content PRs that merged directly to the release branch outside the listed staging PRs: #23769 (Ramp callback), #25252 (JWT OAuth2 override), #25254 (AWS GovCloud mode), #25258 (batch-limit cleanup), #25334 (router custom_llm_provider), #25345 (Triton embeddings), #25347 (tag-based routing), #25358 (Baseten pricing attribution)
- Add @kedarthakkar to new contributors (first-ever PR via #23769)
- Update RELEASE_NOTES_GENERATION_INSTRUCTIONS: require walking git log range between release tags in addition to staging PRs, and verify new-contributor status per author rather than trusting the GH release body floor
Pre-select "Internal User Viewer" in the Global Proxy Role dropdown
on both the standalone and embedded Invite User forms so admins don't
have to remember to pick a role, and the default lands on the least
privileged option rather than silently posting an undefined role.
user_dashboard.tsx imports getCookie from @/utils/cookieUtils, but the
vi.mock factory in user_dashboard.test.tsx only exports clearTokenCookies.
Vitest throws `No "getCookie" export is defined on the "@/utils/cookieUtils"
mock`, breaking all three beforeunload-listener tests.
Add getCookie to the mock factory so it matches the current imports.
The Logs view's Team ID filter dropdown was reading `allTeams` from the
root `teams` state in page.tsx, which the Teams page search overwrites
with its filtered subset. Applying a team search on the Teams page made
filtered-out teams disappear from the Logs filter dropdown.
Swap the Team ID filter to use the existing `TeamDropdown` component via
a small `FilterTeamDropdown` wrapper that adapts it to the filter slot's
`FilterOptionCustomComponentProps` contract. The dropdown now drives its
own `useInfiniteTeams` query against `/v2/team/list` with server-side
search and an isolated react-query cache, unreachable from root state.
Rename the now-unused `hookAllTeams` destructure to `allTeams` so the
`KeyInfoView` passthrough receives the hook's unpolluted fetch instead
of the polluted prop, and drop the dead `allTeams` prop from
`SpendLogsTable` and both of its call sites.
Vertex AI rejects requests containing both search tools (googleSearch,
enterpriseWebSearch, urlContext) and function declarations with error:
'Multiple tools are supported only when they are all search tools.'
When _merge_tools_from_deployment() combines deployment-level search
tools with user-request function tools (e.g. via MCP), the mixed tool
list causes a 400 error. This fix detects the conflict in _map_function()
and drops search tools, keeping function declarations.
Non-search tools like code_execution and computerUse are preserved.
Fixes#23337
* fix(router): discard oldest entry when trimming latency list in lowest_latency strategy
The lowest_latency routing strategy keeps a rolling window of the most
recent latency and time-to-first-token measurements per deployment. When
the window is full, the strategy was discarding the *newest* value
instead of the oldest, because the trim used
`[: max_latency_list_size - 1]` (keeping indices 0..N-2) rather than
`[1:]` (dropping index 0 and keeping indices 1..N-1).
Since new values are appended at the end, the bug meant the most recent
measurement was always dropped once the list reached capacity. The
routing decisions then relied on stale data (including any early-spike
values that never aged out), and timeout penalties written via
`async_log_failure_event` were silently discarded as well.
Fix the slice in all five call sites (sync + async log_success_event for
both latency and time_to_first_token, and async_log_failure_event for
the timeout penalty) and add regression tests covering each path.
* test(router): cover async TTFT trim path in lowest_latency regression tests
Adds test_ttft_list_trimming_discards_oldest_entry_async, an async
counterpart to test_ttft_list_trimming_discards_oldest_entry that drives
async_log_success_event with a ModelResponse and completion_start_time so
the async time_to_first_token trim branch is actually exercised.
Previously no test touched that code path: the sync TTFT test used
log_success_event, and the async latency test passed a plain dict
response_obj without stream/completion_start_time, so TTFT was never
computed and the async trim was unreached. Verified load-bearing by
reverting only the async TTFT slice — the new test fails and all others
pass.
* format
* fix#25506
* address greptile review feedback
* [Test] UI - Models: Add E2E tests for Add Model flow
Add E2E tests covering:
- Test connection with bad credentials shows failure modal
- Adding a specific model and verifying it appears in All Models table
- Adding a wildcard route and verifying it appears in All Models table
- Verifying model dropdown shows provider-specific models (existing test updated)
Added data-testid attributes to UI components to support stable test selectors.
Tests verified passing 3/3 consecutive runs with zero flakiness.
* address greptile review feedback (greploop iteration 1)
Add cleanup helper to delete models created during tests, preventing
stale data accumulation across repeated test runs.
* fix CI: replace data-testid selectors with text/role-based selectors
The data-testid attributes added to React components are not present
in the CI-built UI output. Switch to using getByRole and getByText
selectors which work with the rendered DOM regardless of build cache.
* remove unnecessary cleanup helper
The database is freshly seeded on every test run via seed.sql,
so per-test cleanup is not needed.
---------
Co-authored-by: Yuneng Jiang <yuneng@berri.ai>
Co-authored-by: Krrish Dholakia <krrish+github@berri.ai>
* fix: emit input_json_delta for tool args bundled in first streaming chunk
Some providers (xAI, Gemini) include tool_call function arguments in the
same streaming chunk as the function name/id. The AnthropicStreamWrapper
was discarding the trigger chunk entirely when starting a new content
block, which silently dropped the input_json_delta carrying tool
arguments. This caused tool_use blocks to arrive with empty input {}.
Now queue the processed_chunk after content_block_start when it carries
non-empty input_json_delta data. Backward compatible: providers that send
empty arguments in the first chunk (OpenAI-style) are unaffected since
the condition checks for truthy partial_json.
* test: add tests for input_json_delta emission on bundled tool args
Covers the fix for providers (xAI, Gemini) that bundle tool_call
arguments in the same streaming chunk as the function name/id.
Verifies the AnthropicStreamWrapper emits input_json_delta after
content_block_start, and that empty-arg chunks (OpenAI-style) are
unaffected.
* style: apply Black formatting to streaming_iterator.py
* fix: mirror input_json_delta fix to sync __next__ and add sync tests
* test: make no_extra_delta tests assert explicitly instead of passing silently
### Background
The Gemini batchEmbedContents response handler hardcoded `index=0` for
every embedding in the response. Any consumer relying on the OpenAI-format
`index` field to match embeddings back to inputs would silently get wrong
associations.
### Changes
Use `enumerate` in `process_response` so each embedding gets its
positional index instead of 0.
### Test Plan
Added unit test asserting sequential indices and correct vector ordering
for a 3-element batch response.
shivam