Add detection for Cerebras's context window exceeded error format:
"Current length is X while limit is Y"
This ensures LiteLLM raises ContextWindowExceededError instead of
generic BadRequestError when Cerebras API calls exceed the model's
context limit, enabling downstream libraries like DSPy to properly
catch and handle these errors for automatic context management.
The 'user' parameter was being ignored when using responses API models
(e.g., model="openai/responses/gpt-4.1") because the model name check
in get_supported_openai_params() didn't account for the "responses/" prefix.
Fix: Normalize the model name by stripping "responses/" prefix before
checking if the model is in the list of supported OpenAI models.
This is a minimal, non-breaking change that:
- Adds 2 lines of code in gpt_transformation.py
- Only affects the parameter support check, not the model variable itself
- Includes unit and integration tests
When OpenAI Responses API returns both text AND tool_calls, the bridge
transformation was emitting is_finished=True after the text message completed,
causing subsequent tool_call chunks to be dropped.
The fix:
- response.output_item.done for messages no longer emits is_finished=True
- Added handler for response.completed to properly signal stream end
* refactor: remove api-key conversion logic for Azure Anthropic
Co-authored-by: Erdem Halil <erdemhalil@users.noreply.github.com>
* fix(passthrough): pass custom_llm_provider to completion_cost for Azure AI Anthropic
The passthrough logging for Anthropic was failing when using Azure AI Anthropic
because the completion_cost function was not receiving the custom_llm_provider
parameter, causing it to fail with "LLM Provider NOT provided" error.
This fix:
- Retrieves custom_llm_provider from logging_obj.model_call_details
- Prepends provider prefix to model name for cost calculation
- Passes both formatted model and custom_llm_provider to completion_cost
- Centralizes provider prefix logic in _create_anthropic_response_logging_payload
This ensures cost calculation works correctly for Azure AI Anthropic requests
with models like azure_ai/claude-sonnet-4-5_gb_20250929.
Co-authored-by: Erdem Halil <erdemhalil@users.noreply.github.com>
* test: add unit tests for Azure AI Anthropic fixes
- Add tests for custom_llm_provider cost calculation in passthrough logging
- Add tests for ProviderConfigManager returning AzureAnthropicMessagesConfig
- Update existing tests to reflect removal of api-key to x-api-key conversion
Co-authored-by: Erdem Halil <erdemhalil@users.noreply.github.com>
---------
Co-authored-by: Erdem Halil <erdemhalil@users.noreply.github.com>
When using litellm with a recent pydantic, the following deprecation warnings are shown
```
litellm/types/llms/anthropic.py:531: PydanticDeprecatedSince20: Support for class-based `config` is deprecated, use ConfigDict instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.12/migration/
class AnthropicResponseContentBlockToolUse(BaseModel):
litellm/types/rag.py:181: PydanticDeprecatedSince20: Support for class-based `config` is deprecated, use ConfigDict instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.12/migration/
class RAGIngestRequest(BaseModel):
```
This aligns with multiple existing places within the litellm codebase that uses ConfigDict to configure pydantic model behavior.
* docs: update Getting Started page with accurate endpoints and fix exception handling
- Update endpoints list to include /responses, /audio, /batches
- Change "Consistent output" to be endpoint-agnostic
- Clarify Response Format title as "OpenAI Chat Completions Format"
- Fix exception handling example: use litellm exceptions instead of deprecated openai.error
- Add model prefix (anthropic/) to example
* docs: reorganize sidebar and improve SDK documentation structure
Sidebar changes:
- Reorder: Python SDK first, then AI Gateway (Proxy)
- Rename "LiteLLM - Getting Started" to "Getting Started"
- Restructure SDK section with Core Functions, Configuration subsections
- Move budget_manager to Guides
- Move sdk_custom_pricing and migration to Extras
- Remove duplicate embedding/async_embedding and embedding/moderation
Content changes:
- Add Response Format section to response_api.md
- Add async aembedding() section to supported_embedding.md
* docs: add deprecation notice for OpenAI Assistants API
OpenAI has deprecated the Assistants API, shutting down on August 26, 2026.
Added warning banner directing users to the Responses API.
* docs: expand Core Functions in SDK sidebar
Add more SDK functions to Core Functions category:
- text_completion()
- image_generation()
- transcription()
- speech()
- Link to "All Supported Endpoints" for complete list
* Rename Sidebar Item
* docs: revert Getting Started label to original
* Rename sidebar label from 'LiteLLM - Getting Started' to 'Getting Started'
* Fix test_delete_polling_removes_from_cache mock setup
- Mock async_delete_cache to properly execute the real implementation path
- Ensures init_async_client() is called and delete() is invoked on the returned client
- Fixes AssertionError: Expected 'delete' to be called once. Called 0 times.
* fix: resolve timeout in add_model_tab test by mocking useProviderFields hook
- Mock useProviderFields hook to prevent network calls and React Query delays
- Use waitFor to properly handle async operations
- Test now passes reliably without 10s timeout
* fix: add test timeout to prevent CI timeout failure
- Add 15 second timeout to 'should display Test Connect and Add Model buttons' test
- Test takes ~6 seconds locally, but CI was timing out at default 5 second limit
- Ensures test has sufficient time to complete in CI environment
* test: quarantine flaky test_oidc_circleci_with_azure
Quarantine test that fails with 401 Unauthorized from Azure OAuth.
The test is flaky and blocks CI builds. Marked with @pytest.mark.skip
until Azure authentication can be fixed or migrated to our own account.
* fix: correct type annotations for anthropic streaming handlers
- Fix return type of _handle_accumulated_json_chunk from Optional[GenericStreamingChunk] to Optional[ModelResponseStream]
- Fix return type of _parse_sse_data from Optional[GenericStreamingChunk] to Optional[ModelResponseStream]
- Add type annotation for output_items in background_streaming.py
These changes align type annotations with actual return values from chunk_parser() which returns ModelResponseStream.
* docs: add missing ONYX_API_KEY and ONYX_API_BASE to environment variables reference
- Add ONYX_API_BASE documentation entry
- Add ONYX_API_KEY documentation entry
- Fixes test_env_keys.py test failure
Fixes#17473 - Anthropic streaming fails with JSONDecodeError when
network fragmentation causes SSE data to arrive in partial chunks.
Changes:
- Add accumulated_json buffer and chunk_type to ModelResponseIterator
- Add _handle_accumulated_json_chunk() to accumulate partial JSON
- Add _parse_sse_data() to handle both complete and partial chunks
- Modify __next__ and __anext__ to use accumulation logic
- Add unit tests for partial chunk handling
The async_post_call_streaming_iterator_hook function was broken:
1. Was a sync function (def) not async generator
2. Returned AsyncGenerator without iterating it
3. Callback generators were chained but never consumed
This fix:
1. Makes the function an async generator (async def + yield)
2. Actually iterates through the chained callbacks with 'async for'
3. Properly yields chunks to the caller
Fixes#9639
* Fix: Support generic_api_compatible_callbacks.json in callback initialization
- Added check in _add_custom_callback_generic_api_str to load callbacks from generic_api_compatible_callbacks.json
- Added SumoLogic webhook integration to generic_api_compatible_callbacks.json
- Fixes bug where callbacks in JSON file were not being loaded
* Added 3 unit tests for JSON callback loading
Update the setup instructions to guide contributors to fork the repository
on GitHub before cloning, which is the standard GitHub workflow for
open source contributions.
yuneng-jiang