## Problem
The `extra_body` parameter in `litellm.responses()` and `litellm.aresponses()`
was being accepted but never passed to the HTTP request sent to the LLM provider.
This prevented users from sending custom/experimental parameters to provider APIs.
## Changes
- Added `data.update(extra_body)` in `async_response_api_handler` (line 2138)
- Added `data.update(extra_body)` in `response_api_handler` (line 2012)
- Added tests to `test_openai_responses_api.py` for extra_body functionality
## Testing
- Tests verify extra_body params are passed in both sync and async modes
- Existing Responses API tests continue to pass
- Manually verified with OpenAI API that custom params are sent correctly
## Impact
Users can now pass custom/experimental parameters via extra_body:
```python
litellm.aresponses(
model="gpt-4o",
input="hello",
extra_body={"custom_param": "value"} # Now works!
)
```
This aligns with the OpenAI SDK pattern and matches behavior in other
LiteLLM endpoints (completion, embedding, etc.) that already support extra_body.
* Update MCP version from 1.10.1 to 1.20.0
- Update mcp dependency: 1.10.1 -> 1.20.0 in requirements.txt, pyproject.toml, and CI config
- Update uvicorn dependency: 0.29.0 -> 0.31.1 (required by MCP 1.20.0)
- Update PyJWT constraint to support newer versions required by MCP
- Update all CI pipeline references to MCP 1.20.0
- Add test to verify MCP version and import compatibility
MCP 1.20.0 requires uvicorn >=0.31.1 and PyJWT >=2.10.1.
MCP package remains Python >=3.10 only (no change to version constraint).
* Update poetry.lock for MCP 1.20.0
* Fix bug, add new unit test
* Extract payload builder code to a separate namespace
* Update opik.py to use logic from the new namespace
* Code cleanup, type hints improvements
* Run linter
* Log model name as span field
* Reformat arguments in payload builders
* Use dataclasses for payloads, use opik native client if it's available
* Add cost and provider
* Add provider mapping
- Add HCP_VAULT_MOUNT_NAME env var to override default 'secret' mount
- Add HCP_VAULT_PATH_PREFIX env var to add prefix to secret paths
- Update get_url() method to construct URLs with configurable mount and prefix
- Add test coverage for custom mount names and path prefixes
- Maintain backward compatibility with existing configurations
This allows users to configure Vault paths like:
- Custom mount: {VAULT_ADDR}/v1/{MOUNT_NAME}/data/{SECRET}
- With prefix: {VAULT_ADDR}/v1/secret/data/{PREFIX}/{SECRET}
- Both: {VAULT_ADDR}/v1/{MOUNT_NAME}/data/{PREFIX}/{SECRET}
Resolves issue where mount name was hardcoded and path prefixes weren't supported.
* KeyManagementSystem add cyberark
* add CyberArkSecretManager
* add CyberArkSecretManager
* add CyberArkSecretManager
* docs add CyberArkSecretManager
* docs
* refactor to use get_secret_from_manager
* Potential fix for code scanning alert no. 3645: Clear-text logging of sensitive information
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* Potential fix for code scanning alert no. 3650: Clear-text logging of sensitive information
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* Potential fix for code scanning alert no. 3649: Clear-text logging of sensitive information
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* Potential fix for code scanning alert no. 3646: Clear-text logging of sensitive information
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
---------
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* noma support v2 api and images with during call
* supporting streams and images with texts
* Supporting text now
* annonymization works
* removing function
* fixing noma.py
* all old tests pass
* adding new tests
* removing changes
* Fixing application id headers
* fix whitespace
* deleting unused imports
* Add gemini api key in the custom api url
* Update tests
* Use api key n the header
* Use api key n the header
* fix mypy error
* fix mypy error
* fix test gemini auth
* fix(redis): handle float redis_version from AWS ElastiCache Valkey
AWS ElastiCache Valkey returns redis_version as a float (7.0) instead
of a string ('7.0.0'), causing AttributeError: 'float' object has no
attribute 'split' in async_lpop when parsing version for LPOP count.
Changes:
- Extract version parsing into _parse_redis_major_version() helper
- Add DEFAULT_REDIS_MAJOR_VERSION constant (replaces magic number)
- Support multiple version formats: string, float, int, malformed
- Add comprehensive test coverage for all version format edge cases
Fixes: 'LiteLLM Redis Cache LPOP: - Got exception from REDIS' error
during db_spend_update_job cronjobs
* refactor: move DEFAULT_REDIS_MAJOR_VERSION to constants.py
This commit fixes two bugs in Responses API streaming tests:
1. **Usage field naming bug**: Tests were using `input_tokens` and
`output_tokens` but the Usage object uses `prompt_tokens` and
`completion_tokens`.
2. **Missing cost in streaming usage**: When `include_cost_in_streaming_usage`
was enabled, the cost was calculated and added to ResponseAPIUsage, but was
lost during the transformation to the Usage object.
Changes:
- Updated test assertions to use correct field names (prompt_tokens, completion_tokens)
- Added cost preservation logic in FakeStreamerResponsesAPIIterator
- Modified _transform_response_api_usage_to_chat_usage() to preserve cost attribute
All streaming tests now pass successfully.
* add helper functions
* update generic_cost_per_token function
* add test
* formatting
* add examples in docstring for _calculate_tiered_cost
* Restore files to upstream/main version
* dashscope specific calculation
* improve for different costs
* remove _calculate_flat_cost function
* fix(anthropic-adapter): properly translate Anthropic image format to OpenAI
Fixed bug where images were stripped during Anthropic Messages API to Azure
OpenAI translation. Image source data was being stringified instead of having
fields properly extracted.
- Added _translate_anthropic_image_to_openai() helper method
- Support both base64 and URL image formats per Anthropic API spec
- Refactored user message and tool result image handling
* test(anthropic-adapter): add comprehensive image translation tests
Add 5 unit tests covering image translation from Anthropic to OpenAI format:
- User messages with base64 images
- User messages with URL images
- Tool results with base64 images
- Tool results with URL images
- Mixed content with multiple images
* Add v1 cut of container api
* fix lint errors
* Add proxy support to container apis & logging support (#16049)
* Add proxy support to container apis
* Add logging support
* Add cost tracking support for containers and documentation
* Add new constant documentation
* Add container cost in model map
* fix failing azure tests
* Update tests based on model map changes
* fix model map tests
* fix model map tests
* Container modeshould be container
* Container tests fix
* Merge branch 'main' into litellm_sameer_oct_staging_2
* Add Prometheus metric to track callback logging failures in S3 (#16102)
* Add proxy support to container apis
* Add logging support
* prometheus metric measures how often s3_v2 is failing
* remove not needed files
* remove not needed files
* remove not needed files
* fix mypy errors
* Use logging_callback_manager to get all the callbacks
---------
Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com>
* feat(llm_passthrough_endpoints.py): support milvus passthrough api
* fix(llm_passthrough_endpoints.py): move streaming request value to the top of the function
* docs: document new milvus vector store passthrough flow
Ishaan Jaffer