Per maintainer feedback, FastAPI should always be available in proxy code.
The issue was that MCP tests were importing from proxy_server unnecessarily,
pulling in all proxy dependencies including policy_resolve_endpoints.
Fix:
- Revert policy_resolve_endpoints.py to use direct FastAPI imports
- Update MCP tests to import LiteLLM_ObjectPermissionTable from litellm.proxy._types
instead of litellm.proxy.proxy_server
This avoids importing the entire proxy_server module with all its dependencies
when tests only need specific types.
Addresses: https://github.com/BerriAI/litellm/pull/21075/changes#r2802201174
Greptile's previous suggestion accidentally removed the if TYPE_CHECKING block,
leaving an orphan else statement that caused a syntax error.
This commit restores the proper structure.
Adds try/except around FastAPI imports with fallback mock classes.
This allows the module to be imported in test environments where
proxy dependencies (FastAPI) may not be installed.
Fixes NameError when MCP tests try to import from proxy_server which
imports from this module:
- NameError: name 'APIRouter' is not defined
- NameError: name 'Depends' is not defined
- NameError: name 'HTTPException' is not defined
- NameError: name 'Query' is not defined
The test-complete aggregate job adds no value as GitHub Actions
already provides visibility into matrix job results.
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
poetry.lock was out of sync with pyproject.toml, causing CI failures
across all PRs with "Run `poetry lock` to fix the lock file" error.
Regenerated the lock file to resolve the sync issue.
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
* feat(bedrock): add DeepSeek V3.2 pricing and region support
* feat(bedrock): add minimax.minimax-m2.1 pricing and region support
* feat(bedrock): add moonshotai.kimi-k2.5 pricing and region support
* feat(bedrock): add qwen.qwen3-coder-next
pricing and region support
* add some sanity unit tests for the bedrock beta models added
* --amend
* resolve greptileai comments and suggestions
* docs: add reference to example_openai_endpoint repo for self-hosting fake OpenAI proxy (#21006)
- Updated benchmarks.md with a section on setting up fake OpenAI endpoints
- Updated load_test.md to mention the self-hosted option
- Updated load_test_advanced.md with a tip box about the example repo
Reference: https://github.com/BerriAI/example_openai_endpoint
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
* MCP fixes
* fix(oldteams.tsx): show policies when creating
* fix(proxy/_types.py): ensure mcp rest endpoints can be called by virtual key
ensures UI works with virtual key testing mcp endpoints
* refactor: migrate get object permissions table logic to happen in user api key auth - allows functions to trust user api key object they receive has what they need
* fix(rest_endpoints.py): filter for allowed tools based on what key has access to
* fix(mcp_server_manager.py): ensure only allowed MCP's are returned to the user, via rest endpoints
* Guardrails - add toxic/abusive content filter guardrails
* fix(streaming): preserve usage data from post-finish_reason chunks in OpenAI-compatible streaming
Fixes#16112
OpenRouter and other OpenAI-compatible providers send a usage chunk after
the finish_reason='stop' chunk when stream_options.include_usage is True.
The OpenAIChatCompletionStreamingHandler.chunk_parser() was not passing
the usage field to ModelResponseStream, causing real token counts from the
provider to be lost and falling back to inaccurate estimates.
* fix: resolve merge conflict in test file
- Fix typo in test method name (extra space)
- Move test_prompt_cache_key_in_optional_params to its own class
---------
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
* fix(vertex_ai): forward extra_body to completion transformation handler
The responses() function accepted extra_body as a named parameter but
did not pass it to response_api_handler when responses_api_provider_config
was None (completion transformation path), silently dropping it.
Also adds deep-merge support for extra_body in Vertex AI Gemini
transformation, so dict values like generationConfig are merged rather
than replaced.
* refactor(vertex_ai): extract _merge_extra_body to fix PLR0915 lint
Move the extra_body merge loop into a helper function to keep
_transform_request_body under the 50-statement limit.
When litellm_metadata is explicitly set to None in litellm_params,
`dict.get("litellm_metadata", {})` returns None (not the default {}),
because the key exists. The subsequent .get() call on None raises
`AttributeError: 'NoneType' object has no attribute 'get'`.
Use `or {}` instead, consistent with line 4924 in the same file.
Fixes#15836
* fix: allow Management keys to access user/daily/activity and team/daily/activity
* feat(vertex): surface trafficType via generic provider_specific_fields in Responses API
Extract Vertex AI's trafficType from usageMetadata in both streaming and
non-streaming paths, storing it in _hidden_params["provider_specific_fields"].
The Responses API transformation layer generically passes any
_hidden_params["provider_specific_fields"] dict to the ResponsesAPIResponse,
avoiding provider-specific logic in the bridge.
Also fix stream_chunk_builder to propagate _hidden_params from the last
streaming chunk to the rebuilt ModelResponse, ensuring provider metadata
survives the chunk→response rebuild.
---------
Co-authored-by: naaa760 <neh6a683@gmail.com>
Co-authored-by: yuneng-jiang <yuneng.jiang@gmail.com>
Julio Quinteros Pro