* add NewModelGroupRequest
* add endpoint for create_model_group
* fix model_access_group_management_router
* add UpdateModelGroupRequest, info and delete
* fix model management tag
* fix validate_models_exist
* fix get_all_access_groups_from_db
* test_create_duplicate_access_group_fails
* test fixes
* fix working create access groups
* fix access group management endpoints
* add is db model checks for model access groups
* Refactor proxy embeddings to use shared processor
- allow ProxyBaseLLMRequestProcessing to accept the aembedding route so embeddings requests reuse the base pipeline hooks
- route embeddings requests through base_process_llm_request, sharing logging, hook execution, retries, and header handling with chat/responses
- tighten token array decoding logic by using router deployment lookups and the unified error handler
* Fix: Correctly process embedding requests with token arrays
The `test_embedding_input_array_of_tokens` test was failing due to a regression that caused embedding requests with token arrays to be processed incorrectly. This prevented the `aembedding` function from being called as expected.
This was caused by a combination of three distinct issues:
1. In `litellm/proxy/common_request_processing.py`, the `function_setup` utility was called with `aembedding` as the `original_function` for embedding routes. This has been corrected to `embedding` to ensure proper request setup.
2. In `litellm/proxy/proxy_server.py`, a `TypeError` occurred because the `get_deployment` method was called with the `model_name` keyword argument instead of the expected `model_id`. This has been corrected. Additionally, the check for token arrays was improved to validate that all elements in the input subarray are integers.
3. In `litellm/proxy/litellm_pre_call_utils.py`, the check for the `enforced_params` enterprise feature was too strict. It blocked valid requests even when the `enforced_params` list was empty. The condition has been adjusted to trigger the check only for non-empty lists.
Finally, the `test_embedding_input_array_of_tokens` assertion was updated to be more robust. The previous `assert_called_once_with` was overly strict, causing failures when unrelated internal parameters were added to the function call. The test now first asserts that `aembedding` is called and then separately verifies the `model` and `input` arguments. This makes the test more resilient to future changes without sacrificing its ability to catch regressions.
* test: align proxy embedding assertions
Update the embedding proxy test to match the new request pipeline: keep the data the proxy builds, expect the extra control kwargs, let the post-call hook return the actual response, and assert the normalized 'embeddings' hook type. This proves the refactor still forwards metadata and returns the mocked payload.
* Update proxy exception test
The proxy now forwards additional kwargs (request_timeout, litellm_call_id, litellm_logging_obj) to llm_router.aembedding. The test needs to accept these to match the real call signature and keep validating the error path instead of the kwargs list.
* testing: unsure of this change
I don't remember why I changed this, will revert and see if any tests fail since the manual test isn't failing without it.
* fix: remove unrelated change
This change was not related to the embeddings refactor and actually belonged to a different branch.
* Use end user budget instead of key budget when creating new team
* Fixed implementation to use user's max budget from the UserTable instead of EndUserTable
Fixes#16470
Video generation endpoints (/v1/videos, /videos/{video_id}, etc.) were
incorrectly restricted to proxy_admin role only. These routes are now
added to openai_routes list, making them accessible to internal_user
role as they should be - video generation is a legitimate user feature,
not a management/admin operation.
Changes:
- Added 8 video route patterns to LiteLLMRoutes.openai_routes in _types.py
- Added comprehensive tests verifying internal_user and virtual key access
- All existing route permission tests continue to pass
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude <noreply@anthropic.com>
* fix: ensure end user endpoints use "handle_exception_on_proxy" correctly
* test 404 on info and update for non-existent user
* test 404 for no customer found
* fix 404 handling for customer endpoints
* test_error_schema_consistency
* test_customer_endpoints_error_schema_consistency
* feat(guardrails): Add deduplication and session tracking
- Implement deduplication logic to prevent duplicate scans (via call_id; add _check_and_mark_scanned) caused by LiteLLM callback system
- Add session tracking using litellm_trace_id as AI Session ID for Prisma AIRS SCM logging
- Extract helper methods _extract_prompt_from_request maintainability
- Use httpxSpecialProvider import (LoggingCallback -> GuardrailCallback)
- Add comprehensive tests for deduplication and session tracking (7 new tests)
- Update documentation with multi-turn conversation tracking examples
* docs: update PANW Prisma AIRS multi-turn conversation example to use industry-standard terminology
- Clearer example for conversation tracking
- Updated terminology from 'AI Session ID' to 'Prisma AIRS AI Session ID' for clarity
* fix: remove unused asyncio import
* fix: correct mypy type ignore comment
* noma support v2 api and images with during call
* supporting streams and images with texts
* Supporting text now
* annonymization works
* removing function
* fixing noma.py
* all old tests pass
* adding new tests
* removing changes
* Fixing application id headers
* fix whitespace
* deleting unused imports
* feat(llm_passthrough_endpoints.py): support milvus passthrough api
* fix(llm_passthrough_endpoints.py): move streaming request value to the top of the function
* docs: document new milvus vector store passthrough flow
* feat(vector_store_endpoints/endpoints.py): add new index_create endpoint
allows admin to create a virtual index, to do permission management for
* feat(key_management_endpoints.py): enable setting allowed_vector_store_indexes on keys
proxy admin can enable dev to create an index on a vector stor
* feat: initial commit adding vector store index passthrough logic to litellm
* feat: add vector store table
* fix(azure_ai/transformation.py): fix headers
* feat: track read/write endpoints by vector store integration
enables permissions by index to work
* fix: azure_ai/vector_stores/search
document the vector store endpoints correctly
ensures permission management works as expected
* fix(proxy/utils.py): improve error message
* docs(azure_ai_vector_stores_passthrough.md): document azure ai passthrough vector store support
* docs(create.md): document azure ai support via passthrough for vector store create
* fix: fix code qa errors
* fix: document new allowed_vector_store_indexes endpoint
* feat(milvus/): initial commit adding milvus vector store support to LiteLLM
allows querying milvus vector store through litellm
* feat(bedrock/vector_stores): support translating openai filters param to aws kb
adds filtering to aws kb
* feat(milvus/): add milvus vector store unified search support
allows calling milvus vector store in through chat completions
* docs(milvus_vector_stores.md): document new milvus vector search integration
* feat(pass_through_endpoints.py): support passing form data through to a passthrough endpoint
Closes LIT-1147
* fix: fix linting errors
* UI Config Guardrails should not be editable and guardrails info definition location fix
* Remove unused import
* Added literals for guardrail definition location
* fix(mcp): respect X-Forwarded-Proto header in OAuth endpoints
When LiteLLM proxy is deployed behind a reverse proxy (like nginx or a load balancer) that terminates SSL/TLS, the proxy receives HTTP requests internally but should expose HTTPS URLs externally. This change detects the X-Forwarded-Proto header and uses it to construct correct redirect URIs and endpoint URLs.
Changes:
- Added X-Forwarded-Proto detection to authorize, token, oauth_protected_resource_mcp, oauth_authorization_server_mcp, and register_client endpoints
- Added comprehensive tests for X-Forwarded-Proto header support across all affected endpoints
- Fixed existing tests to properly mock request.headers
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix formatting
* feat(mcp): support X-Forwarded-Host for proxy base URL reconstruction
Extended X-Forwarded-Proto support to also handle X-Forwarded-Host and X-Forwarded-Port headers. This allows LiteLLM to correctly construct redirect URIs and endpoint URLs when deployed behind a reverse proxy that changes the host/port.
Example scenario:
- Internal URL: http://localhost:8888/github/mcp
- External URL: https://proxy.abc.com/github/mcp
- Proxy sets: X-Forwarded-Proto: https, X-Forwarded-Host: proxy.abc.com
Changes:
- Added get_request_base_url() helper function to centralize X-Forwarded-* header handling
- Replaced all inline X-Forwarded-Proto checks with calls to the helper function
- Helper handles X-Forwarded-Proto, X-Forwarded-Host, and X-Forwarded-Port
- Added tests for X-Forwarded-Host scenarios in authorize and token endpoints
Fixes issue where protected resource URL mismatch occurred:
Error: Protected resource http://proxy.abc.com:8888/github/mcp
does not match expected https://proxy.abc.com/github/mcp🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* chore: replace Yelp-specific hostnames with generic examples
Changed all references from chatproxy.yelpcorp.com to proxy.example.com in:
- test_proxy_forwarding.py (default host parameter)
- TEST_PROXY_FORWARDING.md (documentation examples)
- discoverable_endpoints.py (docstring example)
- test_discoverable_endpoints.py (test mock data)
This makes the code more generic and suitable for open source.
All 13 tests still passing.
* remove accidentally added files
* fix formatting
* add new test for get_base_url
---------
Co-authored-by: Claude <noreply@anthropic.com>
* perf: Skip sleep delays in base_mail.py during tests to improve test speed
* perf: Mock datetime.now in parallel_request_limiter_v3.py to improve test speed
* pref: Mock urllib system calls in test_aiohttp_transport.py to improve test speed
* chore: add --durations=50 to visualize slowest tests
* pref: reduce setup phase overhead by widening fixture scope in conftest.py
* test: stabilize flaky tests
* fix: minor issue
