* feat(guardrails): Add deduplication and session tracking
- Implement deduplication logic to prevent duplicate scans (via call_id; add _check_and_mark_scanned) caused by LiteLLM callback system
- Add session tracking using litellm_trace_id as AI Session ID for Prisma AIRS SCM logging
- Extract helper methods _extract_prompt_from_request maintainability
- Use httpxSpecialProvider import (LoggingCallback -> GuardrailCallback)
- Add comprehensive tests for deduplication and session tracking (7 new tests)
- Update documentation with multi-turn conversation tracking examples
* docs: update PANW Prisma AIRS multi-turn conversation example to use industry-standard terminology
- Clearer example for conversation tracking
- Updated terminology from 'AI Session ID' to 'Prisma AIRS AI Session ID' for clarity
* fix: remove unused asyncio import
* fix: correct mypy type ignore comment
* noma support v2 api and images with during call
* supporting streams and images with texts
* Supporting text now
* annonymization works
* removing function
* fixing noma.py
* all old tests pass
* adding new tests
* removing changes
* Fixing application id headers
* fix whitespace
* deleting unused imports
* feat(llm_passthrough_endpoints.py): support milvus passthrough api
* fix(llm_passthrough_endpoints.py): move streaming request value to the top of the function
* docs: document new milvus vector store passthrough flow
* feat(vector_store_endpoints/endpoints.py): add new index_create endpoint
allows admin to create a virtual index, to do permission management for
* feat(key_management_endpoints.py): enable setting allowed_vector_store_indexes on keys
proxy admin can enable dev to create an index on a vector stor
* feat: initial commit adding vector store index passthrough logic to litellm
* feat: add vector store table
* fix(azure_ai/transformation.py): fix headers
* feat: track read/write endpoints by vector store integration
enables permissions by index to work
* fix: azure_ai/vector_stores/search
document the vector store endpoints correctly
ensures permission management works as expected
* fix(proxy/utils.py): improve error message
* docs(azure_ai_vector_stores_passthrough.md): document azure ai passthrough vector store support
* docs(create.md): document azure ai support via passthrough for vector store create
* fix: fix code qa errors
* fix: document new allowed_vector_store_indexes endpoint
* feat(milvus/): initial commit adding milvus vector store support to LiteLLM
allows querying milvus vector store through litellm
* feat(bedrock/vector_stores): support translating openai filters param to aws kb
adds filtering to aws kb
* feat(milvus/): add milvus vector store unified search support
allows calling milvus vector store in through chat completions
* docs(milvus_vector_stores.md): document new milvus vector search integration
* feat(pass_through_endpoints.py): support passing form data through to a passthrough endpoint
Closes LIT-1147
* fix: fix linting errors
* UI Config Guardrails should not be editable and guardrails info definition location fix
* Remove unused import
* Added literals for guardrail definition location
* fix(mcp): respect X-Forwarded-Proto header in OAuth endpoints
When LiteLLM proxy is deployed behind a reverse proxy (like nginx or a load balancer) that terminates SSL/TLS, the proxy receives HTTP requests internally but should expose HTTPS URLs externally. This change detects the X-Forwarded-Proto header and uses it to construct correct redirect URIs and endpoint URLs.
Changes:
- Added X-Forwarded-Proto detection to authorize, token, oauth_protected_resource_mcp, oauth_authorization_server_mcp, and register_client endpoints
- Added comprehensive tests for X-Forwarded-Proto header support across all affected endpoints
- Fixed existing tests to properly mock request.headers
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix formatting
* feat(mcp): support X-Forwarded-Host for proxy base URL reconstruction
Extended X-Forwarded-Proto support to also handle X-Forwarded-Host and X-Forwarded-Port headers. This allows LiteLLM to correctly construct redirect URIs and endpoint URLs when deployed behind a reverse proxy that changes the host/port.
Example scenario:
- Internal URL: http://localhost:8888/github/mcp
- External URL: https://proxy.abc.com/github/mcp
- Proxy sets: X-Forwarded-Proto: https, X-Forwarded-Host: proxy.abc.com
Changes:
- Added get_request_base_url() helper function to centralize X-Forwarded-* header handling
- Replaced all inline X-Forwarded-Proto checks with calls to the helper function
- Helper handles X-Forwarded-Proto, X-Forwarded-Host, and X-Forwarded-Port
- Added tests for X-Forwarded-Host scenarios in authorize and token endpoints
Fixes issue where protected resource URL mismatch occurred:
Error: Protected resource http://proxy.abc.com:8888/github/mcp
does not match expected https://proxy.abc.com/github/mcp🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* chore: replace Yelp-specific hostnames with generic examples
Changed all references from chatproxy.yelpcorp.com to proxy.example.com in:
- test_proxy_forwarding.py (default host parameter)
- TEST_PROXY_FORWARDING.md (documentation examples)
- discoverable_endpoints.py (docstring example)
- test_discoverable_endpoints.py (test mock data)
This makes the code more generic and suitable for open source.
All 13 tests still passing.
* remove accidentally added files
* fix formatting
* add new test for get_base_url
---------
Co-authored-by: Claude <noreply@anthropic.com>
* perf: Skip sleep delays in base_mail.py during tests to improve test speed
* perf: Mock datetime.now in parallel_request_limiter_v3.py to improve test speed
* pref: Mock urllib system calls in test_aiohttp_transport.py to improve test speed
* chore: add --durations=50 to visualize slowest tests
* pref: reduce setup phase overhead by widening fixture scope in conftest.py
* test: stabilize flaky tests
* fix: minor issue
* fix(support-model-specific-tpm/rpm-limits): Allows setting rate limits by tpm/rpm for models by team
* fix(key_management_endpoints.py): enforce guaranteed throughput with key-level model tpm/rpm limits, when team-level tpm/rpm limits are set
* test: add unit testing
* feat(schema.prisma): add metadata to litellm budget table
* feat(proxy/utils.py): add org limits to user api key auth
allows org level tpm/rpm limiting to work
* feat: add org level tpm/rpm limits + inherit org id in key from team
enables org level tpm/rpm limits
* feat: validated working org tpm/rpm limits
* feat: support updating org level, model specific tpm/rpm limits
* fix: working key validation for org level tpm/rpm limits
* fix: working validation for orgs when giving tpm/rpm to teams
* fix(key_management_endpoints.py): fix tpm/rpm limits on orgs
* fix(key_management_endpoints.py): support limits
* refactor: remove duplicate var
* fix: refactor to avoid ruff errors
* fix: fix typign
* fix: fix linting error
* fix: fix testing
* fix(key_management_endpoints.py): document params
* fix(presidio.py): handle content as a list of texts
covers openai + anthropic messages api
* fix(presidio.py): safe get messages
* test: add unit testing for presidio guardrails
* fix(unified_guardrail.py): initial commit
* fix(enkryptai.py): implement apply_guardrail to enkrypt guardrail
* fix(unified_guardrail.py): support unified guardrail on input
* feat(unified_guardrail.py): add post call success hook implementation
allows us to just have 1 place to handle llm translation to guardrail api spec
* refactor: refactor initial unified guardrail component
* refactor: more refactoring
* feat(responses/): add guardrails to responses api
allows existing guardrails to work for new llm endpoints
* docs(adding_guardrail_support.md): document new guardrail endpoint support
* test: add unit tests
* feat(image_generation/): add guardrail support for image generation endpoint
* feat(openai/text_completion): support guardrails on `/v1/completions` API
* docs: document guardrails support on new endpoints
* docs: clarify when guardrails run
* feat(openai/speech): add guardrail support for input
* docs(rerank/): add guardrail support on input query
* fix: fix ruff check
* Addd v2/chat support for cohere
* fix streaming
* Use v2_transformation for logging passthrough:
* Use v2_transformation for logging passthrough:
* Add test for checking if document and citation_options is getting passed
* Update the cohere model
* Add cost tracking for vertex ai passthrough batch jobs
* Add full passthrough support
* refactor code according to the comments
* Add passthrough handler
* remove invalid params
* Updated documentation
* Updated documentation
* Updated documentation
* Correct the import
* Add openai videos generation and retrieval support
* add retrieval endpoint
* Add docs
* Add imports
* remove orjson
* remove double import
* fix openai videos format
* remove mock code
* remove not required comments
* Add tests
* Add tests
* Add other video endpoints
* Fix cost calculation and transformation
* Fixed mypy tests
* remove not used imports
* fix documentation for get batch req (#15742)
* Add grounding info to responses API (#15737)
* Add grounding info to responses API
* fix lint errors
* Use typed objects for annotations
* Use typed objects for annotations
* fix mypy error
* Litellm fix json serialize alreting 2 (#15741)
* fix json serializable error for alerts
* Add test
* fix mypt errors
* fix mypt errors
* Add Qwen3 imported model support for AWS Bedrock (#15783)
* Add qwen imported model support
* fix mypy errors
* fix empty user message error (#15784)
* fix typed dict for list
* Add azure supported videos endpoint
* fix mapped tests
* add azure sora models to model map
* Add OpenAI video generation and content retrieval support (#15745)
* Add openai videos generation and retrieval support
* add retrieval endpoint
* Add docs
* Add imports
* remove orjson
* remove double import
* fix openai videos format
* remove mock code
* remove not required comments
* Add tests
* Add tests
* Add other video endpoints
* Fix cost calculation and transformation
* Fixed mypy tests
* remove not used imports
* fix typed dict for list
* fix mypy errors
* move directory
* make v2 chat default
* Fix mypy tests
* Fix mypy tests
* Fix mypy tests
* Fix mypy tests
* Revert "Add Azure Video Generation Support with Sora Integration"
* refactor videos repo
* add test
* Add azure openai videos support
* Add azure openai videos support
* Add router endpoint support for videos
* fix mypy error
* add azure models
* fix mapped test
* fix mypy error
* Add proxy router test
* Add proxy router test
* remove deprecated model name from tests
* fix import error
* fix import error
* Add gaurdrail integration in videos endpoint
* Add logging support for videos endpoint
* Add final documentation supporting videos integration
* fix model name and document input
* Update literals to avoid mypy errors
* Remove unused imports and print statements
* revert guardrail support for video generation and video remix
* revert guardrail support for video generation and video remix
* Fix failing mapped and llm translation tests
* 1. add v3 classify
2. add new classifix for masking
3. support same id for the conversation for pre and post
working with duplicates
* clean code, remove some debug and run tests
* update liter errors
* improvment for Code Organization, httpx Error Handling Specificity, Logging Improvements and Type
* transfer test test_lasso_guard_config to the new location
* Fix type hints and linting errors in lasso.py
- Add type: ignore for httpx module when None
- Fix return type issues in _handle_classification and _handle_masking
- Ensure masked_messages is not None before passing to _apply_masking_to_model_response
- Convert LassoResponse to dict for _log_masking_applied call
* fix: Add response_type parameter to OAuth authorization endpoint
Fixes#15684
OAuth providers like Google require the response_type parameter during
the authorization flow. This commit adds response_type=code to the
authorization redirect parameters, which is required by the OAuth 2.0
specification (RFC 6749 Section 4.1.1).
Changes:
- Added response_type=code to authorization params in discoverable_endpoints.py
- Added test coverage for the response_type parameter
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix oauth flow by forwarding code_challenge and forwarding code_verifier
---------
Co-authored-by: Claude <noreply@anthropic.com>
* feat(guardrails): Add content masking and streaming support to PANW Prisma AIRS
- Add mask_request_content and mask_response_content parameters
- Implement content masking for prompts and responses
- Add streaming support with real-time masking
- Add comprehensive test coverage (28 tests)
- Update documentation with masking examples and security notes
* fix(guardrails): Fix PANW Prisma AIRS env var fallback and text completion support
* fix(support-model-specific-tpm/rpm-limits): Allows setting rate limits by tpm/rpm for models by team
* fix(key_management_endpoints.py): enforce guaranteed throughput with key-level model tpm/rpm limits, when team-level tpm/rpm limits are set
* test: add unit testing
* fix: fix minor linting errors
* fix: refactor
Ishaan Jaffer