Commit Graph

4632 Commits

Author SHA1 Message Date
Cesar Garcia d65a29b88d docs: fix image generation response format from 'image' to 'images' (#16378)
Update documentation to reflect actual API response format:
- Change singular 'image' field to plural 'images' array
- Add complete ImageURLListItem structure with index and type fields
- Update all code examples to use message.images instead of message.image
- Fix streaming examples to access images[0]["image_url"]["url"]

The documentation was incorrectly showing 'image' (singular object)
but the actual implementation returns 'images' (array of ImageURLListItem).

Related to issue #16227
2025-11-07 19:06:03 -08:00
Krrish Dholakia 532ebf43d0 docs(moderation.md): fix moderation quick start docs 2025-11-07 16:25:08 -08:00
Sameer Kankute faae0ff0dc Fix Azure DALL-E-3 health check content policy violation by using safe default prompt (#16329)
* Add custom health check prompt support

* Add constant for health check prompt

* Add constant for health check prompt
2025-11-07 15:30:56 -08:00
Krrish Dholakia 9059905d25 docs(openai/videos.md): document proxy usage on openai docs for video gen 2025-11-07 15:27:18 -08:00
Ishaan Jaff 2bd85dc455 [Feat] Add DD Agent Host support for datadog callback (#16379)
* add DD_AGENT_HOST

* docs DD Agent

* test_datadog_agent_configuration

* DD_AGENT_HOST
2025-11-07 15:18:23 -08:00
Ishaan Jaff a6b0993405 [Feat] Secret Manager - Hashicorp, add auth via approle (#16374)
* add _verify_required_credentials_exist and _auth_via_approle

* test_hashicorp_secret_manager_approle_auth

* docs hcorp auth
2025-11-07 14:39:33 -08:00
Ishaan Jaffer ea4048324b docs fix 2025-11-07 14:39:24 -08:00
Jason Roberts 5733f711fd feat(guardrails): panw prisma airs guardrail deduplication and enhanced session tracking (#16273)
* feat(guardrails): Add deduplication and session tracking

- Implement deduplication logic to prevent duplicate scans (via call_id; add _check_and_mark_scanned) caused by LiteLLM callback system
- Add session tracking using litellm_trace_id as AI Session ID for Prisma AIRS SCM logging
- Extract helper methods _extract_prompt_from_request maintainability
- Use httpxSpecialProvider import (LoggingCallback -> GuardrailCallback)
- Add comprehensive tests for deduplication and session tracking (7 new tests)
- Update documentation with multi-turn conversation tracking examples

* docs: update PANW Prisma AIRS multi-turn conversation example to use industry-standard terminology

- Clearer example for conversation tracking
- Updated terminology from 'AI Session ID' to 'Prisma AIRS AI Session ID' for clarity

* fix: remove unused asyncio import

* fix: correct mypy type ignore comment
2025-11-06 19:34:37 -08:00
Sameer Kankute 83998d3573 Update the fireworks url in tests and doc (#16346) 2025-11-06 19:22:21 -08:00
Ishaan Jaffer a05159dc6e docs fix 2025-11-06 17:21:23 -08:00
Ishaan Jaffer 830a9e9585 docs fix 2025-11-06 16:22:17 -08:00
Ishaan Jaff 18a5c4f75a [Feat] Add LiteLLM Gateway built in guardrail (#16338)
* add ContentFilterAction

* store pre-built regex patterns

* add v0 of content filter guard

* add _filter_messages

* test content filter guard

* init ContentFilterGuardrail

* fix ContentFilterGuardrail enums

* rename folder

* fix litellm_content_filter

* refactor content filter guard

* test content filter

* add streaming for ContentFilterGuardrail

* test_streaming_hook_mask

* add litellm_content_filter

* docs show litellm content filter

* docs litellm content filter

* fix lnting

* Potential fix for code scanning alert no. 3675: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-11-06 16:02:28 -08:00
Sameer Kankute 9dc22eed5c Update container documentation to be similar to others (#16327) 2025-11-06 10:08:45 -08:00
Ishaan Jaff 630a746c84 [Feat] Add Custom Secret Manager - Allow users to define and write a custom secret manager (#16297)
* add CustomSecretManager class

* docs custom secret manager

* add TestCustomSecretManager

* add KeyManagementSystem.CUSTOM

* add get_secret_from_manager

* add custom secret manager

* Potential fix for code scanning alert no. 3662: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* docs fix

* load_custom_secret_manager

* initialize_secret_manager

* add custom_secret_manager

* fix add custom secret manager

* add custom secret manager to KeyManagementSystem

* fix KeyManagementSystem.CUSTOM

* fix custom secret manager within cookbook

* fix link for custom secret manager

* Potential fix for code scanning alert no. 3663: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-11-05 17:20:26 -08:00
Krrish Dholakia fa278a1886 docs: move guardrails and responses api out of beta 2025-11-05 16:48:54 -08:00
Ishaan Jaff 2541a54023 [Feat] Email Notifications - Ensure Users get Key Rotated Email (#16292)
* add virtual_key_rotated

* add KEY_ROTATED_EMAIL_TEMPLATE

* fix base email

* add _send_key_rotated_email

* fix _send_key_rotated_email

* key rotated email fix

* fix: async_key_generated_hook ensure behavior matches regenerate

* test_send_key_rotated_email

* expose EMAIL_INCLUDE_API_KEY

* docs mask api keys

* add email regen

* docs key regen

* docs email

* docs email

* docs fix

* fix code qa check
2025-11-05 15:56:49 -08:00
Alexsander Hamir e06d1d623f Add: benchmark comparison with other AI gateways (#16248)
* fix: benchmark comparison location & content

* add: pros and cons

* remove
2025-11-05 15:44:30 -08:00
Deepanshu Lulla 3ef210e5f4 Add allowing Key based prefix to s3 path (#16237)
* add Key based prefix

* add Key based prefix

* update documentation

* correct doc

---------

Co-authored-by: deepanshu <deepanshu.lulla@hq.bill.com>
2025-11-05 14:43:21 -08:00
Ishaan Jaff 466e7d178c [Feat] Cyber Ark - Add Key Rotations support (#16289)
* KeyManagementSystem add cyberark

* add CyberArkSecretManager

* add CyberArkSecretManager

* add CyberArkSecretManager

* docs add CyberArkSecretManager

* docs

* refactor to use get_secret_from_manager

* fix async roate for cyber ark, re-use base class

* fixes

* cyber ark

* docs fix

* docs fix

* docs cyberark

* fix linting

* fix get_secret_from_manager
2025-11-05 14:03:43 -08:00
Ishaan Jaff 9a372bfad6 [Feat] Add CyberArk Secrets Manager Integration (#16278)
* KeyManagementSystem add cyberark

* add CyberArkSecretManager

* add CyberArkSecretManager

* add CyberArkSecretManager

* docs add CyberArkSecretManager

* docs

* refactor to use get_secret_from_manager

* Potential fix for code scanning alert no. 3645: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* Potential fix for code scanning alert no. 3650: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* Potential fix for code scanning alert no. 3649: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* Potential fix for code scanning alert no. 3646: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-11-05 14:00:45 -08:00
Krrish Dholakia 9b925c7e47 docs: remove old doc
causing search issues
2025-11-05 08:23:35 -08:00
Ishaan Jaff 60f3a3b0ad [Feat] add serxng search API provider (#16259)
* TestFirecrawlSearch

* add SearchProviders

* add to get_provider_search_config

* add FirecrawlSearchConfig

* add FirecrawlSearchRequest

* add firecrawl API docs

* add pricing firecrawl/search

* add new search APIs

* add SearXNGSearchConfig

* add searxng/search

* add serxng params

* TestSearXNGSearch

* docs serxng

* docs fix

* docs fix

* docs serxng
2025-11-04 17:56:07 -08:00
Ishaan Jaff af78a93ecf [Feat] /search API - add firecrawl search API support (#16257)
* TestFirecrawlSearch

* add SearchProviders

* add to get_provider_search_config

* add FirecrawlSearchConfig

* add FirecrawlSearchRequest

* add firecrawl API docs

* add pricing firecrawl/search

* add new search APIs
2025-11-04 17:52:12 -08:00
Krrish Dholakia a3810a5f3a docs(openai_passthrough.md): document how to make openai passthrough route work 2025-11-04 17:12:33 -08:00
Ishaan Jaff c1ac6e25aa [Feat] Add Bedrock Agentcore as a provider on LiteLLM Python SDK and LiteLLM AI Gateway (#16252)
* add agentcore in get_bedrock_route

* add AmazonAgentCoreConfig

* fix get_runtime_endpoint

* init AmazonAgentCoreConfig

* add get_bedrock_chat_config

* get_bedrock_chat_config

* add AmazonAgentCoreConfig

* fix get_complete_url

* refactor transform response

* test agentcore

* test_bedrock_agentcore_with_streaming

* fix _parse_json_response

* fix _calculate_usage

* test_bedrock_agentcore_basic

* add AgentCoreSSEStreamIterator

* add native streaming for agentcore

* test_bedrock_agentcore_with_streaming

* test_bedrock_agentcore_basic

* add agentcore

* _calculate_usage

* fix linting
2025-11-04 16:35:12 -08:00
Deepanshu Lulla 812ea03d28 Add tags and descriptions support to aws secrets manager (#16224)
* Add tags and descriptions support to aws secrets manager

* add tags

---------

Co-authored-by: deepanshu <deepanshu.lulla@hq.bill.com>
2025-11-04 16:11:51 -08:00
Krish Dholakia 726452dfe8 Revert "add: comparison with portkey (#16145)" (#16247)
This reverts commit 844ace1283.
2025-11-04 10:37:56 -08:00
Alexsander Hamir 844ace1283 add: comparison with portkey (#16145) 2025-11-04 10:37:17 -08:00
YutaSaito 8e27b6c0b4 [MCP] configure static mcp header (#16179)
* feat: configure extra mcp headers in ui

* doc: static header

* build: add new migration file

* chore: add missing image file

* fix: test
2025-11-03 21:06:36 -08:00
Ishaan Jaff 57295cedef [Feat] Add Azure AI Doc Intelligence OCR (#16219)
* TestAzureDocumentIntelligenceOCR

* add AZURE_DOCUMENT_INTELLIGENCE_API_VERSION

* add AzureDocumentIntelligenceOCRConfig

* add async_transform_ocr_response

* use async transform

* add AzureDocumentIntelligenceOCRConfig

* add AzureDocumentIntelligenceOCRConfig

* add AzureDocumentIntelligenceOCRConfig

* add get_azure_ai_ocr_config

* add azure_ai/doc-intelligence

* add azure_ai/doc-intelligence

* docs fix

* docs fix

* add azure doc intel

* fix lint error
2025-11-03 17:22:19 -08:00
Alexsander Hamir a73e890d8f fix: broken link on model_management.md (#16217) 2025-11-03 17:00:03 -08:00
Ishaan Jaff 71c61c274f [Feat] /ocr - Add VertexAI OCR provider support + cost tracking (#16216)
* add VertexAIOCRConfig

* __all__ = ["VertexAIOCRConfig"]
add

* add get_provider_ocr_config

* use GenericLiteLLMParams for litellm params

* fix _async_prepare_ocr_request

* fix _prepare_ocr_request

* fix get_complete_url

* fix validate_environment

* add safe_get_vertex_ai_project

* add VertexAIOCRConfig

* fix get_complete_url

* add TestVertexAIOCR

* add mistral-ocr-2505 cost

* add OCR to provider info

* docs vertex ai ocr

* fix _handle_rate_limits

* Potential fix for code scanning alert no. 3632: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-11-03 15:56:49 -08:00
Krish Dholakia 07d2a27f14 Milvus - Passthrough API support - adds create + read vector store support via passthrough API's (#16170)
* feat(llm_passthrough_endpoints.py): support milvus passthrough api

* fix(llm_passthrough_endpoints.py): move streaming request value to the top of the function

* docs: document new milvus vector store passthrough flow
2025-11-02 09:47:58 -08:00
YutaSaito 6ed76ff809 feat: change guardrail_information to list type (#16127)
* feat: change guardrail_information to list type to support displaying multiple guardrails

* fix: add missing commit and revert auto-format changes in utils.py

---------

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
2025-11-02 09:47:49 -08:00
Deepanshu Lulla 20b95e9a80 strip base64 in s3 (#16157)
* strip base64

* strip base64

* s3 use key prefix

* s3 use key prefix

* strip base64 doc

---------

Co-authored-by: deepanshu <deepanshu.lulla@hq.bill.com>
2025-11-02 09:06:53 -08:00
yuneng-jiang 48822c2fa2 [Feature] UI - Add Model Existing Credentials Improvement (#16166)
* Model Settings UI Improvement

* Doc changes from Anthropic SDK to Passthrough

* Tests
2025-11-02 09:04:36 -08:00
Ishaan Jaffer b111bcd910 docs fix 2025-11-01 16:56:44 -07:00
Ishaan Jaff 61b6ee0cf8 docs - v1.79.1 (#16163)
* v1

* fixes

* docs update

* doc fix

* docs fix

* docs fix

* Dependency Upgrades

* docs fix

* docs fix

* docs fix
2025-11-01 16:53:46 -07:00
Sameer Kankute 0c743e1adc Add E2E Container API Support (#16136)
* Add v1 cut of container api

* fix lint errors

* Add proxy support to container apis & logging support (#16049)

* Add proxy support to container apis

* Add logging support

* Add cost tracking support for containers and documentation

* Add new constant documentation

* Add container cost in model map

* fix failing azure tests

* Update tests based on model map changes

* fix model map tests

* fix model map tests

* Container modeshould be container

* Container tests fix

* Merge branch 'main' into litellm_sameer_oct_staging_2

---------

Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com>
2025-11-01 14:03:51 -07:00
Ishaan Jaffer 571f94777c docs fix 2025-11-01 13:51:21 -07:00
Krrish Dholakia bb0caee896 docs: document vector store usage with chat completions api 2025-11-01 13:47:24 -07:00
Alexsander Hamir 542378e604 add: minimum resource requirement (#16146) 2025-11-01 13:17:37 -07:00
Krrish Dholakia ef7d81865b build: update dependencies 2025-11-01 12:58:39 -07:00
Sameer Kankute 058b6a66eb Add custom_llm_provider support for video endpoints (non-generation) (#16121)
* fix model error for apis which don't need model

* fix print statments:

* fix mypy lint errors

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
2025-11-01 12:09:11 -07:00
Krish Dholakia 43aacf2dc0 (feat) Azure AI Vector Stores - support "virtual" indexes + create vector store on passthrough API (#16160)
* feat(vector_store_endpoints/endpoints.py): add new index_create endpoint

allows admin to create a virtual index, to do permission management for

* feat(key_management_endpoints.py): enable setting allowed_vector_store_indexes on keys

proxy admin can enable dev to create an index on a vector stor

* feat: initial commit adding vector store index passthrough logic to litellm

* feat: add vector store table

* fix(azure_ai/transformation.py): fix headers

* feat: track read/write endpoints by vector store integration

enables permissions by index to work

* fix: azure_ai/vector_stores/search

document the vector store endpoints correctly

 ensures permission management works as expected

* fix(proxy/utils.py): improve error message

* docs(azure_ai_vector_stores_passthrough.md): document azure ai passthrough vector store support

* docs(create.md): document azure ai support via passthrough for vector store create

* fix: fix code qa errors

* fix: document new allowed_vector_store_indexes endpoint
2025-11-01 12:01:32 -07:00
Krish Dholakia b02be1ba70 (feat) Milvus - search vector store support + (fix) Passthrough Endpoints - support multi-part form data on passthrough (#16035)
* feat(milvus/): initial commit adding milvus vector store support to LiteLLM

allows querying milvus vector store through litellm

* feat(bedrock/vector_stores): support translating openai filters param to aws kb

adds filtering to aws kb

* feat(milvus/): add milvus vector store unified search support

allows calling milvus vector store in through chat completions

* docs(milvus_vector_stores.md): document new milvus vector search integration

* feat(pass_through_endpoints.py): support passing form data through to a passthrough endpoint

Closes LIT-1147

* fix: fix linting errors
2025-11-01 12:00:29 -07:00
Ishaan Jaffer 22d35e2552 docs fix 2025-11-01 11:13:25 -07:00
Boxuan Li 297c2a0bf4 prompt_caching.md: Fix wrong prompt_tokens definition (#16044) 2025-10-30 20:13:13 -07:00
John Lathouwers a6f740f28b Add OCI Signer Authentication. Closes #16048, Closes #15654 (#16064)
* Add OCI Signer Authentication. Closes #16048,  Closes #15654

* Fix linting error

* Remove Recommended, Catch None, Trim Whitespace

* Make method clear re Proxy vs SDK
2025-10-30 19:59:01 -07:00
Krrish Dholakia 30f343c3cb docs(deploy.md): document new terraform module 2025-10-30 17:11:55 -07:00