Commit Graph
620 Commits
Author SHA1 Message Date
Ishaan JaffGitHubCopilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
18a5c4f75a [Feat] Add LiteLLM Gateway built in guardrail (#16338)
* add ContentFilterAction

* store pre-built regex patterns

* add v0 of content filter guard

* add _filter_messages

* test content filter guard

* init ContentFilterGuardrail

* fix ContentFilterGuardrail enums

* rename folder

* fix litellm_content_filter

* refactor content filter guard

* test content filter

* add streaming for ContentFilterGuardrail

* test_streaming_hook_mask

* add litellm_content_filter

* docs show litellm content filter

* docs litellm content filter

* fix lnting

* Potential fix for code scanning alert no. 3675: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-11-06 16:02:28 -08:00
Ishaan JaffGitHubCopilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
630a746c84 [Feat] Add Custom Secret Manager - Allow users to define and write a custom secret manager (#16297)
* add CustomSecretManager class

* docs custom secret manager

* add TestCustomSecretManager

* add KeyManagementSystem.CUSTOM

* add get_secret_from_manager

* add custom secret manager

* Potential fix for code scanning alert no. 3662: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* docs fix

* load_custom_secret_manager

* initialize_secret_manager

* add custom_secret_manager

* fix add custom secret manager

* add custom secret manager to KeyManagementSystem

* fix KeyManagementSystem.CUSTOM

* fix custom secret manager within cookbook

* fix link for custom secret manager

* Potential fix for code scanning alert no. 3663: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-11-05 17:20:26 -08:00
Krrish Dholakia fa278a1886 docs: move guardrails and responses api out of beta 2025-11-05 16:48:54 -08:00
Ishaan JaffandGitHub 2541a54023 [Feat] Email Notifications - Ensure Users get Key Rotated Email (#16292)
* add virtual_key_rotated

* add KEY_ROTATED_EMAIL_TEMPLATE

* fix base email

* add _send_key_rotated_email

* fix _send_key_rotated_email

* key rotated email fix

* fix: async_key_generated_hook ensure behavior matches regenerate

* test_send_key_rotated_email

* expose EMAIL_INCLUDE_API_KEY

* docs mask api keys

* add email regen

* docs key regen

* docs email

* docs email

* docs fix

* fix code qa check
2025-11-05 15:56:49 -08:00
Ishaan JaffGitHubCopilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
9a372bfad6 [Feat] Add CyberArk Secrets Manager Integration (#16278)
* KeyManagementSystem add cyberark

* add CyberArkSecretManager

* add CyberArkSecretManager

* add CyberArkSecretManager

* docs add CyberArkSecretManager

* docs

* refactor to use get_secret_from_manager

* Potential fix for code scanning alert no. 3645: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* Potential fix for code scanning alert no. 3650: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* Potential fix for code scanning alert no. 3649: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* Potential fix for code scanning alert no. 3646: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-11-05 14:00:45 -08:00
Ishaan JaffandGitHub 60f3a3b0ad [Feat] add serxng search API provider (#16259)
* TestFirecrawlSearch

* add SearchProviders

* add to get_provider_search_config

* add FirecrawlSearchConfig

* add FirecrawlSearchRequest

* add firecrawl API docs

* add pricing firecrawl/search

* add new search APIs

* add SearXNGSearchConfig

* add searxng/search

* add serxng params

* TestSearXNGSearch

* docs serxng

* docs fix

* docs fix

* docs serxng
2025-11-04 17:56:07 -08:00
Ishaan JaffandGitHub c1ac6e25aa [Feat] Add Bedrock Agentcore as a provider on LiteLLM Python SDK and LiteLLM AI Gateway (#16252)
* add agentcore in get_bedrock_route

* add AmazonAgentCoreConfig

* fix get_runtime_endpoint

* init AmazonAgentCoreConfig

* add get_bedrock_chat_config

* get_bedrock_chat_config

* add AmazonAgentCoreConfig

* fix get_complete_url

* refactor transform response

* test agentcore

* test_bedrock_agentcore_with_streaming

* fix _parse_json_response

* fix _calculate_usage

* test_bedrock_agentcore_basic

* add AgentCoreSSEStreamIterator

* add native streaming for agentcore

* test_bedrock_agentcore_with_streaming

* test_bedrock_agentcore_basic

* add agentcore

* _calculate_usage

* fix linting
2025-11-04 16:35:12 -08:00
Ishaan JaffandGitHub 57295cedef [Feat] Add Azure AI Doc Intelligence OCR (#16219)
* TestAzureDocumentIntelligenceOCR

* add AZURE_DOCUMENT_INTELLIGENCE_API_VERSION

* add AzureDocumentIntelligenceOCRConfig

* add async_transform_ocr_response

* use async transform

* add AzureDocumentIntelligenceOCRConfig

* add AzureDocumentIntelligenceOCRConfig

* add AzureDocumentIntelligenceOCRConfig

* add get_azure_ai_ocr_config

* add azure_ai/doc-intelligence

* add azure_ai/doc-intelligence

* docs fix

* docs fix

* add azure doc intel

* fix lint error
2025-11-03 17:22:19 -08:00
Ishaan JaffGitHubCopilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
71c61c274f [Feat] /ocr - Add VertexAI OCR provider support + cost tracking (#16216)
* add VertexAIOCRConfig

* __all__ = ["VertexAIOCRConfig"]
add

* add get_provider_ocr_config

* use GenericLiteLLMParams for litellm params

* fix _async_prepare_ocr_request

* fix _prepare_ocr_request

* fix get_complete_url

* fix validate_environment

* add safe_get_vertex_ai_project

* add VertexAIOCRConfig

* fix get_complete_url

* add TestVertexAIOCR

* add mistral-ocr-2505 cost

* add OCR to provider info

* docs vertex ai ocr

* fix _handle_rate_limits

* Potential fix for code scanning alert no. 3632: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-11-03 15:56:49 -08:00
0c743e1adc Add E2E Container API Support (#16136)
* Add v1 cut of container api

* fix lint errors

* Add proxy support to container apis & logging support (#16049)

* Add proxy support to container apis

* Add logging support

* Add cost tracking support for containers and documentation

* Add new constant documentation

* Add container cost in model map

* fix failing azure tests

* Update tests based on model map changes

* fix model map tests

* fix model map tests

* Container modeshould be container

* Container tests fix

* Merge branch 'main' into litellm_sameer_oct_staging_2

---------

Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com>
2025-11-01 14:03:51 -07:00
Krish DholakiaandGitHub 43aacf2dc0 (feat) Azure AI Vector Stores - support "virtual" indexes + create vector store on passthrough API (#16160)
* feat(vector_store_endpoints/endpoints.py): add new index_create endpoint

allows admin to create a virtual index, to do permission management for

* feat(key_management_endpoints.py): enable setting allowed_vector_store_indexes on keys

proxy admin can enable dev to create an index on a vector stor

* feat: initial commit adding vector store index passthrough logic to litellm

* feat: add vector store table

* fix(azure_ai/transformation.py): fix headers

* feat: track read/write endpoints by vector store integration

enables permissions by index to work

* fix: azure_ai/vector_stores/search

document the vector store endpoints correctly

 ensures permission management works as expected

* fix(proxy/utils.py): improve error message

* docs(azure_ai_vector_stores_passthrough.md): document azure ai passthrough vector store support

* docs(create.md): document azure ai support via passthrough for vector store create

* fix: fix code qa errors

* fix: document new allowed_vector_store_indexes endpoint
2025-11-01 12:01:32 -07:00
Krish DholakiaandGitHub b02be1ba70 (feat) Milvus - search vector store support + (fix) Passthrough Endpoints - support multi-part form data on passthrough (#16035)
* feat(milvus/): initial commit adding milvus vector store support to LiteLLM

allows querying milvus vector store through litellm

* feat(bedrock/vector_stores): support translating openai filters param to aws kb

adds filtering to aws kb

* feat(milvus/): add milvus vector store unified search support

allows calling milvus vector store in through chat completions

* docs(milvus_vector_stores.md): document new milvus vector search integration

* feat(pass_through_endpoints.py): support passing form data through to a passthrough endpoint

Closes LIT-1147

* fix: fix linting errors
2025-11-01 12:00:29 -07:00
Ishaan JaffandGitHub 99feefd614 [Feat] Add FAL AI Image Generations on LiteLLM (#16067)
* add fal-ai provider

* fix image_generation_handler

* init FalAIImageGenerationConfig

* init cost_calculator

* init FAL AI

* TestFAL_AI_ImageGeneration

* fix load_custom_provider_entrypoints

* TestFAL_AI_ImageGeneration

* add imagen4 transform FAL AI

* add FAL AI imagen 4 transform

* BaseImageGenTest

* test_fal_ai_image_generation_basic

* add BRIA + Recraft img gen

* add recraft + BRIA

* test_fal_ai_image_generation_basic

* tests for flux PRO v11

* Add FAL AI SD

* test FAL AI SD

* docs FAL AI

* docs fal ai

* Using Model-Specific Parameters

* add fal ai model prices

* add fall_ai JPG logo

* ui fixes FAL AI

* fix linting

* fix linting

* fix bedrock test_get_request_body_stability3

* test_custom_llm_provider_entrypoint
2025-10-29 13:10:51 -07:00
Sameer KankuteandGitHub 59189c0579 fix errors in videos documentation (#15996) 2025-10-28 14:48:04 -07:00
Krrish Dholakia e1f54ef02c docs: refactor placement of adding guardrails to endpoints doc 2025-10-27 10:26:00 -07:00
Sameer KankuteandGitHub 70650a044c Add all sora models (#15937) 2025-10-26 10:10:57 -07:00
Krish DholakiaandGitHub 346e036399 fix(opentelemetry.py): fix issue where headers were not being split correctly + feat(bedrock): add titan image generations w/ cost tracking (#15916)
* fix(opentelemetry.py): fix issue where headers were not being split correctly

* feat(bedrock/image): Support bedrock titan image generation

Closes https://github.com/BerriAI/litellm/issues/361

* build(model_prices_and_context_window.json): track titan image gen pricing

enables cost tracking per request

* feat(amazon_titan_transformation.py): support titan image generation cost tracking

* docs: document new model

* docs: update docs to indicate cost tracking + refactor rerank into separate doc

* fix: fix mypy linting error

* fix: fix type ignore
2025-10-25 13:45:13 -07:00
Krish DholakiaandGitHub 2bd41dc034 Guardrails - Responses API, Image Gen, Text completions, Audio transcriptions, Audio Speech, Rerank, Anthropic Messages API support via the unified apply_guardrails function (#15706)
* fix(presidio.py): handle content as a list of texts

covers openai + anthropic messages api

* fix(presidio.py): safe get messages

* test: add unit testing for presidio guardrails

* fix(unified_guardrail.py): initial commit

* fix(enkryptai.py): implement apply_guardrail to enkrypt guardrail

* fix(unified_guardrail.py): support unified guardrail on input

* feat(unified_guardrail.py): add post call success hook implementation

allows us to just have 1 place to handle llm translation to guardrail api spec

* refactor: refactor initial unified guardrail component

* refactor: more refactoring

* feat(responses/): add guardrails to responses api

allows existing guardrails to work for new llm endpoints

* docs(adding_guardrail_support.md): document new guardrail endpoint support

* test: add unit tests

* feat(image_generation/): add guardrail support for image generation endpoint

* feat(openai/text_completion): support guardrails on `/v1/completions` API

* docs: document guardrails support on new endpoints

* docs: clarify when guardrails run

* feat(openai/speech): add guardrail support for input

* docs(rerank/): add guardrail support on input query

* fix: fix ruff check
2025-10-25 13:38:57 -07:00
Krish DholakiaandGitHub 86524fcaf5 VertexAI Search Vector Store - Passthrough endpoint support + Vector store search Cost tracking support (#15824)
* feat(vector_stores/): initial commit adding Vertex AI Search API support for litellm

new vector store provider

* feat(vector_store/): use vector store id for vertex ai search api

* fix: transformation.py

cleanup

* fix: implement abstract function

* fix: fix linting error

* fix: main.py

fix check

* feat: initial commit with working passthrough support for vertex ai search api through litellm

* feat(llm_passthrough_endpoints.py): fix passing correct project on datastore passthrough

* feat(vertex_ai/): support passthrough call for vertex ai search vector store

* docs(vertex_ai_search_datastore.md): document new vertex ai passthrough endpoint

* docs(sidebars.js): document new endpoint

* feat: initial commit adding logging for vertex ai passthrough api

 allows vertex ai vector search api to work with cost calculation

* feat(vertex_ai/): search vector store cost tracking

* fix(vertex_passthrough_logging_handler.py): log the cost

* fix: improve logged response

* fix(vertex_passthrough_logging_handler.py): logging

* feat(litellm_logging): main.py

add cost tracking for vertex ai search api via unified api

* refactor: fix ruff checks

* fix(llm_passthrough_endpoints.py): fix linting
2025-10-25 13:17:15 -07:00
Krish DholakiaandGitHub f8d6a6edb9 fix(managed_files.py): don't raise error if managed object is not found + (Feat) Azure AI - Search Vector Stores + (Fix) Batches - “User default_user_id does not have access to the object” when object not in db + (fix) Vector Stores - show config.yaml vector stores on UI (#15873)
* fix(managed_files.py): don't raise error if managed object is not found

* feat(vector_stores): add azure ai search vector store support

Enables direct querying a vector store on azure

* fix(azure/vector_stores): working azure ai search api vector stores

allows azure direct querying on vector stores

* test: update env vars

* docs(docs/): document new azure ai vector store search

* docs(azure_ai_vector_stores.md): add table

* docs: clarify support for 'create' vector stores

* fix(vector_stores/endpoints.py): Fixes https://github.com/BerriAI/litellm/issues/14606

* fix: fix linting errors
2025-10-25 12:06:24 -07:00
Krish DholakiaandGitHub ddacaf6c32 (feat) Organizations: allow org admins to create teams on UI + (feat) IBM Guardrails (#15924)
* fix(oldteams.tsx): allow org admin to create team on ui

* fix(oldteams.tsx): show org admin a dropdown of allowed orgs for team creation

* docs(access_control.md): cleanup doc

* feat(ibm_guardrails/): initial commit adding support for ibm guardrails on litellm

allows user to use self-hosted ibm guardrails

* feat(ibm_detector.py): working detector

* docs(ibm_guardrails.md): document new ibm guardrails

* fix: fix linting errors
2025-10-25 11:13:39 -07:00
Ishaan JaffandGitHub e4d5f00990 [Feat] New Guardrail - Dynamo AI Guardrail (#15920)
* add dynamo types

* fix Dynamo guard

* add dynamo guardrail

* add dynamo ai docs guard

* docs fix

* test dynamo

* test LASSO
2025-10-24 17:11:04 -07:00
Ishaan JaffandGitHub 3e4b5ef3a5 [Feat] Add cost tracking for Search API requests - Google PSE, Tavily, Parallel AI, Exa AI (#15821)
* add search cost tracking

* add cost tracking for tavily tiers

* add search to call types

* add search_provider_cost_per_query

* add cost tracking for search APIs

* add cost tracking search APIs

* docs cost tracking search

* docs search

* fix linting
2025-10-22 17:29:09 -07:00
Ishaan JaffandGitHub f5a80110c1 [Feat] Add /search endpoint on LiteLLM Gateway (#15780)
* add SearchProvider

* add SearchToolTypedDict

* add search

* add SearchAPIRouter

* working router level search

* add search to allowed llm / ocr routes

* feat: add search_router

* add routing + proxy for search APIs

* /v1/search/{search_tool_name}

* fix search routing

* feat: parse_search_tools

* clean up sidebar

* docs fix

* router tests for search tools

* docs fix
2025-10-21 19:05:20 -07:00
Krrish Dholakia 2a1dbb5b9e docs(creating_adapters.md): document how to write an adapter 2025-10-21 16:20:31 -07:00
Krrish Dholakia 1e0368521e refactor: cleanup 2025-10-21 13:46:19 -07:00
Ishaan JaffandGitHub 8ad9bbbd02 [Docs] Add Azure AI - OCR to docs (#15768)
* add Azure OCR to docs

* docs fix

* docs fix

* docs fix

* docs OCR
2025-10-21 13:10:45 -07:00
Ishaan Jaffer 185182bebc Revert "add Azure OCR to docs"
This reverts commit a3699e28a4.
2025-10-21 13:02:31 -07:00
Ishaan Jaffer a3699e28a4 add Azure OCR to docs 2025-10-21 13:02:21 -07:00
Ishaan Jaffer 6605aba307 docs grayswan 2025-10-21 11:16:25 -07:00
Ishaan JaffandGitHub 73a23a6c78 [Feat] Add Azure AVA TTS integration (#15749)
* add AzureBaseIssueTokenHandler

* add BaseTextToSpeechConfig

* async_text_to_speech_handler

* add AzureAVATextToSpeechConfig

* add get_provider_text_to_speech_config

* add AzureAVATextToSpeechConfig

* fixes for base_llm_http_handler

* fix transform_text_to_speech_request

* test_azure_ava_tts_async

* test_azure_ava_tts_async

* fix TextToSpeechRequestData

* fix transform_text_to_speech_request

* add text_to_speech_handler in LLMHttpHandler

* remove old file

* fix transform_text_to_speech_request

* fix dispatch_text_to_speech

* fix azure TTS

* fix AVA TTS

* fix transform

* fix linting

* ci/cd - use one job for audio testing

* fix tests

* fix llm http handler debugging

* unit tests azure tts

* docs Azure speech

* docs fix

* docs azure AVA

* docs azure AVA

* fix handlers

* test_async_realtime_uses_max_size_parameter
2025-10-20 16:52:23 -07:00
Krrish Dholakia bbb7b75672 docs: sidebars.js
add vertex ai live api to docs
2025-10-17 23:55:44 -07:00
Ishaan Jaffer ea69f4547d Merge branch 'main' into litellm_oct_staging2 2025-10-16 17:06:29 -07:00
Ishaan Jaffer 79516563a9 docs add /invoke and /converse routes 2025-10-16 16:25:27 -07:00
TensorNull 55a6dd3a8b feat(cometapi): Add CometAPI provider support (embeddings, image generation, docs)
- Add CometAPI embedding and image generation transformations and configs
- Add image cost calculator and export/init files
- Register provider in constants, utils, main (embedding path) and sidebars
- Add CometAPI docs page and cookbook notebook (Colab) for usage examples
2025-10-16 13:08:14 +08:00
Ishaan JaffGitHubCopilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
bc26845ec4 [Feat] Native /ocr endpoint support (#15573)
* [Feat] Add native litellm.ocr() functions (#15567)

* fix get_supported_ocr_params

* add get_provider_ocr_config

* init OCR

* init ocr functions

* add OCRResponse Base Model

* add ocr to llm http handlers

* add main.py for OCR

* fix linting for OCR

* TestMistralOCR

* update to use DocumentType for Mistral

* fix _prepare_ocr_request

* fix transform

* add main.py for OCR

* add spec to init

* fix OCR

* TestMistralOCR

* ruff fix

* Potential fix for code scanning alert no. 3521: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* [Feat] Add /ocr route on LiteLLM AI Gateway - Adds support for native mistral ocr calling (#15571)

* fix get_supported_ocr_params

* add get_provider_ocr_config

* init OCR

* init ocr functions

* add OCRResponse Base Model

* add ocr to llm http handlers

* add main.py for OCR

* fix linting for OCR

* TestMistralOCR

* update to use DocumentType for Mistral

* fix _prepare_ocr_request

* fix transform

* add main.py for OCR

* add spec to init

* fix OCR

* TestMistralOCR

* ruff fix

* add router.ocr() methods

* add OCR routes

* feat add ocr routes

* add OCR routes

* feat: add OCR routes in proxy server

* working /ocr routes

* test_router_aocr_with_mistral

* docs Mistral OCR

* docs OCR

* [Feat] Add Azure AI Mistral OCR Integration  (#15572)

* fix get_supported_ocr_params

* add get_provider_ocr_config

* init OCR

* init ocr functions

* add OCRResponse Base Model

* add ocr to llm http handlers

* add main.py for OCR

* fix linting for OCR

* TestMistralOCR

* update to use DocumentType for Mistral

* fix _prepare_ocr_request

* fix transform

* add main.py for OCR

* add spec to init

* fix OCR

* TestMistralOCR

* ruff fix

* add router.ocr() methods

* add OCR routes

* feat add ocr routes

* add OCR routes

* feat: add OCR routes in proxy server

* working /ocr routes

* test_router_aocr_with_mistral

* docs Mistral OCR

* docs OCR

* add azure ai to get_provider_ocr_config

* add AzureAIOCRConfig

* TestAzureAIOCR

* TestAzureAIOCR

* test fixes for azure ai ocr

* fix async OCR transform for Azure

* fix transform_ocr_request

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-10-15 17:20:01 -07:00
Ishaan JaffandGitHub a6c57cb5bd [Feat] Cost Tracking - specify a global vendor discount for costs. (#15546)
* fix cost_discount_config

* add CostBreakdown

* fix: set_cost_breakdown

* test_cost_discount_vertex_ai

* docs fix

* docs fix discounts

* docs fix

* docs custom pricing

* docs fix

* fixes for getting cost breakdown in response headers

* test - response headers wth discount
2025-10-14 20:07:04 -07:00
Ishaan Jaffer 7288f4355f docs fix 2025-10-11 16:24:21 -07:00
Ishaan Jaffer 54b8bde60c docs fix 2025-10-11 16:23:46 -07:00
Ishaan JaffandGitHub 527c8f59fa [Feat] Tag Management - Add support for setting tag based budgets (#15433)
* feat: add LiteLLM_TagTable

* fix: use new table for tag management

* fix - allow setting budgets for tags

* working tag creation

* fix schema.prisma

* add tag info

* ui fixes

* ui fix tag info

* TAG_CACHE_IN_MEMORY_TTL_SECONDS

* add Litellm_EntityType

* fix get_aggregated_db_spend_update_transactions

* fix: _update_entity_spend_in_db

* fix _tag_max_budget_check

* add tag budget check

* add tag_list_transactions

* test_get_tag_objects_batch

* test_update_tag_db_without_prisma_client

* fix get_tags_from_request_body

* get_tags_from_request_body

* fix get_tags_from_request_body

* fix spend tracking utils

* get_tags_from_request_body

* test_get_tags_from_request_body_with_metadata_tags

* feat: add _update_tag_cache spend tracking

* fix _PROXY_track_cost_callback

* test_tag_cache_update_multiple_tags

* fix tag info

* docs fix

* docs tag budgets

* doc fix

* docs fix

* fix tag budget

* docs tag budgets

* docs fix

* ruff fix
2025-10-10 19:24:50 -07:00
Ishaan JaffandGitHub bf209415da [Feat] VertexAI Gemma model family streaming support + Added MedGemma (#15427)
* test_acompletion_filters_stream_and_stream_options

* fix: stream_options

* docs medgemma

* lint fix

* docs
2025-10-10 14:22:27 -07:00
Krish DholakiaandGitHub d944717d4b Merge pull request #15278 from xprilion/wandb-inference-docs
Add W&B Inference documentation
2025-10-09 22:35:16 -07:00
Ishaan JaffandGitHub 6cf57d85c3 [Feat] Add EnkryptAI Guardrails on LiteLLM (#15390)
* feat: add EnkryptAIGuardrail

* fix types

* fix EnkryptAIGuardrails

* __all__ = ["EnkryptAIGuardrail"]
fix

* fix EnkryptAIGuardrail

* TestEnkryptAIExtraction

* fix mypy linting

* docs enkrypt ai

* add EnkryptAIGuardrailConfigModel

* fix EnkryptAIGuardrailConfigModel

* fix import struct

* fix EnkryptAIGuardrails
2025-10-09 16:27:04 -07:00
Anubhav SinghandGitHub f59f1971cc Merge branch 'BerriAI:main' into wandb-inference-docs 2025-10-09 22:26:37 +05:30
Krrish Dholakia f88e3ebdfa docs: refactor to have function call / computer use / etc. on top level 2025-10-09 07:25:25 -07:00
Ishaan JaffandGitHub 36c971a6fd [MCP Gateway] QA/Fixes - Ensure Team/Key level enforcement works for MCPs (#15305)
* fix: _set_object_permission

* fix: _set_object_permission on teams

* fix: _set_object_permission

* fixes for team/key permissions

* statsh: object permission view

* fix: MCPServerPermissions

* fix: _get_team_object_permission

* test mcp checks for permissions

* fix server checks with prefix names

* test_list_tools_strips_prefix_when_matching_permissions

* ruff fix

* docs - refactor MCP

* docs update MCP docs

* docs allowed tools
2025-10-07 17:34:48 -07:00
xprilion 888391127f Add W&B Inference documentation 2025-10-07 19:18:45 +05:30
Krrish Dholakia fbf7ca17ba docs(security_encryption_faq.md): answers common questions about data at rest vs. in transit 2025-10-06 17:34:10 -07:00
Ishaan JaffandGitHub efa782d6d2 [Feat] Add Nvidia NIM Rerank Support (#15152)
* feat: add NvidiaNimRerankConfig

* fix: NvidiaNimRerankConfig

* fix: NvidiaNimRerankConfig

* fix routing to nvidia nim

* docs nvidia nim rerank

* TestNvidiaNim

* nvidia nim rerank fixes

* fix rerank

* transform_rerank_response

* Usage with LiteLLM Proxy

* fixes linting

* NvidiaNimRerankConfig.DEFAULT_NIM_RERANK_API_BASE

* fix Custom API Base URL

* fix rerank base

* fix main.py

* fix transform

* fix linting

* map_cohere_rerank_params

* ruff fix

* linting fixes

* ruff fix
2025-10-02 18:58:52 -07:00
Amir RefaeeandGitHub 8991657d67 added railtracks to projects using litellm (#15144) 2025-10-02 14:21:38 -07:00