Commit Graph
585 Commits
Author SHA1 Message Date
TensorNull 55a6dd3a8b feat(cometapi): Add CometAPI provider support (embeddings, image generation, docs)
- Add CometAPI embedding and image generation transformations and configs
- Add image cost calculator and export/init files
- Register provider in constants, utils, main (embedding path) and sidebars
- Add CometAPI docs page and cookbook notebook (Colab) for usage examples
2025-10-16 13:08:14 +08:00
Ishaan JaffGitHubCopilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
bc26845ec4 [Feat] Native /ocr endpoint support (#15573)
* [Feat] Add native litellm.ocr() functions (#15567)

* fix get_supported_ocr_params

* add get_provider_ocr_config

* init OCR

* init ocr functions

* add OCRResponse Base Model

* add ocr to llm http handlers

* add main.py for OCR

* fix linting for OCR

* TestMistralOCR

* update to use DocumentType for Mistral

* fix _prepare_ocr_request

* fix transform

* add main.py for OCR

* add spec to init

* fix OCR

* TestMistralOCR

* ruff fix

* Potential fix for code scanning alert no. 3521: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* [Feat] Add /ocr route on LiteLLM AI Gateway - Adds support for native mistral ocr calling (#15571)

* fix get_supported_ocr_params

* add get_provider_ocr_config

* init OCR

* init ocr functions

* add OCRResponse Base Model

* add ocr to llm http handlers

* add main.py for OCR

* fix linting for OCR

* TestMistralOCR

* update to use DocumentType for Mistral

* fix _prepare_ocr_request

* fix transform

* add main.py for OCR

* add spec to init

* fix OCR

* TestMistralOCR

* ruff fix

* add router.ocr() methods

* add OCR routes

* feat add ocr routes

* add OCR routes

* feat: add OCR routes in proxy server

* working /ocr routes

* test_router_aocr_with_mistral

* docs Mistral OCR

* docs OCR

* [Feat] Add Azure AI Mistral OCR Integration  (#15572)

* fix get_supported_ocr_params

* add get_provider_ocr_config

* init OCR

* init ocr functions

* add OCRResponse Base Model

* add ocr to llm http handlers

* add main.py for OCR

* fix linting for OCR

* TestMistralOCR

* update to use DocumentType for Mistral

* fix _prepare_ocr_request

* fix transform

* add main.py for OCR

* add spec to init

* fix OCR

* TestMistralOCR

* ruff fix

* add router.ocr() methods

* add OCR routes

* feat add ocr routes

* add OCR routes

* feat: add OCR routes in proxy server

* working /ocr routes

* test_router_aocr_with_mistral

* docs Mistral OCR

* docs OCR

* add azure ai to get_provider_ocr_config

* add AzureAIOCRConfig

* TestAzureAIOCR

* TestAzureAIOCR

* test fixes for azure ai ocr

* fix async OCR transform for Azure

* fix transform_ocr_request

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-10-15 17:20:01 -07:00
Ishaan Jaffer 7288f4355f docs fix 2025-10-11 16:24:21 -07:00
Ishaan Jaffer 54b8bde60c docs fix 2025-10-11 16:23:46 -07:00
Ishaan JaffandGitHub 527c8f59fa [Feat] Tag Management - Add support for setting tag based budgets (#15433)
* feat: add LiteLLM_TagTable

* fix: use new table for tag management

* fix - allow setting budgets for tags

* working tag creation

* fix schema.prisma

* add tag info

* ui fixes

* ui fix tag info

* TAG_CACHE_IN_MEMORY_TTL_SECONDS

* add Litellm_EntityType

* fix get_aggregated_db_spend_update_transactions

* fix: _update_entity_spend_in_db

* fix _tag_max_budget_check

* add tag budget check

* add tag_list_transactions

* test_get_tag_objects_batch

* test_update_tag_db_without_prisma_client

* fix get_tags_from_request_body

* get_tags_from_request_body

* fix get_tags_from_request_body

* fix spend tracking utils

* get_tags_from_request_body

* test_get_tags_from_request_body_with_metadata_tags

* feat: add _update_tag_cache spend tracking

* fix _PROXY_track_cost_callback

* test_tag_cache_update_multiple_tags

* fix tag info

* docs fix

* docs tag budgets

* doc fix

* docs fix

* fix tag budget

* docs tag budgets

* docs fix

* ruff fix
2025-10-10 19:24:50 -07:00
Ishaan JaffandGitHub bf209415da [Feat] VertexAI Gemma model family streaming support + Added MedGemma (#15427)
* test_acompletion_filters_stream_and_stream_options

* fix: stream_options

* docs medgemma

* lint fix

* docs
2025-10-10 14:22:27 -07:00
Krish DholakiaandGitHub d944717d4b Merge pull request #15278 from xprilion/wandb-inference-docs
Add W&B Inference documentation
2025-10-09 22:35:16 -07:00
Ishaan JaffandGitHub 6cf57d85c3 [Feat] Add EnkryptAI Guardrails on LiteLLM (#15390)
* feat: add EnkryptAIGuardrail

* fix types

* fix EnkryptAIGuardrails

* __all__ = ["EnkryptAIGuardrail"]
fix

* fix EnkryptAIGuardrail

* TestEnkryptAIExtraction

* fix mypy linting

* docs enkrypt ai

* add EnkryptAIGuardrailConfigModel

* fix EnkryptAIGuardrailConfigModel

* fix import struct

* fix EnkryptAIGuardrails
2025-10-09 16:27:04 -07:00
Anubhav SinghandGitHub f59f1971cc Merge branch 'BerriAI:main' into wandb-inference-docs 2025-10-09 22:26:37 +05:30
Krrish Dholakia f88e3ebdfa docs: refactor to have function call / computer use / etc. on top level 2025-10-09 07:25:25 -07:00
Ishaan JaffandGitHub 36c971a6fd [MCP Gateway] QA/Fixes - Ensure Team/Key level enforcement works for MCPs (#15305)
* fix: _set_object_permission

* fix: _set_object_permission on teams

* fix: _set_object_permission

* fixes for team/key permissions

* statsh: object permission view

* fix: MCPServerPermissions

* fix: _get_team_object_permission

* test mcp checks for permissions

* fix server checks with prefix names

* test_list_tools_strips_prefix_when_matching_permissions

* ruff fix

* docs - refactor MCP

* docs update MCP docs

* docs allowed tools
2025-10-07 17:34:48 -07:00
xprilion 888391127f Add W&B Inference documentation 2025-10-07 19:18:45 +05:30
Krrish Dholakia fbf7ca17ba docs(security_encryption_faq.md): answers common questions about data at rest vs. in transit 2025-10-06 17:34:10 -07:00
Ishaan JaffandGitHub efa782d6d2 [Feat] Add Nvidia NIM Rerank Support (#15152)
* feat: add NvidiaNimRerankConfig

* fix: NvidiaNimRerankConfig

* fix: NvidiaNimRerankConfig

* fix routing to nvidia nim

* docs nvidia nim rerank

* TestNvidiaNim

* nvidia nim rerank fixes

* fix rerank

* transform_rerank_response

* Usage with LiteLLM Proxy

* fixes linting

* NvidiaNimRerankConfig.DEFAULT_NIM_RERANK_API_BASE

* fix Custom API Base URL

* fix rerank base

* fix main.py

* fix transform

* fix linting

* map_cohere_rerank_params

* ruff fix

* linting fixes

* ruff fix
2025-10-02 18:58:52 -07:00
Amir RefaeeandGitHub 8991657d67 added railtracks to projects using litellm (#15144) 2025-10-02 14:21:38 -07:00
Eddie Richter 929510ef5d Adding unit tests and documentation 2025-09-30 12:12:24 -06:00
abhijitjavelinandGitHub adea1bfbc4 Merge branch 'BerriAI:main' into main 2025-09-28 01:08:16 +05:30
Abhijit L 7bba372ac0 feat: add javelin guardrails 2025-09-28 01:07:08 +05:30
Krish DholakiaandGitHub 3e3e3837a1 Merge branch 'main' into litellm_dev_09_24_2025_p1 2025-09-27 11:21:07 -07:00
Krrish Dholakia 5c6a863e40 docs: cleanup 2025-09-27 11:10:44 -07:00
fdfff39222 Add Azure passthrough documentation and sidebar entry (#14958)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-09-27 10:13:16 -07:00
Ishaan JaffandGitHub d04c6d4eea [Feat] Add new anthropic web fetch tool support (#14951)
* add web_fetch tool for ANTHROPIC_HOSTED_TOOLS

* add ANTHROPIC_BETA_HEADER_VALUES, ANTHROPIC_HOSTED_TOOLS

* feat: add web fetch tool anthropic

* test_anthropic_tool_use

* docs web fetch

* docs fix

* docs fix
2025-09-26 11:04:11 -07:00
dcbccd1fea Corrected docs updates sept 2025 (#14916)
* docs: Corrected documentation updates from Sept 2025

This PR contains the actual intended documentation changes, properly synced with main:

 Real changes applied:
- Added AWS authentication link to bedrock guardrails documentation
- Updated Vertex AI with Gemini API alternative configuration
- Added async_post_call_success_hook code snippet to custom callback docs
- Added SSO free for up to 5 users information to enterprise and custom_sso docs
- Added SSO free information block to security.md
- Added cancel response API usage and curl example to response_api.md
- Added image for modifying default user budget via admin UI
- Re-ordered sidebars in documentation

 Sync issues resolved:
- Kept all upstream changes that were added to main after branch diverged
- Preserved Provider-Specific Metadata Parameters section that was added upstream
- Maintained proper curl parameter formatting (-d instead of -D)

This corrects the sync issues from the original PR #14769.

* docs: Restore missing files from original PR

Added back ~16 missing documentation files that were part of the original PR:

 Restored files:
- docs/my-website/docs/completion/usage.md
- docs/my-website/docs/fine_tuning.md
- docs/my-website/docs/getting_started.md
- docs/my-website/docs/image_edits.md
- docs/my-website/docs/image_generation.md
- docs/my-website/docs/index.md
- docs/my-website/docs/moderation.md
- docs/my-website/docs/observability/callbacks.md
- docs/my-website/docs/providers/bedrock.md
- docs/my-website/docs/proxy/caching.md
- docs/my-website/docs/proxy/config_settings.md
- docs/my-website/docs/proxy/db_deadlocks.md
- docs/my-website/docs/proxy/load_balancing.md
- docs/my-website/docs/proxy_api.md
- docs/my-website/docs/rerank.md

 Fixed context-caching issue:
- Restored provider_specific_params.md to main version (preserving Provider-Specific Metadata Parameters section)
- Your original PR didn't intend to modify this file - it was just a sync issue

Now includes all ~26 documentation files from the original PR #14769.

* docs: Remove files that were deleted in original PR

- Removed docs/my-website/docs/providers/azure_ai_img_edit.md (was deleted in original PR)
- sdk/headers.md was already not present

Now matches the complete intended changes from original PR #14769.

* docs: Restore azure_ai_img_edit.md from main

- Restored docs/my-website/docs/providers/azure_ai_img_edit.md from main branch
- This file should not have been deleted as it was a newer commit
- SDK headers file doesn't exist in main (was reverted) and wasn't part of your original changes

Fixes the file restoration issues.

* docs: Fix vertex.md - preserve context caching from newer commit

- Restored vertex.md to main version to preserve context caching content (lines 817-887)
- Added back only your intended change: alternative gemini config example
- Context caching content from newer commit is now preserved

Fixes the vertex.md sync issue where newer content was incorrectly deleted.

* docs: Fix providers/bedrock.md - restore deleted content from newer commit

- Restored providers/bedrock.md to main version
- Preserves 'Usage - Request Metadata' section that was added in newer commit
- Your actual intended change was to proxy/guardrails/bedrock.md (authentication tip) which is preserved
- Now only has additions, no subtractions as intended

Fixes the bedrock.md sync issue.

* docs: Restore missing IAM policy section in bedrock.md

Added back your intended IAM policy documentation that was lost when restoring main version:

 Added IAM AssumeRole Policy section:
- Explains requirement for sts:AssumeRole permission
- Shows error message example when permission missing
- Provides complete IAM policy JSON example
- Links to AWS AssumeRole documentation
- Clarifies trust policy requirements

Now bedrock.md has both:
- All newer content preserved (Request Metadata section)
- Your intended IAM policy addition restored

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2025-09-25 15:49:19 -07:00
Mubashir OsmaniandGitHub fc1c39645a docs: Letta Guide (#14798)
* fix: flaky passthrough tests

* Revert "fix: flaky passthrough tests"

This reverts commit ffe692e017600a8853ab7c31f95485958ab74c5f.

* fix: serialize prisma objects

* docs: added letta

* arranged endpoints in alphabetical order

* added user_api_key fields to custom auth docs

* user onboarding

* cleaning up

This reverts commit 40e2aade73632cea371c6092d766ba193f9ac4a3.

* cleaning up
This reverts commit ffe692e017600a8853ab7c31f95485958ab74c5f.

* Revert "fix: serialize prisma objects"

This reverts commit 1e7bd13c26dcac8bff1f818c39b2b78f22f0ba28.
2025-09-23 16:18:57 -07:00
Ishaan Jaffer 45f835227b Revert "Merge pull request #14769 from TeddyAmkie/doc-updates-sept-2025"
This reverts commit 666bcac2af, reversing
changes made to b7803bcbb0.
2025-09-23 14:14:58 -07:00
Krish DholakiaandGitHub af151e6ba8 Merge branch 'main' into doc-updates-sept-2025 2025-09-22 22:41:02 -07:00
Krish DholakiaandGitHub 66f7e4a832 Merge pull request #14790 from eycjur/support_flux_image_edit
[Feat] Support flux image edit
2025-09-22 22:38:45 -07:00
eycjur f24d65b807 add docs 2025-09-23 02:51:19 +00:00
Krrish Dholakia 4c66e1f6bf docs(provider_specific_params.md): fix docs 2025-09-22 17:18:57 -07:00
berri-teddy d9417f70bb Re-ordered sidebars in docs 2025-09-21 20:31:45 -07:00
Ishaan Jaffer ea24d88458 docs fix 2025-09-20 17:22:41 -07:00
Ishaan Jaffer 39cb7f67f0 fix sidebar 2025-09-20 12:30:55 -07:00
Ishaan Jaffer d8d4e46c97 docs fix 2025-09-20 10:24:55 -07:00
Felipe Gare d59e813b58 docs: add Vertex batch provider documentation and sidebar entry 2025-09-20 00:20:51 -03:00
Ishaan JaffandGitHub 4c983f985a [Feat] Add Bedrock Twelve Labs embedding provider support (#14697)
* fix: add 12 labs to bedrock embedding

* fix: get_bedrock_embedding_provider

* test: test_text_embedding

* fix: 12 labs embedding transform

* fix: refactor 12 labs transform logic

* fix: test_e2e_bedrock_embedding

* fix: test_e2e_bedrock_embedding

* feat: add bedrock twelvelabs pricing

* DOCS: docs bedrock embedding

* DOCS: 12 labs bedrock overview

* fix: bedrock embeddings 12 labs
2025-09-18 17:16:45 -07:00
Tim Elfrink 9521414efa Resolve merge conflict by including both CompactifAI and OVHCloud providers
- Keep CompactifAI provider detection logic
- Include new OVHCloud provider from main branch
- Both providers now work correctly with model prefix detection
2025-09-14 23:03:18 +02:00
Krish DholakiaandGitHub 56fd60b140 Merge pull request #14494 from eliasto/feat/ovhcloud-ai-edpoints-provider
feat: Add OVHCloud AI Endpoints as a provider
2025-09-14 00:45:08 -07:00
Krrish Dholakia 7ef8c808cf docs: update doc 2025-09-14 00:28:11 -07:00
Krish DholakiaandGitHub ad9515a81e Merge branch 'main' into litellm_contributor_prs_09_12_2025_p1 2025-09-13 09:38:43 -07:00
Tim Elfrink 1987556a50 Add CompactifAI provider documentation and config
- Create comprehensive provider documentation with usage examples
- Cover basic completion, streaming, async, and function calling
- Document AWS Marketplace subscription and API key setup process
- Include proxy configuration and advanced parameter examples
- Add error handling examples and model information
- Update website sidebar to include CompactifAI in provider list
- Update README.md with CompactifAI provider reference
2025-09-13 08:42:56 +02:00
Ishaan Jaff 8b3b943557 docs fix 2025-09-12 19:51:14 -07:00
Krrish Dholakia 05e3e3685b docs(forward_client_headers.md): document new forwarding client headers to llm api feature 2025-09-12 19:38:07 -07:00
Ishaan JaffandGitHub 93af8fd6ba [QA] E2E - Testing for bedrock batches api (#14525)
* add bedrock/batch-anthropic.claude-3-5-sonnet-20240620-v1:0

* test_bedrock_batches_api

* fix

* fix import

* test_bedrock_batches_api
2025-09-12 19:31:19 -07:00
Ishaan JaffandGitHub 20c390dc71 [Docs] - Bedrock Batches API (#14526)
* docs bedrock batch

* docs fix

* docs bedrock batches

* fix Create Batch Request

* docs fix
2025-09-12 19:08:13 -07:00
Elias TOURNEUX ef9d1ddc40 feat: Add OVHCloud AI Endpoints as a provider 2025-09-12 13:37:03 +02:00
Krish DholakiaandGitHub d72081113e Merge pull request #14111 from dharamendrak/feature/aiohttp-dependency-injection
feat: Add dependency injection support to BaseLLMAIOHTTPHandler
2025-09-09 22:35:30 -07:00
Dharamendra Kumar 92596e48fe Merge branch 'main' into feature/aiohttp-dependency-injection 2025-09-07 17:54:55 -07:00
Dharamendra Kumar 009106def0 docs for custom aiohttp session 2025-09-07 17:42:26 -07:00
Krish DholakiaandGitHub ba10173ec7 Merge branch 'main' into heroku-llms 2025-09-06 22:10:20 -07:00
Krish DholakiaandGitHub 8b11452d9e Merge branch 'main' into add-vercel-ai-gateway-provider 2025-08-29 22:27:08 -07:00