tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-07-11 05:03:57 +00:00
Code Issues Packages Projects Releases Wiki Activity
27,276 Commits 2,072 Branches 1,425 Tags
443bada4259f835fa5487d19682181bbf5bb66bb
Commit Graph

4656 Commits

Author SHA1 Message Date
jwang-gif 443bada425 Add Zscaler AI Guard hook (#15691) 
* Add Zscaler AI Guard hook

Co-authored-by: Angela Tao <atao@zscaler.com>

* Fix lint error, update document

* Fix lint error, update document

* update document

* fix mypy type error

* fix mypy issue

* fix test

* fix test

* improve document

* remove unuseful code

* use litellm httphandler

* update test cases

* revover guardrail_initializers.py and guardrail_registry.py

* remove unuse import

* app apply_guardrail

* remove functions repleased by apply_guardrail, update test and doc

* remove functions repleased by apply_guardrail, update test and doc

---------

Co-authored-by: Angela Tao <atao@zscaler.com>
 2025-11-11 15:34:27 -08:00
Ishaan Jaffer be05324645 docs fix MAX_LANGFUSE_INITIALIZED_CLIENTS 2025-11-11 11:26:33 -08:00
Ishaan Jaff 5c9f50d584 [AI Gateway] - End User Budgets - Allow pointing max_end_user budget to an id, so the default ID applies to all end users (#16456) 
* add _apply_budget_limits_to_end_user_params

* add _apply_budget_limits_to_end_user_params

* add _apply_budget_limits_to_end_user_params

* test_default_budget_applied_to_end_user_without_budget

* docs fix

* fix config
 2025-11-11 08:20:13 -08:00
Sameer Kankute aaa8cba00b Add docs for tracking callback failure (#16474) 2025-11-10 19:15:14 -08:00
Sameer Kankute bf363cdf10 Add sdk focused examples (#16441) 2025-11-10 18:39:34 -08:00
Sameer Kankute be3c09e6d5 Add GET list of providers endpoint (#16432) 2025-11-10 18:38:09 -08:00
‮Artem a97676b8ae Add softgen to projects that are using litellm (#16423) 
* Add Softgen project documentation

* Add Softgen project to sidebars

* Update Softgen
 2025-11-10 11:09:29 -08:00
Sameer Kankute 013c195ab7 Fix container api link (#16440) 2025-11-10 11:09:10 -08:00
Krrish Dholakia 50f35fca47 docs: reorder docs 2025-11-10 07:33:22 -08:00
Ishaan Jaffer c088483da9 docs playground 2025-11-08 18:15:52 -08:00
Ishaan Jaffer f2832ba2f8 docs fix 2025-11-08 17:25:01 -08:00
Ishaan Jaffer b803af4aae docs fix 2025-11-08 17:05:20 -08:00
Ishaan Jaffer 36d181e398 docs fix + linting fix 2025-11-08 17:02:00 -08:00
Ishaan Jaffer 42c1e4740a docs built in guard 2025-11-08 16:59:00 -08:00
Ishaan Jaffer a298a9d492 docs fix 2025-11-08 16:52:26 -08:00
Ishaan Jaffer 7fb46e0b29 docs fix 2025-11-08 16:50:51 -08:00
Ishaan Jaffer 4647b41d7f docs fix 2025-11-08 16:50:06 -08:00
Ishaan Jaff 08cfc2bf7e [Docs] Litellm 1 79 2 rc (#16415) 
* stash - v1

* docs fix

* docs fix

* docs

* folder fix

* docs fix
 2025-11-08 16:47:15 -08:00
Alexsander Hamir 277b564385 add: performance improvements to release notes (#16401) 
* add: performance improvements to release notes

* fix: mention which endpoint is getting faster

* fix: clarify that reported latency is end-to-end
 2025-11-08 16:42:58 -08:00
Ishaan Jaff 20180ff93b [Docs] litellm content filter guard (#16413) 
* add gif 1

* add guard 2

* add guard 3

* guard 5

* docs fix

* docs fix
 2025-11-08 16:30:13 -08:00
Sameer Kankute 86d73c918c Adds support for returning Azure Content Policy error information when exceptions from Azure OpenAI occur (#16231) 
* add provider_specific_fields to ContentPolicyViolationError

* use provider_specific_fields in ProxyException

* update openai_exception_handler

* fix use exception checker for content policy violation azure

* add AzureOpenAIExceptionMapping

* test_azure_with_content_safety_error

* Accessing Provider-Specific Error Details

* TestExceptionCheckers

* unit test got provider_specific_fields=

* add clear types for error dict

* fix test_azure_with_content_safety_error

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
 2025-11-08 16:04:36 -08:00
Sameer Kankute e037d9315d Add Vertex and Gemini Videos API with Cost Tracking + UI support (#16323) 
* Use video id for videos api

* remove mock code

* Potential fix for code scanning alert no. 3630: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* remove print statements

* Update video prefix for 'video_'

* Add veo with openai videos unified specs

* Add videos testing to UI

* remove mock code

* Remove not need ui changes:

* Fix mypy errors related to gemini

* fix test_transform_video_create_request

* Add vertex ai veo config

* Add vertex ai veo config

* Add cost tracking for gemini and add optional param passing

* fix bugs related to vertex ai veo

* Add Gemini Veo Video Generation in Openai Videos Unified Spec (#16229)

* Add veo with openai videos unified specs

* Add videos testing to UI

* remove mock code

* Remove not need ui changes:

* Fix mypy errors related to gemini

* fix test_transform_video_create_request

* Add contant video duration for gemini and vertex

* Fix litellm_mapped_tests tests

* fix azure videos issue

* Added doc for videos vertex ai

* fix seconds param error

* fix lint errors

* test_transform_video_create_response_cost_tracking_no_duration

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com>
 2025-11-08 16:03:51 -08:00
Alan Ponnachan 30873688f2 docs: Add documentation for Anthropic memory tool (#16388) 2025-11-08 12:38:57 -08:00
Ishaan Jaffer bae1857787 fix doc 2025-11-08 10:31:19 -08:00
Cesar Garcia d65a29b88d docs: fix image generation response format from 'image' to 'images' (#16378) 
Update documentation to reflect actual API response format:
- Change singular 'image' field to plural 'images' array
- Add complete ImageURLListItem structure with index and type fields
- Update all code examples to use message.images instead of message.image
- Fix streaming examples to access images[0]["image_url"]["url"]

The documentation was incorrectly showing 'image' (singular object)
but the actual implementation returns 'images' (array of ImageURLListItem).

Related to issue #16227
 2025-11-07 19:06:03 -08:00
Krrish Dholakia 532ebf43d0 docs(moderation.md): fix moderation quick start docs 2025-11-07 16:25:08 -08:00
Sameer Kankute faae0ff0dc Fix Azure DALL-E-3 health check content policy violation by using safe default prompt (#16329) 
* Add custom health check prompt support

* Add constant for health check prompt

* Add constant for health check prompt
 2025-11-07 15:30:56 -08:00
Krrish Dholakia 9059905d25 docs(openai/videos.md): document proxy usage on openai docs for video gen 2025-11-07 15:27:18 -08:00
Ishaan Jaff 2bd85dc455 [Feat] Add DD Agent Host support for datadog callback (#16379) 
* add DD_AGENT_HOST

* docs DD Agent

* test_datadog_agent_configuration

* DD_AGENT_HOST
 2025-11-07 15:18:23 -08:00
Ishaan Jaff a6b0993405 [Feat] Secret Manager - Hashicorp, add auth via approle (#16374) 
* add _verify_required_credentials_exist and _auth_via_approle

* test_hashicorp_secret_manager_approle_auth

* docs hcorp auth
 2025-11-07 14:39:33 -08:00
Ishaan Jaffer ea4048324b docs fix 2025-11-07 14:39:24 -08:00
Jason Roberts 5733f711fd feat(guardrails): panw prisma airs guardrail deduplication and enhanced session tracking (#16273) 
* feat(guardrails): Add deduplication and session tracking

- Implement deduplication logic to prevent duplicate scans (via call_id; add _check_and_mark_scanned) caused by LiteLLM callback system
- Add session tracking using litellm_trace_id as AI Session ID for Prisma AIRS SCM logging
- Extract helper methods _extract_prompt_from_request maintainability
- Use httpxSpecialProvider import (LoggingCallback -> GuardrailCallback)
- Add comprehensive tests for deduplication and session tracking (7 new tests)
- Update documentation with multi-turn conversation tracking examples

* docs: update PANW Prisma AIRS multi-turn conversation example to use industry-standard terminology

- Clearer example for conversation tracking
- Updated terminology from 'AI Session ID' to 'Prisma AIRS AI Session ID' for clarity

* fix: remove unused asyncio import

* fix: correct mypy type ignore comment
 2025-11-06 19:34:37 -08:00
Sameer Kankute 83998d3573 Update the fireworks url in tests and doc (#16346) 2025-11-06 19:22:21 -08:00
Ishaan Jaffer a05159dc6e docs fix 2025-11-06 17:21:23 -08:00
Ishaan Jaffer 830a9e9585 docs fix 2025-11-06 16:22:17 -08:00
Ishaan Jaff 18a5c4f75a [Feat] Add LiteLLM Gateway built in guardrail (#16338) 
* add ContentFilterAction

* store pre-built regex patterns

* add v0 of content filter guard

* add _filter_messages

* test content filter guard

* init ContentFilterGuardrail

* fix ContentFilterGuardrail enums

* rename folder

* fix litellm_content_filter

* refactor content filter guard

* test content filter

* add streaming for ContentFilterGuardrail

* test_streaming_hook_mask

* add litellm_content_filter

* docs show litellm content filter

* docs litellm content filter

* fix lnting

* Potential fix for code scanning alert no. 3675: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
 2025-11-06 16:02:28 -08:00
Sameer Kankute 9dc22eed5c Update container documentation to be similar to others (#16327) 2025-11-06 10:08:45 -08:00
Ishaan Jaff 630a746c84 [Feat] Add Custom Secret Manager - Allow users to define and write a custom secret manager (#16297) 
* add CustomSecretManager class

* docs custom secret manager

* add TestCustomSecretManager

* add KeyManagementSystem.CUSTOM

* add get_secret_from_manager

* add custom secret manager

* Potential fix for code scanning alert no. 3662: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* docs fix

* load_custom_secret_manager

* initialize_secret_manager

* add custom_secret_manager

* fix add custom secret manager

* add custom secret manager to KeyManagementSystem

* fix KeyManagementSystem.CUSTOM

* fix custom secret manager within cookbook

* fix link for custom secret manager

* Potential fix for code scanning alert no. 3663: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
 2025-11-05 17:20:26 -08:00
Krrish Dholakia fa278a1886 docs: move guardrails and responses api out of beta 2025-11-05 16:48:54 -08:00
Ishaan Jaff 2541a54023 [Feat] Email Notifications - Ensure Users get Key Rotated Email (#16292) 
* add virtual_key_rotated

* add KEY_ROTATED_EMAIL_TEMPLATE

* fix base email

* add _send_key_rotated_email

* fix _send_key_rotated_email

* key rotated email fix

* fix: async_key_generated_hook ensure behavior matches regenerate

* test_send_key_rotated_email

* expose EMAIL_INCLUDE_API_KEY

* docs mask api keys

* add email regen

* docs key regen

* docs email

* docs email

* docs fix

* fix code qa check
 2025-11-05 15:56:49 -08:00
Alexsander Hamir e06d1d623f Add: benchmark comparison with other AI gateways (#16248) 
* fix: benchmark comparison location & content

* add: pros and cons

* remove
 2025-11-05 15:44:30 -08:00
Deepanshu Lulla 3ef210e5f4 Add allowing Key based prefix to s3 path (#16237) 
* add Key based prefix

* add Key based prefix

* update documentation

* correct doc

---------

Co-authored-by: deepanshu <deepanshu.lulla@hq.bill.com>
 2025-11-05 14:43:21 -08:00
Ishaan Jaff 466e7d178c [Feat] Cyber Ark - Add Key Rotations support (#16289) 
* KeyManagementSystem add cyberark

* add CyberArkSecretManager

* add CyberArkSecretManager

* add CyberArkSecretManager

* docs add CyberArkSecretManager

* docs

* refactor to use get_secret_from_manager

* fix async roate for cyber ark, re-use base class

* fixes

* cyber ark

* docs fix

* docs fix

* docs cyberark

* fix linting

* fix get_secret_from_manager
 2025-11-05 14:03:43 -08:00
Ishaan Jaff 9a372bfad6 [Feat] Add CyberArk Secrets Manager Integration (#16278) 
* KeyManagementSystem add cyberark

* add CyberArkSecretManager

* add CyberArkSecretManager

* add CyberArkSecretManager

* docs add CyberArkSecretManager

* docs

* refactor to use get_secret_from_manager

* Potential fix for code scanning alert no. 3645: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* Potential fix for code scanning alert no. 3650: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* Potential fix for code scanning alert no. 3649: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* Potential fix for code scanning alert no. 3646: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
 2025-11-05 14:00:45 -08:00
Krrish Dholakia 9b925c7e47 docs: remove old doc 
causing search issues
 2025-11-05 08:23:35 -08:00
Ishaan Jaff 60f3a3b0ad [Feat] add serxng search API provider (#16259) 
* TestFirecrawlSearch

* add SearchProviders

* add to get_provider_search_config

* add FirecrawlSearchConfig

* add FirecrawlSearchRequest

* add firecrawl API docs

* add pricing firecrawl/search

* add new search APIs

* add SearXNGSearchConfig

* add searxng/search

* add serxng params

* TestSearXNGSearch

* docs serxng

* docs fix

* docs fix

* docs serxng
 2025-11-04 17:56:07 -08:00
Ishaan Jaff af78a93ecf [Feat] /search API - add firecrawl search API support (#16257) 
* TestFirecrawlSearch

* add SearchProviders

* add to get_provider_search_config

* add FirecrawlSearchConfig

* add FirecrawlSearchRequest

* add firecrawl API docs

* add pricing firecrawl/search

* add new search APIs
 2025-11-04 17:52:12 -08:00
Krrish Dholakia a3810a5f3a docs(openai_passthrough.md): document how to make openai passthrough route work 2025-11-04 17:12:33 -08:00
Ishaan Jaff c1ac6e25aa [Feat] Add Bedrock Agentcore as a provider on LiteLLM Python SDK and LiteLLM AI Gateway (#16252) 
* add agentcore in get_bedrock_route

* add AmazonAgentCoreConfig

* fix get_runtime_endpoint

* init AmazonAgentCoreConfig

* add get_bedrock_chat_config

* get_bedrock_chat_config

* add AmazonAgentCoreConfig

* fix get_complete_url

* refactor transform response

* test agentcore

* test_bedrock_agentcore_with_streaming

* fix _parse_json_response

* fix _calculate_usage

* test_bedrock_agentcore_basic

* add AgentCoreSSEStreamIterator

* add native streaming for agentcore

* test_bedrock_agentcore_with_streaming

* test_bedrock_agentcore_basic

* add agentcore

* _calculate_usage

* fix linting
 2025-11-04 16:35:12 -08:00
Deepanshu Lulla 812ea03d28 Add tags and descriptions support to aws secrets manager (#16224) 
* Add tags and descriptions support to aws secrets manager

* add tags

---------

Co-authored-by: deepanshu <deepanshu.lulla@hq.bill.com>
 2025-11-04 16:11:51 -08:00