tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-08-06 00:23:19 +00:00
Code Issues Packages Projects Releases Wiki Activity
23,576 Commits 2,072 Branches 1,425 Tags
7bb48ffaf948602d0d74b434d6864400a276c2ab
Commit Graph
23576 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Ishaan Jaff 7bb48ffaf9 docs auto routing 2025-07-25 08:50:16 -07:00
Ishaan Jaff 50574c0c3e docs - auto routing 2025-07-25 07:31:39 -07:00
Ishaan Jaff 20450bfe94 fix mapped test 2025-07-25 07:24:42 -07:00
Ishaan Jaff ecfafd1243 docs update 2025-07-25 07:19:32 -07:00
Ishaan Jaff 7745f61fb1 docs update 2025-07-25 07:19:20 -07:00
Ishaan Jaff 9c76ccd92c docs Health Check Server 2025-07-25 07:17:26 -07:00
Krish DholakiaandGitHub 6d3ce76e1c Guardrails - support model-level guardrails (#12968) 
* fix(custom_guardrail.py): initial logic for model level guardrails

* feat(custom_guardrail.py): working pre call guardrails

* fix(custom_guardrails.py): check if custom guardrails set before running event hook

* test(test_custom_guardrail.py): add unit tests for async pre call deployment hook on custom guardrail

* feat(custom_guardrail.py): add post call processing support for guardrails

allows model based guardrails to run on the post call event for that model only

* fix(utils.py): only run if call type is in enum

* test: update unit tests to work
 2025-07-24 22:38:03 -07:00
Krish DholakiaandGitHub dd607cd301 Show global retry policy on UI (#12969) 
* fix(router.py): return global retry policy on `get/config/callbacks`

Partial fix for https://github.com/BerriAI/litellm/issues/12855

* fix(model_dashboard.tsx): accept global retry policy

Fixes https://github.com/BerriAI/litellm/issues/12855

* fix(model_dashboard.tsx): update global retry policy, if that's what was edited
 2025-07-24 22:37:15 -07:00
Dmitriy AlergantandGitHub 0cc320cc0a GuardrailsAI: use validatedOutput to allow usage of "fix" guards. Previously "fix" guards had no effect in llmOutput mode. (#12891) 2025-07-24 22:07:06 -07:00
sings-to-bees-on-wednesdaysandGitHub eb96fb78bc fix(auth_utils): make header comparison case-insensitive (#12950) 
If the user specified in the configuration e.g. "user_header_name:
X-OpenWebUI-User-Email", here we were looking for a dict key
"X-OpenWebUI-User-Email" when the dict actually contained
"x-openwebui-user-email".

Switch to iteration and case insensitive string comparison instead to
fix this.

This fixes customer budget enforcement when the customer ID is passed
in as a header rather than as a "user" value in the body.
 2025-07-24 22:06:12 -07:00
Cole McIntoshandGitHub 3ca65b3064 feat: Update model pricing and context window configurations (#12910) 
- Adjusted input and output cost per token for existing models.
- Added new model configuration for "openrouter/qwen/qwen3-coder" with specified token limits and costs.
 2025-07-24 21:57:15 -07:00
GastonandGitHub 6cfaf674e4 Fix: Shorten Gemini tool_call_id for Azure compatibility (#12941) 2025-07-24 21:52:16 -07:00
Murad KhafizovandGitHub 650ae0ef88 Added documentation about metadata exposed over the /v1/models endpoint (#12942) 2025-07-24 21:51:16 -07:00
Santosh DhaladhuliandGitHub d9d53466e8 [Feat] Add inpainting support and corresponding tests for Amazon Nova Canvas (#12949) 2025-07-24 21:46:06 -07:00
Ishaan Jaff fdb2ebf344 bump: version 1.74.8 → 1.74.9 2025-07-24 21:27:15 -07:00
Ishaan Jaff d892e969ab ui new build 2025-07-24 21:27:06 -07:00
Ishaan Jaff 91eee78a8c Revert "EditAutoRouterTabProps" 
This reverts commit 2835d3a3743e6411b9914a0b01381050e2273ad7.
 2025-07-24 21:26:01 -07:00
Ishaan Jaff 0630b52044 EditAutoRouterTabProps 2025-07-24 21:26:01 -07:00
Ishaan JaffandGitHub 39b0fe0bf7 [Feat] Edit Auto Router Settings on UI (#12966) 
* EditAutoRouterTabProps

* Revert "EditAutoRouterTabProps"

This reverts commit 2835d3a3743e6411b9914a0b01381050e2273ad7.

* add EditAutoRouterTab

* delete edit

* fixes for edit auto-router

* fix accessing model edit

* working edit auto router

* fix - edit remove custom model name

* fixes for edit auto router settings

* qa for adding a model router

* test fix
 2025-07-24 21:25:48 -07:00
Ishaan Jaff 29cff4f4e6 pip install semantic_router --no-deps 2025-07-24 20:09:37 -07:00
Ishaan Jaff 6e0c84363e test fix test_litellm_anthropic_prompt_caching_tools 2025-07-24 20:06:22 -07:00
Ishaan JaffandGitHub 106a298f0a [Feat] UI - Allow Adding LiteLLM Auto Router on UI (#12960) 
* add router.json

* test_router_auto_router

* async_pre_routing_hook

* fixes for auto router

* add async_pre_routing_hook

* add LiteLLMRouterEncoder

* update test auto_router_embedding_model

* add auto_router_embedding_model

* add AutoRouter

* fix async_pre_routing_hook

* update async_pre_routing_hook

* fix auto router

* fix router.json

* working router init

* working embedding encoder

* working auto router

* test_router_auto_router

* test auto router

* add semantic-router as optional for litellm

* add extras

* semantic_router==0.1.10

* ruff fix

* use aiohttp==3.10.11

* python-dotenv==1.0.1

* test auto router

* test_router_auto_router

* semantic_router

* test_is_auto_router_deployment

* fix check

* fix docker build step

* add semantic_router

* UI  - Add auto router on litellm

* working utterances config

* fix route config builder

* kind of working add automodel router

* move loc of add deployment

* fixes for AutoRouter

* add auto_router_config in types.py

* fixes for init_auto_router_deployment

* fix adding auto router models

* working auto-router with dB

* Revert "add semantic_router"

This reverts commit 537b67288798731a119d811f643b682086377ee9.

* TestAutoRouter

* fix linting

* add semantic router to docker

* test fix

* fix router config builder

* remove export button
 2025-07-24 19:58:49 -07:00
Jugal D. BhattandGitHub e63b163578 added new realtime model for openai (#12946) 2025-07-24 19:57:39 -07:00
Ishaan JaffandGitHub 6174609685 [LLM Translation] - Bug fix Anthropic Tool calling (#12959) 
* fix _map_tool_helper

* fix linting
 2025-07-24 19:55:52 -07:00
Ishaan Jaff 939f44f398 docs key_type 2025-07-24 18:46:13 -07:00
Ishaan Jaff 2bf0c1187c v1.74.7-stable 2025-07-24 18:44:21 -07:00
Ishaan Jaff 095ddea0dc docs index 2025-07-24 18:39:11 -07:00
Ishaan JaffandGitHub b8e404dd95 [Feat] Backend Router - Add Auto-Router powered by semantic-router (#12955) 
* add router.json

* test_router_auto_router

* async_pre_routing_hook

* fixes for auto router

* add async_pre_routing_hook

* add LiteLLMRouterEncoder

* update test auto_router_embedding_model

* add auto_router_embedding_model

* add AutoRouter

* fix async_pre_routing_hook

* update async_pre_routing_hook

* fix auto router

* fix router.json

* working router init

* working embedding encoder

* working auto router

* test_router_auto_router

* test auto router

* add semantic-router as optional for litellm

* add extras

* semantic_router==0.1.10

* ruff fix

* use aiohttp==3.10.11

* python-dotenv==1.0.1

* test auto router

* test_router_auto_router

* semantic_router

* test_is_auto_router_deployment

* fix check

* fix docker build step

* add semantic_router

* Revert "add semantic_router"

This reverts commit 537b67288798731a119d811f643b682086377ee9.
 2025-07-24 18:32:56 -07:00
Krish DholakiaandGitHub 8aa4beff5f fix(internal_user_endpoints.py): delete member from team table on /user/delete (#12926) 
removes user from team when user is deleted
 2025-07-24 16:42:28 -07:00
Krish DholakiaandGitHub 1a57875d24 Proxy - specify key_type - allows specifying if key can call LLM API routes vs. Management routes only (#12909) 
* feat(key_management_endpoints.py): Support new 'key_type' field

allow user to specify if key should be 'management' or 'llm api' key

Security fix

* test(test_route_checks.py): add unit tests

* fix(create_key_button.tsx): add ui component to select key type

allows specifying if key can call llm api vs. management routes

* feat(create_key_button.tsx): add specifying key type to ui

* fix(route_checks.py): add sensitive data masker for user id on not allowed error message

prevent leaking sensitive information
 2025-07-24 16:40:40 -07:00
Krish DholakiaandGitHub 550211ba47 Prometheus - tags, fix '[tag]="false"' when tag is set (#12916) 
* fix(prometheus.py): sanitize tag-based labels to handle colons (:) and spaces ( )

* fix(prometheus.py): working tag based metrics

* fix(prometheus.py): emit request tags on post call success hook

* fix(prometheus.py): add user agent tags on request failure

* fix(prometheus.py): add request tags to deployment failure metric

s
 2025-07-24 16:39:13 -07:00
Ishaan Jaff 0a07698d9d docs update 2025-07-24 11:26:30 -07:00
Cole McIntoshandGitHub 8c99c71f38 Merge pull request #12939 from superpoussin22/correct_doc 
Update control_plane_and_data_plane.md
 2025-07-24 09:17:01 -06:00
superpoussin22andGitHub ca9712c587 Update control_plane_and_data_plane.md 2025-07-24 16:15:55 +02:00
Ishaan JaffandGitHub 3e529fae7f [UI] Allow setting up CloudZero Usage through LiteLLM UI (#12923) 
* refactor cz settings

* allow viewing/setting CZ settings

* working button

* fixes for export button

* nicer button

* fix className
 2025-07-23 21:17:12 -07:00
Apoorv SharmaandGitHub 29475ab4b0 Add GA version of gemini 2.5 flash lite for both vertex and gemini (#12920) 
This PR adds the config for gemini 2.5 flash lite which is out of preview now
 2025-07-23 18:47:46 -07:00
Ishaan JaffandGitHub e3b752d3dc [Feat] LiteLLM CloudZero Integration updates - using LiteLLM_SpendLogs Table (#12922) 
* use spend logs in transform.py

* fixes for using spend logs

* get_usage_data

* update get_usage_data_for_hour

* fix CloudZeroLogger

* fixes for cloudzero_dry_run_export

* working DB read

* fix _aggregate_to_hourly

* update cloudzero endpoints

* fix cloudzero_export

* fixes for CBF transformer

* use init_background_job on saving cz settings

* add is_cloudzero_setup_in_db

* init cz on startup

* fixes for CZ transform

* fix ruff check

* fix LOC violation
 2025-07-23 18:46:50 -07:00
Ishaan Jaff 99031bf8b6 ci/cd new release v1.74.8-nightly 2025-07-23 13:50:36 -07:00
Ishaan Jaff 461cd0c30a test_completion_cost_deepseek 2025-07-23 13:16:12 -07:00
Ishaan Jaff 2d0187824c test_proxy_model_resolution_with_custom_names_documentation 2025-07-23 13:13:46 -07:00
Ishaan Jaff 79a0841719 test_router_content_policy_fallbacks 2025-07-23 13:04:28 -07:00
Ishaan JaffandGitHub 99ad24c683 [Feat] LiteLLM x Cloudzero integration - Allow exporting spend to cloudzero (#12908) 
* add cz init

* add DataAnalyzer

* add CZRNGenerator -> LiteLLM resource generator

* add CloudZeroStreamer

* add CBFTransformer

* add cloudzero DB connection

* add config

* add cli.py

* cleanup

* add CloudZeroLogger

* add CloudZeroLogger

* update CloudZeroLogger

* add cloudzero_router to litellm proxy

* add CloudZeroInitRequest

* add cloudzero spend endpoints

* simple dry run endpoint

* refactor dir structure

* add well types CBFRecord

* TestCBFTransformer

* TestCloudZeroStreamer

* ruff fix

* add polars pip + docker requirements.txt

* _group_by_date

* update code qa check

* docs for CZ params
 2025-07-23 13:00:58 -07:00
Krish DholakiaandGitHub c5c2a68510 Swagger - bump DOMPurify version + add back Support local swagger bundle (good for airgapped environments) + support local swagger on custom root paths (#12911) 
* fix(proxy_server.py): update swagger-ui-bundle.js + bring back swagger in airgapped environments

* fix(proxy_server.py): support local swagger on custom root path

enables on prem usage of litellm
 2025-07-23 11:22:56 -07:00
Ishaan Jaff 0d2e280e78 ci/cd new release 2025-07-23 09:04:26 -07:00
Ishaan Jaff 00fd020291 fix test 2025-07-23 09:04:09 -07:00
Krrish Dholakia 4277becdc2 docs(credentials.tsx): cleanup reference to dead link 2025-07-23 09:02:39 -07:00
Ishaan Jaff 60ed272ae0 ci/cd litellm 2025-07-23 07:15:33 -07:00
Ishaan Jaff 5616aaee6d docs pillar guard 2025-07-23 07:15:02 -07:00
Ishaan Jaff 477ca5c0f6 test_proxy_all_models 2025-07-23 07:11:35 -07:00
Krrish Dholakia 663dea37a8 docs: fix docs 2025-07-22 23:30:50 -07:00