tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-07-19 20:18:02 +00:00
Code Issues Packages Projects Releases Wiki Activity
23,559 Commits 2,072 Branches 1,425 Tags
0630b52044700ea40e72fe2230231e375cd6cf9c
Commit Graph
23559 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Ishaan Jaff 0630b52044 EditAutoRouterTabProps 2025-07-24 21:26:01 -07:00
Ishaan JaffandGitHub 39b0fe0bf7 [Feat] Edit Auto Router Settings on UI (#12966) 
* EditAutoRouterTabProps

* Revert "EditAutoRouterTabProps"

This reverts commit 2835d3a3743e6411b9914a0b01381050e2273ad7.

* add EditAutoRouterTab

* delete edit

* fixes for edit auto-router

* fix accessing model edit

* working edit auto router

* fix - edit remove custom model name

* fixes for edit auto router settings

* qa for adding a model router

* test fix
 2025-07-24 21:25:48 -07:00
Ishaan Jaff 29cff4f4e6 pip install semantic_router --no-deps 2025-07-24 20:09:37 -07:00
Ishaan Jaff 6e0c84363e test fix test_litellm_anthropic_prompt_caching_tools 2025-07-24 20:06:22 -07:00
Ishaan JaffandGitHub 106a298f0a [Feat] UI - Allow Adding LiteLLM Auto Router on UI (#12960) 
* add router.json

* test_router_auto_router

* async_pre_routing_hook

* fixes for auto router

* add async_pre_routing_hook

* add LiteLLMRouterEncoder

* update test auto_router_embedding_model

* add auto_router_embedding_model

* add AutoRouter

* fix async_pre_routing_hook

* update async_pre_routing_hook

* fix auto router

* fix router.json

* working router init

* working embedding encoder

* working auto router

* test_router_auto_router

* test auto router

* add semantic-router as optional for litellm

* add extras

* semantic_router==0.1.10

* ruff fix

* use aiohttp==3.10.11

* python-dotenv==1.0.1

* test auto router

* test_router_auto_router

* semantic_router

* test_is_auto_router_deployment

* fix check

* fix docker build step

* add semantic_router

* UI  - Add auto router on litellm

* working utterances config

* fix route config builder

* kind of working add automodel router

* move loc of add deployment

* fixes for AutoRouter

* add auto_router_config in types.py

* fixes for init_auto_router_deployment

* fix adding auto router models

* working auto-router with dB

* Revert "add semantic_router"

This reverts commit 537b67288798731a119d811f643b682086377ee9.

* TestAutoRouter

* fix linting

* add semantic router to docker

* test fix

* fix router config builder

* remove export button
 2025-07-24 19:58:49 -07:00
Jugal D. BhattandGitHub e63b163578 added new realtime model for openai (#12946) 2025-07-24 19:57:39 -07:00
Ishaan JaffandGitHub 6174609685 [LLM Translation] - Bug fix Anthropic Tool calling (#12959) 
* fix _map_tool_helper

* fix linting
 2025-07-24 19:55:52 -07:00
Ishaan Jaff 939f44f398 docs key_type 2025-07-24 18:46:13 -07:00
Ishaan Jaff 2bf0c1187c v1.74.7-stable 2025-07-24 18:44:21 -07:00
Ishaan Jaff 095ddea0dc docs index 2025-07-24 18:39:11 -07:00
Ishaan JaffandGitHub b8e404dd95 [Feat] Backend Router - Add Auto-Router powered by semantic-router (#12955) 
* add router.json

* test_router_auto_router

* async_pre_routing_hook

* fixes for auto router

* add async_pre_routing_hook

* add LiteLLMRouterEncoder

* update test auto_router_embedding_model

* add auto_router_embedding_model

* add AutoRouter

* fix async_pre_routing_hook

* update async_pre_routing_hook

* fix auto router

* fix router.json

* working router init

* working embedding encoder

* working auto router

* test_router_auto_router

* test auto router

* add semantic-router as optional for litellm

* add extras

* semantic_router==0.1.10

* ruff fix

* use aiohttp==3.10.11

* python-dotenv==1.0.1

* test auto router

* test_router_auto_router

* semantic_router

* test_is_auto_router_deployment

* fix check

* fix docker build step

* add semantic_router

* Revert "add semantic_router"

This reverts commit 537b67288798731a119d811f643b682086377ee9.
 2025-07-24 18:32:56 -07:00
Krish DholakiaandGitHub 8aa4beff5f fix(internal_user_endpoints.py): delete member from team table on /user/delete (#12926) 
removes user from team when user is deleted
 2025-07-24 16:42:28 -07:00
Krish DholakiaandGitHub 1a57875d24 Proxy - specify key_type - allows specifying if key can call LLM API routes vs. Management routes only (#12909) 
* feat(key_management_endpoints.py): Support new 'key_type' field

allow user to specify if key should be 'management' or 'llm api' key

Security fix

* test(test_route_checks.py): add unit tests

* fix(create_key_button.tsx): add ui component to select key type

allows specifying if key can call llm api vs. management routes

* feat(create_key_button.tsx): add specifying key type to ui

* fix(route_checks.py): add sensitive data masker for user id on not allowed error message

prevent leaking sensitive information
 2025-07-24 16:40:40 -07:00
Krish DholakiaandGitHub 550211ba47 Prometheus - tags, fix '[tag]="false"' when tag is set (#12916) 
* fix(prometheus.py): sanitize tag-based labels to handle colons (:) and spaces ( )

* fix(prometheus.py): working tag based metrics

* fix(prometheus.py): emit request tags on post call success hook

* fix(prometheus.py): add user agent tags on request failure

* fix(prometheus.py): add request tags to deployment failure metric

s
 2025-07-24 16:39:13 -07:00
Ishaan Jaff 0a07698d9d docs update 2025-07-24 11:26:30 -07:00
Cole McIntoshandGitHub 8c99c71f38 Merge pull request #12939 from superpoussin22/correct_doc 
Update control_plane_and_data_plane.md
 2025-07-24 09:17:01 -06:00
superpoussin22andGitHub ca9712c587 Update control_plane_and_data_plane.md 2025-07-24 16:15:55 +02:00
Ishaan JaffandGitHub 3e529fae7f [UI] Allow setting up CloudZero Usage through LiteLLM UI (#12923) 
* refactor cz settings

* allow viewing/setting CZ settings

* working button

* fixes for export button

* nicer button

* fix className
 2025-07-23 21:17:12 -07:00
Apoorv SharmaandGitHub 29475ab4b0 Add GA version of gemini 2.5 flash lite for both vertex and gemini (#12920) 
This PR adds the config for gemini 2.5 flash lite which is out of preview now
 2025-07-23 18:47:46 -07:00
Ishaan JaffandGitHub e3b752d3dc [Feat] LiteLLM CloudZero Integration updates - using LiteLLM_SpendLogs Table (#12922) 
* use spend logs in transform.py

* fixes for using spend logs

* get_usage_data

* update get_usage_data_for_hour

* fix CloudZeroLogger

* fixes for cloudzero_dry_run_export

* working DB read

* fix _aggregate_to_hourly

* update cloudzero endpoints

* fix cloudzero_export

* fixes for CBF transformer

* use init_background_job on saving cz settings

* add is_cloudzero_setup_in_db

* init cz on startup

* fixes for CZ transform

* fix ruff check

* fix LOC violation
 2025-07-23 18:46:50 -07:00
Ishaan Jaff 99031bf8b6 ci/cd new release v1.74.8-nightly 2025-07-23 13:50:36 -07:00
Ishaan Jaff 461cd0c30a test_completion_cost_deepseek 2025-07-23 13:16:12 -07:00
Ishaan Jaff 2d0187824c test_proxy_model_resolution_with_custom_names_documentation 2025-07-23 13:13:46 -07:00
Ishaan Jaff 79a0841719 test_router_content_policy_fallbacks 2025-07-23 13:04:28 -07:00
Ishaan JaffandGitHub 99ad24c683 [Feat] LiteLLM x Cloudzero integration - Allow exporting spend to cloudzero (#12908) 
* add cz init

* add DataAnalyzer

* add CZRNGenerator -> LiteLLM resource generator

* add CloudZeroStreamer

* add CBFTransformer

* add cloudzero DB connection

* add config

* add cli.py

* cleanup

* add CloudZeroLogger

* add CloudZeroLogger

* update CloudZeroLogger

* add cloudzero_router to litellm proxy

* add CloudZeroInitRequest

* add cloudzero spend endpoints

* simple dry run endpoint

* refactor dir structure

* add well types CBFRecord

* TestCBFTransformer

* TestCloudZeroStreamer

* ruff fix

* add polars pip + docker requirements.txt

* _group_by_date

* update code qa check

* docs for CZ params
 2025-07-23 13:00:58 -07:00
Krish DholakiaandGitHub c5c2a68510 Swagger - bump DOMPurify version + add back Support local swagger bundle (good for airgapped environments) + support local swagger on custom root paths (#12911) 
* fix(proxy_server.py): update swagger-ui-bundle.js + bring back swagger in airgapped environments

* fix(proxy_server.py): support local swagger on custom root path

enables on prem usage of litellm
 2025-07-23 11:22:56 -07:00
Ishaan Jaff 0d2e280e78 ci/cd new release 2025-07-23 09:04:26 -07:00
Ishaan Jaff 00fd020291 fix test 2025-07-23 09:04:09 -07:00
Krrish Dholakia 4277becdc2 docs(credentials.tsx): cleanup reference to dead link 2025-07-23 09:02:39 -07:00
Ishaan Jaff 60ed272ae0 ci/cd litellm 2025-07-23 07:15:33 -07:00
Ishaan Jaff 5616aaee6d docs pillar guard 2025-07-23 07:15:02 -07:00
Ishaan Jaff 477ca5c0f6 test_proxy_all_models 2025-07-23 07:11:35 -07:00
Krrish Dholakia 663dea37a8 docs: fix docs 2025-07-22 23:30:50 -07:00
Igal BoxermanandGitHub d118cc7098 feat: Add Pillar Security guardrail integration (#12791) 
* feat: Add Pillar Security guardrail integration

Implements comprehensive LLM security guardrails using Pillar Security API with support for prompt injection detection, PII/secret detection, content moderation, and multi-mode execution (pre_call, during_call, post_call). Includes complete documentation, testing, and configurable actions on flagged content.

* fix: Resolve MyPy type error in Pillar guardrail config

Restructure PillarGuardrailConfigModel to properly inherit from GuardrailConfigModel[T]
and resolve return type compatibility issue in get_config_model method.

* fix: Resolve MyPy type error in Pillar guardrail config

Restructure PillarGuardrailConfigModel to properly inherit from GuardrailConfigModel[T]
and resolve return type compatibility issue in get_config_model method.

* fix docs

* fix docs

* improved docs

* fix examples, READY
 2025-07-22 23:17:33 -07:00
Murad KhafizovandGitHub a6ddf5c744 feat: extended /v1/models endpoint, now it returns with fallbacks on demand (#12811) 
* Extended `/v1/model` endpoint to support fallbacks

* unit tests reworked

* linting fixes

* fix lining error

* fix linting
 2025-07-22 23:16:46 -07:00
Stuart GeigerandGitHub 37c626a9a0 rm retired anthropic models from model_prices_and_context_window.json (#12864) 
rm claude-2, claude-2.1, claude-3-sonnet-20240229
 2025-07-22 23:14:37 -07:00
Jugal D. BhattandGitHub 03d5102044 [Add health check] add architecture diagram (#12879) 
* add architecture diagram

* add image path

* add new image
 2025-07-22 23:12:19 -07:00
Jugal D. BhattandGitHub 405eb7f243 [Docs ]Litellm mcp access group doc (#12883) 
* add example for config

* fix typo
 2025-07-22 23:11:53 -07:00
Krish DholakiaandGitHub d26f74ade5 Request Headers - support x-litellm-num-retries + Usage - support usage by model group (#12890) 
* feat(litellm_pre_call_utils.py): add num_retries to litellm data for backend call

allow user to pass in num retries via request headers

* test(test_litellm_pre_call_utils.py): add unit test

* docs(request_headers.md): document new request header

* fix(common_daily_activity.py): show spend breakdown by model group

Partial fix for https://github.com/BerriAI/litellm/issues/12887

* feat(new_usage.tsx): new tab switcher for viewing usage by model group vs. received model

Closes https://github.com/BerriAI/litellm/issues/12887
 2025-07-22 23:09:36 -07:00
Ishaan Jaff a0846c3f31 ci/cd new release 2025-07-22 21:08:02 -07:00
Ishaan Jaff 642cfa26b0 remove deprecated 2025-07-22 20:59:34 -07:00
Ishaan Jaff 3a76b2762f ci/cd new release 2025-07-22 20:41:49 -07:00
Ishaan Jaff 5741cdae0f fix openrouter/qwen/qwen-vl-plus 2025-07-22 20:37:57 -07:00
Krish DholakiaandGitHub 8cd6c25e1e Fix async retryer on .acompletion() + forward clientside headers - filter out content-type from clientside request (causes llm api call to hang) (#12886) 
* fix(main.py): fix async retryer

Fixes https://github.com/BerriAI/litellm/issues/12830

* fix(forward_clientside_headers_by_model_group.py): filter out 'content-type' from forwardable headers

clientside content-type != proxy content type, can cause requests to hang
 2025-07-22 19:50:05 -07:00
Ishaan JaffandGitHub b41ce5c92f [Feat] - Track cost + add tags for health checks done by LiteLLM Proxy (#12880) 
* refactor to use add_user_api_key_auth_to_request_metadata

* add get_litellm_internal_health_check_user_api_key_auth

* add get_litellm_internal_health_check_user_api_key_auth

* add _update_model_params_with_health_check_tracking_information

* add HealthCheckHelpers

* refactor to use clean helpers

* test_update_model_params_with_health_check_tracking_information

* test_get_litellm_internal_health_check_user_api_key_auth

* test_add_user_api_key_auth_to_request_metadata

* fix _update_model_params_with_health_check_tracking_information
 2025-07-22 18:45:57 -07:00
Ishaan Jaff c21dc46a33 fix morph api tests 2025-07-22 18:44:44 -07:00
Ishaan Jaff bd8b21e700 fix mypy linting 2025-07-22 18:38:54 -07:00
Ishaan Jaff d93e85282e test_bad_database_url 2025-07-22 18:37:51 -07:00
Ishaan Jaff bf300f8ca7 Revert "Litellm dev 07 21 2025 p1 (#12848)" 
This reverts commit e4e10aa4ed.
 2025-07-22 18:28:36 -07:00
Ishaan Jaff f0a8abb911 fix cost_calculator 2025-07-22 18:16:53 -07:00