tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-08-06 06:24:16 +00:00
Code Issues Packages Projects Releases Wiki Activity
24,921 Commits 2,072 Branches 1,425 Tags
ef9d1ddc40cdbff2cd61ca1dfad71ec8533ea6bc
Commit Graph
235 Commits
Author SHA1 Message Date
Ishaan JaffGitHubCopilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
 		555efd60b5 [Security] Fix: Ensure .info() logs are not used for request/responses + Add code QA check for possible violations (#14386) 
* test: info log

* fixes for debug

* add info_log_check

* fix: debug level

* fix: debug level

* fix: use .debug

* fix info statements

* fix info log check

* fix: fix emit_standard_logging_payload

* fix RequestBody

* Potential fix for code scanning alert no. 3339: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* fix check

* fix code qa check

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
 2025-09-09 13:55:56 -07:00
mubashir1osmani 2626123aca added litellm_spend_metric 2025-09-01 11:33:34 -04:00
mubashir1osmani 5649c0f0b9 Revert "feat: add persistence and management for guardrails on virtual keys" 
This reverts commit 6cd5afa8b1.
 2025-09-01 10:45:46 -04:00
mubashir1osmani a6790ac158 Revert "feat: add UI support for displaying and managing guardrails on virtual keys" 
This reverts commit 83654c5608.
 2025-09-01 10:45:44 -04:00
mubashir1osmani 31cfe1fe0a Revert "fix: import verbose_proxy_logger to resolve undefined name errors" 
This reverts commit a8f2ebed9c.
 2025-09-01 10:45:38 -04:00
mubashir1osmani 8de65e8448 fix: added prometheus missing metrics 2025-09-01 10:36:51 -04:00
mubashir1osmani af2f447698 added prometheus metrics 2025-09-01 10:28:48 -04:00
mubashir1osmani (aider) a8f2ebed9c fix: import verbose_proxy_logger to resolve undefined name errors 2025-09-01 01:17:46 -04:00
mubashir1osmani (aider) 83654c5608 feat: add UI support for displaying and managing guardrails on virtual keys 2025-09-01 01:17:08 -04:00
mubashir1osmani (aider) 6cd5afa8b1 feat: add persistence and management for guardrails on virtual keys 2025-09-01 00:56:22 -04:00
Krrish Dholakia dba78046b9 fix(internal_user_endpoints.py): show remaining users when free SSO users is enabled 2025-08-20 22:26:01 -07:00
Ishaan JaffandGitHub 4d941c914e [Feat] Responses API Session Handling - Multi media support (#13347) 
* rename ResponsesSessionHandler

* use ResponsesSessionHandler

* test session handler

* refactor ResponsesSessionHandler

* fix get_proxy_server_request_from_spend_log

* use constant for LITELLM_TRUNCATED_PAYLOAD_FIELD

* add _should_check_cold_storage_for_full_payload

* add get_class_type_for_custom_logger_name

* get_active_custom_logger_for_callback_name

* add get_proxy_server_request_from_cold_storage to CustomLogger

* add ColdStorageHandler

* start using cold storage integration

* add get_proxy_server_request_from_cold_storage

* fixes from manual testing

* s3 v2 fix getting region name

* ChatCompletionImageUrlObject

* use _get_configured_cold_storage_custom_logger

* fixes for _should_check_cold_storage_for_full_payload

* fix _download_object_from_s3

* test_s3_v2_with_cold_storage

* add cold_storage_object_key to StandardLoggingMetadata

* use get_proxy_server_request_from_cold_storage_with_object_key

* add cold_storage_object_key to SpendLogsMetadata

* add cold_storage_object_key

* get_proxy_server_request_from_cold_storage_with_object_key

* use get_proxy_server_request_from_cold_storage_with_object_key

* test responses API

* add get_proxy_server_request_from_cold_storage_with_object_key

* session handler fixes

* test session handler

* fix ruff checks

* _download_object_from_s3

* cleanup

* test

* lint fix

* test_e2e_cold_storage_successful_retrieval

* test_e2e_generate_cold_storage_object_key_successful

* test_async_gcs_pub_sub_v1

* test fix

* test fix

* test fix

* test_standard_logging_metadata_has_cold_storage_object_key_field

* test_sanitize_request_body_for_spend_logs_payload_basic

* test_transform_input_image_item_to_image_item_with_image_data
 2025-08-07 10:59:53 -07:00
Ishaan Jaff da40b5ec47 bump litellm enterprise 2025-08-05 17:50:40 -07:00
Ishaan Jaff 0308a835b8 bump: version 0.1.17 → 0.1.18 2025-08-05 17:47:16 -07:00
Ishaan JaffandGitHub dab8ba03e3 [Feat] - When using custom tags on prometheus allow using wildcard patterns (#13316) 
* _tag_matches_wildcard_configured_pattern

* test_get_custom_labels_from_tags_wildcard_patterns

* docs Custom Tags

* docs how custom tags work

* fix
 2025-08-05 17:46:13 -07:00
Krish DholakiaandGitHub eb49f987de Ensure disable_llm_api_endpoints works + Add wildcard model support for 'team-byok' model (#13278) 
* fix(route_checks.py): ensure disable llm api endpoints is correctly set

* fix(route_checks.py): raise httpexception

raise expected exceptions

* fix(router.py): handle team only wildcard models

fixes issue where team only wildcard models were not considered during auth checks

* fix(router.py): handle team only wildcard models

fixes issue where team only wildcard models were not considered during auth checks
 2025-08-04 23:19:51 -07:00
Ishaan Jaff ad26cba8b6 add litellm-enterprise==0.1.17 2025-08-04 17:26:50 -07:00
Ishaan JaffandGitHub ba1882fdd5 [Bug Fix] Prometheus - fix for litellm_input_tokens_metric, litellm_output_tokens_metric - Note this updates the metric name (#13271) 
* fixes for litellm_tokens_metric

* test_prometheus_token_metrics_with_prometheus_config
 2025-08-04 17:22:21 -07:00
Jugal D. BhattandGitHub 900c7f45c0 [MCP Gateway] Litellm mcp pre and during guardrails (#13188) 
* add guardrail support

* add guardrail support

* guardrails for MCP

* added changes

* add mcp guardrails

* added test

* add ui

* fix guardrail form

* working with cursor

* remvoe print

* fix mcp servertests

* fix mypy and remove console logs

* fix mypy and remove console logs

* fix mypy tests
 2025-08-01 20:02:25 -07:00
Krish DholakiaandGitHub b7c5073d28 Custom Auth - bubble up custom exceptions (#13093) 
* fix(enterprise/litellm_enterprise/proxy/auth/user_api_key_auth.py): bubble up exception if type is ProxyException

* docs(custom_auth.md): doc on bubbling up custom exceptions
 2025-07-29 16:28:25 -07:00
Krish DholakiaandGitHub d6423d913d build: update pip package (#12998) 2025-07-25 16:23:53 -07:00
Krish DholakiaandGitHub 550211ba47 Prometheus - tags, fix '[tag]="false"' when tag is set (#12916) 
* fix(prometheus.py): sanitize tag-based labels to handle colons (:) and spaces ( )

* fix(prometheus.py): working tag based metrics

* fix(prometheus.py): emit request tags on post call success hook

* fix(prometheus.py): add user agent tags on request failure

* fix(prometheus.py): add request tags to deployment failure metric

s
 2025-07-24 16:39:13 -07:00
Ishaan Jaff bf300f8ca7 Revert "Litellm dev 07 21 2025 p1 (#12848)" 
This reverts commit e4e10aa4ed.
 2025-07-22 18:28:36 -07:00
Krish DholakiaandGitHub e4e10aa4ed Litellm dev 07 21 2025 p1 (#12848) 
* fix(main.py): fix async retryer

Fixes https://github.com/BerriAI/litellm/issues/12830

* fix(forward_clientside_headers_by_model_group.py): filter out 'content-type' from forwardable headers

clientside content-type != proxy content type, can cause requests to hang

* test(tests/): update tests
 2025-07-21 22:09:39 -07:00
Krish DholakiaandGitHub 00677afefe Litellm batch cost tracking debug (#12782) 
* feat(proxy_server.py): support batch polling interval

allows admin to control batch polling interval (default is 3600s)

easier debugging

* fix(proxy_settings_endpoint.py): ensure value is actually set before updating env var
 2025-07-21 20:17:56 -07:00
Ishaan Jaff 96f7eb6f78 bump litellm enterprise version 2025-07-19 10:12:33 -07:00
Ishaan JaffandGitHub 06574a72b5 [Feat] Backend - Add support for disabling callbacks in request body (#12762) 
* allow using standard_callback_dynamic_params to disable callbacks

* fix is_callback_disabled_dynamically

* test_callback_disabled_via_request_body_multiple
 2025-07-19 10:10:30 -07:00
Ishaan JaffandGitHub 99e2ea081d [Feat] UI - Allow clicking into Vector Stores (#12741) 
* Add View Vector Store

* add /info for vector store

* fix updated_at
 2025-07-18 14:28:57 -07:00
Jugal D. BhattGitHubKrrish Dholakia
 		a46b9d376f [Prometheus] Move Prometheus to enterprise folder (#12659) 
* fix tools fetch for keys

* add promethues to enterprise

* remove old prom

* remove old prom

* fix tests

* safe imports

* add if

* fix enterprise test

* rename imports

* added label import

* added label import

* move tests to enterprise

* fix tests

* add log

* build: update versions

---------

Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>
 2025-07-18 11:54:47 -07:00
Ishaan JaffandGitHub 3051a9c68a [Bug Fix] QA - Use PG Vector Vector Store with LiteLLM (#12716) 
* store in generic litellm params

* handle storing litellm_params_json

* add litellm_params

* update file loc

* fix startup issue

* ui fix litellm_params

* fix typing

* fix adding PG Vector

* use litellm params

* fix transform_search_vector_store_request

* fix transform

* fix URL

* test fix

* test_pg_vector_search_request_construction
 2025-07-18 08:41:18 -07:00
Ishaan JaffandGitHub 95f7290e44 [Feat] Proxy - New LLM API Routes /v1/vector_stores and /v1/vector_stores/vs_abc123/search (#12699) 
* add Search + Create vector stores

* add new vector_stores routes

* add avector_store_search routes

* docs vector stores

* docs vector stores

* Create Vector Store

* fix - refactor

* add vector_store_router

* fix imports

* fix ruff
 2025-07-17 11:19:25 -07:00
yeahyungandGitHub 782969bb05 (#11794) use upsert for managed object table rather than create to avoid UniqueViolationError (#11795) 
* (#11794) use upsert for managed object table rather than create to avoid UniqueViolationError

* (#11794) use upsert for managed object table rather than create to avoid UniqueViolationError
 2025-07-15 20:20:01 -07:00
Krish DholakiaandGitHub 7c392475e6 Control Plane + Data Plane support (#12601) 
* feat(route_checks.py): allow admin to disable proxy management endpoints on instance

useful for preventing multiple instances from doing admin actions

* docs(scaling_multiple_instances.md): add architecture doc on scaling multiple litellm instances

provide guidance on scaling proxy

* docs(scaling_multiple_instances.md): add doc on scaling across multiple regions for litellm

* fix(route_checks.py): allow disabling llm api endpoints on an instance

allows pure admin instance to exist

* refactor(enterprise/route_checks.py): refactor env var checks

* refactor: finish refactoring

* docs(control_plane_and_data_plane.md): refactor docs

* test: update tests
 2025-07-14 21:31:56 -07:00
Ishaan JaffandGitHub 57b0b4edf3 [Bug fix] [Bug]: Verbose log is enabled by default (#12596) 
* test find_set_verbose_assignments

* fix set verbose

* test set verbose

* fix unused import
 2025-07-14 20:06:16 -07:00
Ishaan Jaff 08ce596c55 bump litellm enterprise pkg 2025-07-12 18:31:52 -07:00
Krrish Dholakia 8b4673fe84 build(ui/): UI - Public Model Hub v2 (#12532) 
Closes https://github.com/BerriAI/litellm/pull/12532
 2025-07-11 22:24:31 -07:00
Krish DholakiaandGitHub 569ddde9fe [Enterprise] Support tag based mode for guardrails (#12508) 
* feat(custom_guardrail.py): allow admin to run guardrails on specific modes based on request tags

Enables pre-call checks for openwebui but logging for claude code guardrails

* refactor(custom_guardrail.py): move tag based mode to enterprise

* test(test_custom_guardrail.py): update test

* feat(quick_start.md): add doc on tag-based guardrail mode

* docs(quick_start.md): cleanup doc

* docs: cleanup

* docs(openweb_ui.md): add tutorial to doc

* fix(custom_guardrail.py): ensure user is premium user

* test: update test
 2025-07-10 22:27:30 -07:00
Ishaan Jaff a37dacb770 bump to litellm-enterprise==0.1.12 2025-07-09 18:35:28 -07:00
Ishaan JaffandGitHub 17ea770ab5 [Feat] SSO - Allow users to run a custom sso login handler (#12465) 
* fix return_ui_sso_redirect_response_after_sign_in

* add CustomSSOLoginHandler

* add handle_custom_ui_sso_sign_in

* fix handle_custom_ui_sso_sign_in

* fixes for /sso/key/generate

* add test_handle_custom_ui_sso_sign_in_success

* fixed for loading config

* add example custom handler

* test_custom_ui_sso_sign_in_handler_config_loading

* docs - custom sso

* move to enterprise folder

* EnterpriseCustomSSOHandler

* handle_custom_ui_sso_sign_in

* docs custom sso

* docs sso

* docs custom sso

* get_redirect_response_from_openid

* fix mock tests
 2025-07-09 16:36:30 -07:00
Jugal D. BhattandGitHub d55712b783 Add MCP url masking on frontend (#12247) 
* add url masking on frontend

* fix test

* add signature
 2025-07-02 16:32:26 -07:00
Ishaan Jaff 1ce71d6d61 bump litellm-enterprise = {version = "0.1.11", optional = true} 2025-07-02 15:00:28 -07:00
Jugal Bhatt e53eb6eb37 change logger 2025-07-01 16:28:53 -07:00
Jugal Bhatt 7e6d27e6c3 change logger 2025-07-01 16:28:34 -07:00
Jugal Bhatt fefc5fc33a add warning message for non premium users 2025-07-01 16:27:31 -07:00
Jugal Bhatt b63d64dcf5 add subject and signature tests 2025-07-01 15:15:40 -07:00
Jugal Bhatt 69670001de email_templates 2025-07-01 14:34:15 -07:00
Ishaan Jaff a021e7cb9d bump poetry 2025-06-28 21:16:52 -07:00
Ishaan JaffandGitHub cfc07f361b [Bug Fix] Invite links email should contain the correct invite id (#12130) 
* use common helper create_invitation_for_user

* use common util in proxy

* fix create_invitation_for_user

* refactor base email

* test_get_invitation_link_creates_new_when_none_exist

* fix code QA checks
 2025-06-28 09:19:32 -07:00
Krish DholakiaandGitHub d895ae827d Litellm batch api background cost calc (#12125) 
* feat(check_batch_cost.py): emit spend log on successful request

ensures cost tracked for batch requests

* feat(proxy_server.py): add background job to poll completed batch jobs

used for calculating cost for batch jobs

* fix(proxy_server.py): run batch cost tracking job every hour

batch jobs take time to complete, no need to run every few seconds

* feat(proxy_server.py): run batch cost tracking job every hour
 2025-06-27 21:46:25 -07:00
Krrish Dholakia c3857e60f2 Store batch output file id in DB + Store batch file status in DB + (experimental) BATCH API COST TRACKING 2025-06-25 22:41:22 -07:00