Commit Graph
250 Commits
Author SHA1 Message Date
Krrish Dholakia b4c4776f56 fix: fix linting errors 2025-10-04 14:47:13 -07:00
Krrish Dholakia a1a0e99638 fix(prometheus.py): working e2e calls w/ userapikeymetadata 2025-09-30 21:23:25 -07:00
Krrish Dholakia d6800ee706 feat(prometheus.py): initial working commit of passing team/key metadata as prometheus metrics
Closes LIT-1006
2025-09-30 17:02:40 -07:00
Krrish Dholakia 6ca7752381 fix(prometheus.py): don't require metadata labels to be set for all requests
add a default value if metadata label not set
2025-09-30 15:46:30 -07:00
Krrish Dholakia c2e2b699e6 fix: fix linting error 2025-09-27 12:43:06 -07:00
Alexsander HamirandGitHub eaa04cd8ce fix: use fastuuid helper (#14903)
* fix: use fastuuid helper across the codebase

First batch of changes, simple drop in replacement.

* second batch of changes

* fixed: script mistake on helper file
2025-09-25 15:47:01 -07:00
Ishaan JaffGitHubCopilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
b9ffa98c55 [Feat] Proxy CLI: Create a python method to login using litellm proxy (#14782)
* fix: cli auth with SSO okta

* fix: add LITTELM_CLI_SERVICE_ACCOUNT_NAME

* fix: get_litellm_cli_user_api_key_auth

* use existing_key CLI

* fix: use existing key

* test auth commands

* test_cli_sso_callback_regenerate_vs_create_flow

* feat: add CLI Token Utilities

* fix: get_stored_api_key

* move file

* fix: get_valid_models

* fix config.yaml

* TestCLITokenUtils

* TestGetValidModelsWithCLI

* fix: tie user id to keys created through CLI

* fix: add teams interface to CLI

* add /keys/update to the list client commands

* fix /sso/cli/poll to return the user_id

* fix: working TeamsManagementClient

* fix CLI Login command

* fixes for auth

* Potential fix for code scanning alert no. 3400: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* ruff fix

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-09-22 21:28:38 -07:00
Ishaan Jaffer c656982f18 test fix 2025-09-20 16:55:23 -07:00
Krish DholakiaandGitHub 270d612029 Merge branch 'main' into litellm_dev_09_10_2025_p1 2025-09-19 22:01:57 -07:00
Krrish Dholakia 565eeca92a build(pyproject.toml): bump versions 2025-09-18 19:24:34 -07:00
Krrish Dholakia 4d87199266 fix(prometheus.py): fix spend metrics 2025-09-18 19:12:07 -07:00
8b804303ed fix: ci/cd tests + lint errors (#14646)
* fix: lint errors + tests

* fixed ci tests

* fixed tests

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
2025-09-17 17:06:43 -07:00
Krrish Dholakia fc2d1f2646 fix: fix import errors 2025-09-14 09:32:21 -07:00
Krrish Dholakia 459e66a9cd fix: fix test 2025-09-13 13:38:35 -07:00
Krrish Dholakia a9fddbf4ad fix(prometheus.py): make prometheus work for multiple workers 2025-09-10 16:20:17 -07:00
Ishaan JaffGitHubCopilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
555efd60b5 [Security] Fix: Ensure .info() logs are not used for request/responses + Add code QA check for possible violations (#14386)
* test: info log

* fixes for debug

* add info_log_check

* fix: debug level

* fix: debug level

* fix: use .debug

* fix info statements

* fix info log check

* fix: fix emit_standard_logging_payload

* fix RequestBody

* Potential fix for code scanning alert no. 3339: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* fix check

* fix code qa check

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-09-09 13:55:56 -07:00
mubashir1osmani 2626123aca added litellm_spend_metric 2025-09-01 11:33:34 -04:00
mubashir1osmani 5649c0f0b9 Revert "feat: add persistence and management for guardrails on virtual keys"
This reverts commit 6cd5afa8b1.
2025-09-01 10:45:46 -04:00
mubashir1osmani a6790ac158 Revert "feat: add UI support for displaying and managing guardrails on virtual keys"
This reverts commit 83654c5608.
2025-09-01 10:45:44 -04:00
mubashir1osmani 31cfe1fe0a Revert "fix: import verbose_proxy_logger to resolve undefined name errors"
This reverts commit a8f2ebed9c.
2025-09-01 10:45:38 -04:00
mubashir1osmani 8de65e8448 fix: added prometheus missing metrics 2025-09-01 10:36:51 -04:00
mubashir1osmani af2f447698 added prometheus metrics 2025-09-01 10:28:48 -04:00
mubashir1osmani (aider) a8f2ebed9c fix: import verbose_proxy_logger to resolve undefined name errors 2025-09-01 01:17:46 -04:00
mubashir1osmani (aider) 83654c5608 feat: add UI support for displaying and managing guardrails on virtual keys 2025-09-01 01:17:08 -04:00
mubashir1osmani (aider) 6cd5afa8b1 feat: add persistence and management for guardrails on virtual keys 2025-09-01 00:56:22 -04:00
Krrish Dholakia dba78046b9 fix(internal_user_endpoints.py): show remaining users when free SSO users is enabled 2025-08-20 22:26:01 -07:00
Ishaan JaffandGitHub 4d941c914e [Feat] Responses API Session Handling - Multi media support (#13347)
* rename ResponsesSessionHandler

* use ResponsesSessionHandler

* test session handler

* refactor ResponsesSessionHandler

* fix get_proxy_server_request_from_spend_log

* use constant for LITELLM_TRUNCATED_PAYLOAD_FIELD

* add _should_check_cold_storage_for_full_payload

* add get_class_type_for_custom_logger_name

* get_active_custom_logger_for_callback_name

* add get_proxy_server_request_from_cold_storage to CustomLogger

* add ColdStorageHandler

* start using cold storage integration

* add get_proxy_server_request_from_cold_storage

* fixes from manual testing

* s3 v2 fix getting region name

* ChatCompletionImageUrlObject

* use _get_configured_cold_storage_custom_logger

* fixes for _should_check_cold_storage_for_full_payload

* fix _download_object_from_s3

* test_s3_v2_with_cold_storage

* add cold_storage_object_key to StandardLoggingMetadata

* use get_proxy_server_request_from_cold_storage_with_object_key

* add cold_storage_object_key to SpendLogsMetadata

* add cold_storage_object_key

* get_proxy_server_request_from_cold_storage_with_object_key

* use get_proxy_server_request_from_cold_storage_with_object_key

* test responses API

* add get_proxy_server_request_from_cold_storage_with_object_key

* session handler fixes

* test session handler

* fix ruff checks

* _download_object_from_s3

* cleanup

* test

* lint fix

* test_e2e_cold_storage_successful_retrieval

* test_e2e_generate_cold_storage_object_key_successful

* test_async_gcs_pub_sub_v1

* test fix

* test fix

* test fix

* test_standard_logging_metadata_has_cold_storage_object_key_field

* test_sanitize_request_body_for_spend_logs_payload_basic

* test_transform_input_image_item_to_image_item_with_image_data
2025-08-07 10:59:53 -07:00
Ishaan Jaff da40b5ec47 bump litellm enterprise 2025-08-05 17:50:40 -07:00
Ishaan Jaff 0308a835b8 bump: version 0.1.17 → 0.1.18 2025-08-05 17:47:16 -07:00
Ishaan JaffandGitHub dab8ba03e3 [Feat] - When using custom tags on prometheus allow using wildcard patterns (#13316)
* _tag_matches_wildcard_configured_pattern

* test_get_custom_labels_from_tags_wildcard_patterns

* docs Custom Tags

* docs how custom tags work

* fix
2025-08-05 17:46:13 -07:00
Krish DholakiaandGitHub eb49f987de Ensure disable_llm_api_endpoints works + Add wildcard model support for 'team-byok' model (#13278)
* fix(route_checks.py): ensure disable llm api endpoints is correctly set

* fix(route_checks.py): raise httpexception

raise expected exceptions

* fix(router.py): handle team only wildcard models

fixes issue where team only wildcard models were not considered during auth checks

* fix(router.py): handle team only wildcard models

fixes issue where team only wildcard models were not considered during auth checks
2025-08-04 23:19:51 -07:00
Ishaan Jaff ad26cba8b6 add litellm-enterprise==0.1.17 2025-08-04 17:26:50 -07:00
Ishaan JaffandGitHub ba1882fdd5 [Bug Fix] Prometheus - fix for litellm_input_tokens_metric, litellm_output_tokens_metric - Note this updates the metric name (#13271)
* fixes for litellm_tokens_metric

* test_prometheus_token_metrics_with_prometheus_config
2025-08-04 17:22:21 -07:00
Jugal D. BhattandGitHub 900c7f45c0 [MCP Gateway] Litellm mcp pre and during guardrails (#13188)
* add guardrail support

* add guardrail support

* guardrails for MCP

* added changes

* add mcp guardrails

* added test

* add ui

* fix guardrail form

* working with cursor

* remvoe print

* fix mcp servertests

* fix mypy and remove console logs

* fix mypy and remove console logs

* fix mypy tests
2025-08-01 20:02:25 -07:00
Krish DholakiaandGitHub b7c5073d28 Custom Auth - bubble up custom exceptions (#13093)
* fix(enterprise/litellm_enterprise/proxy/auth/user_api_key_auth.py): bubble up exception if type is ProxyException

* docs(custom_auth.md): doc on bubbling up custom exceptions
2025-07-29 16:28:25 -07:00
Krish DholakiaandGitHub d6423d913d build: update pip package (#12998) 2025-07-25 16:23:53 -07:00
Krish DholakiaandGitHub 550211ba47 Prometheus - tags, fix '[tag]="false"' when tag is set (#12916)
* fix(prometheus.py): sanitize tag-based labels to handle colons (:) and spaces ( )

* fix(prometheus.py): working tag based metrics

* fix(prometheus.py): emit request tags on post call success hook

* fix(prometheus.py): add user agent tags on request failure

* fix(prometheus.py): add request tags to deployment failure metric

s
2025-07-24 16:39:13 -07:00
Ishaan Jaff bf300f8ca7 Revert "Litellm dev 07 21 2025 p1 (#12848)"
This reverts commit e4e10aa4ed.
2025-07-22 18:28:36 -07:00
Krish DholakiaandGitHub e4e10aa4ed Litellm dev 07 21 2025 p1 (#12848)
* fix(main.py): fix async retryer

Fixes https://github.com/BerriAI/litellm/issues/12830

* fix(forward_clientside_headers_by_model_group.py): filter out 'content-type' from forwardable headers

clientside content-type != proxy content type, can cause requests to hang

* test(tests/): update tests
2025-07-21 22:09:39 -07:00
Krish DholakiaandGitHub 00677afefe Litellm batch cost tracking debug (#12782)
* feat(proxy_server.py): support batch polling interval

allows admin to control batch polling interval (default is 3600s)

easier debugging

* fix(proxy_settings_endpoint.py): ensure value is actually set before updating env var
2025-07-21 20:17:56 -07:00
Ishaan Jaff 96f7eb6f78 bump litellm enterprise version 2025-07-19 10:12:33 -07:00
Ishaan JaffandGitHub 06574a72b5 [Feat] Backend - Add support for disabling callbacks in request body (#12762)
* allow using standard_callback_dynamic_params to disable callbacks

* fix is_callback_disabled_dynamically

* test_callback_disabled_via_request_body_multiple
2025-07-19 10:10:30 -07:00
Ishaan JaffandGitHub 99e2ea081d [Feat] UI - Allow clicking into Vector Stores (#12741)
* Add View Vector Store

* add /info for vector store

* fix updated_at
2025-07-18 14:28:57 -07:00
a46b9d376f [Prometheus] Move Prometheus to enterprise folder (#12659)
* fix tools fetch for keys

* add promethues to enterprise

* remove old prom

* remove old prom

* fix tests

* safe imports

* add if

* fix enterprise test

* rename imports

* added label import

* added label import

* move tests to enterprise

* fix tests

* add log

* build: update versions

---------

Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>
2025-07-18 11:54:47 -07:00
Ishaan JaffandGitHub 3051a9c68a [Bug Fix] QA - Use PG Vector Vector Store with LiteLLM (#12716)
* store in generic litellm params

* handle storing litellm_params_json

* add litellm_params

* update file loc

* fix startup issue

* ui fix litellm_params

* fix typing

* fix adding PG Vector

* use litellm params

* fix transform_search_vector_store_request

* fix transform

* fix URL

* test fix

* test_pg_vector_search_request_construction
2025-07-18 08:41:18 -07:00
Ishaan JaffandGitHub 95f7290e44 [Feat] Proxy - New LLM API Routes /v1/vector_stores and /v1/vector_stores/vs_abc123/search (#12699)
* add Search + Create vector stores

* add new vector_stores routes

* add avector_store_search routes

* docs vector stores

* docs vector stores

* Create Vector Store

* fix - refactor

* add vector_store_router

* fix imports

* fix ruff
2025-07-17 11:19:25 -07:00
yeahyungandGitHub 782969bb05 (#11794) use upsert for managed object table rather than create to avoid UniqueViolationError (#11795)
* (#11794) use upsert for managed object table rather than create to avoid UniqueViolationError

* (#11794) use upsert for managed object table rather than create to avoid UniqueViolationError
2025-07-15 20:20:01 -07:00
Krish DholakiaandGitHub 7c392475e6 Control Plane + Data Plane support (#12601)
* feat(route_checks.py): allow admin to disable proxy management endpoints on instance

useful for preventing multiple instances from doing admin actions

* docs(scaling_multiple_instances.md): add architecture doc on scaling multiple litellm instances

provide guidance on scaling proxy

* docs(scaling_multiple_instances.md): add doc on scaling across multiple regions for litellm

* fix(route_checks.py): allow disabling llm api endpoints on an instance

allows pure admin instance to exist

* refactor(enterprise/route_checks.py): refactor env var checks

* refactor: finish refactoring

* docs(control_plane_and_data_plane.md): refactor docs

* test: update tests
2025-07-14 21:31:56 -07:00
Ishaan JaffandGitHub 57b0b4edf3 [Bug fix] [Bug]: Verbose log is enabled by default (#12596)
* test find_set_verbose_assignments

* fix set verbose

* test set verbose

* fix unused import
2025-07-14 20:06:16 -07:00
Ishaan Jaff 08ce596c55 bump litellm enterprise pkg 2025-07-12 18:31:52 -07:00