Krish Dholakia and GitHub
8aa4beff5f
fix(internal_user_endpoints.py): delete member from team table on /user/delete ( #12926 )
...
removes user from team when user is deleted
2025-07-24 16:42:28 -07:00
Krish Dholakia and GitHub
1a57875d24
Proxy - specify key_type - allows specifying if key can call LLM API routes vs. Management routes only ( #12909 )
...
* feat(key_management_endpoints.py): Support new 'key_type' field
allow user to specify if key should be 'management' or 'llm api' key
Security fix
* test(test_route_checks.py): add unit tests
* fix(create_key_button.tsx): add ui component to select key type
allows specifying if key can call llm api vs. management routes
* feat(create_key_button.tsx): add specifying key type to ui
* fix(route_checks.py): add sensitive data masker for user id on not allowed error message
prevent leaking sensitive information
2025-07-24 16:40:40 -07:00
Krish Dholakia and GitHub
550211ba47
Prometheus - tags, fix '[tag]="false"' when tag is set ( #12916 )
...
* fix(prometheus.py): sanitize tag-based labels to handle colons (:) and spaces ( )
* fix(prometheus.py): working tag based metrics
* fix(prometheus.py): emit request tags on post call success hook
* fix(prometheus.py): add user agent tags on request failure
* fix(prometheus.py): add request tags to deployment failure metric
s
2025-07-24 16:39:13 -07:00
Ishaan Jaff
0a07698d9d
docs update
2025-07-24 11:26:30 -07:00
Cole McIntosh and GitHub
8c99c71f38
Merge pull request #12939 from superpoussin22/correct_doc
...
Update control_plane_and_data_plane.md
2025-07-24 09:17:01 -06:00
superpoussin22 and GitHub
ca9712c587
Update control_plane_and_data_plane.md
2025-07-24 16:15:55 +02:00
Ishaan Jaff and GitHub
3e529fae7f
[UI] Allow setting up CloudZero Usage through LiteLLM UI ( #12923 )
...
* refactor cz settings
* allow viewing/setting CZ settings
* working button
* fixes for export button
* nicer button
* fix className
2025-07-23 21:17:12 -07:00
Apoorv Sharma and GitHub
29475ab4b0
Add GA version of gemini 2.5 flash lite for both vertex and gemini ( #12920 )
...
This PR adds the config for gemini 2.5 flash lite which is out of preview now
2025-07-23 18:47:46 -07:00
Ishaan Jaff and GitHub
e3b752d3dc
[Feat] LiteLLM CloudZero Integration updates - using LiteLLM_SpendLogs Table ( #12922 )
...
* use spend logs in transform.py
* fixes for using spend logs
* get_usage_data
* update get_usage_data_for_hour
* fix CloudZeroLogger
* fixes for cloudzero_dry_run_export
* working DB read
* fix _aggregate_to_hourly
* update cloudzero endpoints
* fix cloudzero_export
* fixes for CBF transformer
* use init_background_job on saving cz settings
* add is_cloudzero_setup_in_db
* init cz on startup
* fixes for CZ transform
* fix ruff check
* fix LOC violation
2025-07-23 18:46:50 -07:00
Ishaan Jaff
99031bf8b6
ci/cd new release
2025-07-23 13:50:36 -07:00
Ishaan Jaff
461cd0c30a
test_completion_cost_deepseek
2025-07-23 13:16:12 -07:00
Ishaan Jaff
2d0187824c
test_proxy_model_resolution_with_custom_names_documentation
2025-07-23 13:13:46 -07:00
Ishaan Jaff
79a0841719
test_router_content_policy_fallbacks
2025-07-23 13:04:28 -07:00
Ishaan Jaff and GitHub
99ad24c683
[Feat] LiteLLM x Cloudzero integration - Allow exporting spend to cloudzero ( #12908 )
...
* add cz init
* add DataAnalyzer
* add CZRNGenerator -> LiteLLM resource generator
* add CloudZeroStreamer
* add CBFTransformer
* add cloudzero DB connection
* add config
* add cli.py
* cleanup
* add CloudZeroLogger
* add CloudZeroLogger
* update CloudZeroLogger
* add cloudzero_router to litellm proxy
* add CloudZeroInitRequest
* add cloudzero spend endpoints
* simple dry run endpoint
* refactor dir structure
* add well types CBFRecord
* TestCBFTransformer
* TestCloudZeroStreamer
* ruff fix
* add polars pip + docker requirements.txt
* _group_by_date
* update code qa check
* docs for CZ params
2025-07-23 13:00:58 -07:00
Krish Dholakia and GitHub
c5c2a68510
Swagger - bump DOMPurify version + add back Support local swagger bundle (good for airgapped environments) + support local swagger on custom root paths ( #12911 )
...
* fix(proxy_server.py): update swagger-ui-bundle.js + bring back swagger in airgapped environments
* fix(proxy_server.py): support local swagger on custom root path
enables on prem usage of litellm
2025-07-23 11:22:56 -07:00
Ishaan Jaff
0d2e280e78
ci/cd new release
2025-07-23 09:04:26 -07:00
Ishaan Jaff
00fd020291
fix test
2025-07-23 09:04:09 -07:00
Krrish Dholakia
4277becdc2
docs(credentials.tsx): cleanup reference to dead link
2025-07-23 09:02:39 -07:00
Ishaan Jaff
60ed272ae0
ci/cd litellm
2025-07-23 07:15:33 -07:00
Ishaan Jaff
5616aaee6d
docs pillar guard
2025-07-23 07:15:02 -07:00
Ishaan Jaff
477ca5c0f6
test_proxy_all_models
2025-07-23 07:11:35 -07:00
Krrish Dholakia
663dea37a8
docs: fix docs
2025-07-22 23:30:50 -07:00
Igal Boxerman and GitHub
d118cc7098
feat: Add Pillar Security guardrail integration ( #12791 )
...
* feat: Add Pillar Security guardrail integration
Implements comprehensive LLM security guardrails using Pillar Security API with support for prompt injection detection, PII/secret detection, content moderation, and multi-mode execution (pre_call, during_call, post_call). Includes complete documentation, testing, and configurable actions on flagged content.
* fix: Resolve MyPy type error in Pillar guardrail config
Restructure PillarGuardrailConfigModel to properly inherit from GuardrailConfigModel[T]
and resolve return type compatibility issue in get_config_model method.
* fix: Resolve MyPy type error in Pillar guardrail config
Restructure PillarGuardrailConfigModel to properly inherit from GuardrailConfigModel[T]
and resolve return type compatibility issue in get_config_model method.
* fix docs
* fix docs
* improved docs
* fix examples, READY
2025-07-22 23:17:33 -07:00
Murad Khafizov and GitHub
a6ddf5c744
feat: extended /v1/models endpoint, now it returns with fallbacks on demand ( #12811 )
...
* Extended `/v1/model` endpoint to support fallbacks
* unit tests reworked
* linting fixes
* fix lining error
* fix linting
2025-07-22 23:16:46 -07:00
Stuart Geiger and GitHub
37c626a9a0
rm retired anthropic models from model_prices_and_context_window.json ( #12864 )
...
rm claude-2, claude-2.1, claude-3-sonnet-20240229
2025-07-22 23:14:37 -07:00
Jugal D. Bhatt and GitHub
03d5102044
[Add health check] add architecture diagram ( #12879 )
...
* add architecture diagram
* add image path
* add new image
2025-07-22 23:12:19 -07:00
Jugal D. Bhatt and GitHub
405eb7f243
[Docs ]Litellm mcp access group doc ( #12883 )
...
* add example for config
* fix typo
2025-07-22 23:11:53 -07:00
Krish Dholakia and GitHub
d26f74ade5
Request Headers - support x-litellm-num-retries + Usage - support usage by model group ( #12890 )
...
* feat(litellm_pre_call_utils.py): add num_retries to litellm data for backend call
allow user to pass in num retries via request headers
* test(test_litellm_pre_call_utils.py): add unit test
* docs(request_headers.md): document new request header
* fix(common_daily_activity.py): show spend breakdown by model group
Partial fix for https://github.com/BerriAI/litellm/issues/12887
* feat(new_usage.tsx): new tab switcher for viewing usage by model group vs. received model
Closes https://github.com/BerriAI/litellm/issues/12887
2025-07-22 23:09:36 -07:00
Ishaan Jaff
a0846c3f31
ci/cd new release
2025-07-22 21:08:02 -07:00
Ishaan Jaff
642cfa26b0
remove deprecated
2025-07-22 20:59:34 -07:00
Ishaan Jaff
3a76b2762f
ci/cd new release
2025-07-22 20:41:49 -07:00
Ishaan Jaff
5741cdae0f
fix openrouter/qwen/qwen-vl-plus
2025-07-22 20:37:57 -07:00
Krish Dholakia and GitHub
8cd6c25e1e
Fix async retryer on .acompletion() + forward clientside headers - filter out content-type from clientside request (causes llm api call to hang) ( #12886 )
...
* fix(main.py): fix async retryer
Fixes https://github.com/BerriAI/litellm/issues/12830
* fix(forward_clientside_headers_by_model_group.py): filter out 'content-type' from forwardable headers
clientside content-type != proxy content type, can cause requests to hang
2025-07-22 19:50:05 -07:00
Ishaan Jaff and GitHub
b41ce5c92f
[Feat] - Track cost + add tags for health checks done by LiteLLM Proxy ( #12880 )
...
* refactor to use add_user_api_key_auth_to_request_metadata
* add get_litellm_internal_health_check_user_api_key_auth
* add get_litellm_internal_health_check_user_api_key_auth
* add _update_model_params_with_health_check_tracking_information
* add HealthCheckHelpers
* refactor to use clean helpers
* test_update_model_params_with_health_check_tracking_information
* test_get_litellm_internal_health_check_user_api_key_auth
* test_add_user_api_key_auth_to_request_metadata
* fix _update_model_params_with_health_check_tracking_information
2025-07-22 18:45:57 -07:00
Ishaan Jaff
c21dc46a33
fix morph api tests
2025-07-22 18:44:44 -07:00
Ishaan Jaff
bd8b21e700
fix mypy linting
2025-07-22 18:38:54 -07:00
Ishaan Jaff
d93e85282e
test_bad_database_url
2025-07-22 18:37:51 -07:00
Ishaan Jaff
bf300f8ca7
Revert "Litellm dev 07 21 2025 p1 ( #12848 )"
...
This reverts commit e4e10aa4ed
2025-07-22 18:28:36 -07:00
Ishaan Jaff
f0a8abb911
fix cost_calculator
2025-07-22 18:16:53 -07:00
Ishaan Jaff
4b741adadb
fix recraft cost calc
2025-07-22 18:12:12 -07:00
Ishaan Jaff
e5debfc84c
fix model cost map for recraft
2025-07-22 18:11:10 -07:00
Ishaan Jaff
1910cf8496
test fix vertex ai
2025-07-22 18:06:38 -07:00
Ishaan Jaff and GitHub
b026f6b280
[Feat] Add cost tracking for new vertex_ai/llama-3 API models ( #12878 )
...
* add vertex_ai/meta/llama-3.1-405b-instruct-maas
* notes - vertex_ai/meta/llama-3.2-90b-vision-instruct-maas
2025-07-22 16:22:09 -07:00
c9899b5c06
[LLM Translation] Litellm gemini 2.0 live support ( #12839 )
...
* add gemini 2.0 live to model context and priceS
* added files to dump
* Update litellm/model_prices_and_context_window_backup.json
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* Update model_prices_and_context_window.json
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
* added vertex ai live preview
* added vertex ai change
* add input video and image cosT
* add input video and image cosT
---------
Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2025-07-22 15:14:52 -07:00
Ishaan Jaff and GitHub
d5ee93aa0c
[Feat] Add Recraft API - Image Edits Support ( #12874 )
...
* test_recraft_image_edit_api
* add RecraftImageEditConfig
* complete RecraftImageEditConfig
* add RecraftImageEditRequestParams in types
* update RecraftImageEditRequestParams
* working
* transform_image_edit_request
* Image Edit docs recraft
* working transform_image_edit_request
* TestRecraftImageEditTransformation
2025-07-22 15:03:08 -07:00
Ishaan Jaff
31e9303232
remove old test
2025-07-22 14:11:43 -07:00
Jugal D. Bhatt and GitHub
aa1ebf37b2
pass in cmd args ( #12871 )
2025-07-22 14:10:23 -07:00
Ishaan Jaff and GitHub
934af0e9a0
add azure-keyvault==4.2.0 ( #12873 )
2025-07-22 14:08:33 -07:00
d4900d7dc0
build(deps): bump form-data from 4.0.3 to 4.0.4 in /docs/my-website ( #12867 )
...
---
updated-dependencies:
- dependency-name: form-data
dependency-version: 4.0.4
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-07-22 11:23:41 -07:00
Ishaan Jaff
6f0c211788
[Feat] MCP Gateway - allow using MCPs with all LLM APIs when using /responses with LiteLLM ( #12546 )
...
* add MCPResponsesAPIHelper
* rename LiteLLM_Proxy_MCP_Handler
* aresponses_api_with_mcp
* mock_responses_api_response
* test response with litellm proxy MCP
* add _should_use_litellm_mcp_gateway
* fix transform_mcp_tool_to_openai_responses_api_tool
* use correct _transform_mcp_tools_to_openai
* fix config.yaml
* fixes for native MCP handling
* docs MCP with litellm proxy
* aresponses_api_with_mcp
* fix linting
* fix mypy
* fix linting
* test_aresponses_api_with_mcp_mock_integration
* docs How it works when server_url="litellm_proxy"
2025-07-22 10:22:45 -07:00
