Commit Graph
23118 Commits
Author SHA1 Message Date
Ishaan Jaff 9846abb5c0 allow using output parsing for guardrails 2025-05-17 14:52:39 -07:00
Ishaan Jaff ddf1b20f4f Revert "git commit fix linting error on cryptography bindings"
This reverts commit 2615f04afd.
2025-05-17 14:49:54 -07:00
Ishaan Jaff 2615f04afd git commit fix linting error on cryptography bindings 2025-05-17 12:16:37 -07:00
Ishaan Jaff db624c90f6 (UI) new build 2025-05-17 11:38:45 -07:00
Krrish Dholakia 61d7704057 bump: version 1.70.0 → 1.70.1 v1.70.1-stable 2025-05-17 09:10:52 -07:00
Krrish Dholakia 9199078f36 test: update test 2025-05-17 07:56:05 -07:00
Ishaan Jaff 88153e9a37 (ui) cleanup logs filter 2025-05-17 07:45:48 -07:00
Krrish Dholakia 5c5699b65d build: merge in https://github.com/BerriAI/litellm/pull/10909
Closes https://github.com/BerriAI/litellm/pull/10909
2025-05-17 07:36:56 -07:00
Krish DholakiaandGitHub cc626ad3ec Handle openai gpt file data + add openai 'supports_pdf_input' to all vision models + Support bedrock tool cache pointing (#10897)
* fix(openai/gpt_transformation.py): handle missing filename for openai file data call

* fix(openai/gpt_transformation.py): clean handling for sync + async pdf url transformation flows

Fixes https://github.com/BerriAI/litellm/issues/10820

* build(model_prices_and_context_window.json): add 'supports_pdf_input' for all openai models which have 'vision' support

Follows openai guidelines

* feat(bedrock/chat): support cache pointing tool calls on Bedrock

Closes https://github.com/BerriAI/litellm/pull/10613

* fix: fix linting error
2025-05-17 07:29:01 -07:00
Ishaan Jaff e0cccb80dd ui - fixes leftnav 2025-05-16 20:55:09 -07:00
Ishaan Jaff 3bbfdcb865 fix: linting error v1.70.0-nightly 2025-05-16 20:15:42 -07:00
Ishaan Jaff 8a7fdf3dd5 bump: version 1.69.3 → 1.70.0 2025-05-16 18:38:10 -07:00
Ishaan Jaff 148814658e ui new build 2025-05-16 18:37:42 -07:00
Ishaan JaffandGitHub 76de6e374c [UI] Allow editing guardrails (#10907)
* ui qa guardrails

* ui fixes guardrail info

* feat: add patch guardrail endpoint

* fix: update_guardrail_in_db

* ui add support for editing guardrails
2025-05-16 18:35:03 -07:00
Jugal D. BhattandGitHub d8525ecbfa Remove extra test file (#10889) 2025-05-16 18:34:22 -07:00
Jugal D. BhattandGitHub 7702663666 add constant to config settings.md (#10908) 2025-05-16 18:33:19 -07:00
Ishaan JaffandGitHub ffafc77791 [UI] Fixes for using Lakera Guardrail on LiteLLM UI (#10906)
* add lakera v2 on ui

* fix: lakera init on ui

* ui fixes for exposed guardrails

* fix: lakera pii masking

* fix: lakera pii masking
2025-05-16 18:24:40 -07:00
Jugal D. BhattandGitHub 44c100d05d [Feat] handle multi-pod deployment for SpendLogs Retention (#10895)
* handle multi-pod deployment

* fix utc and ruff errors

* add constants.py

* add lock duration acc to interval

* add lock duration on pod lock manager

* update tests to use redis

* update comments from review

* update config_Settings.md

* lint errors

* remove custom ttl setting

* add constants.py

* add constants.py

* add check for pod lock manager, and allow otherwise

* remove dup try except and move to finally

remove args
2025-05-16 18:18:51 -07:00
tanjiroandGitHub fdacf07ccb Key alias filter on logs (#10905)
* key hash filter added

* key alias filter added
2025-05-16 17:49:14 -07:00
Ishaan JaffandGitHub bd1e0634bf [Feat] Add Lakera v2 Guardrail Support (#10880)
* feat: add types for lakera v2

* feat: add call v2 guard for lakera

* feat: add call v2 guard for lakera

* feat: add LAKERA_V2

* feat: add LAKERA_V2 params

* feat: add initialize_lakera_v2

* fix: lakera pii masking

* test: lakera pii masking

* fix: lakera pii masking with tracing

* fix: lakera pii masking with tracing

* fix: fix linting errors

* fix: lakera ai docs
2025-05-16 17:08:21 -07:00
Ishaan JaffandGitHub 888c4130bb [Feat] UI: Allow clicking into a guardrail (#10904)
* ui allow clicking into guardrail

* feat: add guardrail info endpoint

* fix: guardrails/{guardrail_id}/info

* ui click into guard

* fix guardrail info view

* fix linting error
2025-05-16 16:58:40 -07:00
tanjiroandGitHub dc8945cce8 key hash filter added (#10903) 2025-05-16 16:53:52 -07:00
Ishaan JaffandGitHub 1a7932a262 Feat: add guardrail tracing to OTEL, Arize phoenix (#10896)
* feat: add guardrail tracing to OTEL, Arize phoenix

* fix: code qa check

* test: trace guard on OTEL

* fix: linting
2025-05-16 13:38:11 -07:00
Will ChenandGitHub 0942a9d51d Fix example commands in reliability.md (#10894)
-D is incorrect and dumps the headers.
The correct flag is -d which sets it as data
2025-05-16 13:21:34 -07:00
Ishaan JaffandGitHub 298912bd38 [UI] - Add Guardrail Tracing to LiteLLM SpendLogs (#10893)
* feat: trace guardrail SLP in spendLogs

* test: trace guardrail SLP in spendLogs

* add guardrail viewer

* checkpoint - working guardrail view on logs

* ui add guardrail view to SpendLogs

* test: fixes guardrails

* trace: fixes guardrails
2025-05-16 12:20:20 -07:00
Ishaan JaffandGitHub b9460dfe37 [Feat] Add Tracing for guardrails in StandardLoggingPayload, Langfuse (#10890)
* test:test_standard_logging_payload_includes_guardrail_information

* feat: add tracing for presidio pii masking

* feat: add tracing for guardrails on langfuse

* test: test_langfuse_trace_includes_guardrail_information

* feat: add tracing for guardrails on langfuse

* fix: working guardrail trace

* test: guardrail trace lands on langfuse

* fix: linting

* fix: code qa check

* test: fixes for presidio test

* test: fixes for presidio test

* test: guardrail trace lands on langfuse

* test: guardrail trace lands on langfuse

* test: guardrail trace lands on langfuse

* test: guardrail trace lands on langfuse
2025-05-16 10:47:12 -07:00
Krrish Dholakia cec4535cd8 fix: fix unit tests 2025-05-16 10:33:44 -07:00
JasonandGitHub 803dda05ed fix: fix novita ai docs (#10882) 2025-05-16 10:04:55 -07:00
f335bb91dd Model filter added on logs tab. (#10877) (#10891)
* model filter added

* Make status logic it's own helper + add a unit test in test_spend_management_endpoints

Co-authored-by: tanjiro <56165694+NANDINI-star@users.noreply.github.com>
2025-05-16 10:04:02 -07:00
Jugal D. BhattandGitHub d5d5166777 Feat log retention cleanup (#10872)
* Batch deletion with tasks

* reset utils

* change print to logger

* remove print for lint

* fix lint

* local testing

* remove constants.py var

* ruff fix

* type fix

* - make sure job doesn't get added if maximum_spend_logs_retention_period is not set
- enable user to override scheduler time
- Upper bound on while true

* update and fix test
2025-05-15 22:43:39 -07:00
Krish DholakiaandGitHub fdfef04d93 Gemini Multimodal Live API support (#10841)
* fix: initial commit

* refactor(gemini/realtime/transformation.py): initial instrumentation

* fix: fix default api base if not set

* feat(gemini/): passes initial user request to backend

* feat(realtime_streaming.py): support transforming message input before sending it

enables gemini realtime streaming to be sent in correct format

* feat: initial working commit of setup message being sent + working

* fix(gemini/): initial commit supporting realtime response transformation

* feat(gemini/realtime): transform session.created event correctly

* fix(realtime_streaming.py): more gemini/realtime response mapping - handle new message

* test(gemini/realtime/test_): add more unit tests

* feat(gemini/realtime): handles consecutive deltas

* feat(gemini/realtime): support openai 'response.text.done' event

* feat(gemini/realtime): add openai 'response.text.done' and 'response.content_part.done' event support

* fix(gemini/realtime): add openai 'response.done' event support

unified realtime api support

* fix: fix linting errors

* fix: fix linting error

* fix: fix linting error

* fix: handle infinite loop

* fix: fix linting error

* fix: fix recursive detector

* fix: fix file

* fix: fix linting error

* fix: fix linting error
2025-05-15 22:39:09 -07:00
Krrish Dholakia ffcdf441d2 fix: fix docs 2025-05-15 22:32:02 -07:00
93639df4c3 Feat/support anonymize in aim guardrail (#10757)
* Enable update/delete org members on UI  (#8560)

* feat(organization_endpoints.py): expose new `/organization/delete` endpoint. Cascade org deletion to member, teams and keys

Ensures any org deletion is handled correctly

* test(test_organizations.py): add simple test to ensure org deletion works

* feat(organization_endpoints.py): expose /organization/update endpoint, and define response models for org delete + update

* fix(organizations.tsx): support org delete on UI + move org/delete endpoint to use DELETE

* feat(organization_endpoints.py): support `/organization/member_update` endpoint

Allow admin to update member's role within org

* feat(organization_endpoints.py): support deleting member from org

* test(test_organizations.py): add e2e test to ensure org member flow works

* fix(organization_endpoints.py): fix code qa check

* fix(schema.prisma): don't introduce ondelete:cascade - breaking change

* docs(organization_endpoints.py): document missing params

* support anonymize and deanonymize

* use new response schema

* don't use detected because action already means there are detections

* log to debug

* CR fixes

* lint

* add tests

* use single quotes in deanonymiztion

* remove engage action case

* set max entities to 100 to prevent memory leak

* add test case for de-anonymization of llm response

---------

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
2025-05-15 22:18:58 -07:00
Krrish Dholakia b54e2ae98b test: update unit test 2025-05-15 22:18:15 -07:00
Krrish Dholakia 2767266db3 build: update poetry lock 2025-05-15 22:07:38 -07:00
Damian GleumesandGitHub 384a7ba94d [Feat]: Configure LiteLLM to Parse User Headers from Open Web UI (#9802)
* add user_header_name

* docs: add per-user tracking to Open WebUI with LiteLLM doc

* docs: standardize "OpenWeb UI" spelling across openweb_ui.md

* docs: improve wording for openweb_ui guide

* fix end_user_id not being set

- move user header parsing to add_litellm_data_to_request
- also set  user_api_key_dict.end_user_id from user header
2025-05-15 22:01:12 -07:00
Eric MaandGitHub 172b30cec1 build(dependencies)📦: Add numpydoc to dependencies in pyproject.toml (#10736)
* build(dependencies)📦: Add numpydoc to dependencies in pyproject.toml

- Include numpydoc for documentation generation in utils.py.

* Update poetry lock file.

* chore(dependencies)🔧: Update optional dependencies and markers in poetry.lock and pyproject.toml

- Set several packages as optional in poetry.lock.
- Add 'extra == "utils"' marker to various packages in poetry.lock.
- Update numpydoc dependency to be optional in pyproject.toml.
- Add 'utils' extra section in pyproject.toml with numpydoc.
2025-05-15 21:53:46 -07:00
Krish DholakiaandGitHub ad9f6b295a fix(streaming_handler.py): patch vertex ai meta llama streaming error (#10878)
Fixes issue where meta llama model response was being nested
2025-05-15 21:44:57 -07:00
Krish DholakiaandGitHub 32478de7a9 fix(proxy/_types.py): add missing comma for /v2/rerank (#10861)
* fix(proxy/_types.py): add missing comma for `/v2/rerank`

Enables non admins to access `/v2/rerank` endpoint

* fix(proxy_track_cost_callback.py): add patch to handle scenario where both 'litellm_metadata' and 'metadata' exist
2025-05-15 21:37:46 -07:00
Ishaan JaffandGitHub dc16e47df6 [UI] Allow adding Bedrock, Presidio, Lakera, AIM guardrails on UI (#10874)
* ui fix bedrock guard

* polish: logo should appear after selecting provider

* fix ui config bedrock

* fix: refactor - use specific configs per provider

* fix: refactor - use specific configs per provider

* feat: ui, show provider specific params for guardrails

* fix: updated type of LiteLLM params for guardrails

* fix: updated type of LiteLLM params for guardrails

* ui, use endpoint for adding presidio, bedrock guardrails

* fix: linting error

* add llama guard and secret detector on UI

* add aim on ui

* allow adding lakera AI on litellm ui

* fix: fixes for params to init guardrails

* test: test_guardrail_info_response

* test: test_initialize_presidio_guardrail

* fix: init guardrails

* fix: init guardrails

* add showSearch

* working bedrock guard
2025-05-15 21:22:56 -07:00
Krrish Dholakia c6d36e8912 test: fix test 2025-05-15 15:40:00 -07:00
f2dc261efc [Feat] UI: Add support for Deleting Guardrail on UI (#10869)
* show id on guardrails UI

* feat: show guardrail_id on UI

* guardrailToDelete

* ui - add DELETE guardrail

* Update ui/litellm-dashboard/src/components/guardrails.tsx

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update ui/litellm-dashboard/src/components/guardrails.tsx

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix ui linting

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-05-15 15:38:18 -07:00
Ishaan JaffandGitHub 263f447795 [CI/CD] Test: fix mapped tests (#10870)
* test: fix mapped tests

* test: fix mapped tests

* test: fix mapped tests

* test: fix mapped tests
2025-05-15 15:36:15 -07:00
Ishaan JaffandGitHub 3d37cc68f6 [Feat - Guardrails] Expose /apply_guardrail endpoint for directly calling guardrail (#10867)
* refactor: add types for mask pii request/response

* feat: add mask pii hook for guardrails

* feat: rename to apply_guardrail

* test: test_presidio_apply_guardrail

* fix: code QA

* fix: linting error

* feat: allow non-admins to access apply_guardrail routes
2025-05-15 14:52:24 -07:00
Ishaan JaffandGitHub 86a9b9902e refactor: fix send_management_endpoint_alert (#10863) 2025-05-15 10:35:27 -07:00
Ishaan JaffandGitHub faed9860c0 [Refactor] Move enterprise_routes within litellm_enterprise (#10860)
* fix: move enterprise routes to litellm_enterprise

* refactor: move enterprise routes to litellm_enterprise

* fix: litellm_enterprise routes

* fix test litellm on github workflow
2025-05-15 10:34:26 -07:00
Ishaan JaffandGitHub 42e6e664b2 [Refactor] Make Pagerduty a free feature (#10857)
* refactor: make pagerduty free

* refactor: make pagerduty free

* fix: pagerduty loc

* fix: linting error
2025-05-15 10:12:06 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
1e08816489 build(deps): bump undici from 6.21.1 to 6.21.3 in /docs/my-website (#10855)
Bumps [undici](https://github.com/nodejs/undici) from 6.21.1 to 6.21.3.
- [Release notes](https://github.com/nodejs/undici/releases)
- [Commits](https://github.com/nodejs/undici/compare/v6.21.1...v6.21.3)

---
updated-dependencies:
- dependency-name: undici
  dependency-version: 6.21.3
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-15 08:33:12 -07:00
Jugal D. BhattandGitHub 16280d9fa9 added fix to resolve delete API (#10845)
* added fix to resolve delete API

* add get requests to ignore model and add test cases
2025-05-15 08:31:37 -07:00
Anibal AnguloandGitHub 0434ca1781 Updated bedrock invoke transform for mistral to support pixtral models (#10439) 2025-05-14 23:08:37 -07:00