tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-08-20 08:23:47 +00:00
Code Issues Packages Projects Releases Wiki Activity
22,032 Commits 2,072 Branches 1,425 Tags
d367eb346036f1bca7cd5ff810975f979badcd15
Commit Graph
22032 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Ishaan JaffandGitHub d367eb3460 [Docs] Pii masking with LiteLLM (#10924) 
* doc pii masking

* remove old doc

* docs add deployment options

* docs guardrail pii masking

* docs pii masking / tracing

* add images of tracing guards litellm ui

* docs presidio pii
 2025-05-17 18:35:33 -07:00
Ishaan JaffandGitHub cdf6026fdd docs apply guard (#10923) 2025-05-17 18:34:35 -07:00
Krish DholakiaandGitHub b4fc703d3b LiteLLM Stable release notes (#10919) 
* docs(index/v1.70.1-stable): style improvements

* style: add style improvements to docs

* docs: cleanup docs

* docs: more style improvements

* docs: style improvements

* docs(gemini/realtime): add docs on realtime api via Google AI Studio

* docs: add openai example to anthropic web search docs

* docs: add missing doc links

* docs: doc cleanup

* docs: add more doc links

* fix: cleanup

* docs: add docker information

* docs: update doc links

* docs: add demo instance details to doc

s
 2025-05-17 17:54:25 -07:00
Ishaan Jaff ac541392ed fix: apply_guardrail endpoint 2025-05-17 17:51:19 -07:00
Ishaan Jaff 22aa33e8b5 bump litellm_enterprise-0.1.4 2025-05-17 17:28:59 -07:00
Ishaan Jaff b861a650ad ui new build 2025-05-17 17:22:32 -07:00
Ishaan JaffandGitHub 5ab9fc0b7a fix linting cryptography (#10921) 2025-05-17 17:21:22 -07:00
Ishaan Jaff e87b536d03 fix config.yml 2025-05-17 16:59:03 -07:00
Ishaan Jaff 01d0983c29 fix config.yml 2025-05-17 16:57:17 -07:00
Ishaan Jaff c79cba47ec fix linting errors from ryptography.hazmat.bindings._rust 2025-05-17 16:31:30 -07:00
Ishaan JaffandGitHub 93ec0c48bc [UI QA Guardrails] - Minor UI Fixes (#10920) 
* fix: presidio params on ui

* move mode and always on to the top

* explain each mode

* show switch for default on / off for guards

* allow multiple modes for guardrails

* fix: presidio output parsing with stream

* Revert "git commit fix linting error on cryptography bindings"

This reverts commit 2615f04afd.
 2025-05-17 16:27:57 -07:00
Ishaan JaffandGitHub 50a6f289cd [QA/Guardrails] Allow editing guardrail entity configs (#10918) 
* allow editing guardrail entity

* fix: allow patching guardrail entity config

* fix: test init guardrails

* fix: update in mem guardrail params

* fix: update in mem guardrail params

* allow editing guardrail entities on ui

* fix pii entity config

* fixes for unselect/select entities

* fix: test fixes
 2025-05-17 16:27:02 -07:00
Krrish Dholakia d79cd70a1a docs(index/v1.70.1-stable): add initial release note 2025-05-17 16:13:44 -07:00
Jugal D. BhattandGitHub b342aa9253 add docs for spend logs (#10913) 
* add docs for spend logs

* change docs

* change docs

* change docs and change default run loops.

There was a calc error previously, it was not a million when run 100 times, but 100k, now default run would delete 500k records in 50s (running 500 loops of function before exiting)

* add docs changes to ui logs to metion deletion
 2025-05-17 16:00:28 -07:00
Ishaan Jaff 9846abb5c0 allow using output parsing for guardrails 2025-05-17 14:52:39 -07:00
Ishaan Jaff ddf1b20f4f Revert "git commit fix linting error on cryptography bindings" 
This reverts commit 2615f04afd.
 2025-05-17 14:49:54 -07:00
Ishaan Jaff 2615f04afd git commit fix linting error on cryptography bindings 2025-05-17 12:16:37 -07:00
Ishaan Jaff db624c90f6 (UI) new build 2025-05-17 11:38:45 -07:00
Krrish Dholakia 61d7704057 bump: version 1.70.0 → 1.70.1 v1.70.1-stable 2025-05-17 09:10:52 -07:00
Krrish Dholakia 9199078f36 test: update test 2025-05-17 07:56:05 -07:00
Ishaan Jaff 88153e9a37 (ui) cleanup logs filter 2025-05-17 07:45:48 -07:00
Krrish Dholakia 5c5699b65d build: merge in https://github.com/BerriAI/litellm/pull/10909 
Closes https://github.com/BerriAI/litellm/pull/10909
 2025-05-17 07:36:56 -07:00
Krish DholakiaandGitHub cc626ad3ec Handle openai gpt file data + add openai 'supports_pdf_input' to all vision models + Support bedrock tool cache pointing (#10897) 
* fix(openai/gpt_transformation.py): handle missing filename for openai file data call

* fix(openai/gpt_transformation.py): clean handling for sync + async pdf url transformation flows

Fixes https://github.com/BerriAI/litellm/issues/10820

* build(model_prices_and_context_window.json): add 'supports_pdf_input' for all openai models which have 'vision' support

Follows openai guidelines

* feat(bedrock/chat): support cache pointing tool calls on Bedrock

Closes https://github.com/BerriAI/litellm/pull/10613

* fix: fix linting error
 2025-05-17 07:29:01 -07:00
Ishaan Jaff e0cccb80dd ui - fixes leftnav 2025-05-16 20:55:09 -07:00
Ishaan Jaff 3bbfdcb865 fix: linting error v1.70.0-nightly 2025-05-16 20:15:42 -07:00
Ishaan Jaff 8a7fdf3dd5 bump: version 1.69.3 → 1.70.0 2025-05-16 18:38:10 -07:00
Ishaan Jaff 148814658e ui new build 2025-05-16 18:37:42 -07:00
Ishaan JaffandGitHub 76de6e374c [UI] Allow editing guardrails (#10907) 
* ui qa guardrails

* ui fixes guardrail info

* feat: add patch guardrail endpoint

* fix: update_guardrail_in_db

* ui add support for editing guardrails
 2025-05-16 18:35:03 -07:00
Jugal D. BhattandGitHub d8525ecbfa Remove extra test file (#10889) 2025-05-16 18:34:22 -07:00
Jugal D. BhattandGitHub 7702663666 add constant to config settings.md (#10908) 2025-05-16 18:33:19 -07:00
Ishaan JaffandGitHub ffafc77791 [UI] Fixes for using Lakera Guardrail on LiteLLM UI (#10906) 
* add lakera v2 on ui

* fix: lakera init on ui

* ui fixes for exposed guardrails

* fix: lakera pii masking

* fix: lakera pii masking
 2025-05-16 18:24:40 -07:00
Jugal D. BhattandGitHub 44c100d05d [Feat] handle multi-pod deployment for SpendLogs Retention (#10895) 
* handle multi-pod deployment

* fix utc and ruff errors

* add constants.py

* add lock duration acc to interval

* add lock duration on pod lock manager

* update tests to use redis

* update comments from review

* update config_Settings.md

* lint errors

* remove custom ttl setting

* add constants.py

* add constants.py

* add check for pod lock manager, and allow otherwise

* remove dup try except and move to finally

remove args
 2025-05-16 18:18:51 -07:00
tanjiroandGitHub fdacf07ccb Key alias filter on logs (#10905) 
* key hash filter added

* key alias filter added
 2025-05-16 17:49:14 -07:00
Ishaan JaffandGitHub bd1e0634bf [Feat] Add Lakera v2 Guardrail Support (#10880) 
* feat: add types for lakera v2

* feat: add call v2 guard for lakera

* feat: add call v2 guard for lakera

* feat: add LAKERA_V2

* feat: add LAKERA_V2 params

* feat: add initialize_lakera_v2

* fix: lakera pii masking

* test: lakera pii masking

* fix: lakera pii masking with tracing

* fix: lakera pii masking with tracing

* fix: fix linting errors

* fix: lakera ai docs
 2025-05-16 17:08:21 -07:00
Ishaan JaffandGitHub 888c4130bb [Feat] UI: Allow clicking into a guardrail (#10904) 
* ui allow clicking into guardrail

* feat: add guardrail info endpoint

* fix: guardrails/{guardrail_id}/info

* ui click into guard

* fix guardrail info view

* fix linting error
 2025-05-16 16:58:40 -07:00
tanjiroandGitHub dc8945cce8 key hash filter added (#10903) 2025-05-16 16:53:52 -07:00
Ishaan JaffandGitHub 1a7932a262 Feat: add guardrail tracing to OTEL, Arize phoenix (#10896) 
* feat: add guardrail tracing to OTEL, Arize phoenix

* fix: code qa check

* test: trace guard on OTEL

* fix: linting
 2025-05-16 13:38:11 -07:00
Will ChenandGitHub 0942a9d51d Fix example commands in reliability.md (#10894) 
-D is incorrect and dumps the headers.
The correct flag is -d which sets it as data
 2025-05-16 13:21:34 -07:00
Ishaan JaffandGitHub 298912bd38 [UI] - Add Guardrail Tracing to LiteLLM SpendLogs (#10893) 
* feat: trace guardrail SLP in spendLogs

* test: trace guardrail SLP in spendLogs

* add guardrail viewer

* checkpoint - working guardrail view on logs

* ui add guardrail view to SpendLogs

* test: fixes guardrails

* trace: fixes guardrails
 2025-05-16 12:20:20 -07:00
Ishaan JaffandGitHub b9460dfe37 [Feat] Add Tracing for guardrails in StandardLoggingPayload, Langfuse (#10890) 
* test:test_standard_logging_payload_includes_guardrail_information

* feat: add tracing for presidio pii masking

* feat: add tracing for guardrails on langfuse

* test: test_langfuse_trace_includes_guardrail_information

* feat: add tracing for guardrails on langfuse

* fix: working guardrail trace

* test: guardrail trace lands on langfuse

* fix: linting

* fix: code qa check

* test: fixes for presidio test

* test: fixes for presidio test

* test: guardrail trace lands on langfuse

* test: guardrail trace lands on langfuse

* test: guardrail trace lands on langfuse

* test: guardrail trace lands on langfuse
 2025-05-16 10:47:12 -07:00
Krrish Dholakia cec4535cd8 fix: fix unit tests 2025-05-16 10:33:44 -07:00
JasonandGitHub 803dda05ed fix: fix novita ai docs (#10882) 2025-05-16 10:04:55 -07:00
Ishaan JaffGitHubtanjiro
 		f335bb91dd Model filter added on logs tab. (#10877) (#10891) 
* model filter added

* Make status logic it's own helper + add a unit test in test_spend_management_endpoints

Co-authored-by: tanjiro <56165694+NANDINI-star@users.noreply.github.com>
 2025-05-16 10:04:02 -07:00
Jugal D. BhattandGitHub d5d5166777 Feat log retention cleanup (#10872) 
* Batch deletion with tasks

* reset utils

* change print to logger

* remove print for lint

* fix lint

* local testing

* remove constants.py var

* ruff fix

* type fix

* - make sure job doesn't get added if maximum_spend_logs_retention_period is not set
- enable user to override scheduler time
- Upper bound on while true

* update and fix test
 2025-05-15 22:43:39 -07:00
Krish DholakiaandGitHub fdfef04d93 Gemini Multimodal Live API support (#10841) 
* fix: initial commit

* refactor(gemini/realtime/transformation.py): initial instrumentation

* fix: fix default api base if not set

* feat(gemini/): passes initial user request to backend

* feat(realtime_streaming.py): support transforming message input before sending it

enables gemini realtime streaming to be sent in correct format

* feat: initial working commit of setup message being sent + working

* fix(gemini/): initial commit supporting realtime response transformation

* feat(gemini/realtime): transform session.created event correctly

* fix(realtime_streaming.py): more gemini/realtime response mapping - handle new message

* test(gemini/realtime/test_): add more unit tests

* feat(gemini/realtime): handles consecutive deltas

* feat(gemini/realtime): support openai 'response.text.done' event

* feat(gemini/realtime): add openai 'response.text.done' and 'response.content_part.done' event support

* fix(gemini/realtime): add openai 'response.done' event support

unified realtime api support

* fix: fix linting errors

* fix: fix linting error

* fix: fix linting error

* fix: handle infinite loop

* fix: fix linting error

* fix: fix recursive detector

* fix: fix file

* fix: fix linting error

* fix: fix linting error
 2025-05-15 22:39:09 -07:00
Krrish Dholakia ffcdf441d2 fix: fix docs 2025-05-15 22:32:02 -07:00
Dror BaronGitHubKrish Dholakia
 		93639df4c3 Feat/support anonymize in aim guardrail (#10757) 
* Enable update/delete org members on UI  (#8560)

* feat(organization_endpoints.py): expose new `/organization/delete` endpoint. Cascade org deletion to member, teams and keys

Ensures any org deletion is handled correctly

* test(test_organizations.py): add simple test to ensure org deletion works

* feat(organization_endpoints.py): expose /organization/update endpoint, and define response models for org delete + update

* fix(organizations.tsx): support org delete on UI + move org/delete endpoint to use DELETE

* feat(organization_endpoints.py): support `/organization/member_update` endpoint

Allow admin to update member's role within org

* feat(organization_endpoints.py): support deleting member from org

* test(test_organizations.py): add e2e test to ensure org member flow works

* fix(organization_endpoints.py): fix code qa check

* fix(schema.prisma): don't introduce ondelete:cascade - breaking change

* docs(organization_endpoints.py): document missing params

* support anonymize and deanonymize

* use new response schema

* don't use detected because action already means there are detections

* log to debug

* CR fixes

* lint

* add tests

* use single quotes in deanonymiztion

* remove engage action case

* set max entities to 100 to prevent memory leak

* add test case for de-anonymization of llm response

---------

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
 2025-05-15 22:18:58 -07:00
Krrish Dholakia b54e2ae98b test: update unit test 2025-05-15 22:18:15 -07:00
Krrish Dholakia 2767266db3 build: update poetry lock 2025-05-15 22:07:38 -07:00
Damian GleumesandGitHub 384a7ba94d [Feat]: Configure LiteLLM to Parse User Headers from Open Web UI (#9802) 
* add user_header_name

* docs: add per-user tracking to Open WebUI with LiteLLM doc

* docs: standardize "OpenWeb UI" spelling across openweb_ui.md

* docs: improve wording for openweb_ui guide

* fix end_user_id not being set

- move user header parsing to add_litellm_data_to_request
- also set  user_api_key_dict.end_user_id from user header
 2025-05-15 22:01:12 -07:00