tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-07-13 17:10:08 +00:00
Code Issues Packages Projects Releases Wiki Activity
28,119 Commits 2,072 Branches 1,425 Tags
a4341ccf833c19f716e00a8fedc66fb257d5bcd6
Commit Graph

232 Commits

Author SHA1 Message Date
Krish Dholakia 4c7a988454 Guardrail API V2 - user api key metadata, session id, specify input type (request/response), image support (#17338) 
* refactor(generic_guardrail_api.py): refactor to update to new guardrail api logic

* refactor: refactor llm api integrations to support passing in text as a list[str] instead of one at a time

* refactor: fix linting errors

* refactor: pass request type to guardrail api

allows request vs. response processing to occur

* feat: pass user api key dict information to the guardrail api

* fix: pass user api key dict information to the guardrail api

* feat: pass litellm call id + trace id, if present

* docs: update docs
 2025-12-01 20:11:58 -08:00
Krish Dholakia b6d6f834e0 (feat) Generic Guardrail API - allows guardrail providers to add INSTANT support for LiteLLM w/out PR to repo (#17175) 
* feat(generic_guardrail_api.py): new generic api for guardrails

Allows guardrail providers to work with litellm for guardrails without needing to make a PR to LiteLLM

* docs(generic_guardrail_api.md): document new generic guardrail api

* Fix: Improve PII detection and guardrail API integration

Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>

* feat: correctly extract raw request from guardrail api

* docs(generic_guardrail_api.md): document this is a beta feature

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
 2025-12-01 14:29:52 -08:00
Ishaan Jaffer b43b68a072 docs fix 2025-11-22 14:02:14 -08:00
Ishaan Jaffer badbadba0d fix img URL for tests 2025-11-22 09:41:15 -08:00
Ishaan Jaff 661117678c Revert "remove deprecated embedding model (#16724)" (#16970) 
This reverts commit b9bc903536.
 2025-11-22 09:34:53 -08:00
Ishaan Jaffer 95caa2e3de bump openai 2.8.0 2025-11-19 17:47:18 -08:00
Sameer Kankute b9bc903536 remove deprecated embedding model (#16724) 2025-11-17 18:46:20 -08:00
Ishaan Jaff 630a746c84 [Feat] Add Custom Secret Manager - Allow users to define and write a custom secret manager (#16297) 
* add CustomSecretManager class

* docs custom secret manager

* add TestCustomSecretManager

* add KeyManagementSystem.CUSTOM

* add get_secret_from_manager

* add custom secret manager

* Potential fix for code scanning alert no. 3662: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* docs fix

* load_custom_secret_manager

* initialize_secret_manager

* add custom_secret_manager

* fix add custom secret manager

* add custom secret manager to KeyManagementSystem

* fix KeyManagementSystem.CUSTOM

* fix custom secret manager within cookbook

* fix link for custom secret manager

* Potential fix for code scanning alert no. 3663: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
 2025-11-05 17:20:26 -08:00
TensorNull e3566faf15 fix(cookbook): Remove the CometAPI key used for testing. 2025-10-16 14:14:58 +08:00
TensorNull 55a6dd3a8b feat(cometapi): Add CometAPI provider support (embeddings, image generation, docs) 
- Add CometAPI embedding and image generation transformations and configs
- Add image cost calculator and export/init files
- Register provider in constants, utils, main (embedding path) and sidebars
- Add CometAPI docs page and cookbook notebook (Colab) for usage examples
 2025-10-16 13:08:14 +08:00
Ishaan Jaffer 10a801ce83 docs fix 2025-09-27 16:52:02 -07:00
Alexsander Hamir eaa04cd8ce fix: use fastuuid helper (#14903) 
* fix: use fastuuid helper across the codebase

First batch of changes, simple drop in replacement.

* second batch of changes

* fixed: script mistake on helper file
 2025-09-25 15:47:01 -07:00
Ishaan Jaff b9ffa98c55 [Feat] Proxy CLI: Create a python method to login using litellm proxy (#14782) 
* fix: cli auth with SSO okta

* fix: add LITTELM_CLI_SERVICE_ACCOUNT_NAME

* fix: get_litellm_cli_user_api_key_auth

* use existing_key CLI

* fix: use existing key

* test auth commands

* test_cli_sso_callback_regenerate_vs_create_flow

* feat: add CLI Token Utilities

* fix: get_stored_api_key

* move file

* fix: get_valid_models

* fix config.yaml

* TestCLITokenUtils

* TestGetValidModelsWithCLI

* fix: tie user id to keys created through CLI

* fix: add teams interface to CLI

* add /keys/update to the list client commands

* fix /sso/cli/poll to return the user_id

* fix: working TeamsManagementClient

* fix CLI Login command

* fixes for auth

* Potential fix for code scanning alert no. 3400: Clear-text logging of sensitive information

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* ruff fix

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
 2025-09-22 21:28:38 -07:00
Ishaan Jaff 10de012e12 [Docs] - v1.77.3 (#14751) 
* fix sidebar

* v1

* docs fix

* docs fix

* docs fix

* docs fix

* docs fix

* docs fix

* docs fix

* docs fix

* docs fix

* docs fix
 2025-09-20 15:23:44 -07:00
Ishaan Jaff f37dd6bb95 Litellm 1.77.2 stable notes (#14544) 
* fix release notes instructions

* docs v1

* fix doc

* fix highlights

* docs fix

* docs fix
 2025-09-13 18:41:34 -07:00
Ishaan Jaff 075a089d82 [Feat] Bedrock Batches - Ensure correct transformation applied to incoming requests (#14522) 
* use is_batch_jsonl_file

* fix valid_content_type

* fix transform_create_file_request

* fix _transform_openai_jsonl_content_to_bedrock_jsonl_content

* test_transform_openai_jsonl_content_to_bedrock_jsonl_content

* fix mypy linting errors

* fix BEDROCK_BATCH_MODEL

* fix working sample

* fix comment

* fix model list

* fix: use with managed batches

* refactor
 2025-09-12 18:32:57 -07:00
Ishaan Jaff a13aa4740a [Fixes] Bug fixes to using LiteLLM MCP Gateway (#14392) 
* fix: use _get_mcp_servers_in_path

* fix checks for using litellm_proxy as MCP tool provider

* fix: fix mcp_tools_with_litellm_proxy

* fix: fix aresponses_api_with_mcp

* aresponses_api_with_mcp

* test_mcp_allowed_tools_filtering

* fix: _filter_mcp_tools_by_allowed_tools

* fix: _filter_mcp_tools_by_allowed_tools

* test_streaming_responses_api_with_mcp_tools

* fixes: test tools transfrom MCP->OpenaI spec

* test_streaming_responses_api_with_mcp_tools

* fix: chat ui allow multi select with allowed tools

* fix: use correct MCP events with litellm proxy response API

* fix get_event_model_class

* fix litellm proxy MCP handler

* fix MCPEnhancedStreamingIterator

* chat ui show list tools result

* UI: show MCP events

* fix stream iterator

* fixes: litellm proxy mcp handler

* test responses + mcp

* fix: update responses api with mcp handling

* ruff check fix

* central: _process_mcp_tools_to_openai_format

* fix: refactor code

* test_mcp_allowed_tools_filtering

* test mcp with litellm proxy

* fix mcp call

* demo: video using MCP ui

* fixes for using stream iterator

* test_no_duplicate_mcp_tools_in_streaming_e2e

* docs fix

* fix code snippet
 2025-09-10 19:12:11 -07:00
Ishaan Jaff 23ae7170d1 [Feat] Allow using Veo Video Generation through LiteLLM Pass through routes (#14228) 
* fix: add follow_redirects=True,

* test_pass_through_with_httpbin_redirect

* cook book veo video

* docs Veo Video Generation with Google AI Studio

* add veo-3.0-generate-preview cost tracking details

* track vertex_video_models
 2025-09-03 18:25:43 -07:00
Philip Kiely 7c3d522435 Update Baseten LiteLLM integration 2025-08-19 12:21:05 -07:00
Ishaan Jaff 4d941c914e [Feat] Responses API Session Handling - Multi media support (#13347) 
* rename ResponsesSessionHandler

* use ResponsesSessionHandler

* test session handler

* refactor ResponsesSessionHandler

* fix get_proxy_server_request_from_spend_log

* use constant for LITELLM_TRUNCATED_PAYLOAD_FIELD

* add _should_check_cold_storage_for_full_payload

* add get_class_type_for_custom_logger_name

* get_active_custom_logger_for_callback_name

* add get_proxy_server_request_from_cold_storage to CustomLogger

* add ColdStorageHandler

* start using cold storage integration

* add get_proxy_server_request_from_cold_storage

* fixes from manual testing

* s3 v2 fix getting region name

* ChatCompletionImageUrlObject

* use _get_configured_cold_storage_custom_logger

* fixes for _should_check_cold_storage_for_full_payload

* fix _download_object_from_s3

* test_s3_v2_with_cold_storage

* add cold_storage_object_key to StandardLoggingMetadata

* use get_proxy_server_request_from_cold_storage_with_object_key

* add cold_storage_object_key to SpendLogsMetadata

* add cold_storage_object_key

* get_proxy_server_request_from_cold_storage_with_object_key

* use get_proxy_server_request_from_cold_storage_with_object_key

* test responses API

* add get_proxy_server_request_from_cold_storage_with_object_key

* session handler fixes

* test session handler

* fix ruff checks

* _download_object_from_s3

* cleanup

* test

* lint fix

* test_e2e_cold_storage_successful_retrieval

* test_e2e_generate_cold_storage_object_key_successful

* test_async_gcs_pub_sub_v1

* test fix

* test fix

* test fix

* test_standard_logging_metadata_has_cold_storage_object_key_field

* test_sanitize_request_body_for_spend_logs_payload_basic

* test_transform_input_image_item_to_image_item_with_image_data
 2025-08-07 10:59:53 -07:00
Krish Dholakia d37cc63250 Add new model provider Novita AI (#7582) (#9527) 
* Add new model provider Novita AI (#7582)

* feat: add new model provider Novita AI

* feat: use deepseek r1 model for examples in Novita AI docs

* fix: fix tests

* fix: fix tests for novita

* fix: fix novita transformation

* ci: fix ci yaml

* fix: fix novita transformation and test (#10056)

---------

Co-authored-by: Jason <ggbbddjm@gmail.com>
 2025-05-12 21:49:30 -07:00
Ishaan Jaff 2cc4a87861 [Docs] Using litellm with Google ADK (#10777) 
* docs litellm ADK usage

* docs litellm google adk

* docs litellm ADK

* docs litellm with ADK usage examples

* docs litellm proxy with ADK

* cookbook litellm ADK
 2025-05-12 16:41:49 -07:00
minatoaquaMK2 65b99d6bc3 feat(grafana_dashboard): enable datasource selection via templating (#10257) 
This commit updates the Grafana dashboard configuration to include a datasource template variable. This allows users to dynamically select the datasource directly within the Grafana dashboard, improving flexibility and user experience.
 2025-04-25 08:49:29 -07:00
Krish Dholakia 34bdf36eab Add inference providers support for Hugging Face (#8258) (#9738) (#9773) 
* Add inference providers support for Hugging Face (#8258)

* add first version of inference providers for huggingface

* temporarily skipping tests

* Add documentation

* Fix titles

* remove max_retries from params and clean up

* add suggestions

* use llm http handler

* update doc

* add suggestions

* run formatters

* add tests

* revert

* revert

* rename file

* set maxsize for lru cache

* fix embeddings

* fix inference url

* fix tests following breaking change in main

* use ChatCompletionRequest

* fix tests and lint

* [Hugging Face] Remove outdated chat completion tests and fix embedding tests (#9749)

* remove or fix tests

* fix link in doc

* fix(config_settings.md): document hf api key

---------

Co-authored-by: célina <hanouticelina@gmail.com>
 2025-04-05 10:50:15 -07:00
Ishaan Jaff 5965680176 fix dev release.txt 2025-04-01 12:02:51 -07:00
dependabot[bot] 8f35bdffb0 build(deps): bump litellm in /cookbook/litellm-ollama-docker-image 
Bumps [litellm](https://github.com/BerriAI/litellm) from 1.55.3 to 1.61.15.
- [Release notes](https://github.com/BerriAI/litellm/releases)
- [Commits](https://github.com/BerriAI/litellm/commits)

---
updated-dependencies:
- dependency-name: litellm
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-03-20 21:03:29 +00:00
Ishaan Jaff e1c0025a55 docs litellm x langfuse cookbook 2025-02-14 09:31:51 -08:00
Ishaan Jaff a551f978fb update cookbook 2025-02-14 09:23:04 -08:00
Ishaan Jaff 9511cdddcd docs litellm langfuse cookbook 2025-02-14 08:16:27 -08:00
Ishaan Jaff 1e423cadf4 cookbook litellm proxy langfuse (#8541) 2025-02-14 08:07:59 -08:00
exiao 85491a0bab Add Arize Cookbook for Turning on LiteLLM Proxy (#8336) 
* Add files via upload

* Update arize_integration.md
 2025-02-06 16:16:28 -08:00
Ishaan Jaff 6f6c651ee0 (Feat) Add input_cost_per_token_batches, output_cost_per_token_batches for OpenAI cost tracking Batches API (#7391) 
* add input_cost_per_token_batches

* input_cost_per_token_batches
 2024-12-23 17:42:58 -08:00
Ishaan Jaff c7f14e936a (code quality) run ruff rule to ban unused imports (#7313) 
* remove unused imports

* fix AmazonConverseConfig

* fix test

* fix import

* ruff check fixes

* test fixes

* fix testing

* fix imports
 2024-12-19 12:33:42 -08:00
Krish Dholakia e332e93786 Litellm security fixes (#7282) 
* build(Dockerfile): bump node version

* build(Dockerfile): bump python version

fix critical errors

* build(requirements.txt): fix snyk errors
 2024-12-18 09:38:52 -08:00
Ishaan Jaff 7a5f997fc9 (refactor) remove berrispendLogger - unused logging integration (#6363) 
* fix remove berrispendLogger

* remove unused clickhouse logger
 2024-10-22 16:53:25 +05:30
Krish Dholakia f252350881 LiteLLM Minor Fixes & Improvements (10/17/2024) (#6293) 
* fix(ui_sso.py): fix faulty admin only check

Fixes https://github.com/BerriAI/litellm/issues/6286

* refactor(sso_helper_utils.py): refactor /sso/callback to use helper utils, covered by unit testing

Prevent future regressions

* feat(prompt_factory): support 'ensure_alternating_roles' param

Closes https://github.com/BerriAI/litellm/issues/6257

* fix(proxy/utils.py): add dailytagspend to expected views

* feat(auth_utils.py): support setting regex for clientside auth credentials

Fixes https://github.com/BerriAI/litellm/issues/6203

* build(cookbook): add tutorial for mlflow + langchain + litellm proxy tracing

* feat(argilla.py): add argilla logging integration

Closes https://github.com/BerriAI/litellm/issues/6201

* fix: fix linting errors

* fix: fix ruff error

* test: fix test

* fix: update vertex ai assumption - parts not always guaranteed (#6296)

* docs(configs.md): add argila env var to docs
 2024-10-17 22:09:11 -07:00
Ishaan Jaff 4d1b4beb3d (refactor) caching use LLMCachingHandler for async_get_cache and set_cache (#6208) 
* use folder for caching

* fix importing caching

* fix clickhouse pyright

* fix linting

* fix correctly pass kwargs and args

* fix test case for embedding

* fix linting

* fix embedding caching logic

* fix refactor handle utils.py

* fix test_embedding_caching_azure_individual_items_reordered
 2024-10-14 16:34:01 +05:30
Ishaan Jaff b1e9d344b2 Update readme.md 2024-10-07 18:15:15 +05:30
Ishaan Jaff a0cbf31fcf Update readme.md 2024-10-07 18:12:43 +05:30
Ishaan Jaff 1bafbf8382 (feat proxy) add v2 maintained LiteLLM grafana dashboard (#6098) 
* add new grafana dashboard litellm

* add v2 grafana dashboard
 2024-10-07 18:11:20 +05:30
Victor Mustar b2d62f00a3 update cookbook 
simplify
 2024-08-28 17:19:48 +02:00
Ishaan Jaff 5bc8b59b11 docs - use consistent name for LiteLLM proxy server 2024-08-03 12:49:35 -07:00
Krrish Dholakia c5ee6fe4b9 build: cookbook on migrating to litellm proxy from openai/azure sdk 2024-07-27 16:09:50 -07:00
Ishaan Jaff c99a5a58ec docs grafana dashboard litellm 2024-07-05 14:08:55 -07:00
Ishaan Jaff 9c6080be1a Create readme.md 2024-07-05 14:04:47 -07:00
Ishaan Jaff 8b741f5efc add example grafana dashboard 2024-07-05 14:02:05 -07:00
Ishaan Jaff 166eb24c54 script to add new models 2024-07-04 20:52:58 -07:00
Ishaan Jaff 778b41cea5 fix - update migration script to ensure api_key in script 2024-05-22 12:22:23 -07:00
mogith-pn 723ef9963e Clarifai - Added streaming and async completion support 2024-05-03 14:03:38 +00:00
mogith-pn d770df2259 Merge branch 'main' into main 2024-04-30 22:48:52 +05:30