tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-08-24 10:27:13 +00:00
Code Issues Packages Projects Releases Wiki Activity
18,968 Commits 2,072 Branches 1,425 Tags
efe236d05b136cd1d89aaabb28ae183310dd6e7d
Commit Graph
18968 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Krrish Dholakia efe236d05b bump: version 1.57.5 → 1.57.6 2025-01-09 22:56:50 -08:00
Krrish Dholakia 3e3c78a200 build(ui/): update ui build 2025-01-09 22:44:05 -08:00
Krish DholakiaandGitHub 63926f484c feat(ui_sso.py): Allows users to use test key pane, and have team budget limits be enforced for their use-case (#7666) 2025-01-09 22:12:45 -08:00
Ishaan JaffandGitHub 6df8faf15f (minor latency fixes / proxy) - use verbose_proxy_logger.debug() instead of litellm.print_verbose (#7664) 
* minor latency fixes

* fix code quality
 2025-01-09 21:06:09 -08:00
Ishaan Jaff 7fcd130e1d bump: version 1.57.4 → 1.57.5 v1.57.5 2025-01-09 20:05:42 -08:00
Ishaan JaffandGitHub 51b17be3ab use asyncio tasks for logging db metrics (#7663) 2025-01-09 19:59:32 -08:00
Ishaan JaffandGitHub 13f364682d (Feat - Batches API) add support for retrieving vertex api batch jobs (#7661) 
* add _async_retrieve_batch

* fix aretrieve_batch

* fix _get_batch_id_from_vertex_ai_batch_response

* fix batches docs
 2025-01-09 18:35:03 -08:00
Ishaan JaffandGitHub 2507c275f6 (proxy perf improvement) - use uvloop for higher RPS (10%-20% higher RPS) (#7662) 
* uvicorn use uvloop

* fix uvloop==0.21.0

* add uvloop to pyproject

* test_completion_response_ratelimit_headers
 2025-01-09 18:11:20 -08:00
Ishaan JaffandGitHub a85de46ef7 (proxy - RPS) - Get 2K RPS at 4 instances, minor fix aiohttp_openai/ (#7659) 
* speed up transform_response

* use 2 workers

* undo changes to uvicorn

* ci/cd run again
 2025-01-09 17:24:18 -08:00
Ishaan JaffandGitHub fd46482916 fix 1 - latency fix (#7655) 2025-01-09 15:57:05 -08:00
Krish DholakiaandGitHub 907bcd3a62 Litellm dev 01 08 2025 p1 (#7640) 
* feat(ui_sso.py): support reading team ids from sso token

* feat(ui_sso.py): working upsert sso user teams membership in litellm - if team exists

Adds user to relevant teams, if user is part of teams and team exists on litellm

* fix(ui_sso.py): safely handle add team member task

* build(ui/): support setting team id when creating team on UI

* build(ui/): teams.tsx

allow setting team id on ui

* build(circle_ci/requirements.txt): add fastapi-sso to ci/cd testing

* fix: fix linting errors
 2025-01-08 22:08:20 -08:00
Krish DholakiaandGitHub 1e3370f3cb LiteLLM Minor Fixes & Improvements (01/08/2025) - p2 (#7643) 
* fix(streaming_chunk_builder_utils.py): add test for groq tool calling + streaming + combine chunks

Addresses https://github.com/BerriAI/litellm/issues/7621

* fix(streaming_utils.py): fix modelresponseiterator for openai like chunk parser

ensures chunk parser uses the correct tool call id when translating the chunk

 Fixes https://github.com/BerriAI/litellm/issues/7621

* build(model_hub.tsx): display cost pricing on model hub

* build(model_hub.tsx): show cost per token pricing + complete model information

* fix(types/utils.py): fix usage object handling
 2025-01-08 19:45:19 -08:00
Krrish Dholakia 39ee4c6bb4 docs(intro.md): add a section on 'why pass through endpoints' 
helps proxy admin understand when these would be useful
 2025-01-08 19:15:41 -08:00
Krrish Dholakia 3193cb3912 build(model_prices_and_context_window.json): omni-moderation-latest-intents 2025-01-08 19:06:04 -08:00
Ishaan Jaff 782b59788f bump: version 1.57.3 → 1.57.4 v1.57.4 2025-01-08 18:50:11 -08:00
Ishaan JaffandGitHub 7259452285 (helm) - bug fix - allow using migrationJob.enabled variable within job (#7639) 
* wrap migration job in conditional

* use enabled flag in values.yaml

* run migration job

* argo cd fixes

* fix values in image repo
 2025-01-08 18:47:33 -08:00
Ishaan JaffandGitHub 48d4f79206 fix is llm api route check (#7631) 2025-01-08 18:45:59 -08:00
Ishaan Jaff 9eb24df113 ci/cd run again 2025-01-08 18:36:39 -08:00
Krish DholakiaandGitHub 4af23353d6 Allow assigning teams to org on UI + OpenAI omni-moderation cost model tracking (#7566) 
* feat(cost_calculator.py): add cost tracking ($0) for openai moderations endpoint

removes sentry cost tracking errors caused by this

* build(teams.tsx): allow assigning teams to orgs
 2025-01-08 16:58:21 -08:00
Krish DholakiaGitHubWolfram Ravenwolf
 		0ffc5379ea Litellm dev 01 07 2025 p2 (#7622) 
* build(ui/): update ui

* fix: drop unsupported non-whitespace characters for real when calling… (#7484)

* fix: drop unsupported non-whitespace characters for real when calling anthropic with stop sequences

* test: add parameterized test for _map_stop_sequences method in AnthropicConfig

---------

Co-authored-by: Wolfram Ravenwolf <52386626+WolframRavenwolf@users.noreply.github.com>
 2025-01-08 16:56:39 -08:00
Ishaan JaffandGitHub fd0a03f719 (feat) - allow building litellm proxy from pip package (#7633) 
* fix working build from pip

* add tests for proxy_build_from_pip_tests

* doc clean up for deployment

* docs cleanup

* docs build from pip

* fix cd docker/build_from_pip
 2025-01-08 16:36:57 -08:00
Ishaan Jaff 43566e9842 fix docs 2025-01-08 12:51:59 -08:00
Ishaan Jaff e5717d2cb0 update load test docs 2025-01-08 12:48:21 -08:00
Ishaan Jaff 74b41d29d3 sort rn 2025-01-08 12:16:01 -08:00
Ishaan Jaff f95439af26 docs v1.57.3 2025-01-08 12:08:19 -08:00
Krish DholakiaGitHubhi019H
 		a187cee538 Litellm dev 01 07 2025 p3 (#7635) 
* fix(__init__.py): fix mistral large tool calling

map bedrock mistral large to converse endpoint

Fixes https://github.com/BerriAI/litellm/issues/7521

* braintrust logging: respect project_id, add more metrics + more (#7613)

* braintrust logging: respect project_id, add more metrics

* braintrust logger: improve json formatting

* braintrust logger: add test for passing specific project_id

* rm unneeded import

* braintrust logging: rm unneeded var in tets

* add project_name

* update docs

---------

Co-authored-by: H <no@email.com>

---------

Co-authored-by: hi019 <65871571+hi019@users.noreply.github.com>
Co-authored-by: H <no@email.com>
 2025-01-08 11:46:24 -08:00
Krish DholakiaandGitHub 07c5f136f1 fix(utils.py): fix select tokenizer for custom tokenizer (#7599) 
* fix(utils.py): fix select tokenizer for custom tokenizer

* fix(router.py): fix 'utils/token_counter' endpoint
 2025-01-07 22:37:09 -08:00
Ishaan Jaff 04eb718f7a update docs 2025-01-07 22:35:07 -08:00
Krrish Dholakia 7bf6769aa7 bump: version 1.57.2 → 1.57.3 v1.57.3 2025-01-07 22:23:19 -08:00
Krish DholakiaandGitHub e8ed40a27b Litellm dev 01 01 2025 p2 (#7615) 
* fix(utils.py): prevent double logging when passing 'fallbacks=' to .completion()

Fixes https://github.com/BerriAI/litellm/issues/7477

* fix(utils.py): fix vertex anthropic check

* fix(utils.py): ensure supported params is always set

Fixes https://github.com/BerriAI/litellm/issues/7470

* test(test_optional_params.py): add unit testing to prevent mistranslation

Fixes https://github.com/BerriAI/litellm/issues/7470

* fix: fix linting error

* test: cleanup
v1.57.2 		2025-01-07 21:40:33 -08:00
Ishaan JaffandGitHub 081826a5d6 (Feat) soft budget alerts on keys (#7623) 
* class WebhookEvent(CallInfo):
Add

* handle soft budget alerts

* handle soft budget

* fix budget alerts

* fix CallInfo

* fix _get_user_info_str

* test_soft_budget_alerts

* test_soft_budget_alert
 2025-01-07 21:36:34 -08:00
Krish DholakiaandGitHub 4e69711411 Litellm dev 01 07 2025 p1 (#7618) 
* fix(main.py): pass custom llm provider on litellm logging provider update

* fix(cost_calculator.py): don't append provider name to return model if existing llm provider

Fixes https://github.com/BerriAI/litellm/issues/7607

* fix(prometheus_services.py): fix prometheus system health error logging

Fixes https://github.com/BerriAI/litellm/issues/7611
 2025-01-07 21:22:31 -08:00
Ishaan JaffandGitHub 60c89a3e8a (Fix) security of base image (#7620) 
* fix security of base images

* fix dockerfile
 2025-01-07 20:35:57 -08:00
Ishaan Jaff 59133e8fe6 ci/cd run again 2025-01-07 10:01:29 -08:00
Ishaan Jaff 55139b8fd6 update tests 2025-01-06 22:36:00 -08:00
Krrish Dholakia d5a288e29e docs: cleanup keys 2025-01-06 21:57:18 -08:00
Ishaan Jaff ae3d8feb50 bump: version 1.57.1 → 1.57.2 2025-01-06 21:48:25 -08:00
Ishaan JaffandGitHub 2ca0977921 aiohttp_openai/ fixes - allow using aiohttp_openai/gpt-4o (#7598) 
* fixes for get_complete_url

* update aiohttp tests

* fix event loop for aiohtto

* ci/cd run again

* test_aiohttp_openai
 2025-01-06 21:39:11 -08:00
Ishaan Jaff 744beac754 ci/cd run again 2025-01-06 21:35:34 -08:00
Krish DholakiaandGitHub fef7839e8a Litellm dev 01 06 2025 p1 (#7594) 
* fix(custom_logger.py): expose new 'async_get_chat_completion_prompt' event hook

* fix(custom_logger.py): langfuse_prompt_management.py

remove 'headers' from custom logger 'async_get_chat_completion_prompt' and 'get_chat_completion_prompt' event hooks

* feat(router.py): expose new function for prompt management based routing

* feat(router.py): partial working router prompt factory logic

allows load balanced model to be used for model name w/ langfuse prompt management call

* feat(router.py): fix prompt management with load balanced model group

* feat(langfuse_prompt_management.py): support reading in openai params from langfuse

enables user to define optional params on langfuse vs. client code

* test(test_Router.py): add unit test for router based langfuse prompt management

* fix: fix linting errors
 2025-01-06 21:26:21 -08:00
Krrish Dholakia 7133cf5b74 bump: version 1.57.0 → 1.57.1 v1.57.1 2025-01-06 21:13:56 -08:00
Krrish Dholakia 16f13dd55c docs(prompt_management.md): update docs to show how to point to load balanced model name 2025-01-06 21:09:09 -08:00
Krish DholakiaandGitHub 0c3fef24cd Litellm dev 01 06 2025 p2 (#7597) 
* test(test_amazing_vertex_completion.py): fix test

* test: initial working code gecko test

* fix(vertex_ai_non_gemini.py): support vertex ai code gecko fake streaming

Fixes https://github.com/BerriAI/litellm/issues/7360

* test(test_get_model_info.py): add test for getting custom provider model info

Covers https://github.com/BerriAI/litellm/issues/7575

* fix(utils.py): fix get_provider_model_info check

Handle custom llm provider scenario

Fixes https://github.com/
BerriAI/litellm/issues/7575
 2025-01-06 21:04:49 -08:00
Krish DholakiaandGitHub b397dc1497 Litellm dev 01 06 2025 p3 (#7596) 
* build(model_prices_and_context_window.json): add gemini-1.5-pro 'supports_vision' = true

Fixes https://github.com/BerriAI/litellm/issues/7592

* build(model_prices_and_context_window.json): add new mistral models pricing + model info
 2025-01-06 20:44:04 -08:00
fzowlandGitHub c22d9208da Refresh VoyageAI models, prices and context (#7472) 
* Refresh VoyageAI models and prices and context

* Refresh VoyageAI models and prices and context

* Refresh VoyageAI models and prices and context
 2025-01-06 20:38:01 -08:00
Ishaan JaffandGitHub 819079f23b (proxy perf improvement) - remove redundant .copy() operation (#7564) 
* latency fix proxy

* remove useless copy in add_key_level_controls
 2025-01-06 20:36:47 -08:00
Ishaan JaffandGitHub 6125ba1e2b (Feat) - allow including dd-trace in litellm base image (#7587) 
* introduce USE_DDTRACE=true

* update dd tracer

* update

* bump dd trace

* use og slim image

* DD tracing

* fix _init_dd_tracer
 2025-01-06 17:27:09 -08:00
Ishaan JaffandGitHub 0b5c1392f7 fix _return_user_api_key_auth_obj (#7591) 2025-01-06 16:43:14 -08:00
Ishaan JaffandGitHub 61d67cfa43 (perf) - fixes for aiohttp handler to hit 1K RPS (#7590) 
* fix getting aiohttp sesson

* fix _get_async_client_session
 2025-01-06 15:41:39 -08:00
Krrish Dholakia 23685e93f3 test: skip tests pending vertex credentials v1.57.0 2025-01-05 15:29:51 -08:00