Commit Graph
18994 Commits
Author SHA1 Message Date
Krrish Dholakia d988bfb6f8 docs(enterprise.md): clarify sla for patching vulnerabilities 2025-01-11 10:42:32 -08:00
Krish DholakiaandGitHub 5e537fbdb1 fix(model_hub.tsx): clarify cost in model hub is per 1m tokens (#7687)
* fix(model_hub.tsx): clarify cost in model hub is per 1m tokens

* docs: test blog

* docs: improve release note docs

* docs(docs/): new stable release doc

* docs(docs/): specify date in all posts

* docs(docs/): add git diff to stable release docs
2025-01-11 09:57:09 -08:00
Krrish Dholakia 9a1c050cf7 docs: new release notes 2025-01-10 22:49:20 -08:00
Krrish Dholakia f2ca244766 docs(logging.md): docs(logging.md): add docs on s3 bucket logging with team alias prefix 2025-01-10 22:28:05 -08:00
Krish DholakiaandGitHub 599730960a build: new ui build (#7685) 2025-01-10 22:12:17 -08:00
Krrish Dholakia 1f7f11f054 bump: version 1.57.8 → 1.57.9 2025-01-10 21:57:29 -08:00
Krish DholakiaandGitHub 27892acdfc Litellm dev 01 10 2025 p3 (#7682)
* feat(langfuse.py): log the used prompt when prompt management used

* test: fix test

* docs(self_serve.md): add doc on restricting personal key creation on ui

* feat(s3.py): support s3 logging with team alias prefixes (if available)

New preview feature

* fix(main.py): remove old if block - simplify to just await if coroutine returned

fixes lm_studio async embedding error

* fix(langfuse.py): handle get prompt check
2025-01-10 21:56:42 -08:00
Krish DholakiaandGitHub c4780479a9 Litellm dev 01 10 2025 p2 (#7679)
* test(test_basic_python_version.py): assert all optional dependencies are marked as extras on poetry

Fixes https://github.com/BerriAI/litellm/issues/7677

* docs(secret.md): clarify 'read_and_write' secret manager usage on aws

* docs(secret.md): fix doc

* build(ui/teams.tsx): add edit/delete button for updating user / team membership on ui

allows updating user role to admin on ui

* build(ui/teams.tsx): display edit member component on ui, when edit button on member clicked

* feat(team_endpoints.py): support updating team member role to admin via api endpoints

allows team member to become admin post-add

* build(ui/user_dashboard.tsx): if team admin - show all team keys

Fixes https://github.com/BerriAI/litellm/issues/7650

* test(config.yml): add tomli to ci/cd

* test: don't call python_basic_testing in local testing (covered by python 3.13 testing)
2025-01-10 21:50:53 -08:00
Ishaan Jaff 49d74748b0 fix showing release notes 2025-01-10 20:40:50 -08:00
Ishaan Jaff 189b677600 bump: version 1.57.7 → 1.57.8 v1.57.8 2025-01-10 20:39:21 -08:00
Ishaan JaffandGitHub 02f5c44a35 [Bug fix]: Proxy Auth Layer - Allow Azure Realtime routes as llm_api_routes (#7684)
* fix route check azure realtime endpoints

* test_is_llm_api_route

* fix /realtime

* test_routes_on_litellm_proxy
2025-01-10 20:38:06 -08:00
Ishaan JaffandGitHub 2d1c90b688 fix proxy pre call hook - only use if user is using alerting (#7683) 2025-01-10 19:07:05 -08:00
Ishaan JaffandGitHub 9ac18caf24 uvicorn allow setting num workers (#7681) 2025-01-10 19:03:14 -08:00
Ishaan JaffandGitHub 5c870c0c51 (performance improvement - litellm sdk + proxy) - ensure litellm does not create unnecessary threads when running async functions (#7680)
* fix handle_sync_success_callbacks_for_async_calls

* fix handle_sync_success_callbacks_for_async_calls

* fix linting / testing errors

* use handle_sync_success_callbacks_for_async_calls

* add unit testing for logging fixes
2025-01-10 17:57:22 -08:00
Krish DholakiaandGitHub a3e65c9bcb LiteLLM Minor Fixes & Improvements (01/10/2025) - p1 (#7670)
* test(test_get_model_info.py): add unit test confirming router deployment updates global 'get_model_info'

* fix(get_supported_openai_params.py): fix custom llm provider 'get_supported_openai_params'

Fixes https://github.com/BerriAI/litellm/issues/7668

* docs(azure.md): clarify how azure ad token refresh on proxy works

Closes https://github.com/BerriAI/litellm/issues/7665
2025-01-10 17:49:05 -08:00
Hugues ChocartandGitHub 8576ca8ccb feat: allow to pass custom parent run id (#7651) 2025-01-10 17:04:46 -08:00
Ishaan JaffandGitHub 9174a6f349 (litellm sdk - perf improvement) - optimize pre_call_check (#7673)
* latency fix - litellm sdk

* fix linting error

* fix litellm logging
2025-01-10 14:16:39 -08:00
Ishaan JaffandGitHub c999b4efe1 (litellm sdk - perf improvement) - use O(1) set lookups for checking llm providers / models (#7672)
* fix get model info logic to use O(1) lookups

* perf - use O(1) lookup for get llm provider
2025-01-10 14:16:30 -08:00
Ishaan JaffandGitHub b3bd15e35a speed up use_custom_pricing_for_model (#7674) 2025-01-10 14:00:48 -08:00
Ishaan JaffandGitHub af08a0caed latency fix _cache_key_object (#7676) 2025-01-10 13:59:26 -08:00
Ishaan Jaff 00a0f56565 bump: version 1.57.6 → 1.57.7 v1.57.7 2025-01-10 13:48:11 -08:00
Ishaan Jaff 4c608a1a32 fix uvloop critical fix 2025-01-10 13:47:18 -08:00
Krrish Dholakia e98c1b86f4 docs(config_settings.md): update docs to include new athina env var 2025-01-10 10:46:12 -08:00
vivek-athinaandGitHub 8e2653c609 Use environment variable for Athina logging URL (#7628)
* Use environment variable for Athina logging URL

* Added to docs as well

* Changed the env var name
2025-01-10 07:47:12 -08:00
Krish DholakiaandGitHub c10ae8879e fix(vertex_ai/gemini/transformation.py): handle 'http://' in gemini p… (#7660)
* fix(vertex_ai/gemini/transformation.py): handle 'http://' in gemini process url

* refactor(router.py): refactor '_prompt_management_factory' to use logging obj get_chat_completion logic

deduplicates code

* fix(litellm_logging.py): update 'get_chat_completion_prompt' to update logging object messages

* docs(prompt_management.md): update prompt management to be in beta

given feedback - this still needs to be revised (e.g. passing in user message, not ignoring)

* refactor(prompt_management_base.py): introduce base class for prompt management

allows consistent behaviour across prompt management integrations

* feat(prompt_management_base.py): support adding client message to template message + refactor langfuse prompt management to use prompt management base

* fix(litellm_logging.py): log prompt id + prompt variables to langfuse if set

allows tracking what prompt was used for what purpose

* feat(litellm_logging.py): log prompt management metadata in standard logging payload + use in langfuse

allows logging prompt id / prompt variables to langfuse

* test: fix test

* fix(router.py): cleanup unused imports

* fix: fix linting error

* fix: fix trace param typing

* fix: fix linting errors

* fix: fix code qa check
2025-01-10 07:31:59 -08:00
Krish DholakiaandGitHub 865e6d5bda fix(main.py): fix lm_studio/ embedding routing (#7658)
* fix(main.py): fix lm_studio/ embedding routing

adds the mapping + updates docs with example

* docs(self_serve.md): update doc to show how to auto-add sso users to teams

* fix(streaming_handler.py): simplify async iterator check, to just check if streaming response is an async iterable
2025-01-09 23:03:24 -08:00
Krrish Dholakia efe236d05b bump: version 1.57.5 → 1.57.6 2025-01-09 22:56:50 -08:00
Krrish Dholakia 3e3c78a200 build(ui/): update ui build 2025-01-09 22:44:05 -08:00
Krish DholakiaandGitHub 63926f484c feat(ui_sso.py): Allows users to use test key pane, and have team budget limits be enforced for their use-case (#7666) 2025-01-09 22:12:45 -08:00
Ishaan JaffandGitHub 6df8faf15f (minor latency fixes / proxy) - use verbose_proxy_logger.debug() instead of litellm.print_verbose (#7664)
* minor latency fixes

* fix code quality
2025-01-09 21:06:09 -08:00
Ishaan Jaff 7fcd130e1d bump: version 1.57.4 → 1.57.5 v1.57.5 2025-01-09 20:05:42 -08:00
Ishaan JaffandGitHub 51b17be3ab use asyncio tasks for logging db metrics (#7663) 2025-01-09 19:59:32 -08:00
Ishaan JaffandGitHub 13f364682d (Feat - Batches API) add support for retrieving vertex api batch jobs (#7661)
* add _async_retrieve_batch

* fix aretrieve_batch

* fix _get_batch_id_from_vertex_ai_batch_response

* fix batches docs
2025-01-09 18:35:03 -08:00
Ishaan JaffandGitHub 2507c275f6 (proxy perf improvement) - use uvloop for higher RPS (10%-20% higher RPS) (#7662)
* uvicorn use uvloop

* fix uvloop==0.21.0

* add uvloop to pyproject

* test_completion_response_ratelimit_headers
2025-01-09 18:11:20 -08:00
Ishaan JaffandGitHub a85de46ef7 (proxy - RPS) - Get 2K RPS at 4 instances, minor fix aiohttp_openai/ (#7659)
* speed up transform_response

* use 2 workers

* undo changes to uvicorn

* ci/cd run again
2025-01-09 17:24:18 -08:00
Ishaan JaffandGitHub fd46482916 fix 1 - latency fix (#7655) 2025-01-09 15:57:05 -08:00
Krish DholakiaandGitHub 907bcd3a62 Litellm dev 01 08 2025 p1 (#7640)
* feat(ui_sso.py): support reading team ids from sso token

* feat(ui_sso.py): working upsert sso user teams membership in litellm - if team exists

Adds user to relevant teams, if user is part of teams and team exists on litellm

* fix(ui_sso.py): safely handle add team member task

* build(ui/): support setting team id when creating team on UI

* build(ui/): teams.tsx

allow setting team id on ui

* build(circle_ci/requirements.txt): add fastapi-sso to ci/cd testing

* fix: fix linting errors
2025-01-08 22:08:20 -08:00
Krish DholakiaandGitHub 1e3370f3cb LiteLLM Minor Fixes & Improvements (01/08/2025) - p2 (#7643)
* fix(streaming_chunk_builder_utils.py): add test for groq tool calling + streaming + combine chunks

Addresses https://github.com/BerriAI/litellm/issues/7621

* fix(streaming_utils.py): fix modelresponseiterator for openai like chunk parser

ensures chunk parser uses the correct tool call id when translating the chunk

 Fixes https://github.com/BerriAI/litellm/issues/7621

* build(model_hub.tsx): display cost pricing on model hub

* build(model_hub.tsx): show cost per token pricing + complete model information

* fix(types/utils.py): fix usage object handling
2025-01-08 19:45:19 -08:00
Krrish Dholakia 39ee4c6bb4 docs(intro.md): add a section on 'why pass through endpoints'
helps proxy admin understand when these would be useful
2025-01-08 19:15:41 -08:00
Krrish Dholakia 3193cb3912 build(model_prices_and_context_window.json): omni-moderation-latest-intents 2025-01-08 19:06:04 -08:00
Ishaan Jaff 782b59788f bump: version 1.57.3 → 1.57.4 v1.57.4 2025-01-08 18:50:11 -08:00
Ishaan JaffandGitHub 7259452285 (helm) - bug fix - allow using migrationJob.enabled variable within job (#7639)
* wrap migration job in conditional

* use enabled flag in values.yaml

* run migration job

* argo cd fixes

* fix values in image repo
2025-01-08 18:47:33 -08:00
Ishaan JaffandGitHub 48d4f79206 fix is llm api route check (#7631) 2025-01-08 18:45:59 -08:00
Ishaan Jaff 9eb24df113 ci/cd run again 2025-01-08 18:36:39 -08:00
Krish DholakiaandGitHub 4af23353d6 Allow assigning teams to org on UI + OpenAI omni-moderation cost model tracking (#7566)
* feat(cost_calculator.py): add cost tracking ($0) for openai moderations endpoint

removes sentry cost tracking errors caused by this

* build(teams.tsx): allow assigning teams to orgs
2025-01-08 16:58:21 -08:00
0ffc5379ea Litellm dev 01 07 2025 p2 (#7622)
* build(ui/): update ui

* fix: drop unsupported non-whitespace characters for real when calling… (#7484)

* fix: drop unsupported non-whitespace characters for real when calling anthropic with stop sequences

* test: add parameterized test for _map_stop_sequences method in AnthropicConfig

---------

Co-authored-by: Wolfram Ravenwolf <52386626+WolframRavenwolf@users.noreply.github.com>
2025-01-08 16:56:39 -08:00
Ishaan JaffandGitHub fd0a03f719 (feat) - allow building litellm proxy from pip package (#7633)
* fix working build from pip

* add tests for proxy_build_from_pip_tests

* doc clean up for deployment

* docs cleanup

* docs build from pip

* fix cd docker/build_from_pip
2025-01-08 16:36:57 -08:00
Ishaan Jaff 43566e9842 fix docs 2025-01-08 12:51:59 -08:00
Ishaan Jaff e5717d2cb0 update load test docs 2025-01-08 12:48:21 -08:00
Ishaan Jaff 74b41d29d3 sort rn 2025-01-08 12:16:01 -08:00