Commit Graph

19556 Commits

Author SHA1 Message Date
Ishaan Jaff ccfbb77b73 (Redis fix) - use mget_non_atomic (#8682)
* use mget_nonatomic

* redis cluster override MGET op

* fix redis cluster + MGET

* test redis cluster
2025-02-20 17:51:31 -08:00
Ishaan Jaff bb6f43d12e (Bug fix) - Cache Health not working when configured with prometheus service logger (#8687)
* fix serialize on safe json dumps

* test_non_standard_dict_keys_complex

* ui fix HealthCheckCacheParams

* fix HealthCheckCacheParams

* fix code qa

* test_cache_ping_failure

* test_cache_ping_health_check_includes_only_cache_attributes

* test_cache_ping_health_check_includes_only_cache_attributes
2025-02-20 13:41:56 -08:00
Krrish Dholakia 8f7437b4d9 bump: version 1.61.11 → 1.61.12 2025-02-19 22:22:32 -08:00
Krish Dholakia f9df01fbc6 fix(utils.py): handle token counter error when invalid message passed in (#8670)
* fix(utils.py): handle token counter error

* fix(utils.py): testing fixes

* fix(utils.py): fix incr for num tokens from list

* fix(utils.py): fix text str token counting
2025-02-19 22:21:34 -08:00
Krish Dholakia 982ee4b96b LiteLLM Contributor PRs (02/18/2025). (#8643)
* fix: prisma migration script logging #6991 (#8617)

* fix: prisma migration script logging #6991

* chore: refactor usng proxy logger

* remove noqa and move import below abs path

* Fix(litellm-vetexai-gemini): adding the supported files as per gemini documentation (#8559)

* bugfix(litellm-vetexai-gemini): adding the supported files as per gemini documentations

* fix(files.py): correct file type constant from 'text' to 'TXT'

* test: handle internal server error

---------

Co-authored-by: Justin Law <81255462+justinthelaw@users.noreply.github.com>
Co-authored-by: alymedhat10 <48028013+alymedhat10@users.noreply.github.com>
2025-02-19 21:52:46 -08:00
Krish Dholakia cc77138b37 Add all /key/generate api params to UI + add metadata fields on team AND org add/update (#8667)
* feat(create_key_button.tsx): initial commit using openapi.json to ensure all values via api are supported on ui for `/key/generate`

Closes https://github.com/BerriAI/litellm/issues/7763

* style(create_key_button.tsx): put openapi settings inside 'advanced setting' accordion

* fix(check_openapi_schema.tsx): style improvements for advanced settings

* style(create_key_button.tsx): add tooltip explaining what the settings mean

* fix(team_info.tsx): render metadata field on team update

allow updating a team's metadata

* fix(networking.tsx): add 'metadata' field to create team form

* refactor: cleanup dead codeblock

* fix(organization_endpoints.py): fix metadata param support on `/organization/new`

* feat(organization_endpoints.py): support updating metadata for organization on api + ui

* test: mark flaky test
v1.61.11-nightly
2025-02-19 21:13:06 -08:00
Krish Dholakia 40d1576292 extract <think>..</think> block for amazon deepseek r1 and put in reasoning_content (#8664)
* feat(amazon_deepseek_transformation.py): extract thinking content and return in same way as deepseek api

* feat(amazon_deepseek_transformation.py): extract thinking block from deepseek r1 streaming block

* fix: suppress linting warning

* fix: fix linting error

* fix: fix linting error
2025-02-19 21:10:59 -08:00
Krrish Dholakia 9470f57e86 build: extract <think>..</think> block for amazon deepseek r1 and put in reasoning_content 2025-02-19 21:10:38 -08:00
Ishaan Jaff 1dfdad1707 bump: version 1.61.10 → 1.61.11 2025-02-19 21:07:01 -08:00
Ishaan Jaff 1460a79ef9 ui new build 2025-02-19 20:09:51 -08:00
Ishaan Jaff 0d2b0ee1b7 (Bug fix) prometheus - safely set latency metrics (#8669)
* use safe_duration_seconds

* _safe_duration_seconds

* test_set_latency_metrics_missing_timestamps
2025-02-19 20:08:46 -08:00
Ishaan Jaff 045cf3f9e2 (Bug Fix Redis) - Fix running redis.mget operations with None Keys (#8666)
* async_batch_get_cache

* test_batch_get_cache_with_none_keys

* async_batch_get_cache

* fix linting error
2025-02-19 19:56:57 -08:00
Ishaan Jaff fff15543d9 (UI + Proxy) Cache Health Check Page - Cleanup/Improvements (#8665)
* fixes for redis cache ping serialization

* fix cache ping check

* fix cache health check ui

* working error details on ui

* ui expand / collapse error

* move cache health check to diff file

* fix displaying error from cache health check

* ui allow copying errors

* ui cache health fixes

* show redis details

* clean up cache health page

* ui polish fixes

* fix error handling on cache health page

* fix redis_cache_params on cache ping response

* error handling

* cache health ping response

* fx error response from cache ping

* parsedLitellmParams

* fix cache health check

* fix cache health page

* cache safely handle json dumps issues

* test caching routes

* test_primitive_types

* fix caching routes

* litellm_mapped_tests

* fix pytest-mock

* fix _serialize

* fix linting on safe dumps

* test_default_max_depth

* pip install "pytest-mock==3.12.0"

* litellm_mapped_tests_coverage

* add readme on new litellm test dir
2025-02-19 19:08:50 -08:00
Krrish Dholakia 39db3147e8 fix(spend_tracking_utils.py): move info to debug 2025-02-19 15:36:32 -08:00
Krrish Dholakia f2c32bb1c7 bump: version 1.61.9 → 1.61.10 2025-02-18 22:56:17 -08:00
Krish Dholakia 0ff56504d7 build: build ui (#8654) 2025-02-18 22:56:01 -08:00
Krish Dholakia 0319559295 fix(team_endpoints.py): allow team member to view team info (#8644)
* fix(team_endpoints.py): allow team member to view team info

* test: handle model overloaded in tool calling test

* test: handle internal server error
2025-02-18 22:28:57 -08:00
Ishaan Jaff 0d398c2295 test_message_with_name v1.61.9-nightly 2025-02-18 21:53:47 -08:00
Ishaan Jaff 4828fb1544 fix code quality 2025-02-18 21:29:23 -08:00
Ishaan Jaff ed87739561 bump: version 1.61.8 → 1.61.9 2025-02-18 21:27:24 -08:00
Ishaan Jaff 6cae24ed08 ui new build 2025-02-18 21:26:06 -08:00
Ishaan Jaff 7e722225a1 ui QA fix, adding wildcard models 2025-02-18 21:25:23 -08:00
Ishaan Jaff c18dd61044 ui linting fix 2025-02-18 21:14:36 -08:00
Ishaan Jaff 889feb2ea8 patch on LiteLLM_AuditLogs 2025-02-18 21:13:14 -08:00
Ishaan Jaff b88762b63c (Polish/Fixes) - Fixes for Adding Team Specific Models (#8645)
* refactor get model info for team models

* allow adding a model to a team when creating team specific model

* ui update selected Team on Team Dropdown

* test_team_model_association

* testing for team specific models

* test_get_team_specific_model

* test: skip on internal server error

* remove model alias card on teams page

* linting fix _get_team_specific_model

* fix DeploymentTypedDict

* fix linting error

* fix code quality

* fix model info checks

---------

Co-authored-by: Krrish Dholakia <krrishdholakia@gmail.com>
2025-02-18 21:11:57 -08:00
Krish Dholakia 2b7755f8d8 Litellm dev 02 18 2025 p3 (#8640)
* fix(team_endpoints.py): cleanup user <-> team association on team delete

Fixes issue where user table still listed team membership post delete

* test(test_team.py): update e2e test - ensure user/team membership is deleted on team delete

* fix(base_invoke_transformation.py): fix deepseek r1 transformation

remove deepseek name from model url

* test(test_completion.py): assert model route not in url

* feat(base_invoke_transformation.py): infer region name from model arn

prevent errors due to different region name in env var vs. model arn, respect if explicitly set in call though

* test: fix test

* test: skip on internal server error
2025-02-18 19:14:20 -08:00
Krish Dholakia ef87b0eaae Litellm dev 02 18 2025 p2 (#8639)
* fix(parallel_request_limiter.py): improve single instance rate limiting by updating in-memory cache instantly

Fixes issue where parallel request limiter had a leak

* fix(parallel_request_limiter.py): fix parallel request limiter to not decrement val on max limit being reached

* test(test_parallel_request_limiter.py): fix test

* test: fix test

* fix(parallel_request_limiter.py): move to using common enum

* test: fix test
2025-02-18 19:12:16 -08:00
Krrish Dholakia 89c8dfc644 test: handle unstable tests 2025-02-18 18:44:12 -08:00
Krish Dholakia 4e38940694 Add OSS license check to ci/cd (#8626)
* feat: add oss license check for related packages

ensure all packages used are permissable

* ci(config.yml): add check_licenses.py

script to ci/cd

* ci(check_licenses.py): fix license check
2025-02-18 18:11:08 -08:00
elroy-bot 7f47ae88b7 Add Elroy to projects built with litellm (#8642)
Co-authored-by: Tom Bedor <tombedor@gmail.com>
2025-02-18 16:43:30 -08:00
Krish Dholakia 2b71973b17 Litellm dev 02 18 2025 p1 (#8630)
* fix(filter.tsx): align filter icon to button correctly

* style: style improvements for filter icon

* style(filter.tsx): cleanup filter box

* style(filter.tsx): style improvement for team id box on filter

* Fix timeout bug for SageMaker Messages API completion (#8635)

* fix(model_cost_map): fix json parse error on model cost map + add unit test (#8629)

Fixes https://github.com/BerriAI/litellm/pull/8619#issuecomment-2666693045

* Fix timeout bug for SageMaker Messages API completion

---------

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>

---------

Co-authored-by: Bobby Lindsey <bobbywlindsey@users.noreply.github.com>
2025-02-18 15:20:17 -08:00
Ishaan Jaff d1ba04d9d9 [Feature]: Redis Caching - Allow setting a namespace for redis cache (#8624)
* use _add_namespace_to_cache_key

* fix cache_control_args

* test_redis_caching_multiple_namespaces

* test_add_namespace_to_cache_key

* test_redis_caching_multiple_namespaces

* docs redis name space

* test_add_namespace_to_cache_key
2025-02-18 14:47:34 -08:00
Krish Dholakia ea985dda0b fix(model_cost_map): fix json parse error on model cost map + add unit test (#8629)
Fixes https://github.com/BerriAI/litellm/pull/8619#issuecomment-2666693045
2025-02-18 11:18:16 -08:00
Krish Dholakia 49610485a2 feat: add oss license check for related packages (#8623)
ensure all packages used are permissable
2025-02-18 10:40:00 -08:00
HeMuling 369b636b5a add openrouter/google/gemini-2.0-flash-001 (#8619) 2025-02-18 10:14:49 -08:00
Ishaan Jaff 8d7937fd7a (Fix) Redis async context usage for Redis Cluster + 94% lower median latency when using Redis Cluster (#8622)
* fix redis async usage

* re- use redis client across requests
2025-02-18 09:22:27 -08:00
Krish Dholakia 2340f1b31f Pass router tags in request headers - x-litellm-tags (#8609)
* feat(litellm_pre_call_utils.py): support `x-litellm-tags` request header

allow tag based routing + spend tracking via request headers

* docs(request_headers.md): document new `x-litellm-tags` for tag based routing and spend tracking

* docs(tag_routing.md): add to docs

* fix(utils.py): only pass str values for openai metadata param

* fix(utils.py): drop non-str values for metadata param to openai

preview-feature, otel span was being sent in
2025-02-18 08:26:22 -08:00
Krrish Dholakia 7bfd816d3b build: merge commit 1b15568af7f8923a014e6e49f5ec09e5abf5f981
Author: Krrish Dholakia <krrishdholakia@gmail.com>
Date:   Mon Feb 17 21:37:36 2025 -0800

    fix(proxy/_types.py): fix linting error

commit dc4d5cffa69bbcde5a3de17aa7dc934238d81034
Author: Krrish Dholakia <krrishdholakia@gmail.com>
v1.61.7.dev1
2025-02-17 21:56:00 -08:00
Ishaan Jaff e8f387200a ci/cd run again 2025-02-17 21:37:52 -08:00
Ishaan Jaff 52450e9f61 test_openai_fine_tuning v1.61.8-nightly 2025-02-17 21:29:59 -08:00
Ishaan Jaff 296013f722 test_openai_fine_tuning 2025-02-17 21:20:09 -08:00
Ishaan Jaff 04ebca1166 test_openai_fine_tuning 2025-02-17 21:12:40 -08:00
Ishaan Jaff 0ee54908c1 bump: version 1.61.7 → 1.61.8 2025-02-17 21:10:21 -08:00
Ishaan Jaff 3d0f67b260 ui new build 2025-02-17 18:46:09 -08:00
Ishaan Jaff afa2a2b0a3 fix linting 2025-02-17 18:44:46 -08:00
Ishaan Jaff 8024300825 (UI) Improvements to Add Team Model Flow (#8603)
* ui - use common team dropdown component

* re-use team component

* rename org field on add model

* handle add model submit

* working view model_id and team_id on root models page

* cleaner

* show all fields

* working model info view

* working team info selector

* clean up team id

* new component for model dashboard

* ui show table with dropdown

* make public model names like email

* revert changes to litellm model name

* fix litellm model name

* ui fix public model

* fix mappings

* fix conditional text input

* fix message

* ui fix bulk add models

* _add_team_model_to_db

* move model mgmt helper funcs

* test_add_team_model_to_db

* ui - display model team model name

* fix add model tab

* fix remove redundant info tab on models page

* dont pass model mappings all the way through

* fix jarring model name when adding team models

* fix edit model button

* delete button on model info

* ui fix model dashboard

* fix DeploymentTypedDict

* _is_model_access_group_for_wildcard_route

* test _get_public_model_name

* ui fix viewing public model name

* fix linting error

* fix linting errors

* fix selectedModel logic
2025-02-17 18:37:14 -08:00
Ishaan Jaff 4a1476792c (UI) Refactor Add Models for Specific Teams (#8592)
* ui - use common team dropdown component

* re-use team component

* rename org field on add model

* handle add model submit

* working view model_id and team_id on root models page

* cleaner

* show all fields

* working model info view

* working team info selector

* clean up team id

* new component for model dashboard

* ui show table with dropdown

* make public model names like email

* revert changes to litellm model name

* fix litellm model name

* ui fix public model

* fix mappings

* fix conditional text input

* fix message

* ui fix bulk add models
2025-02-17 17:58:29 -08:00
Ishaan Jaff 341a63cf8b (UI) Allow adding models for a Team (#8598) (#8601)
* ui - use common team dropdown component

* re-use team component

* rename org field on add model

* handle add model submit

* working view model_id and team_id on root models page

* cleaner

* show all fields

* working model info view

* working team info selector

* clean up team id
2025-02-17 17:57:56 -08:00
Krrish Dholakia ce1d8026d9 bump: version 1.61.6 → 1.61.7 v1.61.7 v1.61.7-nightly 2025-02-17 17:56:44 -08:00
Krrish Dholakia d328140676 build: build ui 2025-02-17 17:56:31 -08:00