Commit Graph

19071 Commits

Author SHA1 Message Date
Ishaan Jaff 7cfda2fb76 bump: version 1.58.3 → 1.58.4 v1.58.4 2025-01-17 08:29:13 -08:00
Ishaan Jaff 2d5f8ea2c6 Revert "fix custom logger"
This reverts commit 9d2707ecfe.
2025-01-17 08:26:59 -08:00
Ishaan Jaff 9d2707ecfe fix custom logger 2025-01-17 07:39:49 -08:00
Ishaan Jaff 632ba92af1 Revert "fix: fix test"
This reverts commit 0642a78abb.
2025-01-17 07:21:19 -08:00
Ishaan Jaff b30e05b54f Revert "test_completion_mistral_api_mistral_large_function_call"
This reverts commit ef9177f0a8.
2025-01-17 07:20:46 -08:00
Krrish Dholakia 0642a78abb fix: fix test 2025-01-17 07:16:32 -08:00
Ishaan Jaff 15e0a605e4 _handle_tool_call_message linting 2025-01-16 22:34:16 -08:00
Ishaan Jaff c8febaca2e test_watsonx_token_in_env_var 2025-01-16 22:28:37 -08:00
Ishaan Jaff 7f63e7c15a test_completion_mistral_api_mistral_large_function_call 2025-01-16 22:27:48 -08:00
Ishaan Jaff f3268b6b06 sec fix minor (#7810) 2025-01-16 22:03:28 -08:00
Ishaan Jaff 9b944ca60c (Fix + Testing) - Add dd-trace-run to litellm ci/cd pipeline + fix bug caused by dd-trace patching OpenAI sdk (#7820)
* add dd trace to e2e docker run tests

* update dd trace v

* fix entrypoint

* dd trace fixes

* proxy_build_from_pip_tests

* build python3.13

* use py 3.13

* fix build from pip

* dd trace fix

* proxy_build_from_pip_tests

* bump build from pip
2025-01-16 22:03:09 -08:00
Ishaan Jaff b492551d3d (fix) IBM Watsonx using ZenApiKey (#7821)
* ibm watsonx fix

* test ZenAPIKey

* fix zenapikey
2025-01-16 22:02:36 -08:00
Ishaan Jaff 939e1c9b19 (datadog llm observability) - fixes + improvements for using datadog llm observability logging integration (#7824)
* dd llm obs fixes

* _ensure_string_content

* fix _get_dd_llm_obs_payload_metadata
2025-01-16 22:02:24 -08:00
Ishaan Jaff 5b36985c00 run ci/cd again 2025-01-16 22:02:03 -08:00
Ishaan Jaff ef9177f0a8 test_completion_mistral_api_mistral_large_function_call 2025-01-16 21:50:56 -08:00
Ishaan Jaff 117256d264 test_async_vertexai_streaming_response 2025-01-16 21:45:12 -08:00
Ishaan Jaff de41abeace llama-v3p1-8b-instruct 2025-01-16 21:34:42 -08:00
Ishaan Jaff 5458a2ff33 fireworks ai use llama-v3p1-8b-instruct 2025-01-16 21:28:44 -08:00
Krrish Dholakia 8ab1335ae0 test: fix unit test 2025-01-16 21:11:17 -08:00
Ishaan Jaff 2f38e72026 test commit on main 2025-01-16 20:52:55 -08:00
Krish Dholakia 4ec2c2c8e4 build: bump certifi version - see if that fixes asyncio ssl issue on python 3.13 (#7800) 2025-01-15 22:02:45 -08:00
Krish Dholakia c57266c9dc test: initial commit enforcing testing on all anthropic pass through … (#7794)
* test: initial commit enforcing testing on all anthropic pass through functions

prevents future regressions

* test(test_unit_test_anthropic_pass_through.py): add unit test for '_get_user_from_metadata' function

* test(test_unit_test_anthropic_passthrough.py): add unit test for handle_logging_anthropic_collected_chunks

* test(test_unit_test_anthropic_pass_through): add coverage for all anthropic pass through functions
2025-01-15 22:02:35 -08:00
Krish Dholakia 843cd3b7c6 test: initial test to enforce all functions in user_api_key_auth.py h… (#7797)
* test: initial test to enforce all functions in user_api_key_auth.py have direct testing

* test(test_user_api_key_auth.py): add is_allowed_route unit test

* test(test_user_api_key_auth.py): add more tests

* test(test_user_api_key_auth.py): add complete testing coverage for all functions in `user_api_key_auth.py`

* test(test_db_schema_changes.py): add a unit test to ensure all db schema changes are backwards compatible

gives user an easy rollback path

* test: fix schema compatibility test filepath

* test: fix test
2025-01-15 21:52:45 -08:00
Krish Dholakia d4ed985173 Add back in non root image fixes (#7781) (#7795)
* Add back in non root image fixes (#7781)

* Add back in non root image fixes

* Fix dockerfile

* Fix perms

* Add in container structure tests for the nonroot image (#7796)

* feat(helm): add securityContext and pull policy values to migration job (#7652)

* fix(helm): corrected indentation in migration-job.yaml

* feat(helm): add securityContext and pull policy values to migration job

* fix confusing save button label (#7778)

* [integrations/lunary] Improve Lunary documentaiton (#7770)

* update lunary doc

* better title

* tweaks

* Update langchain.md

* Update lunary_integration.md

* Fix wrong URL for internal user invitation (#7762)

* format

* done

* Update instructor tutorial (#7784)

* Add in container structure tests for the nonroot image

---------

Co-authored-by: Zackeus Bengtsson <32719220+Hexoplon@users.noreply.github.com>
Co-authored-by: yujonglee <yujonglee.dev@gmail.com>
Co-authored-by: Hugues Chocart <chocart.hugues@icloud.com>
Co-authored-by: Nikolaiev Dmytro <dima.nikol.99@gmail.com>

---------

Co-authored-by: Rajat Vig <rajatvig@users.noreply.github.com>
Co-authored-by: Zackeus Bengtsson <32719220+Hexoplon@users.noreply.github.com>
Co-authored-by: yujonglee <yujonglee.dev@gmail.com>
Co-authored-by: Hugues Chocart <chocart.hugues@icloud.com>
Co-authored-by: Nikolaiev Dmytro <dima.nikol.99@gmail.com>
2025-01-15 21:49:03 -08:00
Krish Dholakia 80d6bbec29 Litellm dev 01 14 2025 p2 (#7772)
* feat(pass_through_endpoints.py): fix anthropic end user cost tracking

* fix(anthropic/chat/transformation.py): use returned provider model for anthropic

handles anthropic `-latest` tag in request body throwing cost calculation errors

ensures we can be accurate in our model cost tracking

* feat(model_prices_and_context_window.json): add gemini-2.0-flash-thinking-exp pricing

* test: update test to use assumption that user_api_key_dict can get anthropic user id

* test: fix test

* fix: fix test

* fix(anthropic_pass_through.py): uncomment previous anthropic end-user cost tracking code block

can't guarantee user api key dict always has end user id - too many code paths

* fix(user_api_key_auth.py): this allows end user id from request body to always be read and set in auth object

* fix(auth_check.py): fix linting error

* test: fix auth check

* fix(auth_utils.py): fix get end user id to handle metadata = None
2025-01-15 21:34:50 -08:00
Ishaan Jaff 4081aeb15e (helm) - allow specifying envVars on values.yaml + add helm lint test (#7789)
* litellm use envVars values.yaml

* fix values.yaml

* add helm lint to ci/cd pipeline

* working values.yaml

* add helm tests to ci/cd

* fix helm chart testing

* update helm tests

* fix helm test

* fix use test values in ci

* fix busy box testing on helm

* fix test-values.yaml

* update helm tests

* fix test connection
2025-01-15 21:11:17 -08:00
Nikolaiev Dmytro 7b45349145 Update instructor tutorial (#7784) 2025-01-15 15:10:50 -08:00
yujonglee 503b38e609 Fix wrong URL for internal user invitation (#7762)
* format

* done
2025-01-15 15:10:32 -08:00
Hugues Chocart 6fff77d131 [integrations/lunary] Improve Lunary documentaiton (#7770)
* update lunary doc

* better title

* tweaks

* Update langchain.md

* Update lunary_integration.md
2025-01-15 15:00:25 -08:00
yujonglee 01357add4d fix confusing save button label (#7778) 2025-01-15 13:51:07 -08:00
Zackeus Bengtsson 801f450e8c feat(helm): add securityContext and pull policy values to migration job (#7652)
* fix(helm): corrected indentation in migration-job.yaml

* feat(helm): add securityContext and pull policy values to migration job
2025-01-15 12:51:08 -08:00
Krrish Dholakia 8c26e1a66d build: bump: version 1.58.2 → 1.58.3 2025-01-15 07:46:57 -08:00
Krish Dholakia fe60a38c8e Litellm dev 01 2025 p4 (#7776)
* fix(gemini/): support gemini 'frequency_penalty' and 'presence_penalty'

Closes https://github.com/BerriAI/litellm/issues/7748

* feat(proxy_server.py): new env var to disable prisma health check on startup

* test: fix test
2025-01-14 21:49:25 -08:00
Krish Dholakia 8353caa485 build(pyproject.toml): bump uvicorn depedency requirement (#7773)
* build(pyproject.toml): bump uvicorn depedency requirement

Fixes https://github.com/BerriAI/litellm/issues/7768

* fix(anthropic/chat/transformation.py): fix is_vertex_request check to actually use optional param passed in

Fixes https://github.com/BerriAI/litellm/issues/6898#issuecomment-2590860695

* fix(o1_transformation.py): fix azure o1 'is_o1_model' check to just check for o1 in model string

https://github.com/BerriAI/litellm/issues/7743

* test: load vertex creds
2025-01-14 21:47:11 -08:00
Ishaan Jaff 96b70eb209 bump: version 1.58.1 → 1.58.2 v1.58.2 2025-01-14 20:18:29 -08:00
Ishaan Jaff 30bb4c4cdd (fix) BaseAWSLLM - cache IAM role credentials when used (#7775)
* fix base aws llm

* fix auth with aws role

* test aws base llm

* fix base aws llm init

* run ci/cd again

* fix get_credentials

* ci/cd run again

* _auth_with_aws_role
2025-01-14 20:16:22 -08:00
Ishaan Jaff 5fbbf47581 (Feat) prometheus - emit remaining team budget metric on proxy startup (#7777)
* fix get_paginated_teams

* use _initialize_remaining_budget_metrics

* fix prom metric

* run ci/cd again

* fix run async func

* fix _initialize_prometheus_startup_metrics

* fix _initialize_prometheus_startup_metrics

* prom unit tests

* test_get_paginated_teams
2025-01-14 20:08:23 -08:00
Ishaan Jaff df7d500d42 docs iam role based access for bedrock (#7774) 2025-01-14 19:02:02 -08:00
Krish Dholakia 35919d9fec Litellm dev 01 13 2025 p2 (#7758)
* fix(factory.py): fix bedrock document url check

Make check more generic - if starts with 'text' or 'application' assume it's a document and let it go through

 Fixes https://github.com/BerriAI/litellm/issues/7746

* feat(key_management_endpoints.py): support writing new key alias to aws secret manager - on key rotation

adds rotation endpoint to aws key management hook - allows for rotated litellm virtual keys with new key alias to be written to it

* feat(key_management_event_hooks.py): support rotating keys and updating secret manager

* refactor(base_secret_manager.py): support rotate secret at the base level

since it's just an abstraction function, it's easy to implement at the base manager level

* style: cleanup unused imports
2025-01-14 17:04:01 -08:00
Krish Dholakia 7b27cfb0ae Support temporary budget increases on keys (#7754)
* fix(gpt_transformation.py): fix response_format translation check for 4o models

Fixes https://github.com/BerriAI/litellm/issues/7616

* feat(key_management_endpoints.py): support 'temp_budget_increase' and 'temp_budget_expiry' fields

Allow proxy admin to grant temporary budget increases to keys

* fix(proxy/_types.py): enforce temp_budget_increase and temp_budget_expiry are always passed together

* feat(user_api_key_auth.py): initial working temp budget increase logic

ensures key budget exceeded error checks for temp budget in key metadata

* feat(proxy_server.py): return the key max budget and key spend in the response headers

Allows clientside user to know their remaining limits

* test: add unit testing for new proxy utils

Ensures new key budget is correctly handled

* docs(temporary_budget_increase.md): add doc on temporary budget increase

* fix(utils.py): remove 3.5 from response_format check for now

not all azure  3.5 models support response_format

* fix(user_api_key_auth.py): return valid user api key auth object on all paths
2025-01-14 17:03:11 -08:00
Krish Dholakia 29663c2db5 Litellm dev 01 14 2025 p1 (#7771)
* First-class Aim Guardrails support (#7738)

* initial aim support

* add tests

* docs(langsmith_integration.md): cleanup

* style: cleanup unused imports

---------

Co-authored-by: Tomer Bin <117278227+hxtomer@users.noreply.github.com>
2025-01-14 16:18:21 -08:00
yujonglee 293bb4c522 fix (#7769) 2025-01-14 14:31:44 -08:00
Ishaan Jaff 8c016d0184 docs benchmark 2025-01-14 10:48:43 -08:00
Ishaan Jaff eb2770fee2 update benchmarks 2025-01-14 10:45:28 -08:00
Ishaan Jaff 00f50bc201 bump: version 1.58.0 → 1.58.1 v1.58.1 2025-01-13 20:20:00 -08:00
Ishaan Jaff d510f1d517 (fix) health check - allow setting health_check_model (#7752)
* use _update_litellm_params_for_health_check

* fix Wildcard Routes

* test_update_litellm_params_for_health_check

* test_perform_health_check_with_health_check_model

* fix doc string

* huggingface/mistralai/Mistral-7B-Instruct-v0.3
2025-01-13 20:16:44 -08:00
Ishaan Jaff 9daa6fb0b4 (prometheus - minor bug fix) - litellm_llm_api_time_to_first_token_metric not populating for bedrock models (#7740)
* fix prometheus ttft

* fix test_set_latency_metrics

* fix _set_latency_metrics

* fix _set_latency_metrics

* fix test_set_latency_metrics

* test_async_log_success_event

* huggingface/mistralai/Mistral-7B-Instruct-v0.3
2025-01-13 20:16:34 -08:00
Ishaan Jaff d88f01d518 (litellm SDK perf improvements) - handle cases when unable to lookup model in model cost map (#7750)
* use lru cache wrapper

* use lru_cache_wrapper for _cached_get_model_info_helper

* fix _get_traceback_str_for_error

* huggingface/mistralai/Mistral-7B-Instruct-v0.3
2025-01-13 19:58:46 -08:00
Ishaan Jaff c8ac61f117 fix http parsing utils (#7753) 2025-01-13 19:58:26 -08:00
Ishaan Jaff f1335362cf (core sdk fix) - fix fallbacks stuck in infinite loop (#7751)
* test_acompletion_fallbacks_basic

* use common run_async_function

* fix completion_with_fallbacks

* fix completion with fallbacks

* fix fallback utils

* test_acompletion_fallbacks_basic

* test_completion_fallbacks_sync

* huggingface/mistralai/Mistral-7B-Instruct-v0.3
2025-01-13 19:34:34 -08:00