tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-08-05 08:23:17 +00:00
Code Issues Packages Projects Releases Wiki Activity
19,087 Commits 2,072 Branches 1,425 Tags
bc311b7a47566cebfd7d35e2a59de1df2bb700fd
Commit Graph
19087 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Ishaan Jaff bc311b7a47 ui logs - view messages / responses 2025-01-17 20:20:49 -08:00
Krish DholakiaandGitHub 80f7af510b Improve Proxy Resiliency: Cooldown single-deployment model groups if 100% calls failed in high traffic (#7823) 
* refactor(_is_cooldown_required): move '_is_cooldown_required' into cooldown_handlers.py

* refactor(cooldown_handlers.py): move cooldown constants into `.constants.py`

* fix(cooldown_handlers.py): remove if single deployment don't cooldown logic

move to traffic based cooldown logic

Addresses https://github.com/BerriAI/litellm/issues/7822

* fix: add unit tests for '_should_cooldown_deployment'

* test: ensure all tests pass

* test: update test

* fix(cooldown_handlers.py): don't cooldown single deployment models for anything besides traffic related errors

* fix(cooldown_handlers.py): fix cooldown handler logic

* fix(cooldown_handlers.py): fix check
 2025-01-17 20:17:02 -08:00
Krish DholakiaandGitHub d00febcdaa /key/delete - allow team admin to delete team keys (#7846) 
* fix(key_management_endpoints.py): fix key delete to allow team admins + other proxy admins to delete keys

Fixes https://github.com/BerriAI/litellm/issues/7760

* fix(key_management_endpoints.py): remove unused variables

* fix(key_management_endpoints.py): fix linting error
 2025-01-17 20:16:12 -08:00
Krish DholakiaandGitHub c4ff0b6487 refactor: make bedrock image transformation requests async (#7840) 
* refactor: initial commit for using separate sync vs. async transformation routes for bedrock

ensures no blocking calls e.g. when converting image url to b64

* perf(converse_transformation.py): make bedrock converse transformation async

asyncify's the bedrock message transformation - useful for handling image urls for bedrock

* fix(converse_handler.py): fix logging for async streaming

* style: cleanup unused imports
 2025-01-17 20:14:15 -08:00
Ishaan Jaff 2f6829fd2a ui - new build 2025-01-17 20:07:06 -08:00
Ishaan Jaff b701fe1f5e view logs 2025-01-17 20:05:25 -08:00
Krish DholakiaandGitHub 71c41f8f33 QA: ensure all bedrock regional models have same supported_ as base + Anthropic nested pydantic object support (#7844) 
* build: ensure all regional bedrock models have same supported values as base bedrock model

prevents drift

* test(base_llm_unit_tests.py): add testing for nested pydantic objects

* fix(test_utils.py): add test_get_potential_model_names

* fix(anthropic/chat/transformation.py): support nested pydantic objects

Fixes https://github.com/BerriAI/litellm/issues/7755
 2025-01-17 19:49:12 -08:00
Ishaan Jaff 62188a01b0 bump: version 1.58.4 → 1.59.0 2025-01-17 19:35:15 -08:00
Ishaan JaffandGitHub 2c117264a2 [Hashicorp - secret manager] - use vault namespace for tls auth (#7834) 
* hcorp - use x-vault-namespace

* _get_tls_cert_auth_body

* HCP_VAULT_CERT_ROLE

* test_hashicorp_secret_manager_tls_cert_auth

* HCP_VAULT_CERT_ROLE
 2025-01-17 19:27:56 -08:00
Ishaan Jaff 69d876f4c7 ui new build 2025-01-17 19:23:41 -08:00
Ishaan Jaff f54e662c57 rename ui tab to ExperimentOutlined 2025-01-17 19:22:32 -08:00
Ishaan Jaff 6d1a5a0e5d ui new build 2025-01-17 19:14:44 -08:00
Ishaan JaffandGitHub a489c5d95a [fix dd llm obs] - use env vars for setting dd tags, service name (#7835) 
* fix custom logger

* fix debugging dd llm obs
 2025-01-17 18:57:16 -08:00
Ishaan JaffandGitHub d3c2f4331a (UI - View SpendLogs Table) (#7842) 
* litellm log messages / responses

* add messages/response to schema.prisma

* add support for logging messages / responses in DB

* test_spend_logs_payload_with_prompts_enabled

* _get_messages_for_spend_logs_payload

* ui_view_spend_logs endpoint

* add tanstack and moment

* add uiSpendLogsCall

* ui view logs table

* ui view spendLogs table

* ui_view_spend_logs

* fix code quality

* test_spend_logs_payload_with_prompts_enabled

* _get_messages_for_spend_logs_payload

* test_spend_logs_payload_with_prompts_enabled

* test_spend_logs_payload_with_prompts_enabled

* ui view spend logs

* minor ui fix

* ui - update leftnav

* ui - clean up ui

* fix leftnav

* ui fix navbar

* ui fix moving chat ui tab
 2025-01-17 18:53:45 -08:00
Krish DholakiaandGitHub a99deb6d0a fix(key_management_endpoints.py): fix default allowed team member roles (#7843) 
admin and user, not admin and member
 2025-01-17 17:15:22 -08:00
yujongleeandGitHub 7584369fbe add key and team level budget (#7831) 2025-01-17 09:04:12 -08:00
Ishaan Jaff 7cfda2fb76 bump: version 1.58.3 → 1.58.4 v1.58.4 2025-01-17 08:29:13 -08:00
Ishaan Jaff 2d5f8ea2c6 Revert "fix custom logger" 
This reverts commit 9d2707ecfe.
 2025-01-17 08:26:59 -08:00
Ishaan Jaff 9d2707ecfe fix custom logger 2025-01-17 07:39:49 -08:00
Ishaan Jaff 632ba92af1 Revert "fix: fix test" 
This reverts commit 0642a78abb.
 2025-01-17 07:21:19 -08:00
Ishaan Jaff b30e05b54f Revert "test_completion_mistral_api_mistral_large_function_call" 
This reverts commit ef9177f0a8.
 2025-01-17 07:20:46 -08:00
Krrish Dholakia 0642a78abb fix: fix test 2025-01-17 07:16:32 -08:00
Ishaan Jaff 15e0a605e4 _handle_tool_call_message linting 2025-01-16 22:34:16 -08:00
Ishaan Jaff c8febaca2e test_watsonx_token_in_env_var 2025-01-16 22:28:37 -08:00
Ishaan Jaff 7f63e7c15a test_completion_mistral_api_mistral_large_function_call 2025-01-16 22:27:48 -08:00
Ishaan JaffandGitHub f3268b6b06 sec fix minor (#7810) 2025-01-16 22:03:28 -08:00
Ishaan JaffandGitHub 9b944ca60c (Fix + Testing) - Add dd-trace-run to litellm ci/cd pipeline + fix bug caused by dd-trace patching OpenAI sdk (#7820) 
* add dd trace to e2e docker run tests

* update dd trace v

* fix entrypoint

* dd trace fixes

* proxy_build_from_pip_tests

* build python3.13

* use py 3.13

* fix build from pip

* dd trace fix

* proxy_build_from_pip_tests

* bump build from pip
 2025-01-16 22:03:09 -08:00
Ishaan JaffandGitHub b492551d3d (fix) IBM Watsonx using ZenApiKey (#7821) 
* ibm watsonx fix

* test ZenAPIKey

* fix zenapikey
 2025-01-16 22:02:36 -08:00
Ishaan JaffandGitHub 939e1c9b19 (datadog llm observability) - fixes + improvements for using datadog llm observability logging integration (#7824) 
* dd llm obs fixes

* _ensure_string_content

* fix _get_dd_llm_obs_payload_metadata
 2025-01-16 22:02:24 -08:00
Ishaan Jaff 5b36985c00 run ci/cd again 2025-01-16 22:02:03 -08:00
Ishaan Jaff ef9177f0a8 test_completion_mistral_api_mistral_large_function_call 2025-01-16 21:50:56 -08:00
Ishaan Jaff 117256d264 test_async_vertexai_streaming_response 2025-01-16 21:45:12 -08:00
Ishaan Jaff de41abeace llama-v3p1-8b-instruct 2025-01-16 21:34:42 -08:00
Ishaan Jaff 5458a2ff33 fireworks ai use llama-v3p1-8b-instruct 2025-01-16 21:28:44 -08:00
Krrish Dholakia 8ab1335ae0 test: fix unit test 2025-01-16 21:11:17 -08:00
Ishaan Jaff 2f38e72026 test commit on main 2025-01-16 20:52:55 -08:00
Krish DholakiaandGitHub 4ec2c2c8e4 build: bump certifi version - see if that fixes asyncio ssl issue on python 3.13 (#7800) 2025-01-15 22:02:45 -08:00
Krish DholakiaandGitHub c57266c9dc test: initial commit enforcing testing on all anthropic pass through … (#7794) 
* test: initial commit enforcing testing on all anthropic pass through functions

prevents future regressions

* test(test_unit_test_anthropic_pass_through.py): add unit test for '_get_user_from_metadata' function

* test(test_unit_test_anthropic_passthrough.py): add unit test for handle_logging_anthropic_collected_chunks

* test(test_unit_test_anthropic_pass_through): add coverage for all anthropic pass through functions
 2025-01-15 22:02:35 -08:00
Krish DholakiaandGitHub 843cd3b7c6 test: initial test to enforce all functions in user_api_key_auth.py h… (#7797) 
* test: initial test to enforce all functions in user_api_key_auth.py have direct testing

* test(test_user_api_key_auth.py): add is_allowed_route unit test

* test(test_user_api_key_auth.py): add more tests

* test(test_user_api_key_auth.py): add complete testing coverage for all functions in `user_api_key_auth.py`

* test(test_db_schema_changes.py): add a unit test to ensure all db schema changes are backwards compatible

gives user an easy rollback path

* test: fix schema compatibility test filepath

* test: fix test
 2025-01-15 21:52:45 -08:00
Krish DholakiaGitHubRajat VigZackeus BengtssonyujongleeHugues ChocartNikolaiev Dmytro
 		d4ed985173 Add back in non root image fixes (#7781) (#7795) 
* Add back in non root image fixes (#7781)

* Add back in non root image fixes

* Fix dockerfile

* Fix perms

* Add in container structure tests for the nonroot image (#7796)

* feat(helm): add securityContext and pull policy values to migration job (#7652)

* fix(helm): corrected indentation in migration-job.yaml

* feat(helm): add securityContext and pull policy values to migration job

* fix confusing save button label (#7778)

* [integrations/lunary] Improve Lunary documentaiton (#7770)

* update lunary doc

* better title

* tweaks

* Update langchain.md

* Update lunary_integration.md

* Fix wrong URL for internal user invitation (#7762)

* format

* done

* Update instructor tutorial (#7784)

* Add in container structure tests for the nonroot image

---------

Co-authored-by: Zackeus Bengtsson <32719220+Hexoplon@users.noreply.github.com>
Co-authored-by: yujonglee <yujonglee.dev@gmail.com>
Co-authored-by: Hugues Chocart <chocart.hugues@icloud.com>
Co-authored-by: Nikolaiev Dmytro <dima.nikol.99@gmail.com>

---------

Co-authored-by: Rajat Vig <rajatvig@users.noreply.github.com>
Co-authored-by: Zackeus Bengtsson <32719220+Hexoplon@users.noreply.github.com>
Co-authored-by: yujonglee <yujonglee.dev@gmail.com>
Co-authored-by: Hugues Chocart <chocart.hugues@icloud.com>
Co-authored-by: Nikolaiev Dmytro <dima.nikol.99@gmail.com>
 2025-01-15 21:49:03 -08:00
Krish DholakiaandGitHub 80d6bbec29 Litellm dev 01 14 2025 p2 (#7772) 
* feat(pass_through_endpoints.py): fix anthropic end user cost tracking

* fix(anthropic/chat/transformation.py): use returned provider model for anthropic

handles anthropic `-latest` tag in request body throwing cost calculation errors

ensures we can be accurate in our model cost tracking

* feat(model_prices_and_context_window.json): add gemini-2.0-flash-thinking-exp pricing

* test: update test to use assumption that user_api_key_dict can get anthropic user id

* test: fix test

* fix: fix test

* fix(anthropic_pass_through.py): uncomment previous anthropic end-user cost tracking code block

can't guarantee user api key dict always has end user id - too many code paths

* fix(user_api_key_auth.py): this allows end user id from request body to always be read and set in auth object

* fix(auth_check.py): fix linting error

* test: fix auth check

* fix(auth_utils.py): fix get end user id to handle metadata = None
 2025-01-15 21:34:50 -08:00
Ishaan JaffandGitHub 4081aeb15e (helm) - allow specifying envVars on values.yaml + add helm lint test (#7789) 
* litellm use envVars values.yaml

* fix values.yaml

* add helm lint to ci/cd pipeline

* working values.yaml

* add helm tests to ci/cd

* fix helm chart testing

* update helm tests

* fix helm test

* fix use test values in ci

* fix busy box testing on helm

* fix test-values.yaml

* update helm tests

* fix test connection
 2025-01-15 21:11:17 -08:00
Nikolaiev DmytroandGitHub 7b45349145 Update instructor tutorial (#7784) 2025-01-15 15:10:50 -08:00
yujongleeandGitHub 503b38e609 Fix wrong URL for internal user invitation (#7762) 
* format

* done
 2025-01-15 15:10:32 -08:00
Hugues ChocartandGitHub 6fff77d131 [integrations/lunary] Improve Lunary documentaiton (#7770) 
* update lunary doc

* better title

* tweaks

* Update langchain.md

* Update lunary_integration.md
 2025-01-15 15:00:25 -08:00
yujongleeandGitHub 01357add4d fix confusing save button label (#7778) 2025-01-15 13:51:07 -08:00
Zackeus BengtssonandGitHub 801f450e8c feat(helm): add securityContext and pull policy values to migration job (#7652) 
* fix(helm): corrected indentation in migration-job.yaml

* feat(helm): add securityContext and pull policy values to migration job
 2025-01-15 12:51:08 -08:00
Krrish Dholakia 8c26e1a66d build: bump: version 1.58.2 → 1.58.3 2025-01-15 07:46:57 -08:00
Krish DholakiaandGitHub fe60a38c8e Litellm dev 01 2025 p4 (#7776) 
* fix(gemini/): support gemini 'frequency_penalty' and 'presence_penalty'

Closes https://github.com/BerriAI/litellm/issues/7748

* feat(proxy_server.py): new env var to disable prisma health check on startup

* test: fix test
 2025-01-14 21:49:25 -08:00
Krish DholakiaandGitHub 8353caa485 build(pyproject.toml): bump uvicorn depedency requirement (#7773) 
* build(pyproject.toml): bump uvicorn depedency requirement

Fixes https://github.com/BerriAI/litellm/issues/7768

* fix(anthropic/chat/transformation.py): fix is_vertex_request check to actually use optional param passed in

Fixes https://github.com/BerriAI/litellm/issues/6898#issuecomment-2590860695

* fix(o1_transformation.py): fix azure o1 'is_o1_model' check to just check for o1 in model string

https://github.com/BerriAI/litellm/issues/7743

* test: load vertex creds
 2025-01-14 21:47:11 -08:00