Commit Graph
23118 Commits
Author SHA1 Message Date
Ishaan Jaff 6e1c28cec9 fix code quality check 2025-01-23 18:06:10 -08:00
Ishaan JaffandGitHub 84fb5aead8 (UI) - Set/edit guardrails on a virtual key (#7954)
* Revert "JWT Auth - `enforce_rbac` support + UI team view, spend calc fix (#7863)"

This reverts commit dca6904937.

* Revert "Litellm dev 01 10 2025 p2 (#7679)"

This reverts commit c4780479a9.

* ui - allow setting guardrails on a key

* working edit guardrails

* fix edit guardrails on a key

* Revert "Revert "JWT Auth - `enforce_rbac` support + UI team view, spend calc fix (#7863)""

This reverts commit 8f7b9ae1af71075f6129a49bfc4af094eb2398c7.

* Revert "Revert "Litellm dev 01 10 2025 p2 (#7679)""

This reverts commit a609139dded26b5f59dc55dd68ad0baa470b6e53.

* fix edit guardrail on ui

* fix list_guardrails
2025-01-23 18:01:54 -08:00
Ishaan JaffandGitHub e6ec4f21e5 ui_view_spend_logs (#7952) 2025-01-23 17:27:01 -08:00
Tyler WagnerandGitHub 7d320d31bd docs: fix typo (#7953) 2025-01-23 16:59:17 -08:00
Ishaan JaffandGitHub 7599c9aebb (Testing + Refactor) - Unit testing for team and virtual key budget checks (#7945)
* unit testing for test_virtual_key_max_budget_check

* refactor _team_max_budget_check

* is_model_allowed_by_pattern
2025-01-23 16:58:16 -08:00
Ishaan Jaff 554489ea18 Revert "set litellm_team_budget_reset_at_metric"
This reverts commit 39d959d72a.
2025-01-23 09:05:14 -08:00
Ishaan Jaff 39d959d72a set litellm_team_budget_reset_at_metric 2025-01-23 08:56:05 -08:00
Ishaan Jaff 6a3e217fdf add litellm_team_budget_reset_at_metric 2025-01-23 08:54:19 -08:00
yujongleeandGitHub f201888f69 Merge pull request #7919 from BerriAI/litellm_refactor_e2e_prometheus
Refactor prometheus e2e test
2025-01-23 18:39:42 +09:00
Krish DholakiaandGitHub 513b1904ab Add attempted-retries and timeout values to response headers + more testing (#7926)
* feat(router.py): add retry headers to response

makes it easy to add testing to ensure model-specific retries are respected

* fix(add_retry_headers.py): clarify attempted retries vs. max retries

* test(test_fallbacks.py): add test for checking if max retries set for model is respected

* test(test_fallbacks.py): assert values for attempted retries and max retries are as expected

* fix(utils.py): return timeout in litellm proxy response headers

* test(test_fallbacks.py): add test to assert model specific timeout used on timeout error

* test: add bad model with timeout to proxy

* fix: fix linting error

* fix(router.py): fix get model list from model alias

* test: loosen test restriction - account for other events on proxy
2025-01-22 22:19:44 -08:00
Ishaan Jaff bc546d82a1 bump: version 1.59.4 → 1.59.5 v1.59.5 2025-01-22 22:15:59 -08:00
Ishaan Jaff b60efd4646 fix test_async_create_batch 2025-01-22 22:15:49 -08:00
Ishaan JaffandGitHub 53a3ea3d06 (Refactor) Langfuse - remove prepare_metadata, langfuse python SDK now handles non-json serializable objects (#7925)
* test_langfuse_logging_completion_with_langfuse_metadata

* fix litellm - remove prepare metadata

* test_langfuse_logging_with_non_serializable_metadata

* detailed e2e langfuse metadata tests

* clean up langfuse logging

* fix langfuse

* remove unused imports

* fix code qa checks

* fix _prepare_metadata
2025-01-22 22:11:40 -08:00
Krish DholakiaandGitHub 27560bd5ad Litellm dev 01 22 2025 p4 (#7932)
* feat(main.py): add new 'provider_specific_header' param

allows passing extra header for specific provider

* fix(litellm_pre_call_utils.py): add unit test for pre call utils

* test(test_bedrock_completion.py): skip test now that bedrock supports this
2025-01-22 21:52:07 -08:00
Krish DholakiaandGitHub 4911cd80a1 fix(utils.py): move adding custom logger callback to success event in… (#7905)
* fix(utils.py): move adding custom logger callback to success event into separate function + don't add success callback to failure event

if user is explicitly choosing 'success' callback, don't log failure as well

* test(test_utils.py): add unit test to ensure custom logger callback only adds callback to specific event

* fix(utils.py): remove string from list of callbacks once corresponding callback class is added

prevents floating values - simplifies testing

* fix(utils.py): fix linting error

* test: cleanup args before test

* test: fix test

* test: update test

* test: fix test
2025-01-22 21:49:09 -08:00
Ishaan JaffandGitHub cefbada875 add deepseek-reasoner (#7935) 2025-01-22 21:26:52 -08:00
Krish DholakiaandGitHub e3bacf7196 Litellm dev 01 22 2025 p1 (#7933)
* docs(docker_quick_start.md): add more troubleshooting guides

* test(test_fallbacks.py): add e2e test for proxy with fallbacks + custom fallback message

* test(test_bedrock_completion.py): skip test now that bedrock supports this behaviour

* test(test_fireworks_ai_translation.py): mock fireworks ai test
2025-01-22 19:55:32 -08:00
Krrish Dholakia 760ba4dfd5 test: skip test - Bedrock now supports this behavior 2025-01-22 18:53:12 -08:00
Krrish Dholakia 049915c14b test: mock fireworks ai test - unstable api 2025-01-22 18:52:11 -08:00
Krrish Dholakia 55546f403b Revert "fix: fix test"
This reverts commit 4e672f6269.
2025-01-22 18:21:07 -08:00
Krrish Dholakia 4e672f6269 fix: fix test 2025-01-22 18:19:49 -08:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
5c88644b87 build(deps): bump undici from 6.21.0 to 6.21.1 in /docs/my-website (#7902)
Bumps [undici](https://github.com/nodejs/undici) from 6.21.0 to 6.21.1.
- [Release notes](https://github.com/nodejs/undici/releases)
- [Commits](https://github.com/nodejs/undici/compare/v6.21.0...v6.21.1)

---
updated-dependencies:
- dependency-name: undici
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-01-22 16:56:23 -08:00
Ishaan JaffandGitHub 1f4ea88228 (Testing) - Add e2e testing for langfuse logging with tags (#7922)
* move langfuse tests

* fix test

* fix completion.json

* working test

* test completion with tags

* langfuse testing fixes

* faster logging testing

* pytest-xdist in testing

* fix langfuse testing flow

* fix testing flow

* fix config for logging tests

* fix langfuse completion with tags stream

* fix _verify_langfuse_call
2025-01-22 09:09:25 -08:00
Ishaan Jaff 0eec1d860e docs on custom tags with langfuse 2025-01-22 08:48:56 -08:00
Ishaan Jaff 9e0ca7da94 docs fix order of logging integrations 2025-01-22 08:45:22 -08:00
yujonglee 84a24d8779 done 2025-01-22 20:19:31 +09:00
Krrish Dholakia d0e68d0f48 bump: version 1.59.3 → 1.59.4 2025-01-21 23:13:32 -08:00
Krish DholakiaandGitHub 76795dba39 Deepseek r1 support + watsonx qa improvements (#7907)
* fix(types/utils.py): support returning 'reasoning_content' for deepseek models

Fixes https://github.com/BerriAI/litellm/issues/7877#issuecomment-2603813218

* fix(convert_dict_to_response.py): return deepseek response in provider_specific_field

allows for separating openai vs. non-openai params in model response

* fix(utils.py): support 'provider_specific_field' in delta chunk as well

allows deepseek reasoning content chunk to be returned to user from stream as well

Fixes https://github.com/BerriAI/litellm/issues/7877#issuecomment-2603813218

* fix(watsonx/chat/handler.py): fix passing space id to watsonx on chat route

* fix(watsonx/): fix watsonx_text/ route with space id

* fix(watsonx/): qa item - also adds better unit testing for watsonx embedding calls

* fix(utils.py): rename to '..fields'

* fix: fix linting errors

* fix(utils.py): fix typing - don't show provider-specific field if none or empty - prevents default respons
e from being non-oai compatible

* fix: cleanup unused imports

* docs(deepseek.md): add docs for deepseek reasoning model
2025-01-21 23:13:15 -08:00
Ishaan Jaff 26a79a533d bump: version 1.59.2 → 1.59.3 v1.59.3 2025-01-21 21:33:50 -08:00
Ishaan Jaff a57b8f6802 fix litellm_overhead_latency_metric 2025-01-21 21:33:31 -08:00
Yuki WatanabeandGitHub 3f053fc99c Update MLflow calllback and documentation (#7809)
* Update MLlfow tracer

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

* doc update

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

* doc update

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

* image rename

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

---------

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
2025-01-21 20:56:48 -08:00
Ishaan Jaff 4978669273 litellm_overhead_latency_metric 2025-01-21 20:51:57 -08:00
Ishaan Jaff aa96c177be fix set_llm_deployment_success_metrics v1.59.2 2025-01-21 20:49:42 -08:00
Ishaan Jaff 695ac73281 bump: version 1.59.1 → 1.59.2 2025-01-21 20:37:14 -08:00
Ishaan JaffandGitHub 4caf4c0277 (Feat - prometheus) - emit litellm_overhead_latency_metric (#7913)
* add track_llm_api_timing

* add track_llm_api_timing

* test_litellm_overhead

* use ResponseMetadata class for setting hidden params and response overhead

* instrument http handler

* fix track_llm_api_timing

* track_llm_api_timing

* emit response overhead on hidden params

* fix resp metadata

* fix make_sync_openai_embedding_request

* test_aaaaatext_completion_endpoint fixes

* _get_value_from_hidden_params

* set_hidden_params

* test_litellm_overhead

* test_litellm_overhead

* test_litellm_overhead

* fix import

* test_litellm_overhead_stream

* add LiteLLMLoggingObject

* use diff folder for testing

* use diff folder for overhead testing

* test litellm overhead

* use typing

* clear typing

* test_litellm_overhead

* fix async_streaming

* update_response_metadata

* move test file

* emit litellm_overhead_latency_metric on prometheus

* add prometheus callback

* litellm_overhead_latency_metric_bucket

* fix apply hidden params

* fix StandardLoggingHiddenParams
2025-01-21 20:36:30 -08:00
Krish DholakiaandGitHub 866fffb50d Litellm dev 01 21 2025 p1 (#7898)
* fix(utils.py): don't pass 'anthropic-beta' header to vertex - will cause request to fail

* fix(utils.py): add flag to allow user to disable filtering invalid headers

ensure user can control behaviour

* style(utils.py): cleanup message

* test(test_utils.py): add unit test to cover invalid header filtering

* fix(proxy_server.py): fix custom openapi schema generation

* fix(utils.py): pass extra headers if set

* fix(main.py): fix image variation to use 'client' param
2025-01-21 20:36:11 -08:00
Ishaan JaffandGitHub dd385410df (Code quality) - Ban recursive functions in codebase (#7910)
* code qa add RecursiveFunctionFinder

* test_recursive_detector

* RecursiveFunctionFinder

* fix check

* recursive_detector
2025-01-21 20:33:32 -08:00
Ishaan JaffandGitHub b6f2e659b9 (Feat) Add x-litellm-overhead-duration-ms and "x-litellm-response-duration-ms" in response from LiteLLM (#7899)
* add track_llm_api_timing

* add track_llm_api_timing

* test_litellm_overhead

* use ResponseMetadata class for setting hidden params and response overhead

* instrument http handler

* fix track_llm_api_timing

* track_llm_api_timing

* emit response overhead on hidden params

* fix resp metadata

* fix make_sync_openai_embedding_request

* test_aaaaatext_completion_endpoint fixes

* _get_value_from_hidden_params

* set_hidden_params

* test_litellm_overhead

* test_litellm_overhead

* test_litellm_overhead

* fix import

* test_litellm_overhead_stream

* add LiteLLMLoggingObject

* use diff folder for testing

* use diff folder for overhead testing

* test litellm overhead

* use typing

* clear typing

* test_litellm_overhead

* fix async_streaming

* update_response_metadata

* move test file

* pply metadata to the response objec
2025-01-21 20:27:55 -08:00
Ishaan JaffandGitHub 63d7d04232 (fix langfuse tags) - read tags from StandardLoggingPayload (#7903)
* fix _get_langfuse_tags

* fix _get_langfuse_tags

* fix _get_langfuse_tags

* _get_langfuse_tags

* test_get_langfuse_tags

* fix langfuse
2025-01-21 20:26:09 -08:00
Ishaan JaffandGitHub 2a71d9e8f1 (Bug fix) - Allow setting null for max_budget, rpm_limit, tpm_limit when updating values on a team (#7912)
* fix update_team

* fix test_key_limit_modifications
2025-01-21 19:19:36 -08:00
b81072d90c fix: add default credential for azure (#7095) (#7891)
* fix: add default credential for azure (#7095)

* fix: fix linting error

* fix: remove redundant test

* test: skip redundant test

---------

Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
2025-01-21 09:01:49 -08:00
Krish DholakiaandGitHub c8aa876785 fix(proxy_server.py): fix get model info when litellm_model_id is set + move model analytics to free (#7886)
* fix(proxy_server.py): fix get model info when litellm_model_id is set

Fixes https://github.com/BerriAI/litellm/issues/7873

* test(test_models.py): add test to ensure get model info on specific deployment has same value as all model info

Fixes https://github.com/BerriAI/litellm/issues/7873

* fix(usage.tsx): make model analytics free

Fixes @iqballx's feedback

* fix(fix(invoke_handler.py):-fix-bedrock-error-chunk-parsing): return correct bedrock status code and error message if chunk in stream

Improves bedrock stream error handling

* fix(proxy_server.py): fix linting errors

* test(test_auth_checks.py): remove redundant test

* fix(proxy_server.py): fix linting errors

* test: fix flaky test

* test: fix test
2025-01-21 08:19:07 -08:00
Ishaan JaffandGitHub 0295f494b6 (e2e testing + minor refactor) - Virtual Key Max budget check (#7888)
* use helper _virtual_key_max_budget_check

* e2e testing for budget exceeded errors

* e2e budget testing

* test_chat_completion_budget_update

* test_chat_completion_high_budget
2025-01-21 06:47:26 -08:00
Krish DholakiaandGitHub 64e1df1f14 Litellm dev 01 20 2025 p3 (#7890)
* fix(router.py): pass stream timeout correctly for non openai / azure models

Fixes https://github.com/BerriAI/litellm/issues/7870

* test(test_router_timeout.py): add test for streaming

* test(test_router_timeout.py): add unit testing for new router functions

* docs(ollama.md): link to section on calling ollama within docker container

* test: remove redundant test

* test: fix test to include timeout value

* docs(config_settings.md): document new router settings param
2025-01-20 21:46:36 -08:00
Krish DholakiaandGitHub 4b23420a20 Litellm dev 01 20 2025 p1 (#7884)
* fix(initial-test-to-return-api-timeout-value-in-openai-timeout-exception): Makes it easier for user to debug why request timed out

* feat(openai.py): return timeout value + time taken on openai timeout errors

helps debug timeout errors

* fix(utils.py): fix num retries extraction logic when num_retries = 0

* fix(config_settings.md): litellm_logging.py

support printing payload to console if 'LITELLM_PRINT_STANDARD_LOGGING_PAYLOAD' is true

 Enables easier debug

* test(test_auth_checks.py'): remove common checks userapikeyauth enforcement check

* fix(litellm_logging.py): fix linting error
v1.59.1
2025-01-20 21:45:48 -08:00
Ishaan JaffandGitHub 806df5d31c (Feat) datadog_llm_observability callback - emit request_tags on logs (#7883)
* dd - emit tags on llm obs payload

* dd  - show requester tags on traces

* test_get_datadog_tags

* _get_datadog_tags

* fix dd POD_NAME

* test_get_datadog_tags
2025-01-20 20:36:27 -08:00
Krish DholakiaandGitHub 4b88635372 fix(fireworks_ai/): fix global disable flag with transform messages helper (#7847)
fixes issue where .get() = none was preventing global disable flag from being picked up
2025-01-20 20:16:11 -08:00
Rafael AcostaandGitHub 05f476d8c7 feat: add new together_ai models (#7882) 2025-01-20 14:16:04 -08:00
Vini murafaandGitHub bc31d8ed6b typo fix README.md (#7879) 2025-01-20 10:50:35 -08:00
KingandGitHub f654567aa2 Fix typo Update alerting.md (#7880) 2025-01-20 08:04:03 -08:00