Commit Graph
19146 Commits
Author SHA1 Message Date
Krrish Dholakia 4e672f6269 fix: fix test 2025-01-22 18:19:49 -08:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
5c88644b87 build(deps): bump undici from 6.21.0 to 6.21.1 in /docs/my-website (#7902)
Bumps [undici](https://github.com/nodejs/undici) from 6.21.0 to 6.21.1.
- [Release notes](https://github.com/nodejs/undici/releases)
- [Commits](https://github.com/nodejs/undici/compare/v6.21.0...v6.21.1)

---
updated-dependencies:
- dependency-name: undici
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-01-22 16:56:23 -08:00
Ishaan JaffandGitHub 1f4ea88228 (Testing) - Add e2e testing for langfuse logging with tags (#7922)
* move langfuse tests

* fix test

* fix completion.json

* working test

* test completion with tags

* langfuse testing fixes

* faster logging testing

* pytest-xdist in testing

* fix langfuse testing flow

* fix testing flow

* fix config for logging tests

* fix langfuse completion with tags stream

* fix _verify_langfuse_call
2025-01-22 09:09:25 -08:00
Ishaan Jaff 0eec1d860e docs on custom tags with langfuse 2025-01-22 08:48:56 -08:00
Ishaan Jaff 9e0ca7da94 docs fix order of logging integrations 2025-01-22 08:45:22 -08:00
Krrish Dholakia d0e68d0f48 bump: version 1.59.3 → 1.59.4 2025-01-21 23:13:32 -08:00
Krish DholakiaandGitHub 76795dba39 Deepseek r1 support + watsonx qa improvements (#7907)
* fix(types/utils.py): support returning 'reasoning_content' for deepseek models

Fixes https://github.com/BerriAI/litellm/issues/7877#issuecomment-2603813218

* fix(convert_dict_to_response.py): return deepseek response in provider_specific_field

allows for separating openai vs. non-openai params in model response

* fix(utils.py): support 'provider_specific_field' in delta chunk as well

allows deepseek reasoning content chunk to be returned to user from stream as well

Fixes https://github.com/BerriAI/litellm/issues/7877#issuecomment-2603813218

* fix(watsonx/chat/handler.py): fix passing space id to watsonx on chat route

* fix(watsonx/): fix watsonx_text/ route with space id

* fix(watsonx/): qa item - also adds better unit testing for watsonx embedding calls

* fix(utils.py): rename to '..fields'

* fix: fix linting errors

* fix(utils.py): fix typing - don't show provider-specific field if none or empty - prevents default respons
e from being non-oai compatible

* fix: cleanup unused imports

* docs(deepseek.md): add docs for deepseek reasoning model
2025-01-21 23:13:15 -08:00
Ishaan Jaff 26a79a533d bump: version 1.59.2 → 1.59.3 v1.59.3 2025-01-21 21:33:50 -08:00
Ishaan Jaff a57b8f6802 fix litellm_overhead_latency_metric 2025-01-21 21:33:31 -08:00
Yuki WatanabeandGitHub 3f053fc99c Update MLflow calllback and documentation (#7809)
* Update MLlfow tracer

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

* doc update

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

* doc update

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

* image rename

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>

---------

Signed-off-by: B-Step62 <yuki.watanabe@databricks.com>
2025-01-21 20:56:48 -08:00
Ishaan Jaff 4978669273 litellm_overhead_latency_metric 2025-01-21 20:51:57 -08:00
Ishaan Jaff aa96c177be fix set_llm_deployment_success_metrics v1.59.2 2025-01-21 20:49:42 -08:00
Ishaan Jaff 695ac73281 bump: version 1.59.1 → 1.59.2 2025-01-21 20:37:14 -08:00
Ishaan JaffandGitHub 4caf4c0277 (Feat - prometheus) - emit litellm_overhead_latency_metric (#7913)
* add track_llm_api_timing

* add track_llm_api_timing

* test_litellm_overhead

* use ResponseMetadata class for setting hidden params and response overhead

* instrument http handler

* fix track_llm_api_timing

* track_llm_api_timing

* emit response overhead on hidden params

* fix resp metadata

* fix make_sync_openai_embedding_request

* test_aaaaatext_completion_endpoint fixes

* _get_value_from_hidden_params

* set_hidden_params

* test_litellm_overhead

* test_litellm_overhead

* test_litellm_overhead

* fix import

* test_litellm_overhead_stream

* add LiteLLMLoggingObject

* use diff folder for testing

* use diff folder for overhead testing

* test litellm overhead

* use typing

* clear typing

* test_litellm_overhead

* fix async_streaming

* update_response_metadata

* move test file

* emit litellm_overhead_latency_metric on prometheus

* add prometheus callback

* litellm_overhead_latency_metric_bucket

* fix apply hidden params

* fix StandardLoggingHiddenParams
2025-01-21 20:36:30 -08:00
Krish DholakiaandGitHub 866fffb50d Litellm dev 01 21 2025 p1 (#7898)
* fix(utils.py): don't pass 'anthropic-beta' header to vertex - will cause request to fail

* fix(utils.py): add flag to allow user to disable filtering invalid headers

ensure user can control behaviour

* style(utils.py): cleanup message

* test(test_utils.py): add unit test to cover invalid header filtering

* fix(proxy_server.py): fix custom openapi schema generation

* fix(utils.py): pass extra headers if set

* fix(main.py): fix image variation to use 'client' param
2025-01-21 20:36:11 -08:00
Ishaan JaffandGitHub dd385410df (Code quality) - Ban recursive functions in codebase (#7910)
* code qa add RecursiveFunctionFinder

* test_recursive_detector

* RecursiveFunctionFinder

* fix check

* recursive_detector
2025-01-21 20:33:32 -08:00
Ishaan JaffandGitHub b6f2e659b9 (Feat) Add x-litellm-overhead-duration-ms and "x-litellm-response-duration-ms" in response from LiteLLM (#7899)
* add track_llm_api_timing

* add track_llm_api_timing

* test_litellm_overhead

* use ResponseMetadata class for setting hidden params and response overhead

* instrument http handler

* fix track_llm_api_timing

* track_llm_api_timing

* emit response overhead on hidden params

* fix resp metadata

* fix make_sync_openai_embedding_request

* test_aaaaatext_completion_endpoint fixes

* _get_value_from_hidden_params

* set_hidden_params

* test_litellm_overhead

* test_litellm_overhead

* test_litellm_overhead

* fix import

* test_litellm_overhead_stream

* add LiteLLMLoggingObject

* use diff folder for testing

* use diff folder for overhead testing

* test litellm overhead

* use typing

* clear typing

* test_litellm_overhead

* fix async_streaming

* update_response_metadata

* move test file

* pply metadata to the response objec
2025-01-21 20:27:55 -08:00
Ishaan JaffandGitHub 63d7d04232 (fix langfuse tags) - read tags from StandardLoggingPayload (#7903)
* fix _get_langfuse_tags

* fix _get_langfuse_tags

* fix _get_langfuse_tags

* _get_langfuse_tags

* test_get_langfuse_tags

* fix langfuse
2025-01-21 20:26:09 -08:00
Ishaan JaffandGitHub 2a71d9e8f1 (Bug fix) - Allow setting null for max_budget, rpm_limit, tpm_limit when updating values on a team (#7912)
* fix update_team

* fix test_key_limit_modifications
2025-01-21 19:19:36 -08:00
b81072d90c fix: add default credential for azure (#7095) (#7891)
* fix: add default credential for azure (#7095)

* fix: fix linting error

* fix: remove redundant test

* test: skip redundant test

---------

Co-authored-by: you-n-g <you-n-g@users.noreply.github.com>
2025-01-21 09:01:49 -08:00
Krish DholakiaandGitHub c8aa876785 fix(proxy_server.py): fix get model info when litellm_model_id is set + move model analytics to free (#7886)
* fix(proxy_server.py): fix get model info when litellm_model_id is set

Fixes https://github.com/BerriAI/litellm/issues/7873

* test(test_models.py): add test to ensure get model info on specific deployment has same value as all model info

Fixes https://github.com/BerriAI/litellm/issues/7873

* fix(usage.tsx): make model analytics free

Fixes @iqballx's feedback

* fix(fix(invoke_handler.py):-fix-bedrock-error-chunk-parsing): return correct bedrock status code and error message if chunk in stream

Improves bedrock stream error handling

* fix(proxy_server.py): fix linting errors

* test(test_auth_checks.py): remove redundant test

* fix(proxy_server.py): fix linting errors

* test: fix flaky test

* test: fix test
2025-01-21 08:19:07 -08:00
Ishaan JaffandGitHub 0295f494b6 (e2e testing + minor refactor) - Virtual Key Max budget check (#7888)
* use helper _virtual_key_max_budget_check

* e2e testing for budget exceeded errors

* e2e budget testing

* test_chat_completion_budget_update

* test_chat_completion_high_budget
2025-01-21 06:47:26 -08:00
Krish DholakiaandGitHub 64e1df1f14 Litellm dev 01 20 2025 p3 (#7890)
* fix(router.py): pass stream timeout correctly for non openai / azure models

Fixes https://github.com/BerriAI/litellm/issues/7870

* test(test_router_timeout.py): add test for streaming

* test(test_router_timeout.py): add unit testing for new router functions

* docs(ollama.md): link to section on calling ollama within docker container

* test: remove redundant test

* test: fix test to include timeout value

* docs(config_settings.md): document new router settings param
2025-01-20 21:46:36 -08:00
Krish DholakiaandGitHub 4b23420a20 Litellm dev 01 20 2025 p1 (#7884)
* fix(initial-test-to-return-api-timeout-value-in-openai-timeout-exception): Makes it easier for user to debug why request timed out

* feat(openai.py): return timeout value + time taken on openai timeout errors

helps debug timeout errors

* fix(utils.py): fix num retries extraction logic when num_retries = 0

* fix(config_settings.md): litellm_logging.py

support printing payload to console if 'LITELLM_PRINT_STANDARD_LOGGING_PAYLOAD' is true

 Enables easier debug

* test(test_auth_checks.py'): remove common checks userapikeyauth enforcement check

* fix(litellm_logging.py): fix linting error
v1.59.1
2025-01-20 21:45:48 -08:00
Ishaan JaffandGitHub 806df5d31c (Feat) datadog_llm_observability callback - emit request_tags on logs (#7883)
* dd - emit tags on llm obs payload

* dd  - show requester tags on traces

* test_get_datadog_tags

* _get_datadog_tags

* fix dd POD_NAME

* test_get_datadog_tags
2025-01-20 20:36:27 -08:00
Krish DholakiaandGitHub 4b88635372 fix(fireworks_ai/): fix global disable flag with transform messages helper (#7847)
fixes issue where .get() = none was preventing global disable flag from being picked up
2025-01-20 20:16:11 -08:00
Rafael AcostaandGitHub 05f476d8c7 feat: add new together_ai models (#7882) 2025-01-20 14:16:04 -08:00
Vini murafaandGitHub bc31d8ed6b typo fix README.md (#7879) 2025-01-20 10:50:35 -08:00
KingandGitHub f654567aa2 Fix typo Update alerting.md (#7880) 2025-01-20 08:04:03 -08:00
Ishaan Jaff f8ea318849 add us.amazon.nova-lite-v1:0 to model cost map 2025-01-20 07:36:12 -08:00
Ishaan Jaff baa9fda9ce docs - Custom Retention Policies 2025-01-20 07:29:48 -08:00
Ishaan Jaff 803da333bf docs Data Retention Policy 2025-01-20 07:00:38 -08:00
Krish DholakiaandGitHub dca6904937 JWT Auth - enforce_rbac support + UI team view, spend calc fix (#7863)
* fix(user_dashboard.tsx): fix spend calculation when team selected

sum all team keys, not user keys

* docs(admin_ui_sso.md): fix docs tabbing

* feat(user_api_key_auth.py): introduce new 'enforce_rbac' param on jwt auth

allows proxy admin to prevent any unmapped yet authenticated jwt tokens from calling proxy

Fixes https://github.com/BerriAI/litellm/issues/6793

* test: more unit testing + refactoring

* fix: fix returning id when obj not found in db

* fix(user_api_key_auth.py): add end user id tracking from jwt auth

* docs(token_auth.md): add doc on rbac with JWTs

* fix: fix unused params

* test: remove old test
2025-01-19 21:28:55 -08:00
Krish DholakiaandGitHub c306c2e0fc Auth checks on invalid fallback models (#7871)
* fix(user_api_key_auth.py): handle clientside fallback model when item in list is dictionary

* fix(auth_checks.py): help user find invalid model names during dev

Ensure fallbacks work in prod

* fix(user_api_key_auth.py): fix linting check

* fix: cleanup unused variables

* fix: fix import

* fix(auth_checks.py): fix auth check
2025-01-19 21:28:10 -08:00
Marty SullivanandGitHub d6e85f7936 add bedrock stability to model_prices_and_context_window.json (#7869) 2025-01-19 15:00:54 -08:00
Krish DholakiaandGitHub 3a7b13efa2 feat(health_check.py): set upperbound for api when making health check call (#7865)
* feat(health_check.py): set upperbound for api when making health check call

prevent bad model from health check to hang and cause pod restarts

* fix(health_check.py): cleanup task once completed

* fix(constants.py): bump default health check timeout to 1min

* docs(health.md): add 'health_check_timeout' to health docs on litellm

* build(proxy_server_config.yaml): add bad model to health check
2025-01-18 19:47:43 -08:00
e67f18b153 LiteLLM Minor Fixes & Improvements (01/18/2025) - p1 (#7857)
* OllamaChatConfig supports JSON schema response format in optional parameters (#7832)

* fix(types/router.py): handle none values for bool types

Fixes https://github.com/BerriAI/litellm/issues/7855#issuecomment-2599781974

* test: handle no hf token in env

---------

Co-authored-by: trislaz <35226192+trislaz@users.noreply.github.com>
2025-01-18 19:03:50 -08:00
Ishaan Jaff f8b059bfa1 docs data sec 2025-01-18 17:44:02 -08:00
Ishaan JaffandGitHub ad4d081a9a litellm sec scans (#7864) 2025-01-18 17:35:00 -08:00
Ishaan Jaff c458c7c801 litellm security page 2025-01-18 17:24:39 -08:00
Ishaan Jaff c0253e17af docs Security Certifications 2025-01-18 17:12:42 -08:00
Ishaan Jaff 40a78253a0 docs data privacy 2025-01-18 17:01:57 -08:00
Ishaan Jaff 447cf5511d fix python 3 install / usage 2025-01-18 16:04:27 -08:00
Ishaan Jaff ac7dc42794 bump: version 1.59.0 → 1.59.1 2025-01-18 15:13:41 -08:00
Ishaan Jaff 7b8fb990db ui new build 2025-01-18 12:56:31 -08:00
Ishaan JaffandGitHub fd5cd422f0 Revert "Remove UI build output" (#7861) 2025-01-18 12:54:25 -08:00
Ishaan JaffandGitHub f6a0bc8bdb (UI Logs) - add pagination + filtering by key name/team name (#7860)
* fix remove emoji on logs page

* fix title of page

* ui - get countryIP

* ui lookup

* ui - get country from ip address

* show team and key alias on root

* working team / key filter

* working filters

* ui filtering by key / team alias

* simple search

* fix add pagination on view logs page

* add start / end time filters

* add custom time filter
2025-01-18 12:47:01 -08:00
Ishaan JaffandGitHub f60756c815 (UI - View Logs Table) - Show country of origin for logs (#7856)
* fix remove emoji on logs page

* fix title of page

* ui - get countryIP

* ui lookup

* ui - get country from ip address
2025-01-18 12:44:26 -08:00
yuu341andGitHub 7b3863b304 Fix: Problem with langfuse_tags when using litellm proxy with langfus… (#7825)
* Fix: Problem with langfuse_tags when using litellm proxy with langfuse integration (#7801)

* Refactor: Create method get_langfuse_tags

* Fix: Exception Handling
2025-01-18 12:39:57 -08:00
Krrish Dholakia 3dc74c6670 build(ui/): update ui 2025-01-18 08:42:51 -08:00