Commit Graph
19121 Commits
Author SHA1 Message Date
Krish DholakiaandGitHub 4b88635372 fix(fireworks_ai/): fix global disable flag with transform messages helper (#7847)
fixes issue where .get() = none was preventing global disable flag from being picked up
2025-01-20 20:16:11 -08:00
Rafael AcostaandGitHub 05f476d8c7 feat: add new together_ai models (#7882) 2025-01-20 14:16:04 -08:00
Vini murafaandGitHub bc31d8ed6b typo fix README.md (#7879) 2025-01-20 10:50:35 -08:00
KingandGitHub f654567aa2 Fix typo Update alerting.md (#7880) 2025-01-20 08:04:03 -08:00
Ishaan Jaff f8ea318849 add us.amazon.nova-lite-v1:0 to model cost map 2025-01-20 07:36:12 -08:00
Ishaan Jaff baa9fda9ce docs - Custom Retention Policies 2025-01-20 07:29:48 -08:00
Ishaan Jaff 803da333bf docs Data Retention Policy 2025-01-20 07:00:38 -08:00
Krish DholakiaandGitHub dca6904937 JWT Auth - enforce_rbac support + UI team view, spend calc fix (#7863)
* fix(user_dashboard.tsx): fix spend calculation when team selected

sum all team keys, not user keys

* docs(admin_ui_sso.md): fix docs tabbing

* feat(user_api_key_auth.py): introduce new 'enforce_rbac' param on jwt auth

allows proxy admin to prevent any unmapped yet authenticated jwt tokens from calling proxy

Fixes https://github.com/BerriAI/litellm/issues/6793

* test: more unit testing + refactoring

* fix: fix returning id when obj not found in db

* fix(user_api_key_auth.py): add end user id tracking from jwt auth

* docs(token_auth.md): add doc on rbac with JWTs

* fix: fix unused params

* test: remove old test
2025-01-19 21:28:55 -08:00
Krish DholakiaandGitHub c306c2e0fc Auth checks on invalid fallback models (#7871)
* fix(user_api_key_auth.py): handle clientside fallback model when item in list is dictionary

* fix(auth_checks.py): help user find invalid model names during dev

Ensure fallbacks work in prod

* fix(user_api_key_auth.py): fix linting check

* fix: cleanup unused variables

* fix: fix import

* fix(auth_checks.py): fix auth check
2025-01-19 21:28:10 -08:00
Marty SullivanandGitHub d6e85f7936 add bedrock stability to model_prices_and_context_window.json (#7869) 2025-01-19 15:00:54 -08:00
Krish DholakiaandGitHub 3a7b13efa2 feat(health_check.py): set upperbound for api when making health check call (#7865)
* feat(health_check.py): set upperbound for api when making health check call

prevent bad model from health check to hang and cause pod restarts

* fix(health_check.py): cleanup task once completed

* fix(constants.py): bump default health check timeout to 1min

* docs(health.md): add 'health_check_timeout' to health docs on litellm

* build(proxy_server_config.yaml): add bad model to health check
2025-01-18 19:47:43 -08:00
e67f18b153 LiteLLM Minor Fixes & Improvements (01/18/2025) - p1 (#7857)
* OllamaChatConfig supports JSON schema response format in optional parameters (#7832)

* fix(types/router.py): handle none values for bool types

Fixes https://github.com/BerriAI/litellm/issues/7855#issuecomment-2599781974

* test: handle no hf token in env

---------

Co-authored-by: trislaz <35226192+trislaz@users.noreply.github.com>
2025-01-18 19:03:50 -08:00
Ishaan Jaff f8b059bfa1 docs data sec 2025-01-18 17:44:02 -08:00
Ishaan JaffandGitHub ad4d081a9a litellm sec scans (#7864) 2025-01-18 17:35:00 -08:00
Ishaan Jaff c458c7c801 litellm security page 2025-01-18 17:24:39 -08:00
Ishaan Jaff c0253e17af docs Security Certifications 2025-01-18 17:12:42 -08:00
Ishaan Jaff 40a78253a0 docs data privacy 2025-01-18 17:01:57 -08:00
Ishaan Jaff 447cf5511d fix python 3 install / usage 2025-01-18 16:04:27 -08:00
Ishaan Jaff ac7dc42794 bump: version 1.59.0 → 1.59.1 2025-01-18 15:13:41 -08:00
Ishaan Jaff 7b8fb990db ui new build 2025-01-18 12:56:31 -08:00
Ishaan JaffandGitHub fd5cd422f0 Revert "Remove UI build output" (#7861) 2025-01-18 12:54:25 -08:00
Ishaan JaffandGitHub f6a0bc8bdb (UI Logs) - add pagination + filtering by key name/team name (#7860)
* fix remove emoji on logs page

* fix title of page

* ui - get countryIP

* ui lookup

* ui - get country from ip address

* show team and key alias on root

* working team / key filter

* working filters

* ui filtering by key / team alias

* simple search

* fix add pagination on view logs page

* add start / end time filters

* add custom time filter
2025-01-18 12:47:01 -08:00
Ishaan JaffandGitHub f60756c815 (UI - View Logs Table) - Show country of origin for logs (#7856)
* fix remove emoji on logs page

* fix title of page

* ui - get countryIP

* ui lookup

* ui - get country from ip address
2025-01-18 12:44:26 -08:00
yuu341andGitHub 7b3863b304 Fix: Problem with langfuse_tags when using litellm proxy with langfus… (#7825)
* Fix: Problem with langfuse_tags when using litellm proxy with langfuse integration (#7801)

* Refactor: Create method get_langfuse_tags

* Fix: Exception Handling
2025-01-18 12:39:57 -08:00
Krrish Dholakia 3dc74c6670 build(ui/): update ui 2025-01-18 08:42:51 -08:00
Krish DholakiaandGitHub 5d065c2c35 fix(admins.tsx): fix logic for getting base url and create common get base url component (#7854)
Resolves https://github.com/BerriAI/litellm/issues/7761
2025-01-18 08:07:39 -08:00
Ishaan Jaff 2fdbcca9ae e2e ui testing fixes v1.59.0 2025-01-18 07:46:55 -08:00
yujongleeandGitHub f4803f63dc refactor (#7851) 2025-01-18 07:25:41 -08:00
yujongleeandGitHub 4e5d68a1d9 Merge pull request #7852 from BerriAI/litellm_fix_base_url
Fix invalid base URL error
2025-01-18 22:14:14 +09:00
yujonglee 0aa8e4a703 fix 2025-01-18 22:13:07 +09:00
yujongleeandGitHub c4c5b3adbd Merge pull request #7849 from BerriAI/litellm_gitignore_ui
Remove UI build output
2025-01-18 16:57:12 +09:00
yujonglee eb91ceb8d5 remore ui build 2025-01-18 16:55:45 +09:00
1bea338597 LiteLLM Minor Fixes & Improvements (2024/16/01) (#7826)
* fix(lm_studio/chat/transformation.py): Fix https://github.com/BerriAI/litellm/issues/7811

* fix(router.py): fix mock timeout check

* fix: drop model name from fallback args since it causes a conflict with the model=model that is provided later on. (#7806)

This error happens if you provide multiple fallback models to the completion function with model name defined in each one.

* fix(router.py): remove mock_timeout before sending to request

prevents reuse in fallbacks

* test: update test

* test: revert test change - wrong pr

---------

Co-authored-by: Dudu Lasry <david1542@users.noreply.github.com>
2025-01-17 20:59:21 -08:00
Ishaan Jaff a2762fb273 ui release note 2025-01-17 20:27:53 -08:00
Ishaan Jaff bc311b7a47 ui logs - view messages / responses 2025-01-17 20:20:49 -08:00
Krish DholakiaandGitHub 80f7af510b Improve Proxy Resiliency: Cooldown single-deployment model groups if 100% calls failed in high traffic (#7823)
* refactor(_is_cooldown_required): move '_is_cooldown_required' into cooldown_handlers.py

* refactor(cooldown_handlers.py): move cooldown constants into `.constants.py`

* fix(cooldown_handlers.py): remove if single deployment don't cooldown logic

move to traffic based cooldown logic

Addresses https://github.com/BerriAI/litellm/issues/7822

* fix: add unit tests for '_should_cooldown_deployment'

* test: ensure all tests pass

* test: update test

* fix(cooldown_handlers.py): don't cooldown single deployment models for anything besides traffic related errors

* fix(cooldown_handlers.py): fix cooldown handler logic

* fix(cooldown_handlers.py): fix check
2025-01-17 20:17:02 -08:00
Krish DholakiaandGitHub d00febcdaa /key/delete - allow team admin to delete team keys (#7846)
* fix(key_management_endpoints.py): fix key delete to allow team admins + other proxy admins to delete keys

Fixes https://github.com/BerriAI/litellm/issues/7760

* fix(key_management_endpoints.py): remove unused variables

* fix(key_management_endpoints.py): fix linting error
2025-01-17 20:16:12 -08:00
Krish DholakiaandGitHub c4ff0b6487 refactor: make bedrock image transformation requests async (#7840)
* refactor: initial commit for using separate sync vs. async transformation routes for bedrock

ensures no blocking calls e.g. when converting image url to b64

* perf(converse_transformation.py): make bedrock converse transformation async

asyncify's the bedrock message transformation - useful for handling image urls for bedrock

* fix(converse_handler.py): fix logging for async streaming

* style: cleanup unused imports
2025-01-17 20:14:15 -08:00
Ishaan Jaff 2f6829fd2a ui - new build 2025-01-17 20:07:06 -08:00
Ishaan Jaff b701fe1f5e view logs 2025-01-17 20:05:25 -08:00
Krish DholakiaandGitHub 71c41f8f33 QA: ensure all bedrock regional models have same supported_ as base + Anthropic nested pydantic object support (#7844)
* build: ensure all regional bedrock models have same supported values as base bedrock model

prevents drift

* test(base_llm_unit_tests.py): add testing for nested pydantic objects

* fix(test_utils.py): add test_get_potential_model_names

* fix(anthropic/chat/transformation.py): support nested pydantic objects

Fixes https://github.com/BerriAI/litellm/issues/7755
2025-01-17 19:49:12 -08:00
Ishaan Jaff 62188a01b0 bump: version 1.58.4 → 1.59.0 2025-01-17 19:35:15 -08:00
Ishaan JaffandGitHub 2c117264a2 [Hashicorp - secret manager] - use vault namespace for tls auth (#7834)
* hcorp - use x-vault-namespace

* _get_tls_cert_auth_body

* HCP_VAULT_CERT_ROLE

* test_hashicorp_secret_manager_tls_cert_auth

* HCP_VAULT_CERT_ROLE
2025-01-17 19:27:56 -08:00
Ishaan Jaff 69d876f4c7 ui new build 2025-01-17 19:23:41 -08:00
Ishaan Jaff f54e662c57 rename ui tab to ExperimentOutlined 2025-01-17 19:22:32 -08:00
Ishaan Jaff 6d1a5a0e5d ui new build 2025-01-17 19:14:44 -08:00
Ishaan JaffandGitHub a489c5d95a [fix dd llm obs] - use env vars for setting dd tags, service name (#7835)
* fix custom logger

* fix debugging dd llm obs
2025-01-17 18:57:16 -08:00
Ishaan JaffandGitHub d3c2f4331a (UI - View SpendLogs Table) (#7842)
* litellm log messages / responses

* add messages/response to schema.prisma

* add support for logging messages / responses in DB

* test_spend_logs_payload_with_prompts_enabled

* _get_messages_for_spend_logs_payload

* ui_view_spend_logs endpoint

* add tanstack and moment

* add uiSpendLogsCall

* ui view logs table

* ui view spendLogs table

* ui_view_spend_logs

* fix code quality

* test_spend_logs_payload_with_prompts_enabled

* _get_messages_for_spend_logs_payload

* test_spend_logs_payload_with_prompts_enabled

* test_spend_logs_payload_with_prompts_enabled

* ui view spend logs

* minor ui fix

* ui - update leftnav

* ui - clean up ui

* fix leftnav

* ui fix navbar

* ui fix moving chat ui tab
2025-01-17 18:53:45 -08:00
Krish DholakiaandGitHub a99deb6d0a fix(key_management_endpoints.py): fix default allowed team member roles (#7843)
admin and user, not admin and member
2025-01-17 17:15:22 -08:00
yujongleeandGitHub 7584369fbe add key and team level budget (#7831) 2025-01-17 09:04:12 -08:00