Commit Graph
23118 Commits
Author SHA1 Message Date
Krish DholakiaandGitHub 222898d727 Fix anthropic thinking + response_format (#9594)
* fix(anthropic/chat/transformation.py): Don't set tool choice on response_format conversion when thinking is enabled

Not allowed by Anthropic

Fixes https://github.com/BerriAI/litellm/issues/8901

* refactor: move test to base anthropic chat tests

ensures consistent behaviour across vertex/anthropic/bedrock

* fix(anthropic/chat/transformation.py): if thinking token is specified and max tokens is not - ensure max token to anthropic is higher than thinking tokens

* feat(converse_transformation.py): correctly handle thinking + response format on Bedrock Converse

Fixes https://github.com/BerriAI/litellm/issues/8901

* fix(converse_transformation.py): correctly handle adding max tokens

* test: handle service unavailable error
2025-03-28 15:57:40 -07:00
Ishaan Jaff df136c5689 add docstring for DBSpendUpdateTransactions 2025-03-28 15:20:28 -07:00
Nicholas Grabar 09daeac188 Rebasing 2 2025-03-28 15:18:09 -07:00
Nicholas Grabar 06a45706b2 Rebase 3 2025-03-28 15:18:05 -07:00
Ishaan Jaff 7c73987066 fix loc of DBSpendUpdateTransactions 2025-03-28 15:17:01 -07:00
Krrish DholakiaandNicholas Grabar 8c9ff23e19 test(test_caching_handler.py): move to in-memory cache - prevent redis flakiness from impacting ci/cd 2025-03-28 15:16:15 -07:00
DbzmanandNicholas Grabar 9e4165ac75 fix: wrong indentation for ttlSecondsAfterFinished (#9611) 2025-03-28 15:16:15 -07:00
1a441def03 fix(logging): add json formatting for uncaught exceptions (#9615) (#9619)
* fix(logging): add json formatting for uncaught exceptions (#9615)

* fix(_logging.py): cleanup logging to catch unhandled exceptions

* fix(_logging.py): avoid using 'print' '

---------

Co-authored-by: Henrique Cavarsan <hcavarsan@yahoo.com.br>
2025-03-28 15:16:15 -07:00
Krish DholakiaandNicholas Grabar 205db622bf fix(proxy_server.py): get master key from environment, if not set in … (#9617)
* fix(proxy_server.py): get master key from environment, if not set in general settings or general settings not set at all

* test: mark flaky test

* test(test_proxy_server.py): mock prisma client

* ci: add new github workflow for testing just the mock tests

* fix: fix linting error

* ci(conftest.py): add conftest.py to isolate proxy tests

* build(pyproject.toml): add respx to dev dependencies

* build(pyproject.toml): add prisma to dev dependencies

* test: fix mock prompt management tests to use a mock anthropic key

* ci(test-litellm.yml): parallelize mock testing

make it run faster

* build(pyproject.toml): add hypercorn as dev dep

* build(pyproject.toml): separate proxy vs. core dev dependencies

make it easier for non-proxy contributors to run tests locally - e.g. no need to install hypercorn

* ci(test-litellm.yml): pin python version

* test(test_rerank.py): move test - cannot be mocked, requires aws credentials for e2e testing

* ci: add thank you message to ci

* test: add mock env var to test

* test: add autouse to tests

* test: test mock env vars for e2e tests
2025-03-28 15:16:15 -07:00
Ishaan Jaff 3df839d878 use helper methods _commit_spend_updates_to_db_without_redis_buffer 2025-03-28 15:12:19 -07:00
Ishaan Jaff 193052ed70 test pod lock manager 2025-03-28 15:05:17 -07:00
Krrish Dholakia 28a9edb547 test(test_caching_handler.py): move to in-memory cache - prevent redis flakiness from impacting ci/cd 2025-03-28 13:32:04 -07:00
Ishaan Jaff 1eaf847f8a test pod lock manager 2025-03-28 13:31:45 -07:00
DbzmanandGitHub 7c85054bad fix: wrong indentation for ttlSecondsAfterFinished (#9611) 2025-03-28 13:20:12 -07:00
Nicholas Grabar 1f2bbda11d Add recursion depth to convert_anyof_null_to_nullable, constants.py. Fix recursive_detector.py raise error state 2025-03-28 13:11:19 -07:00
Ishaan Jaff 021eedaf69 test pod lock manager 2025-03-28 12:59:16 -07:00
Ishaan Jaff c53d172b06 rename pod lock manager 2025-03-28 12:57:00 -07:00
f3ff7e2779 fix(logging): add json formatting for uncaught exceptions (#9615) (#9619)
* fix(logging): add json formatting for uncaught exceptions (#9615)

* fix(_logging.py): cleanup logging to catch unhandled exceptions

* fix(_logging.py): avoid using 'print' '

---------

Co-authored-by: Henrique Cavarsan <hcavarsan@yahoo.com.br>
2025-03-28 12:47:26 -07:00
Krish DholakiaandGitHub 0865e52db3 fix(proxy_server.py): get master key from environment, if not set in … (#9617)
* fix(proxy_server.py): get master key from environment, if not set in general settings or general settings not set at all

* test: mark flaky test

* test(test_proxy_server.py): mock prisma client

* ci: add new github workflow for testing just the mock tests

* fix: fix linting error

* ci(conftest.py): add conftest.py to isolate proxy tests

* build(pyproject.toml): add respx to dev dependencies

* build(pyproject.toml): add prisma to dev dependencies

* test: fix mock prompt management tests to use a mock anthropic key

* ci(test-litellm.yml): parallelize mock testing

make it run faster

* build(pyproject.toml): add hypercorn as dev dep

* build(pyproject.toml): separate proxy vs. core dev dependencies

make it easier for non-proxy contributors to run tests locally - e.g. no need to install hypercorn

* ci(test-litellm.yml): pin python version

* test(test_rerank.py): move test - cannot be mocked, requires aws credentials for e2e testing

* ci: add thank you message to ci

* test: add mock env var to test

* test: add autouse to tests

* test: test mock env vars for e2e tests
v1.65.0-nightly v1.65.0.rc
2025-03-28 12:32:04 -07:00
Shixian ShengandGitHub e729615e41 Merge branch 'BerriAI:main' into main 2025-03-28 14:38:28 -04:00
Ishaan Jaff e7181395ff fix code quality check 2025-03-28 10:32:39 -07:00
Ishaan JaffandGitHub 462e9baffa Merge pull request #9618 from NickGrab/litellm_8864-feature-vertex-anyOf-support
Add support to Vertex AI transformation for anyOf union type with null fields
2025-03-28 10:29:33 -07:00
NickGrabandGitHub b72fbdde74 Merge branch 'main' into litellm_8864-feature-vertex-anyOf-support 2025-03-28 10:25:04 -07:00
Nicholas Grabar 9437ee5e1f Revert "Unit test fixing and poetry update"
This reverts commit 8c79e1902e.
2025-03-28 10:22:32 -07:00
Nicholas Grabar 8c79e1902e Unit test fixing and poetry update 2025-03-28 09:57:53 -07:00
Krrish Dholakia 69e28b92c6 fix: fix python38 linting error 2025-03-28 09:38:32 -07:00
Krish DholakiaandGitHub d60e485c03 Update README.md (#9616) 2025-03-28 09:28:09 -07:00
Shixian ShengandGitHub 74917c7a6f Merge branch 'BerriAI:main' into main 2025-03-28 06:44:51 -04:00
Krrish Dholakia 34a1a48af1 fix(common_utils.py): fix linting error 2025-03-27 23:31:58 -07:00
Krrish Dholakia 3226477464 fix: cleanup 2025-03-27 23:31:12 -07:00
Ishaan Jaff 0edd4aa8a7 fix PodLockManager v1.64.1.dev1 2025-03-27 23:29:48 -07:00
Krish DholakiaandGitHub cdcc8ea9b7 Connect UI to "LiteLLM_DailyUserSpend" spend table - enables usage tab to work at 1m+ spend logs (#9603)
* feat(spend_management_endpoints.py): expose new endpoint for querying user's usage at 1m+ spend logs

Allows user to view their spend at 1m+ spend logs

* build(schema.prisma): add api_requests to dailyuserspend table

* build(migration.sql): add migration file for new column to daily user spend table

* build(prisma_client.py): add logic for copying over migration folder, if deploy/migrations present in expected location

enables easier testing of prisma migration flow

* build(ui/): initial commit successfully using the dailyuserspend table on the UI

* refactor(internal_user_endpoints.py): refactor `/user/daily/activity` to give breakdowns by provider/model/key

* feat: feature parity (cost page) with existing 'usage' page

* build(ui/): add activity tab to new_usage.tsx

gets to feature parity on 'All Up' page of 'usage.tsx'

* fix(proxy/utils.py): count number of api requests in daily user spend table

allows us to see activity by model on new usage tab

* style(new_usage.tsx): fix y-axis to be in ascending order of date

* fix: fix linting errors

* fix: fix ruff check errors
2025-03-27 23:29:15 -07:00
Krrish Dholakia b155f4f4a6 fix(proxy_server.py): Fix https://github.com/BerriAI/litellm/issues/9576 2025-03-27 23:25:46 -07:00
Ishaan Jaff 1ba73f491a working cron job updates 2025-03-27 23:13:01 -07:00
Krrish Dholakia 5203382702 test: run test earlier to catch error 2025-03-27 23:08:52 -07:00
Ishaan Jaff 666cd8d4e3 handle commit updates to the DB 2025-03-27 22:55:36 -07:00
Ishaan Jaff 0cb6cb6c66 fix schema to work with lock acquisition 2025-03-27 22:54:46 -07:00
Krish DholakiaandGitHub ccbac691e5 Support discovering gemini, anthropic, xai models by calling their /v1/model endpoint (#9530)
* fix: initial commit for adding provider model discovery to gemini

* feat(gemini/): add model discovery for gemini/ route

* docs(set_keys.md): update docs to show you can check available gemini models as well

* feat(anthropic/): add model discovery for anthropic api key

* feat(xai/): add model discovery for XAI

enables checking what models an xai key can call

* ci: bump ci config yml

* fix(topaz/common_utils.py): fix linting error

* fix: fix linting error for python38
2025-03-27 22:50:48 -07:00
Ishaan Jaff 758182fc7f fix typo on codebase 2025-03-27 22:36:00 -07:00
Ishaan Jaff fc46f6b861 fix db spend update buffer 2025-03-27 22:34:15 -07:00
Ishaan Jaff 1bfffadd05 fix typo 2025-03-27 22:31:38 -07:00
Krish DholakiaandGitHub 340a63bace fix(mistral_chat_transformation.py): add missing comma (#9606) 2025-03-27 22:16:21 -07:00
Krrish Dholakia 79175ddb53 test: fix test 2025-03-27 22:04:59 -07:00
Ishaan Jaff 9139230531 update the correct set of txs 2025-03-27 21:54:12 -07:00
Ishaan Jaff aa6c34cb97 use lock manager for update spend job 2025-03-27 21:43:02 -07:00
Ishaan Jaff dd00429ade update schema 2025-03-27 21:36:22 -07:00
Ishaan Jaff ea93a09b7a add model CronJob 2025-03-27 21:10:26 -07:00
Krrish Dholakia b3c7785240 test: skip flaky test - failing due to db timeouts - unrelated to test 2025-03-27 20:34:26 -07:00
Krrish Dholakia e2d4597588 test: mark flaky test 2025-03-27 20:10:57 -07:00
Shixian ShengandGitHub 1785d5e4c7 Merge branch 'BerriAI:main' into main 2025-03-27 22:18:42 -04:00