Commit Graph
20950 Commits
Author SHA1 Message Date
Krish DholakiaandGitHub cdcc8ea9b7 Connect UI to "LiteLLM_DailyUserSpend" spend table - enables usage tab to work at 1m+ spend logs (#9603)
* feat(spend_management_endpoints.py): expose new endpoint for querying user's usage at 1m+ spend logs

Allows user to view their spend at 1m+ spend logs

* build(schema.prisma): add api_requests to dailyuserspend table

* build(migration.sql): add migration file for new column to daily user spend table

* build(prisma_client.py): add logic for copying over migration folder, if deploy/migrations present in expected location

enables easier testing of prisma migration flow

* build(ui/): initial commit successfully using the dailyuserspend table on the UI

* refactor(internal_user_endpoints.py): refactor `/user/daily/activity` to give breakdowns by provider/model/key

* feat: feature parity (cost page) with existing 'usage' page

* build(ui/): add activity tab to new_usage.tsx

gets to feature parity on 'All Up' page of 'usage.tsx'

* fix(proxy/utils.py): count number of api requests in daily user spend table

allows us to see activity by model on new usage tab

* style(new_usage.tsx): fix y-axis to be in ascending order of date

* fix: fix linting errors

* fix: fix ruff check errors
2025-03-27 23:29:15 -07:00
Krrish Dholakia b155f4f4a6 fix(proxy_server.py): Fix https://github.com/BerriAI/litellm/issues/9576 2025-03-27 23:25:46 -07:00
Krrish Dholakia 5203382702 test: run test earlier to catch error 2025-03-27 23:08:52 -07:00
Krish DholakiaandGitHub ccbac691e5 Support discovering gemini, anthropic, xai models by calling their /v1/model endpoint (#9530)
* fix: initial commit for adding provider model discovery to gemini

* feat(gemini/): add model discovery for gemini/ route

* docs(set_keys.md): update docs to show you can check available gemini models as well

* feat(anthropic/): add model discovery for anthropic api key

* feat(xai/): add model discovery for XAI

enables checking what models an xai key can call

* ci: bump ci config yml

* fix(topaz/common_utils.py): fix linting error

* fix: fix linting error for python38
2025-03-27 22:50:48 -07:00
Krish DholakiaandGitHub 340a63bace fix(mistral_chat_transformation.py): add missing comma (#9606) 2025-03-27 22:16:21 -07:00
Krrish Dholakia 79175ddb53 test: fix test 2025-03-27 22:04:59 -07:00
Krrish Dholakia b3c7785240 test: skip flaky test - failing due to db timeouts - unrelated to test 2025-03-27 20:34:26 -07:00
Krrish Dholakia e2d4597588 test: mark flaky test 2025-03-27 20:10:57 -07:00
Krish DholakiaandGitHub b9d0f460e8 Revert "Support max_completion_tokens on Mistral (#9589)" (#9604)
This reverts commit fef5d23dd5.
2025-03-27 19:14:26 -07:00
Chris MancusoandGitHub fef5d23dd5 Support max_completion_tokens on Mistral (#9589)
* Support max_completion_tokens on Mistral

* test fix
2025-03-27 17:27:19 -07:00
Krish DholakiaandGitHub fb83567a03 Litellm new UI build (#9601)
* build: new ui build

* build: new ui build

* fix(proxy_server.py): only show user models their key can access on `/models`

* fix(model_management_endpoints.py): ensure team admin can add models

* test: update unit testing to reflect changes

* fix(model_dashboard.tsx): fix sizing on models page

* build: fix ui
2025-03-27 17:15:25 -07:00
Krish DholakiaandGitHub 11838e1c3b Litellm fix db testing (#9593)
* ci: fix test

* test: safely change db url

* fix: print db url

* test: remove delenv
2025-03-27 14:50:41 -07:00
Krrish Dholakia f1f40eba3f docs(index.md): document new team model flow 2025-03-27 12:14:29 -07:00
Krrish Dholakia f1fa0044cb bump: version 1.64.1 → 1.65.0 2025-03-27 12:07:17 -07:00
Krish DholakiaandGitHub 63c9f59373 Allow team admins to add/update/delete models on UI + show api base and model id on request logs (#9572)
* feat(view_logs.tsx): show model id + api base in request logs

easier debugging

* fix(index.tsx): fix length of api base

easier viewing

* refactor(leftnav.tsx): show models tab to team admin

* feat(model_dashboard.tsx): add explainer for what the 'models' page is for team admin

helps them understand how they can use it

* feat(model_management_endpoints.py): restrict model add by team to just team admin

allow team admin to add models via non-team keys (e.g. ui token)

* test(test_add_update_models.py): update unit testing for new behaviour

* fix(model_dashboard.tsx): show user the models

* feat(proxy_server.py): add new query param 'user_models_only' to `/v2/model/info`

Allows user to retrieve just the models they've added

Used in UI to show internal users just the models they've added

* feat(model_dashboard.tsx): allow team admins to view their own models

* fix: allow ui user to fetch model cost map

* feat(add_model_tab.tsx): require team admins to specify team when onboarding models

* fix(_types.py): add `/v1/model/info` to info route

`/model/info` was already there

* fix(model_info_view.tsx): allow user to edit a model they created

* fix(model_management_endpoints.py): allow team admin to update team model

* feat(model_managament_endpoints.py): allow team admin to delete team models

* fix(model_management_endpoints.py): don't require team id to be set when adding a model

* fix(proxy_server.py): fix linting error

* fix: fix ui linting error

* fix(model_management_endpoints.py): ensure consistent auth checks on all model calls

* test: remove old test - function no longer exists in same form

* test: add updated mock testing
2025-03-27 12:06:31 -07:00
Krrish Dholakia 3ca34a181c docs(openai.md): add gpt-4o-transcribe to docs 2025-03-26 23:23:35 -07:00
Krish DholakiaandGitHub 6ba31e1d84 Allow viewing keyinfo on request logs (#9568)
* feat(view_logs.tsx): show model id + api base in request logs

easier debugging

* fix(index.tsx): fix length of api base

easier viewing

* build(ui/): initial commit allowing user to click into key from request logs

allows easier debugging of 'what key is this?
2025-03-26 23:11:15 -07:00
Krish DholakiaandGitHub c0845fec1f Add OpenAI gpt-4o-transcribe support (#9517)
* refactor: introduce new transformation config for gpt-4o-transcribe models

* refactor: expose new transformation configs for audio transcription

* ci: fix config yml

* feat(openai/transcriptions): support provider config transformation on openai audio transcriptions

allows gpt-4o and whisper audio transformation to work as expected

* refactor: migrate fireworks ai + deepgram to new transform request pattern

* feat(openai/): working support for gpt-4o-audio-transcribe

* build(model_prices_and_context_window.json): add gpt-4o-transcribe to model cost map

* build(model_prices_and_context_window.json): specify what endpoints are supported for `/audio/transcriptions`

* fix(get_supported_openai_params.py): fix return

* refactor(deepgram/): migrate unit test to deepgram handler

* refactor: cleanup unused imports

* fix(get_supported_openai_params.py): fix linting error

* test: update test
2025-03-26 23:10:25 -07:00
Krrish Dholakia 109add7946 build(model_prices_and_context_window.json): add gemini multimodal embedding cost 2025-03-26 23:04:24 -07:00
Ishaan JaffandGitHub 7097ce544e Merge pull request #9569 from BerriAI/litellm_fix_db_unavailable
[Reliability Fix] - Allow Pods to startup + passing /health/readiness when `allow_requests_on_db_unavailable: True` and DB is down
2025-03-26 21:18:52 -07:00
Ishaan Jaff 05c38049fe docs prod.md 2025-03-26 21:04:36 -07:00
Ishaan Jaff b6506f7bda test_db_health_readiness_check_with_prisma_error 2025-03-26 20:56:39 -07:00
Ishaan Jaff 87f0201f84 test_handle_db_exception_with_connection_error 2025-03-26 20:44:25 -07:00
Ishaan Jaff 34e58be36d fix order of _setup_prisma_client 2025-03-26 20:42:01 -07:00
Ishaan Jaff 427580eff5 fix _setup_prisma_client 2025-03-26 20:35:27 -07:00
Ishaan Jaff 15c04da735 refactor tests 2025-03-26 20:00:10 -07:00
Ishaan Jaff 88ef97b9d1 allow proxy to startup on DB unavailable 2025-03-26 19:50:57 -07:00
Ishaan Jaff 497570b2a6 bug fix - allow pods to startup when DB is unavailable 2025-03-26 19:49:51 -07:00
Ishaan Jaff 7142b0b610 refactor PrismaDBExceptionHandler 2025-03-26 19:22:24 -07:00
Ishaan JaffandGitHub 0155b0eba2 Merge pull request #9533 from BerriAI/litellm_stability_fixes
[Reliability Fixes] - Gracefully handle exceptions when DB is having an outage
2025-03-26 18:57:38 -07:00
Ishaan Jaff 485aa87e65 allow_requests_on_db_unavailable 2025-03-26 18:48:18 -07:00
Ishaan Jaff 763f853a9f docs fix 2025-03-26 18:32:58 -07:00
Ishaan Jaff 5242c5fbab test - auth exception handler 2025-03-26 18:28:28 -07:00
Ishaan Jaff 15b1a8afb0 test_is_database_connection_error_prisma_errors 2025-03-26 18:27:39 -07:00
Krish DholakiaandGitHub 4351c77253 Support Gemini audio token cost tracking + fix openai audio input token cost tracking (#9535)
* fix(vertex_and_google_ai_studio_gemini.py): log gemini audio tokens in usage object

enables accurate cost tracking

* refactor(vertex_ai/cost_calculator.py): refactor 128k+ token cost calculation to only run if model info has it

Google has moved away from this for gemini-2.0 models

* refactor(vertex_ai/cost_calculator.py): migrate to usage object for more flexible data passthrough

* fix(llm_cost_calc/utils.py): support audio token cost tracking in generic cost per token

enables vertex ai cost tracking to work with audio tokens

* fix(llm_cost_calc/utils.py): default to total prompt tokens if text tokens field not set

* refactor(llm_cost_calc/utils.py): move openai cost tracking to generic cost per token

more consistent behaviour across providers

* test: add unit test for gemini audio token cost calculation

* ci: bump ci config

* test: fix test
2025-03-26 17:26:25 -07:00
Krrish Dholakia 04490c99d7 test: fix test 2025-03-26 17:12:09 -07:00
Krrish Dholakia 72c0ad419f build(migration.sql): add migration file for new dailyusertable
documents prisma db changes
2025-03-26 17:11:25 -07:00
Krrish Dholakia d4adc9764b test(test_db_schema_migration.py): ci/cd test to enforce schema migrations are documented in .sql files 2025-03-26 16:59:50 -07:00
37265c8b1c Add Daily User Spend Aggregate view - allows UI Usage tab to work > 1m rows (#9538)
* ci: update github action

* build(schema.prisma): enable daily user spend table

allows storing aggregate view of user's daily spend

* build(schema.prisma): add new daily user spend table

* feat: working daily user spend tracking

maintains an aggregate view for easier querying in high traffic

* setup_google_dns

* ci: update ci yaml

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
2025-03-26 16:36:36 -07:00
Krrish Dholakia fb9abd40f3 ci(config.yml): add pytest-postgres to ci/cd 2025-03-26 16:25:08 -07:00
Krish DholakiaandGitHub 132d3f7baa feat(prisma-migrations): add baseline db migration file (#9565)
adds initial baseline db migration file

enables future schema changes to be documented via .sql files
2025-03-26 16:22:56 -07:00
Ishaan JaffandGitHub b7f4abd13a Merge pull request #9542 from BerriAI/litellm_fix_vertex_ai_ft_models
[Feature]: Support for Fine-Tuned Vertex AI LLMs
2025-03-26 16:19:41 -07:00
Ishaan Jaff 23aa7f81b5 fix ProxyException 2025-03-26 15:53:33 -07:00
Ishaan Jaff ff33ed020c fix auth checks 2025-03-26 15:45:58 -07:00
Ishaan Jaff 8bd2081dec fix get_key_object 2025-03-26 15:41:40 -07:00
Ishaan Jaff f8caebc54b is_database_connection_error 2025-03-26 14:55:40 -07:00
Ishaan Jaff 4948673e35 fix test changes 2025-03-26 14:51:33 -07:00
Ishaan Jaff 1812ce4a54 undo config.yml changes 2025-03-26 14:43:03 -07:00
Ishaan Jaff 044c56c439 Merge branch 'main' into litellm_stability_fixes 2025-03-26 14:41:43 -07:00
Ishaan Jaff 0aae9aa24a rename _is_model_gemini_spec_model 2025-03-26 14:28:26 -07:00