Commit Graph
23118 Commits
Author SHA1 Message Date
Krrish Dholakia dee5182fc8 fix: fix linting error 2025-04-19 08:04:56 -07:00
Ishaan Jaff a62805f98f fixes for assistans API tests 2025-04-19 07:59:53 -07:00
Ishaan Jaff 5bf76f0bb1 test fixes for azure assistants 2025-04-19 07:36:40 -07:00
Krish DholakiaandGitHub ef6ac42658 Litellm dev 04 18 2025 p2 (#10157)
* fix(proxy/_types.py): allow internal user to call api playground

* fix(new_usage.tsx): cleanup tag based usage - only show for proxy admin

not clear what tags internal user should be allowed to see

* fix(team_endpoints.py): allow internal user view spend for teams they belong to

* fix(team_endpoints.py): return team alias on `/team/daily/activity` API

allows displaying team alias on ui

* fix: fix linting error

* fix(entity_usage.tsx): allow viewing top keys by team

* fix(entity_usage.tsx): show alias, if available in breakdown

allows entity alias to be easily displayed

* Show usage by key (on all up, team, and tag usage dashboards)  (#10152)

* fix(entity_usage.tsx): allow user to select team in team usage tab

* fix(new_usage.tsx): load all tags for filtering

* fix(tag_management_endpoints.py): return dynamic tags from db on `/tag/list`

* fix(litellm_pre_call_utils.py): support x-litellm-tags even if tag based routing not enabled

* fix(new_usage.tsx): show breakdown of usage by api key on dashboard

helpful when looking at spend by team

* fix(networking.tsx): exclude litellm-dashboard team id's from calls

adds noisy ui tokens to key activity

* fix(new_usage.tsx): allow user to see activity by key on main tab

* feat(internal_user_endpoints.py): refactor to use common_daily_activity function

reuses same logic across teams/keys/tags

Allows returning team_alias in api_keys consistently

* fix(leftnav.tsx): swap old usage with new usage tab

* fix(entity_usage.tsx): show breakdown of teams in daily spend chart

* style(new_usage.tsx): show global usage tab if user is admin / has admin view

* fix(new_usage.tsx): add disclaimer for new usage dashboard

* fix(new_usage.tsx): fix linting error

* Allow filtering usage dashboard by team + tag (#10150)

* fix(entity_usage.tsx): allow user to select team in team usage tab

* fix(new_usage.tsx): load all tags for filtering

* fix(tag_management_endpoints.py): return dynamic tags from db on `/tag/list`

* fix(litellm_pre_call_utils.py): support x-litellm-tags even if tag based routing not enabled

* fix: fix linting error
2025-04-19 07:32:23 -07:00
Ishaan Jaff b9756bf006 test_completion_azure 2025-04-19 07:24:11 -07:00
Krish DholakiaandGitHub a1879cfa35 fix(litellm-proxy-extras/utils.py): prisma migrate improvements: handle existing columns in db table (#10138) 2025-04-18 20:36:56 -07:00
Krrish Dholakia 652e1b7f0f test: update test 2025-04-18 20:36:15 -07:00
Ishaan Jaff 6de3481252 doc fix 2025-04-18 19:57:03 -07:00
Ishaan Jaff 76f00a5121 add info on litellm release 2025-04-18 19:56:29 -07:00
Ishaan JaffandGitHub 3d5022bd79 [Feat] Support for all litellm providers on Responses API (works with Codex) - Anthropic, Bedrock API, VertexAI, Ollama (#10132)
* transform request

* basic handler for LiteLLMCompletionTransformationHandler

* complete transform litellm to responses api

* fixes to test

* fix stream=True

* fix streaming iterator

* fixes for transformation

* fixes for anthropic codex support

* fix pass response_api_optional_params

* test anthropic responses api tools

* update responses types

* working codex with litellm

* add session handler

* fixes streaming iterator

* fix handler

* add litellm codex example

* fix code quality

* test fix

* docs litellm codex

* litellm codexdoc

* docs openai codex with litellm

* docs litellm openai codex

* litellm codex

* linting fixes for transforming responses API

* fix import error

* fix responses api test

* add sync iterator support for responses api
v1.66.3.dev5
2025-04-18 19:53:59 -07:00
Krrish Dholakia 3e87ec4f16 test: replace removed fireworks ai models 2025-04-18 14:23:16 -07:00
Krish DholakiaandGitHub 1ea046cc61 test: update tests to new deployment model (#10142)
* test: update tests to new deployment model

* test: update model name

* test: skip cohere rbac issue test

* test: update test - replace gpt-4o model
2025-04-18 14:22:12 -07:00
Krrish Dholakia 415abfc222 test: update test 2025-04-18 13:13:58 -07:00
David EmmanuelandGitHub de3c2d14bf Add Gemini Flash 2.5 Preview Model Price and Context Window (#10125)
* Update model_prices_and_context_window_backup.json

* Update model_prices_and_context_window.json
2025-04-18 09:44:46 -07:00
Krrish Dholakia f7dd688035 test: handle cohere rbac issue (verified happens on calling azure directly) 2025-04-18 08:42:12 -07:00
Krrish Dholakia 809eb859cf fix(azure/o_series_transformation.py): fix azure o4 model routing
Fixes https://github.com/BerriAI/litellm/pull/10065#issuecomment-2814015058
2025-04-17 22:58:01 -07:00
Krrish Dholakia 614d80cb1b build(model_prices_and_context_window.json): add azure gpt-4.1 pricing
ensures cost tracking for gpt-4.1 works
2025-04-17 20:09:17 -07:00
Marc AbramowitzandGitHub 3c71a81100 Remove unnecessary package*.json files (#10075) 2025-04-17 20:03:56 -07:00
Ishaan Jaff 19664960eb docs azure responses API 2025-04-17 18:47:44 -07:00
Marc AbramowitzandGitHub 409dde22f6 UI: Make columns resizable/hideable in Models table (#10119)
* Make columns resizable in Models table

* Make edit and delete buttons sticky on right side

* Add Columns dropdown to control which columns are shown

* Remove unnecessary dependencies

* Fix title of visibility checkboxes for Input Cost and Output Cost

* Make the Columns dropdown close if the user clicks anywhere outside of it
2025-04-17 18:12:20 -07:00
Ishaan JaffandGitHub d3e04eac7f [Feat] Unified Responses API - Add Azure Responses API support (#10116)
* initial commit for azure responses api support

* update get complete url

* fixes for responses API

* working azure responses API

* working responses API

* test suite for responses API

* azure responses API test suite

* fix test with complete url

* fix test refactor

* test fix metadata checks

* fix code quality check
2025-04-17 16:47:59 -07:00
Krrish Dholakia 8be8022914 docs(vertex_ai.md): document new vertex passthrough route 2025-04-16 22:25:16 -07:00
Krrish Dholakia ff81f48af3 bump: version 1.66.2 → 1.66.3 v1.66.2.dev1 v1.66.3-nightly 2025-04-16 22:20:10 -07:00
Krrish Dholakia 78c6d73dea build: new ui build 2025-04-16 22:11:53 -07:00
Ishaan Jaff 257e78ffb5 test fix vertex_ai/mistral-large@2407 2025-04-16 21:52:52 -07:00
Krish DholakiaandGitHub 8ddaf3dfbc fix(o_series_transformation.py): correctly map o4 to openai o_series model (#10079)
Fixes https://github.com/BerriAI/litellm/issues/10066
2025-04-16 21:51:31 -07:00
Krish DholakiaandGitHub c73a6a8d1e Add new /vertex_ai/discovery route - enables calling AgentBuilder API routes (#10084)
* feat(llm_passthrough_endpoints.py): expose new `/vertex_ai/discovery/` endpoint

Allows calling vertex ai discovery endpoints via passthrough

 For agentbuilder api calls

* refactor(llm_passthrough_endpoints.py): use common _base_vertex_proxy_route

Prevents duplicate code

* feat(llm_passthrough_endpoints.py): add vertex endpoint specific passthrough handlers
2025-04-16 21:45:51 -07:00
Ishaan Jaff 198922b26f test fixes for vertex mistral, this model was deprecated on vertex 2025-04-16 20:51:45 -07:00
Ishaan Jaff c38146e180 test fix 2025-04-16 20:13:31 -07:00
Ishaan Jaff cf801f9642 test fix vertex_ai/codestral 2025-04-16 20:01:36 -07:00
0ced13aec8 Virtual Keys: Filter by key alias (#10035) (#10085)
Co-authored-by: Christian Owusu <36159205+crisshaker@users.noreply.github.com>
2025-04-16 19:46:05 -07:00
Ishaan Jaff 2b14978d9d bump litellm proxy extras 2025-04-16 19:28:16 -07:00
Ishaan Jaff 12ccb954a6 ui new build 2025-04-16 19:23:04 -07:00
Ishaan JaffandGitHub 6220f3e7b8 [Feat SSO] Add LiteLLM SCIM Integration for Team and User management (#10072)
* fix NewUser response type

* add scim router

* add v0 scim v2 endpoints

* working scim transformation

* use 1 file for types

* fix scim firstname and givenName storage

* working SCIMErrorResponse

* working team / group provisioning on SCIM

* add SCIMPatchOp

* move scim folder

* fix import scim_router

* fix dont auto create scim keys

* add auth on all scim endpoints

* add is_virtual_key_allowed_to_call_route

* fix allowed routes

* fix for key management

* fix allowed routes check

* clean up error message

* fix code check

* fix for route checks

* ui SCIM support

* add UI tab for SCIM

* fixes SCIM

* fixes for SCIM settings on ui

* scim settings

* clean up scim view

* add migration for allowed_routes in keys table

* refactor scim transform

* fix SCIM linting error

* fix code quality check

* fix ui linting

* test_scim_transformations.py
2025-04-16 19:21:47 -07:00
Krish DholakiaandGitHub 7ca553b235 Add team based usage dashboard at 1m+ spend logs (+ new /team/daily/activity API) (#10081)
* feat(ui/): add team based usage to dashboard

allows admin to see spend across teams + within teams at 1m+ spend logs

* fix(entity_usage.tsx): add activity page to entity usage

* style(entity_usage.tsx): place filter above tab switcher
2025-04-16 18:10:14 -07:00
Krish DholakiaandGitHub c0d7e9f16d Add new /tag/daily/activity endpoint + Add tag dashboard to UI (#10073)
* feat: initial commit adding daily tag spend table to db

* feat(db_spend_update_writer.py): correctly log tag spend transactions

* build(schema.prisma): add new tag table to root

* build: add new migration file

* feat(common_daily_activity.py): add `/tag/daily/activity` API endpoint

allows viewing daily spend by tag

* feat(tag_management_endpoints.py): support comma separated list of tags + tag breakdown metric

allows querying multiple tags + knowing what tags are driving spend

* feat(entity_usage.tsx): initial commit adding tag based usage to litellm dashboard

brings back tag based usage tracking to UI at 1m+ spend logs

* feat(entity_usage.tsx): add top api key view to ui

* feat(entity_usage.tsx): add tag table to ui

* feat(entity_usage.tsx): allow filtering by tag

* refactor(entity_usage.tsx): reorder components

* build(ui/): fix linting error

* fix: fix ruff checks

* fix(schema.prisma): drop uniqueness requirement on tag

allows dailytagspend to have multiple rows with the same tag

* build(schema.prisma): drop uniqueness requirement on tag in dailytagspend

allows tag agg. view to work on multiple rows with same tag

* build(schema.prisma): drop tag uniqueness requirement
2025-04-16 15:24:44 -07:00
Peter Dave HelloandGitHub 5c078af738 Add OpenAI o3 & 4o-mini (#10065)
Reference:
- https://platform.openai.com/docs/models/o3
- https://platform.openai.com/docs/models/o4-mini
2025-04-16 12:40:13 -07:00
Krish DholakiaandGitHub d8a1071bc4 Add aggregate spend by tag (#10071)
* feat: initial commit adding daily tag spend table to db

* feat(db_spend_update_writer.py): correctly log tag spend transactions

* build(schema.prisma): add new tag table to root

* build: add new migration file
2025-04-16 12:26:21 -07:00
Krish DholakiaandGitHub 47e811d6ce fix(llm_http_handler.py): fix fake streaming (#10061)
* fix(llm_http_handler.py): fix fake streaming

allows groq to work with llm_http_handler

* fix(groq.py): migrate groq to openai like config

ensures json mode handling works correctly
v1.66.2-nightly
2025-04-16 10:15:11 -07:00
Krish DholakiaandGitHub c603680d2a fix(stream_chunk_builder_utils.py): don't set index on modelresponse (#10063)
* fix(stream_chunk_builder_utils.py): don't set index on modelresponse

* test: update tests
2025-04-16 10:11:47 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
7b7b43e1a7 build(deps): bump http-proxy-middleware in /docs/my-website (#10064)
Bumps [http-proxy-middleware](https://github.com/chimurai/http-proxy-middleware) from 2.0.7 to 2.0.9.
- [Release notes](https://github.com/chimurai/http-proxy-middleware/releases)
- [Changelog](https://github.com/chimurai/http-proxy-middleware/blob/v2.0.9/CHANGELOG.md)
- [Commits](https://github.com/chimurai/http-proxy-middleware/compare/v2.0.7...v2.0.9)

---
updated-dependencies:
- dependency-name: http-proxy-middleware
  dependency-version: 2.0.9
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-04-16 09:55:44 -07:00
a9e8a36f89 [Bug Fix] Azure Blob Storage fixes (#10059)
* Simple fix for #9339 - upgrade the underlying library and cache the azure storage client (#9965)

* fix -  use constants for caching azure storage client

---------

Co-authored-by: Adrian Lyjak <adrian@chatmeter.com>
2025-04-16 09:47:10 -07:00
Krrish Dholakia a743b6fc1f fix(bedrock/common_utils.py): add us-west-1 to us regions 2025-04-16 08:00:39 -07:00
Krrish Dholakia 6b7d20c911 test: fix test 2025-04-16 07:57:10 -07:00
ChaoFu YangandGitHub c07eea864e /utils/token_counter: get model_info from deployment directly (#10047) 2025-04-16 07:53:18 -07:00
Michael LeshchinskyandGitHub e19d05980c Add litellm call id passing to Aim guardrails on pre and post-hooks calls (#10021)
* Add litellm_call_id passing to aim guardrails on pre and post-hooks

* Add test that ensures that pre_call_hook receives litellm call id when common_request_processing called
2025-04-16 07:41:28 -07:00
Ishaan Jaff ca593e003a bump litellm-proxy-extras==0.1.9 2025-04-15 22:49:24 -07:00
Ishaan Jaff 1d4fea509d ui new build 2025-04-15 22:36:44 -07:00
Ishaan Jaff ad09d250ef test fix azure deprecated mistral 2025-04-15 22:32:14 -07:00
Ishaan JaffandGitHub dcc43e797a [Docs] Auto prompt caching (#10044)
* docs prompt cache controls

* doc fix auto prompt caching
2025-04-15 22:29:47 -07:00