Timothy Lowrimore
bb259a651c
removes extraneous heading from readme
2025-08-12 09:11:16 -06:00
Timothy Lowrimore and GitHub
c0d82545d3
Merge branch 'main' into heroku-llms
2025-08-12 09:02:51 -06:00
cccf41c0df
Update docs/my-website/docs/providers/heroku.md
...
Co-authored-by: Claire Riley <claire.riley@salesforce.com >
2025-08-12 08:46:48 -06:00
1342abf163
Update docs/my-website/docs/providers/heroku.md
...
Co-authored-by: Claire Riley <claire.riley@salesforce.com >
2025-08-12 08:46:16 -06:00
586c0a7be5
Update docs/my-website/docs/providers/heroku.md
...
Co-authored-by: Claire Riley <claire.riley@salesforce.com >
2025-08-12 08:45:47 -06:00
7b0ecb1847
Update docs/my-website/docs/providers/heroku.md
...
Co-authored-by: Claire Riley <claire.riley@salesforce.com >
2025-08-12 08:45:36 -06:00
d4e320a4ce
Update docs/my-website/docs/providers/heroku.md
...
Co-authored-by: Claire Riley <claire.riley@salesforce.com >
2025-08-12 08:45:21 -06:00
5857d17fad
Update docs/my-website/docs/providers/heroku.md
...
Co-authored-by: Claire Riley <claire.riley@salesforce.com >
2025-08-12 08:45:06 -06:00
3cfb85228f
Update docs/my-website/docs/providers/heroku.md
...
Co-authored-by: Claire Riley <claire.riley@salesforce.com >
2025-08-12 08:44:52 -06:00
417b18a727
Update docs/my-website/docs/providers/heroku.md
...
Co-authored-by: Claire Riley <claire.riley@salesforce.com >
2025-08-12 08:42:46 -06:00
fa587c2eba
Update docs/my-website/docs/providers/heroku.md
...
Co-authored-by: Claire Riley <claire.riley@salesforce.com >
2025-08-12 08:42:27 -06:00
1be33ec47f
Update docs/my-website/docs/providers/heroku.md
...
Co-authored-by: Claire Riley <claire.riley@salesforce.com >
2025-08-12 08:41:38 -06:00
00c9f1fe0e
Update docs/my-website/docs/providers/heroku.md
...
Co-authored-by: Claire Riley <claire.riley@salesforce.com >
2025-08-12 08:41:22 -06:00
e97fa833b3
Update docs/my-website/docs/providers/heroku.md
...
Co-authored-by: Claire Riley <claire.riley@salesforce.com >
2025-08-12 08:41:08 -06:00
b20b28a912
Update docs/my-website/docs/providers/heroku.md
...
Co-authored-by: Claire Riley <claire.riley@salesforce.com >
2025-08-12 08:40:54 -06:00
e2165ff76e
Update docs/my-website/docs/providers/heroku.md
...
Co-authored-by: Claire Riley <claire.riley@salesforce.com >
2025-08-12 08:40:43 -06:00
Ishaan Jaff and GitHub
008ea864a7
[Feat] - Add key/team logging for Langfuse OTEL Logger ( #13512 )
...
* feat - add key/team logging for LF
* test_construct_dynamic_otel_headers_with_langfuse_keys
* update LangfuseOtelLogger
* test_construct_dynamic_otel_headers_with_langfuse_keys
* cleanup
* OpenTelemetryConfig fixes
* fix code qa checks
* TestLangfuseOtelIntegration
2025-08-11 22:06:25 -07:00
tanjiro and GitHub
d5135bba31
Allow unsetting TPM and RPM - Teams Settings ( #13430 )
...
* modified tpm rpm for 0 and null values
* show unlimited if tpm/rpm 0 or null
* bring back unlimited
2025-08-11 18:11:43 -07:00
NULL and GitHub
f3dcae2528
feat: add CometAPI provider support with chat completions and streaming ( #13458 )
...
* feat: add CometAPI support with config, error handling and tests
* fix: specify type for extra_body in CometAPIConfig
---------
Signed-off-by: NULL <129579691+TensorNull@users.noreply.github.com >
2025-08-11 18:06:37 -07:00
breno-aumo and GitHub
7484a19edf
Fix OCI streaming ( #13437 )
...
* add oci models to model_prices_and_context_window.json
* remove unsupported and unavailable oci models from docs
* fix: split data chunks when using async streaming
* add oci models to model_prices_and_context_window.json
* fix: split data chunks when using async streaming
* revert model.json modifications
2025-08-11 18:04:58 -07:00
Ishaan Jaff and GitHub
49ca4dc792
Sending tags to LiteLLM Proxy ( #13517 )
2025-08-11 17:58:50 -07:00
Krrish Dholakia
21e067e47f
fix: fix check
2025-08-11 17:56:19 -07:00
Krrish Dholakia
92ebf5b918
fix(router.py): fix print statement
2025-08-11 17:46:14 -07:00
Jugal D. Bhatt and GitHub
5efe59e6e2
[Draft] [LLM Translation] Add model id check ( #13507 )
...
* Add model id check
* add ruff fix
2025-08-11 15:41:08 -07:00
Ishaan Jaff and GitHub
9f78287000
[Bug Fix]: Azure OpenAI GPT-5 max_tokens + reasoning param support ( #13510 )
...
* add AzureOpenAIGPT5Config
* add AzureOpenAIGPT5Config
* add AzureOpenAIGPT5Config
* add AzureOpenAIGPT5Config
* test_azure_gpt5_supports_reasoning_effort
* test_azure_gpt5_reasoning
* test_azure_gpt5_reasoning
* ruff check fixes
* docs azure gpt5
2025-08-11 15:40:53 -07:00
Timothy Lowrimore
cc3ec33fe9
removes unused imports that are causing linting failures
2025-08-11 09:52:20 -06:00
Timothy Lowrimore and GitHub
95d9e30448
Merge branch 'main' into heroku-llms
2025-08-11 09:45:52 -06:00
Timothy Lowrimore
51b52534fb
fixes misconfigured OCI models by setting supports_tool_choice: true
2025-08-11 09:40:40 -06:00
Ishaan Jaff and GitHub
1cd827874f
[Bug Fix] - Allow using reasoning_effort for gpt-5 model family and reasoning for Responses API ( #13475 )
...
* test_openai_gpt5_reasoning
* test_openai_gpt5_reasoning_effort_parameter
* add OpenAIGPT5ResponsesAPIConfig
* test_openai_gpt5_reasoning_effort_parameter
* fixes
2025-08-10 09:55:36 -07:00
Krrish Dholakia
bd8a0ae0d0
docs: fix order
2025-08-10 09:42:55 -07:00
Krrish Dholakia
1dbac75675
docs(index.md): update release with deployment information
2025-08-10 09:31:28 -07:00
Krish Dholakia and GitHub
0aeb4f1653
fix(health_check_helpers.py): set max tokens for wildcard call to 10, fixes calling gpt-5-nano via wildcard on openai ( #13482 )
...
gpt-5-nano raises errors for max_tokens=1
2025-08-10 09:23:36 -07:00
Krish Dholakia and GitHub
184687157e
Litellm model cost map fixes ( #13480 )
...
* build(model_prices_and_context_window.json): fix max token values
* build(model_prices_and_context_window.json): fix max token values
* build(model_prices_and_context_window.json): fix azure gpt-5-chat pricing
2025-08-10 07:38:35 -07:00
Krish Dholakia and GitHub
c742c76288
Litellm release notes 08 10 2025 ( #13479 )
...
* docs(index.md): initial doc
* build(index.md): initial notes
* docs(index.md): add llm translation tickets
* docs(index.md): document new model support
* docs(index.md): document all pricing changes
* docs(index.md): add llm api endpoints
* docs(index.md): add doc on mcp gateway
* docs(index.md): add all remaining rc notes
* docs(index.md): cleanup
2025-08-10 07:32:11 -07:00
Krrish Dholakia
ece2c9c65d
bump: version 1.75.4 → 1.75.5
2025-08-09 16:31:51 -07:00
Krrish Dholakia
0eedf7c447
build: update local model cost map
2025-08-09 16:31:41 -07:00
Krish Dholakia and GitHub
9f6f96d76c
Litellm dev 08 07 2025 p1 ( #13418 )
...
* fix(router.py): support base model for model group usage
allows model group info to show accurate cost information for azure models
* fix(router.py): fix changes
* test: add unit tests
* build(pyproject.toml): bump openai version requirements
support custom tool from responses api
Closes https://github.com/BerriAI/litellm/issues/13391
* docs(responses_api.md): add verbosity + free-form function calling parameters
* docs(responses_api.md): add cfg + minimal reasoning to docs
Closes https://github.com/BerriAI/litellm/issues/13391
* docs(responses_api.md): add proxy examples to docs
* refactor: fix ruff error
2025-08-09 16:30:04 -07:00
Sannan Nasir and GitHub
0e53b1feab
Add digitalocean provider ( #12169 )
...
* Add digitalocean provider
* Add digitalocean provider
* Revert "Add digitalocean provider"
This reverts commit 96dda40f45b3d12ea03e861d060ec81460b7759e.
* changes
* fixes
* Update transformation
* refactoring
* rename provider to Gradient AI
* fixes
* Incorporte review comments
* revert changes
* fix typo
* revert change
* incorporated review comments
* Revert "Incorporte review comments"
This reverts commit 37bd51bd54ef4fd52ccc12866e47f8de9476d597.
* changes
* Revert "Revert "Incorporte review comments"
This reverts commit 37bd51bd54ef4fd52ccc12866e47f8de9476d597."
This reverts commit 68c8a198ee0d6441c3a52f6c6a49c9c95a4cb0a8.
* changes
* fixes
* Update provider_specific_fields.tsx
2025-08-09 16:26:33 -07:00
Ishaan Jaff and GitHub
f60a9cf908
[Bug]: Fix JWTs access not working with model groups ( #13474 )
...
* fix can_team_access_model
* test_find_team_with_model_access_model_group
2025-08-09 16:14:51 -07:00
Jugal D. Bhatt and GitHub
95fbe59c46
Add local storage auth ( #13473 )
2025-08-09 16:13:56 -07:00
Jugal D. Bhatt and GitHub
67833590d6
[Proxy changes] Litellm add model price reload schedule for multi-pod ( #13470 )
...
* added mcp guardrails doc in mcp.md
* add button to reload models
* Added button changes
* added button for scheduling reload
* add multi pod support to reloading the model price json
* fix ruff
2025-08-09 16:12:13 -07:00
Krish Dholakia and GitHub
1c8761111f
Router - reduce p99 latency w/ redis enabled by 50% + OTEL - track pre_call hook latency ( #13362 )
...
* feat(proxy/utils.py): track pre-call hooks in OTEL
some pre call hooks can cause latency in high traffic - make sure this is tracked
* fix(router.py): move redis call on deployment_callback_on_success to pipeline operation
reduces p99 latency by half when redis is enabled
* fix(parallel_request_limiter_v3.py): only run check if any item has rate limits set
Prevents unnecessary latency added by rate limit checks
* test: add unit tests
* Latency Improvements: only track tpm/rpm usage when set on deployment+ LLM Caching - use an in-memory cache to reduce redis calls + OTEL - track time spent on LLM caching (#13472 )
* fix(router.py): only track usage for deployments with tpm/rpm set
ensures additional latency avoided for non-tpm/rpm models
* fix(caching_handler.py): log time spent on request get cache to OTEL
enables easy debugging of call latency
* fix(caching_handler.py): use dual cache object for in-memory caching + trace redis call within caching handler
* fix(caching_handler.py): working in-memory cache for redis calls
ensures dual cache works when redis cache setup for llm calls
makes calls quicker by only checking redis when in-memory cache missed for llm api call
* test: remove redundant test
* test: add unit tests
2025-08-09 16:09:51 -07:00
Ishaan Jaff and GitHub
60306d34a0
[Bug Fix] Allow using Swagger for /chat/completions ( #13469 )
...
* fix get_openapi_schema
* fixes for ProxyChatCompletionRequest
* TestSwaggerChatCompletions
* fix working request body
* fix - add "messages"
* fix messages
* TestSwaggerChatCompletions
* test_messages_field_has_example
* ruff check fix
2025-08-09 15:35:45 -07:00
Jugal D. Bhatt and GitHub
1270df08a4
[Proxy + UI] Litellm add reload model api and button ( #13464 )
...
* added mcp guardrails doc in mcp.md
* add button to reload models
* Added button changes
* remove the model_reload
2025-08-09 13:52:56 -07:00
Jugal D. Bhatt and GitHub
10a1fe21c5
[LLM Translation] Litellm azure o series drop params ( #13353 )
...
* added route check
* fix ruff
* Added support for dropping o_series params
* Added ruff fix
* fix tests
2025-08-09 13:52:45 -07:00
6184e898b7
Generate unique IDs for litellm_call_id and function_id using UUID ( #13468 )
...
Co-authored-by: Cursor Agent <cursoragent@cursor.com >
Co-authored-by: ishaan <ishaan@berri.ai >
2025-08-09 12:59:09 -07:00
Ishaan Jaff and GitHub
eb4bd26f24
[Bug Fix] - Get Routes ( #13466 )
...
* fixes get_routes_for_mounted_app
* fix - use _safe_get_endpoint_name
* fix code QA check
* test_get_routes_for_mounted_app_with_static_files
* test fixes
2025-08-09 12:52:23 -07:00
Ishaan Jaff and GitHub
825ea65b96
[Bug Fix] Responses API - Responses API failed if input containing ResponseReasoningItem ( #13465 )
...
* add test_responses_api_multi_turn_with_reasoning_and_structured_output
* fix transform_responses_api_request
2025-08-09 11:20:34 -07:00
Ishaan Jaff
ee40db7b31
docs native litellm prompts
2025-08-09 09:46:31 -07:00
Ishaan Jaff and GitHub
94c33200a4
docs - native prompt mgmt ( #13463 )
2025-08-09 09:39:16 -07:00
Timothy Lowrimore