Commit Graph
26598 Commits
Author SHA1 Message Date
YutaSaitoandGitHub bb3b77cbeb feat: add guardrail for image generation (#15619)
* feat: add guardrail for image generation

* fix: use get_str_from_messages
2025-10-16 21:50:30 -07:00
Lucas SugiandGitHub be8854f62e fix: Convert object to a correct type (#15634) 2025-10-16 21:47:32 -07:00
Nicholas CoutureandGitHub 8032e73872 [Fix] Ensure guardrail memory sync after database updates (#15633)
* chore: Consistency in install-test-deps using poetry run

* feat: update in-memory guardrails after database CRUD operations

* test: add parameterized tests for guardrail CRUD with memory sync
2025-10-16 21:46:49 -07:00
Krrish Dholakia 814aeb5fad fix(pass_through_endpoints.py): ensure passthrough endpoints are loaded in from db correctly on restarts 2025-10-16 21:39:48 -07:00
3ff073c811 UI - add arize on ui, LLMs - clarifai refactor to openai compatible route, added azure ai/grok-4 model family
* added oauth mcp to docs

* added azure ai/grok-4 model family

* Revert "added oauth mcp to docs"

This reverts commit 950b7cef44f14b2db1429f6fbd32548a7c95d325.

* fix: arize ui integration

* need to remove a file

This reverts commit d6c877b73ac763464f204b77135f3786342373b7.

* fix: add arize from ui

* updated clarifai functions to openai compatible (#15615)

* fix: npm build errors

* Snowflake provider support: added embeddings, PAT, account_id (#15372)

* snowflake support PAT, account_id and embeddings

* format

* test embeddings

* format

* complete test

---------

Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>

* Revert "Snowflake provider support: added embeddings, PAT, account_id (#15372)" (#15632)

This reverts commit c6d58e5b4af8493c020fa519d72ec6ebc90c896b.

---------

Co-authored-by: mubashir1osmani <mubashir.osmani777@gmail.com>
Co-authored-by: Mubashir Osmani <ilikewafflesomcuh@gmail.com>
Co-authored-by: mogith-pn <143642606+mogith-pn@users.noreply.github.com>
Co-authored-by: Andrey <elkin.andr@gmail.com>
2025-10-16 20:39:15 -07:00
Ishaan Jaffer 39b0e76a33 test_completion_cohere_command_r_plus_function_call 2025-10-16 20:33:41 -07:00
Ishaan Jaffer 64876634ef ui new build 2025-10-16 20:32:51 -07:00
Ishaan Jaffer 7a1f654da6 ui new build 2025-10-16 18:37:42 -07:00
Ishaan Jaffer 13e895b9b0 test fix 2025-10-16 18:00:46 -07:00
Ishaan Jaffer 1c21b2e285 fixes: AI/ML API 2025-10-16 17:52:37 -07:00
Ishaan Jaffer 1c53a686d5 fix e2eui 2025-10-16 17:48:27 -07:00
Ishaan Jaffer bf20ecab80 Revert "fix get_tags_from_request_body"
This reverts commit 1c044dad999500b5c11ec96f528212c248f72f61.
2025-10-16 17:40:08 -07:00
Ishaan Jaffer 96449b319c fix get_tags_from_request_body 2025-10-16 17:40:08 -07:00
Ishaan JaffandGitHub 02758ebc28 Merge pull request #15553 from BerriAI/litellm_oct_staging2
[Staging] Oct Staging branch 2
2025-10-16 17:38:20 -07:00
Ishaan Jaffer ea69f4547d Merge branch 'main' into litellm_oct_staging2 2025-10-16 17:06:29 -07:00
Ishaan Jaffer 00e6ec4985 bump: version 1.78.2 → 1.78.3 2025-10-16 16:57:04 -07:00
Ishaan JaffandGitHub f69f7d101b Merge pull request #15618 from BerriAI/litellm_bedrock_invoke_support
[Feat] Allow calling /invoke, /converse routes through AI Gateway + models on config.yaml
2025-10-16 16:56:23 -07:00
Ishaan Jaffer a31a8108b5 fix pass thu ruff check 2025-10-16 16:45:14 -07:00
Ishaan Jaffer 0bef295b15 doc fix 2025-10-16 16:36:24 -07:00
Ishaan Jaffer bbf9d6a57b docs boto3 instructions 2025-10-16 16:34:56 -07:00
Ishaan Jaffer 79516563a9 docs add /invoke and /converse routes 2025-10-16 16:25:27 -07:00
Ishaan Jaffer 8c5d6891d4 docs pt 2025-10-16 16:18:13 -07:00
Ishaan Jaffer fe5dd23a84 docs boto3 pass thoughs 2025-10-16 16:17:02 -07:00
Ishaan Jaffer c2b69e371c docs pass through 2025-10-16 16:13:08 -07:00
Ishaan Jaffer f87c86806c docs invoke 2025-10-16 16:13:01 -07:00
Ishaan JaffandGitHub 5009a43b0e Merge pull request #15608 from BerriAI/litellm_sso_add_pkce
[Feat] UI SSO - Add PKCE for OKTA SSO
2025-10-16 15:48:10 -07:00
Ishaan JaffandGitHub f27f2d4803 Merge branch 'main' into litellm_sso_add_pkce 2025-10-16 15:48:01 -07:00
Ishaan Jaffer bcf53c6ceb working - errors from bedrock through pass throughs 2025-10-16 15:40:42 -07:00
Krrish Dholakia e43c3b9b88 test: add unit testing 2025-10-16 15:37:26 -07:00
Ishaan Jaffer 51f1907e98 test_bedrock_error_handling_returns_actual_error 2025-10-16 15:36:51 -07:00
Krrish Dholakia b85aeb08ce fix(ui_sso.py): support both 'app_roles' and 'roles' claim on jwt decode for SSO handler 2025-10-16 15:29:34 -07:00
Ishaan Jaffer e17636d4a6 add _extract_model_from_bedrock_endpoint 2025-10-16 15:26:32 -07:00
Ishaan Jaffer cc5eac4965 fix _add_deployment_model_to_endpoint_for_llm_passthrough_route 2025-10-16 15:10:54 -07:00
Ishaan JaffandGitHub f98f299854 Merge pull request #15617 from BerriAI/litellm_october_alexsander_stanging
[OCT] Alexsander PERF improvements
2025-10-16 15:10:02 -07:00
Ishaan Jaffer e5956ff0d4 handle_bedrock_passthrough_router_model 2025-10-16 15:08:06 -07:00
Ishaan Jaffer 902cf69ffd fixes for async pass throughs 2025-10-16 14:59:00 -07:00
AlexsanderHamir bdca4db941 Merge branch 'litellm_october_alexsander_stanging' of https://github.com/BerriAI/litellm into litellm_october_alexsander_stanging 2025-10-16 14:43:30 -07:00
AlexsanderHamir aad0efd99a fix(proxy): change check_file_size_under_limit to accept Collection[str]
Fixes mypy type error where router_model_names (a set) was incompatible
with List[str] parameter type.

Changed check_file_size_under_limit to accept Collection[str] instead of
List[str] since the parameter is only used for membership checks, making
it more flexible and avoiding unnecessary type conversions.

Resolves: proxy/proxy_server.py:5107 arg-type error
2025-10-16 14:42:27 -07:00
AlexsanderHamir 220fa3feeb Add missing env key 2025-10-16 14:42:27 -07:00
AlexsanderHamir e154ab6a66 test: ensure model_names is a O(1) datastructure 2025-10-16 14:42:24 -07:00
AlexsanderHamir 3b14866d78 perf(router): optimize model lookups with O(1) data structures
Replace O(n) scans with index map lookups and convert model_names to set. Standardize timing calls.
2025-10-16 14:41:59 -07:00
AlexsanderHamir 76b272dffa test(router): update error message assertion after string concat optimization
Update test_generate_model_id_with_deployment_model_name to accept the new
error message format that results from the list+join optimization.

The function still correctly rejects None values with a TypeError, but the
error message changed from 'unsupported operand type(s) for +=' to
'expected str instance, NoneType found' due to the implementation change
from string concatenation to list joining.
2025-10-16 14:41:59 -07:00
AlexsanderHamir f842b2a6ce perf(router): optimize string concatenation in hash generation
Replace string concatenation in loop with list append + join pattern.
This improves time complexity from O(n²) to O(n) and avoids creating
many temporary string objects during hash ID generation.
2025-10-16 14:41:59 -07:00
AlexsanderHamir 8c58d165a7 perf(router): use shallow copy instead of deepcopy for model aliases
Replace copy.deepcopy() with dict.copy() in _get_all_deployments when
creating model aliases.

Safe because:
1. Only modifies top-level 'model_name' field (isolated by shallow copy)
2. Nested dicts (litellm_params, model_info) are never modified after return
3. When model_alias=None, returns original dict with NO copy, proving
   callers expect nested structures to be read-only
4. Key insight: Code that needs to modify nested structures does deepcopy FIRST.
   This proves the contract is: \"treat returned deployments as read-only for
   nested fields.\" (see line 6894: copy.deepcopy before modifying litellm_params)

Performance: ~10-100x faster than deepcopy on nested dict structures.
Tests: All 114 router unit tests pass.
2025-10-16 14:41:58 -07:00
AlexsanderHamir 35de05dc8b test: add static analysis to prevent O(n) linear scans in router
Add AST-based test to detect 'for ... in self.model_list' anti-pattern.
Enforces use of index maps (model_id_to_deployment_index_map and
model_name_to_deployment_indices) for O(1) lookups instead of O(n) iteration.
2025-10-16 14:41:58 -07:00
AlexsanderHamir 254f50a655 perf(router): optimize model lookups with O(1) index maps and standardize timing
- Use model_id_to_deployment_index_map and model_name_to_deployment_indices for O(1) lookups in get_model_info, get_deployment_by_model_group_name, and get_model_ids
2025-10-16 14:41:58 -07:00
Ishaan Jaffer 74a35914c6 _add_deployment_model_to_endpoint_for_llm_passthrough_route 2025-10-16 14:31:13 -07:00
Ishaan Jaffer da72a812c9 test_bedrock_passthrough_router 2025-10-16 14:23:34 -07:00
Ishaan Jaffer 1c8b7abddb GENERIC_CLIENT_USE_PKCE 2025-10-16 13:32:12 -07:00
Ishaan Jaffer 193fa19552 fix Okta PKCE 2025-10-16 13:26:14 -07:00