Commit Graph
29587 Commits
Author SHA1 Message Date
Ishaan JaffandGitHub bf1c5bef59 [UI] Add view for estimating costs across requests (#18645)
* add estimate_cost endpoint

* TestCostEstimateEndpoint

* fix estimate_cost

* add /cost/estimate to spend tracking routes

* fix code QA checks

* fixes endpoint

* v0 cost estimator

* v0 cost estimator

* formatNumberWithCommas

* fix admin view

* docs

* docs fix + export PDF/CSV

* fixes for export
2026-01-05 19:19:58 +05:30
Sameer KankuteandGitHub d0a26dd4bc Merge pull request #18644 from BerriAI/litellm_fix_mapped_tests_05012026
Fix mapped tests 05012026
2026-01-05 17:53:17 +05:30
Sameer Kankute d3107ac61a revert litellm/litellm/llms/gemini/interactions/transformation.py 2026-01-05 17:37:01 +05:30
Sameer KankuteandGitHub 3c60b930e2 Merge pull request #18635 from BerriAI/litellm_completions_api_summary_param
(feat) Add mapping for reasoning effort to summary param of responses API
2026-01-05 17:36:21 +05:30
Sameer KankuteandGitHub 30096ffe5c Merge pull request #18637 from BerriAI/litellm_gemini_api_base_auth_fix
fix: auth header for custom api base in generate Content request
2026-01-05 17:35:47 +05:30
Sameer KankuteandGitHub 23fa88f64e Merge pull request #18638 from BerriAI/litellm_fix_custom_llm_providers_ui_testing
fix: Add custom llm provider to get_llm_provider when sent via UI
2026-01-05 17:34:50 +05:30
Sameer Kankute 2d7f0a1351 fix: test_create_simple_string_input 2026-01-05 17:15:12 +05:30
Sameer Kankute c0f74c994a Fix: test_transform_request_image_extraction 2026-01-05 17:02:54 +05:30
Sameer Kankute 935824e4f9 Fix : test_sap_chat[False] 2026-01-05 16:56:22 +05:30
Ishaan JaffandGitHub 7f4dba2a01 [Feat] Add Cost Estimator for AI Gateway (#18643)
* add estimate_cost endpoint

* TestCostEstimateEndpoint

* fix estimate_cost

* add /cost/estimate to spend tracking routes

* fix code QA checks

* fixes endpoint
2026-01-05 16:51:23 +05:30
Sameer Kankute 840617de73 fix: tests.test_litellm.interactions.test_gemini_interactions.TestGeminiInteractions 2026-01-05 16:41:24 +05:30
Matt GreathouseandGitHub b24156079f Add gemini-3-flash on openrouter (#18247) 2026-01-05 16:37:23 +05:30
Sameer Kankute d590c820c2 Fix: TypeError: Object of type function is not JSON serializable 2026-01-05 16:32:21 +05:30
Sameer KankuteGitHubCopilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
28c7659d3d Potential fix for code scanning alert no. 3954: Clear-text logging of sensitive information
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2026-01-05 16:06:10 +05:30
YutaSaitoandGitHub 165d5f3af1 Merge pull request #18640 from BerriAI/litellm_fix_cloudzero_integration_test
tests: mock LiteLLM_UserTable for CloudZero export
2026-01-05 17:41:22 +09:00
YutaSaitoandGitHub d5e4a43be3 Merge pull request #18639 from BerriAI/litellm_feat_mcp_global_mode
[feat] mcp global mode
2026-01-05 17:34:06 +09:00
Yuta Saito bdc35b4cd2 tests: mock LiteLLM_UserTable for CloudZero export 2026-01-05 17:23:04 +09:00
Yuta Saito ef2f76e5a5 fix: npm build error 2026-01-05 17:06:02 +09:00
Yuta Saito 7213f4a690 chore: lint 2026-01-05 16:57:39 +09:00
Yuta Saito fee9553c51 feat: add allow_all_keys doc 2026-01-05 16:56:19 +09:00
Yuta Saito 38a20b8036 feat: feat: allow_all_keys to mcp server on the UI 2026-01-05 16:28:51 +09:00
Sameer Kankute 6337ea876b fix: Add custom llm provider to get_llm_provider when sent via UI 2026-01-05 12:57:30 +05:30
Sameer Kankute 0730a74dda fix: auth header for custom api base in generate Content request 2026-01-05 12:25:13 +05:30
Yuta Saito f1d77ae14d feat: allow_all_keys to mcp server 2026-01-05 15:49:09 +09:00
Sameer Kankute 02a41a5c13 fix the comment 2026-01-05 11:40:40 +05:30
Sameer Kankute 0f8e4364d6 Replace summary param as detailed 2026-01-05 11:24:35 +05:30
Sameer Kankute 42d4aab3e7 Add mapping for reasoning effort to summary of responses API 2026-01-05 11:21:34 +05:30
YutaSaitoandGitHub b6664669d2 Merge pull request #18480 from hamzaq453/fix/openapi-parameter-names-security
Fix: Remove exec() usage and handle invalid OpenAPI parameter names
2026-01-05 14:34:57 +09:00
YutaSaitoandGitHub c405135c2b Merge pull request #18584 from BerriAI/feat/add_user_email_to_cloudzero
feat: add user email to cloudzero
2026-01-05 14:22:46 +09:00
hamzaq453 9ca7b1ad9b Resolve merge conflicts: integrate path validation with **kwargs approach
- Keep **kwargs approach (no exec()) for security
- Integrate path traversal validation from main branch
- Add URL encoding for path parameters
- Merge both test suites (edge cases + security tests)
- All 14 tests passing
2026-01-05 10:20:26 +05:00
YutaSaitoandGitHub ff6fd0c7f4 Merge pull request #18634 from BerriAI/litellm_refactor_bedrock_guardrail
refactor: remove redundant Bedrock guardrail block handling
2026-01-05 14:12:21 +09:00
Yuta Saito b37076bf2a refactor: remove redundant Bedrock guardrail block handling 2026-01-05 13:55:38 +09:00
YutaSaitoandGitHub fcabc059ca Merge pull request #18630 from BerriAI/litellm_fix_mcp_guardrail
fix: MCP handling in unified guardrail
2026-01-05 12:25:14 +09:00
amangupta-20andGitHub 399579f8ea feat: Add Levo AI integration (#18529) 2026-01-05 08:49:21 +05:30
Yuta Saito 0509fc79da fix: move code from litellm/llms to the mcp_server dir 2026-01-05 12:05:16 +09:00
Yuta Saito c8c73e6fa5 fix: MCP handling in unified guardrail 2026-01-05 10:41:24 +09:00
yuneng-jiangandGitHub 359b8df8b2 Merge pull request #18619 from BerriAI/litellm_ui_keys_loading_2
[Feature] UI - Virtual Keys Table Loading State Between Pages
2026-01-03 19:30:32 -08:00
yuneng-jiang 1112974112 Virtual Keys Table Loading State 2026-01-03 19:22:38 -08:00
yuneng-jiangandGitHub 280085edaa Merge pull request #18618 from BerriAI/litellm_ui_unit_test_coverage
[Infra] UI - Unit Tests: Adding Tests to Increase Coverage
2026-01-03 17:55:53 -08:00
yuneng-jiang 816124a40b Fixign build 2026-01-03 17:39:59 -08:00
yuneng-jiang 1184db079e fixing tests 2026-01-03 17:30:52 -08:00
yuneng-jiang 60c39255d5 Adding unit testing coverage 2026-01-03 17:25:46 -08:00
yuneng-jiangandGitHub c07ff5d9ee Merge pull request #18617 from BerriAI/litellm_ui_sso_loading_state
[Feature] UI - SSO Settings Loading State + Deprecate Previous SSO Flow
2026-01-03 16:33:51 -08:00
yuneng-jiang d6296411ec SSO Settings Loading, deprecate previous flow 2026-01-03 16:23:10 -08:00
Alexsander HamirandGitHub b6d601c2f0 perf(utils): lazy load 15+ unused imports (#18616)
- Move BaseBatchesConfig, BaseContainerConfig, BaseEmbeddingConfig, BaseImageEditConfig, BaseImageGenerationConfig, BaseImageVariationConfig, BasePassthroughConfig, BaseRealtimeConfig, BaseRerankConfig, BaseVectorStoreConfig, BaseVectorStoreFilesConfig, BaseVideoConfig to lazy loading
- Move ANTHROPIC_API_ONLY_HEADERS, AnthropicThinkingParam, RerankResponse to lazy loading
- Move ChatCompletionDeltaToolCallChunk, ChatCompletionToolCallChunk, ChatCompletionToolCallFunctionChunk, LiteLLM_Params to lazy loading
- Add type stubs to TYPE_CHECKING block for mypy support
- Add lazy loading handlers in __getattr__ method
- These imports are not used in utils.py runtime code, only in type annotations (safe with from __future__ import annotations)
2026-01-03 16:16:17 -08:00
Alexsander HamirandGitHub dd1ccec734 refactor(utils): lazy load 15 additional imports to improve import time (#18613)
* refactor(utils): lazy load 15 additional imports to improve import time

- Move Rules, AsyncHTTPHandler, HTTPHandler to lazy loading via __getattr__
- Move get_num_retries_from_retry_policy, reset_retry_policy to lazy loading
- Move get_secret to lazy loading
- Move cached_imports functions (get_coroutine_checker, get_litellm_logging_class, get_set_callbacks) to lazy loading
- Move core_helpers functions (get_litellm_metadata_from_kwargs, map_finish_reason, process_response_headers) to lazy loading
- Move dot_notation_indexing functions (delete_nested_value, is_nested_path) to lazy loading
- Move get_litellm_params functions to lazy loading
- Move _ensure_extra_body_is_safe, get_formatted_prompt, get_response_headers, update_response_metadata to lazy loading
- Move executor to lazy loading
- Move BaseAnthropicMessagesConfig, BaseAudioTranscriptionConfig to lazy loading
- Add type stubs in TYPE_CHECKING block for mypy type checking
- These functions/classes are exported for other modules but not used internally in utils.py, so lazy loading is safe and improves startup performance

* fix(utils): use getattr for Rules and get_coroutine_checker in client decorator

- Update Rules() instantiation in client decorator to use getattr for lazy loading
- Update Rules.has_pre_call_rules() usage in function_setup to use getattr
- Update get_coroutine_checker() usage in client decorator to use getattr
- Fixes NameError: name 'Rules' is not defined error that occurs when Rules is lazy-loaded

* fix(utils): use getattr for get_litellm_logging_class in function_setup

- Update get_litellm_logging_class() usage in function_setup to use getattr for lazy loading
- Fixes NameError: name 'get_litellm_logging_class' is not defined error that occurs when get_litellm_logging_class is lazy-loaded

* fix(utils): use getattr for get_set_callbacks in function_setup

- Update get_set_callbacks() usage in function_setup to use getattr for lazy loading
- Fixes NameError: name 'get_set_callbacks' is not defined error that occurs when get_set_callbacks is lazy-loaded

* fix(utils): use getattr for all lazy-loaded imports in utils.py

- Update update_response_metadata (4 occurrences) to use getattr
- Update executor.submit (1 occurrence) to use getattr
- Update get_num_retries_from_retry_policy (2 occurrences) to use getattr
- Update reset_retry_policy (2 occurrences) to use getattr
- Update is_nested_path and delete_nested_value (1 occurrence each) to use getattr
- Update _ensure_extra_body_is_safe (1 occurrence) to use getattr

Fixes NameError errors that occur when these functions/classes are lazy-loaded but used directly in utils.py

* fix(utils): use getattr for _get_base_model_from_litellm_call_metadata in _get_base_model_from_metadata

- Update _get_base_model_from_litellm_call_metadata usage to use getattr for lazy loading
- Fixes NameError: name '_get_base_model_from_litellm_call_metadata' is not defined

* fix(utils): use getattr for second _get_base_model_from_litellm_call_metadata usage

- Fix the second occurrence of _get_base_model_from_litellm_call_metadata on line 7052
- Both occurrences in _get_base_model_from_metadata now use getattr for lazy loading

* fix(utils): fix indentation in _get_base_model_from_metadata function

* fix(utils): use getattr for get_litellm_metadata_from_kwargs in _get_litellm_params

- Update get_litellm_metadata_from_kwargs usage to use getattr for lazy loading
- Fixes NameError: name 'get_litellm_metadata_from_kwargs' is not defined

* fix(utils): fix syntax error in get_litellm_metadata_from_kwargs fix

- Move getattr call before cast statement to fix syntax error
2026-01-03 15:50:53 -08:00
yuneng-jiangandGitHub bbae987b02 Merge pull request #18615 from BerriAI/litellm_ui_admin_add_model_1
[Infra] UI - E2E Tests: Add Model - See Models from Selected Provider
2026-01-03 15:46:53 -08:00
yuneng-jiang a6c3fb1fb5 E2E test see models for specific provider 2026-01-03 15:10:52 -08:00
Alexsander HamirandGitHub 1c5c303e98 refactor(utils): implement lazy loading for provider configs, model info classes, streaming handlers, and redact utilities (#18611)
* refactor(utils): lazy load redact_messages imports to improve import time

- Move LiteLLMLoggingObject and redact_message_input_output_from_logging to lazy loading via __getattr__
- Add type stubs in TYPE_CHECKING block for mypy type checking
- These are only used in type annotations (with from __future__ import annotations), so lazy loading works correctly

This reduces import time by deferring the redact_messages module import until these are actually accessed.

* refactor(utils): lazy load CustomStreamWrapper to improve import time

- Move CustomStreamWrapper from streaming_handler to lazy loading via __getattr__
- Add type stub in TYPE_CHECKING block for mypy type checking
- CustomStreamWrapper is not used internally in utils.py, only exported for other modules

This reduces import time by deferring the streaming_handler module import until CustomStreamWrapper is actually accessed.

* refactor(utils): lazy load BaseGoogleGenAIGenerateContentConfig to improve import time

- Move BaseGoogleGenAIGenerateContentConfig from google_genai.transformation to lazy loading via __getattr__
- Add type stub in TYPE_CHECKING block for mypy type checking
- BaseGoogleGenAIGenerateContentConfig is only used in type annotations (with from __future__ import annotations), so lazy loading works correctly

This reduces import time by deferring the google_genai.transformation module import until BaseGoogleGenAIGenerateContentConfig is actually accessed.

* refactor(utils): lazy load BaseOCRConfig, BaseSearchConfig, and BaseTextToSpeechConfig

- Move BaseOCRConfig, BaseSearchConfig, and BaseTextToSpeechConfig to lazy loading via __getattr__
- Add type stubs in TYPE_CHECKING block for mypy type checking
- These config classes are only used in quoted type annotations (forward references), so lazy loading works correctly

This reduces import time by deferring the transformation module imports until these config classes are actually accessed.

* refactor(utils): lazy load BedrockModelInfo, CohereModelInfo, and MistralOCRConfig

- Move BedrockModelInfo, CohereModelInfo, and MistralOCRConfig to lazy loading via __getattr__
- Add type stubs in TYPE_CHECKING block for mypy type checking
- Update internal usages to use getattr pattern for accessing lazy-loaded classes
- These provider-specific model info classes are only used in specific code paths, so lazy loading reduces initial import time

This reduces import time by deferring the bedrock, cohere, and mistral module imports until these classes are actually accessed.

* fix(utils): remove duplicate MistralOCRConfig import in TYPE_CHECKING block
2026-01-03 14:07:57 -08:00
Alexsander HamirandGitHub 2cbcaf2abf refactor(utils): lazy load heavy imports to improve import time and memory usage (#18610)
* refactor(utils): lazy load heavy imports to improve import time

- Move BaseVectorStore, CredentialAccessor, and exception_mapping_utils imports to lazy loading via __getattr__
- Add _get_utils_globals() helper function following pattern from _lazy_imports.py
- Refactor __getattr__ to use consistent caching pattern matching __init__.py
- Update load_credentials_from_list to use lazy-loaded CredentialAccessor

This reduces import time and memory usage by only loading these modules when they're actually accessed, not during module import.

* refactor(utils): lazy load additional heavy imports to improve import time

- Move get_llm_provider, _is_non_openai_azure_model to lazy loading
- Move get_supported_openai_params to lazy loading
- Move convert_dict_to_response functions (LiteLLMResponseObjectHandler, convert_to_model_response_object, etc.) to lazy loading
- Move get_api_base and ResponseMetadata to lazy loading
- Move _parse_content_for_reasoning to lazy loading
- Update all internal usages to access via getattr(sys.modules[__name__], ...)

This reduces import time and memory usage by only loading these modules when they're actually accessed, not during module import.

* fix(utils): suppress PLR0915 linter warning for __getattr__ function

The __getattr__ function intentionally has many statements to handle
multiple lazy-loaded imports. Add noqa comment to suppress the warning.

* fix(utils): add type stubs for lazy-loaded functions in TYPE_CHECKING block

Add type imports and declarations in TYPE_CHECKING block to help mypy
understand the types of lazy-loaded functions accessed via __getattr__.
This follows the same pattern used in __init__.py for lazy-loaded items.
2026-01-03 13:34:17 -08:00