When a fetch fails, the button now exits the loading state instead of
staying stuck on "Fetching" indefinitely.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Show a Fetch/Fetching button next to "Showing X of Y results" that acts as
both a manual refetch trigger and a loading indicator. The "Loading keys..."
message now only appears on initial load; subsequent refetches keep the table
visible with stale data (via React Query's keepPreviousData).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix case-insensitive tool name matching in _tool_name_matches() so that
OpenAPI operationIds (camelCase) match lowercase registered tool names
when filtering by allowed_tools
- Fix get_base_url() to resolve relative server URLs (e.g. /api/v3) by
deriving full base URL from spec_path when OpenAPI spec has relative URLs
- Add tests for case-insensitive matching and filter_tools_by_allowed_tools
Made-with: Cursor
`all_models = user_api_key_dict.models` was creating an alias, so
`_get_models_from_access_groups` (which uses `.pop()`/`.extend()`) would
mutate the cached object in-place. Now both `.models` and `.team_models`
assignments create copies via `list()`.
Added test to verify the input is not mutated.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds dedup to get_key_models and get_team_models to prevent duplicate
entries when access group member models overlap with proxy_model_list.
Removes dead assignment of all_models in get_team_models.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When a team has "all-proxy-models", the model list expansion now includes
model access group names so they appear in the UI key creation form.
Also fixes get_key_models not forwarding include_model_access_groups to
_get_models_from_access_groups, and removes unused _unfurl_all_proxy_models.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix perform_redaction to handle dict representation of ModelResponse (from model_dump())
- Preserve full choices structure when redacting, redact content/audio in place
- Add _redact_standard_logging_object helper for standard_logging_object field
- Update test_logging_redaction_e2e_test assertions to expect choices format
- Add charity_engine to provider_endpoints_support.json
Fixes: test_standard_logging_payload, test_standard_logging_payload_audio
Made-with: Cursor
Any param in DEFAULT_CHAT_COMPLETION_PARAM_VALUES that arrives via
completion(**kwargs) is now automatically forwarded to
get_optional_params(), even if it's not a named parameter of
completion().
Previously, get_non_default_completion_params() excluded params in
OPENAI_CHAT_COMPLETION_PARAMS (assuming they'd be forwarded via the
named-param path), while optional_param_args only contained explicitly
named params. Params like 'store' that were in the known-params list
but not named params fell through both paths and were silently dropped.
The fix adds a 7-line loop after building optional_param_args that
forwards any kwargs present in DEFAULT_CHAT_COMPLETION_PARAM_VALUES.
This means new OpenAI params only need to be added to the constants
dict — no boilerplate changes to 3+ function signatures required.
Fixes#23087
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
The SageMaker embedding handler was not using _load_credentials(),
which meant aws_role_name and aws_session_name parameters were
ignored. This prevented cross-account role assumption for embeddings
while it worked for completions.
Changes:
- Replace direct boto3 client creation with _load_credentials() call
- Create boto3.Session with assumed credentials
- Add comprehensive unit tests for role assumption
This aligns the embedding handler behavior with the completion handler,
which already supports role assumption via the BaseAWSLLM.get_credentials()
method.
Fixes cross-account SageMaker embedding access where users need to
assume a role in another account to invoke endpoints.
Any param in DEFAULT_CHAT_COMPLETION_PARAM_VALUES that arrives via
completion(**kwargs) is now automatically forwarded to
get_optional_params(), even if it's not a named parameter of
completion().
Previously, get_non_default_completion_params() excluded params in
OPENAI_CHAT_COMPLETION_PARAMS (assuming they'd be forwarded via the
named-param path), while optional_param_args only contained explicitly
named params. Params like 'store' that were in the known-params list
but not named params fell through both paths and were silently dropped.
The fix adds a 7-line loop after building optional_param_args that
forwards any kwargs present in DEFAULT_CHAT_COMPLETION_PARAM_VALUES.
This means new OpenAI params only need to be added to the constants
dict — no boilerplate changes to 3+ function signatures required.
Fixes#23087
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
* feat(charity_engine): add Charity Engine provider
Charity Engine is a crowdsourced distributed computing platform that
donates processing power to charitable causes. Its inference API
provides OpenAI-compatible chat, completions, and embeddings endpoints.
* test(charity_engine): add provider config and resolution tests
Verify JSONProviderRegistry config, provider list membership,
model routing for charity_engine/<model>, and Router compatibility.
* feat(charity_engine): add Charity Engine to LlmProviders enum
Enables provider_list membership and LlmProviders.CHARITY_ENGINE
resolution required by the provider and test suite.
* fix(charity_engine): remove api_base_env to fix non-deterministic test
The CHARITY_ENGINE_API_BASE env var could override the base_url in CI,
causing test_charity_engine_provider_resolution to fail intermittently.
* fix(charity_engine): remove trailing slash from base_url
Claude's API returns assistant messages with empty text blocks
({"type": "text", "text": ""}) alongside tool_use blocks during
multi-turn tool-use conversations. These blocks are rejected when
sent back to the API with "text content blocks must be non-empty".
Sanitization already exists for other code paths (/v1/chat/completions
for both Anthropic and Bedrock), but NOT for the /v1/messages native
path. This adds the same treatment by stripping empty text blocks
from messages in async_anthropic_messages_handler before they are
forwarded to the provider.
Fixes#22930
_get_openai_compatible_provider_info already returns an api_base
ending in /v1, but get_models prepended another /v1, producing
.../inference/v1/v1/accounts/... which 404s.
Strip the trailing /v1 from api_base before re-adding it so that
both the default and any user-supplied base work correctly.
Add parametrized tests covering the default URL, trailing-slash,
custom base with /v1, and custom base without /v1.
Fixes#23106
In _get_available_deployments, when streaming mode is active the code
sums time-to-first-token values from item_ttft_latency but divides by
len(item_latency) instead of len(item_ttft_latency). These lists can
have different lengths, producing an incorrect average that skews
lowest-latency routing decisions for streaming requests.
Signed-off-by: JiangNan <1394485448@qq.com>
When value > 1 (e.g., "2mo") and current_month + value > 12,
target_month exceeds valid range (1-12), causing ValueError in
datetime constructor. For example, calling duration_in_seconds("2mo")
in November produces target_month=13.
Use modular arithmetic to correctly wrap months and increment year.
Signed-off-by: JiangNan <1394485448@qq.com>
* add new azure gpt models
* add versionless azure/gpt-5.4 models
* Undated azure/gpt-5.4 alias missing supports_service_tier
* indicate service tier support for azure/gpt-5.3-chat
* fix priority tier pricing for new azure/gpt models
The bug occurred when user data inadvertently contained reserved Python
keywords like 'self', 'params', or '__class__' as keys. When such a dict
was unpacked via **kwargs to LiteLLM_Params() or GenericLiteLLMParams(),
Python raised TypeError because 'self' was passed both implicitly and
as a keyword argument.
The fix:
- Add a Pydantic model_validator(mode='before') to GenericLiteLLMParams
that filters out reserved keys ('self', 'params', '__class__') before
validation
- Move the max_retries str-to-int conversion into the same validator
- Remove the custom __init__ methods from both GenericLiteLLMParams and
LiteLLM_Params, since the validator now handles the preprocessing
- Clean up unused VERTEX_CREDENTIALS_TYPES import
This fix applies to all classes that inherit from GenericLiteLLMParams,
including LiteLLM_Params and updateLiteLLMParams.
Added comprehensive tests in tests/test_litellm/test_litellm_params_reserved_keys.py
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
The test calls OpenAI's gpt-4o-audio-preview model which sometimes
doesn't return usage data in the streaming response. Fixed by:
- Adding @pytest.mark.flaky(retries=5, delay=2) for retry handling
- Fixing usage_obj loop to check chunk.usage is not None
- Skipping gracefully when OpenAI doesn't return usage data
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The test_create_skill test was consistently failing in CI with a 500 from
Anthropic because the SKILL.md frontmatter always used the same hardcoded
name (test-skill-litellm). Since test_delete_skill is permanently skipped,
skills accumulate in the CI account, and re-creating with a duplicate name
triggers an Internal Server Error on Anthropic's side.
Fix: pass a timestamp-based unique_suffix to create_skill_zip so each run
produces a distinct skill name in the zip's SKILL.md frontmatter.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The Claude Agent SDK sends max_tokens=32000 for unrecognized model names
(like "bedrock-nova-pro"), which exceeds Nova Pro's 10,000 limit. Enable
modify_params in the test proxy config so LiteLLM clamps max_tokens to the
model's actual limit. Also swap nova-premier to nova-pro since premier
requires provisioned throughput unavailable in CI.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sameer Kankute