Merge pull request #23276 from BerriAI/litellm_oss_staging_03_10_2026

Litellm oss staging 03 10 2026

CLAUDE.md

@@ -110,6 +110,13 @@ LiteLLM is a unified interface for 100+ LLM providers with two main components:
 ### Proxy database access
 - **Do not write raw SQL** for proxy DB operations. Use Prisma model methods instead of `execute_raw` / `query_raw`.
 - Use the generated client: `prisma_client.db.<model>` (e.g. `litellm_tooltable`, `litellm_usertable`) with `.upsert()`, `.find_many()`, `.find_unique()`, `.update()`, `.update_many()` as appropriate. This avoids schema/client drift, keeps code testable with simple mocks, and matches patterns used in spend logs and other proxy code.
+- **No N+1 queries.** Never query the DB inside a loop. Batch-fetch with `{"in": ids}` and distribute in-memory.
+- **Batch writes.** Use `create_many`/`update_many`/`delete_many` instead of individual calls (these return counts only; `update_many`/`delete_many` no-op silently on missing rows). When multiple separate writes target the same table (e.g. in `batch_()`), order by primary key to avoid deadlocks.
+- **Push work to the DB.** Filter, sort, group, and aggregate in SQL, not Python. Verify Prisma generates the expected SQL — e.g. prefer `group_by` over `find_many(distinct=...)` which does client-side processing.
+- **Bound large result sets.** Prisma materializes full results in memory. For results over ~10 MB, paginate with `take`/`skip` or `cursor`/`take`, always with an explicit `order`. Prefer cursor-based pagination (`skip` is O(n)). Don't paginate naturally small result sets.
+- **Limit fetched columns on wide tables.** Use `select` to fetch only needed fields — returns a partial object, so downstream code must not access unselected fields.
+- **Check index coverage.** For new or modified queries, check `schema.prisma` for a supporting index. Prefer extending an existing index (e.g. `@@index([a])` → `@@index([a, b])`) over adding a new one, unless it's a `@@unique`. Only add indexes for large/frequent queries.
+- **Keep schema files in sync.** Apply schema changes to all `schema.prisma` copies (`schema.prisma`, `litellm/proxy/`, `litellm-proxy-extras/`, `litellm-js/spend-logs/` for SpendLogs) with a migration under `litellm-proxy-extras/litellm_proxy_extras/migrations/`.
 
 ### Enterprise Features
 - Enterprise-specific code in `enterprise/` directory

docs/my-website/docs/apply_guardrail.md

@@ -11,6 +11,7 @@ This endpoint supports various guardrail types including:
 - **Presidio** - PII detection and masking
 - **Bedrock** - AWS Bedrock guardrails for content moderation
 - **Lakera** - AI safety guardrails
+- **PANW Prisma AIRS** - Threat detection, DLP, and policy enforcement
 - **Custom guardrails** - User-defined guardrails
 
 ## Configuration

docs/my-website/docs/completion/output.md

@@ -51,6 +51,28 @@ Here's what an example response looks like
 }
 ```
 
+## Native Finish Reason
+
+LiteLLM maps all provider-specific `finish_reason` values to OpenAI-compatible values (`stop`, `length`, `tool_calls`, `function_call`, `content_filter`). When the original provider value differs from the mapped value, it is preserved in `provider_specific_fields["native_finish_reason"]`.
+
+This is useful for agent loops that need to distinguish between different stop conditions (e.g., Gemini's `MALFORMED_FUNCTION_CALL` vs a normal `stop`).
+
+```python
+response = completion(model="gemini/gemini-2.0-flash", messages=messages)
+
+choice = response.choices[0]
+print(choice.finish_reason)  # "stop" (OpenAI-compatible)
+
+# Access the original provider value when it differs:
+if hasattr(choice, "provider_specific_fields") and choice.provider_specific_fields:
+    native = choice.provider_specific_fields.get("native_finish_reason")
+    if native == "MALFORMED_FUNCTION_CALL":
+        # Handle malformed function call differently from a normal stop
+        pass
+```
+
+When the provider already returns an OpenAI-compatible value (e.g., `stop`), `native_finish_reason` is not set.
+
 ## Additional Attributes
 
 You can also access information like latency. 

docs/my-website/docs/contributing/adding_openai_compatible_providers.md

@@ -80,6 +80,36 @@ That's it! The provider is now available.
 }
 ```
 
+## Responses API Support
+
+If your provider also supports the OpenAI Responses API (`/v1/responses`), add `supported_endpoints`:
+
+```json
+{
+  "your_provider": {
+    "base_url": "https://api.yourprovider.com/v1",
+    "api_key_env": "YOUR_PROVIDER_API_KEY",
+    "supported_endpoints": ["/v1/chat/completions", "/v1/responses"]
+  }
+}
+```
+
+This enables `litellm.responses()` with zero additional code:
+
+```python
+import litellm
+
+response = litellm.responses(
+    model="your_provider/model-name",
+    input="Hello, what can you do?",
+)
+print(response.output)
+```
+
+If `supported_endpoints` is omitted, it defaults to `[]`. Chat completions is always enabled for JSON providers regardless of this field.
+
+The provider inherits all request/response handling from OpenAI's Responses API — streaming, tools, and all standard parameters work out of the box.
+
 ## Usage
 
 ```python
@@ -89,11 +119,17 @@ import os
 # Set your API key
 os.environ["YOUR_PROVIDER_API_KEY"] = "your-key-here"
 
-# Use the provider
+# Chat completions
 response = litellm.completion(
     model="your_provider/model-name",
     messages=[{"role": "user", "content": "Hello"}],
 )
+
+# Responses API (if supported_endpoints includes "/v1/responses")
+response = litellm.responses(
+    model="your_provider/model-name",
+    input="Hello",
+)
 ```
 
 ## When to Use Python Instead
@@ -105,7 +141,9 @@ Use a Python config class if you need:
 - Provider-specific streaming logic
 - Advanced tool calling modifications
 
-For these cases, create a config class in `litellm/llms/your_provider/chat/transformation.py` that inherits from `OpenAIGPTConfig` or `OpenAILikeChatConfig`.
+For chat completions, create a config class in `litellm/llms/your_provider/chat/transformation.py` that inherits from `OpenAIGPTConfig` or `OpenAILikeChatConfig`.
+
+For responses API with small overrides, inherit from `OpenAIResponsesAPIConfig` and override only what's needed. See `litellm/llms/perplexity/responses/transformation.py` for a minimal example (~40 lines vs 400+).
 
 ## Testing
 

docs/my-website/docs/mcp_guardrail.md

@@ -86,4 +86,5 @@ MCP guardrails work with all LiteLLM-supported guardrail providers:
 - **Lakera**: Content moderation
 - **Aporia**: Custom guardrails
 - **Noma**: Noma Security
+- **PANW Prisma AIRS**: Prisma AIRS guardrails
 - **Custom**: Your own guardrail implementations

docs/my-website/docs/provider_registration/add_model_pricing.md

@@ -13,6 +13,7 @@ Here's the full specification with all available fields:
 ```json
 {
     "sample_spec": {
+        "aliases": ["optional list of alternate names for this model, e.g. dated versions like sample_spec-20250101"],
         "code_interpreter_cost_per_session": 0.0,
         "computer_use_input_cost_per_1k_tokens": 0.0,
         "computer_use_output_cost_per_1k_tokens": 0.0,
@@ -121,4 +122,28 @@ Here's the full specification with all available fields:
 }
 ```
 
-That's it! Your PR will be reviewed and merged.
+### Using Aliases
+
+Many providers release the same model under multiple names — for example, a `latest` tag and a dated version like `claude-sonnet-4-5-20250929`. Instead of duplicating the entire entry, you can use the `aliases` field:
+
+```json
+{
+    "claude-sonnet-4-5": {
+        "aliases": ["claude-sonnet-4-5-20250929"],
+        "input_cost_per_token": 3e-06,
+        "output_cost_per_token": 1.5e-05,
+        "litellm_provider": "anthropic",
+        "max_input_tokens": 200000,
+        "max_output_tokens": 64000,
+        "mode": "chat",
+        "supports_function_calling": true,
+        "supports_tool_choice": true
+    }
+}
+```
+
+At load time, each alias is expanded into a top-level entry sharing the same data as the canonical entry. The example above makes both `claude-sonnet-4-5` and `claude-sonnet-4-5-20250929` resolve with the same pricing and capabilities.
+
+:::info
+This is different from [`model_alias_map`](../completion/model_alias.md), which is a runtime SDK/proxy feature for mapping user-facing model names to LiteLLM model identifiers. The `aliases` field here is for the model cost JSON only — it avoids duplicate entries for models that share identical pricing and capabilities.
+:::

docs/my-website/docs/proxy/guardrails/panw_prisma_airs.md

@@ -1,24 +1,15 @@
 import Image from '@theme/IdealImage';
-import Tabs from '@theme/Tabs';
-import TabItem from '@theme/TabItem';
 
 # PANW Prisma AIRS
 
-LiteLLM supports PANW Prisma AIRS (AI Runtime Security) guardrails via the [Prisma AIRS Scan API](https://pan.dev/prisma-airs/api/airuntimesecurity/airuntimesecurityapi//). This integration provides **Security-as-Code** for AI applications using Palo Alto Networks' AI security platform.
+LiteLLM supports PANW Prisma AIRS (AI Runtime Security) guardrails via the [Prisma AIRS Scan API](https://pan.dev/prisma-airs/api/airuntimesecurity/airuntimesecurityapi/). This integration provides Security-as-Code for AI applications using Palo Alto Networks' AI security platform.
 
-## Features
+- **Prompt injection and malicious URL detection** — real-time scanning before or after LLM calls
+- **Data loss prevention (DLP)** — detect and block sensitive data in prompts and responses
+- **Sensitive content masking** — automatically mask PII, credit cards, SSNs instead of blocking
+- **MCP tool call scanning** — scan tool name and arguments on direct MCP tool invocations
+- **Configurable fail-open / fail-closed** — choose between maximum security or high availability
 
-- ✅ **Real-time prompt injection detection**
-- ✅ **Malicious URL detection** 
-- ✅ **Data loss prevention (DLP)**
-- ✅ **Sensitive content masking** - Automatically mask PII, credit cards, SSNs instead of blocking
-- ✅ **Comprehensive threat detection** for AI models and datasets
-- ✅ **Model-agnostic protection** across public and private models
-- ✅ **Synchronous scanning** with immediate response
-- ✅ **Configurable security profiles**
-- ✅ **Streaming support** - Real-time masking for streaming responses
-- ✅ **Multi-turn conversation tracking** - Automatic session grouping in Prisma AIRS SCM logs
-- ✅ **Configurable fail-open/fail-closed** - Choose between maximum security (block on API errors) or high availability (allow on transient errors)
 
 ## Quick Start
 
@@ -32,7 +23,14 @@ For detailed setup instructions, see the [Prisma AIRS API Overview](https://docs
 
 ### 2. Define Guardrails on your LiteLLM config.yaml
 
-Define your guardrails under the `guardrails` section:
+Set `api_base` to the regional endpoint for your Prisma AIRS deployment profile:
+
+| Region | Endpoint |
+|--------|----------|
+| US | `https://service.api.aisecurity.paloaltonetworks.com` |
+| EU (Germany) | `https://service-de.api.aisecurity.paloaltonetworks.com` |
+| India | `https://service-in.api.aisecurity.paloaltonetworks.com` |
+| Singapore | `https://service-sg.api.aisecurity.paloaltonetworks.com` |
 
 ```yaml
 model_list:
@@ -45,21 +43,15 @@ guardrails:
   - guardrail_name: "panw-prisma-airs-guardrail"
     litellm_params:
       guardrail: panw_prisma_airs
-      mode: "pre_call"                    # Run before LLM call
-      api_key: os.environ/PANW_PRISMA_AIRS_API_KEY    # Your Prisma AIRS API key
-      profile_name: os.environ/PANW_PRISMA_AIRS_PROFILE_NAME  # Security profile from Strata Cloud Manager
-      api_base: "https://service.api.aisecurity.paloaltonetworks.com"  
+      mode: "pre_call"
+      api_key: os.environ/PANW_PRISMA_AIRS_API_KEY
+      profile_name: os.environ/PANW_PRISMA_AIRS_PROFILE_NAME
+      api_base: "https://service.api.aisecurity.paloaltonetworks.com"  # US — change to your region
 ```
 
-#### Supported values for `mode`
-
-- `pre_call` Run **before** LLM call, on **input**
-- `post_call` Run **after** LLM call, on **input & output**  
-- `during_call` Run **during** LLM call, on **input**. Same as `pre_call` but runs in parallel with LLM call
-
 ### 3. Start LiteLLM Gateway
 
-```bash title="Set environment variables"
+```bash
 export PANW_PRISMA_AIRS_API_KEY="your-panw-api-key"
 export PANW_PRISMA_AIRS_PROFILE_NAME="your-security-profile"
 export OPENAI_API_KEY="sk-proj-..."
@@ -69,15 +61,8 @@ export OPENAI_API_KEY="sk-proj-..."
 litellm --config config.yaml --detailed_debug
 ```
 
-
 ### 4. Test Request
 
-**[Langchain, OpenAI SDK Usage Examples](../proxy/user_keys#request-format)**
-
-<Tabs>
-<TabItem label="Blocked request" value="blocked">
-
-Expect this to fail due to prompt injection attempt:
 
 ```shell
 curl -i http://localhost:4000/v1/chat/completions \
@@ -92,254 +77,57 @@ curl -i http://localhost:4000/v1/chat/completions \
   }'
 ```
 
-Expected response on failure:
+Expected response when the guardrail blocks:
 
 ```json
 {
   "error": {
-    "message": {
-      "error": "Violated PANW Prisma AIRS guardrail policy",
-      "panw_response": {
-        "action": "block",
-        "category": "malicious",
-        "profile_id": "03b32734-d06d-4bb7-a8df-ac5147630ce8",
-        "profile_name": "dev-block-all-profile",
-        "prompt_detected": {
-          "dlp": false,
-          "injection": true,
-          "toxic_content": false,
-          "url_cats": false
-        },
-        "report_id": "Rbd251eac-6e67-433b-b3ef-8eb42d2c7d2c",
-        "response_detected": {
-          "dlp": false,
-          "toxic_content": false,
-          "url_cats": false
-        },
-        "scan_id": "bd251eac-6e67-433b-b3ef-8eb42d2c7d2c",
-        "tr_id": "string"
-      }
-    },
-    "type": "None",
-    "param": "None",
-    "code": "400"
+    "message": "Prompt blocked by PANW Prisma AI Security policy (Category: malicious)",
+    "type": "guardrail_violation",
+    "code": "panw_prisma_airs_blocked",
+    "guardrail": "panw-prisma-airs-guardrail",
+    "category": "malicious"
   }
 }
 ```
 
-</TabItem>
-<TabItem label="Successful Call" value="allowed">
+LiteLLM wraps this detail in an endpoint-specific HTTP error envelope. Optional fields that may also appear: `scan_id`, `report_id`, `profile_name`, `profile_id`, `tr_id`, `prompt_detected`.
 
-```shell
-curl -i http://localhost:4000/v1/chat/completions \
-  -H "Content-Type: application/json" \
-  -H "Authorization: Bearer sk-your-api-key" \
-  -d '{
-    "model": "gpt-4o",
-    "messages": [
-      {"role": "user", "content": "What is the weather like today?"}
-    ],
-    "guardrails": ["panw-prisma-airs-guardrail"]
-  }'
-```
+On success, the guardrail name appears in the `x-litellm-applied-guardrails` response header.
 
-Expected successful response:
+## Configuration
 
-```json
-{
-  "choices": [
-    {
-      "finish_reason": "stop",
-      "index": 0,
-      "message": {
-        "content": "I don't have access to real-time weather data, but I can help you find weather information through various weather services or apps...",
-        "role": "assistant",
-        "tool_calls": null,
-        "function_call": null,
-        "annotations": []
-      }
-    }
-  ],
-  "created": 1736028456,
-  "id": "chatcmpl-AqQj8example",
-  "model": "gpt-4o",
-  "object": "chat.completion",
-  "usage": {
-    "completion_tokens": 25,
-    "prompt_tokens": 12,
-    "total_tokens": 37
-  },
-  "x-litellm-panw-scan": {
-    "action": "allow",
-    "category": "benign",
-    "profile_id": "03b32734-d06d-4bb7-a8df-ac5147630ce8",
-    "profile_name": "dev-block-all-profile",
-    "prompt_detected": {
-      "dlp": false,
-      "injection": false,
-      "toxic_content": false,
-      "url_cats": false
-    },
-    "report_id": "Rbd251eac-6e67-433b-b3ef-8eb42d2c7d2c",
-    "response_detected": {
-      "dlp": false,
-      "toxic_content": false,
-      "url_cats": false
-    },
-    "scan_id": "bd251eac-6e67-433b-b3ef-8eb42d2c7d2c",
-    "tr_id": "string"
-  }
-}
-```
+### Supported Modes
 
-</TabItem>
-</Tabs>
+| Mode | Timing | What is scanned |
+|------|--------|-----------------|
+| `pre_call` | Before LLM call | Request input |
+| `during_call` | Parallel with LLM call | Request input |
+| `post_call` | After LLM call | Response output |
+| `pre_mcp_call` | Before MCP tool execution | MCP tool input |
+| `during_mcp_call` | Parallel with MCP tool execution | MCP tool input |
 
-## Configuration Parameters
+
+### Configuration Parameters
 
 | Parameter | Required | Description | Default |
 |-----------|----------|-------------|---------|
 | `api_key` | Yes | Your PANW Prisma AIRS API key from Strata Cloud Manager | - |
 | `profile_name` | No | Security profile name configured in Strata Cloud Manager. Optional if API key has linked profile | - |
-| `app_name` | No | Application identifier for tracking in Prisma AIRS analytics (will be prefixed with "LiteLLM-") | `LiteLLM` |
-| `api_base` | No | Regional API endpoint (see [Regional Endpoints](#regional-endpoints) below) | `https://service.api.aisecurity.paloaltonetworks.com` (US) |
-| `mode` | No | When to run the guardrail | `pre_call` |
-| `fallback_on_error` | No | Action when PANW API is unavailable: `"block"` (fail-closed, default) or `"allow"` (fail-open). Config errors always block. | `block` |
-| `timeout` | No | PANW API call timeout in seconds (1-60) | `10.0` |
-| `violation_message_template` | No | Custom template for error message when request is blocked. Supports `{guardrail_name}`, `{category}`, `{action_type}`, `{default_message}` placeholders. | - |
+| `app_name` | No | Application identifier for tracking in Prisma AIRS analytics (prefixed with "LiteLLM-") | `LiteLLM` |
+| `api_base` | No | Regional API endpoint. US: `https://service.api.aisecurity.paloaltonetworks.com`, EU: `https://service-de.api.aisecurity.paloaltonetworks.com`, India: `https://service-in.api.aisecurity.paloaltonetworks.com`, Singapore: `https://service-sg.api.aisecurity.paloaltonetworks.com` | US |
+| `mode` | No | When to run the guardrail (see mode table above) | `pre_call` |
+| `fallback_on_error` | No | Action when PANW API is unavailable: `"block"` (fail-closed) or `"allow"` (fail-open). Config errors always block. | `block` |
+| `timeout` | No | PANW API call timeout in seconds (recommended: 1-60) | `10.0` |
+| `violation_message_template` | No | Custom template for blocked requests. Supports `{guardrail_name}`, `{category}`, `{action_type}`, `{default_message}` placeholders. | - |
+| `mask_request_content` | No | Mask sensitive data in prompts instead of blocking | `false` |
+| `mask_response_content` | No | Mask sensitive data in responses instead of blocking | `false` |
+| `mask_on_block` | No | Backwards-compatible flag that enables both request and response masking | `false` |
+| `experimental_use_latest_role_message_only` | No | Anthropic `/v1/messages` only. When unset: scans only latest user message on request side. Set `false` to scan all user/system/developer messages. Non-Anthropic unaffected. | Unset (true for Anthropic) |
 
-### Regional Endpoints
+Use the regional `api_base` that matches your Prisma AIRS deployment profile region for lower latency and data residency compliance.
 
-PANW Prisma AIRS supports multiple regional endpoints based on your deployment profile region:
-
-| Region | API Base URL |
-|--------|--------------|
-| **US** (default) | `https://service.api.aisecurity.paloaltonetworks.com` |
-| **EU (Germany)** | `https://service-de.api.aisecurity.paloaltonetworks.com` |
-| **India** | `https://service-in.api.aisecurity.paloaltonetworks.com` |
-
-**Example configuration for EU region:**
-
-```yaml
-guardrails:
-  - guardrail_name: "panw-eu"
-    litellm_params:
-      guardrail: panw_prisma_airs
-      api_key: os.environ/PANW_PRISMA_AIRS_API_KEY
-      api_base: "https://service-de.api.aisecurity.paloaltonetworks.com"
-      profile_name: "production"
-```
-
-:::tip Region Selection
-Use the regional endpoint that matches your Prisma AIRS deployment profile region configured in Strata Cloud Manager. Using the correct region ensures:
-- Lower latency (requests stay in-region)
-- Compliance with data residency requirements
-- Optimal performance
-:::
-
-## Per-Request Metadata Overrides
-
-You can override guardrail settings on a per-request basis using the `metadata` field:
-
-```json
-{
-  "model": "gpt-4",
-  "messages": [...],
-  "metadata": {
-    "profile_name": "dev-allow-all",            // Override profile name
-    "profile_id": "uuid-here",                  // Override profile ID (takes precedence)
-    "user_ip": "192.168.1.100",                 // Track user IP
-    "app_name": "MyApp"                         // Custom app name (becomes "LiteLLM-MyApp")
-  }
-}
-```
-
-**Supported Metadata Fields:**
-
-| Field | Description | Priority |
-|-------|-------------|----------|
-| `profile_name` | PANW AI security profile name | Per-request > config |
-| `profile_id` | PANW AI security profile ID (takes precedence over profile_name) | Per-request only |
-| `user_ip` | User IP address for tracking in Prisma AIRS | Per-request only |
-| `app_name` | Application identifier (prefixed with "LiteLLM-") | Per-request > config > "LiteLLM" |
-| `app_user` | Custom user identifier for tracking in Prisma AIRS | `app_user` > `user` > "litellm_user" |
-
-:::info Profile Resolution
-- If both `profile_id` and `profile_name` are provided, PANW API uses `profile_id` (it takes precedence)
-- If no profile is specified in metadata, uses the config `profile_name`
-- If no profile is specified at all, PANW API will use the profile linked to your API key in Strata Cloud Manager
-- **Note:** If your API key is not linked to a profile, you must provide `profile_name` or `profile_id`
-:::
-
-## Multi-Turn Conversation Tracking
-
-PANW Prisma AIRS automatically tracks multi-turn conversations using LiteLLM's `litellm_trace_id`. This enables you to:
-
-- **Group related requests** - All requests in a conversation share the same AI Session ID in Prisma AIRS SCM logs
-- **Track conversation context** - See the full history of prompts and responses for a user session
-- **Analyze attack patterns** - Identify sophisticated multi-turn attacks across conversation history
-
-### How It Works
-
-LiteLLM automatically generates a unique `litellm_trace_id` for each conversation session. The PANW guardrail uses this as the PANW transaction ID (which maps to "AI Session ID" in Strata Cloud Manager):
-
-```
-Conversation Session: litellm_trace_id = "abc-123-def-456"
-
-Turn 1 (User):    "What's the capital of France?"
-  → Scan ID: scan_001 | Prisma AIRS AI Session ID: abc-123-def-456
-
-Turn 2 (Assistant): "Paris is the capital of France."
-  → Scan ID: scan_002 | Prisma AIRS AI Session ID: abc-123-def-456
-
-Turn 3 (User):    "What's the population?"
-  → Scan ID: scan_003 | Prisma AIRS AI Session ID: abc-123-def-456
-
-Turn 4 (Assistant): "Paris has approximately 2.1 million residents."
-  → Scan ID: scan_004 | Prisma AIRS AI Session ID: abc-123-def-456
-```
-
-All scans appear under the same AI Session ID in Prisma AIRS logs, making it easy to:
-- Review complete conversation history (all 4 turns grouped together)
-- Identify patterns across multiple turns
-- Correlate security events within a session
-- Track the flow of user prompts and AI responses
-
-### Session Tracking
-
-LiteLLM automatically generates a unique `litellm_trace_id` for each request, which the PANW guardrail uses as the AI Session ID in Strata Cloud Manager. All prompt and response scans for a request are automatically grouped under the same session.
-
-#### Custom Session IDs (Per-App Tracking)
-
-You can provide your own `litellm_trace_id` to track sessions on a per-app or per-conversation basis:
-
-```bash
-curl -X POST http://localhost:4000/v1/chat/completions \
-  -H "Content-Type: application/json" \
-  -H "Authorization: Bearer sk-1234" \
-  -d '{
-    "model": "gpt-3.5-turbo",
-    "messages": [{"role": "user", "content": "capital of France"}],
-    "litellm_trace_id": "my-app-session-123",             # Custom AI Session ID
-    "metadata": {
-      "profile_name": "dev-allow-all-profile",            # Override security profile
-      "user_ip": "192.168.1.1",                           # Track user IP
-      "app_name": "eng"                                   # Custom app identifier
-    },
-    "guardrails": ["panw-prisma-airs-pre-guard", "panw-prisma-airs-post-guard"]
-  }'
-```
-
-**Result in PANW SCM:**
-- AI Session ID: `my-app-session-123`
-- All prompt and response scans will be grouped under this custom session ID
-- Perfect for tracking multi-turn conversations or per-application sessions
-
-:::tip Viewing Sessions in Prisma AIRS SCM Logs
-In Strata Cloud Manager, navigate to **AI Runtime > Sessions** to view all AI Session IDs and their associated scans. Click on a session to see the complete conversation history with security analysis.
-:::
-
-## Environment Variables
+### Environment Variables
 
 ```bash
 export PANW_PRISMA_AIRS_API_KEY="your-panw-api-key"
@@ -348,12 +136,31 @@ export PANW_PRISMA_AIRS_PROFILE_NAME="your-security-profile"
 export PANW_PRISMA_AIRS_API_BASE="https://custom-endpoint.com"
 ```
 
-## Advanced Configuration
+### Per-Request Metadata Overrides
+
+| Field | Description | Priority |
+|-------|-------------|----------|
+| `profile_name` | PANW AI security profile name | Per-request > config |
+| `profile_id` | PANW AI security profile ID (takes precedence over `profile_name`) | Per-request only |
+| `user_ip` | User IP address for tracking in Prisma AIRS | Per-request only |
+| `app_name` | Application identifier (prefixed with "LiteLLM-") | Per-request > config > "LiteLLM" |
+| `app_user` | Custom user identifier for tracking in Prisma AIRS | `app_user` > `user` > "litellm_user" |
+
+```json
+{
+  "model": "gpt-4",
+  "messages": [...],
+  "metadata": {
+    "profile_name": "dev-allow-all",
+    "profile_id": "uuid-here",
+    "user_ip": "192.168.1.100",
+    "app_name": "MyApp"
+  }
+}
+```
 
 ### Multiple Security Profiles
 
-You can configure different security profiles for different use cases:
-
 ```yaml
 guardrails:
   - guardrail_name: "panw-strict-security"
@@ -361,126 +168,40 @@ guardrails:
       guardrail: panw_prisma_airs
       mode: "pre_call"
       api_key: os.environ/PANW_PRISMA_AIRS_API_KEY
-      profile_name: "strict-policy"       # High security profile
-      
-  - guardrail_name: "panw-permissive-security"  
+      profile_name: "strict-policy"
+
+  - guardrail_name: "panw-permissive-security"
     litellm_params:
       guardrail: panw_prisma_airs
       mode: "post_call"
       api_key: os.environ/PANW_PRISMA_AIRS_API_KEY
-      profile_name: "permissive-policy"   # Lower security profile
+      profile_name: "permissive-policy"
 ```
 
-### Multiple API Keys (Multi-Tenant)
-
-For multi-tenant deployments where different customers need different PANW API keys, create separate guardrail instances:
-
-```yaml
-guardrails:
-  - guardrail_name: "panw-customer-a"
-    litellm_params:
-      guardrail: panw_prisma_airs
-      mode: "pre_call"
-      api_key: os.environ/PANW_CUSTOMER_A_KEY  # Linked to Customer A profile in SCM
-      
-  - guardrail_name: "panw-customer-b"
-    litellm_params:
-      guardrail: panw_prisma_airs
-      mode: "pre_call"
-      api_key: os.environ/PANW_CUSTOMER_B_KEY  # Linked to Customer B profile in SCM
-```
-
-Then route requests to the appropriate guardrail:
-
-```bash
-curl -X POST http://localhost:4000/v1/chat/completions \
-  -H "Content-Type: application/json" \
-  -H "Authorization: Bearer sk-1234" \
-  -d '{
-    "model": "gpt-4",
-    "messages": [{"role": "user", "content": "Hello"}],
-    "guardrails": ["panw-customer-a"]
-  }'
-```
-
-**Use Cases:**
-- **Multi-tenant deployments**: Different customers with different security policies
-- **Environment-specific policies**: Dev/staging/prod with different API keys and profiles
-- **A/B testing**: Compare different security profiles side-by-side
-
 ### Content Masking
 
-PANW Prisma AIRS can automatically mask sensitive content (PII, credit cards, SSNs, etc.) instead of blocking requests. This allows your application to continue functioning while protecting sensitive data.
-
-#### How It Works
-
-1. **Detection**: PANW scans content and identifies sensitive data
-2. **Masking**: Sensitive data is replaced with placeholders (e.g., `XXXXXXXXXX` or `{PHONE}`)
-3. **Pass-through**: Masked content is sent to the LLM or returned to the user
-
-#### Configuration Options
+:::warning Important: Masking is Controlled by PANW Security Profile
+The actual masking behavior (what content gets masked and how) is controlled by your PANW Prisma AIRS security profile in Strata Cloud Manager. The LiteLLM flags (`mask_request_content`, `mask_response_content`) only control whether to apply the masked content and allow the request to continue, or block entirely.
+:::
 
 ```yaml
 guardrails:
   - guardrail_name: "panw-with-masking"
     litellm_params:
       guardrail: panw_prisma_airs
-      mode: "post_call"                      # Scan response output
+      mode: "post_call"
       api_key: os.environ/PANW_PRISMA_AIRS_API_KEY
       profile_name: "default"
-      mask_request_content: true             # Mask sensitive data in prompts
-      mask_response_content: true            # Mask sensitive data in responses
+      mask_request_content: true
+      mask_response_content: true
 ```
 
-**Masking Parameters:**
-
-- `mask_request_content: true` - When PANW detects sensitive data in prompts, mask it instead of blocking
-- `mask_response_content: true` - When PANW detects sensitive data in responses, mask it instead of blocking  
-- `mask_on_block: true` - Backwards compatible flag that enables both request and response masking
-
-:::warning Important: Masking is Controlled by PANW Security Profile
-The **actual masking behavior** (what content gets masked and how) is controlled by your **PANW Prisma AIRS security profile** configured in Strata Cloud Manager. The LiteLLM config settings (`mask_request_content`, `mask_response_content`) only control whether to:
-- **Apply the masked content** returned by PANW and allow the request to continue, OR
-- **Block the request** entirely when sensitive data is detected
-
-LiteLLM does not alter or configure your PANW security profile. To change what content gets masked, update your profile settings in Strata Cloud Manager.
-:::
-
-:::info Security Posture
-The guardrail is **fail-closed** by default - if the PANW API is unavailable, requests are blocked to ensure no unscanned content reaches your LLM. This provides maximum security.
-:::
-
-### Custom Violation Messages
-
-You can customize the error message returned to the user when a request is blocked by configuring the `violation_message_template` parameter. This is useful for providing user-friendly feedback instead of technical details.
-
-```yaml
-guardrails:
-  - guardrail_name: "panw-custom-message"
-    litellm_params:
-      guardrail: panw_prisma_airs
-      api_key: os.environ/PANW_PRISMA_AIRS_API_KEY
-      # Simple message
-      violation_message_template: "Your request was blocked by our AI Security Policy."
-
-  - guardrail_name: "panw-detailed-message"
-    litellm_params:
-      guardrail: panw_prisma_airs
-      api_key: os.environ/PANW_PRISMA_AIRS_API_KEY
-      # Message with placeholders
-      violation_message_template: "{action_type} blocked due to {category} violation. Please contact support."
-```
-
-**Supported Placeholders:**
-- `{guardrail_name}`: Name of the guardrail (e.g. "panw-custom-message")
-- `{category}`: Violation category (e.g. "malicious", "injection", "dlp")
-- `{action_type}`: "Prompt" or "Response"
-- `{default_message}`: The original technical error message
+- `mask_request_content: true` — mask sensitive data in prompts instead of blocking
+- `mask_response_content: true` — mask sensitive data in responses instead of blocking
+- `mask_on_block: true` — backwards-compatible flag that enables both request and response masking
 
 ### Fail-Open Configuration
 
-By default, the PANW guardrail operates in **fail-closed** mode for maximum security. If the PANW API is unavailable (timeout, rate limit, network error), requests are blocked. You can configure **fail-open** mode for high-availability scenarios where service continuity is critical.
-
 ```yaml
 guardrails:
   - guardrail_name: "panw-high-availability"
@@ -488,135 +209,86 @@ guardrails:
       guardrail: panw_prisma_airs
       api_key: os.environ/PANW_PRISMA_AIRS_API_KEY
       profile_name: "production"
-      fallback_on_error: "allow"  # Enable fail-open mode
-      timeout: 5.0                 # Shorter timeout for fail-open
+      fallback_on_error: "allow"
+      timeout: 5.0
 ```
 
-**Configuration Options:**
-
-| Parameter | Value | Behavior |
-|-----------|-------|----------|
-| `fallback_on_error` | `"block"` (default) | **Fail-closed**: Block requests when API unavailable (maximum security) |
-| `fallback_on_error` | `"allow"` | **Fail-open**: Allow requests when API unavailable (high availability) |
-| `timeout` | `1.0` - `60.0` | API call timeout in seconds (default: `10.0`) |
-
 **Error Handling Matrix:**
 
 | Error Type | `fallback_on_error="block"` | `fallback_on_error="allow"` |
 |------------|----------------------------|----------------------------|
-| 401 Unauthorized | Block (500) | Block (500) ⚠️ |
-| 403 Forbidden | Block (500) | Block (500) ⚠️ |
-| Profile Error | Block (500) | Block (500) ⚠️ |
+| 401 Unauthorized | Block (500) | Block (500) |
+| 403 Forbidden | Block (500) | Block (500) |
+| Profile Error | Block (500) | Block (500) |
 | 429 Rate Limit | Block (500) | Allow (`:unscanned`) |
 | Timeout | Block (500) | Allow (`:unscanned`) |
 | Network Error | Block (500) | Allow (`:unscanned`) |
 | 5xx Server Error | Block (500) | Allow (`:unscanned`) |
 | Content Blocked | Block (400) | Block (400) |
 
-⚠️ = Always blocks regardless of fail-open setting 
+Authentication and configuration errors (401, 403, invalid profile) always block. Only transient errors (429, timeout, network) trigger fail-open.
 
-:::warning Security Trade-Off
-Enabling `fallback_on_error="allow"` reduces security in exchange for availability. Requests may proceed **without scanning** when the PANW API is unavailable. Use only when:
-- Service availability is more critical than security scanning
-- You have other security controls in place
-- You monitor the `:unscanned` header for audit trails
+When fail-open is triggered, the response includes a tracking header: `X-LiteLLM-Applied-Guardrails: panw-airs:unscanned`
 
-**Authentication and configuration errors (401, 403, invalid profile) always block** - only transient errors (429, timeout, network) trigger fail-open behavior.
-:::
-
-**Observability:**
-
-When fail-open is triggered, the response includes a special header for tracking:
-
-```
-X-LiteLLM-Applied-Guardrails: panw-airs:unscanned
-```
-
-This allows you to:
-- Track which requests bypassed scanning
-- Alert on unscanned request volumes
-- Audit compliance requirements
-
-#### Example: Masking Credit Card Numbers
-
-<Tabs>
-<TabItem label="Without Masking" value="no-mask">
-
-**Request:**
-```json
-{
-  "messages": [
-    {"role": "user", "content": "My credit card is 4929-3813-3266-4295"}
-  ]
-}
-```
-
-**Response:** ❌ **Blocked with 400 error**
-
-</TabItem>
-<TabItem label="With Masking" value="with-mask">
-
-**Request:**
-```json
-{
-  "messages": [
-    {"role": "user", "content": "My credit card is 4929-3813-3266-4295"}
-  ]
-}
-```
-
-**Masked prompt sent to LLM:**
-```json
-{
-  "messages": [
-    {"role": "user", "content": "My credit card is XXXXXXXXXXXXXXXXXX"}
-  ]
-}
-```
-
-**Response:** ✅ **Allowed with masked content**
-
-</TabItem>
-</Tabs>
-
-#### Masking Capabilities
-
-The guardrail masks sensitive content in:
-
-- ✅ **Chat messages** - User prompts and assistant responses
-- ✅ **Streaming responses** - Real-time masking of streamed content
-- ✅ **Multi-choice responses** - All choices in the response
-- ✅ **Tool/function calls** - Arguments passed to tools and functions
-- ✅ **Content lists** - Mixed content types (text, images, etc.)
-
-#### Complete Example
+### Custom Violation Messages
 
 ```yaml
 guardrails:
-  - guardrail_name: "panw-production-security"
+  - guardrail_name: "panw-custom-message"
     litellm_params:
       guardrail: panw_prisma_airs
-      mode: "post_call"                      # Scan input and output
       api_key: os.environ/PANW_PRISMA_AIRS_API_KEY
-      profile_name: "production-profile"
-      mask_request_content: true             # Mask sensitive prompts
-      mask_response_content: true            # Mask sensitive responses
+      violation_message_template: "Your request was blocked by our AI Security Policy."
+
+  - guardrail_name: "panw-detailed-message"
+    litellm_params:
+      guardrail: panw_prisma_airs
+      api_key: os.environ/PANW_PRISMA_AIRS_API_KEY
+      violation_message_template: "{action_type} blocked due to {category} violation. Please contact support."
 ```
 
-## Use Cases
+**Supported Placeholders:** `{guardrail_name}`, `{category}`, `{action_type}`, `{default_message}`
 
-From [official Prisma AIRS documentation](https://docs.paloaltonetworks.com/ai-runtime-security/activation-and-onboarding/ai-runtime-security-api-intercept-overview):
+## Behavior and Limitations
 
-- **Secure AI models in production**: Validate prompt requests and responses to protect deployed AI models
-- **Detect data poisoning**: Identify contaminated training data before fine-tuning
-- **Protect against adversarial input**: Safeguard AI agents from malicious inputs and outputs
-- **Prevent sensitive data leakage**: Use API-based threat detection to block sensitive data leaks
+### Transaction Tracking
+
+For standard request/response scans, `tr_id` maps to `litellm_call_id`. MCP tool scans use the parent `litellm_call_id` when available; if missing, PANW synthesizes a fallback MCP transaction ID. The real limitation is correlation loss — synthesized MCP `tr_id` values are not grouped with the parent request's prompt/response scans in AIRS dashboards.
+
+By default, LiteLLM generates a UUID for `litellm_call_id`. To provide your own:
+
+```bash
+curl -X POST http://localhost:4000/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -H "Authorization: Bearer sk-1234" \
+  -H "x-litellm-call-id: my-custom-call-id-789" \
+  -d '{
+    "model": "gpt-3.5-turbo",
+    "messages": [{"role": "user", "content": "capital of France"}],
+    "guardrails": ["panw-prisma-airs-guardrail"]
+  }'
+```
+
+The `x-litellm-call-id` is also returned in response headers. If you pass `litellm_trace_id` in request metadata (or via the `x-litellm-trace-id` header), it is included in the PANW API payload metadata but does not affect `tr_id` or appear in Prisma AIRS.
+
+### Streaming
+
+- Response masking works on OpenAI chat streaming (`mask_response_content: true`)
+- `/v1/messages` and `/v1/responses` raw streaming blocks instead of masking when violations are detected
+- Request-side masking (`mask_request_content`) is unaffected by endpoint type
+- When `fallback_on_error: "allow"` is set, streaming responses fail open on transient PANW API errors (timeout, 5xx, network) — original chunks are yielded unchanged
+
+## MCP Tool Security
+
+Tool invocations are sent to AIRS as structured `tool_event` payloads containing tool name, ecosystem, and serialized arguments. Tool-event scans always use request mode.
+
+**What is scanned:** LLM-driven `tool_calls` (name + arguments) and MCP request-side invocations when `mcp_tool_name` (or fallback `name`) is present. Response-side OpenAI-compatible `tool_calls` are also scanned when surfaced into `apply_guardrail()`.
+
+**What is not scanned:** Tool definitions in `inputs["tools"]` and post-MCP tool results (no `post_mcp_call` hook exists yet).
 
 
-## Next Steps
+### Current Limitations
 
-- Configure your security policies in [Strata Cloud Manager](https://apps.paloaltonetworks.com/)
-- Review the [Prisma AIRS API documentation](https://pan.dev/airs/) for advanced features
-- Set up monitoring and alerting for threat detections in your PANW dashboard
-- Consider implementing both pre_call and post_call guardrails for comprehensive protection
-- Monitor detection events and tune your security profiles based on your application needs
+- **No post-MCP response scanning.** Actual post-MCP tool-result scanning is not supported because there is no `post_mcp_call` hook in the framework. Response-side MCP events are only scanned when they appear as regular `tool_calls` in the LLM response.
+- **Guardrail selection not inherited by MCP sub-calls.** With `default_on: false`, MCP request-side child-call scans can be skipped because the parent request's guardrail selection is not propagated to the synthetic MCP payload. Workaround: use a dedicated guardrail with `mode: pre_mcp_call` and `default_on: true`.
+- **MCP transaction correlation.** MCP tool scans use the parent `litellm_call_id` when available; otherwise a fallback ID is synthesized and will not be grouped with the parent request in AIRS dashboards.

litellm-proxy-extras/litellm_proxy_extras/migrations/20260309115809_add_missing_indexes/migration.sql

@@ -0,0 +1,13 @@
+-- SkipTransactionBlock
+
+-- Drop invalid indexes left behind by failed CONCURRENTLY builds
+DROP INDEX CONCURRENTLY IF EXISTS "LiteLLM_VerificationToken_key_alias_idx";
+
+-- CreateIndex
+CREATE INDEX CONCURRENTLY "LiteLLM_VerificationToken_key_alias_idx" ON "LiteLLM_VerificationToken"("key_alias");
+
+-- Drop invalid indexes left behind by failed CONCURRENTLY builds
+DROP INDEX CONCURRENTLY IF EXISTS "LiteLLM_SpendLogs_user_startTime_idx";
+
+-- CreateIndex
+CREATE INDEX CONCURRENTLY "LiteLLM_SpendLogs_user_startTime_idx" ON "LiteLLM_SpendLogs"("user", "startTime");

litellm-proxy-extras/litellm_proxy_extras/schema.prisma

@@ -388,6 +388,9 @@ model LiteLLM_VerificationToken {
 
     // SELECT ... FROM "public"."LiteLLM_VerificationToken" WHERE (("public"."LiteLLM_VerificationToken"."expires" IS NULL OR "public"."LiteLLM_VerificationToken"."expires" > $1) AND "public"."LiteLLM_VerificationToken"."budget_reset_at" < $2) OFFSET $3
     @@index([budget_reset_at, expires])
+
+    // SELECT ... FROM "public"."LiteLLM_VerificationToken" WHERE (...) ORDER BY "public"."LiteLLM_VerificationToken"."key_alias" ASC
+    @@index([key_alias])
 }
 
 model LiteLLM_JWTKeyMapping {
@@ -553,6 +556,9 @@ model LiteLLM_SpendLogs {
   @@index([startTime, request_id])
   @@index([end_user])
   @@index([session_id])
+
+  // SELECT ... FROM "LiteLLM_SpendLogs" WHERE ("startTime" >= $1 AND "startTime" <= $2 AND "user" = $3) GROUP BY ...
+  @@index([user, startTime])
 }
 
 // View spend, model, api_key per request

litellm/caching/dual_cache.py

@@ -346,8 +346,6 @@ class DualCache(BaseCache):
         )
         try:
             if self.in_memory_cache is not None:
-                if "ttl" not in kwargs and self.default_in_memory_ttl is not None:
-                    kwargs["ttl"] = self.default_in_memory_ttl
                 await self.in_memory_cache.async_set_cache(key, value, **kwargs)
 
             if self.redis_cache is not None and local_only is False:
@@ -369,8 +367,6 @@ class DualCache(BaseCache):
         )
         try:
             if self.in_memory_cache is not None:
-                if "ttl" not in kwargs and self.default_in_memory_ttl is not None:
-                    kwargs["ttl"] = self.default_in_memory_ttl
                 await self.in_memory_cache.async_set_cache_pipeline(
                     cache_list=cache_list, **kwargs
                 )

litellm/completion_extras/litellm_responses_transformation/transformation.py

@@ -390,7 +390,6 @@ class LiteLLMResponsesTransformationHandler(CompletionTransformationBridge):
             ResponseOutputMessage,
             ResponseReasoningItem,
         )
-        from openai.types.responses.response_output_item import ResponseApplyPatchToolCall
 
         from litellm.types.utils import Choices, Message
 
@@ -449,18 +448,6 @@ class LiteLLMResponsesTransformationHandler(CompletionTransformationBridge):
                 accumulated_tool_calls.append(tool_call_dict)
                 tool_call_index += 1
 
-            elif isinstance(item, ResponseApplyPatchToolCall):
-                from litellm.responses.litellm_completion_transformation.transformation import (
-                    LiteLLMCompletionResponsesConfig,
-                )
-
-                tool_call_dict = LiteLLMCompletionResponsesConfig.convert_apply_patch_tool_call_to_chat_completion_tool_call(
-                    tool_call_item=item,
-                    index=tool_call_index,
-                )
-                accumulated_tool_calls.append(tool_call_dict)
-                tool_call_index += 1
-
             elif isinstance(item, dict) and handle_raw_dict_callback is not None:
                 # Handle raw dict responses (e.g., from GPT-5 Codex)
                 choice, index = handle_raw_dict_callback(item=item, index=index)
@@ -1108,12 +1095,6 @@ class OpenAiResponsesToChatCompletionStreamIterator(BaseModelResponseIterator):
 
             finish_reason = "tool_calls" if has_function_calls else "stop"
 
-            usage = None
-            if response_data.get("usage"):
-                from litellm.responses.utils import ResponseAPILoggingUtils
-                usage = ResponseAPILoggingUtils._transform_response_api_usage_to_chat_usage(
-                    response_data.get("usage")
-                )
             return ModelResponseStream(
                 choices=[
                     StreamingChoices(
@@ -1121,8 +1102,7 @@ class OpenAiResponsesToChatCompletionStreamIterator(BaseModelResponseIterator):
                         delta=Delta(content=""),
                         finish_reason=finish_reason,
                     )
-                ],
-                usage=usage
+                ]
             )
         else:
             pass

litellm/constants.py

@@ -1214,12 +1214,8 @@ OPENAI_FINISH_REASONS = [
     "stop",
     "length",
     "function_call",
+    "tool_calls",
     "content_filter",
-    "null",
-    "finish_reason_unspecified",
-    "malformed_function_call",
-    "guardrail_intervened",
-    "eos",
 ]
 HUMANLOOP_PROMPT_CACHE_TTL_SECONDS = int(
     os.getenv("HUMANLOOP_PROMPT_CACHE_TTL_SECONDS", 60)

litellm/google_genai/adapters/transformation.py

@@ -770,8 +770,6 @@ class GoogleGenAIAdapter:
             "content_filter": "SAFETY",
             "tool_calls": "STOP",
             "function_call": "STOP",
-            "finish_reason_unspecified": "FINISH_REASON_UNSPECIFIED",
-            "malformed_function_call": "MALFORMED_FUNCTION_CALL",
         }
 
         return mapping.get(finish_reason, "STOP")

litellm/litellm_core_utils/core_helpers.py

@@ -1,11 +1,11 @@
 # What is this?
 ## Helper utilities
-from typing import TYPE_CHECKING, Any, Iterable, List, Literal, Optional, Union
+from typing import TYPE_CHECKING, Any, Iterable, List, Literal, Optional, Union, get_args
 
 import httpx
 
 from litellm._logging import verbose_logger
-from litellm.types.llms.openai import AllMessageValues
+from litellm.types.llms.openai import AllMessageValues, OpenAIChatCompletionFinishReason
 
 if TYPE_CHECKING:
     from opentelemetry.trace import Span as _Span
@@ -58,45 +58,55 @@ def safe_divide(
     return numerator / denominator
 
 
-def map_finish_reason(
-    finish_reason: str,
-):  # openai supports 5 stop sequences - 'stop', 'length', 'function_call', 'content_filter', 'null'
-    # anthropic mapping
-    if finish_reason == "stop_sequence":
+_FINISH_REASON_MAP: dict[str, OpenAIChatCompletionFinishReason] = {
+    # Anthropic
+    "stop_sequence": "stop",
+    "end_turn": "stop",
+    "max_tokens": "length",
+    "tool_use": "tool_calls",
+    "compaction": "length",
+    # Cohere
+    "COMPLETE": "stop",
+    "ERROR_TOXIC": "content_filter",
+    "ERROR": "stop",
+    # HuggingFace / Together AI
+    "eos_token": "stop",
+    "eos": "stop",
+    # Gemini / Vertex AI
+    "STOP": "stop",
+    "MAX_TOKENS": "length",
+    "SAFETY": "content_filter",
+    "RECITATION": "content_filter",
+    "FINISH_REASON_UNSPECIFIED": "stop",
+    "MALFORMED_FUNCTION_CALL": "stop",
+    "LANGUAGE": "content_filter",
+    "OTHER": "content_filter",
+    "BLOCKLIST": "content_filter",
+    "PROHIBITED_CONTENT": "content_filter",
+    "SPII": "content_filter",
+    "IMAGE_SAFETY": "content_filter",
+    "IMAGE_PROHIBITED_CONTENT": "content_filter",
+    "TOO_MANY_TOOL_CALLS": "stop",
+    "MALFORMED_RESPONSE": "stop",
+    # Bedrock
+    "guardrail_intervened": "content_filter",
+    # OpenAI passthrough
+    "stop": "stop",
+    "length": "length",
+    "tool_calls": "tool_calls",
+    "function_call": "function_call",
+    "content_filter": "content_filter",
+}
+
+
+def map_finish_reason(finish_reason: str) -> OpenAIChatCompletionFinishReason:
+    mapped = _FINISH_REASON_MAP.get(finish_reason)
+    if mapped is None:
+        verbose_logger.warning(
+            "Unmapped finish_reason '%s', defaulting to 'stop'", finish_reason
+        )
         return "stop"
-    # cohere mapping - https://docs.cohere.com/reference/generate
-    elif finish_reason == "COMPLETE":
-        return "stop"
-    elif finish_reason == "MAX_TOKENS":  # cohere + vertex ai
-        return "length"
-    elif finish_reason == "ERROR_TOXIC":
-        return "content_filter"
-    elif (
-        finish_reason == "ERROR"
-    ):  # openai currently doesn't support an 'error' finish reason
-        return "stop"
-    # huggingface mapping https://huggingface.github.io/text-generation-inference/#/Text%20Generation%20Inference/generate_stream
-    elif finish_reason == "eos_token" or finish_reason == "stop_sequence":
-        return "stop"
-    elif (
-        finish_reason == "FINISH_REASON_UNSPECIFIED"
-    ):  # vertex ai - got from running `print(dir(response_obj.candidates[0].finish_reason))`: ['FINISH_REASON_UNSPECIFIED', 'MAX_TOKENS', 'OTHER', 'RECITATION', 'SAFETY', 'STOP',]
-        return "finish_reason_unspecified"
-    elif finish_reason == "MALFORMED_FUNCTION_CALL":
-        return "malformed_function_call"
-    elif finish_reason == "SAFETY" or finish_reason == "RECITATION":  # vertex ai
-        return "content_filter"
-    elif finish_reason == "STOP":  # vertex ai
-        return "stop"
-    elif finish_reason == "end_turn" or finish_reason == "stop_sequence":  # anthropic
-        return "stop"
-    elif finish_reason == "max_tokens":  # anthropic
-        return "length"
-    elif finish_reason == "tool_use":  # anthropic
-        return "tool_calls"
-    elif finish_reason == "compaction":
-        return "length"
-    return finish_reason
+    return mapped
 
 
 def remove_index_from_tool_calls(

litellm/litellm_core_utils/duration_parser.py

@@ -64,10 +64,12 @@ def duration_in_seconds(duration: str) -> int:
         now = time.time()
         current_time = datetime.fromtimestamp(now)
 
-        # Calculate target month and year, handling overflow past December
-        total_months = current_time.month - 1 + value  # 0-indexed months
-        target_year = current_time.year + total_months // 12
-        target_month = total_months % 12 + 1  # back to 1-indexed
+        if current_time.month == 12:
+            target_year = current_time.year + 1
+            target_month = 1
+        else:
+            target_year = current_time.year
+            target_month = current_time.month + value
 
         # Determine the day to set for next month
         target_day = current_time.day

litellm/litellm_core_utils/get_model_cost_map.py

@@ -11,7 +11,7 @@ export LITELLM_LOCAL_MODEL_COST_MAP=True
 import json
 import os
 from importlib.resources import files
-from typing import Optional
+from typing import Dict, List, Optional
 
 import httpx
 
@@ -183,6 +183,61 @@ def get_model_cost_map_source_info() -> dict:
     }
 
 
+def _expand_model_aliases(model_cost: dict) -> dict:
+    """
+    Expand ``aliases`` lists in model cost entries into top-level entries.
+
+    Each alias gets a reference to the **same** dict object as the canonical
+    entry (zero memory overhead).  The ``aliases`` key is removed from the
+    entry so downstream code never sees it.
+
+    If an alias collides with an existing canonical entry the alias is
+    skipped and a warning is logged.
+    """
+    aliases_to_add: Dict[str, dict] = {}
+    keys_with_aliases: List[str] = []
+
+    for model_name, model_info in model_cost.items():
+        aliases: Optional[list] = model_info.get("aliases")
+        if aliases is None:
+            continue
+        keys_with_aliases.append(model_name)
+        if not isinstance(aliases, list):
+            verbose_logger.warning(
+                "LiteLLM model alias field for '%s' is not a list (got %s) — skipping.",
+                model_name,
+                type(aliases).__name__,
+            )
+            continue
+        if not aliases:
+            continue
+        for alias in aliases:
+            if alias in model_cost:
+                verbose_logger.warning(
+                    "LiteLLM model alias conflict: alias '%s' (from '%s') "
+                    "already exists as a canonical entry — skipping.",
+                    alias,
+                    model_name,
+                )
+                continue
+            if alias in aliases_to_add:
+                verbose_logger.warning(
+                    "LiteLLM model alias conflict: alias '%s' (from '%s') "
+                    "was already claimed by another entry — skipping.",
+                    alias,
+                    model_name,
+                )
+                continue
+            aliases_to_add[alias] = model_info  # same dict reference
+
+    # Remove the ``aliases`` key from entries so it doesn't pollute model info
+    for key in keys_with_aliases:
+        model_cost[key].pop("aliases", None)
+
+    model_cost.update(aliases_to_add)
+    return model_cost
+
+
 def get_model_cost_map(url: str) -> dict:
     """
     Public entry point — returns the model cost map dict.
@@ -202,7 +257,7 @@ def get_model_cost_map(url: str) -> dict:
         _cost_map_source_info.url = None
         _cost_map_source_info.is_env_forced = True
         _cost_map_source_info.fallback_reason = None
-        return GetModelCostMap.load_local_model_cost_map()
+        return _expand_model_aliases(GetModelCostMap.load_local_model_cost_map())
 
     _cost_map_source_info.url = url
     _cost_map_source_info.is_env_forced = False
@@ -218,7 +273,7 @@ def get_model_cost_map(url: str) -> dict:
         )
         _cost_map_source_info.source = "local"
         _cost_map_source_info.fallback_reason = f"Remote fetch failed: {str(e)}"
-        return GetModelCostMap.load_local_model_cost_map()
+        return _expand_model_aliases(GetModelCostMap.load_local_model_cost_map())
 
     # Validate using cached count (cheap int comparison, no file I/O)
     if not GetModelCostMap.validate_model_cost_map(
@@ -232,8 +287,8 @@ def get_model_cost_map(url: str) -> dict:
         )
         _cost_map_source_info.source = "local"
         _cost_map_source_info.fallback_reason = "Remote data failed integrity validation"
-        return GetModelCostMap.load_local_model_cost_map()
+        return _expand_model_aliases(GetModelCostMap.load_local_model_cost_map())
 
     _cost_map_source_info.source = "remote"
     _cost_map_source_info.fallback_reason = None
-    return content
+    return _expand_model_aliases(content)

litellm/litellm_core_utils/prompt_templates/factory.py

@@ -2493,74 +2493,257 @@ def anthropic_messages_pt(  # noqa: PLR0915
                     assistant_content.extend(_compaction_blocks)  # type: ignore
 
             thinking_blocks = assistant_content_block.get("thinking_blocks", None)
+
+            # Check if tool_calls contain server tool calls (web search, etc.)
+            # If so, we need to interleave thinking blocks with tool call groups
+            # to preserve the original content block ordering.
+            # Fixes: https://github.com/BerriAI/litellm/issues/23047
+            assistant_tool_calls = assistant_content_block.get("tool_calls")
+            _has_server_tool_calls = False
+            if assistant_tool_calls is not None:
+                for _tc in assistant_tool_calls:
+                    _tc_id = (
+                        _tc.get("id")
+                        if isinstance(_tc, dict)
+                        else getattr(_tc, "id", None)
+                    )
+                    if _tc_id and isinstance(_tc_id, str) and _tc_id.startswith("srvtoolu_"):
+                        _has_server_tool_calls = True
+                        break
+
             if (
                 thinking_blocks is not None
-            ):  # IMPORTANT: ADD THIS FIRST, ELSE ANTHROPIC WILL RAISE AN ERROR
-                assistant_content.extend(thinking_blocks)
-            if "content" in assistant_content_block and isinstance(
-                assistant_content_block["content"], list
+                and _has_server_tool_calls
+                and isinstance(
+                    assistant_content_block.get("content", None), (str, type(None))
+                )
             ):
-                for m in assistant_content_block["content"]:
-                    # handle thinking blocks
-                    thinking_block = cast(str, m.get("thinking", ""))
-                    text_block = cast(str, m.get("text", ""))
-                    if (
-                        m.get("type", "") == "thinking" and len(thinking_block) > 0
-                    ):  # don't pass empty text blocks. anthropic api raises errors.
-                        anthropic_message: Union[
-                            ChatCompletionThinkingBlock,
-                            AnthropicMessagesTextParam,
-                        ] = cast(ChatCompletionThinkingBlock, m)
-                        assistant_content.append(anthropic_message)
-                    # handle text
-                    elif (
-                        m.get("type", "") == "text" and len(text_block) > 0
-                    ):  # don't pass empty text blocks. anthropic api raises errors.
-                        anthropic_message = AnthropicMessagesTextParam(
-                            type="text", text=text_block
-                        )
-                        _cached_message = add_cache_control_to_content(
-                            anthropic_content_element=anthropic_message,
-                            original_content_element=dict(m),
-                        )
+                # INTERLEAVED MODE: When we have both thinking blocks and server
+                # tool calls (e.g. web search), Anthropic's original response
+                # interleaves them: [thinking_1, server_tool_use_1, result_1,
+                # thinking_2, text, server_tool_use_2, result_2, ...].
+                # We must preserve this interleaved order because Anthropic
+                # verifies thinking block signatures based on position.
 
-                        assistant_content.append(
-                            cast(AnthropicMessagesTextParam, _cached_message)
-                        )
-                    # handle server_tool_use blocks (tool search, web search, etc.)
-                    # Pass through as-is since these are Anthropic-native content types
-                    elif m.get("type", "") == "server_tool_use":
-                        assistant_content.append(m)  # type: ignore
-                    # handle all *_tool_result blocks (tool_search_tool_result,
-                    # web_search_tool_result, bash_code_execution_tool_result, etc.)
-                    # Pass through as-is since these are Anthropic-native content types
-                    elif m.get("type", "").endswith("_tool_result"):
-                        assistant_content.append(m)  # type: ignore
-            elif (
-                "content" in assistant_content_block
-                and isinstance(assistant_content_block["content"], str)
-                and assistant_content_block[
-                    "content"
-                ]  # don't pass empty text blocks. anthropic api raises errors.
-            ):
-                _anthropic_text_content_element = AnthropicMessagesTextParam(
-                    type="text",
-                    text=assistant_content_block["content"],
+                # Build the tool call groups (server_tool_use + its result)
+                _provider_specific_fields_raw_tc = assistant_content_block.get(
+                    "provider_specific_fields"
+                )
+                _provider_specific_fields_tc: Dict[str, Any] = {}
+                if isinstance(_provider_specific_fields_raw_tc, dict):
+                    _provider_specific_fields_tc = cast(
+                        Dict[str, Any], _provider_specific_fields_raw_tc
+                    )
+                _web_search_results_tc = _provider_specific_fields_tc.get(
+                    "web_search_results"
+                )
+                _tool_results_tc = _provider_specific_fields_tc.get("tool_results")
+                tool_invoke_results = convert_to_anthropic_tool_invoke(
+                    assistant_tool_calls,  # type: ignore
+                    web_search_results=_web_search_results_tc,
+                    tool_results=_tool_results_tc,
                 )
 
-                _content_element = add_cache_control_to_content(
-                    anthropic_content_element=_anthropic_text_content_element,
-                    original_content_element=dict(assistant_content_block),
+                # Group tool invoke results into (server_tool_use, result) pairs
+                # and separate regular tool_use blocks
+                server_tool_groups: List[List[Any]] = []
+                regular_tool_uses: List[Any] = []
+                _current_group: List[Any] = []
+                for item in tool_invoke_results:
+                    item_type = (
+                        item.get("type", "")
+                        if isinstance(item, dict)
+                        else getattr(item, "type", "")
+                    )
+                    if item_type == "server_tool_use":
+                        if _current_group:
+                            server_tool_groups.append(_current_group)
+                        _current_group = [item]
+                    elif item_type.endswith("_tool_result"):
+                        _current_group.append(item)
+                    elif item_type == "tool_use":
+                        regular_tool_uses.append(item)
+                    else:
+                        _current_group.append(item)
+                if _current_group:
+                    server_tool_groups.append(_current_group)
+
+                # Build the text block if content is a non-empty string
+                text_element = None
+                if (
+                    isinstance(assistant_content_block.get("content"), str)
+                    and assistant_content_block["content"]
+                ):
+                    _anthropic_text_content_element = AnthropicMessagesTextParam(
+                        type="text",
+                        text=assistant_content_block["content"],
+                    )
+                    _content_element = add_cache_control_to_content(
+                        anthropic_content_element=_anthropic_text_content_element,
+                        original_content_element=dict(assistant_content_block),
+                    )
+                    if "cache_control" in _content_element:
+                        _anthropic_text_content_element["cache_control"] = (
+                            _content_element["cache_control"]
+                        )
+                    text_element = _anthropic_text_content_element
+
+                # Interleave: each thinking block precedes its server tool group.
+                # Pattern: thinking[0], group[0], thinking[1], group[1], ...
+                # Any remaining thinking blocks (after all groups) go before text.
+                # Any remaining groups (after all thinking blocks) go after.
+                tb_idx = 0
+                grp_idx = 0
+                num_tb = len(thinking_blocks) if thinking_blocks else 0
+                num_grp = len(server_tool_groups)
+
+                while tb_idx < num_tb or grp_idx < num_grp:
+                    if tb_idx < num_tb and grp_idx < num_grp:
+                        # Emit thinking block then its tool group
+                        assistant_content.append(thinking_blocks[tb_idx])
+                        tb_idx += 1
+                        for block in server_tool_groups[grp_idx]:
+                            item_id = (
+                                block.get("id")
+                                if isinstance(block, dict)
+                                else getattr(block, "id", None)
+                            )
+                            if item_id and item_id in unique_tool_ids:
+                                continue
+                            if item_id:
+                                unique_tool_ids.add(item_id)
+                            assistant_content.append(
+                                cast(AnthropicMessagesAssistantMessageValues, block)
+                            )
+                        grp_idx += 1
+                    elif tb_idx < num_tb:
+                        # More thinking blocks than tool groups - emit before text
+                        assistant_content.append(thinking_blocks[tb_idx])
+                        tb_idx += 1
+                    else:
+                        # More tool groups than thinking blocks - emit remaining
+                        for block in server_tool_groups[grp_idx]:
+                            item_id = (
+                                block.get("id")
+                                if isinstance(block, dict)
+                                else getattr(block, "id", None)
+                            )
+                            if item_id and item_id in unique_tool_ids:
+                                continue
+                            if item_id:
+                                unique_tool_ids.add(item_id)
+                            assistant_content.append(
+                                cast(AnthropicMessagesAssistantMessageValues, block)
+                            )
+                        grp_idx += 1
+
+                # Add text block (if any)
+                if text_element is not None:
+                    assistant_content.append(text_element)
+
+                # Add regular (non-server) tool calls at the end
+                for item in regular_tool_uses:
+                    item_id = (
+                        item.get("id")
+                        if isinstance(item, dict)
+                        else getattr(item, "id", None)
+                    )
+                    if item_id and item_id in unique_tool_ids:
+                        continue
+                    if item_id:
+                        unique_tool_ids.add(item_id)
+                    assistant_content.append(
+                        cast(AnthropicMessagesAssistantMessageValues, item)
+                    )
+
+                # Mark tool_calls as already processed so they are not added again
+                assistant_tool_calls = None
+
+            else:
+                # SEQUENTIAL MODE: No server tool calls, or no thinking blocks,
+                # or content is a list. Use the original sequential approach.
+
+                # When content is a list, check if it already contains thinking
+                # blocks inline. If so, skip prepending thinking_blocks to avoid
+                # duplication and preserve the original interleaved order.
+                # Fixes the gap where list-content messages bypass INTERLEAVED
+                # MODE and still get thinking blocks prepended out of order.
+                _content_is_list = "content" in assistant_content_block and isinstance(
+                    assistant_content_block["content"], list
                 )
+                _list_has_thinking = False
+                if _content_is_list:
+                    for _item in assistant_content_block["content"]:
+                        if isinstance(_item, dict) and _item.get("type") in ("thinking", "redacted_thinking"):
+                            _list_has_thinking = True
+                            break
 
-                if "cache_control" in _content_element:
-                    _anthropic_text_content_element["cache_control"] = _content_element[
-                        "cache_control"
-                    ]
+                if (
+                    thinking_blocks is not None
+                    and not _list_has_thinking
+                ):  # IMPORTANT: ADD THIS FIRST, ELSE ANTHROPIC WILL RAISE AN ERROR
+                    assistant_content.extend(thinking_blocks)
+                if _content_is_list:
+                    for m in assistant_content_block["content"]:
+                        # handle thinking blocks
+                        thinking_block = cast(str, m.get("thinking", ""))
+                        text_block = cast(str, m.get("text", ""))
+                        if (
+                            m.get("type", "") == "thinking" and len(thinking_block) > 0
+                        ):  # don't pass empty text blocks. anthropic api raises errors.
+                            anthropic_message: Union[
+                                ChatCompletionThinkingBlock,
+                                AnthropicMessagesTextParam,
+                            ] = cast(ChatCompletionThinkingBlock, m)
+                            assistant_content.append(anthropic_message)
+                        # handle text
+                        elif (
+                            m.get("type", "") == "text" and len(text_block) > 0
+                        ):  # don't pass empty text blocks. anthropic api raises errors.
+                            anthropic_message = AnthropicMessagesTextParam(
+                                type="text", text=text_block
+                            )
+                            _cached_message = add_cache_control_to_content(
+                                anthropic_content_element=anthropic_message,
+                                original_content_element=dict(m),
+                            )
 
-                assistant_content.append(_anthropic_text_content_element)
+                            assistant_content.append(
+                                cast(AnthropicMessagesTextParam, _cached_message)
+                            )
+                        # handle server_tool_use blocks (tool search, web search, etc.)
+                        # Pass through as-is since these are Anthropic-native content types
+                        elif m.get("type", "") == "server_tool_use":
+                            assistant_content.append(m)  # type: ignore
+                        # handle all *_tool_result blocks (tool_search_tool_result,
+                        # web_search_tool_result, bash_code_execution_tool_result, etc.)
+                        # Pass through as-is since these are Anthropic-native content types
+                        elif m.get("type", "").endswith("_tool_result"):
+                            assistant_content.append(m)  # type: ignore
+                elif (
+                    "content" in assistant_content_block
+                    and isinstance(assistant_content_block["content"], str)
+                    and assistant_content_block[
+                        "content"
+                    ]  # don't pass empty text blocks. anthropic api raises errors.
+                ):
+                    _anthropic_text_content_element = AnthropicMessagesTextParam(
+                        type="text",
+                        text=assistant_content_block["content"],
+                    )
+
+                    _content_element = add_cache_control_to_content(
+                        anthropic_content_element=_anthropic_text_content_element,
+                        original_content_element=dict(assistant_content_block),
+                    )
+
+                    if "cache_control" in _content_element:
+                        _anthropic_text_content_element["cache_control"] = _content_element[
+                            "cache_control"
+                        ]
+
+                    assistant_content.append(_anthropic_text_content_element)
 
-            assistant_tool_calls = assistant_content_block.get("tool_calls")
             if (
                 assistant_tool_calls is not None
             ):  # support assistant tool invoke conversion

litellm/litellm_core_utils/redact_messages.py

@@ -73,53 +73,6 @@ def _redact_responses_api_output(output_items):
                         summary_item.text = "redacted-by-litellm"
 
 
-def _redact_standard_logging_object(model_call_details: dict):
-    """Redact messages and response inside standard_logging_object if present."""
-    standard_logging_object = model_call_details.get("standard_logging_object")
-    if standard_logging_object is None:
-        return
-
-    redacted_str = "redacted-by-litellm"
-
-    if standard_logging_object.get("messages") is not None:
-        standard_logging_object["messages"] = [
-            {"role": "user", "content": redacted_str}
-        ]
-
-    response = standard_logging_object.get("response")
-    if response is not None:
-        if isinstance(response, dict) and "output" in response:
-            # ResponsesAPIResponse format - redact content in output items
-            if isinstance(response.get("output"), list):
-                for output_item in response["output"]:
-                    if isinstance(output_item, dict) and "content" in output_item:
-                        if isinstance(output_item["content"], list):
-                            for content_item in output_item["content"]:
-                                if (
-                                    isinstance(content_item, dict)
-                                    and "text" in content_item
-                                ):
-                                    content_item["text"] = redacted_str
-        elif isinstance(response, dict) and "choices" in response:
-            # ModelResponse dict format - redact content in choices
-            if isinstance(response.get("choices"), list):
-                for choice in response["choices"]:
-                    if isinstance(choice, dict):
-                        if "message" in choice and isinstance(choice["message"], dict):
-                            choice["message"]["content"] = redacted_str
-                            if "audio" in choice["message"]:
-                                choice["message"]["audio"] = None
-                        elif "delta" in choice and isinstance(choice["delta"], dict):
-                            choice["delta"]["content"] = redacted_str
-                            if "audio" in choice["delta"]:
-                                choice["delta"]["audio"] = None
-        elif isinstance(response, str):
-            standard_logging_object["response"] = redacted_str
-        else:
-            # For other formats (empty dict, None, etc.), use simple text format
-            standard_logging_object["response"] = {"text": redacted_str}
-
-
 def perform_redaction(model_call_details: dict, result):
     """
     Performs the actual redaction on the logging object and result.
@@ -161,29 +114,6 @@ def perform_redaction(model_call_details: dict, result):
             if hasattr(_result, "choices") and _result.choices is not None:
                 for choice in _result.choices:
                     _redact_choice_content(choice)
-        elif isinstance(_result, dict) and "choices" in _result:
-            # Handle dict representation of ModelResponse (e.g., from model_dump())
-            if _result.get("choices") is not None:
-                for choice in _result["choices"]:
-                    if isinstance(choice, dict):
-                        if "message" in choice and isinstance(choice["message"], dict):
-                            choice["message"]["content"] = "redacted-by-litellm"
-                            if "reasoning_content" in choice["message"]:
-                                choice["message"]["reasoning_content"] = "redacted-by-litellm"
-                            if "thinking_blocks" in choice["message"]:
-                                choice["message"]["thinking_blocks"] = None
-                            if "audio" in choice["message"]:
-                                choice["message"]["audio"] = None
-                        elif "delta" in choice and isinstance(choice["delta"], dict):
-                            choice["delta"]["content"] = "redacted-by-litellm"
-                            if "reasoning_content" in choice["delta"]:
-                                choice["delta"]["reasoning_content"] = "redacted-by-litellm"
-                            if "thinking_blocks" in choice["delta"]:
-                                choice["delta"]["thinking_blocks"] = None
-                            if "audio" in choice["delta"]:
-                                choice["delta"]["audio"] = None
-                    else:
-                        _redact_choice_content(choice)
         elif isinstance(_result, litellm.ResponsesAPIResponse):
             if hasattr(_result, "output"):
                 _redact_responses_api_output(_result.output)

litellm/llms/azure/chat/gpt_5_transformation.py

@@ -4,10 +4,7 @@ from typing import List
 
 import litellm
 from litellm.exceptions import UnsupportedParamsError
-from litellm.llms.openai.chat.gpt_5_transformation import (
-    OpenAIGPT5Config,
-    _get_effort_level,
-)
+from litellm.llms.openai.chat.gpt_5_transformation import OpenAIGPT5Config
 from litellm.types.llms.openai import AllMessageValues
 
 from .gpt_transformation import AzureOpenAIConfig
@@ -84,21 +81,20 @@ class AzureOpenAIGPT5Config(AzureOpenAIConfig, OpenAIGPT5Config):
             non_default_params.get("reasoning_effort")
             or optional_params.get("reasoning_effort")
         )
-        effective_effort = _get_effort_level(reasoning_effort_value)
 
         # gpt-5.1/5.2/5.4 support reasoning_effort='none', but other gpt-5 models don't
         # See: https://learn.microsoft.com/en-us/azure/ai-foundry/openai/how-to/reasoning
         supports_none = self._supports_reasoning_effort_level(model, "none")
 
-        if effective_effort == "none" and not supports_none:
+        if reasoning_effort_value == "none" and not supports_none:
             if litellm.drop_params is True or (
                 drop_params is not None and drop_params is True
             ):
                 non_default_params = non_default_params.copy()
                 optional_params = optional_params.copy()
-                if _get_effort_level(non_default_params.get("reasoning_effort")) == "none":
+                if non_default_params.get("reasoning_effort") == "none":
                     non_default_params.pop("reasoning_effort")
-                if _get_effort_level(optional_params.get("reasoning_effort")) == "none":
+                if optional_params.get("reasoning_effort") == "none":
                     optional_params.pop("reasoning_effort")
             else:
                 raise UnsupportedParamsError(
@@ -121,19 +117,9 @@ class AzureOpenAIGPT5Config(AzureOpenAIConfig, OpenAIGPT5Config):
         )
 
         # Only drop reasoning_effort='none' for models that don't support it
-        result_effort = _get_effort_level(result.get("reasoning_effort"))
-        if result_effort == "none" and not supports_none:
+        if result.get("reasoning_effort") == "none" and not supports_none:
             result.pop("reasoning_effort")
 
-        # Azure Chat Completions: gpt-5.4+ does not support tools + reasoning together.
-        # Drop reasoning_effort when both are present (OpenAI routes to Responses API; Azure does not).
-        if self.is_model_gpt_5_4_plus_model(model):
-            has_tools = bool(
-                non_default_params.get("tools") or optional_params.get("tools")
-            )
-            if has_tools and result_effort not in (None, "none"):
-                result.pop("reasoning_effort", None)
-
         return result
 
     def transform_request(

litellm/llms/bedrock/chat/converse_transformation.py

@@ -51,7 +51,6 @@ from litellm.types.llms.openai import (
 )
 from litellm.types.utils import (
     ChatCompletionMessageToolCall,
-    CompletionTokensDetailsWrapper,
     Function,
     Message,
     ModelResponse,
@@ -64,7 +63,6 @@ from litellm.utils import (
     has_tool_call_blocks,
     last_assistant_with_tool_calls_has_no_thinking_blocks,
     supports_reasoning,
-    token_counter,
 )
 
 from ..common_utils import (
@@ -1208,7 +1206,6 @@ class AmazonConverseConfig(BaseConfig):
             self._validate_request_metadata(request_metadata)
 
         output_config: Optional[OutputConfigBlock] = inference_params.pop("outputConfig", None)
-        inference_params.pop("output_config", None)  # Bedrock Converse doesn't support it
 
         # keep supported params in 'inference_params', and set all model-specific params in 'additional_request_params'
         additional_request_params = {
@@ -1623,11 +1620,7 @@ class AmazonConverseConfig(BaseConfig):
                 thinking_blocks_list.append(_redacted_block)
         return thinking_blocks_list
 
-    def _transform_usage(
-        self,
-        usage: ConverseTokenUsageBlock,
-        reasoning_content: Optional[str] = None,
-    ) -> Usage:
+    def _transform_usage(self, usage: ConverseTokenUsageBlock) -> Usage:
         input_tokens = usage["inputTokens"]
         output_tokens = usage["outputTokens"]
         total_tokens = usage["totalTokens"]
@@ -1644,19 +1637,6 @@ class AmazonConverseConfig(BaseConfig):
         prompt_tokens_details = PromptTokensDetailsWrapper(
             cached_tokens=cache_read_input_tokens
         )
-        reasoning_tokens = (
-            token_counter(text=reasoning_content, count_response_tokens=True)
-            if reasoning_content
-            else 0
-        )
-        completion_tokens_details = CompletionTokensDetailsWrapper(
-            reasoning_tokens=reasoning_tokens,
-            text_tokens=(
-                output_tokens - reasoning_tokens
-                if reasoning_tokens > 0
-                else output_tokens
-            ),
-        )
         openai_usage = Usage(
             prompt_tokens=input_tokens,
             completion_tokens=output_tokens,
@@ -1664,7 +1644,6 @@ class AmazonConverseConfig(BaseConfig):
             prompt_tokens_details=prompt_tokens_details,
             cache_creation_input_tokens=cache_creation_input_tokens,
             cache_read_input_tokens=cache_read_input_tokens,
-            completion_tokens_details=completion_tokens_details,
         )
         return openai_usage
 
@@ -2001,10 +1980,7 @@ class AmazonConverseConfig(BaseConfig):
             chat_completion_message["tool_calls"] = filtered_tools
 
         ## CALCULATING USAGE - bedrock returns usage in the headers
-        usage = self._transform_usage(
-            completion_response["usage"],
-            reasoning_content=chat_completion_message.get("reasoning_content"),
-        )
+        usage = self._transform_usage(completion_response["usage"])
 
         ## HANDLE TOOL CALLS
         _message = Message(**chat_completion_message)

litellm/llms/fireworks_ai/chat/transformation.py

@@ -426,11 +426,8 @@ class FireworksAIConfig(OpenAIGPTConfig):
                 "FIREWORKS_ACCOUNT_ID is not set. Please set the environment variable, to query Fireworks AI's `/models` endpoint."
             )
 
-        base = api_base.rstrip("/")
-        if base.endswith("/v1"):
-            base = base[: -len("/v1")]
         response = litellm.module_level_client.get(
-            url=f"{base}/v1/accounts/{account_id}/models",
+            url=f"{api_base}/v1/accounts/{account_id}/models",
             headers={"Authorization": f"Bearer {api_key}"},
         )
 

litellm/llms/openai/chat/gpt_5_transformation.py

@@ -25,22 +25,6 @@ def _normalize_reasoning_effort_for_chat_completion(
     return None
 
 
-def _get_effort_level(value: Union[str, dict, None]) -> Optional[str]:
-    """Extract the effective effort level from reasoning_effort (string or dict).
-
-    Use this for guards that compare effort level (e.g. xhigh validation, "none" checks).
-    Ensures dict inputs like {"effort": "none", "summary": "detailed"} are correctly
-    treated as effort="none" for validation purposes.
-    """
-    if value is None:
-        return None
-    if isinstance(value, str):
-        return value
-    if isinstance(value, dict) and "effort" in value:
-        return value["effort"]
-    return None
-
-
 class OpenAIGPT5Config(OpenAIGPTConfig):
     """Configuration for gpt-5 models including GPT-5-Codex variants.
 
@@ -86,19 +70,6 @@ class OpenAIGPT5Config(OpenAIGPTConfig):
         model_name = model.split("/")[-1]
         return model_name.startswith("gpt-5.4")
 
-    @classmethod
-    def is_model_gpt_5_4_plus_model(cls, model: str) -> bool:
-        """Check if the model is gpt-5.4 or newer (5.4, 5.5, 5.6, etc., including pro)."""
-        model_name = model.split("/")[-1]
-        if not model_name.startswith("gpt-5."):
-            return False
-        try:
-            version_str = model_name.replace("gpt-5.", "").split("-")[0]
-            major = version_str.split(".")[0]
-            return int(major) >= 4
-        except (ValueError, IndexError):
-            return False
-
     @classmethod
     def _supports_reasoning_effort_level(cls, model: str, level: str) -> bool:
         """Check if the model supports a specific reasoning_effort level.
@@ -179,32 +150,21 @@ class OpenAIGPT5Config(OpenAIGPTConfig):
                 drop_params=drop_params,
             )
 
-        # Get raw reasoning_effort and effective effort level for all guards.
-        # Use effective_effort (extracted string) for xhigh validation, "none" checks, and
-        # tool/sampling guards — dict inputs like {"effort": "none", "summary": "detailed"}
-        # must be treated as effort="none" to avoid incorrect tool-drop or sampling errors.
+        # Normalize reasoning_effort: chat completion API expects a string, not a dict
+        # (e.g. {'effort': 'high', 'summary': 'detailed'} -> 'high')
         raw_reasoning_effort = (
             non_default_params.get("reasoning_effort")
             or optional_params.get("reasoning_effort")
         )
-        effective_effort = _get_effort_level(raw_reasoning_effort)
+        normalized = _normalize_reasoning_effort_for_chat_completion(raw_reasoning_effort)
+        if raw_reasoning_effort is not None and normalized is not None:
+            if "reasoning_effort" in non_default_params:
+                non_default_params["reasoning_effort"] = normalized
+            if "reasoning_effort" in optional_params:
+                optional_params["reasoning_effort"] = normalized
 
-        # Normalize to string for Chat Completions API when dict has only "effort".
-        # Preserve full dict (e.g. {"effort": "high", "summary": "detailed"}) for Responses API.
-        if isinstance(raw_reasoning_effort, dict) and set(raw_reasoning_effort.keys()) <= {"effort"}:
-            normalized = _normalize_reasoning_effort_for_chat_completion(raw_reasoning_effort)
-            if normalized is not None:
-                if "reasoning_effort" in non_default_params:
-                    non_default_params["reasoning_effort"] = normalized
-                if "reasoning_effort" in optional_params:
-                    optional_params["reasoning_effort"] = normalized
-
-        reasoning_effort = (
-            non_default_params.get("reasoning_effort")
-            or optional_params.get("reasoning_effort")
-            or raw_reasoning_effort
-        )
-        if effective_effort is not None and effective_effort == "xhigh":
+        reasoning_effort = normalized or raw_reasoning_effort
+        if reasoning_effort is not None and reasoning_effort == "xhigh":
             if not self._supports_reasoning_effort_level(model, "xhigh"):
                 if litellm.drop_params or drop_params:
                     non_default_params.pop("reasoning_effort", None)
@@ -231,20 +191,17 @@ class OpenAIGPT5Config(OpenAIGPTConfig):
             has_tools = bool(
                 non_default_params.get("tools") or optional_params.get("tools")
             )
-            if has_tools and effective_effort not in (None, "none"):
-                # Check if this will be routed to Responses API
-                # If so, keep reasoning_effort; otherwise drop it for chat completions API
-                if not self.is_model_gpt_5_4_plus_model(model):
-                    non_default_params.pop("reasoning_effort", None)
-                    optional_params.pop("reasoning_effort", None)
-                    reasoning_effort = None
+            if has_tools and reasoning_effort not in (None, "none"):
+                non_default_params.pop("reasoning_effort", None)
+                optional_params.pop("reasoning_effort", None)
+                reasoning_effort = None
 
         # gpt-5.1/5.2 support logprobs, top_p, top_logprobs only when reasoning_effort="none"
         supports_none = self._supports_reasoning_effort_level(model, "none")
         if supports_none:
             sampling_params = ["logprobs", "top_logprobs", "top_p"]
             has_sampling = any(p in non_default_params for p in sampling_params)
-            if has_sampling and effective_effort not in (None, "none"):
+            if has_sampling and reasoning_effort not in (None, "none"):
                 if litellm.drop_params or drop_params:
                     for p in sampling_params:
                         non_default_params.pop(p, None)
@@ -254,7 +211,7 @@ class OpenAIGPT5Config(OpenAIGPTConfig):
                             "gpt-5.1/5.2/5.4 only support logprobs, top_p, top_logprobs when "
                             "reasoning_effort='none'. Current reasoning_effort='{}'. "
                             "To drop unsupported params set `litellm.drop_params = True`"
-                        ).format(effective_effort),
+                        ).format(reasoning_effort),
                         status_code=400,
                     )
 
@@ -262,7 +219,7 @@ class OpenAIGPT5Config(OpenAIGPTConfig):
             temperature_value: Optional[float] = non_default_params.pop("temperature")
             if temperature_value is not None:
                 # models supporting reasoning_effort="none" also support flexible temperature
-                if supports_none and (effective_effort == "none" or effective_effort is None):
+                if supports_none and (reasoning_effort == "none" or reasoning_effort is None):
                     optional_params["temperature"] = temperature_value
                 elif temperature_value == 1:
                     optional_params["temperature"] = temperature_value

litellm/llms/openai/image_edit/transformation.py

@@ -40,6 +40,7 @@ class OpenAIImageEditConfig(BaseImageEditConfig):
             "image",
             "prompt",
             "background",
+            "input_fidelity",
             "mask",
             "model",
             "n",

litellm/llms/openai_like/README.md

@@ -10,8 +10,9 @@ Instead of creating a full Python module for simple OpenAI-compatible providers,
 
 - `providers.json` - Configuration file for all JSON-based providers
 - `json_loader.py` - Loads and parses the JSON configuration
-- `dynamic_config.py` - Generates Python config classes from JSON
-- `chat/` - Existing OpenAI-like chat completion handlers
+- `dynamic_config.py` - Generates Python config classes from JSON (chat + responses)
+- `chat/` - OpenAI-like chat completion handlers
+- `responses/` - OpenAI-like Responses API handlers
 
 ## Adding a New Provider
 
@@ -96,6 +97,32 @@ response = litellm.completion(
 )
 ```
 
+## Responses API Support
+
+Providers that support the OpenAI Responses API (`/v1/responses`) can declare it via `supported_endpoints`:
+
+```json
+{
+  "your_provider": {
+    "base_url": "https://api.yourprovider.com/v1",
+    "api_key_env": "YOUR_PROVIDER_API_KEY",
+    "supported_endpoints": ["/v1/chat/completions", "/v1/responses"]
+  }
+}
+```
+
+This enables `litellm.responses(model="your_provider/model-name", ...)` with zero Python code.
+The provider inherits all request/response handling from OpenAI's Responses API config.
+
+If `supported_endpoints` is omitted, it defaults to `[]` (only chat completions, which is always enabled for JSON providers).
+
+### How It Works
+
+1. `json_loader.py` checks `supported_endpoints` for `/v1/responses`
+2. `dynamic_config.py` generates a responses config class (inherits from `OpenAIResponsesAPIConfig`)
+3. `ProviderConfigManager.get_provider_responses_api_config()` returns the generated config
+4. Request/response transformation is inherited from OpenAI — no custom code needed
+
 ## Benefits
 
 - **Simple**: 2-5 lines of JSON vs 100+ lines of Python
@@ -112,6 +139,10 @@ Use a Python config class if you need:
 - Provider-specific streaming logic
 - Advanced tool calling transformations
 
+For providers that are *mostly* OpenAI-compatible but need small overrides (e.g. preset model handling),
+you can inherit from `OpenAIResponsesAPIConfig` and override only what's needed — see
+`litellm/llms/perplexity/responses/transformation.py` for a minimal example (~40 lines).
+
 ## Implementation Details
 
 ### How It Works
@@ -125,5 +156,6 @@ Use a Python config class if you need:
 
 The JSON system is integrated at:
 - `litellm/litellm_core_utils/get_llm_provider_logic.py` - Provider resolution
-- `litellm/utils.py` - ProviderConfigManager
+- `litellm/utils.py` - ProviderConfigManager (chat + responses)
+- `litellm/responses/main.py` - Responses API routing
 - `litellm/constants.py` - openai_compatible_providers list

litellm/llms/openai_like/dynamic_config.py

@@ -166,3 +166,63 @@ def create_config_class(provider: SimpleProviderConfig):
             return provider.slug
 
     return JSONProviderConfig
+
+
+_responses_config_cache: dict = {}
+
+
+def create_responses_config_class(provider: SimpleProviderConfig):
+    """Generate a Responses API config class dynamically from JSON configuration.
+
+    Parallel to create_config_class() but for /v1/responses endpoints.
+    Classes are cached per provider slug to avoid regeneration on every request.
+    """
+    if provider.slug in _responses_config_cache:
+        return _responses_config_cache[provider.slug]
+
+    from litellm.llms.openai_like.responses.transformation import (
+        OpenAILikeResponsesConfig,
+    )
+    from litellm.types.router import GenericLiteLLMParams
+
+    class JSONProviderResponsesConfig(OpenAILikeResponsesConfig):
+        @property
+        def custom_llm_provider(self):  # type: ignore[override]
+            return provider.slug
+
+        def validate_environment(
+            self,
+            headers: dict,
+            model: str,
+            litellm_params: Optional[GenericLiteLLMParams],
+        ) -> dict:
+            litellm_params = litellm_params or GenericLiteLLMParams()
+            api_key = (
+                litellm_params.api_key
+                or get_secret_str(provider.api_key_env)
+            )
+            if api_key:
+                headers["Authorization"] = f"Bearer {api_key}"
+            return headers
+
+        def get_complete_url(
+            self,
+            api_base: Optional[str],
+            litellm_params: dict,
+        ) -> str:
+            if not api_base:
+                if provider.api_base_env:
+                    api_base = get_secret_str(provider.api_base_env)
+                if not api_base:
+                    api_base = provider.base_url
+
+            if api_base is None:
+                raise ValueError(
+                    f"api_base is required for provider {provider.slug}"
+                )
+
+            api_base = api_base.rstrip("/")
+            return f"{api_base}/responses"
+
+    _responses_config_cache[provider.slug] = JSONProviderResponsesConfig
+    return JSONProviderResponsesConfig

litellm/llms/openai_like/json_loader.py

@@ -21,6 +21,7 @@ class SimpleProviderConfig:
         self.param_mappings = data.get("param_mappings", {})
         self.constraints = data.get("constraints", {})
         self.special_handling = data.get("special_handling", {})
+        self.supported_endpoints = data.get("supported_endpoints", [])
 
 
 class JSONProviderRegistry:
@@ -64,6 +65,14 @@ class JSONProviderRegistry:
         """Check if a provider is defined via JSON"""
         return slug in cls._providers
 
+    @classmethod
+    def supports_responses_api(cls, slug: str) -> bool:
+        """Check if a JSON provider supports the Responses API"""
+        provider = cls._providers.get(slug)
+        if provider is None:
+            return False
+        return "/v1/responses" in provider.supported_endpoints
+
     @classmethod
     def list_providers(cls) -> list:
         """List all registered provider slugs"""

litellm/llms/openai_like/responses/__init__.py

@@ -0,0 +1,5 @@
+from litellm.llms.openai_like.responses.transformation import (
+    OpenAILikeResponsesConfig,
+)
+
+__all__ = ["OpenAILikeResponsesConfig"]

litellm/llms/openai_like/responses/transformation.py

@@ -0,0 +1,51 @@
+"""
+OpenAI-like Responses API transformation.
+
+Base class for JSON-declared providers that support the /v1/responses endpoint.
+Inherits everything from OpenAIResponsesAPIConfig; subclasses only override
+provider-specific resolution (slug, API key env var, base URL).
+"""
+
+from typing import Optional, Union
+
+from litellm.llms.openai.responses.transformation import OpenAIResponsesAPIConfig
+from litellm.secret_managers.main import get_secret_str
+from litellm.types.router import GenericLiteLLMParams
+from litellm.types.utils import LlmProviders
+
+
+class OpenAILikeResponsesConfig(OpenAIResponsesAPIConfig):
+    """
+    Responses API config for OpenAI-compatible providers declared via JSON.
+
+    Concrete per-provider classes are generated dynamically in dynamic_config.py.
+    This base provides the three overridable hooks that the dynamic generator
+    fills in: custom_llm_provider, validate_environment, get_complete_url.
+    """
+
+    @property
+    def custom_llm_provider(self) -> Union[str, LlmProviders]:  # type: ignore[override]
+        return "openai_like"
+
+    def validate_environment(
+        self,
+        headers: dict,
+        model: str,
+        litellm_params: Optional[GenericLiteLLMParams],
+    ) -> dict:
+        litellm_params = litellm_params or GenericLiteLLMParams()
+        api_key = litellm_params.api_key or get_secret_str("OPENAI_LIKE_API_KEY")
+        if api_key:
+            headers["Authorization"] = f"Bearer {api_key}"
+        return headers
+
+    def get_complete_url(
+        self,
+        api_base: Optional[str],
+        litellm_params: dict,
+    ) -> str:
+        api_base = api_base or get_secret_str("OPENAI_LIKE_API_BASE")
+        if not api_base:
+            raise ValueError("api_base is required for openai_like provider")
+        api_base = api_base.rstrip("/")
+        return f"{api_base}/responses"

litellm/llms/perplexity/responses/transformation.py

@@ -1,54 +1,31 @@
 """
-Transformation logic for Perplexity Agent API (Responses API)
+Perplexity Responses API — OpenAI-compatible.
 
-This module handles the translation between OpenAI's Responses API format
-and Perplexity's Responses API format, which supports:
-- Third-party model access (OpenAI, Anthropic, Google, xAI, etc.)
-- Presets for optimized configurations
-- Web search and URL fetching tools
-- Reasoning effort control
-- Instructions parameter for system-level guidance
+The only provider quirks:
+- cost returned as dict → handled by ResponseAPIUsage.parse_cost validator
+- preset models (preset/pro-search) → handled by transform_responses_api_request
+- HTTP 200 with status:"failed" → raised as exception in transform_response_api_response
+
+Ref: https://docs.perplexity.ai/api-reference/responses-post
 """
 
 from typing import Any, Dict, List, Optional, Union
 
 import httpx
 
-from litellm._logging import verbose_logger
 from litellm.litellm_core_utils.litellm_logging import Logging as LiteLLMLoggingObj
 from litellm.llms.base_llm.chat.transformation import BaseLLMException
 from litellm.llms.openai.responses.transformation import OpenAIResponsesAPIConfig
 from litellm.secret_managers.main import get_secret_str
-from litellm.types.llms.openai import (
-    ResponseAPIUsage,
-    ResponseInputParam,
-    ResponsesAPIOptionalRequestParams,
-    ResponsesAPIResponse,
-    ResponsesAPIStreamingResponse,
-)
+from litellm.types.llms.openai import ResponseInputParam, ResponsesAPIResponse
 from litellm.types.router import GenericLiteLLMParams
 from litellm.types.utils import LlmProviders
 
 
 class PerplexityResponsesConfig(OpenAIResponsesAPIConfig):
-    """
-    Configuration for Perplexity Agent API (Responses API)
-
-
-    Reference: https://docs.perplexity.ai/docs/agent-api/overview
-    """
-
-    @property
-    def custom_llm_provider(self) -> LlmProviders:
-        return LlmProviders.PERPLEXITY
 
     def get_supported_openai_params(self, model: str) -> list:
-        """
-        Perplexity Responses API supports a different set of parameters
-
-        Ref: https://docs.perplexity.ai/api-reference/responses-post
-        Params aligned with response-echo fields and Open Responses spec.
-        """
+        """Ref: https://docs.perplexity.ai/api-reference/responses-post"""
         return [
             "max_output_tokens",
             "stream",
@@ -56,200 +33,45 @@ class PerplexityResponsesConfig(OpenAIResponsesAPIConfig):
             "top_p",
             "tools",
             "reasoning",
-            "preset",
             "instructions",
-            "models",  # Model fallback support
-            "tool_choice",
-            "parallel_tool_calls",
-            "max_tool_calls",
-            "text",
-            "previous_response_id",
-            "store",
-            "background",
-            "truncation",
-            "metadata",
-            "safety_identifier",
-            "user",
-            "stream_options",
-            "top_logprobs",
-            "prompt_cache_key",
-            "frequency_penalty",
-            "presence_penalty",
-            "service_tier",
+            "models",
         ]
 
+    @property
+    def custom_llm_provider(self) -> LlmProviders:
+        return LlmProviders.PERPLEXITY
+
     def validate_environment(
         self, headers: dict, model: str, litellm_params: Optional[GenericLiteLLMParams]
     ) -> dict:
-        """Validate environment and set up headers"""
-        # Get API key from environment
-        api_key = get_secret_str("PERPLEXITYAI_API_KEY") or get_secret_str(
-            "PERPLEXITY_API_KEY"
+        litellm_params = litellm_params or GenericLiteLLMParams()
+        api_key = (
+            litellm_params.api_key
+            or get_secret_str("PERPLEXITYAI_API_KEY")
+            or get_secret_str("PERPLEXITY_API_KEY")
         )
-
         if api_key:
             headers["Authorization"] = f"Bearer {api_key}"
-
-        headers["Content-Type"] = "application/json"
-
         return headers
 
-    def get_complete_url(
-        self,
-        api_base: Optional[str],
-        litellm_params: dict,
-    ) -> str:
-        """Get the complete URL for the Perplexity Responses API"""
-        if api_base is None:
-            api_base = (
-                get_secret_str("PERPLEXITY_API_BASE") or "https://api.perplexity.ai"
-            )
+    def get_complete_url(self, api_base: Optional[str], litellm_params: dict) -> str:
+        api_base = api_base or get_secret_str("PERPLEXITY_API_BASE") or "https://api.perplexity.ai"
+        return f"{api_base.rstrip('/')}/v1/responses"
 
-        # Ensure api_base doesn't end with a slash
-        api_base = api_base.rstrip("/")
-
-        # Add the responses endpoint
-        return f"{api_base}/v1/responses"
-
-    def map_openai_params(  # noqa: PLR0915
-        self,
-        response_api_optional_params: ResponsesAPIOptionalRequestParams,
-        model: str,
-        drop_params: bool,
-    ) -> Dict:
-        """
-        Map OpenAI Responses API parameters to Perplexity format
-
-        Key differences:
-        - Supports 'preset' parameter for predefined configurations
-        - Supports 'instructions' parameter for system-level guidance
-        - Tools are specified differently (web_search, fetch_url)
-        """
-        mapped_params: Dict[str, Any] = {}
-
-        # Map standard parameters
-        if response_api_optional_params.get("max_output_tokens"):
-            mapped_params["max_output_tokens"] = response_api_optional_params[
-                "max_output_tokens"
-            ]
-
-        if response_api_optional_params.get("temperature"):
-            mapped_params["temperature"] = response_api_optional_params["temperature"]
-
-        if response_api_optional_params.get("top_p"):
-            mapped_params["top_p"] = response_api_optional_params["top_p"]
-
-        if response_api_optional_params.get("stream"):
-            mapped_params["stream"] = response_api_optional_params["stream"]
-
-        if response_api_optional_params.get("stream_options"):
-            mapped_params["stream_options"] = response_api_optional_params[
-                "stream_options"
-            ]
-
-        # Map Perplexity-specific parameters (using .get() with Any dict access)
-        preset = response_api_optional_params.get("preset")  # type: ignore
-        if preset:
-            mapped_params["preset"] = preset
-
-        instructions = response_api_optional_params.get("instructions")  # type: ignore
-        if instructions:
-            mapped_params["instructions"] = instructions
-
-        if response_api_optional_params.get("reasoning"):
-            mapped_params["reasoning"] = response_api_optional_params["reasoning"]
-
-        tools = response_api_optional_params.get("tools")
-        if tools:
-            # Convert tools to list of dicts for transformation
-            tools_list = [dict(tool) if hasattr(tool, "__dict__") else tool for tool in tools]  # type: ignore
-            mapped_params["tools"] = self._transform_tools(tools_list)  # type: ignore
-
-        # Tool control
-        if response_api_optional_params.get("tool_choice"):
-            mapped_params["tool_choice"] = response_api_optional_params["tool_choice"]
-        if response_api_optional_params.get("parallel_tool_calls") is not None:
-            mapped_params["parallel_tool_calls"] = response_api_optional_params[
-                "parallel_tool_calls"
-            ]
-        if response_api_optional_params.get("max_tool_calls"):
-            mapped_params["max_tool_calls"] = response_api_optional_params[
-                "max_tool_calls"
-            ]
-
-        # Structured outputs
-        text_param = response_api_optional_params.get("text")
-        if text_param:
-            mapped_params["text"] = text_param
-
-        # Conversation continuity
-        if response_api_optional_params.get("previous_response_id"):
-            mapped_params["previous_response_id"] = response_api_optional_params[
-                "previous_response_id"
-            ]
-
-        # Storage and lifecycle
-        if response_api_optional_params.get("store") is not None:
-            mapped_params["store"] = response_api_optional_params["store"]
-        if response_api_optional_params.get("background") is not None:
-            mapped_params["background"] = response_api_optional_params["background"]
-        if response_api_optional_params.get("truncation"):
-            mapped_params["truncation"] = response_api_optional_params["truncation"]
-
-        # Metadata
-        if response_api_optional_params.get("metadata"):
-            mapped_params["metadata"] = response_api_optional_params["metadata"]
-        if response_api_optional_params.get("safety_identifier"):
-            mapped_params["safety_identifier"] = response_api_optional_params[
-                "safety_identifier"
-            ]
-        if response_api_optional_params.get("user"):
-            mapped_params["user"] = response_api_optional_params["user"]
-
-        # Additional
-        if response_api_optional_params.get("top_logprobs") is not None:
-            mapped_params["top_logprobs"] = response_api_optional_params["top_logprobs"]
-        if response_api_optional_params.get("prompt_cache_key"):
-            mapped_params["prompt_cache_key"] = response_api_optional_params[
-                "prompt_cache_key"
-            ]
-        if response_api_optional_params.get("frequency_penalty") is not None:
-            mapped_params["frequency_penalty"] = response_api_optional_params[
-                "frequency_penalty"  # type: ignore[typeddict-item]
-            ]
-        if response_api_optional_params.get("presence_penalty") is not None:
-            mapped_params["presence_penalty"] = response_api_optional_params[
-                "presence_penalty"  # type: ignore[typeddict-item]
-            ]
-        if response_api_optional_params.get("service_tier"):
-            mapped_params["service_tier"] = response_api_optional_params["service_tier"]
-
-        return mapped_params
-
-    def _transform_tools(self, tools: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
-        """
-        Transform tools to Perplexity format.
-
-        Perplexity supports (per public OpenAPI spec):
-        - web_search: Performs web searches
-        - fetch_url: Fetches content from URLs
-        - function: Function Calling
-        """
-        perplexity_tools = []
-
-        for tool in tools:
-            if isinstance(tool, dict):
-                tool_type = tool.get("type", "")
-
-                # Direct Perplexity tool format
-                if tool_type in ["web_search", "fetch_url"]:
-                    perplexity_tools.append(tool)
-
-                # Function tools: Perplexity supports them natively
-                elif tool_type == "function":
-                    perplexity_tools.append(tool)
-
-        return perplexity_tools
+    def _ensure_message_type(
+        self, input: Union[str, ResponseInputParam]
+    ) -> Union[str, List[Dict[str, Any]]]:
+        """Ensure list input items have type='message' (required by Perplexity)."""
+        if isinstance(input, str):
+            return input
+        if isinstance(input, list):
+            result = []
+            for item in input:
+                if isinstance(item, dict) and "type" not in item:
+                    item = {**item, "type": "message"}
+                result.append(item)
+            return result
+        return input
 
     def transform_responses_api_request(
         self,
@@ -259,62 +81,23 @@ class PerplexityResponsesConfig(OpenAIResponsesAPIConfig):
         litellm_params: GenericLiteLLMParams,
         headers: dict,
     ) -> Dict:
-        """
-        Transform request to Perplexity Responses API format
-        """
-        # Check if the model is a preset (format: preset/preset-name)
+        """Handle preset/ model prefix: send as {"preset": name} instead of {"model": name}."""
+        input = self._ensure_message_type(input)
         if model.startswith("preset/"):
-            preset_name = model.replace("preset/", "")
-            data = {
-                "preset": preset_name,
-                "input": self._format_input(input),
+            input = self._validate_input_param(input)
+            data: Dict = {
+                "preset": model[len("preset/"):],
+                "input": input,
             }
-        # Check if preset is explicitly provided in params
-        elif response_api_optional_request_params.get("preset"):
-            data = {
-                "preset": response_api_optional_request_params.pop("preset"),
-                "input": self._format_input(input),
-            }
-        else:
-            # Full request format for third-party models
-            data = {
-                "model": model,
-                "input": self._format_input(input),
-            }
-
-        # Add all optional parameters
-        for key, value in response_api_optional_request_params.items():
-            data[key] = value
-
-        return data
-
-    def _format_input(
-        self, input: Union[str, ResponseInputParam]
-    ) -> Union[str, List[Dict[str, Any]]]:
-        """
-        Format input for Perplexity Responses API
-
-        The API accepts either:
-        - A simple string for single-turn queries
-        - An array of message objects for multi-turn conversations
-        """
-        if isinstance(input, str):
-            return input
-
-        # Handle ResponseInputParam format
-        if isinstance(input, list):
-            formatted_messages = []
-            for item in input:
-                if isinstance(item, dict):
-                    formatted_message = {
-                        "type": "message",
-                        "role": item.get("role"),
-                        "content": item.get("content", ""),
-                    }
-                    formatted_messages.append(formatted_message)
-            return formatted_messages
-
-        return str(input)
+            data.update(response_api_optional_request_params)
+            return data
+        return super().transform_responses_api_request(
+            model=model,
+            input=input,
+            response_api_optional_request_params=response_api_optional_request_params,
+            litellm_params=litellm_params,
+            headers=headers,
+        )
 
     def transform_response_api_response(
         self,
@@ -322,174 +105,27 @@ class PerplexityResponsesConfig(OpenAIResponsesAPIConfig):
         raw_response: httpx.Response,
         logging_obj: LiteLLMLoggingObj,
     ) -> ResponsesAPIResponse:
-        """
-        Transform Perplexity Responses API response to OpenAI Responses API format
-        """
+        """Check for Perplexity's status:'failed' on HTTP 200 before delegating to base."""
         try:
             raw_response_json = raw_response.json()
-        except Exception as e:
-            raise BaseLLMException(
-                status_code=raw_response.status_code,
-                message=f"Failed to parse response: {str(e)}",
-            )
-
-        # Check for error status
-        status = raw_response_json.get("status")
-        if status == "failed":
-            error = raw_response_json.get("error", {})
-            error_message = error.get("message", "Unknown error")
-            raise BaseLLMException(
-                status_code=raw_response.status_code,
-                message=error_message,
-            )
-
-        # Transform usage to handle Perplexity's cost structure
-        usage_data = raw_response_json.get("usage", {})
-        transformed_usage_dict = self._transform_usage(usage_data)
-
-        # Convert usage dict to ResponseAPIUsage object
-        usage_obj = (
-            ResponseAPIUsage(**transformed_usage_dict)
-            if transformed_usage_dict
-            else None
-        )
-
-        # Map Perplexity response to OpenAI Responses API format
-        response = ResponsesAPIResponse(
-            id=raw_response_json.get("id", ""),
-            object="response",
-            created_at=raw_response_json.get("created_at", 0),
-            status=raw_response_json.get("status", "completed"),
-            model=raw_response_json.get("model", model),
-            output=raw_response_json.get("output", []),
-            usage=usage_obj,
-        )
-
-        return response
-
-    def _transform_usage(self, usage_data: Dict[str, Any]) -> Dict[str, Any]:
-        """
-        Transform Perplexity usage data to OpenAI format
-
-        Perplexity returns:
-        {
-            "input_tokens": 100,
-            "output_tokens": 200,
-            "total_tokens": 300,
-            "cost": {
-                "currency": "USD",
-                "input_cost": 0.0001,
-                "output_cost": 0.0002,
-                "total_cost": 0.0003
-            }
-        }
-
-        OpenAI expects:
-        {
-            "input_tokens": 100,
-            "output_tokens": 200,
-            "total_tokens": 300,
-            "cost": 0.0003
-        }
-        """
-        transformed = {
-            "input_tokens": usage_data.get("input_tokens", 0),
-            "output_tokens": usage_data.get("output_tokens", 0),
-            "total_tokens": usage_data.get("total_tokens", 0),
-        }
-
-        # Transform cost from Perplexity format (dict) to OpenAI format (float)
-        cost_obj = usage_data.get("cost")
-        if isinstance(cost_obj, dict) and "total_cost" in cost_obj:
-            transformed["cost"] = cost_obj["total_cost"]
-            verbose_logger.debug(
-                "Transformed Perplexity cost object to float: %s -> %s",
-                cost_obj,
-                cost_obj["total_cost"],
-            )
-        elif cost_obj is not None:
-            # If cost is already a float/number, use it as-is
-            transformed["cost"] = cost_obj
-
-        # Add input_tokens_details if present
-        if "input_tokens_details" in usage_data:
-            transformed["input_tokens_details"] = usage_data["input_tokens_details"]
-
-        # Add output_tokens_details if present
-        if "output_tokens_details" in usage_data:
-            transformed["output_tokens_details"] = usage_data["output_tokens_details"]
-
-        return transformed
-
-    def transform_streaming_response(
-        self,
-        model: str,
-        parsed_chunk: dict,
-        logging_obj: LiteLLMLoggingObj,
-    ) -> ResponsesAPIStreamingResponse:
-        """
-        Transform a parsed streaming response chunk into a ResponsesAPIStreamingResponse
-        """
-        # Get the event type from the chunk
-        verbose_logger.debug("Raw Perplexity Chunk=%s", parsed_chunk)
-        event_type = str(parsed_chunk.get("type"))
-        event_pydantic_model = PerplexityResponsesConfig.get_event_model_class(
-            event_type=event_type
-        )
-
-        # Transform Perplexity-specific fields to OpenAI format
-        parsed_chunk = self._transform_perplexity_chunk(parsed_chunk)
-
-        # Defensive: Handle error.code being null (similar to OpenAI implementation)
-        try:
-            error_obj = parsed_chunk.get("error")
-            if isinstance(error_obj, dict) and error_obj.get("code") is None:
-                # Preserve other fields, but ensure `code` is a non-null string
-                parsed_chunk = dict(parsed_chunk)
-                parsed_chunk["error"] = dict(error_obj)
-                parsed_chunk["error"]["code"] = "unknown_error"
         except Exception:
-            # If anything unexpected happens here, fall back to attempting
-            # instantiation and let higher-level handlers manage errors.
-            verbose_logger.debug("Failed to coalesce error.code in parsed_chunk")
+            raw_response_json = None
 
-        return event_pydantic_model(**parsed_chunk)
+        if (
+            isinstance(raw_response_json, dict)
+            and raw_response_json.get("status") == "failed"
+        ):
+            error = raw_response_json.get("error", {})
+            raise BaseLLMException(
+                status_code=raw_response.status_code,
+                message=error.get("message", "Unknown Perplexity error"),
+            )
 
-    def _transform_perplexity_chunk(self, chunk: dict) -> dict:
-        """
-        Transform Perplexity-specific fields in a streaming chunk to OpenAI format.
-
-        This handles:
-        - Converting Perplexity's cost object to a simple float
-        """
-        # Make a copy to avoid modifying the original
-        chunk = dict(chunk)
-
-        # Transform usage.cost from Perplexity format to OpenAI format
-        # Perplexity: {"currency": "USD", "input_cost": 0.0001, "output_cost": 0.0002, "total_cost": 0.0003}
-        # OpenAI: 0.0003 (just the total_cost as a float)
-        try:
-            response_obj = chunk.get("response")
-            if isinstance(response_obj, dict):
-                usage_obj = response_obj.get("usage")
-                if isinstance(usage_obj, dict):
-                    cost_obj = usage_obj.get("cost")
-                    if isinstance(cost_obj, dict) and "total_cost" in cost_obj:
-                        # Replace the cost object with just the total_cost value
-                        chunk = dict(chunk)
-                        chunk["response"] = dict(response_obj)
-                        chunk["response"]["usage"] = dict(usage_obj)
-                        chunk["response"]["usage"]["cost"] = cost_obj["total_cost"]
-                        verbose_logger.debug(
-                            "Transformed Perplexity cost object to float: %s -> %s",
-                            cost_obj,
-                            cost_obj["total_cost"],
-                        )
-        except Exception as e:
-            # If transformation fails, log and continue with original chunk
-            verbose_logger.debug("Failed to transform Perplexity cost object: %s", e)
-
-        return chunk
+        return super().transform_response_api_response(
+            model=model,
+            raw_response=raw_response,
+            logging_obj=logging_obj,
+        )
 
     def supports_native_websocket(self) -> bool:
         """Perplexity does not support native WebSocket for Responses API"""

litellm/llms/sagemaker/completion/handler.py

@@ -583,17 +583,35 @@ class SagemakerLLM(BaseAWSLLM):
         ### BOTO3 INIT
         import boto3
 
-        # Use _load_credentials to support role assumption (aws_role_name, aws_session_name)
-        credentials, aws_region_name = self._load_credentials(optional_params)
+        # pop aws_secret_access_key, aws_access_key_id, aws_region_name from kwargs, since completion calls fail with them
+        aws_secret_access_key = optional_params.pop("aws_secret_access_key", None)
+        aws_access_key_id = optional_params.pop("aws_access_key_id", None)
+        aws_region_name = optional_params.pop("aws_region_name", None)
 
-        # Create boto3 session with the loaded credentials
-        session = boto3.Session(
-            aws_access_key_id=credentials.access_key,
-            aws_secret_access_key=credentials.secret_key,
-            aws_session_token=credentials.token,
-            region_name=aws_region_name,
-        )
-        client = session.client(service_name="sagemaker-runtime")
+        if aws_access_key_id is not None:
+            # uses auth params passed to completion
+            # aws_access_key_id is not None, assume user is trying to auth using litellm.completion
+            client = boto3.client(
+                service_name="sagemaker-runtime",
+                aws_access_key_id=aws_access_key_id,
+                aws_secret_access_key=aws_secret_access_key,
+                region_name=aws_region_name,
+            )
+        else:
+            # aws_access_key_id is None, assume user is trying to auth using env variables
+            # boto3 automaticaly reads env variables
+
+            # we need to read region name from env
+            # I assume majority of users use .env for auth
+            region_name = (
+                get_secret("AWS_REGION_NAME")
+                or aws_region_name  # get region from config file if specified
+                or "us-west-2"  # default to us-west-2 if region not specified
+            )
+            client = boto3.client(
+                service_name="sagemaker-runtime",
+                region_name=region_name,
+            )
 
         # pop streaming if it's in the optional params as 'stream' raises an error with sagemaker
         inference_params = deepcopy(optional_params)
@@ -610,9 +628,7 @@ class SagemakerLLM(BaseAWSLLM):
         #### EMBEDDING LOGIC
         # Transform request based on model type
         provider_config = SagemakerEmbeddingConfig.get_model_config(model)
-        request_data = provider_config.transform_embedding_request(
-            model, input, optional_params, {}
-        )
+        request_data = provider_config.transform_embedding_request(model, input, optional_params, {})
         data = json.dumps(request_data).encode("utf-8")
 
         ## LOGGING
@@ -657,19 +673,19 @@ class SagemakerLLM(BaseAWSLLM):
         )
 
         print_verbose(f"raw model_response: {response}")
-
+        
         # Transform response based on model type
         from httpx import Response as HttpxResponse
-
+        
         # Create a mock httpx Response object for the transformation
         mock_response = HttpxResponse(
             status_code=200,
-            content=json.dumps(response).encode("utf-8"),
-            headers={"content-type": "application/json"},
+            content=json.dumps(response).encode('utf-8'),
+            headers={"content-type": "application/json"}
         )
-
+        
         model_response = EmbeddingResponse()
-
+        
         # Use the request_data that was already transformed above
         return provider_config.transform_embedding_response(
             model=model,
@@ -679,5 +695,5 @@ class SagemakerLLM(BaseAWSLLM):
             api_key=None,
             request_data=request_data,
             optional_params=optional_params,
-            litellm_params=litellm_params or {},
+            litellm_params=litellm_params or {}
         )

litellm/llms/snowflake/chat/transformation.py

@@ -208,27 +208,28 @@ class SnowflakeConfig(SnowflakeBaseConfig, OpenAIGPTConfig):
 
     def _transform_tool_choice(
         self, tool_choice: Union[str, Dict[str, Any]]
-    ) -> Union[str, Dict[str, Any]]:
+    ) -> Dict[str, Any]:
         """
         Transform OpenAI tool_choice format to Snowflake format.
 
+        Snowflake requires tool_choice to be an object, not a string.
+        Ref: https://docs.snowflake.com/en/developer-guide/snowflake-rest-api/reference/cortex-inference#post--api-v2-cortex-inference-complete-req-body-schema
+
         Args:
             tool_choice: Tool choice in OpenAI format (str or dict)
 
         Returns:
-            Tool choice in Snowflake format
+            Tool choice in Snowflake format (always an object)
 
-        OpenAI format:
-        {"type": "function", "function": {"name": "get_weather"}}
+        OpenAI format (string): "auto", "required", "none"
+        OpenAI format (object): {"type": "function", "function": {"name": "get_weather"}}
 
-        Snowflake format:
-        {"type": "tool", "name": ["get_weather"]}
-
-        Note: String values ("auto", "required", "none") pass through unchanged.
+        Snowflake format (string values become objects): {"type": "auto"}
+        Snowflake format (specific tool): {"type": "tool", "name": ["get_weather"]}
         """
         if isinstance(tool_choice, str):
-            # "auto", "required", "none" pass through as-is
-            return tool_choice
+            # Snowflake requires object format: {"type": "auto"} not string "auto"
+            return {"type": tool_choice}
 
         if isinstance(tool_choice, dict):
             if tool_choice.get("type") == "function":

litellm/llms/vertex_ai/common_utils.py

@@ -516,6 +516,29 @@ def _build_vertex_schema(parameters: dict, add_property_ordering: bool = False):
     return parameters
 
 
+def _build_vertex_schema_for_gemini_2(parameters: dict) -> dict:
+    """
+    Minimal schema builder for Gemini 2.0+ tool parameters.
+
+    Gemini 2.0+ accepts standard JSON Schema natively in tool parameters,
+    including lowercase types, anyOf with null, and bare {} (TYPE_UNSPECIFIED).
+    The only transformation needed is resolving $ref/$defs, which Gemini does
+    NOT support in tool parameters (returns 400).
+
+    This avoids the harmful transforms in _build_vertex_schema that break
+    JsonValue/Any semantics by coercing {} to {"type": "object"}.
+    """
+    valid_schema_fields = set(get_type_hints(Schema).keys())
+
+    parameters = dict(parameters)  # shallow copy to avoid mutating caller's dict
+    defs = parameters.pop("$defs", {})
+    unpack_defs(parameters, defs)
+
+    parameters = filter_schema_fields(parameters, valid_schema_fields)
+
+    return parameters
+
+
 def _build_json_schema(parameters: dict) -> dict:
     """
     Build a JSON Schema for use with Gemini's responseJsonSchema parameter.

litellm/llms/vertex_ai/gemini/transformation.py

@@ -583,18 +583,12 @@ def _gemini_convert_messages_with_history(  # noqa: PLR0915
         raise e
 
 
-# Keys that LiteLLM consumes internally and must never be forwarded to the
-_LITELLM_INTERNAL_EXTRA_BODY_KEYS: frozenset = frozenset({"cache", "tags"})
-
-
 def _pop_and_merge_extra_body(data: RequestBody, optional_params: dict) -> None:
     """Pop extra_body from optional_params and shallow-merge into data, deep-merging dict values."""
     extra_body: Optional[dict] = optional_params.pop("extra_body", None)
     if extra_body is not None:
         data_dict: dict = data  # type: ignore[assignment]
         for k, v in extra_body.items():
-            if k in _LITELLM_INTERNAL_EXTRA_BODY_KEYS:
-                continue
             if k in data_dict and isinstance(data_dict[k], dict) and isinstance(v, dict):
                 data_dict[k].update(v)
             else:

litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py

@@ -97,6 +97,7 @@ from ..common_utils import (
     VertexAIError,
     _build_json_schema,
     _build_vertex_schema,
+    _build_vertex_schema_for_gemini_2,
     supports_response_json_schema,
 )
 from ..vertex_llm_base import VertexBase
@@ -467,7 +468,7 @@ class VertexGeminiConfig(VertexAIBaseConfig, BaseConfig):
             return None
 
     def _map_function(  # noqa: PLR0915
-        self, value: List[dict], optional_params: dict
+        self, value: List[dict], optional_params: dict, model: str = ""
     ) -> List[Tools]:
         """
         Map OpenAI-style tools/functions to Vertex AI format.
@@ -510,10 +511,21 @@ class VertexGeminiConfig(VertexAIBaseConfig, BaseConfig):
                     "parameters" in _openai_function_object
                     and _openai_function_object["parameters"] is not None
                     and isinstance(_openai_function_object["parameters"], dict)
-                ):  # OPENAI accepts JSON Schema, Google accepts OpenAPI schema.
-                    _openai_function_object["parameters"] = _build_vertex_schema(
-                        _openai_function_object["parameters"]
-                    )
+                ):
+                    if supports_response_json_schema(model):
+                        # Gemini 2.0+: minimal transform (resolve $ref only)
+                        _openai_function_object["parameters"] = (
+                            _build_vertex_schema_for_gemini_2(
+                                _openai_function_object["parameters"]
+                            )
+                        )
+                    else:
+                        # Gemini 1.5: full OpenAPI-style transform
+                        _openai_function_object["parameters"] = (
+                            _build_vertex_schema(
+                                _openai_function_object["parameters"]
+                            )
+                        )
 
                 openai_function_object = _openai_function_object
 
@@ -1051,7 +1063,7 @@ class VertexGeminiConfig(VertexAIBaseConfig, BaseConfig):
             ):
                 # Pass optional_params so _map_function can add toolConfig if needed
                 mapped_tools = self._map_function(
-                    value=value, optional_params=optional_params
+                    value=value, optional_params=optional_params, model=model
                 )
                 optional_params = self._add_tools_to_optional_params(
                     optional_params, mapped_tools
@@ -1230,27 +1242,25 @@ class VertexGeminiConfig(VertexAIBaseConfig, BaseConfig):
             "IMAGE_PROHIBITED_CONTENT": "The token generation was stopped as the response was flagged for prohibited image content.",
         }
 
+    _GEMINI_FINISH_REASON_KEYS = frozenset({
+        "STOP", "MAX_TOKENS", "SAFETY", "RECITATION", "FINISH_REASON_UNSPECIFIED",
+        "MALFORMED_FUNCTION_CALL", "LANGUAGE", "OTHER", "BLOCKLIST",
+        "PROHIBITED_CONTENT", "SPII", "IMAGE_SAFETY", "IMAGE_PROHIBITED_CONTENT",
+        "TOO_MANY_TOOL_CALLS", "MALFORMED_RESPONSE",
+    })
+
     @staticmethod
     def get_finish_reason_mapping() -> Dict[str, OpenAIChatCompletionFinishReason]:
         """
-        Return Dictionary of finish reasons which indicate response was flagged
-
-        and what it means
+        Return Dictionary of Gemini/Vertex AI finish reasons and their
+        OpenAI-compatible mappings.
         """
+        from litellm.litellm_core_utils.core_helpers import _FINISH_REASON_MAP
+
         return {
-            "FINISH_REASON_UNSPECIFIED": "finish_reason_unspecified",
-            "STOP": "stop",
-            "MAX_TOKENS": "length",
-            "SAFETY": "content_filter",
-            "RECITATION": "content_filter",
-            "LANGUAGE": "content_filter",
-            "OTHER": "content_filter",
-            "BLOCKLIST": "content_filter",
-            "PROHIBITED_CONTENT": "content_filter",
-            "SPII": "content_filter",
-            "MALFORMED_FUNCTION_CALL": "malformed_function_call",  # openai doesn't have a way of representing this
-            "IMAGE_SAFETY": "content_filter",
-            "IMAGE_PROHIBITED_CONTENT": "content_filter",
+            k: v
+            for k, v in _FINISH_REASON_MAP.items()
+            if k in VertexGeminiConfig._GEMINI_FINISH_REASON_KEYS
         }
 
     def translate_exception_str(self, exception_string: str):
@@ -1769,15 +1779,14 @@ class VertexGeminiConfig(VertexAIBaseConfig, BaseConfig):
         chat_completion_message: Optional[ChatCompletionResponseMessage],
         finish_reason: Optional[str],
     ) -> OpenAIChatCompletionFinishReason:
-        mapped_finish_reason = VertexGeminiConfig.get_finish_reason_mapping()
+        from litellm.litellm_core_utils.core_helpers import map_finish_reason
+
         if chat_completion_message and chat_completion_message.get("function_call"):
             return "function_call"
         elif chat_completion_message and chat_completion_message.get("tool_calls"):
             return "tool_calls"
-        elif (
-            finish_reason and finish_reason in mapped_finish_reason.keys()
-        ):  # vertex ai
-            return mapped_finish_reason[finish_reason]
+        elif finish_reason:
+            return map_finish_reason(finish_reason)
         else:
             return "stop"
 

litellm/proxy/_experimental/mcp_server/openapi_to_mcp_generator.py

@@ -92,24 +92,7 @@ def get_base_url(spec: Dict[str, Any], spec_path: Optional[str] = None) -> str:
     """Extract base URL from OpenAPI spec."""
     # OpenAPI 3.x
     if "servers" in spec and spec["servers"]:
-        server_url = spec["servers"][0]["url"]
-        
-        # If the server URL is relative (starts with /), derive base from spec_path
-        if server_url.startswith("/") and spec_path:
-            if spec_path.startswith("http://") or spec_path.startswith("https://"):
-                # Extract base URL from spec_path (e.g., https://petstore3.swagger.io/api/v3/openapi.json)
-                # Combine domain with the relative server URL
-                from urllib.parse import urlparse
-                parsed = urlparse(spec_path)
-                base_domain = f"{parsed.scheme}://{parsed.netloc}"
-                full_base_url = base_domain + server_url
-                verbose_logger.info(
-                    f"OpenAPI spec has relative server URL '{server_url}'. "
-                    f"Deriving base from spec_path: {full_base_url}"
-                )
-                return full_base_url
-        
-        return server_url
+        return spec["servers"][0]["url"]
     # OpenAPI 2.x (Swagger)
     elif "host" in spec:
         scheme = spec.get("schemes", ["https"])[0]

litellm/proxy/_experimental/mcp_server/server.py

@@ -711,7 +711,6 @@ if MCP_AVAILABLE:
 
         Checks both the full tool name and unprefixed version (without server prefix).
         This allows users to configure simple tool names regardless of prefixing.
-        Comparison is case-insensitive to handle OpenAPI operationIds that may be in camelCase.
 
         Args:
             tool_name: The tool name to check (may be prefixed like "server-tool_name")
@@ -724,15 +723,13 @@ if MCP_AVAILABLE:
             split_server_prefix_from_name,
         )
 
-        # Normalize filter list to lowercase for case-insensitive comparison
-        filter_list_lower = [f.lower() for f in filter_list]
-
-        if tool_name.lower() in filter_list_lower:
+        # Check if the full name is in the list
+        if tool_name in filter_list:
             return True
 
-        # Check if the unprefixed name is in the list (case-insensitive)
+        # Check if the unprefixed name is in the list
         unprefixed_name, _ = split_server_prefix_from_name(tool_name)
-        return unprefixed_name.lower() in filter_list_lower
+        return unprefixed_name in filter_list
 
     def filter_tools_by_allowed_tools(
         tools: List[MCPTool],

litellm/proxy/auth/model_checks.py

@@ -108,23 +108,16 @@ def get_key_models(
     """
     all_models: List[str] = []
     if len(user_api_key_dict.models) > 0:
-        all_models = list(user_api_key_dict.models)  # copy to avoid mutating cached objects
+        all_models = user_api_key_dict.models
         if SpecialModelNames.all_team_models.value in all_models:
-            all_models = list(user_api_key_dict.team_models)  # copy to avoid mutating cached objects
+            all_models = user_api_key_dict.team_models
         if SpecialModelNames.all_proxy_models.value in all_models:
-            all_models = list(proxy_model_list)  # copy to avoid mutating caller's list
-            if include_model_access_groups:
-                all_models.extend(model_access_groups.keys())
+            all_models = proxy_model_list
 
     all_models = _get_models_from_access_groups(
-        model_access_groups=model_access_groups,
-        all_models=all_models,
-        include_model_access_groups=include_model_access_groups,
+        model_access_groups=model_access_groups, all_models=all_models
     )
 
-    # deduplicate while preserving order
-    all_models = list(dict.fromkeys(all_models))
-
     verbose_proxy_logger.debug("ALL KEY MODELS - {}".format(len(all_models)))
     return all_models
 
@@ -148,8 +141,8 @@ def get_team_models(
             all_models_set.update(team_models)
         if SpecialModelNames.all_proxy_models.value in all_models_set:
             all_models_set.update(proxy_model_list)
-            if include_model_access_groups:
-                all_models_set.update(model_access_groups.keys())
+
+    all_models = list(all_models_set)
 
     all_models = _get_models_from_access_groups(
         model_access_groups=model_access_groups,
@@ -157,9 +150,6 @@ def get_team_models(
         include_model_access_groups=include_model_access_groups,
     )
 
-    # deduplicate while preserving order
-    all_models = list(dict.fromkeys(all_models))
-
     verbose_proxy_logger.debug("ALL TEAM MODELS - {}".format(len(all_models)))
     return all_models
 

litellm/proxy/credential_endpoints/endpoints.py

@@ -142,47 +142,17 @@ async def get_credentials(
     tags=["credential management"],
     response_model=CredentialItem,
 )
-async def get_credential_by_name(
-    request: Request,
-    fastapi_response: Response,
-    credential_name: str = Path(..., description="The credential name, percent-decoded; may contain slashes"),
-    user_api_key_dict: UserAPIKeyAuth = Depends(user_api_key_auth),
-):
-    """
-    [BETA] endpoint. This might change unexpectedly.
-    """
-    try:
-        for credential in litellm.credential_list:
-            if credential.credential_name == credential_name:
-                masked_credential = CredentialItem(
-                    credential_name=credential.credential_name,
-                    credential_values=_get_masked_values(
-                        credential.credential_values,
-                        unmasked_length=4,
-                        number_of_asterisks=4,
-                    ),
-                    credential_info=credential.credential_info,
-                )
-                return masked_credential
-        raise HTTPException(
-            status_code=404,
-            detail="Credential not found. Got credential name: " + credential_name,
-        )
-    except Exception as e:
-        verbose_proxy_logger.exception(e)
-        raise handle_exception_on_proxy(e)
-
-
 @router.get(
     "/credentials/by_model/{model_id}",
     dependencies=[Depends(user_api_key_auth)],
     tags=["credential management"],
     response_model=CredentialItem,
 )
-async def get_credential_by_model(
+async def get_credential(
     request: Request,
     fastapi_response: Response,
-    model_id: str = Path(..., description="The model ID to look up credentials for"),
+    credential_name: str = Path(..., description="The credential name, percent-decoded; may contain slashes"),
+    model_id: Optional[str] = None,
     user_api_key_dict: UserAPIKeyAuth = Depends(user_api_key_auth),
 ):
     """
@@ -191,25 +161,48 @@ async def get_credential_by_model(
     from litellm.proxy.proxy_server import llm_router
 
     try:
-        if llm_router is None:
-            raise HTTPException(status_code=500, detail="LLM router not found")
-        model = llm_router.get_deployment(model_id)
-        if model is None:
-            raise HTTPException(status_code=404, detail="Model not found")
-        credential_values = llm_router.get_deployment_credentials(model_id)
-        if credential_values is None:
-            raise HTTPException(status_code=404, detail="Model not found")
-        masked_credential_values = _get_masked_values(
-            credential_values,
-            unmasked_length=4,
-            number_of_asterisks=4,
-        )
-        credential = CredentialItem(
-            credential_name="{}-credential-{}".format(model.model_name, model_id),
-            credential_values=masked_credential_values,
-            credential_info={},
-        )
-        return credential
+        if model_id:
+            if llm_router is None:
+                raise HTTPException(status_code=500, detail="LLM router not found")
+            model = llm_router.get_deployment(model_id)
+            if model is None:
+                raise HTTPException(status_code=404, detail="Model not found")
+            credential_values = llm_router.get_deployment_credentials(model_id)
+            if credential_values is None:
+                raise HTTPException(status_code=404, detail="Model not found")
+            masked_credential_values = _get_masked_values(
+                credential_values,
+                unmasked_length=4,
+                number_of_asterisks=4,
+            )
+            credential = CredentialItem(
+                credential_name="{}-credential-{}".format(model.model_name, model_id),
+                credential_values=masked_credential_values,
+                credential_info={},
+            )
+            # return credential object
+            return credential
+        elif credential_name:
+            for credential in litellm.credential_list:
+                if credential.credential_name == credential_name:
+                    masked_credential = CredentialItem(
+                        credential_name=credential.credential_name,
+                        credential_values=_get_masked_values(
+                            credential.credential_values,
+                            unmasked_length=4,
+                            number_of_asterisks=4,
+                        ),
+                        credential_info=credential.credential_info,
+                    )
+                    return masked_credential
+            raise HTTPException(
+                status_code=404,
+                detail="Credential not found. Got credential name: " + credential_name,
+            )
+        else:
+            raise HTTPException(
+                status_code=404, detail="Credential name or model ID required"
+            )
     except Exception as e:
         verbose_proxy_logger.exception(e)
         raise handle_exception_on_proxy(e)

litellm/proxy/guardrails/guardrail_hooks/panw_prisma_airs/__init__.py

@@ -19,7 +19,7 @@ def initialize_guardrail(litellm_params: "LitellmParams", guardrail: "Guardrail"
 
     _panw_callback = PanwPrismaAirsHandler(
         **{
-            **litellm_params.model_dump(),
+            **litellm_params.model_dump(exclude_unset=True),
             "guardrail_name": guardrail_name,
             "event_hook": litellm_params.mode,
             "default_on": litellm_params.default_on or False,

litellm/proxy/management_endpoints/team_endpoints.py

@@ -2827,6 +2827,21 @@ async def validate_membership(
     )
 
 
+def _unfurl_all_proxy_models(
+    team_info: LiteLLM_TeamTable, llm_router: Router
+) -> LiteLLM_TeamTable:
+    if (
+        SpecialModelNames.all_proxy_models.value in team_info.models
+        and llm_router is not None
+    ):
+        team_models: set[str] = set()  # make set to avoid duplicates
+        for model in team_info.models:
+            if model != SpecialModelNames.all_proxy_models.value:
+                team_models.add(model)
+        for model in llm_router.get_model_names():
+            team_models.add(model)
+        team_info.models = list(team_models)
+    return team_info
 
 
 async def _add_team_member_budget_table(
@@ -2957,6 +2972,9 @@ async def team_info(
                 team_info_response_object=_team_info,
             )
 
+        # ## UNFURL 'all-proxy-models' into the team_info.models list ##
+        # if llm_router is not None:
+        #     _team_info = _unfurl_all_proxy_models(_team_info, llm_router)
         response_object = TeamInfoResponseObject(
             team_id=team_id,
             team_info=_team_info,

litellm/proxy/pass_through_endpoints/pass_through_endpoints.py

@@ -2062,8 +2062,7 @@ class InitPassThroughEndpointHelpers:
         """
         ## CHECK IF MAPPED PASS THROUGH ENDPOINT
         for mapped_route in LiteLLMRoutes.mapped_pass_through_routes.value:
-            full_mapped_route = InitPassThroughEndpointHelpers._build_full_path_with_root(mapped_route)
-            if route.startswith(full_mapped_route):
+            if route.startswith(mapped_route):
                 return True
 
         # Fast path: check if any registered route key contains this path

litellm/proxy/schema.prisma

@@ -397,6 +397,9 @@ model LiteLLM_VerificationToken {
 
     // SELECT ... FROM "public"."LiteLLM_VerificationToken" WHERE (("public"."LiteLLM_VerificationToken"."expires" IS NULL OR "public"."LiteLLM_VerificationToken"."expires" > $1) AND "public"."LiteLLM_VerificationToken"."budget_reset_at" < $2) OFFSET $3
     @@index([budget_reset_at, expires])
+
+    // SELECT ... FROM "public"."LiteLLM_VerificationToken" WHERE (...) ORDER BY "public"."LiteLLM_VerificationToken"."key_alias" ASC
+    @@index([key_alias])
 }
 
 model LiteLLM_JWTKeyMapping {
@@ -562,6 +565,9 @@ model LiteLLM_SpendLogs {
   @@index([startTime, request_id])
   @@index([end_user])
   @@index([session_id])
+
+  // SELECT ... FROM "LiteLLM_SpendLogs" WHERE ("startTime" >= $1 AND "startTime" <= $2 AND "user" = $3) GROUP BY ...
+  @@index([user, startTime])
 }
 
 // View spend, model, api_key per request

litellm/proxy/spend_tracking/spend_management_endpoints.py

@@ -1461,21 +1461,11 @@ async def _get_spend_report_for_time_range(
     dependencies=[Depends(user_api_key_auth)],
     responses={
         200: {
-            "description": "The calculated cost",
-            "content": {
-                "application/json": {
-                    "schema": {
-                        "type": "object",
-                        "properties": {
-                            "cost": {
-                                "type": "number",
-                                "description": "The calculated cost",
-                                "example": 0.0,
-                            }
-                        },
-                    }
-                }
-            },
+            "cost": {
+                "description": "The calculated cost",
+                "example": 0.0,
+                "type": "float",
+            }
         }
     },
 )

litellm/responses/litellm_completion_transformation/transformation.py

@@ -292,14 +292,14 @@ class LiteLLMCompletionResponsesConfig:
             )
         _messages = litellm_completion_request.get("messages") or []
         session_messages = chat_completion_session.get("messages") or []
-
+        
         # If session messages are empty (e.g., no database in test environment),
         # we still need to process the new input messages
         # Store original _messages before combining for safety check
         original_new_messages = _messages.copy() if _messages else []
-
+        
         combined_messages = session_messages + _messages
-
+        
         # Fix: Ensure tool_results have corresponding tool_calls in previous assistant message
         # Pass tools parameter to help reconstruct tool_calls if not in cache
         tools = litellm_completion_request.get("tools") or []
@@ -307,7 +307,7 @@ class LiteLLMCompletionResponsesConfig:
             messages=combined_messages,
             tools=tools
         )
-
+        
         # Safety check: Ensure we don't end up with empty messages
         # This can happen when using previous_response_id without a database (e.g., in tests)
         # and session messages are empty but new input messages exist
@@ -338,7 +338,7 @@ class LiteLLMCompletionResponsesConfig:
                     model=litellm_completion_request.get("model", ""),
                     llm_provider=litellm_completion_request.get("custom_llm_provider", ""),
                 )
-
+        
         litellm_completion_request["messages"] = combined_messages
         litellm_completion_request["litellm_trace_id"] = chat_completion_session.get(
             "litellm_session_id"
@@ -384,45 +384,10 @@ class LiteLLMCompletionResponsesConfig:
                     if call_id_raw:
                         existing_tool_call_ids.add(str(call_id_raw))
 
-                    #########################################################
-                    # Merge consecutive function_call items into a single assistant
-                    # message. Anthropic requires that all tool_use blocks appear in
-                    # ONE assistant message immediately followed by the tool_result
-                    # blocks. Without this merging, each function_call creates its own
-                    # assistant message, producing back-to-back assistant messages that
-                    # Anthropic rejects with "tool_use ids were found without
-                    # tool_result blocks immediately after".
-                    #########################################################
-                    if messages:
-                        last_msg = messages[-1]
-                        last_role = (
-                            last_msg.get("role")
-                            if isinstance(last_msg, dict)
-                            else getattr(last_msg, "role", None)
-                        )
-                        if last_role == "assistant":
-                            for new_msg in chat_completion_messages:
-                                new_role = (
-                                    new_msg.get("role")
-                                    if isinstance(new_msg, dict)
-                                    else getattr(new_msg, "role", None)
-                                )
-                                if new_role == "assistant":
-                                    new_tcs = (
-                                        new_msg.get("tool_calls")
-                                        if isinstance(new_msg, dict)
-                                        else getattr(new_msg, "tool_calls", None)
-                                    ) or []
-                                    for tc in new_tcs:
-                                        LiteLLMCompletionResponsesConfig._add_tool_call_to_assistant(
-                                            last_msg, tc
-                                        )
-                            continue
-
                 #########################################################
                 # If Input Item is a Tool Call Output, add it to the tool_call_output_messages list
-                # preserving the ordering of tool call outputs. Some models require the tool
-                # result to immediately follow the assistant tool call.
+                # preserving the ordering of tool call outputs. Some models require the tool 
+                # result to immediately follow the assistant tool call. 
                 #########################################################
                 if LiteLLMCompletionResponsesConfig._is_input_item_tool_call_output(
                     input_item=_input
@@ -795,14 +760,14 @@ class LiteLLMCompletionResponsesConfig:
     ]:
         """
         Ensure that tool_result messages have corresponding tool_calls in the previous assistant message.
-
+        
         This is critical for Anthropic API which requires that each tool_result block has a
         corresponding tool_use block in the previous assistant message.
-
+        
         Args:
             messages: List of messages that may include tool_result messages
             tools: Optional list of tools that can be used to reconstruct tool_calls if not in cache
-
+            
         Returns:
             List of messages with tool_calls added to assistant messages when needed
         """
@@ -821,29 +786,29 @@ class LiteLLMCompletionResponsesConfig:
             ]
         ] = list(copy.deepcopy(messages))
         messages_to_remove = []
-
+        
         # Count non-tool messages to avoid removing all messages
         # This prevents empty messages list when using previous_response_id without a database
         non_tool_messages_count = sum(
             1 for msg in fixed_messages if msg.get("role") != "tool"
         )
-
+        
         for i, message in enumerate(fixed_messages):
             # Only process tool messages - check role first to narrow the type
             if message.get("role") != "tool":
                 continue
-
+                
             # At this point, we know it's a tool message, so it should have tool_call_id
             # Use get() with default to safely access tool_call_id
             tool_call_id_raw = message.get("tool_call_id") if isinstance(message, dict) else getattr(message, "tool_call_id", None)
             tool_call_id: str = (
                 str(tool_call_id_raw) if tool_call_id_raw is not None else ""
             )
-
+            
             prev_assistant_idx = LiteLLMCompletionResponsesConfig._find_previous_assistant_idx(
                 fixed_messages, i
             )
-
+            
             # Try to recover empty tool_call_id from previous assistant message
             if not tool_call_id and prev_assistant_idx is not None:
                 prev_assistant = fixed_messages[prev_assistant_idx]
@@ -858,7 +823,7 @@ class LiteLLMCompletionResponsesConfig:
                         message_dict["tool_call_id"] = tool_call_id
                     elif hasattr(message, "tool_call_id"):
                         setattr(message, "tool_call_id", tool_call_id)
-
+            
             # Only remove messages with empty tool_call_id if we have other non-tool messages
             # This prevents ending up with an empty messages list when using previous_response_id
             # without a database (e.g., in tests where session messages are empty)
@@ -870,7 +835,7 @@ class LiteLLMCompletionResponsesConfig:
                 # If no non-tool messages, keep the tool message even with empty call_id
                 # The API will return a proper error message about the missing tool_use block
                 continue
-
+            
             # Check if the previous assistant message has the corresponding tool_call
             # This needs to run for ALL tool messages with a valid tool_call_id,
             # not just those that had an empty tool_call_id initially
@@ -879,12 +844,12 @@ class LiteLLMCompletionResponsesConfig:
                 tool_calls = LiteLLMCompletionResponsesConfig._get_tool_calls_list(
                     prev_assistant
                 )
-
+                
                 if not LiteLLMCompletionResponsesConfig._check_tool_call_exists(
                     tool_calls, tool_call_id
                 ):
                     _tool_use_definition = TOOL_CALLS_CACHE.get_cache(key=tool_call_id)
-
+                    
                     if not _tool_use_definition and tools:
                         _tool_use_definition = (
                             LiteLLMCompletionResponsesConfig._reconstruct_tool_call_from_tools(
@@ -909,11 +874,11 @@ class LiteLLMCompletionResponsesConfig:
                         LiteLLMCompletionResponsesConfig._add_tool_call_to_assistant(
                             prev_assistant, tool_call_chunk
                         )
-
+        
         # Remove messages with empty tool_call_id that couldn't be fixed
         for idx in reversed(messages_to_remove):
             fixed_messages.pop(idx)
-
+        
         return fixed_messages
 
     @staticmethod
@@ -1558,39 +1523,6 @@ class LiteLLMCompletionResponsesConfig:
 
         return tool_call_dict
 
-    @staticmethod
-    def convert_apply_patch_tool_call_to_chat_completion_tool_call(
-        tool_call_item: Any,
-        index: int = 0,
-    ) -> Dict[str, Any]:
-        """
-        Convert ResponseApplyPatchToolCall to ChatCompletionToolCallChunk format.
-
-        The operation (create_file / update_file / delete_file) is serialised
-        as JSON so it appears in function.arguments, just like any other
-        tool call.
-
-        Args:
-            tool_call_item: ResponseApplyPatchToolCall object with call_id and operation
-            index: The index of this tool call
-
-        Returns:
-            Dictionary in ChatCompletionToolCallChunk format
-        """
-        import json
-
-        operation_dict = tool_call_item.operation.model_dump()
-        tool_call_dict: Dict[str, Any] = {
-            "id": tool_call_item.call_id,
-            "function": {
-                "name": "apply_patch",
-                "arguments": json.dumps(operation_dict),
-            },
-            "type": "function",
-            "index": index,
-        }
-        return tool_call_dict
-
     @staticmethod
     def transform_chat_completion_response_to_responses_api_response(
         request_input: Union[str, ResponseInputParam],

litellm/responses/main.py

@@ -686,7 +686,7 @@ def responses(
             BaseResponsesAPIConfig
         ] = ProviderConfigManager.get_provider_responses_api_config(
             model=model,
-            provider=litellm.LlmProviders(custom_llm_provider),
+            provider=custom_llm_provider,
         )
 
         local_vars.update(kwargs)
@@ -902,7 +902,7 @@ def delete_responses(
             BaseResponsesAPIConfig
         ] = ProviderConfigManager.get_provider_responses_api_config(
             model=None,
-            provider=litellm.LlmProviders(custom_llm_provider),
+            provider=custom_llm_provider,
         )
 
         if responses_api_provider_config is None:
@@ -1082,7 +1082,7 @@ def get_responses(
             BaseResponsesAPIConfig
         ] = ProviderConfigManager.get_provider_responses_api_config(
             model=None,
-            provider=litellm.LlmProviders(custom_llm_provider),
+            provider=custom_llm_provider,
         )
 
         if responses_api_provider_config is None:
@@ -1239,7 +1239,7 @@ def list_input_items(
             BaseResponsesAPIConfig
         ] = ProviderConfigManager.get_provider_responses_api_config(
             model=None,
-            provider=litellm.LlmProviders(custom_llm_provider),
+            provider=custom_llm_provider,
         )
 
         if responses_api_provider_config is None:
@@ -1397,7 +1397,7 @@ def cancel_responses(
             BaseResponsesAPIConfig
         ] = ProviderConfigManager.get_provider_responses_api_config(
             model=None,
-            provider=litellm.LlmProviders(custom_llm_provider),
+            provider=custom_llm_provider,
         )
 
         if responses_api_provider_config is None:
@@ -1584,7 +1584,7 @@ def compact_responses(
             BaseResponsesAPIConfig
         ] = ProviderConfigManager.get_provider_responses_api_config(
             model=model,
-            provider=litellm.LlmProviders(custom_llm_provider),
+            provider=custom_llm_provider,
         )
 
         if responses_api_provider_config is None:

litellm/router.py

@@ -5505,10 +5505,6 @@ class Router:
                     return response
 
                 except Exception as e:
-                    # Always track the latest error so we raise the most
-                    # recent exception instead of the first one.
-                    original_exception = e
-
                     ## LOGGING
                     kwargs = self.log_retry(kwargs=kwargs, e=e)
                     remaining_retries = num_retries - current_attempt - 1
@@ -5523,24 +5519,6 @@ class Router:
                         )
                     else:
                         _healthy_deployments = []
-
-                    # Check if this error is non-retryable (e.g., 400 context
-                    # window exceeded). If so, raise immediately instead of
-                    # continuing the retry loop. Respect retry policy
-                    # precedence - only check when no retry policy applies.
-                    if not _retry_policy_applies:
-                        try:
-                            self.should_retry_this_error(
-                                error=e,
-                                healthy_deployments=_healthy_deployments,
-                                all_deployments=_all_deployments,
-                                context_window_fallbacks=context_window_fallbacks,
-                                regular_fallbacks=fallbacks,
-                                content_policy_fallbacks=content_policy_fallbacks,
-                            )
-                        except Exception:
-                            raise e
-
                     _timeout = self._time_to_sleep_before_retry(
                         e=e,
                         remaining_retries=remaining_retries,

litellm/router_strategy/lowest_latency.py

@@ -490,22 +490,20 @@ class LowestLatencyLoggingHandler(CustomLogger):
 
             # get average latency or average ttft (depending on streaming/non-streaming)
             total: float = 0.0
-            use_ttft = (
+            if (
                 request_kwargs is not None
                 and request_kwargs.get("stream", None) is not None
                 and request_kwargs["stream"] is True
                 and len(item_ttft_latency) > 0
-            )
-            if use_ttft:
+            ):
                 for _call_latency in item_ttft_latency:
                     if isinstance(_call_latency, float):
                         total += _call_latency
-                item_latency = total / len(item_ttft_latency)
             else:
                 for _call_latency in item_latency:
                     if isinstance(_call_latency, float):
                         total += _call_latency
-                item_latency = total / len(item_latency)
+            item_latency = total / len(item_latency)
 
             # -------------- #
             # Debugging Logic

litellm/types/images/main.py

@@ -13,6 +13,7 @@ class ImageEditOptionalRequestParams(TypedDict, total=False):
     """
 
     background: Optional[Literal["transparent", "opaque", "auto"]]
+    input_fidelity: Optional[Literal["high", "low"]]
     mask: Optional[str]
     n: Optional[int]
     quality: Optional[Literal["high", "medium", "low", "standard", "auto"]]

litellm/types/llms/openai.py

@@ -1199,6 +1199,14 @@ class ResponseAPIUsage(BaseLiteLLMOpenAIResponseObject):
     cost: Optional[float] = None
     """The cost of the request."""
 
+    @field_validator("cost", mode="before")
+    @classmethod
+    def parse_cost(cls, v: Any) -> Optional[float]:
+        """Normalise cost: accept either a float or a dict with a ``total_cost`` key."""
+        if isinstance(v, dict):
+            return v.get("total_cost")
+        return v
+
     model_config = {"extra": "allow"}
 
 
@@ -2110,7 +2118,7 @@ class OpenAIBatchResult(TypedDict, total=False):
 
 
 OpenAIChatCompletionFinishReason = Literal[
-    "stop", "content_filter", "function_call", "tool_calls", "length", "guardrail_intervened", "eos", "finish_reason_unspecified", "malformed_function_call" # last 2 are vertex ai specific, guardrail_intervened is bedrock specific
+    "stop", "content_filter", "function_call", "tool_calls", "length"
 ]
 
 

litellm/types/proxy/guardrails/guardrail_hooks/panw_prisma_airs.py

@@ -52,6 +52,13 @@ class PanwPrismaAirsGuardrailConfigModel(GuardrailConfigModel):
         description="PANW API call timeout in seconds (1-60).",
     )
 
+    experimental_use_latest_role_message_only: Optional[bool] = Field(
+        default=None,
+        description="Anthropic /v1/messages only. When unset: scans only latest user/developer "
+        "message on request side. Set false to scan all user/system/developer messages. "
+        "Non-Anthropic unaffected.",
+    )
+
     @staticmethod
     def ui_friendly_name() -> str:
         return "PANW Prisma AIRS"

litellm/types/utils.py

@@ -1333,7 +1333,11 @@ class Choices(SafeAttributeModel, OpenAIObject):
         **params,
     ):
         if finish_reason is not None:
-            params["finish_reason"] = map_finish_reason(finish_reason)
+            mapped = map_finish_reason(finish_reason)
+            params["finish_reason"] = mapped
+            if finish_reason != mapped:
+                provider_specific_fields = dict(provider_specific_fields) if provider_specific_fields else {}
+                provider_specific_fields["native_finish_reason"] = finish_reason
         else:
             params["finish_reason"] = "stop"
         if index is not None:

litellm/utils.py

@@ -8101,17 +8101,8 @@ class ProviderConfigManager:
         Returns the provider config for a given provider.
 
         Uses O(1) dictionary lookup for fast provider resolution.
+        Python classes take priority over JSON (they have custom overrides).
         """
-        # Check JSON providers FIRST (these override standard mappings)
-        from litellm.llms.openai_like.dynamic_config import create_config_class
-        from litellm.llms.openai_like.json_loader import JSONProviderRegistry
-
-        if JSONProviderRegistry.exists(provider.value):
-            provider_config = JSONProviderRegistry.get(provider.value)
-            if provider_config is None:
-                raise ValueError(f"Provider {provider.value} not found")
-            return create_config_class(provider_config)()
-
         # Handle OpenAI special cases (O-series and GPT-5 models)
         if provider == LlmProviders.OPENAI:
             if litellm.openaiOSeriesConfig.is_model_o_series_model(model=model):
@@ -8125,18 +8116,24 @@ class ProviderConfigManager:
                 ProviderConfigManager._build_provider_config_map()
             )
 
-        # O(1) dictionary lookup
+        # O(1) dictionary lookup — Python classes first (custom overrides take priority)
         config_entry = ProviderConfigManager._PROVIDER_CONFIG_MAP.get(provider)
-        if config_entry is None:
-            return None
+        if config_entry is not None:
+            config_factory, needs_model = config_entry
+            if needs_model:
+                return config_factory(model)  # type: ignore
+            else:
+                return config_factory()  # type: ignore
 
-        # Unpack factory function and whether it needs model parameter
-        # This avoids expensive inspect.signature() calls at runtime
-        config_factory, needs_model = config_entry
-        if needs_model:
-            return config_factory(model)  # type: ignore
-        else:
-            return config_factory()  # type: ignore
+        # Fall back to JSON providers (generic OpenAI-compatible)
+        from litellm.llms.openai_like.dynamic_config import create_config_class
+        from litellm.llms.openai_like.json_loader import JSONProviderRegistry
+
+        if JSONProviderRegistry.exists(provider.value):
+            provider_config = JSONProviderRegistry.get(provider.value)
+            if provider_config is None:
+                raise ValueError(f"Provider {provider.value} not found")
+            return create_config_class(provider_config)()
 
     @staticmethod
     def get_provider_embedding_config(
@@ -8341,9 +8338,52 @@ class ProviderConfigManager:
 
     @staticmethod
     def get_provider_responses_api_config(
-        provider: LlmProviders,
+        provider: Union[LlmProviders, str],
         model: Optional[str] = None,
     ) -> Optional[BaseResponsesAPIConfig]:
+        from litellm.llms.openai_like.dynamic_config import (
+            create_responses_config_class,
+        )
+        from litellm.llms.openai_like.json_loader import JSONProviderRegistry
+
+        # Resolve provider string for JSON lookup
+        provider_str = provider.value if isinstance(provider, LlmProviders) else str(provider)
+
+        # Try to convert to enum for Python class lookup first.
+        # Python classes take priority over JSON (they have custom overrides).
+        provider_enum: Optional[LlmProviders] = None
+        if isinstance(provider, LlmProviders):
+            provider_enum = provider
+        else:
+            try:
+                provider_enum = LlmProviders(provider)
+            except ValueError:
+                pass
+
+        # Check Python classes first (custom overrides take priority)
+        result = ProviderConfigManager._get_python_responses_api_config(
+            provider_enum, model
+        )
+        if result is not None:
+            return result
+
+        # Fall back to JSON providers (generic OpenAI-compatible)
+        if JSONProviderRegistry.exists(provider_str) and JSONProviderRegistry.supports_responses_api(provider_str):
+            provider_config = JSONProviderRegistry.get(provider_str)
+            if provider_config is not None:
+                return create_responses_config_class(provider_config)()
+
+        return None
+
+    @staticmethod
+    def _get_python_responses_api_config(
+        provider: Optional[LlmProviders],
+        model: Optional[str] = None,
+    ) -> Optional[BaseResponsesAPIConfig]:
+        """Check for Python-class-based responses API configs (custom overrides)."""
+        if provider is None:
+            return None
+
         if litellm.LlmProviders.OPENAI == provider:
             return litellm.OpenAIResponsesAPIConfig()
         elif litellm.LlmProviders.AZURE == provider:

provider_endpoints_support.json

@@ -458,24 +458,6 @@
         "interactions": true
       }
     },
-    "charity_engine": {
-      "display_name": "Charity Engine (`charity_engine`)",
-      "url": "https://docs.litellm.ai/docs/providers/charity_engine",
-      "endpoints": {
-        "chat_completions": true,
-        "messages": true,
-        "responses": true,
-        "embeddings": false,
-        "image_generations": false,
-        "audio_transcriptions": false,
-        "audio_speech": false,
-        "moderations": false,
-        "batches": false,
-        "rerank": false,
-        "a2a": false,
-        "interactions": false
-      }
-    },
     "chutes": {
       "display_name": "Chutes (`chutes`)",
       "endpoints": {

schema.prisma

@@ -388,6 +388,9 @@ model LiteLLM_VerificationToken {
 
     // SELECT ... FROM "public"."LiteLLM_VerificationToken" WHERE (("public"."LiteLLM_VerificationToken"."expires" IS NULL OR "public"."LiteLLM_VerificationToken"."expires" > $1) AND "public"."LiteLLM_VerificationToken"."budget_reset_at" < $2) OFFSET $3
     @@index([budget_reset_at, expires])
+
+    // SELECT ... FROM "public"."LiteLLM_VerificationToken" WHERE (...) ORDER BY "public"."LiteLLM_VerificationToken"."key_alias" ASC
+    @@index([key_alias])
 }
 
 model LiteLLM_JWTKeyMapping {
@@ -553,6 +556,9 @@ model LiteLLM_SpendLogs {
   @@index([startTime, request_id])
   @@index([end_user])
   @@index([session_id])
+
+  // SELECT ... FROM "LiteLLM_SpendLogs" WHERE ("startTime" >= $1 AND "startTime" <= $2 AND "user" = $3) GROUP BY ...
+  @@index([user, startTime])
 }
 
 // View spend, model, api_key per request

tests/llm_translation/test_prompt_factory.py

@@ -1601,3 +1601,346 @@ def test_parse_tool_call_arguments_still_raises_for_unrepairable():
     error_msg = str(exc_info.value)
     assert "test_tool" in error_msg
     assert "test context" in error_msg
+
+
+
+def test_anthropic_messages_pt_interleave_thinking_with_server_tool_calls():
+    """
+    Test that thinking blocks are interleaved with server tool calls (web search)
+    instead of being prepended all at once.
+
+    When Anthropic returns a response with extended thinking + multiple web searches,
+    the content blocks are interleaved:
+    [thinking_1, server_tool_use_1, result_1, thinking_2, server_tool_use_2, result_2]
+
+    On round-trip through OpenAI format, thinking_blocks and tool_calls are separate
+    fields. anthropic_messages_pt must reconstruct the interleaved order, otherwise
+    Anthropic rejects the request because thinking block signatures are position-dependent.
+
+    Fixes: https://github.com/BerriAI/litellm/issues/23047
+    """
+    messages = [
+        {"role": "user", "content": "Search for news about fast.ai and answer.ai"},
+        {
+            "role": "assistant",
+            "content": "Here is what I found.",
+            "thinking_blocks": [
+                {
+                    "type": "thinking",
+                    "thinking": "I need to search for fast.ai news.",
+                    "signature": "sig_thinking_1",
+                },
+                {
+                    "type": "thinking",
+                    "thinking": "Now I should also search for answer.ai.",
+                    "signature": "sig_thinking_2",
+                },
+            ],
+            "tool_calls": [
+                {
+                    "id": "srvtoolu_01SEARCH1",
+                    "type": "function",
+                    "function": {
+                        "name": "web_search",
+                        "arguments": '{"query": "fast.ai news"}',
+                    },
+                },
+                {
+                    "id": "srvtoolu_01SEARCH2",
+                    "type": "function",
+                    "function": {
+                        "name": "web_search",
+                        "arguments": '{"query": "answer.ai news"}',
+                    },
+                },
+            ],
+            "provider_specific_fields": {
+                "web_search_results": [
+                    {
+                        "type": "web_search_tool_result",
+                        "tool_use_id": "srvtoolu_01SEARCH1",
+                        "content": [
+                            {
+                                "type": "web_search_result",
+                                "url": "https://fast.ai",
+                                "title": "fast.ai",
+                                "snippet": "fast.ai news",
+                            }
+                        ],
+                    },
+                    {
+                        "type": "web_search_tool_result",
+                        "tool_use_id": "srvtoolu_01SEARCH2",
+                        "content": [
+                            {
+                                "type": "web_search_result",
+                                "url": "https://answer.ai",
+                                "title": "answer.ai",
+                                "snippet": "answer.ai news",
+                            }
+                        ],
+                    },
+                ]
+            },
+        },
+        {"role": "user", "content": "Now search for news about solveit"},
+    ]
+
+    result = anthropic_messages_pt(
+        messages, model="claude-sonnet-4-5", llm_provider="anthropic"
+    )
+
+    # Find the assistant message
+    assistant_msg = next(m for m in result if m["role"] == "assistant")
+    content = assistant_msg["content"]
+
+    # Extract types in order
+    types = [c.get("type") for c in content]
+
+    # The correct interleaved order should be:
+    # thinking_1, server_tool_use_1, web_search_tool_result_1,
+    # thinking_2, server_tool_use_2, web_search_tool_result_2,
+    # text
+    assert types == [
+        "thinking",
+        "server_tool_use",
+        "web_search_tool_result",
+        "thinking",
+        "server_tool_use",
+        "web_search_tool_result",
+        "text",
+    ], f"Expected interleaved order but got: {types}"
+
+    # Verify thinking blocks preserved their content and signatures
+    thinking_1 = content[0]
+    assert thinking_1["thinking"] == "I need to search for fast.ai news."
+    assert thinking_1["signature"] == "sig_thinking_1"
+
+    thinking_2 = content[3]
+    assert thinking_2["thinking"] == "Now I should also search for answer.ai."
+    assert thinking_2["signature"] == "sig_thinking_2"
+
+    # Verify server_tool_use blocks preserved their IDs
+    assert content[1]["id"] == "srvtoolu_01SEARCH1"
+    assert content[4]["id"] == "srvtoolu_01SEARCH2"
+
+    # Verify web_search_tool_result blocks are paired correctly
+    assert content[2]["tool_use_id"] == "srvtoolu_01SEARCH1"
+    assert content[5]["tool_use_id"] == "srvtoolu_01SEARCH2"
+
+    # Verify text block is present at the end
+    assert content[6]["text"] == "Here is what I found."
+
+
+def test_anthropic_messages_pt_thinking_blocks_no_server_tools_unchanged():
+    """
+    Test that the existing behavior is preserved when thinking blocks exist
+    but there are no server tool calls (only regular tool_use).
+
+    Thinking blocks should still be prepended first in this case.
+    """
+    messages = [
+        {"role": "user", "content": "What is the weather?"},
+        {
+            "role": "assistant",
+            "content": "Let me check.",
+            "thinking_blocks": [
+                {
+                    "type": "thinking",
+                    "thinking": "I should check the weather.",
+                    "signature": "sig_1",
+                },
+            ],
+            "tool_calls": [
+                {
+                    "id": "toolu_01REG",
+                    "type": "function",
+                    "function": {
+                        "name": "get_weather",
+                        "arguments": '{"location": "SF"}',
+                    },
+                },
+            ],
+        },
+        {
+            "role": "tool",
+            "tool_call_id": "toolu_01REG",
+            "content": "72F and sunny",
+        },
+    ]
+
+    result = anthropic_messages_pt(
+        messages, model="claude-sonnet-4-5", llm_provider="anthropic"
+    )
+
+    assistant_msg = next(m for m in result if m["role"] == "assistant")
+    content = assistant_msg["content"]
+    types = [c.get("type") for c in content]
+
+    # Original behavior: thinking first, then text, then tool_use
+    assert types == ["thinking", "text", "tool_use"], f"Expected sequential order but got: {types}"
+
+
+def test_anthropic_messages_pt_interleave_more_thinking_than_tool_groups():
+    """
+    Test interleaving when there are more thinking blocks than server tool groups.
+    Extra thinking blocks should appear before the text block.
+    """
+    messages = [
+        {"role": "user", "content": "Search for something"},
+        {
+            "role": "assistant",
+            "content": "Found it.",
+            "thinking_blocks": [
+                {
+                    "type": "thinking",
+                    "thinking": "First thought",
+                    "signature": "sig_1",
+                },
+                {
+                    "type": "thinking",
+                    "thinking": "Second thought",
+                    "signature": "sig_2",
+                },
+                {
+                    "type": "thinking",
+                    "thinking": "Third thought after search",
+                    "signature": "sig_3",
+                },
+            ],
+            "tool_calls": [
+                {
+                    "id": "srvtoolu_01ONLY",
+                    "type": "function",
+                    "function": {
+                        "name": "web_search",
+                        "arguments": '{"query": "something"}',
+                    },
+                },
+            ],
+            "provider_specific_fields": {
+                "web_search_results": [
+                    {
+                        "type": "web_search_tool_result",
+                        "tool_use_id": "srvtoolu_01ONLY",
+                        "content": [{"type": "web_search_result", "url": "https://example.com", "title": "Test", "snippet": "result"}],
+                    },
+                ]
+            },
+        },
+    ]
+
+    result = anthropic_messages_pt(
+        messages, model="claude-sonnet-4-5", llm_provider="anthropic"
+    )
+
+    assistant_msg = next(m for m in result if m["role"] == "assistant")
+    content = assistant_msg["content"]
+    types = [c.get("type") for c in content]
+
+    # thinking_1 paired with tool group, thinking_2 and thinking_3 before text
+    assert types == [
+        "thinking",            # paired with tool group
+        "server_tool_use",
+        "web_search_tool_result",
+        "thinking",            # extra - before text
+        "thinking",            # extra - before text
+        "text",
+    ], f"Expected order but got: {types}"
+
+
+def test_anthropic_messages_pt_list_content_with_thinking_preserves_order():
+    """
+    Test that when assistant content is already a list containing interleaved
+    thinking blocks and server tool blocks, the thinking_blocks from
+    provider_specific_fields are NOT duplicated/prepended.
+
+    This covers the gap identified by Greptile where list-content messages
+    bypass INTERLEAVED MODE and fall into SEQUENTIAL MODE, which previously
+    would prepend all thinking_blocks again, causing duplication and
+    breaking Anthropic's position-dependent signature verification.
+
+    Fixes: https://github.com/BerriAI/litellm/issues/23047
+    """
+    messages = [
+        {"role": "user", "content": "Search for AI news"},
+        {
+            "role": "assistant",
+            # Content is already a list with interleaved thinking + server tool blocks
+            "content": [
+                {
+                    "type": "thinking",
+                    "thinking": "Let me search for AI news.",
+                    "signature": "sig_1",
+                },
+                {
+                    "type": "server_tool_use",
+                    "id": "srvtoolu_01SEARCH1",
+                    "name": "web_search",
+                    "input": {"query": "AI news"},
+                },
+                {
+                    "type": "web_search_tool_result",
+                    "tool_use_id": "srvtoolu_01SEARCH1",
+                    "content": [
+                        {
+                            "type": "web_search_result",
+                            "url": "https://example.com",
+                            "title": "AI News",
+                            "snippet": "Latest AI news",
+                        }
+                    ],
+                },
+                {
+                    "type": "thinking",
+                    "thinking": "Now let me summarize.",
+                    "signature": "sig_2",
+                },
+                {
+                    "type": "text",
+                    "text": "Here is the AI news summary.",
+                },
+            ],
+            # thinking_blocks also present in provider_specific_fields
+            "thinking_blocks": [
+                {
+                    "type": "thinking",
+                    "thinking": "Let me search for AI news.",
+                    "signature": "sig_1",
+                },
+                {
+                    "type": "thinking",
+                    "thinking": "Now let me summarize.",
+                    "signature": "sig_2",
+                },
+            ],
+        },
+        {"role": "user", "content": "Tell me more"},
+    ]
+
+    result = anthropic_messages_pt(
+        messages, model="claude-sonnet-4-5", llm_provider="anthropic"
+    )
+
+    assistant_msg = next(m for m in result if m["role"] == "assistant")
+    content = assistant_msg["content"]
+    types = [c.get("type") for c in content]
+
+    # The list content already has the correct interleaved order.
+    # thinking_blocks should NOT be prepended again (which would cause
+    # duplication and break signature verification).
+    assert types == [
+        "thinking",
+        "server_tool_use",
+        "web_search_tool_result",
+        "thinking",
+        "text",
+    ], f"Expected preserved list order without duplicate thinking blocks, but got: {types}"
+
+    # Verify no duplicate thinking blocks
+    thinking_count = sum(1 for t in types if t == "thinking")
+    assert thinking_count == 2, f"Expected 2 thinking blocks, got {thinking_count} (duplication detected)"
+
+    # Verify signatures preserved in correct positions
+    assert content[0]["signature"] == "sig_1"
+    assert content[3]["signature"] == "sig_2"

tests/llm_translation/test_skills_api.py

@@ -44,25 +44,19 @@ def create_skill_zip(skill_name: str, unique_suffix: Optional[str] = None):
     skill_dir = test_dir / skill_name
 
     # Create a zip file containing the skill directory
-    # When unique_suffix is set, folder name must match skill name in SKILL.md (Anthropic requirement)
-    zip_folder_name = f"{skill_name}-{unique_suffix}" if unique_suffix else skill_name
     zip_path = test_dir / f"{skill_name}.zip"
     with zipfile.ZipFile(zip_path, "w", zipfile.ZIP_DEFLATED) as zf:
+        zf.write(skill_dir, arcname=skill_name)
+
         if unique_suffix is not None:
-            # Rewrite SKILL.md with a unique name and use matching folder name
+            # Rewrite SKILL.md with a unique name to avoid API conflicts
             skill_md = (skill_dir / "SKILL.md").read_text()
             skill_md = skill_md.replace(
                 f"name: {skill_name}",
-                f"name: {zip_folder_name}",
+                f"name: {skill_name}-{unique_suffix}",
             )
-            zf.writestr(f"{zip_folder_name}/SKILL.md", skill_md)
-            # Add any other files in the skill dir (e.g. subdirs) under the new folder name
-            for f in skill_dir.rglob("*"):
-                if f.is_file() and f.name != "SKILL.md":
-                    rel = f.relative_to(skill_dir)
-                    zf.write(f, arcname=f"{zip_folder_name}/{rel}")
+            zf.writestr(f"{skill_name}/SKILL.md", skill_md)
         else:
-            zf.write(skill_dir, arcname=skill_name)
             zf.write(skill_dir / "SKILL.md", arcname=f"{skill_name}/SKILL.md")
 
     try:

tests/local_testing/test_custom_callback_input.py

@@ -1300,11 +1300,9 @@ def test_logging_async_cache_hit_sync_call(turn_off_message_logging):
                 "redacted-by-litellm"
                 == standard_logging_object["messages"][0]["content"]
             )
-            # response is a full ModelResponse dict (choices format) since d84e5e381acf
-            assert (
-                standard_logging_object["response"]["choices"][0]["message"]["content"]
-                == "redacted-by-litellm"
-            )
+            assert {"text": "redacted-by-litellm"} == standard_logging_object[
+                "response"
+            ]
 
 
 def test_logging_standard_payload_failure_call():

tests/logging_callback_tests/test_logging_redaction_e2e_test.py

@@ -45,8 +45,7 @@ async def test_global_redaction_on():
     await asyncio.sleep(1)
     standard_logging_payload = test_custom_logger.logged_standard_logging_payload
     assert standard_logging_payload is not None
-    response = standard_logging_payload["response"]
-    assert response["choices"][0]["message"]["content"] == "redacted-by-litellm"
+    assert standard_logging_payload["response"] == {"text": "redacted-by-litellm"}
     assert standard_logging_payload["messages"][0]["content"] == "redacted-by-litellm"
     print(
         "logged standard logging payload",
@@ -76,8 +75,7 @@ async def test_global_redaction_with_dynamic_params(turn_off_message_logging):
     )
 
     if turn_off_message_logging is True:
-        response = standard_logging_payload["response"]
-        assert response["choices"][0]["message"]["content"] == "redacted-by-litellm"
+        assert standard_logging_payload["response"] == {"text": "redacted-by-litellm"}
         assert (
             standard_logging_payload["messages"][0]["content"] == "redacted-by-litellm"
         )
@@ -110,8 +108,7 @@ async def test_global_redaction_off_with_dynamic_params(turn_off_message_logging
         json.dumps(standard_logging_payload, indent=2),
     )
     if turn_off_message_logging is True:
-        response = standard_logging_payload["response"]
-        assert response["choices"][0]["message"]["content"] == "redacted-by-litellm"
+        assert standard_logging_payload["response"] == {"text": "redacted-by-litellm"}
         assert (
             standard_logging_payload["messages"][0]["content"] == "redacted-by-litellm"
         )
@@ -393,8 +390,7 @@ async def test_redaction_with_streaming_response():
     assert standard_logging_payload is not None
     
     # Verify that redaction worked without pickle errors
-    response = standard_logging_payload["response"]
-    assert response["choices"][0]["message"]["content"] == "redacted-by-litellm"
+    assert standard_logging_payload["response"] == {"text": "redacted-by-litellm"}
     assert standard_logging_payload["messages"][0]["content"] == "redacted-by-litellm"
     print(
         "logged standard logging payload for streaming with coroutine handling",
@@ -481,6 +477,5 @@ async def test_redaction_with_metadata_completion_api():
     
     # Verify the helper function works correctly - with get_metadata_variable_name_from_kwargs,
     # the system checks the appropriate field for headers
-    response = standard_logging_payload["response"]
-    assert response["choices"][0]["message"]["content"] == "redacted-by-litellm"
+    assert standard_logging_payload["response"] == {"text": "redacted-by-litellm"}
     assert standard_logging_payload["messages"][0]["content"] == "redacted-by-litellm"

tests/test_litellm/caching/test_dual_cache.py

@@ -1,11 +1,9 @@
 import asyncio
-import time
 from unittest.mock import AsyncMock, MagicMock, patch
 
 import pytest
 
 from litellm.caching.dual_cache import DualCache
-from litellm.caching.in_memory_cache import InMemoryCache
 from litellm.caching.redis_cache import RedisCache
 
 
@@ -58,104 +56,3 @@ async def test_dual_cache_async_batch_get_cache_rolls_back_redis_reservation_on_
         assert mock_async_batch_get_cache.call_count == 2
         assert "shared_a" not in dual_cache.last_redis_batch_access_time
         assert "shared_b" not in dual_cache.last_redis_batch_access_time
-
-
-@pytest.mark.asyncio
-async def test_dual_cache_async_set_cache_injects_default_in_memory_ttl():
-    """
-    Test that async_set_cache injects default_in_memory_ttl into kwargs
-    when no explicit ttl is provided, matching the sync set_cache behavior.
-
-    Regression test for: async_set_cache was missing the TTL injection that
-    sync set_cache has, causing InMemoryCache to use its own default_ttl (600s)
-    instead of DualCache's default_in_memory_ttl.
-    """
-    in_memory_cache = InMemoryCache(default_ttl=600)
-    dual_cache = DualCache(
-        in_memory_cache=in_memory_cache,
-        default_in_memory_ttl=60,
-    )
-
-    before = time.time()
-    await dual_cache.async_set_cache(key="test_key", value="test_value")
-    after = time.time()
-
-    # The TTL stored should reflect default_in_memory_ttl (60s), not
-    # InMemoryCache's default_ttl (600s)
-    expiry = in_memory_cache.ttl_dict["test_key"]
-    assert expiry >= before + 60
-    assert expiry <= after + 60
-
-
-@pytest.mark.asyncio
-async def test_dual_cache_async_set_cache_respects_explicit_ttl():
-    """
-    Test that async_set_cache does NOT override an explicitly provided ttl.
-    """
-    in_memory_cache = InMemoryCache(default_ttl=600)
-    dual_cache = DualCache(
-        in_memory_cache=in_memory_cache,
-        default_in_memory_ttl=60,
-    )
-
-    before = time.time()
-    await dual_cache.async_set_cache(key="test_key", value="test_value", ttl=30)
-    after = time.time()
-
-    # The explicit ttl=30 should be used, not default_in_memory_ttl (60)
-    expiry = in_memory_cache.ttl_dict["test_key"]
-    assert expiry >= before + 30
-    assert expiry <= after + 30
-
-
-@pytest.mark.asyncio
-async def test_dual_cache_async_set_cache_pipeline_injects_default_in_memory_ttl():
-    """
-    Test that async_set_cache_pipeline injects default_in_memory_ttl into kwargs
-    when no explicit ttl is provided.
-    """
-    in_memory_cache = InMemoryCache(default_ttl=600)
-    dual_cache = DualCache(
-        in_memory_cache=in_memory_cache,
-        default_in_memory_ttl=60,
-    )
-
-    cache_list = [("key_a", "value_a"), ("key_b", "value_b")]
-
-    before = time.time()
-    await dual_cache.async_set_cache_pipeline(cache_list=cache_list)
-    after = time.time()
-
-    for key in ["key_a", "key_b"]:
-        expiry = in_memory_cache.ttl_dict[key]
-        assert expiry >= before + 60
-        assert expiry <= after + 60
-
-
-@pytest.mark.asyncio
-async def test_dual_cache_sync_and_async_set_cache_use_same_ttl():
-    """
-    Test that sync set_cache and async async_set_cache produce the same TTL
-    when no explicit ttl is provided, ensuring parity between the two paths.
-    """
-    in_memory_sync = InMemoryCache(default_ttl=600)
-    dual_cache_sync = DualCache(
-        in_memory_cache=in_memory_sync,
-        default_in_memory_ttl=60,
-    )
-
-    in_memory_async = InMemoryCache(default_ttl=600)
-    dual_cache_async = DualCache(
-        in_memory_cache=in_memory_async,
-        default_in_memory_ttl=60,
-    )
-
-    dual_cache_sync.set_cache(key="test_key", value="test_value")
-    await dual_cache_async.async_set_cache(key="test_key", value="test_value")
-
-    sync_expiry = in_memory_sync.ttl_dict["test_key"]
-    async_expiry = in_memory_async.ttl_dict["test_key"]
-
-    # Both should use default_in_memory_ttl=60, so their expiry times
-    # should be within a small tolerance of each other
-    assert abs(sync_expiry - async_expiry) < 1.0

tests/test_litellm/completion_extras/litellm_responses_transformation/test_completion_extras_litellm_responses_transformation_transformation.py

@@ -738,58 +738,7 @@ def test_response_completed_with_message_only_emits_stop_finish_reason():
     )
 
 
-
-def test_response_completed_preserves_usage_with_cached_tokens():
-    """
-    Test that response.completed correctly translates Responses API usage
-    (input_tokens_details) to chat completion usage (prompt_tokens_details).
-
-    This is a regression test for an issue where streaming with models that
-    use the Responses API bridge (e.g. gpt-5.2-codex) would drop
-    prompt_tokens_details, causing cached_tokens to always be None.
-    """
-    from litellm.completion_extras.litellm_responses_transformation.transformation import (
-        OpenAiResponsesToChatCompletionStreamIterator,
-    )
-
-    iterator = OpenAiResponsesToChatCompletionStreamIterator(streaming_response=None, sync_stream=True)
-
-    chunk = {
-        "type": "response.completed",
-        "response": {
-            "id": "resp_789",
-            "status": "completed",
-            "output": [
-                {
-                    "type": "message",
-                    "id": "msg_abc",
-                    "role": "assistant",
-                    "content": [{"type": "output_text", "text": "Six"}],
-                    "status": "completed",
-                }
-            ],
-            "usage": {
-                "input_tokens": 1226,
-                "output_tokens": 5,
-                "total_tokens": 1231,
-                "input_tokens_details": {"cached_tokens": 1024},
-                "output_tokens_details": {"reasoning_tokens": 0},
-            },
-        },
-    }
-
-    result = iterator.chunk_parser(chunk)
-
-    assert result.usage is not None, "usage should be set on response.completed chunk"
-    assert result.usage.prompt_tokens == 1226, "prompt_tokens should map from input_tokens"
-    assert result.usage.completion_tokens == 5, "completion_tokens should map from output_tokens"
-    assert result.usage.prompt_tokens_details is not None, "prompt_tokens_details should be set"
-    assert result.usage.prompt_tokens_details.cached_tokens == 1024, (
-        "cached_tokens should be preserved from input_tokens_details"
-    )
-
-
-def test_function_call_done_emits_is_finished():
+def test_function_call_done_does_not_emit_finish_reason():
     """
     Test that OUTPUT_ITEM_DONE for a function_call does NOT emit finish_reason.
     The response.completed event handles the terminal finish_reason correctly.
@@ -1378,138 +1327,6 @@ def test_transform_response_preserves_annotations():
     print("✓ Annotations from Responses API are correctly preserved in Chat Completions format")
 
 
-def test_apply_patch_tool_call_converted_to_chat_completion_tool_call():
-    """
-    Test that ResponseApplyPatchToolCall items from the Responses API are
-    correctly converted to ChatCompletions-style tool calls by the bridge.
-
-    This is a regression test for a bug where litellm.completion() with a
-    responses/ model prefix crashed when the model returned an
-    apply_patch_call, because _convert_response_output_to_choices did not
-    handle ResponseApplyPatchToolCall items. The model DID use the tool,
-    but the bridge silently dropped it (or raised an error), while the
-    native litellm.responses() path worked correctly.
-    """
-    import json
-    from unittest.mock import Mock
-
-    from openai.types.responses.response_apply_patch_tool_call import (
-        OperationCreateFile,
-    )
-    from openai.types.responses.response_output_item import (
-        ResponseApplyPatchToolCall,
-    )
-
-    from litellm.completion_extras.litellm_responses_transformation.transformation import (
-        LiteLLMResponsesTransformationHandler,
-    )
-    from litellm.types.llms.openai import (
-        InputTokensDetails,
-        OutputTokensDetails,
-        ResponseAPIUsage,
-        ResponsesAPIResponse,
-    )
-    from litellm.types.utils import ModelResponse, Usage
-
-    handler = LiteLLMResponsesTransformationHandler()
-
-    # Build an apply_patch_call item like the model would return
-    operation = OperationCreateFile(
-        diff="--- /dev/null\n+++ b/hello.py\n@@ -0,0 +1 @@\n+print('hello world')\n",
-        path="hello.py",
-        type="create_file",
-    )
-    apply_patch_item = ResponseApplyPatchToolCall(
-        id="apc_001",
-        call_id="call_patch_hello",
-        operation=operation,
-        status="completed",
-        type="apply_patch_call",
-    )
-
-    # Minimal usage
-    usage = ResponseAPIUsage(
-        input_tokens=30,
-        input_tokens_details=InputTokensDetails(cached_tokens=0),
-        output_tokens=40,
-        output_tokens_details=OutputTokensDetails(reasoning_tokens=0),
-        total_tokens=70,
-    )
-
-    raw_response = ResponsesAPIResponse(
-        id="resp_apply_patch_test",
-        created_at=1234567890,
-        error=None,
-        incomplete_details=None,
-        instructions=None,
-        metadata={},
-        model="gpt-5.2-codex",
-        object="response",
-        output=[apply_patch_item],
-        parallel_tool_calls=True,
-        temperature=1.0,
-        tool_choice="auto",
-        tools=[],
-        top_p=1.0,
-        max_output_tokens=None,
-        previous_response_id=None,
-        reasoning=None,
-        status="completed",
-        text=None,
-        truncation="disabled",
-        usage=usage,
-        user=None,
-        store=True,
-        background=False,
-    )
-
-    model_response = ModelResponse(
-        id="chatcmpl-apply-patch",
-        created=1234567890,
-        model=None,
-        object="chat.completion",
-        choices=[],
-        usage=Usage(completion_tokens=0, prompt_tokens=0, total_tokens=0),
-    )
-
-    logging_obj = Mock()
-
-    result = handler.transform_response(
-        model="gpt-5.2-codex",
-        raw_response=raw_response,
-        model_response=model_response,
-        logging_obj=logging_obj,
-        request_data={"model": "gpt-5.2-codex"},
-        messages=[
-            {"role": "system", "content": "You are a coding assistant."},
-            {"role": "user", "content": "Create hello.py"},
-        ],
-        optional_params={},
-        litellm_params={},
-        encoding=Mock(),
-    )
-
-    # Should have exactly one choice with finish_reason="tool_calls"
-    assert len(result.choices) == 1, f"Expected 1 choice, got {len(result.choices)}"
-
-    choice = result.choices[0]
-    assert choice.finish_reason == "tool_calls"
-
-    # The choice should contain one tool call for apply_patch
-    tool_calls = choice.message.tool_calls
-    assert tool_calls is not None, "tool_calls should not be None"
-    assert len(tool_calls) == 1, f"Expected 1 tool_call, got {len(tool_calls)}"
-
-    tc = tool_calls[0]
-    assert tc["id"] == "call_patch_hello"
-    assert tc["type"] == "function"
-    assert tc["function"]["name"] == "apply_patch"
-
-    # The operation should be serialised as JSON in arguments
-    args = json.loads(tc["function"]["arguments"])
-    assert args["type"] == "create_file"
-    assert args["path"] == "hello.py"
-    assert "print('hello world')" in args["diff"]
 def test_multi_tool_call_stream_no_premature_finish():
     """
     Regression test for multi-tool-call streaming bug.
@@ -1961,35 +1778,3 @@ def test_parallel_tool_calls_comprehensive_streaming_integration():
     )
 
     print("✓ Parallel tool calls with split argument deltas stream correctly end-to-end")
-
-
-def test_map_optional_params_preserves_reasoning_summary():
-    """Test that reasoning_effort dict with summary field is preserved.
-    
-    Regression test for: User reported that summary field was being dropped
-    when routing to Responses API. The dict format should be fully preserved.
-    """
-    from litellm.completion_extras.litellm_responses_transformation.transformation import (
-        LiteLLMResponsesTransformationHandler,
-    )
-    from litellm.types.llms.openai import ResponsesAPIOptionalRequestParams
-
-    handler = LiteLLMResponsesTransformationHandler()
-
-    optional_params = {
-        "stream": False,
-        "tools": [{"type": "function", "function": {"name": "test_tool"}}],
-        "tool_choice": "auto",
-        "reasoning_effort": {"effort": "high", "summary": "detailed"},
-    }
-
-    responses_api_request = ResponsesAPIOptionalRequestParams()
-    handler._map_optional_params_to_responses_api_request(
-        optional_params, responses_api_request
-    )
-
-    # Verify reasoning_effort dict with summary was fully preserved
-    assert "reasoning" in responses_api_request
-    assert responses_api_request["reasoning"] == {"effort": "high", "summary": "detailed"}
-    assert responses_api_request["reasoning"]["effort"] == "high"
-    assert responses_api_request["reasoning"]["summary"] == "detailed"

tests/test_litellm/litellm_core_utils/test_core_helpers.py

@@ -1,6 +1,12 @@
 """Tests for litellm_core_utils.core_helpers module."""
 
-from litellm.litellm_core_utils.core_helpers import reconstruct_model_name
+import pytest
+
+from litellm.litellm_core_utils.core_helpers import (
+    _FINISH_REASON_MAP,
+    map_finish_reason,
+    reconstruct_model_name,
+)
 
 
 def test_reconstruct_model_name_prefers_deployment_value():
@@ -43,3 +49,102 @@ def test_reconstruct_model_name_returns_original_for_other_providers():
     )
 
     assert result == "claude-3-sonnet"
+
+
+# ---------------------------------------------------------------------------
+# map_finish_reason tests
+# ---------------------------------------------------------------------------
+
+VALID_OPENAI_FINISH_REASONS = {"stop", "length", "tool_calls", "function_call", "content_filter"}
+
+
+class TestMapFinishReasonAnthropic:
+    def test_stop_sequence(self):
+        assert map_finish_reason("stop_sequence") == "stop"
+
+    def test_end_turn(self):
+        assert map_finish_reason("end_turn") == "stop"
+
+    def test_max_tokens(self):
+        assert map_finish_reason("max_tokens") == "length"
+
+    def test_tool_use(self):
+        assert map_finish_reason("tool_use") == "tool_calls"
+
+    def test_compaction(self):
+        assert map_finish_reason("compaction") == "length"
+
+
+class TestMapFinishReasonGemini:
+    @pytest.mark.parametrize(
+        "gemini_reason,expected",
+        [
+            ("STOP", "stop"),
+            ("MAX_TOKENS", "length"),
+            ("SAFETY", "content_filter"),
+            ("RECITATION", "content_filter"),
+            ("FINISH_REASON_UNSPECIFIED", "stop"),
+            ("MALFORMED_FUNCTION_CALL", "stop"),
+            ("LANGUAGE", "content_filter"),
+            ("OTHER", "content_filter"),
+            ("BLOCKLIST", "content_filter"),
+            ("PROHIBITED_CONTENT", "content_filter"),
+            ("SPII", "content_filter"),
+            ("IMAGE_SAFETY", "content_filter"),
+            ("IMAGE_PROHIBITED_CONTENT", "content_filter"),
+            ("TOO_MANY_TOOL_CALLS", "stop"),
+            ("MALFORMED_RESPONSE", "stop"),
+        ],
+    )
+    def test_gemini_finish_reasons(self, gemini_reason, expected):
+        assert map_finish_reason(gemini_reason) == expected
+
+
+class TestMapFinishReasonCohere:
+    def test_complete(self):
+        assert map_finish_reason("COMPLETE") == "stop"
+
+    def test_error_toxic(self):
+        assert map_finish_reason("ERROR_TOXIC") == "content_filter"
+
+    def test_error(self):
+        assert map_finish_reason("ERROR") == "stop"
+
+
+class TestMapFinishReasonHuggingFace:
+    def test_eos_token(self):
+        assert map_finish_reason("eos_token") == "stop"
+
+    def test_eos(self):
+        assert map_finish_reason("eos") == "stop"
+
+
+class TestMapFinishReasonBedrock:
+    def test_guardrail_intervened(self):
+        assert map_finish_reason("guardrail_intervened") == "content_filter"
+
+
+class TestMapFinishReasonOpenAIPassthrough:
+    @pytest.mark.parametrize(
+        "reason", ["stop", "length", "tool_calls", "function_call", "content_filter"]
+    )
+    def test_openai_values_pass_through(self, reason):
+        assert map_finish_reason(reason) == reason
+
+
+class TestMapFinishReasonUnknown:
+    def test_unknown_value_defaults_to_stop(self):
+        assert map_finish_reason("some_unknown_value") == "stop"
+
+    def test_empty_string_defaults_to_stop(self):
+        assert map_finish_reason("") == "stop"
+
+
+class TestFinishReasonMapOutputsAreValid:
+    def test_all_mapped_values_are_valid_openai_reasons(self):
+        """Every value in _FINISH_REASON_MAP must be a valid OpenAI finish reason."""
+        for provider_reason, openai_reason in _FINISH_REASON_MAP.items():
+            assert openai_reason in VALID_OPENAI_FINISH_REASONS, (
+                f"Mapped value '{openai_reason}' (from '{provider_reason}') "
+                f"is not a valid OpenAI finish reason"
+            )

tests/test_litellm/llms/azure/chat/test_azure_gpt5_transformation.py

@@ -192,23 +192,6 @@ def test_azure_gpt5_1_series_temperature_handling(config: AzureOpenAIGPT5Config)
     assert params["temperature"] == 0.6
 
 
-def test_azure_gpt5_4_drops_reasoning_effort_when_tools_present(config: AzureOpenAIGPT5Config):
-    """Azure Chat Completions: gpt-5.4+ drops reasoning_effort when tools are present.
-
-    OpenAI routes tools+reasoning to Responses API; Azure does not, so we drop reasoning_effort.
-    """
-    tools = [{"type": "function", "function": {"name": "test", "description": "test"}}]
-    params = config.map_openai_params(
-        non_default_params={"reasoning_effort": "high", "tools": tools},
-        optional_params={},
-        model="gpt5_series/gpt-5.4",
-        drop_params=False,
-        api_version="2024-05-01-preview",
-    )
-    assert "reasoning_effort" not in params
-    assert params["tools"] == tools
-
-
 def test_azure_gpt5_reasoning_effort_none_error(config: AzureOpenAIGPT5Config):
     """Test that Azure GPT-5 (non-5.1) raises error for reasoning_effort='none' when drop_params=False."""
     with pytest.raises(litellm.utils.UnsupportedParamsError):

tests/test_litellm/llms/bedrock/chat/test_converse_transformation.py

@@ -43,29 +43,6 @@ def test_transform_usage():
     )
     assert openai_usage._cache_creation_input_tokens == usage["cacheWriteInputTokens"]
     assert openai_usage._cache_read_input_tokens == usage["cacheReadInputTokens"]
-    # completion_tokens_details should always be populated
-    assert openai_usage.completion_tokens_details is not None
-    assert openai_usage.completion_tokens_details.reasoning_tokens == 0
-    assert openai_usage.completion_tokens_details.text_tokens == usage["outputTokens"]
-
-
-def test_transform_usage_with_reasoning_content():
-    """Test that completion_tokens_details correctly tracks reasoning vs text tokens."""
-    usage = ConverseTokenUsageBlock(
-        **{
-            "inputTokens": 10,
-            "outputTokens": 100,
-            "totalTokens": 110,
-        }
-    )
-    config = AmazonConverseConfig()
-    reasoning_text = "Let me think about this step by step."
-    openai_usage = config._transform_usage(usage, reasoning_content=reasoning_text)
-    assert openai_usage.completion_tokens_details is not None
-    assert openai_usage.completion_tokens_details.reasoning_tokens > 0
-    assert openai_usage.completion_tokens_details.text_tokens == (
-        usage["outputTokens"] - openai_usage.completion_tokens_details.reasoning_tokens
-    )
 
 
 def test_transform_system_message():
@@ -3193,33 +3170,6 @@ def test_transform_request_with_output_config():
     assert result["outputConfig"]["textFormat"]["structure"]["jsonSchema"]["name"] == "TestSchema"
 
 
-def test_output_config_snake_case_stripped_from_bedrock_converse_request():
-    """Test that output_config (snake_case) is stripped from Bedrock Converse requests.
-
-    Bedrock Converse API doesn't support the output_config parameter (Anthropic-only).
-    Nova and other Converse models reject requests with extraneous output_config.
-    """
-    config = AmazonConverseConfig()
-    messages = [{"role": "user", "content": "test"}]
-    optional_params = {
-        "output_config": {"effort": "high"},
-    }
-
-    result = config._transform_request(
-        model="us.amazon.nova-pro-v1:0",
-        messages=messages,
-        optional_params=optional_params,
-        litellm_params={},
-        headers={},
-    )
-
-    # output_config must not appear in additionalModelRequestFields
-    additional = result.get("additionalModelRequestFields", {})
-    assert "output_config" not in additional, (
-        f"output_config should be stripped for Bedrock Converse, got: {list(additional.keys())}"
-    )
-
-
 def test_transform_response_native_structured_output():
     """Test response handling when model returns JSON as text content (native structured output)."""
     response_json = {

tests/test_litellm/llms/fireworks_ai/chat/test_fireworks_ai_chat_transformation.py

@@ -110,60 +110,6 @@ def test_get_supported_openai_params_reasoning_effort():
     assert "reasoning_effort" not in unsupported_params
 
 
-@pytest.mark.parametrize(
-    "api_base, expected_url_prefix",
-    [
-        (
-            "https://api.fireworks.ai/inference/v1",
-            "https://api.fireworks.ai/inference/v1/accounts/",
-        ),
-        (
-            "https://api.fireworks.ai/inference/v1/",
-            "https://api.fireworks.ai/inference/v1/accounts/",
-        ),
-        (
-            "https://custom-host.example.com/v1",
-            "https://custom-host.example.com/v1/accounts/",
-        ),
-        (
-            "https://custom-host.example.com/api",
-            "https://custom-host.example.com/api/v1/accounts/",
-        ),
-    ],
-    ids=["default", "trailing-slash", "custom-with-v1", "custom-without-v1"],
-)
-def test_get_models_url_no_double_v1(api_base, expected_url_prefix):
-    """Ensure get_models never produces a /v1/v1/ URL segment (fixes #23106)."""
-    config = FireworksAIConfig()
-    account_id = "fireworks"
-
-    mock_response = MagicMock()
-    mock_response.status_code = 200
-    mock_response.json.return_value = {
-        "models": [{"name": "accounts/fireworks/models/llama-v3-70b"}]
-    }
-
-    with (
-        patch("litellm.module_level_client.get", return_value=mock_response) as mock_get,
-        patch(
-            "litellm.llms.fireworks_ai.chat.transformation.get_secret_str",
-            side_effect=lambda key: {
-                "FIREWORKS_API_KEY": "test-key",
-                "FIREWORKS_API_BASE": api_base,
-                "FIREWORKS_ACCOUNT_ID": account_id,
-            }.get(key),
-        ),
-    ):
-        result = config.get_models(api_key="test-key", api_base=api_base)
-
-        called_url = mock_get.call_args.kwargs.get("url") or mock_get.call_args[1].get("url", "")
-        assert "/v1/v1/" not in called_url, f"Double /v1/ detected in URL: {called_url}"
-        assert called_url.startswith(expected_url_prefix), (
-            f"URL {called_url} does not start with {expected_url_prefix}"
-        )
-        assert result == ["fireworks_ai/accounts/fireworks/models/llama-v3-70b"]
-
-
 def test_transform_messages_helper_removes_provider_specific_fields():
     """
     Test that _transform_messages_helper removes provider_specific_fields from messages.

tests/test_litellm/llms/openai/chat/test_openai_gpt_transformation.py

@@ -13,7 +13,6 @@ from litellm.llms.openai.chat.gpt_transformation import (
     OpenAIChatCompletionStreamingHandler,
     OpenAIGPTConfig,
 )
-from litellm.llms.openai.chat.gpt_5_transformation import OpenAIGPT5Config
 
 
 class TestOpenAIGPTConfig:
@@ -325,195 +324,3 @@ class TestPromptCacheParams:
         )
         assert optional_params.get("prompt_cache_key") == "my-cache-key"
         assert optional_params.get("prompt_cache_retention") == "24h"
-
-
-class TestGPT5ReasoningEffortPreservation:
-    """Tests for GPT-5 reasoning_effort dict preservation for Responses API."""
-
-    def setup_method(self):
-        self.config = OpenAIGPT5Config()
-
-    def test_reasoning_effort_string_preserved(self):
-        """Test that reasoning_effort as string is preserved."""
-        non_default_params = {"reasoning_effort": "high"}
-        optional_params = {}
-        
-        self.config.map_openai_params(
-            non_default_params=non_default_params,
-            optional_params=optional_params,
-            model="gpt-5.4",
-            drop_params=False,
-        )
-        
-        # String format should be preserved
-        assert non_default_params.get("reasoning_effort") == "high"
-
-    def test_reasoning_effort_dict_with_only_effort_normalized(self):
-        """Test that reasoning_effort dict with only 'effort' key is normalized to string."""
-        non_default_params = {"reasoning_effort": {"effort": "high"}}
-        optional_params = {}
-        
-        self.config.map_openai_params(
-            non_default_params=non_default_params,
-            optional_params=optional_params,
-            model="gpt-5.4",
-            drop_params=False,
-        )
-        
-        # Dict with only 'effort' should be normalized to string
-        assert non_default_params.get("reasoning_effort") == "high"
-
-    def test_reasoning_effort_dict_with_summary_preserved(self):
-        """Test that reasoning_effort dict with 'summary' field is preserved for Responses API.
-        
-        Regression test for: User reported that summary field was being dropped when
-        routing to Responses API. The dict format with additional fields should be
-        preserved so it can be properly handled by the Responses API transformation.
-        """
-        non_default_params = {"reasoning_effort": {"effort": "high", "summary": "detailed"}}
-        optional_params = {}
-        
-        self.config.map_openai_params(
-            non_default_params=non_default_params,
-            optional_params=optional_params,
-            model="gpt-5.4",
-            drop_params=False,
-        )
-        
-        # Dict with additional fields should be preserved
-        assert non_default_params.get("reasoning_effort") == {"effort": "high", "summary": "detailed"}
-        assert isinstance(non_default_params.get("reasoning_effort"), dict)
-        assert non_default_params["reasoning_effort"]["effort"] == "high"
-        assert non_default_params["reasoning_effort"]["summary"] == "detailed"
-
-    def test_reasoning_effort_dict_with_generate_summary_preserved(self):
-        """Test that reasoning_effort dict with 'generate_summary' field is preserved."""
-        non_default_params = {"reasoning_effort": {"effort": "medium", "generate_summary": "auto"}}
-        optional_params = {}
-        
-        self.config.map_openai_params(
-            non_default_params=non_default_params,
-            optional_params=optional_params,
-            model="gpt-5.4",
-            drop_params=False,
-        )
-        
-        # Dict with additional fields should be preserved
-        assert non_default_params.get("reasoning_effort") == {"effort": "medium", "generate_summary": "auto"}
-        assert isinstance(non_default_params.get("reasoning_effort"), dict)
-
-    def test_reasoning_effort_dict_with_all_fields_preserved(self):
-        """Test that reasoning_effort dict with all fields is preserved."""
-        non_default_params = {
-            "reasoning_effort": {
-                "effort": "high",
-                "summary": "detailed",
-                "generate_summary": "concise"
-            }
-        }
-        optional_params = {}
-        
-        self.config.map_openai_params(
-            non_default_params=non_default_params,
-            optional_params=optional_params,
-            model="gpt-5.4",
-            drop_params=False,
-        )
-        
-        # Dict with all fields should be preserved
-        reasoning = non_default_params.get("reasoning_effort")
-        assert isinstance(reasoning, dict)
-        assert reasoning["effort"] == "high"
-        assert reasoning["summary"] == "detailed"
-        assert reasoning["generate_summary"] == "concise"
-
-    def test_reasoning_effort_dict_xhigh_triggers_validation(self):
-        """xhigh-dict: effective effort is extracted for model-support validation.
-        
-        When reasoning_effort={"effort": "xhigh", "summary": "detailed"} is passed to a model
-        that doesn't support xhigh (e.g. gpt-5.1), the xhigh guard must fire.
-        """
-        import litellm
-
-        non_default_params = {"reasoning_effort": {"effort": "xhigh", "summary": "detailed"}}
-        optional_params = {}
-
-        with pytest.raises(litellm.utils.UnsupportedParamsError):
-            self.config.map_openai_params(
-                non_default_params=non_default_params,
-                optional_params=optional_params,
-                model="gpt-5.1",
-                drop_params=False,
-            )
-
-    def test_reasoning_effort_dict_xhigh_dropped_when_requested(self):
-        """xhigh-dict with drop_params=True: reasoning_effort is dropped."""
-        non_default_params = {"reasoning_effort": {"effort": "xhigh", "summary": "detailed"}}
-        optional_params = {}
-
-        self.config.map_openai_params(
-            non_default_params=non_default_params,
-            optional_params=optional_params,
-            model="gpt-5.1",
-            drop_params=True,
-        )
-
-        assert "reasoning_effort" not in non_default_params
-
-    def test_reasoning_effort_dict_none_treated_as_none_for_tools(self):
-        """none-dict: {"effort": "none", "summary": "detailed"} is treated as effort=none.
-        
-        Tool-drop guard should NOT fire; reasoning_effort should be kept.
-        """
-        tools = [{"type": "function", "function": {"name": "test", "description": "test"}}]
-        non_default_params = {"reasoning_effort": {"effort": "none", "summary": "detailed"}, "tools": tools}
-        optional_params = {}
-
-        self.config.map_openai_params(
-            non_default_params=non_default_params,
-            optional_params=optional_params,
-            model="gpt-5.4",
-            drop_params=False,
-        )
-
-        assert non_default_params.get("reasoning_effort") == {"effort": "none", "summary": "detailed"}
-        assert non_default_params.get("tools") == tools
-
-    def test_reasoning_effort_dict_none_treated_as_none_for_sampling(self):
-        """none-dict: {"effort": "none", "summary": "detailed"} allows logprobs/top_p.
-        
-        Sampling-param guard should NOT fire; logprobs should be kept.
-        """
-        non_default_params = {
-            "reasoning_effort": {"effort": "none", "summary": "detailed"},
-            "logprobs": True,
-        }
-        optional_params = {}
-
-        self.config.map_openai_params(
-            non_default_params=non_default_params,
-            optional_params=optional_params,
-            model="gpt-5.1",
-            drop_params=False,
-        )
-
-        assert non_default_params.get("reasoning_effort") == {"effort": "none", "summary": "detailed"}
-        assert non_default_params.get("logprobs") is True
-
-    def test_reasoning_effort_dict_none_allows_temperature(self):
-        """none-dict: {"effort": "none", "summary": "detailed"} allows non-default temperature."""
-        non_default_params = {
-            "reasoning_effort": {"effort": "none", "summary": "detailed"},
-            "temperature": 0.5,
-        }
-        optional_params = {}
-
-        self.config.map_openai_params(
-            non_default_params=non_default_params,
-            optional_params=optional_params,
-            model="gpt-5.1",
-            drop_params=False,
-        )
-
-        assert optional_params.get("temperature") == 0.5
-        assert non_default_params.get("reasoning_effort") == {"effort": "none", "summary": "detailed"}

tests/test_litellm/llms/openai/test_gpt5_transformation.py

@@ -324,11 +324,10 @@ def test_gpt5_4_pro_allows_reasoning_effort_xhigh(config: OpenAIConfig):
     assert params["reasoning_effort"] == "xhigh"
 
 
-def test_gpt5_preserves_reasoning_effort_dict_with_summary(config: OpenAIConfig):
-    """Dict with summary/generate_summary is preserved for Responses API.
+def test_gpt5_normalizes_reasoning_effort_dict_to_string(config: OpenAIConfig):
+    """Chat completion API expects reasoning_effort as a string, not a dict.
 
     Config/deployments may pass Responses API format: {'effort': 'high', 'summary': 'detailed'}.
-    We preserve the full dict so it reaches the Responses API transformation.
     """
     params = config.map_openai_params(
         non_default_params={"reasoning_effort": {"effort": "high", "summary": "detailed"}},
@@ -336,82 +335,18 @@ def test_gpt5_preserves_reasoning_effort_dict_with_summary(config: OpenAIConfig)
         model="gpt-5.4",
         drop_params=False,
     )
-    assert params["reasoning_effort"] == {"effort": "high", "summary": "detailed"}
+    assert params["reasoning_effort"] == "high"
 
 
-def test_gpt5_xhigh_dict_triggers_validation(config: OpenAIConfig):
-    """Dict with effort='xhigh' triggers xhigh model-support validation.
-
-    Regression: when reasoning_effort is a dict, effective_effort must be used for
-    the xhigh guard so validation is not silently skipped.
-    """
-    with pytest.raises(litellm.utils.UnsupportedParamsError):
-        config.map_openai_params(
-            non_default_params={"reasoning_effort": {"effort": "xhigh", "summary": "detailed"}},
-            optional_params={},
-            model="gpt-5.1",
-            drop_params=False,
-        )
-
-
-def test_gpt5_xhigh_dict_accepted_for_supported_model(config: OpenAIConfig):
-    """Dict with effort='xhigh' passes through for gpt-5.4+."""
-    params = config.map_openai_params(
-        non_default_params={"reasoning_effort": {"effort": "xhigh", "summary": "detailed"}},
-        optional_params={},
-        model="gpt-5.4",
-        drop_params=False,
-    )
-    assert params["reasoning_effort"] == {"effort": "xhigh", "summary": "detailed"}
-
-
-def test_gpt5_none_dict_with_tools_no_tool_drop(config: OpenAIConfig):
-    """Dict with effort='none' and tools: no tool-drop, reasoning_effort preserved.
-
-    Regression: effective_effort='none' must be used for tool-drop guard so
-    {"effort": "none", "summary": "detailed"} is not incorrectly treated as non-none.
-    """
-    tools = [{"type": "function", "function": {"name": "test", "description": "test"}}]
-    params = config.map_openai_params(
-        non_default_params={"reasoning_effort": {"effort": "none", "summary": "detailed"}, "tools": tools},
-        optional_params={},
-        model="gpt-5.4",
-        drop_params=False,
-    )
-    assert params["reasoning_effort"] == {"effort": "none", "summary": "detailed"}
-    assert params["tools"] == tools
-
-
-def test_gpt5_none_dict_with_sampling_params_allowed(config: OpenAIConfig):
-    """Dict with effort='none' allows logprobs/top_p/top_logprobs.
-
-    Regression: effective_effort='none' must be used for sampling guard so
-    {"effort": "none", "summary": "detailed"} does not incorrectly trigger sampling errors.
-    """
-    params = config.map_openai_params(
-        non_default_params={
-            "reasoning_effort": {"effort": "none", "summary": "detailed"},
-            "logprobs": True,
-            "top_p": 0.9,
-        },
-        optional_params={},
-        model="gpt-5.1",
-        drop_params=False,
-    )
-    assert params["reasoning_effort"] == {"effort": "none", "summary": "detailed"}
-    assert params["logprobs"] is True
-    assert params["top_p"] == 0.9
-
-
-def test_gpt5_preserves_reasoning_effort_dict_with_summary_from_optional_params(config: OpenAIConfig):
-    """reasoning_effort dict with summary in optional_params is preserved."""
+def test_gpt5_normalizes_reasoning_effort_dict_from_optional_params(config: OpenAIConfig):
+    """reasoning_effort dict in optional_params (e.g. from model config) is normalized."""
     params = config.map_openai_params(
         non_default_params={},
         optional_params={"reasoning_effort": {"effort": "medium", "summary": "detailed"}},
         model="gpt-5.4",
         drop_params=False,
     )
-    assert params["reasoning_effort"] == {"effort": "medium", "summary": "detailed"}
+    assert params["reasoning_effort"] == "medium"
 
 
 def test_gpt5_4_drops_reasoning_effort_when_tools_present(config: OpenAIConfig):

tests/test_litellm/llms/openai/test_openai_image_edit_transformation.py

@@ -4,6 +4,7 @@ from typing import Dict
 import pytest
 
 from litellm import image_edit
+from litellm.images.utils import ImageEditRequestUtils
 from litellm.llms.openai.image_edit.transformation import OpenAIImageEditConfig
 from litellm.types.router import GenericLiteLLMParams
 
@@ -254,3 +255,50 @@ def test_transform_image_edit_request_with_mask_list(image_edit_config: OpenAIIm
     mask_file = next(f for f in files if f[0] == "mask")
     assert mask_file[1][1] == mask1  # Should be the first mask, not the second
 
+
+def test_transform_image_edit_request_with_input_fidelity(
+    image_edit_config: OpenAIImageEditConfig,
+):
+    """Test that input_fidelity is included in the data dict when provided"""
+    model = "gpt-image-1"
+    prompt = "Make the background blue"
+    image = b"fake_image_data"
+    image_edit_optional_request_params = {"input_fidelity": "high"}
+    litellm_params = GenericLiteLLMParams()
+    headers = {}
+
+    data, files = image_edit_config.transform_image_edit_request(
+        model=model,
+        prompt=prompt,
+        image=image,
+        image_edit_optional_request_params=image_edit_optional_request_params,
+        litellm_params=litellm_params,
+        headers=headers,
+    )
+
+    assert data["input_fidelity"] == "high"
+    assert data["model"] == model
+    assert data["prompt"] == prompt
+    assert "image" not in data
+
+
+def test_get_supported_openai_params_includes_input_fidelity(
+    image_edit_config: OpenAIImageEditConfig,
+):
+    """Test that input_fidelity is in the supported params list"""
+    supported = image_edit_config.get_supported_openai_params(model="gpt-image-1")
+    assert "input_fidelity" in supported
+
+
+def test_input_fidelity_passes_through_optional_param_filter():
+    """Test that input_fidelity is not dropped by get_requested_image_edit_optional_param"""
+    params = {
+        "input_fidelity": "low",
+        "quality": "high",
+        "unknown_param": "should_be_dropped",
+    }
+    filtered = ImageEditRequestUtils.get_requested_image_edit_optional_param(params)
+    assert filtered["input_fidelity"] == "low"
+    assert filtered["quality"] == "high"
+    assert "unknown_param" not in filtered
+

tests/test_litellm/llms/openai_like/responses/test_openai_like_responses.py

@@ -0,0 +1,341 @@
+"""
+Tests for OpenAI-like Responses API support in the JSON provider system.
+"""
+
+import os
+import sys
+from unittest.mock import patch
+
+import pytest
+
+sys.path.insert(
+    0, os.path.abspath(os.path.join(os.path.dirname(__file__), "../../../../.."))
+)
+
+
+class TestSimpleProviderConfigSupportedEndpoints:
+    """Test the supported_endpoints field on SimpleProviderConfig."""
+
+    def test_default_supported_endpoints(self):
+        """supported_endpoints defaults to [] (chat always enabled, nothing else)"""
+        from litellm.llms.openai_like.json_loader import SimpleProviderConfig
+
+        config = SimpleProviderConfig("test", {"base_url": "https://example.com", "api_key_env": "TEST_KEY"})
+        assert config.supported_endpoints == []
+
+    def test_custom_supported_endpoints(self):
+        """supported_endpoints can be set explicitly"""
+        from litellm.llms.openai_like.json_loader import SimpleProviderConfig
+
+        config = SimpleProviderConfig(
+            "test",
+            {
+                "base_url": "https://example.com",
+                "api_key_env": "TEST_KEY",
+                "supported_endpoints": ["/v1/chat/completions", "/v1/responses"],
+            },
+        )
+        assert "/v1/responses" in config.supported_endpoints
+        assert "/v1/chat/completions" in config.supported_endpoints
+
+    def test_responses_only_endpoint(self):
+        """A provider can support only responses"""
+        from litellm.llms.openai_like.json_loader import SimpleProviderConfig
+
+        config = SimpleProviderConfig(
+            "test",
+            {
+                "base_url": "https://example.com",
+                "api_key_env": "TEST_KEY",
+                "supported_endpoints": ["/v1/responses"],
+            },
+        )
+        assert config.supported_endpoints == ["/v1/responses"]
+
+
+class TestJSONProviderRegistryResponsesAPI:
+    """Test supports_responses_api on JSONProviderRegistry."""
+
+    def test_existing_provider_no_responses(self):
+        """Existing providers without supported_endpoints don't support responses"""
+        from litellm.llms.openai_like.json_loader import JSONProviderRegistry
+
+        # publicai has no supported_endpoints in JSON, defaults to []
+        assert JSONProviderRegistry.supports_responses_api("publicai") is False
+
+    def test_nonexistent_provider(self):
+        """Non-existent provider returns False"""
+        from litellm.llms.openai_like.json_loader import JSONProviderRegistry
+
+        assert JSONProviderRegistry.supports_responses_api("nonexistent_provider_xyz") is False
+
+    def test_provider_with_responses_endpoint(self):
+        """A provider with /v1/responses in supported_endpoints returns True"""
+        from litellm.llms.openai_like.json_loader import (
+            JSONProviderRegistry,
+            SimpleProviderConfig,
+        )
+
+        # Temporarily inject a test provider
+        test_config = SimpleProviderConfig(
+            "test_responses_provider",
+            {
+                "base_url": "https://test.example.com",
+                "api_key_env": "TEST_API_KEY",
+                "supported_endpoints": ["/v1/chat/completions", "/v1/responses"],
+            },
+        )
+        JSONProviderRegistry._providers["test_responses_provider"] = test_config
+        try:
+            assert JSONProviderRegistry.supports_responses_api("test_responses_provider") is True
+        finally:
+            del JSONProviderRegistry._providers["test_responses_provider"]
+
+
+class TestCreateResponsesConfigClass:
+    """Test dynamic responses config class generation."""
+
+    def _make_test_provider(self):
+        from litellm.llms.openai_like.json_loader import SimpleProviderConfig
+
+        return SimpleProviderConfig(
+            "test_resp",
+            {
+                "base_url": "https://api.testresp.com/v1",
+                "api_key_env": "TEST_RESP_API_KEY",
+                "api_base_env": "TEST_RESP_API_BASE",
+                "supported_endpoints": ["/v1/responses"],
+            },
+        )
+
+    def test_generated_class_custom_llm_provider(self):
+        """Generated class returns the provider slug"""
+        from litellm.llms.openai_like.dynamic_config import (
+            create_responses_config_class,
+        )
+
+        provider = self._make_test_provider()
+        config_cls = create_responses_config_class(provider)
+        config = config_cls()
+        assert config.custom_llm_provider == "test_resp"
+
+    def test_generated_class_get_complete_url(self):
+        """Generated class builds correct responses URL"""
+        from litellm.llms.openai_like.dynamic_config import (
+            create_responses_config_class,
+        )
+
+        provider = self._make_test_provider()
+        config_cls = create_responses_config_class(provider)
+        config = config_cls()
+
+        url = config.get_complete_url(api_base=None, litellm_params={})
+        assert url == "https://api.testresp.com/v1/responses"
+
+    def test_generated_class_get_complete_url_with_override(self):
+        """api_base override takes precedence"""
+        from litellm.llms.openai_like.dynamic_config import (
+            create_responses_config_class,
+        )
+
+        provider = self._make_test_provider()
+        config_cls = create_responses_config_class(provider)
+        config = config_cls()
+
+        url = config.get_complete_url(api_base="https://custom.api.com/v1", litellm_params={})
+        assert url == "https://custom.api.com/v1/responses"
+
+    def test_generated_class_get_complete_url_strips_trailing_slash(self):
+        """Trailing slashes are stripped from base URL"""
+        from litellm.llms.openai_like.dynamic_config import (
+            create_responses_config_class,
+        )
+
+        provider = self._make_test_provider()
+        config_cls = create_responses_config_class(provider)
+        config = config_cls()
+
+        url = config.get_complete_url(api_base="https://custom.api.com/v1/", litellm_params={})
+        assert url == "https://custom.api.com/v1/responses"
+
+    def test_generated_class_validate_environment(self):
+        """validate_environment sets Authorization header from env"""
+        from litellm.llms.openai_like.dynamic_config import (
+            create_responses_config_class,
+        )
+
+        provider = self._make_test_provider()
+        config_cls = create_responses_config_class(provider)
+        config = config_cls()
+
+        with patch(
+            "litellm.llms.openai_like.dynamic_config.get_secret_str",
+            return_value="sk-test-key-123",
+        ):
+            headers = config.validate_environment(headers={}, model="test-model", litellm_params=None)
+        assert headers["Authorization"] == "Bearer sk-test-key-123"
+
+    def test_generated_class_validate_environment_litellm_params_override(self):
+        """api_key from litellm_params takes precedence over env"""
+        from litellm.llms.openai_like.dynamic_config import (
+            create_responses_config_class,
+        )
+        from litellm.types.router import GenericLiteLLMParams
+
+        provider = self._make_test_provider()
+        config_cls = create_responses_config_class(provider)
+        config = config_cls()
+
+        litellm_params = GenericLiteLLMParams(api_key="sk-override-key")
+        headers = config.validate_environment(
+            headers={}, model="test-model", litellm_params=litellm_params
+        )
+        assert headers["Authorization"] == "Bearer sk-override-key"
+
+    def test_generated_class_inherits_openai_responses_methods(self):
+        """Generated class inherits OpenAI Responses API transformation methods"""
+        from litellm.llms.openai.responses.transformation import (
+            OpenAIResponsesAPIConfig,
+        )
+        from litellm.llms.openai_like.dynamic_config import (
+            create_responses_config_class,
+        )
+
+        provider = self._make_test_provider()
+        config_cls = create_responses_config_class(provider)
+        config = config_cls()
+
+        # Should have inherited methods from OpenAIResponsesAPIConfig
+        assert hasattr(config, "get_supported_openai_params")
+        assert hasattr(config, "map_openai_params")
+        assert hasattr(config, "transform_responses_api_request")
+        assert hasattr(config, "transform_response_api_response")
+        assert hasattr(config, "transform_streaming_response")
+
+        # Verify inheritance chain
+        assert isinstance(config, OpenAIResponsesAPIConfig)
+
+    def test_generated_class_get_complete_url_uses_api_base_env(self):
+        """get_complete_url falls back to api_base_env when api_base is None"""
+        from litellm.llms.openai_like.dynamic_config import (
+            create_responses_config_class,
+        )
+
+        provider = self._make_test_provider()
+        config_cls = create_responses_config_class(provider)
+        config = config_cls()
+
+        with patch(
+            "litellm.llms.openai_like.dynamic_config.get_secret_str",
+            return_value="https://env-override.example.com/v1",
+        ):
+            url = config.get_complete_url(api_base=None, litellm_params={})
+        assert url == "https://env-override.example.com/v1/responses"
+
+
+class TestProviderConfigManagerResponsesAPI:
+    """Test that ProviderConfigManager integrates JSON responses providers."""
+
+    def test_json_provider_with_responses_returns_config(self):
+        """A JSON provider with /v1/responses returns a responses config"""
+        from litellm.llms.openai_like.json_loader import (
+            JSONProviderRegistry,
+            SimpleProviderConfig,
+        )
+        from litellm.utils import ProviderConfigManager
+
+        test_config = SimpleProviderConfig(
+            "test_pcm_resp",
+            {
+                "base_url": "https://api.testpcm.com/v1",
+                "api_key_env": "TEST_PCM_KEY",
+                "supported_endpoints": ["/v1/responses"],
+            },
+        )
+        JSONProviderRegistry._providers["test_pcm_resp"] = test_config
+        try:
+            config = ProviderConfigManager.get_provider_responses_api_config(
+                provider="test_pcm_resp",
+                model="some-model",
+            )
+            assert config is not None
+            assert config.custom_llm_provider == "test_pcm_resp"
+        finally:
+            del JSONProviderRegistry._providers["test_pcm_resp"]
+
+    def test_json_provider_without_responses_returns_none(self):
+        """A JSON provider without /v1/responses returns None"""
+        from litellm.utils import ProviderConfigManager
+
+        # publicai only supports chat completions
+        config = ProviderConfigManager.get_provider_responses_api_config(
+            provider="publicai",
+            model="some-model",
+        )
+        assert config is None
+
+    def test_unknown_provider_returns_none(self):
+        """A completely unknown provider returns None"""
+        from litellm.utils import ProviderConfigManager
+
+        config = ProviderConfigManager.get_provider_responses_api_config(
+            provider="totally_unknown_provider_xyz",
+            model="some-model",
+        )
+        assert config is None
+
+    def test_standard_providers_still_work(self):
+        """Existing enum-based providers still resolve correctly"""
+        from litellm.types.utils import LlmProviders
+        from litellm.utils import ProviderConfigManager
+
+        config = ProviderConfigManager.get_provider_responses_api_config(
+            provider=LlmProviders.OPENAI,
+            model="gpt-4o",
+        )
+        assert config is not None
+
+    def test_standard_provider_as_string_still_works(self):
+        """Passing 'openai' as a string also works"""
+        from litellm.utils import ProviderConfigManager
+
+        config = ProviderConfigManager.get_provider_responses_api_config(
+            provider="openai",
+            model="gpt-4o",
+        )
+        assert config is not None
+
+    def test_python_class_takes_priority_over_json(self):
+        """If a provider has both a Python class and JSON config, Python wins"""
+        from litellm.llms.openai_like.json_loader import (
+            JSONProviderRegistry,
+            SimpleProviderConfig,
+        )
+        from litellm.llms.perplexity.responses.transformation import (
+            PerplexityResponsesConfig,
+        )
+        from litellm.utils import ProviderConfigManager
+
+        # Inject perplexity into JSON registry with responses support
+        test_config = SimpleProviderConfig(
+            "perplexity",
+            {
+                "base_url": "https://api.perplexity.ai",
+                "api_key_env": "PERPLEXITY_API_KEY",
+                "supported_endpoints": ["/v1/responses"],
+            },
+        )
+        original = JSONProviderRegistry._providers.get("perplexity")
+        JSONProviderRegistry._providers["perplexity"] = test_config
+        try:
+            config = ProviderConfigManager.get_provider_responses_api_config(
+                provider="perplexity",
+                model="some-model",
+            )
+            # Should be the Python class, not the JSON-generated one
+            assert isinstance(config, PerplexityResponsesConfig)
+        finally:
+            if original is not None:
+                JSONProviderRegistry._providers["perplexity"] = original
+            else:
+                del JSONProviderRegistry._providers["perplexity"]

tests/test_litellm/llms/perplexity/responses/test_perplexity_responses_transformation.py

@@ -7,11 +7,17 @@ transformations for the Agent API (Responses API).
 Source: litellm/llms/perplexity/responses/transformation.py
 """
 
+import json
 import os
 import sys
 
+import httpx
+import pytest
+
 sys.path.insert(0, os.path.abspath("../../../../.."))
 
+from litellm.litellm_core_utils.litellm_logging import Logging as LiteLLMLoggingObj
+from litellm.llms.base_llm.chat.transformation import BaseLLMException
 from litellm.llms.perplexity.responses.transformation import PerplexityResponsesConfig
 from litellm.types.llms.openai import ResponsesAPIOptionalRequestParams
 from litellm.types.utils import LlmProviders
@@ -260,10 +266,12 @@ class TestPerplexityResponsesTransformation:
         assert result.get("user") == "user_456"
 
     def test_all_supported_params_declared(self):
-        """get_supported_openai_params returns complete list"""
+        """get_supported_openai_params returns Perplexity-specific restricted list"""
         config = PerplexityResponsesConfig()
         supported = config.get_supported_openai_params("perplexity/openai/gpt-5.2")
 
+        # Perplexity Responses API supports a restricted set of params
+        # Ref: https://docs.perplexity.ai/api-reference/responses-post
         expected = [
             "max_output_tokens",
             "stream",
@@ -271,68 +279,46 @@ class TestPerplexityResponsesTransformation:
             "top_p",
             "tools",
             "reasoning",
-            "preset",
             "instructions",
             "models",
-            "tool_choice",
-            "parallel_tool_calls",
-            "max_tool_calls",
-            "text",
-            "previous_response_id",
-            "store",
-            "background",
-            "truncation",
-            "metadata",
-            "safety_identifier",
-            "user",
-            "stream_options",
-            "top_logprobs",
-            "prompt_cache_key",
-            "frequency_penalty",
-            "presence_penalty",
-            "service_tier",
         ]
 
         for param in expected:
             assert param in supported, f"Missing supported param: {param}"
 
-    def test_cost_transformation(self):
-        """Perplexity cost dict to OpenAI float"""
-        config = PerplexityResponsesConfig()
+    def test_cost_dict_to_float_via_validator(self):
+        """Perplexity cost dict is parsed by generic ResponseAPIUsage.parse_cost validator"""
+        from litellm.types.llms.openai import ResponseAPIUsage
 
-        usage_data = {
-            "input_tokens": 100,
-            "output_tokens": 200,
-            "total_tokens": 300,
-            "cost": {
+        usage = ResponseAPIUsage(
+            input_tokens=100,
+            output_tokens=200,
+            total_tokens=300,
+            cost={
                 "currency": "USD",
                 "input_cost": 0.0001,
                 "output_cost": 0.0002,
                 "total_cost": 0.0003,
             },
-        }
+        )
 
-        result = config._transform_usage(usage_data)
+        assert usage.input_tokens == 100
+        assert usage.output_tokens == 200
+        assert usage.total_tokens == 300
+        assert usage.cost == 0.0003
 
-        assert result["input_tokens"] == 100
-        assert result["output_tokens"] == 200
-        assert result["total_tokens"] == 300
-        assert result["cost"] == 0.0003
+    def test_cost_float_passthrough_via_validator(self):
+        """Cost already float passes through validator unchanged"""
+        from litellm.types.llms.openai import ResponseAPIUsage
 
-    def test_cost_transformation_float_passthrough(self):
-        """Cost already float passes through"""
-        config = PerplexityResponsesConfig()
+        usage = ResponseAPIUsage(
+            input_tokens=100,
+            output_tokens=200,
+            total_tokens=300,
+            cost=0.0005,
+        )
 
-        usage_data = {
-            "input_tokens": 100,
-            "output_tokens": 200,
-            "total_tokens": 300,
-            "cost": 0.0005,
-        }
-
-        result = config._transform_usage(usage_data)
-
-        assert result["cost"] == 0.0005
+        assert usage.cost == 0.0005
 
     def test_preset_handling(self):
         """Preset model names work"""
@@ -350,6 +336,85 @@ class TestPerplexityResponsesTransformation:
         assert data["input"] == "What is AI?"
         assert "temperature" in data
 
+    def test_preset_handling_list_input(self):
+        """Preset with list input preserves type field"""
+        config = PerplexityResponsesConfig()
+
+        list_input = [
+            {"type": "message", "role": "user", "content": "What is AI?"},
+        ]
+
+        data = config.transform_responses_api_request(
+            model="preset/pro-search",
+            input=list_input,
+            response_api_optional_request_params={"temperature": 0.7},
+            litellm_params={},
+            headers={},
+        )
+
+        assert data["preset"] == "pro-search"
+        assert isinstance(data["input"], list)
+        assert data["input"][0]["type"] == "message"
+        assert data["input"][0]["role"] == "user"
+
+    def test_non_preset_list_input(self):
+        """Non-preset with list input preserves type field"""
+        config = PerplexityResponsesConfig()
+
+        list_input = [
+            {"type": "message", "role": "user", "content": "Hello"},
+        ]
+
+        data = config.transform_responses_api_request(
+            model="openai/gpt-5.2",
+            input=list_input,
+            response_api_optional_request_params={},
+            litellm_params={},
+            headers={},
+        )
+
+        assert data["model"] == "openai/gpt-5.2"
+        assert isinstance(data["input"], list)
+        assert data["input"][0]["type"] == "message"
+
+    def test_list_input_adds_type_message_when_missing(self):
+        """Input items without type get type='message' added automatically"""
+        config = PerplexityResponsesConfig()
+
+        list_input = [
+            {"role": "user", "content": "Hello"},
+        ]
+
+        data = config.transform_responses_api_request(
+            model="openai/gpt-5.2",
+            input=list_input,
+            response_api_optional_request_params={},
+            litellm_params={},
+            headers={},
+        )
+
+        assert data["input"][0]["type"] == "message"
+        assert data["input"][0]["role"] == "user"
+        assert data["input"][0]["content"] == "Hello"
+
+    def test_list_input_preserves_existing_type(self):
+        """Input items that already have type are not modified"""
+        config = PerplexityResponsesConfig()
+
+        list_input = [
+            {"type": "function_call_output", "call_id": "123", "output": "{}"},
+        ]
+
+        data = config.transform_responses_api_request(
+            model="openai/gpt-5.2",
+            input=list_input,
+            response_api_optional_request_params={},
+            litellm_params={},
+            headers={},
+        )
+
+        assert data["input"][0]["type"] == "function_call_output"
+
     def test_get_complete_url(self):
         """Correct endpoint URL"""
         config = PerplexityResponsesConfig()
@@ -379,3 +444,149 @@ class TestPerplexityResponsesTransformation:
         assert config is not None
         assert isinstance(config, PerplexityResponsesConfig)
         assert config.custom_llm_provider == LlmProviders.PERPLEXITY
+
+    def test_failed_status_raises_exception(self):
+        """Perplexity HTTP 200 with status:'failed' must raise BaseLLMException"""
+        config = PerplexityResponsesConfig()
+
+        failed_body = {
+            "status": "failed",
+            "error": {"message": "Model quota exceeded"},
+        }
+
+        raw_response = httpx.Response(
+            status_code=200,
+            json=failed_body,
+            request=httpx.Request("POST", "https://api.perplexity.ai/v1/responses"),
+        )
+
+        logging_obj = LiteLLMLoggingObj(
+            model="perplexity/openai/gpt-5.2",
+            messages=[],
+            stream=False,
+            call_type="responses",
+            start_time=None,
+            litellm_call_id="test",
+            function_id="test",
+        )
+
+        with pytest.raises(BaseLLMException) as exc_info:
+            config.transform_response_api_response(
+                model="perplexity/openai/gpt-5.2",
+                raw_response=raw_response,
+                logging_obj=logging_obj,
+            )
+
+        assert "Model quota exceeded" in str(exc_info.value.message)
+
+    def test_successful_response_passes_through(self):
+        """Normal completed response delegates to base OpenAI handler"""
+        config = PerplexityResponsesConfig()
+
+        success_body = {
+            "id": "resp_123",
+            "object": "response",
+            "created_at": 1700000000,
+            "status": "completed",
+            "model": "openai/gpt-5.2",
+            "output": [
+                {
+                    "type": "message",
+                    "id": "msg_123",
+                    "role": "assistant",
+                    "status": "completed",
+                    "content": [
+                        {"type": "output_text", "text": "Hello!", "annotations": []}
+                    ],
+                }
+            ],
+            "usage": {
+                "input_tokens": 10,
+                "output_tokens": 5,
+                "total_tokens": 15,
+            },
+        }
+
+        raw_response = httpx.Response(
+            status_code=200,
+            json=success_body,
+            request=httpx.Request("POST", "https://api.perplexity.ai/v1/responses"),
+        )
+
+        logging_obj = LiteLLMLoggingObj(
+            model="perplexity/openai/gpt-5.2",
+            messages=[],
+            stream=False,
+            call_type="responses",
+            start_time=None,
+            litellm_call_id="test",
+            function_id="test",
+        )
+
+        response = config.transform_response_api_response(
+            model="perplexity/openai/gpt-5.2",
+            raw_response=raw_response,
+            logging_obj=logging_obj,
+        )
+
+        assert response.id == "resp_123"
+        assert response.status == "completed"
+
+    def test_streaming_cost_dict_to_float_via_validator(self):
+        """Cost dict in a streaming response.completed chunk is converted to float
+        end-to-end through transform_streaming_response via pydantic's recursive
+        construction of ResponsesAPIResponse → ResponseAPIUsage.parse_cost."""
+        config = PerplexityResponsesConfig()
+
+        completed_chunk = {
+            "type": "response.completed",
+            "response": {
+                "id": "resp_streaming_123",
+                "object": "response",
+                "created_at": 1700000000,
+                "status": "completed",
+                "model": "openai/gpt-5.2",
+                "output": [
+                    {
+                        "type": "message",
+                        "id": "msg_123",
+                        "role": "assistant",
+                        "status": "completed",
+                        "content": [
+                            {"type": "output_text", "text": "Hello!", "annotations": []}
+                        ],
+                    }
+                ],
+                "usage": {
+                    "input_tokens": 100,
+                    "output_tokens": 200,
+                    "total_tokens": 300,
+                    "cost": {
+                        "currency": "USD",
+                        "input_cost": 0.0001,
+                        "output_cost": 0.0002,
+                        "total_cost": 0.0003,
+                    },
+                },
+            },
+        }
+
+        logging_obj = LiteLLMLoggingObj(
+            model="perplexity/openai/gpt-5.2",
+            messages=[],
+            stream=True,
+            call_type="responses",
+            start_time=None,
+            litellm_call_id="test",
+            function_id="test",
+        )
+
+        result = config.transform_streaming_response(
+            model="perplexity/openai/gpt-5.2",
+            parsed_chunk=completed_chunk,
+            logging_obj=logging_obj,
+        )
+
+        assert result.type == "response.completed"
+        assert result.response.usage.cost == 0.0003
+        assert isinstance(result.response.usage.cost, float)

tests/test_litellm/llms/sagemaker/test_sagemaker_embedding_role_assumption.py

@@ -1,243 +0,0 @@
-"""
-Test cases for SageMaker embedding role assumption support
-
-This module tests that the SageMaker embedding handler properly supports
-AWS IAM role assumption via aws_role_name and aws_session_name parameters,
-matching the behavior of the completion handler.
-"""
-
-import json
-import os
-import sys
-from datetime import timezone
-from unittest.mock import MagicMock, call, patch
-
-sys.path.insert(0, os.path.abspath("../../../../.."))
-
-from botocore.credentials import Credentials
-
-from litellm.llms.sagemaker.completion.handler import SagemakerLLM
-from litellm.types.utils import EmbeddingResponse
-
-
-class TestSagemakerEmbeddingRoleAssumption:
-    """Test that SageMaker embedding supports role assumption like completion does"""
-
-    def setup_method(self):
-        self.sagemaker_llm = SagemakerLLM()
-
-    def test_embedding_uses_load_credentials(self):
-        """
-        Test that embedding() calls _load_credentials() to support role assumption.
-        This ensures aws_role_name and aws_session_name parameters are properly handled.
-        """
-        # Mock credentials that would be returned after role assumption
-        mock_credentials = Credentials(
-            access_key="assumed-access-key",
-            secret_key="assumed-secret-key",
-            token="assumed-session-token",
-        )
-
-        # Mock the SageMaker client response
-        mock_sagemaker_client = MagicMock()
-        mock_sagemaker_client.invoke_endpoint.return_value = {
-            "Body": MagicMock(
-                read=MagicMock(return_value=json.dumps({"embedding": [[0.1, 0.2, 0.3]]}).encode())
-            )
-        }
-
-        # Mock boto3.Session to return our mock client
-        mock_session = MagicMock()
-        mock_session.client.return_value = mock_sagemaker_client
-
-        with patch.object(
-            self.sagemaker_llm, "_load_credentials", return_value=(mock_credentials, "us-east-1")
-        ) as mock_load_creds, patch("boto3.Session", return_value=mock_session):
-
-            # Create mock logging object
-            mock_logging = MagicMock()
-
-            optional_params = {
-                "aws_role_name": "arn:aws:iam::123456789012:role/TestRole",
-                "aws_session_name": "test-session",
-            }
-
-            self.sagemaker_llm.embedding(
-                model="test-endpoint",
-                input=["hello world"],
-                model_response=EmbeddingResponse(),
-                print_verbose=print,
-                encoding=None,
-                logging_obj=mock_logging,
-                optional_params=optional_params,
-            )
-
-            # Verify _load_credentials was called with the optional_params
-            mock_load_creds.assert_called_once()
-
-            # Verify boto3.Session was created with the assumed credentials
-            mock_session_calls = mock_session.client.call_args_list
-            assert len(mock_session_calls) == 1
-            assert mock_session_calls[0] == call(service_name="sagemaker-runtime")
-
-    def test_embedding_role_assumption_with_sts(self):
-        """
-        Test the full role assumption flow for embeddings, similar to completion.
-        Verifies that STS assume_role is called when aws_role_name is provided.
-        """
-        # Mock the STS client for role assumption
-        mock_sts_client = MagicMock()
-
-        # Mock the STS response with proper expiration handling
-        mock_expiry = MagicMock()
-        mock_expiry.tzinfo = timezone.utc
-        time_diff = MagicMock()
-        time_diff.total_seconds.return_value = 3600
-        mock_expiry.__sub__ = MagicMock(return_value=time_diff)
-
-        mock_sts_response = {
-            "Credentials": {
-                "AccessKeyId": "assumed-access-key",
-                "SecretAccessKey": "assumed-secret-key",
-                "SessionToken": "assumed-session-token",
-                "Expiration": mock_expiry,
-            }
-        }
-        mock_sts_client.assume_role.return_value = mock_sts_response
-
-        # Mock the SageMaker client response
-        mock_sagemaker_client = MagicMock()
-        mock_sagemaker_client.invoke_endpoint.return_value = {
-            "Body": MagicMock(
-                read=MagicMock(return_value=json.dumps({"embedding": [[0.1, 0.2, 0.3]]}).encode())
-            )
-        }
-
-        # Mock boto3.Session for SageMaker client creation
-        mock_session = MagicMock()
-        mock_session.client.return_value = mock_sagemaker_client
-
-        def mock_boto3_client(service_name, **kwargs):
-            if service_name == "sts":
-                return mock_sts_client
-            return mock_sagemaker_client
-
-        with patch("boto3.client", side_effect=mock_boto3_client), \
-             patch("boto3.Session", return_value=mock_session):
-
-            mock_logging = MagicMock()
-
-            optional_params = {
-                "aws_role_name": "arn:aws:iam::123456789012:role/CrossAccountRole",
-                "aws_session_name": "litellm-embedding-session",
-                "aws_region_name": "us-east-1",
-            }
-
-            self.sagemaker_llm.embedding(
-                model="test-endpoint",
-                input=["hello world"],
-                model_response=EmbeddingResponse(),
-                print_verbose=print,
-                encoding=None,
-                logging_obj=mock_logging,
-                optional_params=optional_params,
-            )
-
-            # Verify STS assume_role was called with correct parameters
-            mock_sts_client.assume_role.assert_called_once()
-            call_args = mock_sts_client.assume_role.call_args
-            assert call_args[1]["RoleArn"] == "arn:aws:iam::123456789012:role/CrossAccountRole"
-            assert call_args[1]["RoleSessionName"] == "litellm-embedding-session"
-
-    def test_embedding_without_role_assumption(self):
-        """
-        Test that embedding works without role assumption when aws_role_name is not provided.
-        Should use default credentials from environment/instance profile.
-        """
-        # Mock the SageMaker client response
-        mock_sagemaker_client = MagicMock()
-        mock_sagemaker_client.invoke_endpoint.return_value = {
-            "Body": MagicMock(
-                read=MagicMock(return_value=json.dumps({"embedding": [[0.1, 0.2, 0.3]]}).encode())
-            )
-        }
-
-        mock_session = MagicMock()
-        mock_session.client.return_value = mock_sagemaker_client
-
-        # Mock credentials returned from environment
-        mock_credentials = Credentials(
-            access_key="env-access-key",
-            secret_key="env-secret-key",
-            token=None,
-        )
-
-        with patch.object(
-            self.sagemaker_llm, "_load_credentials", return_value=(mock_credentials, "us-west-2")
-        ), patch("boto3.Session", return_value=mock_session):
-
-            mock_logging = MagicMock()
-
-            # No aws_role_name provided
-            optional_params = {
-                "aws_region_name": "us-west-2",
-            }
-
-            result = self.sagemaker_llm.embedding(
-                model="test-endpoint",
-                input=["hello world"],
-                model_response=EmbeddingResponse(),
-                print_verbose=print,
-                encoding=None,
-                logging_obj=mock_logging,
-                optional_params=optional_params,
-            )
-
-            # Should still work and return embeddings
-            assert result is not None
-
-    def test_embedding_session_created_with_assumed_credentials(self):
-        """
-        Test that boto3.Session is created with the credentials from role assumption.
-        This verifies the credentials flow from _load_credentials to the SageMaker client.
-        """
-        mock_credentials = Credentials(
-            access_key="assumed-key",
-            secret_key="assumed-secret",
-            token="assumed-token",
-        )
-
-        mock_sagemaker_client = MagicMock()
-        mock_sagemaker_client.invoke_endpoint.return_value = {
-            "Body": MagicMock(
-                read=MagicMock(return_value=json.dumps({"embedding": [[0.1, 0.2, 0.3]]}).encode())
-            )
-        }
-
-        with patch.object(
-            self.sagemaker_llm, "_load_credentials", return_value=(mock_credentials, "us-east-1")
-        ), patch("boto3.Session") as mock_session_class:
-
-            mock_session = MagicMock()
-            mock_session.client.return_value = mock_sagemaker_client
-            mock_session_class.return_value = mock_session
-
-            mock_logging = MagicMock()
-
-            self.sagemaker_llm.embedding(
-                model="test-endpoint",
-                input=["hello world"],
-                model_response=EmbeddingResponse(),
-                print_verbose=print,
-                encoding=None,
-                logging_obj=mock_logging,
-                optional_params={},
-            )
-
-            # Verify Session was created with the assumed credentials
-            mock_session_class.assert_called_once_with(
-                aws_access_key_id="assumed-key",
-                aws_secret_access_key="assumed-secret",
-                aws_session_token="assumed-token",
-                region_name="us-east-1",
-            )

tests/test_litellm/llms/snowflake/chat/test_snowflake_chat_transformation.py

@@ -105,7 +105,10 @@ class TestSnowflakeToolTransformation:
 
     def test_transform_request_with_string_tool_choice(self):
         """
-        Test that string tool_choice values pass through unchanged.
+        Test that string tool_choice values are transformed to Snowflake object format.
+
+        Snowflake requires tool_choice to be an object, not a string.
+        Ref: https://docs.snowflake.com/en/developer-guide/snowflake-rest-api/reference/cortex-inference#post--api-v2-cortex-inference-complete-req-body-schema
         """
         config = SnowflakeConfig()
 
@@ -120,7 +123,8 @@ class TestSnowflakeToolTransformation:
                 headers={},
             )
 
-            assert transformed_request["tool_choice"] == value
+            # Snowflake requires object format: {"type": "auto"} not string "auto"
+            assert transformed_request["tool_choice"] == {"type": value}
 
     def test_transform_response_with_tool_calls(self):
         """

tests/test_litellm/llms/vertex_ai/gemini/test_vertex_ai_gemini_transformation.py

@@ -128,75 +128,6 @@ def test_vertex_ai_includes_labels():
 
 
 
-def test_extra_body_cache_not_forwarded_to_vertex_ai():
-    """
-    'cache' inside extra_body is a LiteLLM-internal proxy caching control.
-    It must NOT be forwarded to the Vertex AI request body.
-
-    Regression test for: "Invalid JSON payload received. Unknown name \"cache\": Cannot find field."
-    Vertex AI enforces a strict JSON schema and rejects any unknown field.
-    """
-    messages = [{"role": "user", "content": "test"}]
-    optional_params = {
-        "extra_body": {
-            "cache": {"use-cache": True, "ttl": 86400},  # LiteLLM-internal
-            "some_vertex_param": "value",                 # legitimate provider extra
-        },
-    }
-    litellm_params = {}
-
-    result = _transform_request_body(
-        messages=messages,
-        model="gemini-2.5-pro",
-        optional_params=optional_params,
-        custom_llm_provider="vertex_ai",
-        litellm_params=litellm_params,
-        cached_content=None,
-    )
-
-    # 'cache' must be stripped — Vertex AI has no such field
-    assert "cache" not in result, (
-        "extra_body.cache must not be forwarded to Vertex AI. "
-        "Vertex AI rejects it with 400: Unknown name \"cache\": Cannot find field."
-    )
-
-    # Other legitimate extra_body keys should still pass through
-    assert "some_vertex_param" in result
-    assert result["some_vertex_param"] == "value"
-
-    # Core request fields must be present
-    assert "contents" in result
-
-
-def test_extra_body_tags_not_forwarded_to_vertex_ai():
-    """
-    'tags' inside extra_body is a LiteLLM-internal param for logging/tracking.
-    It must NOT be forwarded to the Vertex AI request body.
-    Documented in litellm_proxy.md: "Send tags by including them in the extra_body parameter"
-    """
-    messages = [{"role": "user", "content": "test"}]
-    optional_params = {
-        "extra_body": {
-            "tags": ["user:alice", "env:prod"],
-            "custom_param": "allowed",
-        },
-    }
-    litellm_params = {}
-
-    result = _transform_request_body(
-        messages=messages,
-        model="gemini-2.5-pro",
-        optional_params=optional_params,
-        custom_llm_provider="vertex_ai",
-        litellm_params=litellm_params,
-        cached_content=None,
-    )
-
-    assert "tags" not in result
-    assert "custom_param" in result
-    assert result["custom_param"] == "allowed"
-
-
 def test_metadata_to_labels_vertex_only():
     """Test that metadata->labels conversion only happens for Vertex AI"""
     messages = [{"role": "user", "content": "test"}]

tests/test_litellm/llms/vertex_ai/gemini/test_vertex_and_google_ai_studio_gemini.py

@@ -674,34 +674,32 @@ def test_check_finish_reason():
 
 def test_finish_reason_unspecified_and_malformed_function_call():
     """
-    Test that FINISH_REASON_UNSPECIFIED and MALFORMED_FUNCTION_CALL 
-    return their lowercase values instead of being mapped to 'stop'
-    since we don't have good mappings for these.
+    Test that FINISH_REASON_UNSPECIFIED and MALFORMED_FUNCTION_CALL
+    are mapped to OpenAI-compatible 'stop' finish reason.
     """
     finish_reason_mappings = VertexGeminiConfig.get_finish_reason_mapping()
-    
-    # Test FINISH_REASON_UNSPECIFIED returns lowercase version
-    assert finish_reason_mappings["FINISH_REASON_UNSPECIFIED"] == "finish_reason_unspecified"
+
+    # Test FINISH_REASON_UNSPECIFIED maps to "stop"
+    assert finish_reason_mappings["FINISH_REASON_UNSPECIFIED"] == "stop"
     assert (
         VertexGeminiConfig._check_finish_reason(
             chat_completion_message=None, finish_reason="FINISH_REASON_UNSPECIFIED"
         )
-        == "finish_reason_unspecified"
+        == "stop"
     )
-    
-    # Test MALFORMED_FUNCTION_CALL returns lowercase version
-    assert finish_reason_mappings["MALFORMED_FUNCTION_CALL"] == "malformed_function_call"
+
+    # Test MALFORMED_FUNCTION_CALL maps to "stop"
+    assert finish_reason_mappings["MALFORMED_FUNCTION_CALL"] == "stop"
     assert (
         VertexGeminiConfig._check_finish_reason(
             chat_completion_message=None, finish_reason="MALFORMED_FUNCTION_CALL"
         )
-        == "malformed_function_call"
+        == "stop"
     )
-    
-    # Ensure these values are in the OpenAI finish reasons constant
-    from litellm import OPENAI_FINISH_REASONS
-    assert "finish_reason_unspecified" in OPENAI_FINISH_REASONS
-    assert "malformed_function_call" in OPENAI_FINISH_REASONS
+
+    # Test new Gemini finish reasons
+    assert finish_reason_mappings["TOO_MANY_TOOL_CALLS"] == "stop"
+    assert finish_reason_mappings["MALFORMED_RESPONSE"] == "stop"
 
 
 def test_vertex_ai_usage_metadata_response_token_count():

tests/test_litellm/llms/vertex_ai/test_vertex_ai_common_utils.py

@@ -11,6 +11,7 @@ sys.path.insert(
 )  # Adds the parent directory to the system path
 
 from litellm.llms.vertex_ai.common_utils import (
+    _build_vertex_schema_for_gemini_2,
     _get_vertex_url,
     convert_anyof_null_to_nullable,
     get_vertex_location_from_url,
@@ -1402,3 +1403,93 @@ def test_add_object_type_does_not_add_type_when_anyof_present():
 
     # Verify type was not added (anyOf handles the type)
     assert "type" not in input_schema, "type should not be added when anyOf is present"
+
+
+class TestBuildVertexSchemaForGemini2:
+    """Tests for _build_vertex_schema_for_gemini_2 — minimal transform for Gemini 2.0+ tools."""
+
+    def test_jsonvalue_standalone_preserved(self):
+        """JsonValue (bare {}) should NOT be coerced to {"type": "object"}."""
+        schema = {
+            "type": "object",
+            "properties": {
+                "name": {"type": "string"},
+                "value": {},
+            },
+            "required": ["name", "value"],
+        }
+        result = _build_vertex_schema_for_gemini_2(schema)
+        assert result["properties"]["value"] == {}
+
+    def test_optional_jsonvalue_anyof_preserved(self):
+        """Optional[JsonValue] anyOf with null should be preserved, not converted to nullable."""
+        schema = {
+            "type": "object",
+            "properties": {
+                "name": {"type": "string"},
+                "value": {
+                    "anyOf": [
+                        {"type": "array", "items": {}},
+                        {},
+                        {"type": "null"},
+                    ]
+                },
+            },
+            "required": ["name"],
+        }
+        result = _build_vertex_schema_for_gemini_2(schema)
+        value_schema = result["properties"]["value"]
+        assert "anyOf" in value_schema
+        assert len(value_schema["anyOf"]) == 3
+        assert {"type": "null"} in value_schema["anyOf"]
+        assert {} in value_schema["anyOf"]
+
+    def test_ref_defs_resolved(self):
+        """$ref/$defs should be resolved since Gemini doesn't support them in tool params."""
+        schema = {
+            "type": "object",
+            "properties": {
+                "value": {"$ref": "#/$defs/JsonValue"},
+            },
+            "$defs": {"JsonValue": {}},
+        }
+        result = _build_vertex_schema_for_gemini_2(schema)
+        assert "$ref" not in result["properties"]["value"]
+        assert "$defs" not in result
+        assert result["properties"]["value"] == {}
+
+    def test_unsupported_fields_stripped(self):
+        """Fields not in Vertex Schema TypedDict should be removed."""
+        schema = {
+            "type": "object",
+            "properties": {
+                "name": {"type": "string", "additionalProperties": False},
+            },
+            "additionalProperties": False,
+            "$schema": "http://json-schema.org/draft-07/schema#",
+        }
+        result = _build_vertex_schema_for_gemini_2(schema)
+        assert "additionalProperties" not in result
+        assert "$schema" not in result
+
+    def test_no_type_coercion(self):
+        """Schemas without type should NOT have type: object added."""
+        schema = {
+            "type": "object",
+            "properties": {
+                "data": {"description": "Any data"},
+            },
+        }
+        result = _build_vertex_schema_for_gemini_2(schema)
+        assert "type" not in result["properties"]["data"]
+
+    def test_items_empty_preserved(self):
+        """items: {} should NOT be coerced to items: {"type": "object"}."""
+        schema = {
+            "type": "object",
+            "properties": {
+                "values": {"type": "array", "items": {}},
+            },
+        }
+        result = _build_vertex_schema_for_gemini_2(schema)
+        assert result["properties"]["values"]["items"] == {}

tests/test_litellm/proxy/_experimental/mcp_server/test_mcp_server.py

@@ -2093,150 +2093,3 @@ async def test_get_tools_from_mcp_servers_logs_list_tools_to_spendlogs_when_enab
     assert spend_meta["tool_count_total"] == 1
     assert spend_meta["allowed_server_count"] == 1
     assert spend_meta["per_server_tool_counts"]["server_a"] == 1
-
-
-def test_tool_name_matches_case_insensitive():
-    """Test that _tool_name_matches performs case-insensitive comparison.
-    
-    This is critical for OpenAPI-based MCP servers where:
-    1. operationIds are often in camelCase (e.g., 'addPet', 'updatePet')
-    2. Tool names are lowercased during registration (e.g., 'addpet', 'updatepet')
-    3. allowed_tools configuration may use the original camelCase names
-    
-    Without case-insensitive matching, all tools would be filtered out.
-    """
-    try:
-        from litellm.proxy._experimental.mcp_server.server import _tool_name_matches
-    except ImportError:
-        pytest.skip("MCP server not available")
-
-    # Test case 1: Unprefixed tool name with camelCase in filter list
-    assert _tool_name_matches("addpet", ["addPet", "updatePet"]) is True
-    assert _tool_name_matches("updatepet", ["addPet", "updatePet"]) is True
-    assert _tool_name_matches("deletepet", ["addPet", "updatePet"]) is False
-
-    # Test case 2: Prefixed tool name with camelCase in filter list
-    assert _tool_name_matches("per_store-addpet", ["addPet", "updatePet"]) is True
-    assert _tool_name_matches("per_store-updatepet", ["addPet", "updatePet"]) is True
-    assert _tool_name_matches("per_store-deletepet", ["addPet", "updatePet"]) is False
-
-    # Test case 3: Mixed case variations
-    assert _tool_name_matches("findPetsByStatus", ["findpetsbystatus"]) is True
-    assert _tool_name_matches("findpetsbystatus", ["findPetsByStatus"]) is True
-    assert _tool_name_matches("FINDPETSBYSTATUS", ["findPetsByStatus"]) is True
-
-    # Test case 4: Full prefixed name in filter list (case-insensitive)
-    assert _tool_name_matches("server-addPet", ["server-addpet"]) is True
-    assert _tool_name_matches("server-addpet", ["server-addPet"]) is True
-
-    # Test case 5: Ensure non-matching names still don't match
-    assert _tool_name_matches("addpet", ["deletePet", "updatePet"]) is False
-    assert _tool_name_matches("server-addpet", ["deletePet", "updatePet"]) is False
-
-
-def test_filter_tools_by_allowed_tools_case_insensitive():
-    """Test that filter_tools_by_allowed_tools handles case-insensitive matching.
-    
-    Ensures that OpenAPI tools with lowercase names can be filtered using
-    camelCase allowed_tools configuration from the OpenAPI spec.
-    """
-    try:
-        from litellm.proxy._experimental.mcp_server.server import (
-            filter_tools_by_allowed_tools,
-        )
-        from litellm.types.mcp_server.tool_registry import MCPTool
-    except ImportError:
-        pytest.skip("MCP server not available")
-
-    # Mock handler function
-    def mock_handler(**kwargs):
-        return kwargs
-
-    # Create mock tools with lowercase names (as registered from OpenAPI)
-    tools = [
-        MCPTool(
-            name="per_store-addpet",
-            description="Add a pet",
-            input_schema={"type": "object"},
-            handler=mock_handler,
-        ),
-        MCPTool(
-            name="per_store-updatepet",
-            description="Update a pet",
-            input_schema={"type": "object"},
-            handler=mock_handler,
-        ),
-        MCPTool(
-            name="per_store-deletepet",
-            description="Delete a pet",
-            input_schema={"type": "object"},
-            handler=mock_handler,
-        ),
-        MCPTool(
-            name="per_store-findpetsbystatus",
-            description="Find pets by status",
-            input_schema={"type": "object"},
-            handler=mock_handler,
-        ),
-    ]
-
-    # Create mock server with camelCase allowed_tools (as from OpenAPI spec)
-    server = MCPServer(
-        server_id="test-server",
-        name="per_store",
-        transport=MCPTransport.http,
-        allowed_tools=["addPet", "updatePet", "findPetsByStatus"],
-    )
-
-    # Filter tools
-    filtered_tools = filter_tools_by_allowed_tools(tools, server)
-
-    # Should return 3 tools (case-insensitive match)
-    assert len(filtered_tools) == 3
-    assert any(t.name == "per_store-addpet" for t in filtered_tools)
-    assert any(t.name == "per_store-updatepet" for t in filtered_tools)
-    assert any(t.name == "per_store-findpetsbystatus" for t in filtered_tools)
-    assert not any(t.name == "per_store-deletepet" for t in filtered_tools)
-
-
-def test_filter_tools_by_allowed_tools_no_filter():
-    """Test that filter_tools_by_allowed_tools returns all tools when no filter is set."""
-    try:
-        from litellm.proxy._experimental.mcp_server.server import (
-            filter_tools_by_allowed_tools,
-        )
-        from litellm.types.mcp_server.tool_registry import MCPTool
-    except ImportError:
-        pytest.skip("MCP server not available")
-
-    # Mock handler function
-    def mock_handler(**kwargs):
-        return kwargs
-
-    tools = [
-        MCPTool(
-            name="fusion_litellm_mcp-model_list",
-            description="List models",
-            input_schema={"type": "object"},
-            handler=mock_handler,
-        ),
-        MCPTool(
-            name="fusion_litellm_mcp-chat_completion",
-            description="Chat completion",
-            input_schema={"type": "object"},
-            handler=mock_handler,
-        ),
-    ]
-
-    # Server with no allowed_tools filter
-    server = MCPServer(
-        server_id="test-server",
-        name="fusion_litellm_mcp",
-        transport=MCPTransport.http,
-        allowed_tools=None,
-    )
-
-    filtered_tools = filter_tools_by_allowed_tools(tools, server)
-
-    # Should return all tools when no filter is configured
-    assert len(filtered_tools) == 2

tests/test_litellm/proxy/auth/test_model_checks.py

@@ -21,140 +21,6 @@ def test_get_team_models_for_all_models_and_team_only_models():
     assert set(result) == set(combined_models)
 
 
-def test_get_team_models_all_proxy_models_includes_access_groups():
-    """
-    When a team has 'all-proxy-models' and include_model_access_groups=True,
-    the result should include model access group names (e.g. 'claude-model-group')
-    in addition to individual model names.
-    """
-    from litellm.proxy.auth.model_checks import get_team_models
-
-    team_models = ["all-proxy-models"]
-    proxy_model_list = ["model1", "model2"]
-    model_access_groups = {
-        "group-a": ["model1"],
-        "group-b": ["model2"],
-    }
-
-    result = get_team_models(
-        team_models, proxy_model_list, model_access_groups, include_model_access_groups=True
-    )
-    assert "group-a" in result
-    assert "group-b" in result
-    assert "model1" in result
-    assert "model2" in result
-    assert len(result) == len(set(result)), "result should have no duplicates"
-
-
-def test_get_team_models_all_proxy_models_without_include_flag():
-    """
-    When include_model_access_groups=False, access group names should NOT
-    appear in the result even with 'all-proxy-models'.
-    """
-    from litellm.proxy.auth.model_checks import get_team_models
-
-    team_models = ["all-proxy-models"]
-    proxy_model_list = ["model1", "model2"]
-    model_access_groups = {
-        "group-a": ["model1"],
-        "group-b": ["model2"],
-    }
-
-    result = get_team_models(
-        team_models, proxy_model_list, model_access_groups, include_model_access_groups=False
-    )
-    assert "group-a" not in result
-    assert "group-b" not in result
-    assert "model1" in result
-    assert "model2" in result
-
-
-def test_get_key_models_all_proxy_models_includes_access_groups():
-    """
-    When a key has 'all-proxy-models' and include_model_access_groups=True,
-    the result should include model access group names.
-    """
-    from litellm.proxy._types import UserAPIKeyAuth
-    from litellm.proxy.auth.model_checks import get_key_models
-
-    user_api_key_dict = UserAPIKeyAuth(
-        models=["all-proxy-models"],
-        api_key="test-key",
-    )
-    proxy_model_list = ["model1", "model2"]
-    model_access_groups = {
-        "group-a": ["model1"],
-    }
-
-    result = get_key_models(
-        user_api_key_dict=user_api_key_dict,
-        proxy_model_list=proxy_model_list,
-        model_access_groups=model_access_groups,
-        include_model_access_groups=True,
-    )
-    assert "group-a" in result
-    assert "model1" in result
-    assert "model2" in result
-    assert len(result) == len(set(result)), "result should have no duplicates"
-
-
-def test_get_key_models_passes_include_model_access_groups():
-    """
-    When a key explicitly has an access group name in its models list and
-    include_model_access_groups=True, the group name should be retained
-    (not stripped by _get_models_from_access_groups).
-    """
-    from litellm.proxy._types import UserAPIKeyAuth
-    from litellm.proxy.auth.model_checks import get_key_models
-
-    user_api_key_dict = UserAPIKeyAuth(
-        models=["group-a"],
-        api_key="test-key",
-    )
-    proxy_model_list = ["model1", "model2"]
-    model_access_groups = {
-        "group-a": ["model1", "model2"],
-    }
-
-    result = get_key_models(
-        user_api_key_dict=user_api_key_dict,
-        proxy_model_list=proxy_model_list,
-        model_access_groups=model_access_groups,
-        include_model_access_groups=True,
-    )
-    assert "group-a" in result
-    assert "model1" in result
-    assert "model2" in result
-
-
-def test_get_key_models_does_not_mutate_input():
-    """
-    get_key_models must not mutate user_api_key_dict.models in-place.
-    _get_models_from_access_groups uses .pop()/.extend() which would corrupt
-    cached UserAPIKeyAuth objects if all_models were an alias instead of a copy.
-    """
-    from litellm.proxy._types import UserAPIKeyAuth
-    from litellm.proxy.auth.model_checks import get_key_models
-
-    original_models = ["group-a", "extra-model"]
-    user_api_key_dict = UserAPIKeyAuth(
-        models=list(original_models),  # give it a list
-        api_key="test-key",
-    )
-    model_access_groups = {
-        "group-a": ["model1", "model2"],
-    }
-
-    _ = get_key_models(
-        user_api_key_dict=user_api_key_dict,
-        proxy_model_list=["model1", "model2"],
-        model_access_groups=model_access_groups,
-        include_model_access_groups=False,
-    )
-    # The original models list on the auth object must be unchanged
-    assert user_api_key_dict.models == original_models
-
-
 @pytest.mark.parametrize(
     "key_models,team_models,proxy_model_list,model_list,expected",
     [

tests/test_litellm/proxy/pass_through_endpoints/test_pass_through_endpoints.py

@@ -2369,42 +2369,3 @@ def test_get_registered_pass_through_route_with_custom_root():
 
     # Clean up
     _registered_pass_through_routes.clear()
-
-
-def test_mapped_pass_through_routes_with_server_root_path():
-    """
-    Mapped passthrough routes (vertex_ai, bedrock, etc) should match
-    even when SERVER_ROOT_PATH is set and the incoming route is prefixed.
-
-    Regression test for https://github.com/BerriAI/litellm/issues/22272
-    """
-    from litellm.proxy.pass_through_endpoints.pass_through_endpoints import (
-        InitPassThroughEndpointHelpers,
-    )
-
-    with patch(
-        "litellm.proxy.pass_through_endpoints.pass_through_endpoints.get_server_root_path"
-    ) as mock_get_root:
-        mock_get_root.return_value = "/litellm"
-
-        # prefixed route should match mapped routes like /vertex_ai
-        assert (
-            InitPassThroughEndpointHelpers.is_registered_pass_through_route(
-                "/litellm/vertex_ai/v1/projects/foo"
-            )
-            is True
-        )
-        assert (
-            InitPassThroughEndpointHelpers.is_registered_pass_through_route(
-                "/litellm/bedrock/model/invoke"
-            )
-            is True
-        )
-
-        # bare route without prefix should not match when root is set
-        assert (
-            InitPassThroughEndpointHelpers.is_registered_pass_through_route(
-                "/vertex_ai/v1/projects/foo"
-            )
-            is False
-        )

tests/test_litellm/proxy/spend_tracking/test_spend_tracking_utils.py

@@ -1071,10 +1071,9 @@ def test_spend_logs_redacts_request_and_response_when_turn_off_message_logging_e
     response_result = _get_response_for_spend_logs_payload(payload=payload, kwargs=kwargs)
 
     # When redaction is enabled and response is a dict (not ModelResponse),
-    # perform_redaction redacts content in-place within the choices structure
+    # perform_redaction returns {"text": "redacted-by-litellm"}
     parsed_response = json.loads(response_result)
-    assert parsed_response["choices"][0]["message"]["content"] == "redacted-by-litellm"
-    assert parsed_response["choices"][0]["message"]["role"] == "assistant"
+    assert parsed_response == {"text": "redacted-by-litellm"}
 
 
 @patch("litellm.secret_managers.main.get_secret_bool")

tests/test_litellm/proxy/test_openapi_schema_validation.py

@@ -1,142 +0,0 @@
-"""
-Test that the OpenAPI schema generated by FastAPI is valid for specific endpoints.
-
-Validates fixes for:
-- /spend/calculate response schema (must use proper OpenAPI 3.x content wrapper)
-- /credentials/by_model/{model_id} path parameter (must not leak credential_name)
-
-Related issue: https://github.com/BerriAI/litellm/issues/21305
-"""
-
-import pytest
-
-
-class TestSpendCalculateOpenAPISchema:
-    """Test /spend/calculate response schema is valid OpenAPI 3.x."""
-
-    def test_response_schema_has_description(self):
-        """The 200 response must have a 'description' field per OpenAPI 3.x spec."""
-        from litellm.proxy.spend_tracking.spend_management_endpoints import router
-
-        for route in router.routes:
-            if hasattr(route, "path") and route.path == "/spend/calculate":
-                responses = route.responses or {}
-                response_200 = responses.get(200, {})
-                assert "description" in response_200, (
-                    "/spend/calculate 200 response must have a 'description' field"
-                )
-                break
-        else:
-            pytest.fail("/spend/calculate route not found in router")
-
-    def test_response_schema_has_content_wrapper(self):
-        """The 200 response must use 'content' wrapper, not bare properties."""
-        from litellm.proxy.spend_tracking.spend_management_endpoints import router
-
-        for route in router.routes:
-            if hasattr(route, "path") and route.path == "/spend/calculate":
-                responses = route.responses or {}
-                response_200 = responses.get(200, {})
-                # Must NOT have 'cost' as a top-level key (invalid OpenAPI)
-                assert "cost" not in response_200, (
-                    "/spend/calculate 200 response must not have 'cost' as a "
-                    "top-level property - use 'content' wrapper instead"
-                )
-                # Must have 'content' wrapper
-                assert "content" in response_200, (
-                    "/spend/calculate 200 response must have a 'content' field"
-                )
-                content = response_200["content"]
-                assert "application/json" in content
-                assert "schema" in content["application/json"]
-                break
-        else:
-            pytest.fail("/spend/calculate route not found in router")
-
-
-class TestCredentialEndpointsOpenAPISchema:
-    """Test /credentials endpoints have correct path parameters."""
-
-    def test_by_name_and_by_model_are_separate_handlers(self):
-        """
-        /credentials/by_name/{credential_name} and /credentials/by_model/{model_id}
-        must be separate handler functions so each only declares its own path params.
-        """
-        from litellm.proxy.credential_endpoints.endpoints import router
-
-        by_name_routes = []
-        by_model_routes = []
-        for route in router.routes:
-            if not hasattr(route, "path"):
-                continue
-            if "by_name" in route.path:
-                by_name_routes.append(route)
-            elif "by_model" in route.path:
-                by_model_routes.append(route)
-
-        assert len(by_name_routes) == 1, "Expected exactly one by_name route"
-        assert len(by_model_routes) == 1, "Expected exactly one by_model route"
-
-        # They must be different endpoint functions
-        by_name_endpoint = by_name_routes[0].endpoint
-        by_model_endpoint = by_model_routes[0].endpoint
-        assert by_name_endpoint is not by_model_endpoint, (
-            "by_name and by_model must be separate handler functions "
-            "to avoid path parameter conflicts in OpenAPI spec"
-        )
-
-    def test_by_model_route_does_not_require_credential_name(self):
-        """
-        The /credentials/by_model/{model_id} route must NOT have
-        credential_name as a parameter.
-        """
-        import inspect
-        from litellm.proxy.credential_endpoints.endpoints import (
-            get_credential_by_model,
-        )
-
-        sig = inspect.signature(get_credential_by_model)
-        param_names = list(sig.parameters.keys())
-        assert "credential_name" not in param_names, (
-            "get_credential_by_model must not have a credential_name parameter"
-        )
-
-    def test_by_name_route_does_not_require_model_id(self):
-        """
-        The /credentials/by_name/{credential_name} route must NOT have
-        model_id as a parameter.
-        """
-        import inspect
-        from litellm.proxy.credential_endpoints.endpoints import (
-            get_credential_by_name,
-        )
-
-        sig = inspect.signature(get_credential_by_name)
-        param_names = list(sig.parameters.keys())
-        assert "model_id" not in param_names, (
-            "get_credential_by_name must not have a model_id parameter"
-        )
-
-    def test_by_model_has_model_id_path_param(self):
-        """The by_model handler must accept model_id as a path parameter."""
-        import inspect
-        from litellm.proxy.credential_endpoints.endpoints import (
-            get_credential_by_model,
-        )
-
-        sig = inspect.signature(get_credential_by_model)
-        assert "model_id" in sig.parameters, (
-            "get_credential_by_model must have a model_id parameter"
-        )
-
-    def test_by_name_has_credential_name_path_param(self):
-        """The by_name handler must accept credential_name as a path parameter."""
-        import inspect
-        from litellm.proxy.credential_endpoints.endpoints import (
-            get_credential_by_name,
-        )
-
-        sig = inspect.signature(get_credential_by_name)
-        assert "credential_name" in sig.parameters, (
-            "get_credential_by_name must have a credential_name parameter"
-        )

tests/test_litellm/responses/litellm_completion_transformation/test_litellm_completion_responses.py

@@ -1774,128 +1774,3 @@ class TestStreamingIDConsistency:
         # Verify it matches the cached ID
         assert iterator._cached_item_id is not None
         assert iterator._cached_item_id == text_done_id
-
-    def test_parallel_tool_calls_merged_into_single_assistant_message(self):
-        """
-        Regression test: multi-turn parallel tool calls via the Responses API must
-        produce a single assistant message with all tool_calls, not one assistant
-        message per function_call item.
-
-        When the model responds with two parallel tool calls (e.g. get_weather for
-        SF and NYC), the next Responses API request includes two consecutive
-        function_call items followed by two function_call_output items.
-
-        Without the fix each function_call becomes its own assistant message,
-        producing back-to-back assistant messages that Anthropic/Vertex AI rejects:
-        "tool_use ids were found without tool_result blocks immediately after".
-        """
-        input_items = [
-            {"type": "message", "role": "user", "content": "Weather in SF and NYC?"},
-            # Two parallel tool calls from the previous assistant response
-            {
-                "type": "function_call",
-                "call_id": "toolu_01",
-                "name": "get_weather",
-                "arguments": '{"city": "SF"}',
-            },
-            {
-                "type": "function_call",
-                "call_id": "toolu_02",
-                "name": "get_weather",
-                "arguments": '{"city": "NYC"}',
-            },
-            # Tool results
-            {"type": "function_call_output", "call_id": "toolu_01", "output": "72°F"},
-            {"type": "function_call_output", "call_id": "toolu_02", "output": "55°F"},
-        ]
-
-        messages = LiteLLMCompletionResponsesConfig._transform_response_input_param_to_chat_completion_message(
-            input=input_items
-        )
-
-        roles = [
-            m.get("role") if isinstance(m, dict) else getattr(m, "role", None)
-            for m in messages
-        ]
-
-        # Must not have two consecutive assistant messages
-        for i in range(len(roles) - 1):
-            assert not (
-                roles[i] == "assistant" and roles[i + 1] == "assistant"
-            ), f"Consecutive assistant messages at indices {i} and {i+1}: {roles}"
-
-        # The single assistant message must contain BOTH tool_calls
-        assistant_messages = [
-            m for m in messages
-            if (m.get("role") if isinstance(m, dict) else getattr(m, "role", None))
-            == "assistant"
-        ]
-        assert len(assistant_messages) == 1, (
-            f"Expected 1 assistant message, got {len(assistant_messages)}"
-        )
-
-        assistant_msg = assistant_messages[0]
-        tool_calls = (
-            assistant_msg.get("tool_calls")
-            if isinstance(assistant_msg, dict)
-            else getattr(assistant_msg, "tool_calls", None)
-        )
-        assert tool_calls is not None and len(tool_calls) == 2, (
-            f"Expected 2 tool_calls in the merged assistant message, got: {tool_calls}"
-        )
-
-        call_ids = [
-            (tc.get("id") if isinstance(tc, dict) else getattr(tc, "id", None))
-            for tc in tool_calls
-        ]
-        assert "toolu_01" in call_ids, f"toolu_01 missing from tool_calls: {call_ids}"
-        assert "toolu_02" in call_ids, f"toolu_02 missing from tool_calls: {call_ids}"
-
-        # Both tool messages must be present
-        tool_messages = [
-            m for m in messages
-            if (m.get("role") if isinstance(m, dict) else getattr(m, "role", None))
-            == "tool"
-        ]
-        assert len(tool_messages) == 2, (
-            f"Expected 2 tool messages, got {len(tool_messages)}"
-        )
-
-    def test_single_tool_call_still_works_after_merge_fix(self):
-        """
-        Ensure the parallel-tool-call merging fix does not break the existing
-        single-tool-call path.
-        """
-        input_items = [
-            {"type": "message", "role": "user", "content": "Weather in SF?"},
-            {
-                "type": "function_call",
-                "call_id": "toolu_01",
-                "name": "get_weather",
-                "arguments": '{"city": "SF"}',
-            },
-            {"type": "function_call_output", "call_id": "toolu_01", "output": "72°F"},
-        ]
-
-        messages = LiteLLMCompletionResponsesConfig._transform_response_input_param_to_chat_completion_message(
-            input=input_items
-        )
-
-        roles = [
-            m.get("role") if isinstance(m, dict) else getattr(m, "role", None)
-            for m in messages
-        ]
-
-        assert "user" in roles
-        assert "assistant" in roles
-        assert "tool" in roles
-
-        assistant_messages = [m for m in messages if (m.get("role") if isinstance(m, dict) else getattr(m, "role", None)) == "assistant"]
-        assert len(assistant_messages) == 1
-
-        tool_calls = (
-            assistant_messages[0].get("tool_calls")
-            if isinstance(assistant_messages[0], dict)
-            else getattr(assistant_messages[0], "tool_calls", None)
-        )
-        assert tool_calls is not None and len(tool_calls) == 1

tests/test_litellm/test_model_cost_aliases.py

@@ -0,0 +1,238 @@
+"""
+Tests for the ``aliases`` feature in the model cost map.
+
+The ``_expand_model_aliases`` function processes ``aliases`` lists from model
+entries, creating shared dict references for alias entries at load time.
+"""
+
+import logging
+
+from litellm.litellm_core_utils.get_model_cost_map import _expand_model_aliases
+
+
+# ---------------------------------------------------------------------------
+# Core expansion behaviour
+# ---------------------------------------------------------------------------
+
+
+class TestExpandModelAliases:
+    """Unit tests for _expand_model_aliases."""
+
+    def test_basic_expansion(self):
+        """Aliases are added as top-level entries in model_cost."""
+        model_cost = {
+            "my-model-latest": {
+                "aliases": ["my-model-20250101"],
+                "input_cost_per_token": 1e-06,
+                "litellm_provider": "test",
+                "mode": "chat",
+            },
+        }
+        result = _expand_model_aliases(model_cost)
+
+        assert "my-model-20250101" in result
+        assert result["my-model-20250101"]["input_cost_per_token"] == 1e-06
+        assert result["my-model-20250101"]["litellm_provider"] == "test"
+
+    def test_multiple_aliases(self):
+        """A single entry can declare multiple aliases."""
+        model_cost = {
+            "provider/model-latest": {
+                "aliases": ["provider/model-v1", "provider/model-v2"],
+                "input_cost_per_token": 5e-06,
+                "litellm_provider": "provider",
+                "mode": "chat",
+            },
+        }
+        result = _expand_model_aliases(model_cost)
+
+        assert "provider/model-v1" in result
+        assert "provider/model-v2" in result
+
+    def test_shared_dict_reference(self):
+        """Alias entries share the same dict object as the canonical entry (no copy)."""
+        model_cost = {
+            "canonical-model": {
+                "aliases": ["alias-model"],
+                "input_cost_per_token": 2e-06,
+                "litellm_provider": "test",
+                "mode": "chat",
+            },
+        }
+        result = _expand_model_aliases(model_cost)
+
+        assert result["alias-model"] is result["canonical-model"]
+
+    def test_aliases_key_removed(self):
+        """The ``aliases`` key is removed from the entry after expansion."""
+        model_cost = {
+            "my-model": {
+                "aliases": ["my-model-alias"],
+                "input_cost_per_token": 1e-06,
+                "litellm_provider": "test",
+                "mode": "chat",
+            },
+        }
+        result = _expand_model_aliases(model_cost)
+
+        assert "aliases" not in result["my-model"]
+        assert "aliases" not in result["my-model-alias"]
+
+    def test_entries_without_aliases_unchanged(self):
+        """Entries with no ``aliases`` key are left untouched."""
+        model_cost = {
+            "plain-model": {
+                "input_cost_per_token": 3e-06,
+                "litellm_provider": "test",
+                "mode": "chat",
+            },
+        }
+        result = _expand_model_aliases(model_cost)
+
+        assert "plain-model" in result
+        assert result["plain-model"]["input_cost_per_token"] == 3e-06
+        assert len(result) == 1
+
+    def test_empty_aliases_list(self):
+        """An empty ``aliases`` list is treated the same as no aliases."""
+        model_cost = {
+            "model-a": {
+                "aliases": [],
+                "input_cost_per_token": 1e-06,
+                "litellm_provider": "test",
+                "mode": "chat",
+            },
+        }
+        result = _expand_model_aliases(model_cost)
+
+        assert len(result) == 1
+        assert "model-a" in result
+        assert "aliases" not in result["model-a"]
+
+
+# ---------------------------------------------------------------------------
+# Conflict handling
+# ---------------------------------------------------------------------------
+
+
+class TestAliasConflicts:
+    """Tests for alias conflict detection and handling."""
+
+    def test_alias_conflicts_with_canonical_entry(self, caplog):
+        """Alias that matches an existing canonical entry is skipped with a warning."""
+        model_cost = {
+            "model-latest": {
+                "aliases": ["model-dated"],
+                "input_cost_per_token": 1e-06,
+                "litellm_provider": "test",
+                "mode": "chat",
+            },
+            "model-dated": {
+                "input_cost_per_token": 1e-06,
+                "litellm_provider": "test",
+                "mode": "chat",
+            },
+        }
+        with caplog.at_level(logging.WARNING, logger="LiteLLM"):
+            result = _expand_model_aliases(model_cost)
+
+        # The canonical "model-dated" entry is preserved, not overwritten
+        assert "model-dated" in result
+        assert "alias conflict" in caplog.text.lower()
+
+    def test_duplicate_alias_across_entries(self, caplog):
+        """Same alias claimed by two different entries: second one is skipped."""
+        model_cost = {
+            "model-a": {
+                "aliases": ["shared-alias"],
+                "input_cost_per_token": 1e-06,
+                "litellm_provider": "test",
+                "mode": "chat",
+            },
+            "model-b": {
+                "aliases": ["shared-alias"],
+                "input_cost_per_token": 2e-06,
+                "litellm_provider": "test",
+                "mode": "chat",
+            },
+        }
+        with caplog.at_level(logging.WARNING, logger="LiteLLM"):
+            result = _expand_model_aliases(model_cost)
+
+        # "shared-alias" should point to model-a (first one wins)
+        assert "shared-alias" in result
+        assert result["shared-alias"]["input_cost_per_token"] == 1e-06
+        assert "alias conflict" in caplog.text.lower()
+
+    def test_canonical_entry_not_overwritten_by_alias(self):
+        """An alias must never overwrite an existing canonical entry's data."""
+        original_cost = 9.99e-06
+        model_cost = {
+            "existing-model": {
+                "input_cost_per_token": original_cost,
+                "litellm_provider": "test",
+                "mode": "chat",
+            },
+            "other-model": {
+                "aliases": ["existing-model"],
+                "input_cost_per_token": 1e-06,
+                "litellm_provider": "test",
+                "mode": "chat",
+            },
+        }
+        result = _expand_model_aliases(model_cost)
+
+        # Original entry must be preserved
+        assert result["existing-model"]["input_cost_per_token"] == original_cost
+
+
+# ---------------------------------------------------------------------------
+# Integration with model_cost dict mutation
+# ---------------------------------------------------------------------------
+
+
+class TestAliasIntegration:
+    """Higher-level tests verifying aliases work with the model_cost dict."""
+
+    def test_mutation_through_alias_visible_on_canonical(self):
+        """Since alias is a shared reference, mutations are visible on both."""
+        model_cost = {
+            "canonical": {
+                "aliases": ["alias"],
+                "input_cost_per_token": 1e-06,
+                "litellm_provider": "test",
+                "mode": "chat",
+            },
+        }
+        result = _expand_model_aliases(model_cost)
+
+        # Mutate via alias
+        result["alias"]["input_cost_per_token"] = 999
+        assert result["canonical"]["input_cost_per_token"] == 999
+
+    def test_mixed_entries_with_and_without_aliases(self):
+        """A model_cost dict with a mix of aliased and plain entries."""
+        model_cost = {
+            "model-with-alias": {
+                "aliases": ["alias-1", "alias-2"],
+                "input_cost_per_token": 1e-06,
+                "litellm_provider": "test",
+                "mode": "chat",
+            },
+            "plain-model": {
+                "input_cost_per_token": 2e-06,
+                "litellm_provider": "test",
+                "mode": "chat",
+            },
+        }
+        result = _expand_model_aliases(model_cost)
+
+        assert len(result) == 4  # 2 canonical + 2 aliases
+        assert "alias-1" in result
+        assert "alias-2" in result
+        assert "plain-model" in result
+        assert "model-with-alias" in result
+
+    def test_expand_on_empty_dict(self):
+        """Expanding an empty dict returns an empty dict."""
+        assert _expand_model_aliases({}) == {}

tests/test_litellm/test_router_retry_non_retryable_errors.py

@@ -1,251 +0,0 @@
-"""
-Test that the Router retry loop correctly handles non-retryable errors.
-
-Verifies that:
-1. Non-retryable errors (e.g., 400 ContextWindowExceeded) inside the retry loop
-   break out immediately instead of being swallowed.
-2. original_exception is updated to the latest error, not stuck on the first.
-3. Retryable errors (e.g., 429 RateLimitError) still retry normally.
-
-Regression tests for https://github.com/BerriAI/litellm/issues/21343
-"""
-
-from unittest.mock import AsyncMock, patch
-
-import pytest
-
-import litellm
-from litellm import Router
-
-
-def _make_rate_limit_error(message="Rate limited"):
-    """Create a RateLimitError for testing."""
-    return litellm.RateLimitError(
-        message=message,
-        llm_provider="bedrock",
-        model="anthropic.claude-v2",
-    )
-
-
-def _make_context_window_error(message="prompt is too long: 1205821 tokens > 200000"):
-    """Create a ContextWindowExceededError for testing."""
-    return litellm.ContextWindowExceededError(
-        message=message,
-        llm_provider="vertex_ai",
-        model="claude-3-opus",
-    )
-
-
-def _make_bad_request_error(message="Invalid request"):
-    """Create a BadRequestError for testing."""
-    return litellm.BadRequestError(
-        message=message,
-        llm_provider="openai",
-        model="gpt-4",
-    )
-
-
-def _make_not_found_error(message="Model not found"):
-    """Create a NotFoundError for testing."""
-    return litellm.NotFoundError(
-        message=message,
-        llm_provider="openai",
-        model="gpt-99",
-    )
-
-
-def _create_router(num_retries=2):
-    """Create a Router with two deployments for testing."""
-    return Router(
-        model_list=[
-            {
-                "model_name": "test-model",
-                "litellm_params": {
-                    "model": "openai/gpt-4",
-                    "api_key": "fake-key-1",
-                },
-            },
-            {
-                "model_name": "test-model",
-                "litellm_params": {
-                    "model": "openai/gpt-4",
-                    "api_key": "fake-key-2",
-                },
-            },
-        ],
-        num_retries=num_retries,
-    )
-
-
-def _base_kwargs():
-    """Return kwargs required by async_function_with_retries."""
-    return {
-        "model": "test-model",
-        "messages": [{"role": "user", "content": "test"}],
-        "original_function": AsyncMock(),
-        "metadata": {},
-    }
-
-
-@pytest.mark.asyncio
-async def test_non_retryable_error_in_retry_loop_raises_immediately():
-    """
-    When a non-retryable error (400 ContextWindowExceeded) occurs inside the
-    retry loop, the router should raise it immediately instead of swallowing it
-    and raising the original error.
-
-    Scenario: First call -> 429, Retry -> 400 (non-retryable)
-    Expected: ContextWindowExceededError is raised, NOT RateLimitError
-    """
-    router = _create_router(num_retries=2)
-
-    rate_limit_error = _make_rate_limit_error()
-    context_window_error = _make_context_window_error()
-
-    call_count = 0
-
-    async def mock_make_call(*args, **kwargs):
-        nonlocal call_count
-        call_count += 1
-        if call_count == 1:
-            raise rate_limit_error
-        else:
-            raise context_window_error
-
-    with patch.object(router, "make_call", side_effect=mock_make_call), \
-         patch.object(router, "_async_get_healthy_deployments",
-                      return_value=(["d1", "d2"], ["d1", "d2"])), \
-         patch.object(router, "_time_to_sleep_before_retry", return_value=0), \
-         patch.object(router, "log_retry", side_effect=lambda kwargs, e: kwargs):
-        with pytest.raises(litellm.ContextWindowExceededError):
-            await router.async_function_with_retries(
-                num_retries=2,
-                **_base_kwargs(),
-            )
-
-
-@pytest.mark.asyncio
-async def test_bad_request_error_in_retry_loop_raises_immediately():
-    """
-    A generic 400 BadRequestError inside the retry loop should also break out
-    immediately since 400 is not retryable.
-    """
-    router = _create_router(num_retries=2)
-
-    rate_limit_error = _make_rate_limit_error()
-    bad_request_error = _make_bad_request_error()
-
-    call_count = 0
-
-    async def mock_make_call(*args, **kwargs):
-        nonlocal call_count
-        call_count += 1
-        if call_count == 1:
-            raise rate_limit_error
-        else:
-            raise bad_request_error
-
-    with patch.object(router, "make_call", side_effect=mock_make_call), \
-         patch.object(router, "_async_get_healthy_deployments",
-                      return_value=(["d1", "d2"], ["d1", "d2"])), \
-         patch.object(router, "_time_to_sleep_before_retry", return_value=0), \
-         patch.object(router, "log_retry", side_effect=lambda kwargs, e: kwargs):
-        with pytest.raises(litellm.BadRequestError):
-            await router.async_function_with_retries(
-                num_retries=2,
-                **_base_kwargs(),
-            )
-
-
-@pytest.mark.asyncio
-async def test_original_exception_updated_to_latest_error():
-    """
-    When all retries are exhausted with retryable errors, the LAST error
-    should be raised, not the first one.
-    """
-    router = _create_router(num_retries=2)
-
-    call_count = 0
-
-    async def mock_make_call(*args, **kwargs):
-        nonlocal call_count
-        call_count += 1
-        raise _make_rate_limit_error(f"Rate limit attempt {call_count}")
-
-    with patch.object(router, "make_call", side_effect=mock_make_call), \
-         patch.object(router, "_async_get_healthy_deployments",
-                      return_value=(["d1", "d2"], ["d1", "d2"])), \
-         patch.object(router, "_time_to_sleep_before_retry", return_value=0), \
-         patch.object(router, "log_retry", side_effect=lambda kwargs, e: kwargs):
-        with pytest.raises(litellm.RateLimitError) as exc_info:
-            await router.async_function_with_retries(
-                num_retries=2,
-                **_base_kwargs(),
-            )
-        # Should be the LAST error, not the first
-        assert "Rate limit attempt 3" in str(exc_info.value)
-
-
-@pytest.mark.asyncio
-async def test_retryable_errors_still_retry_normally():
-    """
-    Retryable errors (429 RateLimitError) should still be retried the
-    configured number of times before raising.
-    """
-    router = _create_router(num_retries=3)
-
-    call_count = 0
-
-    async def mock_make_call(*args, **kwargs):
-        nonlocal call_count
-        call_count += 1
-        raise _make_rate_limit_error(f"Rate limit attempt {call_count}")
-
-    with patch.object(router, "make_call", side_effect=mock_make_call), \
-         patch.object(router, "_async_get_healthy_deployments",
-                      return_value=(["d1", "d2"], ["d1", "d2"])), \
-         patch.object(router, "_time_to_sleep_before_retry", return_value=0), \
-         patch.object(router, "log_retry", side_effect=lambda kwargs, e: kwargs):
-        with pytest.raises(litellm.RateLimitError):
-            await router.async_function_with_retries(
-                num_retries=3,
-                **_base_kwargs(),
-            )
-
-        # Initial call + 3 retries = 4 total calls
-        assert call_count == 4
-
-
-@pytest.mark.asyncio
-async def test_not_found_error_in_retry_loop_raises_immediately():
-    """
-    A 404 NotFoundError inside the retry loop should break out immediately.
-    """
-    router = _create_router(num_retries=2)
-
-    rate_limit_error = _make_rate_limit_error()
-    not_found_error = _make_not_found_error()
-
-    call_count = 0
-
-    async def mock_make_call(*args, **kwargs):
-        nonlocal call_count
-        call_count += 1
-        if call_count == 1:
-            raise rate_limit_error
-        else:
-            raise not_found_error
-
-    with patch.object(router, "make_call", side_effect=mock_make_call), \
-         patch.object(router, "_async_get_healthy_deployments",
-                      return_value=(["d1", "d2"], ["d1", "d2"])), \
-         patch.object(router, "_time_to_sleep_before_retry", return_value=0), \
-         patch.object(router, "log_retry", side_effect=lambda kwargs, e: kwargs):
-        with pytest.raises(litellm.NotFoundError):
-            await router.async_function_with_retries(
-                num_retries=2,
-                **_base_kwargs(),
-            )
-
-        # Only 2 calls: initial + first retry that hits non-retryable
-        assert call_count == 2

tests/test_litellm/types/test_types_utils.py

@@ -225,6 +225,60 @@ def test_chat_completion_token_logprob_invalid_top_logprobs_rejected():
         )
 
 
+# ---------------------------------------------------------------------------
+# native_finish_reason in provider_specific_fields
+# ---------------------------------------------------------------------------
+
+
+class TestNativeFinishReason:
+    """Choices exposes the raw provider finish_reason in provider_specific_fields
+    when it differs from the mapped OpenAI-compatible value."""
+
+    def test_provider_reason_exposed_when_mapped(self):
+        from litellm.types.utils import Choices
+
+        choice = Choices(finish_reason="end_turn")
+        assert choice.finish_reason == "stop"
+        assert choice.provider_specific_fields["native_finish_reason"] == "end_turn"
+
+    def test_provider_reason_not_set_when_already_openai(self):
+        from litellm.types.utils import Choices
+
+        choice = Choices(finish_reason="stop")
+        assert choice.finish_reason == "stop"
+        assert not hasattr(choice, "provider_specific_fields")
+
+    def test_provider_reason_merged_with_existing_fields(self):
+        from litellm.types.utils import Choices
+
+        choice = Choices(
+            finish_reason="max_tokens",
+            provider_specific_fields={"citations": [{"url": "http://example.com"}]},
+        )
+        assert choice.finish_reason == "length"
+        assert choice.provider_specific_fields["native_finish_reason"] == "max_tokens"
+        assert choice.provider_specific_fields["citations"] == [{"url": "http://example.com"}]
+
+    def test_gemini_safety_reason_exposed(self):
+        from litellm.types.utils import Choices
+
+        choice = Choices(finish_reason="SAFETY")
+        assert choice.finish_reason == "content_filter"
+        assert choice.provider_specific_fields["native_finish_reason"] == "SAFETY"
+
+    def test_anthropic_tool_use_reason_exposed(self):
+        from litellm.types.utils import Choices
+
+        choice = Choices(finish_reason="tool_use")
+        assert choice.finish_reason == "tool_calls"
+        assert choice.provider_specific_fields["native_finish_reason"] == "tool_use"
+
+    def test_max_tokens_reason_exposed(self):
+        from litellm.types.utils import Choices
+
+        choice = Choices(finish_reason="MAX_TOKENS")
+        assert choice.finish_reason == "length"
+        assert choice.provider_specific_fields["native_finish_reason"] == "MAX_TOKENS"
 def test_delta_maps_reasoning_to_reasoning_content():
     """
     Test that Delta maps 'reasoning' field to 'reasoning_content'.

ui/litellm-dashboard/src/components/VirtualKeysPage/VirtualKeysTable.test.tsx

@@ -275,8 +275,8 @@ it("should display user email correctly", async () => {
   });
 });
 
-it("should show loading message only on initial load (isPending)", () => {
-  // Mock initial loading state
+it("should show skeleton loaders when isLoading is true", () => {
+  // Mock loading state
   mockUseKeys.mockReturnValue({
     data: null,
     isPending: true,
@@ -296,7 +296,7 @@ it("should show loading message only on initial load (isPending)", () => {
 
   renderWithProviders(<VirtualKeysTable {...mockProps} />);
 
-  // Check that loading message is shown on initial load
+  // Check that loading message is shown
   expect(screen.getByText("🚅 Loading keys...")).toBeInTheDocument();
 
   // Check that actual key data is not shown
@@ -810,79 +810,3 @@ describe("pagination display – total count and page count", () => {
     });
   });
 });
-
-describe("refetch button", () => {
-  it("should show Fetch button in normal state", () => {
-    renderWithProviders(<VirtualKeysTable {...defaultMockProps} />);
-
-    const fetchButton = screen.getByTitle("Fetch data");
-    expect(fetchButton).toBeInTheDocument();
-    expect(fetchButton).not.toBeDisabled();
-    expect(screen.getByText("Fetch")).toBeInTheDocument();
-  });
-
-  it("should show Fetching state and keep table data visible during refetch", () => {
-    mockUseKeys.mockReturnValue({
-      data: {
-        keys: [mockKey],
-        total_count: 1,
-        current_page: 1,
-        total_pages: 1,
-      } as KeysResponse,
-      isPending: false,
-      isFetching: true,
-      refetch: vi.fn(),
-    } as any);
-
-    renderWithProviders(<VirtualKeysTable {...defaultMockProps} />);
-
-    // Button should show "Fetching" and be disabled
-    expect(screen.getByText("Fetching")).toBeInTheDocument();
-    const fetchButton = screen.getByTitle("Fetch data");
-    expect(fetchButton).toBeDisabled();
-
-    // Table data should still be visible (stale data)
-    expect(screen.getByText("Test Key Alias")).toBeInTheDocument();
-
-    // "Loading keys..." should NOT appear during refetch
-    expect(screen.queryByText("🚅 Loading keys...")).not.toBeInTheDocument();
-  });
-
-  it("should call refetch when Fetch button is clicked", () => {
-    const mockRefetch = vi.fn();
-    mockUseKeys.mockReturnValue({
-      data: {
-        keys: [mockKey],
-        total_count: 1,
-        current_page: 1,
-        total_pages: 1,
-      } as KeysResponse,
-      isPending: false,
-      isFetching: false,
-      refetch: mockRefetch,
-    } as any);
-
-    renderWithProviders(<VirtualKeysTable {...defaultMockProps} />);
-
-    const fetchButton = screen.getByTitle("Fetch data");
-    fireEvent.click(fetchButton);
-
-    expect(mockRefetch).toHaveBeenCalledTimes(1);
-  });
-
-  it("should show Fetch button enabled on error so user can retry", () => {
-    mockUseKeys.mockReturnValue({
-      data: null,
-      isPending: false,
-      isFetching: false,
-      isError: true,
-      refetch: vi.fn(),
-    } as any);
-
-    renderWithProviders(<VirtualKeysTable {...defaultMockProps} />);
-
-    const fetchButton = screen.getByTitle("Fetch data");
-    expect(fetchButton).not.toBeDisabled();
-    expect(screen.getByText("Fetch")).toBeInTheDocument();
-  });
-});

ui/litellm-dashboard/src/components/VirtualKeysPage/VirtualKeysTable.tsx

@@ -85,7 +85,6 @@ export function VirtualKeysTable({ teams, organizations, onSortChange, currentSo
     data: keys,
     isPending: isLoading,
     isFetching,
-    isError,
     refetch,
   } = useKeys(tablePagination.pageIndex + 1, tablePagination.pageSize, {
     sortBy: sortBy || undefined,
@@ -103,15 +102,6 @@ export function VirtualKeysTable({ teams, organizations, onSortChange, currentSo
       organizations,
     });
 
-  // Defer the transition so the button stays in loading state until the table
-  // has rendered with the new data (mirrors the spend-logs pattern)
-  const isFetchingDeferred = useDeferredValue(isFetching);
-  const isButtonLoading = (isFetching || isFetchingDeferred) && !isError;
-
-  const handleRefresh = () => {
-    refetch();
-  };
-
   const totalCount = filteredTotalCount ?? keys?.total_count ?? 0;
 
   // Add a useEffect to call refresh when a key is created
@@ -679,28 +669,16 @@ export function VirtualKeysTable({ teams, organizations, onSortChange, currentSo
           </div>
 
           <div className="flex items-center justify-between w-full mb-4">
-            <div className="inline-flex items-center gap-2">
-              {isLoading ? (
-                <Skeleton.Node active style={{ width: 200, height: 20 }} />
-              ) : (
-                <span className="inline-flex text-sm text-gray-700">
-                  Showing {rangeLabel} of {totalCount} results
-                </span>
-              )}
-
-              <AntButton
-                type="default"
-                icon={<SyncOutlined spin={isButtonLoading} />}
-                onClick={handleRefresh}
-                disabled={isButtonLoading}
-                title="Fetch data"
-              >
-                {isButtonLoading ? "Fetching" : "Fetch"}
-              </AntButton>
-            </div>
+            {isLoading || isFetching ? (
+              <Skeleton.Node active style={{ width: 200, height: 20 }} />
+            ) : (
+              <span className="inline-flex text-sm text-gray-700">
+                Showing {rangeLabel} of {totalCount} results
+              </span>
+            )}
 
             <div className="inline-flex items-center gap-2">
-              {isLoading ? (
+              {isLoading || isFetching ? (
                 <Skeleton.Node active style={{ width: 74, height: 20 }} />
               ) : (
                 <span className="text-sm text-gray-700">
@@ -708,24 +686,24 @@ export function VirtualKeysTable({ teams, organizations, onSortChange, currentSo
                 </span>
               )}
 
-              {isLoading ? (
+              {isLoading || isFetching ? (
                 <Skeleton.Button active size="small" style={{ width: 84, height: 30 }} />
               ) : (
                 <button
                   onClick={() => table.previousPage()}
-                  disabled={isLoading || !table.getCanPreviousPage()}
+                  disabled={isLoading || isFetching || !table.getCanPreviousPage()}
                   className="px-3 py-1 text-sm border rounded-md hover:bg-gray-50 disabled:opacity-50 disabled:cursor-not-allowed"
                 >
                   Previous
                 </button>
               )}
 
-              {isLoading ? (
+              {isLoading || isFetching ? (
                 <Skeleton.Button active size="small" style={{ width: 58, height: 30 }} />
               ) : (
                 <button
                   onClick={() => table.nextPage()}
-                  disabled={isLoading || !table.getCanNextPage()}
+                  disabled={isLoading || isFetching || !table.getCanNextPage()}
                   className="px-3 py-1 text-sm border rounded-md hover:bg-gray-50 disabled:opacity-50 disabled:cursor-not-allowed"
                 >
                   Next
@@ -810,7 +788,7 @@ export function VirtualKeysTable({ teams, organizations, onSortChange, currentSo
                     ))}
                   </TableHead>
                   <TableBody>
-                    {isLoading ? (
+                    {isLoading || isFetching ? (
                       <TableRow>
                         <TableCell colSpan={columns.length} className="h-8 text-center">
                           <div className="text-center text-gray-500">