merge: resolve conflicts between main and litellm_oss_staging_03_11_2026

2026-08-06 16:24:46 +00:00 · 2026-03-12 09:38:31 -03:00
parent 8d2432b21b
commit 1be6b31e2f
205 changed files with 16204 additions and 10448 deletions
@@ -0,0 +1,169 @@
+---
+slug: gemini_embedding_2_multimodal
+title: "Gemini Embedding 2 Preview: Multimodal Embeddings on LiteLLM"
+date: 2025-03-11T10:00:00
+authors:
+  - name: Sameer Kankute
+    title: SWE @ LiteLLM (LLM Translation)
+    url: https://www.linkedin.com/in/sameer-kankute/
+    image_url: https://pbs.twimg.com/profile_images/2001352686994907136/ONgNuSk5_400x400.jpg
+description: "Generate embeddings from text, images, audio, video, and PDFs with gemini-embedding-2-preview on LiteLLM via Gemini API and Vertex AI."
+tags: [gemini, embeddings, multimodal, vertex ai]
+hide_table_of_contents: false
+---
+
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Gemini Embedding 2 Preview: Multimodal Embeddings
+
+LiteLLM now supports **multimodal embeddings** with `gemini-embedding-2-preview`—generating a single embedding from a mix of text, images, audio, video, and PDF content. Available via both the **Gemini API** (API key) and **Vertex AI** (GCP credentials).
+
+## Supported Input Types
+
+| Modality | Supported Formats | 
+|----------|-------------------|
+| **Text** | Plain text |
+| **Image** | PNG, JPEG | 
+| **Audio** | MP3, WAV | 
+| **Video** | MP4, MOV | 
+| **Documents** | PDF | 
+
+## Input Formats
+
+LiteLLM accepts three input formats for multimodal content:
+
+1. **Data URIs** – Base64-encoded inline: `data:image/png;base64,<encoded_data>`
+2. **GCS URLs** – Cloud Storage paths (Vertex AI): `gs://bucket/path/to/file.png`
+3. **Gemini File References** – Pre-uploaded files (Gemini API): `files/abc123`
+
+## Quick Start
+
+<Tabs>
+<TabItem value="gemini" label="Gemini API">
+
+```python
+from litellm import embedding
+import os
+
+os.environ["GEMINI_API_KEY"] = "your-api-key"
+
+# Text + Image (base64)
+response = embedding(
+    model="gemini/gemini-embedding-2-preview",
+    input=[
+        "The food was delicious and the waiter...",
+        "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAgAAAAIAQMAAAD+wSzIAAAABlBMVEX///+/v7+jQ3Y5AAAADklEQVQI12P4AIX8EAgALgAD/aNpbtEAAAAASUVORK5CYII"
+    ],
+)
+print(response)
+```
+
+</TabItem>
+
+<TabItem value="vertex" label="Vertex AI">
+
+```python
+import litellm
+from litellm import embedding
+
+litellm.vertex_project = "your-project-id"
+litellm.vertex_location = "us-central1"
+
+# Text + Image (GCS URL)
+response = embedding(
+    model="vertex_ai/gemini-embedding-2-preview",
+    input=[
+        "Describe this image",
+        "gs://my-bucket/images/photo.png"
+    ],
+)
+print(response)
+```
+
+</TabItem>
+
+<TabItem value="proxy" label="LiteLLM Proxy">
+
+**1. Config (config.yaml)**
+
+```yaml
+model_list:
+  - model_name: gemini-embedding-2-preview
+    litellm_params:
+      model: gemini/gemini-embedding-2-preview
+      api_key: os.environ/GEMINI_API_KEY
+  - model_name: vertex-gemini-embedding-2-preview
+    litellm_params:
+      model: vertex_ai/gemini-embedding-2-preview
+      vertex_project: os.environ/VERTEXAI_PROJECT
+      vertex_location: os.environ/VERTEXAI_LOCATION
+
+general_settings:
+  master_key: sk-1234
+```
+
+**2. Start proxy**
+
+```bash
+litellm --config config.yaml
+```
+
+**3. Call embeddings**
+
+```bash
+curl -X POST http://localhost:4000/embeddings \
+  -H "Authorization: Bearer sk-1234" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "gemini-embedding-2-preview",
+    "input": [
+      "The food was delicious and the waiter...",
+      "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAgAAAAIAQMAAAD+wSzIAAAABlBMVEX///+/v7+jQ3Y5AAAADklEQVQI12P4AIX8EAgALgAD/aNpbtEAAAAASUVORK5CYII"
+    ]
+  }'
+```
+
+</TabItem>
+</Tabs>
+
+## Input Format Examples
+
+| Format | Example | Provider |
+|--------|---------|----------|
+| **Data URI** | `data:image/png;base64,...` | Gemini, Vertex AI |
+| **GCS URL** | `gs://bucket/path/image.png` | Vertex AI |
+| **File reference** | `files/abc123` | Gemini API only |
+
+### Supported MIME Types for Data URIs
+
+- **Images:** `image/png`, `image/jpeg`
+- **Audio:** `audio/mpeg`, `audio/wav`
+- **Video:** `video/mp4`, `video/quicktime`
+- **Documents:** `application/pdf`
+
+### GCS URL MIME Inference
+
+For Vertex AI, MIME types are inferred from file extensions:
+
+- `.png` → `image/png`
+- `.jpg` / `.jpeg` → `image/jpeg`
+- `.mp3` → `audio/mpeg`
+- `.wav` → `audio/wav`
+- `.mp4` → `video/mp4`
+- `.mov` → `video/quicktime`
+- `.pdf` → `application/pdf`
+
+## Optional Parameters
+
+| Parameter | Description | Maps to |
+|-----------|-------------|---------|
+| `dimensions` | Output embedding size | `outputDimensionality` |
+
+```python
+response = embedding(
+    model="gemini/gemini-embedding-2-preview",
+    input=["text to embed"],
+    dimensions=768,  # Optional: control output vector size
+)
+```
@@ -138,6 +138,7 @@ The `/v1/messages/count_tokens` endpoint automatically routes to the appropriate
 | Provider | Token Counting Method |
 |----------|----------------------|
 | Anthropic | [Anthropic Token Counting API](https://docs.anthropic.com/en/docs/build-with-claude/token-counting) |
+| OpenAI | [OpenAI Responses API `/input_tokens`](https://platform.openai.com/docs/api-reference/responses/input-tokens) — see [Token Counting](./count_tokens.md) |
 | Vertex AI (Claude) | Vertex AI Partner Models Token Counter |
 | Bedrock (Claude) | AWS Bedrock CountTokens API |
 | Gemini | Google AI Studio countTokens API |
@@ -11,6 +11,7 @@ This endpoint supports various guardrail types including:
 - **Presidio** - PII detection and masking
 - **Bedrock** - AWS Bedrock guardrails for content moderation
 - **Lakera** - AI safety guardrails
+- **PANW Prisma AIRS** - Threat detection, DLP, and policy enforcement
 - **Custom guardrails** - User-defined guardrails

 ## Configuration
@@ -13,7 +13,7 @@ import TabItem from '@theme/TabItem';
 | Fallbacks | ✅ | Works between supported models |
 | Loadbalancing | ✅ | Works between supported models |
 | Guardrails | ✅ | Applies to output transcribed text (non-streaming only) |
-| Supported Providers | `openai`, `azure`, `vertex_ai`, `gemini`, `deepgram`, `groq`, `fireworks_ai`, `ovhcloud` | |
+| Supported Providers | `openai`, `azure`, `vertex_ai`, `gemini`, `deepgram`, `groq`, `fireworks_ai`, `ovhcloud`, `mistral` | |

 ## Quick Start

@@ -126,6 +126,7 @@ transcript = client.audio.transcriptions.create(
 - [Fireworks AI](./providers/fireworks_ai.md#audio-transcription)
 - [Groq](./providers/groq.md#speech-to-text---whisper)
 - [Deepgram](./providers/deepgram.md)
+- [Mistral (Voxtral)](./providers/mistral.md#audio-transcription)
 - [OVHcloud AI Endpoints](./providers/ovhcloud.md)

 ---
@@ -51,6 +51,28 @@ Here's what an example response looks like
 }
 ```

+## Native Finish Reason
+
+LiteLLM maps all provider-specific `finish_reason` values to OpenAI-compatible values (`stop`, `length`, `tool_calls`, `function_call`, `content_filter`). When the original provider value differs from the mapped value, it is preserved in `provider_specific_fields["native_finish_reason"]`.
+
+This is useful for agent loops that need to distinguish between different stop conditions (e.g., Gemini's `MALFORMED_FUNCTION_CALL` vs a normal `stop`).
+
+```python
+response = completion(model="gemini/gemini-2.0-flash", messages=messages)
+
+choice = response.choices[0]
+print(choice.finish_reason)  # "stop" (OpenAI-compatible)
+
+# Access the original provider value when it differs:
+if hasattr(choice, "provider_specific_fields") and choice.provider_specific_fields:
+    native = choice.provider_specific_fields.get("native_finish_reason")
+    if native == "MALFORMED_FUNCTION_CALL":
+        # Handle malformed function call differently from a normal stop
+        pass
+```
+
+When the provider already returns an OpenAI-compatible value (e.g., `stop`), `native_finish_reason` is not set.
+
 ## Additional Attributes

 You can also access information like latency. 
@@ -115,6 +115,11 @@ print(response)

 Web fetch is available on the following Anthropic API models:

+- `claude-opus-4-6` (Claude Opus 4.6)
+- `claude-sonnet-4-6` (Claude Sonnet 4.6)
+- `claude-opus-4-5` (Claude Opus 4.5)
+- `claude-sonnet-4-5` (Claude Sonnet 4.5)
+- `claude-haiku-4-5` (Claude Haiku 4.5)
 - `claude-opus-4-1-20250805` (Claude Opus 4.1)
 - `claude-opus-4-20250514` (Claude Opus 4)
 - `claude-sonnet-4-20250514` (Claude Sonnet 4)
@@ -80,6 +80,36 @@ That's it! The provider is now available.
 }
 ```

+## Responses API Support
+
+If your provider also supports the OpenAI Responses API (`/v1/responses`), add `supported_endpoints`:
+
+```json
+{
+  "your_provider": {
+    "base_url": "https://api.yourprovider.com/v1",
+    "api_key_env": "YOUR_PROVIDER_API_KEY",
+    "supported_endpoints": ["/v1/chat/completions", "/v1/responses"]
+  }
+}
+```
+
+This enables `litellm.responses()` with zero additional code:
+
+```python
+import litellm
+
+response = litellm.responses(
+    model="your_provider/model-name",
+    input="Hello, what can you do?",
+)
+print(response.output)
+```
+
+If `supported_endpoints` is omitted, it defaults to `[]`. Chat completions is always enabled for JSON providers regardless of this field.
+
+The provider inherits all request/response handling from OpenAI's Responses API — streaming, tools, and all standard parameters work out of the box.
+
 ## Usage

 ```python
@@ -89,11 +119,17 @@ import os
 # Set your API key
 os.environ["YOUR_PROVIDER_API_KEY"] = "your-key-here"

-# Use the provider
+# Chat completions
 response = litellm.completion(
    model="your_provider/model-name",
    messages=[{"role": "user", "content": "Hello"}],
 )
+
+# Responses API (if supported_endpoints includes "/v1/responses")
+response = litellm.responses(
+    model="your_provider/model-name",
+    input="Hello",
+)
 ```

 ## When to Use Python Instead
@@ -105,7 +141,9 @@ Use a Python config class if you need:
 - Provider-specific streaming logic
 - Advanced tool calling modifications

-For these cases, create a config class in `litellm/llms/your_provider/chat/transformation.py` that inherits from `OpenAIGPTConfig` or `OpenAILikeChatConfig`.
+For chat completions, create a config class in `litellm/llms/your_provider/chat/transformation.py` that inherits from `OpenAIGPTConfig` or `OpenAILikeChatConfig`.
+
+For responses API with small overrides, inherit from `OpenAIResponsesAPIConfig` and override only what's needed. See `litellm/llms/perplexity/responses/transformation.py` for a minimal example (~40 lines vs 400+).

 ## Testing

@@ -0,0 +1,189 @@
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Token Counting
+
+## Overview
+
+LiteLLM provides exact token counting by calling provider-specific token counting APIs. This gives you accurate token counts before sending requests, helping with cost estimation and context window management.
+
+| Feature | Details |
+|---------|---------|
+| SDK Method | `litellm.acount_tokens()` |
+| Proxy Endpoints | `/v1/messages/count_tokens` (Anthropic format), `/v1/responses/input_tokens` (OpenAI format) |
+| Fallback | Local tiktoken-based counting for unsupported providers |
+
+## Supported Providers
+
+| Provider | Token Counting API | Format |
+|----------|-------------------|--------|
+| OpenAI | [Responses API `/input_tokens`](https://platform.openai.com/docs/api-reference/responses/input-tokens) | OpenAI Responses |
+| Anthropic | [Messages `/count_tokens`](https://docs.anthropic.com/en/docs/build-with-claude/token-counting) | Anthropic Messages |
+| Vertex AI (Claude) | Vertex AI Partner Models Token Counter | Anthropic Messages |
+| Bedrock (Claude) | AWS Bedrock CountTokens API | Anthropic Messages |
+| Gemini | Google AI Studio countTokens API | Anthropic Messages |
+| Vertex AI (Gemini) | Vertex AI countTokens API | Anthropic Messages |
+| Other providers | Local tiktoken fallback | N/A |
+
+## SDK Usage
+
+### Basic Usage
+
+```python
+import asyncio
+import litellm
+
+async def main():
+    # OpenAI
+    result = await litellm.acount_tokens(
+        model="openai/gpt-4o",
+        messages=[{"role": "user", "content": "Hello, how are you?"}],
+    )
+    print(f"Token count: {result.total_tokens}")
+    print(f"Tokenizer: {result.tokenizer_type}")  # "openai_api"
+
+    # Anthropic
+    result = await litellm.acount_tokens(
+        model="anthropic/claude-3-5-sonnet-20241022",
+        messages=[{"role": "user", "content": "Hello, how are you?"}],
+    )
+    print(f"Token count: {result.total_tokens}")
+    print(f"Tokenizer: {result.tokenizer_type}")  # "anthropic_api"
+
+asyncio.run(main())
+```
+
+### With Tools and System Message
+
+```python
+import asyncio
+import litellm
+
+async def main():
+    result = await litellm.acount_tokens(
+        model="openai/gpt-4o",
+        messages=[{"role": "user", "content": "What's the weather in Paris?"}],
+        tools=[{
+            "type": "function",
+            "function": {
+                "name": "get_weather",
+                "description": "Get weather for a city",
+                "parameters": {
+                    "type": "object",
+                    "properties": {"city": {"type": "string"}},
+                },
+            },
+        }],
+        system="You are a helpful weather assistant.",
+    )
+    print(f"Token count (with tools): {result.total_tokens}")
+
+asyncio.run(main())
+```
+
+### Response Format
+
+`litellm.acount_tokens()` returns a `TokenCountResponse`:
+
+```python
+TokenCountResponse(
+    total_tokens=15,           # Token count
+    request_model="openai/gpt-4o",  # Model requested
+    model_used="gpt-4o",      # Model used for counting
+    tokenizer_type="openai_api",    # "openai_api", "anthropic_api", "local_tokenizer"
+    original_response={"input_tokens": 15},  # Raw API response
+    error=False,               # True if counting failed
+    error_message=None,        # Error details if failed
+)
+```
+
+### Fallback Behavior
+
+If a provider doesn't support a token counting API, or if the API key is missing, `acount_tokens()` automatically falls back to local tiktoken-based counting:
+
+```python
+# Unsupported provider → automatic fallback
+result = await litellm.acount_tokens(
+    model="together_ai/meta-llama/Llama-3-8b-chat-hf",
+    messages=[{"role": "user", "content": "Hello"}],
+)
+print(result.tokenizer_type)  # "local_tokenizer"
+```
+
+## Proxy Usage
+
+### OpenAI Format — `/v1/responses/input_tokens`
+
+<Tabs>
+<TabItem value="curl" label="curl">
+
+```bash
+curl -X POST "http://localhost:4000/v1/responses/input_tokens" \
+  -H "Content-Type: application/json" \
+  -H "Authorization: Bearer sk-1234" \
+  -d '{
+    "model": "gpt-4o",
+    "input": "Hello, how are you?"
+  }'
+```
+
+</TabItem>
+<TabItem value="python" label="Python (httpx)">
+
+```python
+import httpx
+
+response = httpx.post(
+    "http://localhost:4000/v1/responses/input_tokens",
+    headers={
+        "Content-Type": "application/json",
+        "Authorization": "Bearer sk-1234"
+    },
+    json={
+        "model": "gpt-4o",
+        "input": "Hello, how are you?"
+    }
+)
+
+print(response.json())
+# {"input_tokens": 7}
+```
+
+</TabItem>
+</Tabs>
+
+**Response:**
+```json
+{"input_tokens": 7}
+```
+
+### Anthropic Format — `/v1/messages/count_tokens`
+
+See [Anthropic Token Counting](./anthropic_count_tokens.md) for full documentation.
+
+```bash
+curl -X POST "http://localhost:4000/v1/messages/count_tokens" \
+  -H "Content-Type: application/json" \
+  -H "Authorization: Bearer sk-1234" \
+  -d '{
+    "model": "claude-3-5-sonnet-20241022",
+    "messages": [
+      {"role": "user", "content": "Hello, how are you?"}
+    ]
+  }'
+```
+
+## Proxy Configuration
+
+```yaml
+model_list:
+  - model_name: gpt-4o
+    litellm_params:
+      model: openai/gpt-4o
+      api_key: os.environ/OPENAI_API_KEY
+
+  - model_name: claude-3-5-sonnet
+    litellm_params:
+      model: anthropic/claude-3-5-sonnet-20241022
+      api_key: os.environ/ANTHROPIC_API_KEY
+```
@@ -514,6 +514,57 @@ All models listed [here](https://ai.google.dev/gemini-api/docs/models/gemini) ar
 | Model Name         | Function Call                                         |
 | :---               | :---                                                  |
 | text-embedding-004 | `embedding(model="gemini/text-embedding-004", input)` |
+| gemini-embedding-2-preview | `embedding(model="gemini/gemini-embedding-2-preview", input)` | [Multimodal docs](#gemini-embedding-2-preview-multimodal) |
+
+### Gemini Embedding 2 Preview (Multimodal)
+
+`gemini-embedding-2-preview` supports **multimodal embeddings**—text, images, audio, video, and PDF in a single request. See [blog post](/blog/gemini_embedding_2_multimodal) for details.
+
+**Input formats:**
+- **Data URIs:** `data:image/png;base64,<encoded_data>`
+- **Gemini file references:** `files/abc123` (pre-uploaded via Gemini Files API)
+
+**Supported MIME types:** `image/png`, `image/jpeg`, `audio/mpeg`, `audio/wav`, `video/mp4`, `video/quicktime`, `application/pdf`
+
+<Tabs>
+<TabItem value="sdk" label="SDK">
+
+```python
+from litellm import embedding
+import os
+os.environ["GEMINI_API_KEY"] = ""
+
+# Text + Image (base64)
+response = embedding(
+    model="gemini/gemini-embedding-2-preview",
+    input=[
+        "The food was delicious and the waiter...",
+        "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAgAAAAIAQMAAAD+wSzIAAAABlBMVEX///+/v7+jQ3Y5AAAADklEQVQI12P4AIX8EAgALgAD/aNpbtEAAAAASUVORK5CYII"
+    ],
+)
+print(response)
+```
+
+</TabItem>
+<TabItem value="proxy" label="PROXY">
+
+```bash
+curl -X POST http://localhost:4000/embeddings \
+  -H "Authorization: Bearer sk-1234" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "gemini-embedding-2-preview",
+    "input": [
+      "The food was delicious and the waiter...",
+      "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAgAAAAIAQMAAAD+wSzIAAAABlBMVEX///+/v7+jQ3Y5AAAADklEQVQI12P4AIX8EAgALgAD/aNpbtEAAAAASUVORK5CYII"
+    ]
+  }'
+```
+
+</TabItem>
+</Tabs>
+
+**Optional:** `dimensions` maps to Gemini's `outputDimensionality`.


 ## Vertex AI Embedding Models
@@ -16,7 +16,7 @@ LiteLLM provides image editing functionality that maps to OpenAI's `/images/edit
 | Supported operations | Create image edits | Single and multiple images supported |
 | Supported LiteLLM SDK Versions | 1.63.8+ | Gemini support requires 1.79.3+ |
 | Supported LiteLLM Proxy Versions | 1.71.1+ | Gemini support requires 1.79.3+ |
-| Supported LLM providers | **OpenAI**, **Gemini (Google AI Studio)**, **Vertex AI**, **OpenRouter**, **Stability AI**, **AWS Bedrock (Stability)** | Gemini supports the new `gemini-2.5-flash-image` family. Vertex AI supports both Gemini and Imagen models. OpenRouter routes image edits through chat completions. Stability AI and Bedrock Stability support various image editing operations. |
+| Supported LLM providers | **OpenAI**, **Gemini (Google AI Studio)**, **Vertex AI**, **OpenRouter**, **Stability AI**, **AWS Bedrock (Stability)**, **Black Forest Labs** | Gemini supports the new `gemini-2.5-flash-image` family. Vertex AI supports both Gemini and Imagen models. OpenRouter routes image edits through chat completions. Stability AI and Bedrock Stability support various image editing operations. Black Forest Labs supports FLUX Kontext models. |

 #### ⚡️See all supported models and providers at [models.litellm.ai](https://models.litellm.ai/)

@@ -199,6 +199,63 @@ for idx, image_obj in enumerate(response.data):

 </TabItem>

+<TabItem value="bfl" label="Black Forest Labs">
+
+#### Basic Image Edit
+```python showLineNumbers title="Black Forest Labs Image Edit"
+import os
+import litellm
+
+os.environ["BFL_API_KEY"] = "your-api-key"
+
+response = litellm.image_edit(
+    model="black_forest_labs/flux-kontext-pro",
+    image=open("original_image.png", "rb"),
+    prompt="Add a green leaf to the scene",
+)
+
+print(response.data[0].url)
+```
+
+#### Inpainting with Mask
+```python showLineNumbers title="Black Forest Labs Inpainting"
+import os
+import litellm
+
+os.environ["BFL_API_KEY"] = "your-api-key"
+
+# Use flux-pro-1.0-fill for inpainting
+response = litellm.image_edit(
+    model="black_forest_labs/flux-pro-1.0-fill",
+    image=open("original_image.png", "rb"),
+    mask=open("mask_image.png", "rb"),
+    prompt="Replace with a garden",
+)
+
+print(response.data[0].url)
+```
+
+#### Outpainting (Expand)
+```python showLineNumbers title="Black Forest Labs Outpainting"
+import os
+import litellm
+
+os.environ["BFL_API_KEY"] = "your-api-key"
+
+# Use flux-pro-1.0-expand to extend image borders
+response = litellm.image_edit(
+    model="black_forest_labs/flux-pro-1.0-expand",
+    image=open("original_image.png", "rb"),
+    prompt="Continue the scene with mountains",
+    top=256,
+    bottom=256,
+)
+
+print(response.data[0].url)
+```
+
+</TabItem>
+
 <TabItem value="vertex_ai" label="Vertex AI">

 #### Basic Image Edit (Gemini)
@@ -392,6 +449,35 @@ curl -X POST "http://0.0.0.0:4000/v1/images/edits" \

 </TabItem>

+<TabItem value="bfl" label="Black Forest Labs">
+
+1. Add Black Forest Labs image edit models to your `config.yaml`:
+```yaml showLineNumbers title="Black Forest Labs Proxy Configuration"
+model_list:
+  - model_name: bfl-kontext-pro
+    litellm_params:
+      model: black_forest_labs/flux-kontext-pro
+      api_key: os.environ/BFL_API_KEY
+    model_info:
+      mode: image_edit
+```
+
+2. Start the LiteLLM proxy server:
+```bash showLineNumbers title="Start LiteLLM Proxy Server"
+litellm --config /path/to/config.yaml
+```
+
+3. Make an image edit request:
+```bash showLineNumbers title="Black Forest Labs Proxy Image Edit"
+curl -X POST "http://0.0.0.0:4000/v1/images/edits" \
+  -H "Authorization: Bearer <YOUR-LITELLM-KEY>" \
+  -F "model=bfl-kontext-pro" \
+  -F "image=@original_image.png" \
+  -F "prompt=Add a sunset in the background"
+```
+
+</TabItem>
+
 <TabItem value="vertex_ai" label="Vertex AI">

 1. Add Vertex AI image edit models to your `config.yaml`:
@@ -15,7 +15,7 @@ import TabItem from '@theme/TabItem';
 | Fallbacks | ✅ | Works between supported models |
 | Loadbalancing | ✅ | Works between supported models |
 | Guardrails | ✅ | Applies to input prompts (non-streaming only) |
-| Supported Providers | OpenAI, Azure, Google AI Studio, Vertex AI, AWS Bedrock, Recraft, OpenRouter, Xinference, Nscale | |
+| Supported Providers | OpenAI, Azure, Google AI Studio, Vertex AI, AWS Bedrock, Black Forest Labs, Recraft, OpenRouter, Xinference, Nscale | |

 ## Quick Start

@@ -133,6 +133,21 @@ LiteLLM attempts [OAuth 2.0 Authorization Server Discovery](https://datatracker.

 <br/>

+### AWS SigV4 Authentication
+
+For MCP servers hosted on [AWS Bedrock AgentCore](https://docs.aws.amazon.com/bedrock/latest/userguide/agentcore.html), select **AWS SigV4** as the authentication type. LiteLLM will sign every outgoing MCP request with your AWS credentials using [Signature Version 4](https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html).
+
+<Image
+  img={require('../img/mcp_aws_sigv4_ui.png')}
+  style={{width: '80%', display: 'block', margin: '0'}}
+/>
+
+Fill in your AWS region, service name (defaults to `bedrock-agentcore`), and optionally your AWS access key and secret. If credentials are omitted, LiteLLM falls back to the boto3 credential chain (IAM roles, environment variables, etc.).
+
+[**See full SigV4 setup guide**](./mcp_aws_sigv4.md)
+
+<br/>
+
 ### Static Headers

 Sometimes your MCP server needs specific headers on every request. Maybe it's an API key, maybe it's a custom header the server expects. Instead of configuring auth, you can just set them directly.
@@ -1,3 +1,7 @@
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+import Image from '@theme/IdealImage';
+
 # MCP - AWS SigV4 Auth

 Use AWS SigV4 authentication to connect LiteLLM to MCP servers hosted on [AWS Bedrock AgentCore](https://docs.aws.amazon.com/bedrock/latest/userguide/agentcore.html).
@@ -10,6 +14,36 @@ LiteLLM's `aws_sigv4` auth type handles this automatically: every outgoing MCP r

 ## Quick Start

+<Tabs>
+<TabItem value="ui" label="LiteLLM UI">
+
+1. Navigate to **MCP Servers** and click **Add New MCP Server**
+2. Set the transport to **Streamable HTTP**
+3. Select **AWS SigV4** as the authentication type
+4. Fill in your AWS credentials:
+
+<Image
+  img={require('../img/mcp_aws_sigv4_ui.png')}
+  style={{width: '80%', display: 'block', margin: '0'}}
+/>
+
+<br/>
+
+| Field | Required | Description |
+|-------|----------|-------------|
+| **AWS Region** | Yes | AWS region for SigV4 signing (e.g., `us-east-1`) |
+| **AWS Service Name** | No | Defaults to `bedrock-agentcore` |
+| **AWS Access Key ID** | No | Falls back to boto3 credential chain if blank |
+| **AWS Secret Access Key** | No | Required if Access Key ID is provided |
+| **AWS Session Token** | No | Only needed for temporary STS credentials |
+
+Once created, LiteLLM will sign every outgoing MCP request with SigV4. The server's tools appear automatically in the MCP Tools list.
+
+**Editing credentials:** When editing an existing SigV4 server, leave credential fields blank to keep the current values. Only fields you fill in will be updated.
+
+</TabItem>
+<TabItem value="config" label="config.yaml">
+
 ### 1. Set AWS credentials

 ```bash
@@ -60,9 +94,12 @@ arn%3Aaws%3Abedrock-agentcore%3Aus-east-1%3A123456789012%3Aruntime%2Fmy-mcp-serv
 litellm --config config.yaml
 ```

-### 4. Use the MCP tools
+</TabItem>
+</Tabs>

-Once started, your AgentCore MCP tools are available through LiteLLM like any other MCP server:
+## Use the MCP tools
+
+Once configured, your AgentCore MCP tools are available through LiteLLM like any other MCP server:

 ```bash title="List available tools"
 curl http://localhost:4000/mcp-rest/tools/list \
@@ -86,4 +86,5 @@ MCP guardrails work with all LiteLLM-supported guardrail providers:
 - **Lakera**: Content moderation
 - **Aporia**: Custom guardrails
 - **Noma**: Noma Security
+- **PANW Prisma AIRS**: Prisma AIRS guardrails
 - **Custom**: Your own guardrail implementations
@@ -13,6 +13,7 @@ Here's the full specification with all available fields:
 ```json
 {
    "sample_spec": {
+        "aliases": ["optional list of alternate names for this model, e.g. dated versions like sample_spec-20250101"],
        "code_interpreter_cost_per_session": 0.0,
        "computer_use_input_cost_per_1k_tokens": 0.0,
        "computer_use_output_cost_per_1k_tokens": 0.0,
@@ -121,4 +122,28 @@ Here's the full specification with all available fields:
 }
 ```

-That's it! Your PR will be reviewed and merged.
+### Using Aliases
+
+Many providers release the same model under multiple names — for example, a `latest` tag and a dated version like `claude-sonnet-4-5-20250929`. Instead of duplicating the entire entry, you can use the `aliases` field:
+
+```json
+{
+    "claude-sonnet-4-5": {
+        "aliases": ["claude-sonnet-4-5-20250929"],
+        "input_cost_per_token": 3e-06,
+        "output_cost_per_token": 1.5e-05,
+        "litellm_provider": "anthropic",
+        "max_input_tokens": 200000,
+        "max_output_tokens": 64000,
+        "mode": "chat",
+        "supports_function_calling": true,
+        "supports_tool_choice": true
+    }
+}
+```
+
+At load time, each alias is expanded into a top-level entry sharing the same data as the canonical entry. The example above makes both `claude-sonnet-4-5` and `claude-sonnet-4-5-20250929` resolve with the same pricing and capabilities.
+
+:::info
+This is different from [`model_alias_map`](../completion/model_alias.md), which is a runtime SDK/proxy feature for mapping user-facing model names to LiteLLM model identifiers. The `aliases` field here is for the model cost JSON only — it avoids duplicate entries for models that share identical pricing and capabilities.
+:::
@@ -0,0 +1,291 @@
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Black Forest Labs Image Generation
+
+Black Forest Labs provides state-of-the-art text-to-image generation using their FLUX models.
+
+## Overview
+
+| Property | Details |
+|----------|---------|
+| Description | Black Forest Labs FLUX models for high-quality text-to-image generation |
+| Provider Route on LiteLLM | `black_forest_labs/` |
+| Provider Doc | [Black Forest Labs API ↗](https://docs.bfl.ai/) |
+| Supported Operations | [`/images/generations`](#image-generation) |
+
+## Setup
+
+### API Key
+
+```python showLineNumbers
+import os
+
+# Set your Black Forest Labs API key
+os.environ["BFL_API_KEY"] = "your-api-key-here"
+```
+
+Get your API key from [Black Forest Labs](https://blackforestlabs.ai/).
+
+## Supported Models
+
+| Model Name | Description | Price |
+|------------|-------------|-------|
+| `black_forest_labs/flux-pro-1.1` | Fast & reliable standard generation | $0.04/image |
+| `black_forest_labs/flux-pro-1.1-ultra` | Ultra high-resolution (up to 4MP) | $0.06/image |
+| `black_forest_labs/flux-dev` | Development/open-source variant | $0.025/image |
+| `black_forest_labs/flux-pro` | Original pro model | $0.05/image |
+
+## Image Generation
+
+### Usage - LiteLLM Python SDK
+
+<Tabs>
+<TabItem value="basic" label="Basic Usage">
+
+```python showLineNumbers title="Basic Image Generation"
+import os
+import litellm
+
+# Set your API key
+os.environ["BFL_API_KEY"] = "your-api-key-here"
+
+# Generate an image
+response = litellm.image_generation(
+    model="black_forest_labs/flux-pro-1.1",
+    prompt="A beautiful sunset over the ocean with sailing boats",
+)
+
+# BFL returns URLs
+print(response.data[0].url)
+```
+
+</TabItem>
+
+<TabItem value="async" label="Async Usage">
+
+```python showLineNumbers title="Async Image Generation"
+import os
+import asyncio
+import litellm
+
+# Set your API key
+os.environ["BFL_API_KEY"] = "your-api-key-here"
+
+async def generate_image():
+    response = await litellm.aimage_generation(
+        model="black_forest_labs/flux-pro-1.1",
+        prompt="A futuristic city skyline at night",
+    )
+    print(response.data[0].url)
+
+# Run the async function
+asyncio.run(generate_image())
+```
+
+</TabItem>
+
+<TabItem value="size" label="Custom Size">
+
+```python showLineNumbers title="Image Generation with Custom Size"
+import os
+import litellm
+
+# Set your API key
+os.environ["BFL_API_KEY"] = "your-api-key-here"
+
+# Generate with specific dimensions
+response = litellm.image_generation(
+    model="black_forest_labs/flux-pro-1.1",
+    prompt="A majestic mountain landscape",
+    size="1792x1024",  # Maps to width/height
+)
+
+print(response.data[0].url)
+```
+
+</TabItem>
+
+<TabItem value="ultra" label="Ultra High-Res">
+
+```python showLineNumbers title="Ultra High Resolution with flux-pro-1.1-ultra"
+import os
+import litellm
+
+# Set your API key
+os.environ["BFL_API_KEY"] = "your-api-key-here"
+
+# Generate ultra high-resolution image
+response = litellm.image_generation(
+    model="black_forest_labs/flux-pro-1.1-ultra",
+    prompt="Detailed portrait of a fantasy character",
+    size="2048x2048",    # Up to 4MP supported
+    quality="hd",        # Maps to raw=True for natural look
+)
+
+print(response.data[0].url)
+```
+
+</TabItem>
+
+<TabItem value="advanced" label="Advanced Parameters">
+
+```python showLineNumbers title="Advanced Image Generation with BFL Parameters"
+import os
+import litellm
+
+# Set your API key
+os.environ["BFL_API_KEY"] = "your-api-key-here"
+
+# Generate with BFL-specific parameters
+response = litellm.image_generation(
+    model="black_forest_labs/flux-pro-1.1",
+    prompt="A cute orange cat sitting on a windowsill",
+    seed=42,                    # For reproducible results
+    output_format="png",        # png or jpeg
+    safety_tolerance=2,         # 0-6, higher = more permissive
+    prompt_upsampling=True,     # Enhance prompt for better results
+)
+
+print(response.data[0].url)
+```
+
+</TabItem>
+</Tabs>
+
+### Usage - LiteLLM Proxy Server
+
+#### 1. Configure your config.yaml
+
+```yaml showLineNumbers title="Black Forest Labs Image Generation Configuration"
+model_list:
+  - model_name: flux-pro
+    litellm_params:
+      model: black_forest_labs/flux-pro-1.1
+      api_key: os.environ/BFL_API_KEY
+    model_info:
+      mode: image_generation
+
+  - model_name: flux-ultra
+    litellm_params:
+      model: black_forest_labs/flux-pro-1.1-ultra
+      api_key: os.environ/BFL_API_KEY
+    model_info:
+      mode: image_generation
+
+  - model_name: flux-dev
+    litellm_params:
+      model: black_forest_labs/flux-dev
+      api_key: os.environ/BFL_API_KEY
+    model_info:
+      mode: image_generation
+
+general_settings:
+  master_key: sk-1234
+```
+
+#### 2. Start LiteLLM Proxy Server
+
+```bash showLineNumbers title="Start LiteLLM Proxy Server"
+litellm --config /path/to/config.yaml
+
+# RUNNING on http://0.0.0.0:4000
+```
+
+#### 3. Make image generation requests
+
+<Tabs>
+<TabItem value="openai-sdk" label="OpenAI SDK">
+
+```python showLineNumbers title="Black Forest Labs via Proxy - OpenAI SDK"
+from openai import OpenAI
+
+# Initialize client with your proxy URL
+client = OpenAI(
+    base_url="http://localhost:4000",
+    api_key="sk-1234"
+)
+
+# Generate image with FLUX Pro
+response = client.images.generate(
+    model="flux-pro",
+    prompt="A beautiful garden with colorful flowers",
+    size="1024x1024",
+)
+
+print(response.data[0].url)
+```
+
+</TabItem>
+
+<TabItem value="curl" label="cURL">
+
+```bash showLineNumbers title="Black Forest Labs via Proxy - cURL"
+curl -X POST 'http://localhost:4000/v1/images/generations' \
+  -H 'Content-Type: application/json' \
+  -H 'Authorization: Bearer sk-1234' \
+  -d '{
+    "model": "flux-pro",
+    "prompt": "A beautiful garden with colorful flowers",
+    "size": "1024x1024"
+  }'
+```
+
+</TabItem>
+</Tabs>
+
+## Supported Parameters
+
+### OpenAI-Compatible Parameters
+
+| Parameter | Type | Description | Mapping |
+|-----------|------|-------------|---------|
+| `prompt` | string | Text description of the image to generate | Direct |
+| `model` | string | The FLUX model to use | Direct |
+| `size` | string | Image dimensions (e.g., `1024x1024`) | Maps to `width` and `height` |
+| `n` | integer | Number of images (ultra model only, up to 4) | Maps to `num_images` |
+| `quality` | string | `hd` for natural look | Maps to `raw=True` for ultra |
+| `response_format` | string | `url` or `b64_json` | Direct |
+
+### Black Forest Labs Specific Parameters
+
+| Parameter | Type | Description | Default |
+|-----------|------|-------------|---------|
+| `width` | integer | Image width (256-1920, multiples of 16) | 1024 |
+| `height` | integer | Image height (256-1920, multiples of 16) | 1024 |
+| `aspect_ratio` | string | Alternative to width/height (e.g., `16:9`, `1:1`) | - |
+| `seed` | integer | Seed for reproducible results | Random |
+| `output_format` | string | Output format: `png` or `jpeg` | `png` |
+| `safety_tolerance` | integer | Safety filter tolerance (0-6, higher = more permissive) | 2 |
+| `prompt_upsampling` | boolean | Enhance prompt for better results | `false` |
+
+### Ultra Model Specific Parameters
+
+| Parameter | Type | Description | Default |
+|-----------|------|-------------|---------|
+| `raw` | boolean | Raw mode for more natural, less synthetic look | `false` |
+| `num_images` | integer | Number of images to generate (1-4) | 1 |
+
+## How It Works
+
+Black Forest Labs uses a polling-based API:
+
+1. **Submit Request**: LiteLLM sends your prompt to BFL
+2. **Get Task ID**: BFL returns a task ID and polling URL
+3. **Poll for Result**: LiteLLM automatically polls until the image is ready
+4. **Return Result**: The generated image URL is returned
+
+This polling is handled automatically by LiteLLM - you just call `image_generation()` and get the result.
+
+## Getting Started
+
+1. Create an account at [Black Forest Labs](https://blackforestlabs.ai/)
+2. Get your API key from the dashboard
+3. Set your `BFL_API_KEY` environment variable
+4. Use `litellm.image_generation()` with any supported model
+
+## Additional Resources
+
+- [Black Forest Labs Documentation](https://docs.bfl.ai/)
+- [Black Forest Labs Image Editing](./black_forest_labs_img_edit.md) - For editing existing images
+- [FLUX Model Information](https://blackforestlabs.ai/)
@@ -0,0 +1,301 @@
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Black Forest Labs Image Editing
+
+Black Forest Labs provides powerful image editing capabilities using their FLUX models to modify existing images based on text descriptions.
+
+## Overview
+
+| Property | Details |
+|----------|---------|
+| Description | Black Forest Labs Image Editing uses FLUX Kontext and other models to modify, inpaint, and expand images based on text prompts. |
+| Provider Route on LiteLLM | `black_forest_labs/` |
+| Provider Doc | [Black Forest Labs API ↗](https://docs.bfl.ai/) |
+| Supported Operations | [`/images/edits`](#image-editing) |
+
+## Setup
+
+### API Key
+
+```python showLineNumbers
+import os
+
+# Set your Black Forest Labs API key
+os.environ["BFL_API_KEY"] = "your-api-key-here"
+```
+
+Get your API key from [Black Forest Labs](https://blackforestlabs.ai/).
+
+## Supported Models
+
+| Model Name | Description | Use Case |
+|------------|-------------|----------|
+| `black_forest_labs/flux-kontext-pro` | FLUX Kontext Pro - General image editing with prompts | General editing, style transfer |
+| `black_forest_labs/flux-kontext-max` | FLUX Kontext Max - Premium quality editing | High-quality edits |
+| `black_forest_labs/flux-pro-1.0-fill` | FLUX Pro Fill - Inpainting with mask | Remove/replace objects |
+| `black_forest_labs/flux-pro-1.0-expand` | FLUX Pro Expand - Outpainting | Expand image borders |
+
+## Image Editing
+
+### Usage - LiteLLM Python SDK
+
+<Tabs>
+<TabItem value="basic-edit" label="Basic Usage">
+
+```python showLineNumbers title="Basic Image Editing"
+import os
+import litellm
+
+# Set your API key
+os.environ["BFL_API_KEY"] = "your-api-key-here"
+
+# Edit an image with a prompt
+response = litellm.image_edit(
+    model="black_forest_labs/flux-kontext-pro",
+    image=open("path/to/your/image.png", "rb"),
+    prompt="Add a green leaf to the scene",
+)
+
+# BFL returns URLs
+print(response.data[0].url)
+```
+
+</TabItem>
+
+<TabItem value="async-edit" label="Async Usage">
+
+```python showLineNumbers title="Async Image Editing"
+import os
+import asyncio
+import litellm
+
+# Set your API key
+os.environ["BFL_API_KEY"] = "your-api-key-here"
+
+async def edit_image():
+    response = await litellm.aimage_edit(
+        model="black_forest_labs/flux-kontext-pro",
+        image=open("path/to/your/image.png", "rb"),
+        prompt="Make this image look like a watercolor painting",
+    )
+    print(response.data[0].url)
+
+# Run the async function
+asyncio.run(edit_image())
+```
+
+</TabItem>
+
+<TabItem value="inpainting" label="Inpainting (Fill)">
+
+```python showLineNumbers title="Inpainting with Mask"
+import os
+import litellm
+
+# Set your API key
+os.environ["BFL_API_KEY"] = "your-api-key-here"
+
+# Use flux-pro-1.0-fill for inpainting
+response = litellm.image_edit(
+    model="black_forest_labs/flux-pro-1.0-fill",
+    image=open("path/to/your/image.png", "rb"),
+    mask=open("path/to/mask.png", "rb"),  # White areas will be edited
+    prompt="Replace with a beautiful garden",
+    steps=50,  # BFL-specific parameter
+    guidance=30,  # BFL-specific parameter
+)
+
+print(response.data[0].url)
+```
+
+</TabItem>
+
+<TabItem value="outpainting" label="Outpainting (Expand)">
+
+```python showLineNumbers title="Outpainting - Expand Image Borders"
+import os
+import litellm
+
+# Set your API key
+os.environ["BFL_API_KEY"] = "your-api-key-here"
+
+# Use flux-pro-1.0-expand to extend image borders
+response = litellm.image_edit(
+    model="black_forest_labs/flux-pro-1.0-expand",
+    image=open("path/to/your/image.png", "rb"),
+    prompt="Continue the scene with a mountain landscape",
+    top=256,     # Expand 256 pixels at top
+    bottom=256,  # Expand 256 pixels at bottom
+    left=128,    # Expand 128 pixels at left
+    right=128,   # Expand 128 pixels at right
+)
+
+print(response.data[0].url)
+```
+
+</TabItem>
+
+<TabItem value="advanced" label="Advanced Parameters">
+
+```python showLineNumbers title="Advanced Image Editing with BFL Parameters"
+import os
+import litellm
+
+# Set your API key
+os.environ["BFL_API_KEY"] = "your-api-key-here"
+
+# Edit image with BFL-specific parameters
+response = litellm.image_edit(
+    model="black_forest_labs/flux-kontext-pro",
+    image=open("path/to/your/image.png", "rb"),
+    prompt="Transform into cyberpunk style with neon lights",
+    seed=42,                    # For reproducible results
+    output_format="png",        # png or jpeg
+    safety_tolerance=2,         # 0-6, higher = more permissive
+    aspect_ratio="16:9",        # Output aspect ratio
+)
+
+print(response.data[0].url)
+```
+
+</TabItem>
+</Tabs>
+
+### Usage - LiteLLM Proxy Server
+
+#### 1. Configure your config.yaml
+
+```yaml showLineNumbers title="Black Forest Labs Image Editing Configuration"
+model_list:
+  - model_name: bfl-kontext-pro
+    litellm_params:
+      model: black_forest_labs/flux-kontext-pro
+      api_key: os.environ/BFL_API_KEY
+    model_info:
+      mode: image_edit
+
+  - model_name: bfl-kontext-max
+    litellm_params:
+      model: black_forest_labs/flux-kontext-max
+      api_key: os.environ/BFL_API_KEY
+    model_info:
+      mode: image_edit
+
+  - model_name: bfl-fill
+    litellm_params:
+      model: black_forest_labs/flux-pro-1.0-fill
+      api_key: os.environ/BFL_API_KEY
+    model_info:
+      mode: image_edit
+
+  - model_name: bfl-expand
+    litellm_params:
+      model: black_forest_labs/flux-pro-1.0-expand
+      api_key: os.environ/BFL_API_KEY
+    model_info:
+      mode: image_edit
+
+general_settings:
+  master_key: sk-1234
+```
+
+#### 2. Start LiteLLM Proxy Server
+
+```bash showLineNumbers title="Start LiteLLM Proxy Server"
+litellm --config /path/to/config.yaml
+
+# RUNNING on http://0.0.0.0:4000
+```
+
+#### 3. Make image editing requests
+
+<Tabs>
+<TabItem value="openai-sdk" label="OpenAI SDK">
+
+```python showLineNumbers title="Black Forest Labs via Proxy - OpenAI SDK"
+from openai import OpenAI
+
+# Initialize client with your proxy URL
+client = OpenAI(
+    base_url="http://localhost:4000",
+    api_key="sk-1234"
+)
+
+# Edit image with FLUX Kontext Pro
+response = client.images.edit(
+    model="bfl-kontext-pro",
+    image=open("path/to/your/image.png", "rb"),
+    prompt="Add magical sparkles and fairy dust",
+)
+
+print(response.data[0].url)
+```
+
+</TabItem>
+
+<TabItem value="curl" label="cURL">
+
+```bash showLineNumbers title="Black Forest Labs via Proxy - cURL"
+curl --location 'http://localhost:4000/v1/images/edits' \
+--header 'Authorization: Bearer sk-1234' \
+--form 'model="bfl-kontext-pro"' \
+--form 'prompt="Add a sunset in the background"' \
+--form 'image=@"path/to/your/image.png"'
+```
+
+</TabItem>
+</Tabs>
+
+## Supported Parameters
+
+### OpenAI-Compatible Parameters
+
+| Parameter | Type | Description | Default |
+|-----------|------|-------------|---------|
+| `image` | file | The image file to edit | Required |
+| `prompt` | string | Text description of the desired changes | Required |
+| `model` | string | The FLUX model to use | Required |
+| `mask` | file | Mask image for inpainting (flux-pro-1.0-fill) | Optional |
+| `n` | integer | Number of images (BFL returns 1 per request) | `1` |
+| `size` | string | Maps to aspect_ratio | Optional |
+| `response_format` | string | `url` or `b64_json` | `url` |
+
+### Black Forest Labs Specific Parameters
+
+| Parameter | Type | Description | Default | Models |
+|-----------|------|-------------|---------|--------|
+| `seed` | integer | Seed for reproducible results | Random | All |
+| `output_format` | string | Output format: `png` or `jpeg` | `png` | All |
+| `safety_tolerance` | integer | Safety filter tolerance (0-6) | 2 | All |
+| `aspect_ratio` | string | Output aspect ratio (e.g., `16:9`, `1:1`) | Original | Kontext models |
+| `steps` | integer | Number of inference steps | Model default | Fill |
+| `guidance` | float | Guidance scale | Model default | Fill |
+| `grow_mask` | integer | Pixels to grow mask | 0 | Fill |
+| `top` | integer | Pixels to expand at top | 0 | Expand |
+| `bottom` | integer | Pixels to expand at bottom | 0 | Expand |
+| `left` | integer | Pixels to expand at left | 0 | Expand |
+| `right` | integer | Pixels to expand at right | 0 | Expand |
+
+## How It Works
+
+Black Forest Labs uses a polling-based API:
+
+1. **Submit Request**: LiteLLM sends your image and prompt to BFL
+2. **Get Task ID**: BFL returns a task ID and polling URL
+3. **Poll for Result**: LiteLLM automatically polls until the image is ready
+4. **Return Result**: The generated image URL is returned
+
+This polling is handled automatically by LiteLLM - you just call `image_edit()` and get the result.
+
+## Getting Started
+
+1. Create an account at [Black Forest Labs](https://blackforestlabs.ai/)
+2. Get your API key from the dashboard
+3. Set your `BFL_API_KEY` environment variable
+4. Use `litellm.image_edit()` with any supported model
+
+## Additional Resources
+
+- [Black Forest Labs Documentation](https://docs.bfl.ai/)
+- [FLUX Model Information](https://blackforestlabs.ai/)
@@ -1562,13 +1562,18 @@ LiteLLM Supports the following image types passed in `url`

 ## Media Resolution Control (Images & Videos)

-For Gemini 3+ models, LiteLLM supports per-part media resolution control using OpenAI's `detail` parameter. This allows you to specify different resolution levels for individual images and videos in your request, whether using `image_url` or `file` content types.
+LiteLLM supports OpenAI's `detail` parameter for specifying the image resolution when using Gemini models. The behavior differs between Gemini versions:
+
+| Gemini Version | Resolution Control | Behavior |
+|----------------|-------------------|----------|
+| Gemini 3+ | Per-part | Each image/video can have its own `detail` setting |
+| Gemini 2.x (2.0, 2.5) | Global | The highest `detail` from all images is applied globally via `mediaResolution` in `generationConfig` |

 **Supported `detail` values:**
- `"low"` - Maps to `media_resolution: "low"` (280 tokens for images, 70 tokens per frame for videos)
- `"medium"` - Maps to `media_resolution: "medium"`
- `"high"` - Maps to `media_resolution: "high"` (1120 tokens for images)
- `"ultra_high"` - Maps to `media_resolution: "ultra_high"`
+- `"low"` - Maps to `MEDIA_RESOLUTION_LOW` (280 tokens for images, 70 tokens per frame for videos)
+- `"medium"` - Maps to `MEDIA_RESOLUTION_MEDIUM`
+- `"high"` - Maps to `MEDIA_RESOLUTION_HIGH` (1120 tokens for images)
+- `"ultra_high"` - Maps to `MEDIA_RESOLUTION_ULTRA_HIGH`
 - `"auto"` or `None` - Model decides optimal resolution (no `media_resolution` set)

 **Usage Examples:**
@@ -1605,8 +1610,9 @@ messages = [
    }
 ]

+# Works with both Gemini 2.x and 3+
 response = completion(
-    model="gemini/gemini-3-pro-preview",
+    model="gemini/gemini-2.5-flash",  # or gemini-3-pro-preview
    messages=messages,
 )
 ```
@@ -1647,7 +1653,9 @@ response = completion(
 </Tabs>

 :::info
-**Per-Part Resolution:** Each image or video in your request can have its own `detail` setting, allowing mixed-resolution requests (e.g., a high-res chart alongside a low-res icon). This feature works with both `image_url` and `file` content types, and is only available for Gemini 3+ models.
+**Gemini 3+ Per-Part Resolution:** Each image or video can have its own `detail` setting, allowing mixed-resolution requests (e.g., a high-res chart alongside a low-res icon). This works with both `image_url` and `file` content types.
+
+**Gemini 2.x Global Resolution:** When multiple images have different `detail` values, LiteLLM uses the highest resolution found and applies it globally via `mediaResolution` in `generationConfig` (e.g., if one image has `"low"` and another has `"high"`, all images will use `"high"`).
 :::

 ## Video Metadata Control
@@ -311,6 +311,79 @@ print(response)
 - **Model Compatibility**: Reasoning parameters only work with magistral models
 - **Backward Compatibility**: Non-magistral models will ignore reasoning parameters and work normally

+## Audio Transcription
+
+Use Mistral's Voxtral models for audio transcription via `litellm.transcription()`.
+
+### SDK Usage
+
+```python
+from litellm import transcription
+import os
+
+os.environ["MISTRAL_API_KEY"] = ""
+
+audio_file = open("path/to/audio.wav", "rb")
+
+response = transcription(
+    model="mistral/voxtral-mini-latest",
+    file=audio_file,
+)
+
+print(response.text)
+```
+
+### With Optional Parameters
+
+```python
+response = transcription(
+    model="mistral/voxtral-mini-latest",
+    file=audio_file,
+    language="en",
+    temperature=0.0,
+    response_format="json",
+)
+```
+
+### Mistral-Specific Parameters
+
+Mistral supports additional parameters beyond the OpenAI-compatible ones:
+
+| Parameter | Type | Description |
+|-----------|------|-------------|
+| `diarize` | `bool` | Enable speaker diarization |
+
+```python
+response = transcription(
+    model="mistral/voxtral-mini-latest",
+    file=audio_file,
+    diarize=True,
+)
+```
+
+### Usage with LiteLLM Proxy
+
+```yaml
+model_list:
+  - model_name: voxtral
+    litellm_params:
+      model: mistral/voxtral-mini-latest
+      api_key: os.environ/MISTRAL_API_KEY
+    model_info:
+      mode: audio_transcription
+```
+
+```bash
+litellm --config /path/to/config.yaml
+```
+
+```bash
+curl --location 'http://0.0.0.0:4000/v1/audio/transcriptions' \
+--header 'Authorization: Bearer sk-1234' \
+--form 'file=@"audio.wav"' \
+--form 'model="voxtral"'
+```
+
 ## Sample Usage - Embedding
 ```python
 from litellm import embedding
@@ -632,7 +632,9 @@ curl -X POST 'http://0.0.0.0:4000/chat/completions' \

 ## OpenAI Chat Completion to Responses API Bridge

-Call any Responses API model from OpenAI's `/chat/completions` endpoint.
+LiteLLM offers a chat completion to Responses API bridge. This lets you use the completion interface while calling the Responses API under the hood.
+
+This is useful when you want to use [Responses API](https://platform.openai.com/docs/api-reference/responses) specific features (like built-in tools, web search preview, or code interpreter).

 :::tip gpt-5.4 + reasoning_effort + function tools

@@ -649,12 +651,54 @@ response = litellm.completion(

 :::

+### When to use the `openai/responses/` prefix
+
+Each model has a `mode` property defined in [`model_prices_and_context_window.json`](https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json) that determines which API endpoint it uses by default:
+
+- **`mode: responses`** - Model automatically uses the Responses API
+- **`mode: chat`** - Model defaults to the Chat Completions API
+
+**Models with `mode: responses`** (automatic Responses API):
+- `o3-deep-research`, `o4-mini-deep-research`
+- `o1-pro`, `o3-pro`
+- `gpt-5.1-codex`, `gpt-5.1-codex-mini`, `gpt-5.1-codex-max`
+- `codex-mini-latest`
+
+**Models with `mode: chat`** (require `openai/responses/` prefix for built-in tools):
+- `gpt-4o`, `gpt-4o-mini`, `gpt-4.1`, `gpt-4.1-mini`
+- `gpt-5`, `gpt-5-mini`
+- `o3`, `o4-mini`
+
+To use built-in tools like `web_search_preview` with `mode: chat` models, add the `openai/responses/` prefix:
+
+```python
+# This will FAIL - gpt-4o has mode: chat, uses Chat Completions API
+response = litellm.completion(
+    model="gpt-4o",
+    messages=[{"role": "user", "content": "What is the weather in Paris today?"}],
+    tools=[{"type": "web_search_preview"}],  # Not supported in Chat Completions
+    # ... other kwargs
+)
+
+# This will WORK - prefix forces Responses API
+response = litellm.completion(
+    model="openai/responses/gpt-4o",
+    messages=[{"role": "user", "content": "What is the weather in Paris today?"}],
+    tools=[{"type": "web_search_preview"}],  # Supported in Responses API
+    # ... other kwargs
+)
+```
+
+### Examples
+
 <Tabs>
 <TabItem value="sdk" label="SDK">

+**Using a model with `mode: responses` (automatic):**
+
 ```python
 import litellm
-import os 
+import os

 os.environ["OPENAI_API_KEY"] = "sk-1234"

@@ -668,6 +712,26 @@ response = litellm.completion(
 )
 print(response)
 ```
+
+**Using a model with `mode: chat` (requires prefix):**
+
+```python
+import litellm
+import os
+
+os.environ["OPENAI_API_KEY"] = "sk-1234"
+
+# Use the openai/responses/ prefix to enable built-in tools
+response = litellm.completion(
+    model="openai/responses/gpt-4o",
+    messages=[{"role": "user", "content": "What is the weather in Paris today?"}],
+    tools=[
+        {"type": "web_search_preview"},
+    ],
+)
+print(response)
+```
+
 </TabItem>
 <TabItem value="proxy" label="PROXY">

@@ -675,10 +739,17 @@ print(response)

 ```yaml
 model_list:
-  - model_name: openai-model
+  # Model with mode: responses (automatic)
+  - model_name: o3-deep-research
    litellm_params:
      model: o3-deep-research-2025-06-26
      api_key: os.environ/OPENAI_API_KEY
+
+  # Model with mode: chat (use prefix for built-in tools)
+  - model_name: gpt-4o-with-tools
+    litellm_params:
+      model: openai/responses/gpt-4o
+      api_key: os.environ/OPENAI_API_KEY
 ```

 2. Start the proxy
@@ -693,15 +764,14 @@ litellm --config config.yaml
 curl -X POST 'http://0.0.0.0:4000/chat/completions' \
 -H 'Content-Type: application/json' \
 -H 'Authorization: Bearer sk-1234' \
-d '{ 
-    "model": "openai-model",
+-d '{
+    "model": "gpt-4o-with-tools",
    "messages": [
-        {"role": "user", "content": "What is the capital of France?"}
+        {"role": "user", "content": "What is the weather in Paris today?"}
    ],
    "tools": [
-        {"type": "web_search_preview"},
-        {"type": "code_interpreter", "container": {"type": "auto"}},
-    ],
+        {"type": "web_search_preview"}
+    ]
 }'
 ```

@@ -79,6 +79,7 @@ All models listed [here](https://github.com/BerriAI/litellm/blob/57f37f743886a02
 | textembedding-gecko@003 | `embedding(model="vertex_ai/textembedding-gecko@003", input)` | 
 | text-embedding-preview-0409 | `embedding(model="vertex_ai/text-embedding-preview-0409", input)` |
 | text-multilingual-embedding-preview-0409 | `embedding(model="vertex_ai/text-multilingual-embedding-preview-0409", input)` | 
+| gemini-embedding-2-preview | `embedding(model="vertex_ai/gemini-embedding-2-preview", input)` | [Multimodal docs](#gemini-embedding-2-preview-multimodal) |
 | Fine-tuned OR Custom Embedding models | `embedding(model="vertex_ai/<your-model-id>", input)` | 

 ### Supported OpenAI (Unified) Params
@@ -257,6 +258,71 @@ model_list:

 ## **Multi-Modal Embeddings**

+### Gemini Embedding 2 Preview (Multimodal)
+
+`gemini-embedding-2-preview` supports **unified multimodal embeddings**—text, images, audio, video, and PDF in a single request. See [blog post](/blog/gemini_embedding_2_multimodal) for details.
+
+**Input formats:**
+- **Data URIs:** `data:image/png;base64,<encoded_data>`
+- **GCS URLs:** `gs://bucket/path/to/file.png` (MIME type inferred from extension)
+
+**Supported MIME types:** `image/png`, `image/jpeg`, `audio/mpeg`, `audio/wav`, `video/mp4`, `video/quicktime`, `application/pdf`
+
+<Tabs>
+<TabItem value="sdk" label="SDK">
+
+```python
+import litellm
+from litellm import embedding
+
+litellm.vertex_project = "your-project-id"
+litellm.vertex_location = "us-central1"
+
+# Text + Image (GCS URL)
+response = embedding(
+    model="vertex_ai/gemini-embedding-2-preview",
+    input=[
+        "Describe this image",
+        "gs://my-bucket/images/photo.png"
+    ],
+)
+
+# Text + Image (base64)
+response = embedding(
+    model="vertex_ai/gemini-embedding-2-preview",
+    input=[
+        "The food was delicious",
+        "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAgAAAAIAQMAAAD+wSzIAAAABlBMVEX///+/v7+jQ3Y5AAAADklEQVQI12P4AIX8EAgALgAD/aNpbtEAAAAASUVORK5CYII"
+    ],
+)
+```
+
+</TabItem>
+<TabItem value="proxy" label="LiteLLM PROXY">
+
+```yaml
+model_list:
+  - model_name: vertex-gemini-embedding-2-preview
+    litellm_params:
+      model: vertex_ai/gemini-embedding-2-preview
+      vertex_project: "your-project-id"
+      vertex_location: "us-central1"
+```
+
+```bash
+curl -X POST http://localhost:4000/embeddings \
+  -H "Authorization: Bearer sk-1234" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "vertex-gemini-embedding-2-preview",
+    "input": ["Describe this", "gs://bucket/image.png"]
+  }'
+```
+
+</TabItem>
+</Tabs>
+
+### multimodalembedding@001 (Legacy)

 Known Limitations:
 - Only supports 1 image / video / image per request
@@ -1,24 +1,15 @@
 import Image from '@theme/IdealImage';
-import Tabs from '@theme/Tabs';
-import TabItem from '@theme/TabItem';

 # PANW Prisma AIRS

-LiteLLM supports PANW Prisma AIRS (AI Runtime Security) guardrails via the [Prisma AIRS Scan API](https://pan.dev/prisma-airs/api/airuntimesecurity/airuntimesecurityapi//). This integration provides **Security-as-Code** for AI applications using Palo Alto Networks' AI security platform.
+LiteLLM supports PANW Prisma AIRS (AI Runtime Security) guardrails via the [Prisma AIRS Scan API](https://pan.dev/prisma-airs/api/airuntimesecurity/airuntimesecurityapi/). This integration provides Security-as-Code for AI applications using Palo Alto Networks' AI security platform.

-## Features
+- **Prompt injection and malicious URL detection** — real-time scanning before or after LLM calls
+- **Data loss prevention (DLP)** — detect and block sensitive data in prompts and responses
+- **Sensitive content masking** — automatically mask PII, credit cards, SSNs instead of blocking
+- **MCP tool call scanning** — scan tool name and arguments on direct MCP tool invocations
+- **Configurable fail-open / fail-closed** — choose between maximum security or high availability

- ✅ **Real-time prompt injection detection**
- ✅ **Malicious URL detection** 
- ✅ **Data loss prevention (DLP)**
- ✅ **Sensitive content masking** - Automatically mask PII, credit cards, SSNs instead of blocking
- ✅ **Comprehensive threat detection** for AI models and datasets
- ✅ **Model-agnostic protection** across public and private models
- ✅ **Synchronous scanning** with immediate response
- ✅ **Configurable security profiles**
- ✅ **Streaming support** - Real-time masking for streaming responses
- ✅ **Multi-turn conversation tracking** - Automatic session grouping in Prisma AIRS SCM logs
- ✅ **Configurable fail-open/fail-closed** - Choose between maximum security (block on API errors) or high availability (allow on transient errors)

 ## Quick Start

@@ -32,7 +23,14 @@ For detailed setup instructions, see the [Prisma AIRS API Overview](https://docs

 ### 2. Define Guardrails on your LiteLLM config.yaml

-Define your guardrails under the `guardrails` section:
+Set `api_base` to the regional endpoint for your Prisma AIRS deployment profile:
+
+| Region | Endpoint |
+|--------|----------|
+| US | `https://service.api.aisecurity.paloaltonetworks.com` |
+| EU (Germany) | `https://service-de.api.aisecurity.paloaltonetworks.com` |
+| India | `https://service-in.api.aisecurity.paloaltonetworks.com` |
+| Singapore | `https://service-sg.api.aisecurity.paloaltonetworks.com` |

 ```yaml
 model_list:
@@ -45,21 +43,15 @@ guardrails:
  - guardrail_name: "panw-prisma-airs-guardrail"
    litellm_params:
      guardrail: panw_prisma_airs
-      mode: "pre_call"                    # Run before LLM call
-      api_key: os.environ/PANW_PRISMA_AIRS_API_KEY    # Your Prisma AIRS API key
-      profile_name: os.environ/PANW_PRISMA_AIRS_PROFILE_NAME  # Security profile from Strata Cloud Manager
-      api_base: "https://service.api.aisecurity.paloaltonetworks.com"  
+      mode: "pre_call"
+      api_key: os.environ/PANW_PRISMA_AIRS_API_KEY
+      profile_name: os.environ/PANW_PRISMA_AIRS_PROFILE_NAME
+      api_base: "https://service.api.aisecurity.paloaltonetworks.com"  # US — change to your region
 ```

-#### Supported values for `mode`
-
- `pre_call` Run **before** LLM call, on **input**
- `post_call` Run **after** LLM call, on **input & output**  
- `during_call` Run **during** LLM call, on **input**. Same as `pre_call` but runs in parallel with LLM call
-
 ### 3. Start LiteLLM Gateway

-```bash title="Set environment variables"
+```bash
 export PANW_PRISMA_AIRS_API_KEY="your-panw-api-key"
 export PANW_PRISMA_AIRS_PROFILE_NAME="your-security-profile"
 export OPENAI_API_KEY="sk-proj-..."
@@ -69,15 +61,8 @@ export OPENAI_API_KEY="sk-proj-..."
 litellm --config config.yaml --detailed_debug
 ```

-
 ### 4. Test Request

-**[Langchain, OpenAI SDK Usage Examples](../proxy/user_keys#request-format)**
-
-<Tabs>
-<TabItem label="Blocked request" value="blocked">
-
-Expect this to fail due to prompt injection attempt:

 ```shell
 curl -i http://localhost:4000/v1/chat/completions \
@@ -92,254 +77,57 @@ curl -i http://localhost:4000/v1/chat/completions \
  }'
 ```

-Expected response on failure:
+Expected response when the guardrail blocks:

 ```json
 {
  "error": {
-    "message": {
-      "error": "Violated PANW Prisma AIRS guardrail policy",
-      "panw_response": {
-        "action": "block",
-        "category": "malicious",
-        "profile_id": "03b32734-d06d-4bb7-a8df-ac5147630ce8",
-        "profile_name": "dev-block-all-profile",
-        "prompt_detected": {
-          "dlp": false,
-          "injection": true,
-          "toxic_content": false,
-          "url_cats": false
-        },
-        "report_id": "Rbd251eac-6e67-433b-b3ef-8eb42d2c7d2c",
-        "response_detected": {
-          "dlp": false,
-          "toxic_content": false,
-          "url_cats": false
-        },
-        "scan_id": "bd251eac-6e67-433b-b3ef-8eb42d2c7d2c",
-        "tr_id": "string"
-      }
-    },
-    "type": "None",
-    "param": "None",
-    "code": "400"
+    "message": "Prompt blocked by PANW Prisma AI Security policy (Category: malicious)",
+    "type": "guardrail_violation",
+    "code": "panw_prisma_airs_blocked",
+    "guardrail": "panw-prisma-airs-guardrail",
+    "category": "malicious"
  }
 }
 ```

-</TabItem>
-<TabItem label="Successful Call" value="allowed">
+LiteLLM wraps this detail in an endpoint-specific HTTP error envelope. Optional fields that may also appear: `scan_id`, `report_id`, `profile_name`, `profile_id`, `tr_id`, `prompt_detected`.

-```shell
-curl -i http://localhost:4000/v1/chat/completions \
-  -H "Content-Type: application/json" \
-  -H "Authorization: Bearer sk-your-api-key" \
-  -d '{
-    "model": "gpt-4o",
-    "messages": [
-      {"role": "user", "content": "What is the weather like today?"}
-    ],
-    "guardrails": ["panw-prisma-airs-guardrail"]
-  }'
-```
+On success, the guardrail name appears in the `x-litellm-applied-guardrails` response header.

-Expected successful response:
+## Configuration

-```json
-{
-  "choices": [
-    {
-      "finish_reason": "stop",
-      "index": 0,
-      "message": {
-        "content": "I don't have access to real-time weather data, but I can help you find weather information through various weather services or apps...",
-        "role": "assistant",
-        "tool_calls": null,
-        "function_call": null,
-        "annotations": []
-      }
-    }
-  ],
-  "created": 1736028456,
-  "id": "chatcmpl-AqQj8example",
-  "model": "gpt-4o",
-  "object": "chat.completion",
-  "usage": {
-    "completion_tokens": 25,
-    "prompt_tokens": 12,
-    "total_tokens": 37
-  },
-  "x-litellm-panw-scan": {
-    "action": "allow",
-    "category": "benign",
-    "profile_id": "03b32734-d06d-4bb7-a8df-ac5147630ce8",
-    "profile_name": "dev-block-all-profile",
-    "prompt_detected": {
-      "dlp": false,
-      "injection": false,
-      "toxic_content": false,
-      "url_cats": false
-    },
-    "report_id": "Rbd251eac-6e67-433b-b3ef-8eb42d2c7d2c",
-    "response_detected": {
-      "dlp": false,
-      "toxic_content": false,
-      "url_cats": false
-    },
-    "scan_id": "bd251eac-6e67-433b-b3ef-8eb42d2c7d2c",
-    "tr_id": "string"
-  }
-}
-```
+### Supported Modes

-</TabItem>
-</Tabs>
+| Mode | Timing | What is scanned |
+|------|--------|-----------------|
+| `pre_call` | Before LLM call | Request input |
+| `during_call` | Parallel with LLM call | Request input |
+| `post_call` | After LLM call | Response output |
+| `pre_mcp_call` | Before MCP tool execution | MCP tool input |
+| `during_mcp_call` | Parallel with MCP tool execution | MCP tool input |

-## Configuration Parameters
+
+### Configuration Parameters

 | Parameter | Required | Description | Default |
 |-----------|----------|-------------|---------|
 | `api_key` | Yes | Your PANW Prisma AIRS API key from Strata Cloud Manager | - |
 | `profile_name` | No | Security profile name configured in Strata Cloud Manager. Optional if API key has linked profile | - |
-| `app_name` | No | Application identifier for tracking in Prisma AIRS analytics (will be prefixed with "LiteLLM-") | `LiteLLM` |
-| `api_base` | No | Regional API endpoint (see [Regional Endpoints](#regional-endpoints) below) | `https://service.api.aisecurity.paloaltonetworks.com` (US) |
-| `mode` | No | When to run the guardrail | `pre_call` |
-| `fallback_on_error` | No | Action when PANW API is unavailable: `"block"` (fail-closed, default) or `"allow"` (fail-open). Config errors always block. | `block` |
-| `timeout` | No | PANW API call timeout in seconds (1-60) | `10.0` |
-| `violation_message_template` | No | Custom template for error message when request is blocked. Supports `{guardrail_name}`, `{category}`, `{action_type}`, `{default_message}` placeholders. | - |
+| `app_name` | No | Application identifier for tracking in Prisma AIRS analytics (prefixed with "LiteLLM-") | `LiteLLM` |
+| `api_base` | No | Regional API endpoint. US: `https://service.api.aisecurity.paloaltonetworks.com`, EU: `https://service-de.api.aisecurity.paloaltonetworks.com`, India: `https://service-in.api.aisecurity.paloaltonetworks.com`, Singapore: `https://service-sg.api.aisecurity.paloaltonetworks.com` | US |
+| `mode` | No | When to run the guardrail (see mode table above) | `pre_call` |
+| `fallback_on_error` | No | Action when PANW API is unavailable: `"block"` (fail-closed) or `"allow"` (fail-open). Config errors always block. | `block` |
+| `timeout` | No | PANW API call timeout in seconds (recommended: 1-60) | `10.0` |
+| `violation_message_template` | No | Custom template for blocked requests. Supports `{guardrail_name}`, `{category}`, `{action_type}`, `{default_message}` placeholders. | - |
+| `mask_request_content` | No | Mask sensitive data in prompts instead of blocking | `false` |
+| `mask_response_content` | No | Mask sensitive data in responses instead of blocking | `false` |
+| `mask_on_block` | No | Backwards-compatible flag that enables both request and response masking | `false` |
+| `experimental_use_latest_role_message_only` | No | Anthropic `/v1/messages` only. When unset: scans only latest user message on request side. Set `false` to scan all user/system/developer messages. Non-Anthropic unaffected. | Unset (true for Anthropic) |

-### Regional Endpoints
+Use the regional `api_base` that matches your Prisma AIRS deployment profile region for lower latency and data residency compliance.

-PANW Prisma AIRS supports multiple regional endpoints based on your deployment profile region:
-
-| Region | API Base URL |
-|--------|--------------|
-| **US** (default) | `https://service.api.aisecurity.paloaltonetworks.com` |
-| **EU (Germany)** | `https://service-de.api.aisecurity.paloaltonetworks.com` |
-| **India** | `https://service-in.api.aisecurity.paloaltonetworks.com` |
-
-**Example configuration for EU region:**
-
-```yaml
-guardrails:
-  - guardrail_name: "panw-eu"
-    litellm_params:
-      guardrail: panw_prisma_airs
-      api_key: os.environ/PANW_PRISMA_AIRS_API_KEY
-      api_base: "https://service-de.api.aisecurity.paloaltonetworks.com"
-      profile_name: "production"
-```
-
-:::tip Region Selection
-Use the regional endpoint that matches your Prisma AIRS deployment profile region configured in Strata Cloud Manager. Using the correct region ensures:
- Lower latency (requests stay in-region)
- Compliance with data residency requirements
- Optimal performance
-:::
-
-## Per-Request Metadata Overrides
-
-You can override guardrail settings on a per-request basis using the `metadata` field:
-
-```json
-{
-  "model": "gpt-4",
-  "messages": [...],
-  "metadata": {
-    "profile_name": "dev-allow-all",            // Override profile name
-    "profile_id": "uuid-here",                  // Override profile ID (takes precedence)
-    "user_ip": "192.168.1.100",                 // Track user IP
-    "app_name": "MyApp"                         // Custom app name (becomes "LiteLLM-MyApp")
-  }
-}
-```
-
-**Supported Metadata Fields:**
-
-| Field | Description | Priority |
-|-------|-------------|----------|
-| `profile_name` | PANW AI security profile name | Per-request > config |
-| `profile_id` | PANW AI security profile ID (takes precedence over profile_name) | Per-request only |
-| `user_ip` | User IP address for tracking in Prisma AIRS | Per-request only |
-| `app_name` | Application identifier (prefixed with "LiteLLM-") | Per-request > config > "LiteLLM" |
-| `app_user` | Custom user identifier for tracking in Prisma AIRS | `app_user` > `user` > "litellm_user" |
-
-:::info Profile Resolution
- If both `profile_id` and `profile_name` are provided, PANW API uses `profile_id` (it takes precedence)
- If no profile is specified in metadata, uses the config `profile_name`
- If no profile is specified at all, PANW API will use the profile linked to your API key in Strata Cloud Manager
- **Note:** If your API key is not linked to a profile, you must provide `profile_name` or `profile_id`
-:::
-
-## Multi-Turn Conversation Tracking
-
-PANW Prisma AIRS automatically tracks multi-turn conversations using LiteLLM's `litellm_trace_id`. This enables you to:
-
- **Group related requests** - All requests in a conversation share the same AI Session ID in Prisma AIRS SCM logs
- **Track conversation context** - See the full history of prompts and responses for a user session
- **Analyze attack patterns** - Identify sophisticated multi-turn attacks across conversation history
-
-### How It Works
-
-LiteLLM automatically generates a unique `litellm_trace_id` for each conversation session. The PANW guardrail uses this as the PANW transaction ID (which maps to "AI Session ID" in Strata Cloud Manager):
-
-```
-Conversation Session: litellm_trace_id = "abc-123-def-456"
-
-Turn 1 (User):    "What's the capital of France?"
-  → Scan ID: scan_001 | Prisma AIRS AI Session ID: abc-123-def-456
-
-Turn 2 (Assistant): "Paris is the capital of France."
-  → Scan ID: scan_002 | Prisma AIRS AI Session ID: abc-123-def-456
-
-Turn 3 (User):    "What's the population?"
-  → Scan ID: scan_003 | Prisma AIRS AI Session ID: abc-123-def-456
-
-Turn 4 (Assistant): "Paris has approximately 2.1 million residents."
-  → Scan ID: scan_004 | Prisma AIRS AI Session ID: abc-123-def-456
-```
-
-All scans appear under the same AI Session ID in Prisma AIRS logs, making it easy to:
- Review complete conversation history (all 4 turns grouped together)
- Identify patterns across multiple turns
- Correlate security events within a session
- Track the flow of user prompts and AI responses
-
-### Session Tracking
-
-LiteLLM automatically generates a unique `litellm_trace_id` for each request, which the PANW guardrail uses as the AI Session ID in Strata Cloud Manager. All prompt and response scans for a request are automatically grouped under the same session.
-
-#### Custom Session IDs (Per-App Tracking)
-
-You can provide your own `litellm_trace_id` to track sessions on a per-app or per-conversation basis:
-
-```bash
-curl -X POST http://localhost:4000/v1/chat/completions \
-  -H "Content-Type: application/json" \
-  -H "Authorization: Bearer sk-1234" \
-  -d '{
-    "model": "gpt-3.5-turbo",
-    "messages": [{"role": "user", "content": "capital of France"}],
-    "litellm_trace_id": "my-app-session-123",             # Custom AI Session ID
-    "metadata": {
-      "profile_name": "dev-allow-all-profile",            # Override security profile
-      "user_ip": "192.168.1.1",                           # Track user IP
-      "app_name": "eng"                                   # Custom app identifier
-    },
-    "guardrails": ["panw-prisma-airs-pre-guard", "panw-prisma-airs-post-guard"]
-  }'
-```
-
-**Result in PANW SCM:**
- AI Session ID: `my-app-session-123`
- All prompt and response scans will be grouped under this custom session ID
- Perfect for tracking multi-turn conversations or per-application sessions
-
-:::tip Viewing Sessions in Prisma AIRS SCM Logs
-In Strata Cloud Manager, navigate to **AI Runtime > Sessions** to view all AI Session IDs and their associated scans. Click on a session to see the complete conversation history with security analysis.
-:::
-
-## Environment Variables
+### Environment Variables

 ```bash
 export PANW_PRISMA_AIRS_API_KEY="your-panw-api-key"
@@ -348,12 +136,31 @@ export PANW_PRISMA_AIRS_PROFILE_NAME="your-security-profile"
 export PANW_PRISMA_AIRS_API_BASE="https://custom-endpoint.com"
 ```

-## Advanced Configuration
+### Per-Request Metadata Overrides
+
+| Field | Description | Priority |
+|-------|-------------|----------|
+| `profile_name` | PANW AI security profile name | Per-request > config |
+| `profile_id` | PANW AI security profile ID (takes precedence over `profile_name`) | Per-request only |
+| `user_ip` | User IP address for tracking in Prisma AIRS | Per-request only |
+| `app_name` | Application identifier (prefixed with "LiteLLM-") | Per-request > config > "LiteLLM" |
+| `app_user` | Custom user identifier for tracking in Prisma AIRS | `app_user` > `user` > "litellm_user" |
+
+```json
+{
+  "model": "gpt-4",
+  "messages": [...],
+  "metadata": {
+    "profile_name": "dev-allow-all",
+    "profile_id": "uuid-here",
+    "user_ip": "192.168.1.100",
+    "app_name": "MyApp"
+  }
+}
+```

 ### Multiple Security Profiles

-You can configure different security profiles for different use cases:
-
 ```yaml
 guardrails:
  - guardrail_name: "panw-strict-security"
@@ -361,126 +168,40 @@ guardrails:
      guardrail: panw_prisma_airs
      mode: "pre_call"
      api_key: os.environ/PANW_PRISMA_AIRS_API_KEY
-      profile_name: "strict-policy"       # High security profile
-      
-  - guardrail_name: "panw-permissive-security"  
+      profile_name: "strict-policy"
+
+  - guardrail_name: "panw-permissive-security"
    litellm_params:
      guardrail: panw_prisma_airs
      mode: "post_call"
      api_key: os.environ/PANW_PRISMA_AIRS_API_KEY
-      profile_name: "permissive-policy"   # Lower security profile
+      profile_name: "permissive-policy"
 ```

-### Multiple API Keys (Multi-Tenant)
-
-For multi-tenant deployments where different customers need different PANW API keys, create separate guardrail instances:
-
-```yaml
-guardrails:
-  - guardrail_name: "panw-customer-a"
-    litellm_params:
-      guardrail: panw_prisma_airs
-      mode: "pre_call"
-      api_key: os.environ/PANW_CUSTOMER_A_KEY  # Linked to Customer A profile in SCM
-      
-  - guardrail_name: "panw-customer-b"
-    litellm_params:
-      guardrail: panw_prisma_airs
-      mode: "pre_call"
-      api_key: os.environ/PANW_CUSTOMER_B_KEY  # Linked to Customer B profile in SCM
-```
-
-Then route requests to the appropriate guardrail:
-
-```bash
-curl -X POST http://localhost:4000/v1/chat/completions \
-  -H "Content-Type: application/json" \
-  -H "Authorization: Bearer sk-1234" \
-  -d '{
-    "model": "gpt-4",
-    "messages": [{"role": "user", "content": "Hello"}],
-    "guardrails": ["panw-customer-a"]
-  }'
-```
-
-**Use Cases:**
- **Multi-tenant deployments**: Different customers with different security policies
- **Environment-specific policies**: Dev/staging/prod with different API keys and profiles
- **A/B testing**: Compare different security profiles side-by-side
-
 ### Content Masking

-PANW Prisma AIRS can automatically mask sensitive content (PII, credit cards, SSNs, etc.) instead of blocking requests. This allows your application to continue functioning while protecting sensitive data.
-
-#### How It Works
-
-1. **Detection**: PANW scans content and identifies sensitive data
-2. **Masking**: Sensitive data is replaced with placeholders (e.g., `XXXXXXXXXX` or `{PHONE}`)
-3. **Pass-through**: Masked content is sent to the LLM or returned to the user
-
-#### Configuration Options
+:::warning Important: Masking is Controlled by PANW Security Profile
+The actual masking behavior (what content gets masked and how) is controlled by your PANW Prisma AIRS security profile in Strata Cloud Manager. The LiteLLM flags (`mask_request_content`, `mask_response_content`) only control whether to apply the masked content and allow the request to continue, or block entirely.
+:::

 ```yaml
 guardrails:
  - guardrail_name: "panw-with-masking"
    litellm_params:
      guardrail: panw_prisma_airs
-      mode: "post_call"                      # Scan response output
+      mode: "post_call"
      api_key: os.environ/PANW_PRISMA_AIRS_API_KEY
      profile_name: "default"
-      mask_request_content: true             # Mask sensitive data in prompts
-      mask_response_content: true            # Mask sensitive data in responses
+      mask_request_content: true
+      mask_response_content: true
 ```

-**Masking Parameters:**
-
- `mask_request_content: true` - When PANW detects sensitive data in prompts, mask it instead of blocking
- `mask_response_content: true` - When PANW detects sensitive data in responses, mask it instead of blocking  
- `mask_on_block: true` - Backwards compatible flag that enables both request and response masking
-
-:::warning Important: Masking is Controlled by PANW Security Profile
-The **actual masking behavior** (what content gets masked and how) is controlled by your **PANW Prisma AIRS security profile** configured in Strata Cloud Manager. The LiteLLM config settings (`mask_request_content`, `mask_response_content`) only control whether to:
- **Apply the masked content** returned by PANW and allow the request to continue, OR
- **Block the request** entirely when sensitive data is detected
-
-LiteLLM does not alter or configure your PANW security profile. To change what content gets masked, update your profile settings in Strata Cloud Manager.
-:::
-
-:::info Security Posture
-The guardrail is **fail-closed** by default - if the PANW API is unavailable, requests are blocked to ensure no unscanned content reaches your LLM. This provides maximum security.
-:::
-
-### Custom Violation Messages
-
-You can customize the error message returned to the user when a request is blocked by configuring the `violation_message_template` parameter. This is useful for providing user-friendly feedback instead of technical details.
-
-```yaml
-guardrails:
-  - guardrail_name: "panw-custom-message"
-    litellm_params:
-      guardrail: panw_prisma_airs
-      api_key: os.environ/PANW_PRISMA_AIRS_API_KEY
-      # Simple message
-      violation_message_template: "Your request was blocked by our AI Security Policy."
-
-  - guardrail_name: "panw-detailed-message"
-    litellm_params:
-      guardrail: panw_prisma_airs
-      api_key: os.environ/PANW_PRISMA_AIRS_API_KEY
-      # Message with placeholders
-      violation_message_template: "{action_type} blocked due to {category} violation. Please contact support."
-```
-
-**Supported Placeholders:**
- `{guardrail_name}`: Name of the guardrail (e.g. "panw-custom-message")
- `{category}`: Violation category (e.g. "malicious", "injection", "dlp")
- `{action_type}`: "Prompt" or "Response"
- `{default_message}`: The original technical error message
+- `mask_request_content: true` — mask sensitive data in prompts instead of blocking
+- `mask_response_content: true` — mask sensitive data in responses instead of blocking
+- `mask_on_block: true` — backwards-compatible flag that enables both request and response masking

 ### Fail-Open Configuration

-By default, the PANW guardrail operates in **fail-closed** mode for maximum security. If the PANW API is unavailable (timeout, rate limit, network error), requests are blocked. You can configure **fail-open** mode for high-availability scenarios where service continuity is critical.
-
 ```yaml
 guardrails:
  - guardrail_name: "panw-high-availability"
@@ -488,135 +209,86 @@ guardrails:
      guardrail: panw_prisma_airs
      api_key: os.environ/PANW_PRISMA_AIRS_API_KEY
      profile_name: "production"
-      fallback_on_error: "allow"  # Enable fail-open mode
-      timeout: 5.0                 # Shorter timeout for fail-open
+      fallback_on_error: "allow"
+      timeout: 5.0
 ```

-**Configuration Options:**
-
-| Parameter | Value | Behavior |
-|-----------|-------|----------|
-| `fallback_on_error` | `"block"` (default) | **Fail-closed**: Block requests when API unavailable (maximum security) |
-| `fallback_on_error` | `"allow"` | **Fail-open**: Allow requests when API unavailable (high availability) |
-| `timeout` | `1.0` - `60.0` | API call timeout in seconds (default: `10.0`) |
-
 **Error Handling Matrix:**

 | Error Type | `fallback_on_error="block"` | `fallback_on_error="allow"` |
 |------------|----------------------------|----------------------------|
-| 401 Unauthorized | Block (500) | Block (500) ⚠️ |
-| 403 Forbidden | Block (500) | Block (500) ⚠️ |
-| Profile Error | Block (500) | Block (500) ⚠️ |
+| 401 Unauthorized | Block (500) | Block (500) |
+| 403 Forbidden | Block (500) | Block (500) |
+| Profile Error | Block (500) | Block (500) |
 | 429 Rate Limit | Block (500) | Allow (`:unscanned`) |
 | Timeout | Block (500) | Allow (`:unscanned`) |
 | Network Error | Block (500) | Allow (`:unscanned`) |
 | 5xx Server Error | Block (500) | Allow (`:unscanned`) |
 | Content Blocked | Block (400) | Block (400) |

-⚠️ = Always blocks regardless of fail-open setting 
+Authentication and configuration errors (401, 403, invalid profile) always block. Only transient errors (429, timeout, network) trigger fail-open.

-:::warning Security Trade-Off
-Enabling `fallback_on_error="allow"` reduces security in exchange for availability. Requests may proceed **without scanning** when the PANW API is unavailable. Use only when:
- Service availability is more critical than security scanning
- You have other security controls in place
- You monitor the `:unscanned` header for audit trails
+When fail-open is triggered, the response includes a tracking header: `X-LiteLLM-Applied-Guardrails: panw-airs:unscanned`

-**Authentication and configuration errors (401, 403, invalid profile) always block** - only transient errors (429, timeout, network) trigger fail-open behavior.
-:::
-
-**Observability:**
-
-When fail-open is triggered, the response includes a special header for tracking:
-
-```
-X-LiteLLM-Applied-Guardrails: panw-airs:unscanned
-```
-
-This allows you to:
- Track which requests bypassed scanning
- Alert on unscanned request volumes
- Audit compliance requirements
-
-#### Example: Masking Credit Card Numbers
-
-<Tabs>
-<TabItem label="Without Masking" value="no-mask">
-
-**Request:**
-```json
-{
-  "messages": [
-    {"role": "user", "content": "My credit card is 4929-3813-3266-4295"}
-  ]
-}
-```
-
-**Response:** ❌ **Blocked with 400 error**
-
-</TabItem>
-<TabItem label="With Masking" value="with-mask">
-
-**Request:**
-```json
-{
-  "messages": [
-    {"role": "user", "content": "My credit card is 4929-3813-3266-4295"}
-  ]
-}
-```
-
-**Masked prompt sent to LLM:**
-```json
-{
-  "messages": [
-    {"role": "user", "content": "My credit card is XXXXXXXXXXXXXXXXXX"}
-  ]
-}
-```
-
-**Response:** ✅ **Allowed with masked content**
-
-</TabItem>
-</Tabs>
-
-#### Masking Capabilities
-
-The guardrail masks sensitive content in:
-
- ✅ **Chat messages** - User prompts and assistant responses
- ✅ **Streaming responses** - Real-time masking of streamed content
- ✅ **Multi-choice responses** - All choices in the response
- ✅ **Tool/function calls** - Arguments passed to tools and functions
- ✅ **Content lists** - Mixed content types (text, images, etc.)
-
-#### Complete Example
+### Custom Violation Messages

 ```yaml
 guardrails:
-  - guardrail_name: "panw-production-security"
+  - guardrail_name: "panw-custom-message"
    litellm_params:
      guardrail: panw_prisma_airs
-      mode: "post_call"                      # Scan input and output
      api_key: os.environ/PANW_PRISMA_AIRS_API_KEY
-      profile_name: "production-profile"
-      mask_request_content: true             # Mask sensitive prompts
-      mask_response_content: true            # Mask sensitive responses
+      violation_message_template: "Your request was blocked by our AI Security Policy."
+
+  - guardrail_name: "panw-detailed-message"
+    litellm_params:
+      guardrail: panw_prisma_airs
+      api_key: os.environ/PANW_PRISMA_AIRS_API_KEY
+      violation_message_template: "{action_type} blocked due to {category} violation. Please contact support."
 ```

-## Use Cases
+**Supported Placeholders:** `{guardrail_name}`, `{category}`, `{action_type}`, `{default_message}`

-From [official Prisma AIRS documentation](https://docs.paloaltonetworks.com/ai-runtime-security/activation-and-onboarding/ai-runtime-security-api-intercept-overview):
+## Behavior and Limitations

- **Secure AI models in production**: Validate prompt requests and responses to protect deployed AI models
- **Detect data poisoning**: Identify contaminated training data before fine-tuning
- **Protect against adversarial input**: Safeguard AI agents from malicious inputs and outputs
- **Prevent sensitive data leakage**: Use API-based threat detection to block sensitive data leaks
+### Transaction Tracking
+
+For standard request/response scans, `tr_id` maps to `litellm_call_id`. MCP tool scans use the parent `litellm_call_id` when available; if missing, PANW synthesizes a fallback MCP transaction ID. The real limitation is correlation loss — synthesized MCP `tr_id` values are not grouped with the parent request's prompt/response scans in AIRS dashboards.
+
+By default, LiteLLM generates a UUID for `litellm_call_id`. To provide your own:
+
+```bash
+curl -X POST http://localhost:4000/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -H "Authorization: Bearer sk-1234" \
+  -H "x-litellm-call-id: my-custom-call-id-789" \
+  -d '{
+    "model": "gpt-3.5-turbo",
+    "messages": [{"role": "user", "content": "capital of France"}],
+    "guardrails": ["panw-prisma-airs-guardrail"]
+  }'
+```
+
+The `x-litellm-call-id` is also returned in response headers. If you pass `litellm_trace_id` in request metadata (or via the `x-litellm-trace-id` header), it is included in the PANW API payload metadata but does not affect `tr_id` or appear in Prisma AIRS.
+
+### Streaming
+
+- Response masking works on OpenAI chat streaming (`mask_response_content: true`)
+- `/v1/messages` and `/v1/responses` raw streaming blocks instead of masking when violations are detected
+- Request-side masking (`mask_request_content`) is unaffected by endpoint type
+- When `fallback_on_error: "allow"` is set, streaming responses fail open on transient PANW API errors (timeout, 5xx, network) — original chunks are yielded unchanged
+
+## MCP Tool Security
+
+Tool invocations are sent to AIRS as structured `tool_event` payloads containing tool name, ecosystem, and serialized arguments. Tool-event scans always use request mode.
+
+**What is scanned:** LLM-driven `tool_calls` (name + arguments) and MCP request-side invocations when `mcp_tool_name` (or fallback `name`) is present. Response-side OpenAI-compatible `tool_calls` are also scanned when surfaced into `apply_guardrail()`.
+
+**What is not scanned:** Tool definitions in `inputs["tools"]` and post-MCP tool results (no `post_mcp_call` hook exists yet).


-## Next Steps
+### Current Limitations

- Configure your security policies in [Strata Cloud Manager](https://apps.paloaltonetworks.com/)
- Review the [Prisma AIRS API documentation](https://pan.dev/airs/) for advanced features
- Set up monitoring and alerting for threat detections in your PANW dashboard
- Consider implementing both pre_call and post_call guardrails for comprehensive protection
- Monitor detection events and tune your security profiles based on your application needs
+- **No post-MCP response scanning.** Actual post-MCP tool-result scanning is not supported because there is no `post_mcp_call` hook in the framework. Response-side MCP events are only scanned when they appear as regular `tool_calls` in the LLM response.
+- **Guardrail selection not inherited by MCP sub-calls.** With `default_on: false`, MCP request-side child-call scans can be skipped because the parent request's guardrail selection is not propagated to the synthetic MCP payload. Workaround: use a dedicated guardrail with `mode: pre_mcp_call` and `default_on: true`.
+- **MCP transaction correlation.** MCP tool scans use the parent `litellm_call_id` when available; otherwise a fallback ID is synthesized and will not be grouped with the parent request in AIRS dashboards.
@@ -814,6 +814,8 @@ const sidebars = {
        "providers/anyscale",
        "providers/apertis",
        "providers/baseten",
+        "providers/black_forest_labs",
+        "providers/black_forest_labs_img_edit",
        "providers/bytez",
        "providers/cerebras",
        "providers/chutes",