Merge pull request #21110 from BerriAI/litellm_litellm_anthropic_remote_url3

Add support for remote URL fetching for anthropic beta header mapping
2026-08-14 10:25:37 +00:00 · 2026-02-14 00:30:51 +05:30
parent a1b7261502 d4dbb460cc
commit 066e694f5e
11 changed files with 947 additions and 40 deletions
@@ -3600,6 +3600,7 @@ jobs:
              -e AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID \
              -e AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY \
              -e AWS_REGION_NAME="us-east-1" \
+              -e LITELLM_LOCAL_ANTHROPIC_BETA_HEADERS="True" \
              --add-host host.docker.internal:host-gateway \
              --name my-app \
              -v $(pwd)/tests/proxy_e2e_anthropic_messages_tests/test_config.yaml:/app/config.yaml \
@@ -0,0 +1,128 @@
+# Auto Sync Anthropic Beta Headers
+
+Automatically keep your Anthropic beta headers configuration up to date without restarting your service. **This allows you to support new Anthropic beta features across all providers without restarting your service.**
+
+## Overview
+
+When Anthropic releases new beta features (e.g., new tool capabilities, extended context windows), you typically need to restart your LiteLLM service to get the latest beta header mappings for different providers (Anthropic, Bedrock, Vertex AI, Azure AI).
+
+With auto-sync, LiteLLM automatically pulls the latest configuration from GitHub's [`anthropic_beta_headers_config.json`](https://github.com/BerriAI/litellm/blob/main/litellm/anthropic_beta_headers_config.json) without requiring a restart. This means:
+
+- **Zero downtime** when new beta features are released
+- **Always up-to-date** provider support mappings
+- **Automatic updates** - set it once and forget it
+
+## Quick Start
+
+**Manual sync:**
+```bash
+curl -X POST "https://your-proxy-url/reload/anthropic_beta_headers" \
+  -H "Authorization: Bearer YOUR_ADMIN_TOKEN" \
+  -H "Content-Type: application/json"
+```
+
+**Automatic sync every 24 hours:**
+```bash
+curl -X POST "https://your-proxy-url/schedule/anthropic_beta_headers_reload?hours=24" \
+  -H "Authorization: Bearer YOUR_ADMIN_TOKEN" \
+  -H "Content-Type: application/json"
+```
+
+## API Endpoints
+
+| Endpoint | Method | Description |
+|----------|--------|-------------|
+| `/reload/anthropic_beta_headers` | POST | Manual sync |
+| `/schedule/anthropic_beta_headers_reload?hours={hours}` | POST | Schedule periodic sync |
+| `/schedule/anthropic_beta_headers_reload` | DELETE | Cancel scheduled sync |
+| `/schedule/anthropic_beta_headers_reload/status` | GET | Check sync status |
+
+**Authentication:** Requires admin role or master key
+
+## Python Example
+
+```python
+import requests
+
+def sync_anthropic_beta_headers(proxy_url, admin_token):
+    response = requests.post(
+        f"{proxy_url}/reload/anthropic_beta_headers",
+        headers={"Authorization": f"Bearer {admin_token}"}
+    )
+    return response.json()
+
+# Usage
+result = sync_anthropic_beta_headers("https://your-proxy-url", "your-admin-token")
+print(result['message'])
+```
+
+## Configuration
+
+**Custom beta headers config URL:**
+```bash
+export LITELLM_ANTHROPIC_BETA_HEADERS_URL="https://raw.githubusercontent.com/BerriAI/litellm/main/litellm/anthropic_beta_headers_config.json"
+```
+
+**Use local beta headers config:**
+```bash
+export LITELLM_LOCAL_ANTHROPIC_BETA_HEADERS=True
+```
+
+## Scheduling Automatic Reloads
+
+Schedule automatic reloads to ensure your proxy always has the latest beta header mappings:
+
+```bash
+# Reload every 24 hours
+curl -X POST "https://your-proxy-url/schedule/anthropic_beta_headers_reload?hours=24" \
+  -H "Authorization: Bearer YOUR_ADMIN_TOKEN"
+```
+
+**Check reload status:**
+```bash
+curl -X GET "https://your-proxy-url/schedule/anthropic_beta_headers_reload/status" \
+  -H "Authorization: Bearer YOUR_ADMIN_TOKEN"
+```
+
+**Response:**
+```json
+{
+  "scheduled": true,
+  "interval_hours": 24,
+  "last_run": "2026-02-13T10:00:00",
+  "next_run": "2026-02-14T10:00:00"
+}
+```
+
+**Cancel scheduled reload:**
+```bash
+curl -X DELETE "https://your-proxy-url/schedule/anthropic_beta_headers_reload" \
+  -H "Authorization: Bearer YOUR_ADMIN_TOKEN"
+```
+
+## Environment Variables
+
+| Variable | Description | Default |
+|----------|-------------|---------|
+| `LITELLM_ANTHROPIC_BETA_HEADERS_URL` | URL to fetch beta headers config from | GitHub main branch |
+| `LITELLM_LOCAL_ANTHROPIC_BETA_HEADERS` | Set to `True` to use local config only | `False` |
+
+## How It Works
+
+1. **Initial Load:** On startup, LiteLLM loads the beta headers configuration from the remote URL (or local file if configured)
+2. **Caching:** The configuration is cached in memory to avoid repeated fetches on every request
+3. **Scheduled Reload:** If configured, the proxy checks every 10 seconds whether it's time to reload based on your schedule
+4. **Manual Reload:** You can trigger an immediate reload via the API endpoint
+5. **Multi-Pod Support:** In multi-pod deployments, the reload configuration is stored in the database so all pods stay in sync
+
+## Benefits
+
+- **No Restarts Required:** Add support for new Anthropic beta features without downtime
+- **Provider Compatibility:** Automatically get updated mappings for Bedrock, Vertex AI, Azure AI, etc.
+- **Performance:** Configuration is cached and only reloaded when needed
+- **Reliability:** Falls back to local configuration if remote fetch fails
+
+## Related
+
+- [Model Cost Map Sync](./sync_models_github.md) - Auto-sync model pricing data
+- [Anthropic Beta Headers](../completion/anthropic.md#beta-features) - Using Anthropic beta features
@@ -92,9 +92,34 @@ Open `anthropic_beta_headers_config.json` and add the new header to each provide
 - **Header transformations**: Some providers use different header names (e.g., Bedrock maps `advanced-tool-use-2025-11-20` to `tool-search-tool-2025-10-19`)
 - **Alphabetical order**: Keep headers sorted alphabetically for maintainability

-### Step 3: Restart Your Application
+### Step 3: Reload Configuration (No Restart Required!)

-After updating the config file, restart your LiteLLM proxy or application:
+**Option 1: Dynamic Reload Without Restart** 
+
+Instead of restarting your application, you can dynamically reload the beta headers configuration using environment variables and API endpoints:
+
+```bash
+# Set environment variable to fetch from remote URL (Do this if you want to point it to some other URL)
+export LITELLM_ANTHROPIC_BETA_HEADERS_URL="https://raw.githubusercontent.com/BerriAI/litellm/main/litellm/anthropic_beta_headers_config.json"
+
+# Manually trigger reload via API (no restart needed!)
+curl -X POST "https://your-proxy-url/reload/anthropic_beta_headers" \
+  -H "Authorization: Bearer YOUR_ADMIN_TOKEN"
+```
+
+**Option 2: Schedule Automatic Reloads** 
+
+Set up automatic reloading to always stay up-to-date with the latest beta headers:
+
+```bash
+# Reload configuration every 24 hours
+curl -X POST "https://your-proxy-url/schedule/anthropic_beta_headers_reload?hours=24" \
+  -H "Authorization: Bearer YOUR_ADMIN_TOKEN"
+```
+
+**Option 3: Traditional Restart**
+
+If you prefer the traditional approach, restart your LiteLLM proxy or application:

 ```bash
 # If using LiteLLM proxy
@@ -104,7 +129,11 @@ litellm --config config.yaml
 # Just restart your Python application
 ```

-The updated configuration will be loaded automatically.
+:::tip Zero-Downtime Updates
+With dynamic reloading, you can fix invalid beta header errors **without restarting your service**! This is especially useful in production environments where downtime is costly.
+
+See [Auto Sync Anthropic Beta Headers](../proxy/sync_anthropic_beta_headers.md) for complete documentation.
+:::

 ## Fixing Invalid Beta Header Errors

@@ -215,6 +244,26 @@ Result sent to Bedrock:
 anthropic-beta: computer-use-2025-01-24
 ```

+## Dynamic Configuration Management (No Restart Required!)
+
+### Environment Variables
+
+Control how LiteLLM loads the beta headers configuration:
+
+| Variable | Description | Default |
+|----------|-------------|---------|
+| `LITELLM_ANTHROPIC_BETA_HEADERS_URL` | URL to fetch config from | GitHub main branch |
+| `LITELLM_LOCAL_ANTHROPIC_BETA_HEADERS` | Set to `True` to use local config only | `False` |
+
+**Example: Use Custom Config URL**
+```bash
+export LITELLM_ANTHROPIC_BETA_HEADERS_URL="https://your-company.com/custom-beta-headers.json"
+```
+
+**Example: Use Local Config Only (No Remote Fetching)**
+```bash
+export LITELLM_LOCAL_ANTHROPIC_BETA_HEADERS=True
+```
 ## Provider-Specific Notes

 ### Bedrock
@@ -1003,6 +1003,7 @@ const sidebars = {
        "tutorials/presidio_pii_masking",
        "tutorials/elasticsearch_logging",
        "tutorials/gemini_realtime_with_audio",
+        "tutorials/claude_code_beta_headers",
        {
          type: "category",
          label: "LiteLLM Python SDK Tutorials",
@@ -338,6 +338,10 @@ model_cost_map_url: str = os.getenv(
    "LITELLM_MODEL_COST_MAP_URL",
    "https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json",
 )
+anthropic_beta_headers_url: str = os.getenv(
+    "LITELLM_ANTHROPIC_BETA_HEADERS_URL",
+    "https://raw.githubusercontent.com/BerriAI/litellm/main/litellm/anthropic_beta_headers_config.json",
+)
 suppress_debug_info = False
 dynamodb_table_name: Optional[str] = None
 s3_callback_params: Optional[Dict] = None
@@ -2,8 +2,8 @@
  "description": "Mapping of Anthropic beta headers for each provider. Keys are input header names, values are provider-specific header names (or null if unsupported). Only headers present in mapping keys with non-null values can be forwarded.",
  "anthropic": {
    "advanced-tool-use-2025-11-20": "advanced-tool-use-2025-11-20",
-    "bash_20241022": "bash_20241022",
-    "bash_20250124": "bash_20250124",
+    "bash_20241022": null,
+    "bash_20250124": null,
    "code-execution-2025-08-25": "code-execution-2025-08-25",
    "compact-2026-01-12": "compact-2026-01-12",
    "computer-use-2025-01-24": "computer-use-2025-01-24",
@@ -13,27 +13,27 @@
    "effort-2025-11-24": "effort-2025-11-24",
    "fast-mode-2026-02-01": "fast-mode-2026-02-01",
    "files-api-2025-04-14": "files-api-2025-04-14",
-    "structured-output-2024-03-01": "structured-output-2024-03-01",
+    "structured-output-2024-03-01": null,
    "fine-grained-tool-streaming-2025-05-14": "fine-grained-tool-streaming-2025-05-14",
    "interleaved-thinking-2025-05-14": "interleaved-thinking-2025-05-14",
    "mcp-client-2025-11-20": "mcp-client-2025-11-20",
    "mcp-client-2025-04-04": "mcp-client-2025-04-04",
-    "mcp-servers-2025-12-04": "mcp-servers-2025-12-04",
+    "mcp-servers-2025-12-04": null,
    "oauth-2025-04-20": "oauth-2025-04-20",
    "output-128k-2025-02-19": "output-128k-2025-02-19",
    "prompt-caching-scope-2026-01-05": "prompt-caching-scope-2026-01-05",
    "skills-2025-10-02": "skills-2025-10-02",
    "structured-outputs-2025-11-13": "structured-outputs-2025-11-13",
-    "text_editor_20241022": "text_editor_20241022",
-    "text_editor_20250124": "text_editor_20250124",
+    "text_editor_20241022": null,
+    "text_editor_20250124": null,
    "token-efficient-tools-2025-02-19": "token-efficient-tools-2025-02-19",
    "web-fetch-2025-09-10": "web-fetch-2025-09-10",
    "web-search-2025-03-05": "web-search-2025-03-05"
  },
  "azure_ai": {
    "advanced-tool-use-2025-11-20": "advanced-tool-use-2025-11-20",
-    "bash_20241022": "bash_20241022",
-    "bash_20250124": "bash_20250124",
+    "bash_20241022": null,
+    "bash_20250124": null,
    "code-execution-2025-08-25": "code-execution-2025-08-25",
    "compact-2026-01-12": null,
    "computer-use-2025-01-24": "computer-use-2025-01-24",
@@ -47,7 +47,7 @@
    "interleaved-thinking-2025-05-14": "interleaved-thinking-2025-05-14",
    "mcp-client-2025-11-20": "mcp-client-2025-11-20",
    "mcp-client-2025-04-04": "mcp-client-2025-04-04",
-    "mcp-servers-2025-12-04": "mcp-servers-2025-12-04",
+    "mcp-servers-2025-12-04": null,
    "output-128k-2025-02-19": null,
    "structured-output-2024-03-01": null,
    "prompt-caching-scope-2026-01-05": "prompt-caching-scope-2026-01-05",
@@ -60,7 +60,7 @@
    "web-search-2025-03-05": "web-search-2025-03-05"
  },
  "bedrock_converse": {
-    "advanced-tool-use-2025-11-20": "tool-search-tool-2025-10-19",
+    "advanced-tool-use-2025-11-20": null,
    "bash_20241022": null,
    "bash_20250124": null,
    "code-execution-2025-08-25": null,
@@ -85,7 +85,7 @@
    "text_editor_20241022": null,
    "text_editor_20250124": null,
    "token-efficient-tools-2025-02-19": null,
-    "tool-search-tool-2025-10-19": "tool-search-tool-2025-10-19",
+    "tool-search-tool-2025-10-19": null,
    "web-fetch-2025-09-10": null,
    "web-search-2025-03-05": null
  },
@@ -5,28 +5,167 @@ This module provides utilities to:
 1. Load beta header configuration from JSON (mapping of supported headers per provider)
 2. Filter and map beta headers based on provider support
 3. Handle provider-specific header name mappings (e.g., advanced-tool-use -> tool-search-tool)
+4. Support remote fetching and caching similar to model cost map

 Design:
 - JSON config contains mapping of beta headers for each provider
 - Keys are input header names, values are provider-specific header names (or null if unsupported)
 - Only headers present in mapping keys with non-null values can be forwarded
 - This enforces stricter validation than the previous unsupported list approach
+
+Configuration can be loaded from:
+- Remote URL (default): Fetches from GitHub repository
+- Local file: Set LITELLM_LOCAL_ANTHROPIC_BETA_HEADERS=True to use bundled config only
+
+Environment Variables:
+- LITELLM_LOCAL_ANTHROPIC_BETA_HEADERS: Set to "True" to disable remote fetching
+- LITELLM_ANTHROPIC_BETA_HEADERS_URL: Custom URL for remote config (optional)
 """

 import json
 import os
+from importlib.resources import files
 from typing import Dict, List, Optional, Set

+import httpx
+
 from litellm.litellm_core_utils.litellm_logging import verbose_logger

 # Cache for the loaded configuration
 _BETA_HEADERS_CONFIG: Optional[Dict] = None


+class GetAnthropicBetaHeadersConfig:
+    """
+    Handles fetching, validating, and loading the Anthropic beta headers configuration.
+    
+    Similar to GetModelCostMap, this class manages the lifecycle of the beta headers
+    configuration with support for remote fetching and local fallback.
+    """
+
+    @staticmethod
+    def load_local_beta_headers_config() -> Dict:
+        """Load the local backup beta headers config bundled with the package."""
+        try:
+            content = json.loads(
+                files("litellm")
+                .joinpath("anthropic_beta_headers_config.json")
+                .read_text(encoding="utf-8")
+            )
+            return content
+        except Exception as e:
+            verbose_logger.error(f"Failed to load local beta headers config: {e}")
+            # Return empty config as fallback
+            return {
+                "anthropic": {},
+                "azure_ai": {},
+                "bedrock": {},
+                "bedrock_converse": {},
+                "vertex_ai": {},
+                "provider_aliases": {}
+            }
+
+    @staticmethod
+    def _check_is_valid_dict(fetched_config: dict) -> bool:
+        """Check if fetched config is a non-empty dict with expected structure."""
+        if not isinstance(fetched_config, dict):
+            verbose_logger.warning(
+                "LiteLLM: Fetched beta headers config is not a dict (type=%s). "
+                "Falling back to local backup.",
+                type(fetched_config).__name__,
+            )
+            return False
+
+        if len(fetched_config) == 0:
+            verbose_logger.warning(
+                "LiteLLM: Fetched beta headers config is empty. "
+                "Falling back to local backup.",
+            )
+            return False
+
+        # Check for at least one provider key
+        provider_keys = ["anthropic", "azure_ai", "bedrock", "bedrock_converse", "vertex_ai"]
+        has_provider = any(key in fetched_config for key in provider_keys)
+        
+        if not has_provider:
+            verbose_logger.warning(
+                "LiteLLM: Fetched beta headers config missing provider keys. "
+                "Falling back to local backup.",
+            )
+            return False
+
+        return True
+
+    @classmethod
+    def validate_beta_headers_config(cls, fetched_config: dict) -> bool:
+        """
+        Validate the integrity of a fetched beta headers config.
+        
+        Returns True if all checks pass, False otherwise.
+        """
+        return cls._check_is_valid_dict(fetched_config)
+
+    @staticmethod
+    def fetch_remote_beta_headers_config(url: str, timeout: int = 5) -> dict:
+        """
+        Fetch the beta headers config from a remote URL.
+        
+        Returns the parsed JSON dict. Raises on network/parse errors
+        (caller is expected to handle).
+        """
+        response = httpx.get(url, timeout=timeout)
+        response.raise_for_status()
+        return response.json()
+
+
+def get_beta_headers_config(url: str) -> dict:
+    """
+    Public entry point — returns the beta headers config dict.
+    
+    1. If ``LITELLM_LOCAL_ANTHROPIC_BETA_HEADERS`` is set, uses the local backup only.
+    2. Otherwise fetches from ``url``, validates integrity, and falls back
+       to the local backup on any failure.
+    
+    Args:
+        url: URL to fetch the remote beta headers configuration from
+        
+    Returns:
+        Dict containing the beta headers configuration
+    """
+    # Check if local-only mode is enabled
+    if os.getenv("LITELLM_LOCAL_ANTHROPIC_BETA_HEADERS", "").lower() == "true":
+        # verbose_logger.debug("Using local Anthropic beta headers config (LITELLM_LOCAL_ANTHROPIC_BETA_HEADERS=True)")
+        return GetAnthropicBetaHeadersConfig.load_local_beta_headers_config()
+
+    try:
+        content = GetAnthropicBetaHeadersConfig.fetch_remote_beta_headers_config(url)
+    except Exception as e:
+        verbose_logger.warning(
+            "LiteLLM: Failed to fetch remote beta headers config from %s: %s. "
+            "Falling back to local backup.",
+            url,
+            str(e),
+        )
+        return GetAnthropicBetaHeadersConfig.load_local_beta_headers_config()
+
+    # Validate the fetched config
+    if not GetAnthropicBetaHeadersConfig.validate_beta_headers_config(fetched_config=content):
+        verbose_logger.warning(
+            "LiteLLM: Fetched beta headers config failed integrity check. "
+            "Using local backup instead. url=%s",
+            url,
+        )
+        return GetAnthropicBetaHeadersConfig.load_local_beta_headers_config()
+
+    return content
+
+
 def _load_beta_headers_config() -> Dict:
    """
-    Load the beta headers configuration from JSON file.
-    Uses caching to avoid repeated file reads.
+    Load the beta headers configuration.
+    Uses caching to avoid repeated fetches/file reads.
+    
+    This function is called by all public API functions and manages the global cache.
    
    Returns:
        Dict containing the beta headers configuration
@@ -36,26 +175,27 @@ def _load_beta_headers_config() -> Dict:
    if _BETA_HEADERS_CONFIG is not None:
        return _BETA_HEADERS_CONFIG
    
-    config_path = os.path.join(
-        os.path.dirname(__file__),
-        "anthropic_beta_headers_config.json"
-    )
+    # Get the URL from environment or use default
+    from litellm import anthropic_beta_headers_url
    
-    try:
-        with open(config_path, "r") as f:
-            _BETA_HEADERS_CONFIG = json.load(f)
-            verbose_logger.debug(f"Loaded beta headers config from {config_path}")
-            return _BETA_HEADERS_CONFIG
-    except Exception as e:
-        verbose_logger.error(f"Failed to load beta headers config: {e}")
-        # Return empty config as fallback (empty mappings)
-        return {
-            "anthropic": {},
-            "azure_ai": {},
-            "bedrock": {},
-            "bedrock_converse": {},
-            "vertex_ai": {}
-        }
+    _BETA_HEADERS_CONFIG = get_beta_headers_config(url=anthropic_beta_headers_url)
+    verbose_logger.debug("Loaded and cached beta headers config")
+    
+    return _BETA_HEADERS_CONFIG
+
+
+def reload_beta_headers_config() -> Dict:
+    """
+    Force reload the beta headers configuration from source (remote or local).
+    Clears the cache and fetches fresh configuration.
+    
+    Returns:
+        Dict containing the newly loaded beta headers configuration
+    """
+    global _BETA_HEADERS_CONFIG
+    _BETA_HEADERS_CONFIG = None
+    verbose_logger.info("Reloading beta headers config (cache cleared)")
+    return _load_beta_headers_config()


 def get_provider_name(provider: str) -> str:
@@ -329,6 +329,9 @@ from litellm.proxy.hooks.prompt_injection_detection import (
 from litellm.proxy.hooks.proxy_track_cost_callback import _ProxyDBLogger
 from litellm.proxy.image_endpoints.endpoints import router as image_router
 from litellm.proxy.litellm_pre_call_utils import add_litellm_data_to_request
+from litellm.proxy.management_endpoints.access_group_endpoints import (
+    router as access_group_router,
+)
 from litellm.proxy.management_endpoints.budget_management_endpoints import (
    router as budget_management_router,
 )
@@ -339,8 +342,8 @@ from litellm.proxy.management_endpoints.callback_management_endpoints import (
    router as callback_management_endpoints_router,
 )
 from litellm.proxy.management_endpoints.common_utils import (
-    admin_can_invite_user,
    _user_has_admin_privileges,
+    admin_can_invite_user,
 )
 from litellm.proxy.management_endpoints.cost_tracking_settings import (
    router as cost_tracking_settings_router,
@@ -393,9 +396,6 @@ from litellm.proxy.management_endpoints.tag_management_endpoints import (
 from litellm.proxy.management_endpoints.team_callback_endpoints import (
    router as team_callback_router,
 )
-from litellm.proxy.management_endpoints.access_group_endpoints import (
-    router as access_group_router,
-)
 from litellm.proxy.management_endpoints.team_endpoints import router as team_router
 from litellm.proxy.management_endpoints.team_endpoints import (
    update_team,
@@ -1486,6 +1486,9 @@ celery_fn = None  # Redis Queue for handling requests
 scheduler = None
 last_model_cost_map_reload = None

+# Global variable for anthropic beta headers reload scheduling
+last_anthropic_beta_headers_reload = None
+

 ### DB WRITER ###
 db_writer_client: Optional[AsyncHTTPHandler] = None
@@ -4149,6 +4152,10 @@ class ProxyConfig:

        if self._should_load_db_object(object_type="model_cost_map"):
            await self._check_and_reload_model_cost_map(prisma_client=prisma_client)
+        
+        if self._should_load_db_object(object_type="anthropic_beta_headers"):
+            await self._check_and_reload_anthropic_beta_headers(prisma_client=prisma_client)
+        
        if self._should_load_db_object(object_type="sso_settings"):
            await self._init_sso_settings_in_db(prisma_client=prisma_client)
        if self._should_load_db_object(object_type="cache_settings"):
@@ -4377,6 +4384,107 @@ class ProxyConfig:
                f"Error in _check_and_reload_model_cost_map: {str(e)}"
            )

+    async def _check_and_reload_anthropic_beta_headers(self, prisma_client: PrismaClient):
+        """
+        Check if anthropic beta headers config needs to be reloaded based on database configuration.
+        This function runs every 10 seconds as part of _init_non_llm_objects_in_db.
+        """
+        try:
+            # Get anthropic beta headers reload configuration from database
+            config_record = await prisma_client.db.litellm_config.find_unique(
+                where={"param_name": "anthropic_beta_headers_reload_config"}
+            )
+
+            if config_record is None or config_record.param_value is None:
+                return  # No configuration found, skip reload
+
+            config = config_record.param_value
+            interval_hours = config.get("interval_hours")
+            force_reload = config.get("force_reload", False)
+
+            if interval_hours is None and force_reload is False:
+                return  # No interval configured, skip reload
+
+            current_time = datetime.utcnow()
+
+            # Check if we need to reload based on interval or force reload
+            should_reload = False
+
+            if force_reload:
+                should_reload = True
+                verbose_proxy_logger.info(
+                    "Anthropic beta headers reload triggered by force reload flag"
+                )
+            elif interval_hours is not None:
+                # Use pod's in-memory last reload time
+                global last_anthropic_beta_headers_reload
+                if last_anthropic_beta_headers_reload is not None:
+                    try:
+                        last_reload_time = datetime.fromisoformat(
+                            last_anthropic_beta_headers_reload
+                        )
+                        time_since_last_reload = current_time - last_reload_time
+                        hours_since_last_reload = (
+                            time_since_last_reload.total_seconds() / 3600
+                        )
+
+                        if hours_since_last_reload >= interval_hours:
+                            should_reload = True
+                            verbose_proxy_logger.info(
+                                f"Anthropic beta headers reload triggered by interval. Hours since last reload: {hours_since_last_reload:.2f}, Interval: {interval_hours}"
+                            )
+                    except Exception as e:
+                        verbose_proxy_logger.warning(
+                            f"Error parsing last reload time: {e}"
+                        )
+                        # If we can't parse the last reload time, reload anyway
+                        should_reload = True
+                else:
+                    # No last reload time recorded, reload now
+                    should_reload = True
+                    verbose_proxy_logger.info(
+                        "Anthropic beta headers reload triggered - no previous reload time recorded"
+                    )
+
+            if should_reload:
+                # Perform the reload
+                from litellm.anthropic_beta_headers_manager import (
+                    reload_beta_headers_config,
+                )
+
+                new_config = reload_beta_headers_config()
+
+                # Update pod's in-memory last reload time
+                last_anthropic_beta_headers_reload = current_time.isoformat()
+
+                # Clear force reload flag in database
+                await prisma_client.db.litellm_config.upsert(
+                    where={"param_name": "anthropic_beta_headers_reload_config"},
+                    data={
+                        "create": {
+                            "param_name": "anthropic_beta_headers_reload_config",
+                            "param_value": safe_dumps(
+                                {
+                                    "interval_hours": interval_hours,
+                                    "force_reload": False,
+                                }
+                            ),
+                        },
+                        "update": {"param_value": safe_dumps({"force_reload": False})},
+                    },
+                )
+
+                # Count providers in config
+                provider_count = sum(1 for k in new_config.keys() if k != "provider_aliases" and k != "description")
+                verbose_proxy_logger.info(
+                    f"Anthropic beta headers config reloaded successfully. Providers: {provider_count}"
+                )
+
+        except Exception as e:
+            verbose_proxy_logger.exception(
+                f"Error in _check_and_reload_anthropic_beta_headers: {str(e)}"
+            )
+
    def _get_prompt_spec_for_db_prompt(self, db_prompt):
        """
        Convert a DB prompt object to a PromptSpec object.
@@ -11887,6 +11995,296 @@ async def get_model_cost_map_reload_status(
        )


+#### ANTHROPIC BETA HEADERS RELOAD ENDPOINTS ####
+
+
+@router.post(
+    "/reload/anthropic_beta_headers",
+    tags=["model management"],
+    dependencies=[Depends(user_api_key_auth)],
+    include_in_schema=False,
+)
+async def reload_anthropic_beta_headers(
+    user_api_key_dict: UserAPIKeyAuth = Depends(user_api_key_auth),
+):
+    """
+    ADMIN ONLY / MASTER KEY Only Endpoint
+
+    Manually reload the Anthropic beta headers configuration from the remote source.
+    This will fetch fresh configuration from the anthropic_beta_headers_config.json file.
+    """
+    # Check if user is admin
+    if user_api_key_dict.user_role != LitellmUserRoles.PROXY_ADMIN:
+        raise HTTPException(
+            status_code=403,
+            detail=f"Access denied. Admin role required. Current role: {user_api_key_dict.user_role}",
+        )
+
+    try:
+        global prisma_client
+        if prisma_client is None:
+            raise HTTPException(
+                status_code=500, detail="Database connection not available"
+            )
+
+        # Immediately reload the beta headers config in the current pod
+        from litellm.anthropic_beta_headers_manager import reload_beta_headers_config
+
+        new_config = reload_beta_headers_config()
+
+        # Update pod's in-memory last reload time
+        global last_anthropic_beta_headers_reload
+        current_time = datetime.utcnow()
+        last_anthropic_beta_headers_reload = current_time.isoformat()
+
+        # Set force reload flag in database for other pods
+        await prisma_client.db.litellm_config.upsert(
+            where={"param_name": "anthropic_beta_headers_reload_config"},
+            data={
+                "create": {
+                    "param_name": "anthropic_beta_headers_reload_config",
+                    "param_value": safe_dumps(
+                        {"interval_hours": None, "force_reload": True}
+                    ),
+                },
+                "update": {"param_value": safe_dumps({"force_reload": True})},
+            },
+        )
+
+        provider_count = sum(1 for k in new_config.keys() if k not in ["provider_aliases", "description"])
+        verbose_proxy_logger.info(
+            f"Anthropic beta headers config reloaded successfully in current pod. Providers: {provider_count}"
+        )
+
+        return {
+            "message": f"Anthropic beta headers configuration reloaded successfully! {provider_count} providers updated.",
+            "status": "success",
+            "providers_count": provider_count,
+            "timestamp": current_time.isoformat(),
+        }
+    except Exception as e:
+        verbose_proxy_logger.exception(f"Failed to reload anthropic beta headers: {str(e)}")
+        raise HTTPException(
+            status_code=500, detail=f"Failed to reload anthropic beta headers: {str(e)}"
+        )
+
+
+@router.post(
+    "/schedule/anthropic_beta_headers_reload",
+    tags=["model management"],
+    dependencies=[Depends(user_api_key_auth)],
+    include_in_schema=False,
+)
+async def schedule_anthropic_beta_headers_reload(
+    hours: int,
+    user_api_key_dict: UserAPIKeyAuth = Depends(user_api_key_auth),
+):
+    """
+    ADMIN ONLY / MASTER KEY Only Endpoint
+
+    Schedule periodic reload of the Anthropic beta headers configuration.
+    This will create a background job that reloads the configuration every specified hours.
+    """
+    # Check if user is admin
+    if user_api_key_dict.user_role != LitellmUserRoles.PROXY_ADMIN:
+        raise HTTPException(
+            status_code=403,
+            detail=f"Access denied. Admin role required. Current role: {user_api_key_dict.user_role}",
+        )
+
+    if hours <= 0:
+        raise HTTPException(status_code=400, detail="Hours must be greater than 0")
+
+    try:
+        global prisma_client
+        if prisma_client is None:
+            raise HTTPException(
+                status_code=500, detail="Database connection not available"
+            )
+
+        # Update database with new reload configuration
+        await prisma_client.db.litellm_config.upsert(
+            where={"param_name": "anthropic_beta_headers_reload_config"},
+            data={
+                "create": {
+                    "param_name": "anthropic_beta_headers_reload_config",
+                    "param_value": safe_dumps(
+                        {"interval_hours": hours, "force_reload": False}
+                    ),
+                },
+                "update": {
+                    "param_value": safe_dumps(
+                        {"interval_hours": hours, "force_reload": False}
+                    )
+                },
+            },
+        )
+
+        verbose_proxy_logger.info(
+            f"Anthropic beta headers reload scheduled for every {hours} hours"
+        )
+
+        return {
+            "message": f"Anthropic beta headers reload scheduled for every {hours} hours",
+            "status": "success",
+            "interval_hours": hours,
+            "timestamp": datetime.utcnow().isoformat(),
+        }
+    except Exception as e:
+        verbose_proxy_logger.exception(
+            f"Failed to schedule anthropic beta headers reload: {str(e)}"
+        )
+        raise HTTPException(
+            status_code=500,
+            detail=f"Failed to schedule anthropic beta headers reload: {str(e)}",
+        )
+
+
+@router.delete(
+    "/schedule/anthropic_beta_headers_reload",
+    tags=["model management"],
+    dependencies=[Depends(user_api_key_auth)],
+    include_in_schema=False,
+)
+async def cancel_anthropic_beta_headers_reload(
+    user_api_key_dict: UserAPIKeyAuth = Depends(user_api_key_auth),
+):
+    """
+    ADMIN ONLY / MASTER KEY Only Endpoint
+
+    Cancel the scheduled periodic reload of the Anthropic beta headers configuration.
+    """
+    # Check if user is admin
+    if user_api_key_dict.user_role != LitellmUserRoles.PROXY_ADMIN:
+        raise HTTPException(
+            status_code=403,
+            detail=f"Access denied. Admin role required. Current role: {user_api_key_dict.user_role}",
+        )
+
+    try:
+        global prisma_client
+        if prisma_client is None:
+            raise HTTPException(
+                status_code=500, detail="Database connection not available"
+            )
+
+        # Remove reload configuration from database
+        await prisma_client.db.litellm_config.delete(
+            where={"param_name": "anthropic_beta_headers_reload_config"}
+        )
+
+        verbose_proxy_logger.info("Anthropic beta headers reload schedule cancelled")
+
+        return {
+            "message": "Anthropic beta headers reload schedule cancelled",
+            "status": "success",
+            "timestamp": datetime.utcnow().isoformat(),
+        }
+    except Exception as e:
+        verbose_proxy_logger.exception(
+            f"Failed to cancel anthropic beta headers reload: {str(e)}"
+        )
+        raise HTTPException(
+            status_code=500, detail=f"Failed to cancel anthropic beta headers reload: {str(e)}"
+        )
+
+
+@router.get(
+    "/schedule/anthropic_beta_headers_reload/status",
+    tags=["model management"],
+    dependencies=[Depends(user_api_key_auth)],
+    include_in_schema=False,
+)
+async def get_anthropic_beta_headers_reload_status(
+    user_api_key_dict: UserAPIKeyAuth = Depends(user_api_key_auth),
+):
+    """
+    ADMIN ONLY / MASTER KEY Only Endpoint
+
+    Get the status of the scheduled Anthropic beta headers reload job.
+    """
+    # Check if user is admin
+    if user_api_key_dict.user_role != LitellmUserRoles.PROXY_ADMIN:
+        raise HTTPException(
+            status_code=403,
+            detail=f"Access denied. Admin role required. Current role: {user_api_key_dict.user_role}",
+        )
+
+    try:
+        global prisma_client, last_anthropic_beta_headers_reload
+
+        verbose_proxy_logger.info(
+            f"Checking anthropic beta headers reload status. Last reload: {last_anthropic_beta_headers_reload}"
+        )
+
+        if prisma_client is None:
+            verbose_proxy_logger.info("No database connection, returning not scheduled")
+            return {
+                "scheduled": False,
+                "interval_hours": None,
+                "last_run": None,
+                "next_run": None,
+            }
+
+        # Get reload configuration from database
+        config_record = await prisma_client.db.litellm_config.find_unique(
+            where={"param_name": "anthropic_beta_headers_reload_config"}
+        )
+
+        if config_record is None or config_record.param_value is None:
+            verbose_proxy_logger.info("No anthropic beta headers reload configuration found")
+            return {
+                "scheduled": False,
+                "interval_hours": None,
+                "last_run": None,
+                "next_run": None,
+            }
+
+        config = config_record.param_value
+        interval_hours = config.get("interval_hours")
+
+        if interval_hours is None:
+            verbose_proxy_logger.info("No interval configured, returning not scheduled")
+            return {
+                "scheduled": False,
+                "interval_hours": None,
+                "last_run": None,
+                "next_run": None,
+            }
+
+        current_time = datetime.utcnow()
+        next_run = None
+
+        # Use pod's in-memory last reload time
+        if last_anthropic_beta_headers_reload is not None:
+            try:
+                last_reload_time = datetime.fromisoformat(last_anthropic_beta_headers_reload)
+                time_since_last_reload = current_time - last_reload_time
+                hours_since_last_reload = time_since_last_reload.total_seconds() / 3600
+
+                if hours_since_last_reload < interval_hours:
+                    next_run = (
+                        last_reload_time + timedelta(hours=interval_hours)
+                    ).isoformat()
+            except Exception as e:
+                verbose_proxy_logger.warning(f"Error parsing last reload time: {e}")
+
+        return {
+            "scheduled": True,
+            "interval_hours": interval_hours,
+            "last_run": last_anthropic_beta_headers_reload,
+            "next_run": next_run,
+        }
+    except Exception as e:
+        verbose_proxy_logger.exception(
+            f"Failed to get anthropic beta headers reload status: {str(e)}"
+        )
+        raise HTTPException(
+            status_code=500,
+            detail=f"Failed to get anthropic beta headers reload status: {str(e)}",
+        )
+
+
@router.get("/", dependencies=[Depends(user_api_key_auth)])
 async def home(request: Request):
    return "LiteLLM: RUNNING"
@@ -0,0 +1,169 @@
+"""
+This test ensures that the proxy can passthrough anthropic requests
+"""
+from pathlib import Path
+import pytest
+import aiohttp
+import json
+
+def get_all_supported_anthropic_beta_headers(provider: str):
+    config_path = (
+        Path(__file__).resolve().parents[2]
+        / "litellm"
+        / "anthropic_beta_headers_config.json"
+    )
+
+    with open(config_path, "r") as f:
+        config = json.load(f)
+
+    anthropic_mapping = config.get(provider, {})
+
+    # Only include headers that have a non-null mapping value
+    return [
+        header_name
+        for header_name, provider_value in anthropic_mapping.items()
+        if provider_value is not None
+    ]
+
+
+@pytest.mark.asyncio
+@pytest.mark.flaky(retries=3, delay=2)
+@pytest.mark.parametrize(
+    "model_name,provider_name",
+    [
+        ("claude-sonnet-4-5-20250929", "anthropic"),
+        ("azure-ai-claude-opus-4.5", "azure_ai"),
+        ("vertex-ai-claude-opus-4-6", "vertex_ai"),
+    ],
+)
+async def test_anthropic_messages_with_all_beta_headers(model_name, provider_name):
+    """
+    Test that v1/messages endpoint works with all non-null Anthropic beta headers
+    and doesn't throw errors
+    """
+    print("Testing v1/messages with all non-null Anthropic beta headers")
+    
+    
+    headers = {
+        "Authorization": "Bearer sk-1234",
+        "Content-Type": "application/json",
+        "anthropic-version": "2023-06-01",
+        "anthropic-beta": ",".join(get_all_supported_anthropic_beta_headers(provider_name)),
+    }
+    
+    payload = {
+        "model": model_name,
+        "max_tokens": 10,
+        "messages": [{"role": "user", "content": "Say 'hello' and nothing else"}],
+        "tools": [{
+            "type": "code_execution_20250825",
+            "name": "code_execution"
+        }]
+    }
+    
+    async with aiohttp.ClientSession() as session:
+        async with session.post(
+            "http://0.0.0.0:4000/v1/messages", 
+            json=payload, 
+            headers=headers
+        ) as response:
+            response_text = await response.text()
+            print(f"Response status: {response.status}")
+            print(f"Response text: {response_text}")
+            
+            # The request should succeed without errors
+            assert response.status == 200, f"Request should succeed, got status {response.status}: {response_text}"
+            
+            response_json = await response.json()
+            print(f"Response JSON: {json.dumps(response_json, indent=4, default=str)}")
+            
+            # Basic response validation
+            assert "id" in response_json, "Response should have an id"
+            assert "content" in response_json, "Response should have content"
+            assert "model" in response_json, "Response should have model"
+            assert "usage" in response_json, "Response should have usage"
+            
+            # Verify usage information
+            usage = response_json["usage"]
+            assert "input_tokens" in usage, "Usage should have input_tokens"
+            assert "output_tokens" in usage, "Usage should have output_tokens"
+            assert usage["input_tokens"] > 0, "Should have some input tokens"
+            assert usage["output_tokens"] > 0, "Should have some output tokens"
+            
+            print(f"✅ Test passed: Request with all beta headers succeeded")
+            print(f"   Model: {response_json['model']}")
+            print(f"   Input tokens: {usage['input_tokens']}")
+            print(f"   Output tokens: {usage['output_tokens']}")
+
+
+
+@pytest.mark.asyncio
+@pytest.mark.flaky(retries=3, delay=2)
+@pytest.mark.parametrize(
+    "model_name,provider_name",
+    [
+        ("bedrock-claude-opus-4.5", "bedrock"),
+        ("bedrock-converse-claude-sonnet-4.5", "bedrock_converse")
+    ],
+)
+async def test_bedrock_invoke_messages_with_all_beta_headers(
+    model_name, provider_name
+):
+    """
+    Test that v1/messages endpoint works with all non-null Anthropic beta headers
+    for both bedrock and bedrock_converse providers.
+    """
+    print(f"Testing v1/messages for model={model_name}, provider={provider_name}")
+
+    beta_headers = get_all_supported_anthropic_beta_headers(provider_name)
+
+    headers = {
+        "Authorization": "Bearer sk-1234",
+        "Content-Type": "application/json",
+        "anthropic-version": "2023-06-01",
+        "anthropic-beta": ",".join(beta_headers),
+    }
+
+    payload = {
+        "model": model_name,
+        "max_tokens": 10,
+        "messages": [
+            {"role": "user", "content": "Say 'hello' and nothing else"}
+        ],
+    }
+
+    async with aiohttp.ClientSession() as session:
+        async with session.post(
+            "http://0.0.0.0:4000/v1/messages",
+            json=payload,
+            headers=headers,
+        ) as response:
+
+            response_text = await response.text()
+            print(f"Response status: {response.status}")
+            print(f"Response text: {response_text}")
+
+            assert (
+                response.status == 200
+            ), f"{provider_name} request failed: {response.status}: {response_text}"
+
+            response_json = await response.json()
+
+            # Basic response validation
+            assert "id" in response_json
+            assert "content" in response_json
+            assert "model" in response_json
+            assert "usage" in response_json
+
+            usage = response_json["usage"]
+
+            assert "input_tokens" in usage
+            assert "output_tokens" in usage
+            assert usage["input_tokens"] > 0
+            assert usage["output_tokens"] > 0
+
+            print("✅ Test passed")
+            print(f"   Provider: {provider_name}")
+            print(f"   Model: {response_json['model']}")
+            print(f"   Input tokens: {usage['input_tokens']}")
+            print(f"   Output tokens: {usage['output_tokens']}")
@@ -29,3 +29,13 @@ model_list:
    litellm_params:
      model: "bedrock/converse/us.anthropic.claude-sonnet-4-5-20250929-v1:0"
      aws_region_name: "us-east-1"
+
+  # Vertex AI models
+  - model_name: vertex-ai-claude-opus-4-6
+    litellm_params:
+      model: "vertex_ai/claude-opus-4-6"
+      vertex_ai_project: "pathrise-convert-1606954137718"
+      vertex_ai_location: "asia-southeast1"
+
+general_settings:  
+  forward_client_headers_to_llm_api: true
@@ -24,8 +24,15 @@ class TestAnthropicBetaHeadersFiltering:
    """Test beta header filtering and mapping for all providers."""

    @pytest.fixture(autouse=True)
-    def setup(self):
+    def setup(self, monkeypatch):
        """Load the beta headers config for testing."""
+        # Force use of local config file for tests
+        monkeypatch.setenv("LITELLM_LOCAL_ANTHROPIC_BETA_HEADERS", "True")
+        
+        # Clear the cached config to ensure fresh load with local config
+        from litellm import anthropic_beta_headers_manager
+        anthropic_beta_headers_manager._BETA_HEADERS_CONFIG = None
+        
        config_path = os.path.join(
            os.path.dirname(litellm.__file__),
            "anthropic_beta_headers_config.json",