diff --git a/docs/my-website/docs/proxy/deploy.md b/docs/my-website/docs/proxy/deploy.md
index d57686dc78..511a9dda08 100644
--- a/docs/my-website/docs/proxy/deploy.md
+++ b/docs/my-website/docs/proxy/deploy.md
@@ -1049,3 +1049,25 @@ export DATABASE_SCHEMA="schema-name" # skip to use the default "public" schema
litellm --config /path/to/config.yaml --iam_token_db_auth
```
+### ✨ Blocking web crawlers
+
+Note: This is an [enterprise only feature](https://docs.litellm.ai/docs/enterprise).
+
+To block web crawlers from indexing the proxy server endpoints, set the `block_robots` setting to `true` in your `litellm_config.yaml` file.
+
+```yaml showLineNumbers title="litellm_config.yaml"
+general_settings:
+ block_robots: true
+```
+
+#### How it works
+
+When this is enabled, the `/robots.txt` endpoint will return a 200 status code with the following content:
+
+```shell showLineNumbers title="robots.txt"
+User-agent: *
+Disallow: /
+```
+
+
+
diff --git a/docs/my-website/docs/proxy/enterprise.md b/docs/my-website/docs/proxy/enterprise.md
index 26dc9d4950..6789fb6ef2 100644
--- a/docs/my-website/docs/proxy/enterprise.md
+++ b/docs/my-website/docs/proxy/enterprise.md
@@ -43,7 +43,9 @@ Features:
- ✅ [Public Model Hub](#public-model-hub)
- ✅ [Custom Email Branding](./email.md#customizing-email-branding)
-## Audit Logs
+## Security
+
+### Audit Logs
Store Audit logs for **Create, Update Delete Operations** done on `Teams` and `Virtual Keys`
@@ -95,7 +97,295 @@ curl --location 'http://0.0.0.0:4000/team/new' \
```
-## Tracking Spend for Custom Tags
+### Blocking web crawlers
+
+To block web crawlers from indexing the proxy server endpoints, set the `block_robots` setting to `true` in your `litellm_config.yaml` file.
+
+```yaml showLineNumbers title="litellm_config.yaml"
+general_settings:
+ block_robots: true
+```
+
+#### How it works
+
+When this is enabled, the `/robots.txt` endpoint will return a 200 status code with the following content:
+
+```shell showLineNumbers title="robots.txt"
+User-agent: *
+Disallow: /
+```
+
+
+
+### Required Params for LLM Requests
+Use this when you want to enforce all requests to include certain params. Example you need all requests to include the `user` and `["metadata]["generation_name"]` params.
+
+
+
+
+
+
+**Step 1** Define all Params you want to enforce on config.yaml
+
+This means `["user"]` and `["metadata]["generation_name"]` are required in all LLM Requests to LiteLLM
+
+```yaml
+general_settings:
+ master_key: sk-1234
+ enforced_params:
+ - user
+ - metadata.generation_name
+```
+
+
+
+
+```bash
+curl -L -X POST 'http://0.0.0.0:4000/key/generate' \
+-H 'Authorization: Bearer sk-1234' \
+-H 'Content-Type: application/json' \
+-d '{
+ "enforced_params": ["user", "metadata.generation_name"]
+}'
+```
+
+
+
+
+**Step 2 Verify if this works**
+
+
+
+
+
+```shell
+curl --location 'http://localhost:4000/chat/completions' \
+ --header 'Authorization: Bearer sk-5fmYeaUEbAMpwBNT-QpxyA' \
+ --header 'Content-Type: application/json' \
+ --data '{
+ "model": "gpt-3.5-turbo",
+ "messages": [
+ {
+ "role": "user",
+ "content": "hi"
+ }
+ ]
+}'
+```
+
+Expected Response
+
+```shell
+{"error":{"message":"Authentication Error, BadRequest please pass param=user in request body. This is a required param","type":"auth_error","param":"None","code":401}}%
+```
+
+
+
+
+
+```shell
+curl --location 'http://localhost:4000/chat/completions' \
+ --header 'Authorization: Bearer sk-5fmYeaUEbAMpwBNT-QpxyA' \
+ --header 'Content-Type: application/json' \
+ --data '{
+ "model": "gpt-3.5-turbo",
+ "user": "gm",
+ "messages": [
+ {
+ "role": "user",
+ "content": "hi"
+ }
+ ],
+ "metadata": {}
+}'
+```
+
+Expected Response
+
+```shell
+{"error":{"message":"Authentication Error, BadRequest please pass param=[metadata][generation_name] in request body. This is a required param","type":"auth_error","param":"None","code":401}}%
+```
+
+
+
+
+
+```shell
+curl --location 'http://localhost:4000/chat/completions' \
+ --header 'Authorization: Bearer sk-5fmYeaUEbAMpwBNT-QpxyA' \
+ --header 'Content-Type: application/json' \
+ --data '{
+ "model": "gpt-3.5-turbo",
+ "user": "gm",
+ "messages": [
+ {
+ "role": "user",
+ "content": "hi"
+ }
+ ],
+ "metadata": {"generation_name": "prod-app"}
+}'
+```
+
+Expected Response
+
+```shell
+{"id":"chatcmpl-9XALnHqkCBMBKrOx7Abg0hURHqYtY","choices":[{"finish_reason":"stop","index":0,"message":{"content":"Hello! How can I assist you today?","role":"assistant"}}],"created":1717691639,"model":"gpt-3.5-turbo-0125","object":"chat.completion","system_fingerprint":null,"usage":{"completion_tokens":9,"prompt_tokens":8,"total_tokens":17}}%
+```
+
+
+
+
+
+
+### Control available public, private routes
+
+**Restrict certain endpoints of proxy**
+
+:::info
+
+❓ Use this when you want to:
+- make an existing private route -> public
+- set certain routes as admin_only routes
+
+:::
+
+#### Usage - Define public, admin only routes
+
+**Step 1** - Set on config.yaml
+
+
+| Route Type | Optional | Requires Virtual Key Auth | Admin Can Access | All Roles Can Access | Description |
+|------------|----------|---------------------------|-------------------|----------------------|-------------|
+| `public_routes` | ✅ | ❌ | ✅ | ✅ | Routes that can be accessed without any authentication |
+| `admin_only_routes` | ✅ | ✅ | ✅ | ❌ | Routes that can only be accessed by [Proxy Admin](./self_serve#available-roles) |
+| `allowed_routes` | ✅ | ✅ | ✅ | ✅ | Routes are exposed on the proxy. If not set then all routes exposed. |
+
+`LiteLLMRoutes.public_routes` is an ENUM corresponding to the default public routes on LiteLLM. [You can see this here](https://github.com/BerriAI/litellm/blob/main/litellm/proxy/_types.py)
+
+```yaml
+general_settings:
+ master_key: sk-1234
+ public_routes: ["LiteLLMRoutes.public_routes", "/spend/calculate"] # routes that can be accessed without any auth
+ admin_only_routes: ["/key/generate"] # Optional - routes that can only be accessed by Proxy Admin
+ allowed_routes: ["/chat/completions", "/spend/calculate", "LiteLLMRoutes.public_routes"] # Optional - routes that can be accessed by anyone after Authentication
+```
+
+**Step 2** - start proxy
+
+```shell
+litellm --config config.yaml
+```
+
+**Step 3** - Test it
+
+
+
+
+
+```shell
+curl --request POST \
+ --url 'http://localhost:4000/spend/calculate' \
+ --header 'Content-Type: application/json' \
+ --data '{
+ "model": "gpt-4",
+ "messages": [{"role": "user", "content": "Hey, how'\''s it going?"}]
+ }'
+```
+
+🎉 Expect this endpoint to work without an `Authorization / Bearer Token`
+
+
+
+
+
+
+**Successful Request**
+
+```shell
+curl --location 'http://0.0.0.0:4000/key/generate' \
+--header 'Authorization: Bearer ' \
+--header 'Content-Type: application/json' \
+--data '{}'
+```
+
+
+**Un-successfull Request**
+
+```shell
+ curl --location 'http://0.0.0.0:4000/key/generate' \
+--header 'Authorization: Bearer ' \
+--header 'Content-Type: application/json' \
+--data '{"user_role": "internal_user"}'
+```
+
+**Expected Response**
+
+```json
+{
+ "error": {
+ "message": "user not allowed to access this route. Route=/key/generate is an admin only route",
+ "type": "auth_error",
+ "param": "None",
+ "code": "403"
+ }
+}
+```
+
+
+
+
+
+
+
+**Successful Request**
+
+```shell
+curl http://localhost:4000/chat/completions \
+-H "Content-Type: application/json" \
+-H "Authorization: Bearer sk-1234" \
+-d '{
+"model": "fake-openai-endpoint",
+"messages": [
+ {"role": "user", "content": "Hello, Claude"}
+]
+}'
+```
+
+
+**Un-successfull Request**
+
+```shell
+curl --location 'http://0.0.0.0:4000/embeddings' \
+--header 'Content-Type: application/json' \
+-H "Authorization: Bearer sk-1234" \
+--data ' {
+"model": "text-embedding-ada-002",
+"input": ["write a litellm poem"]
+}'
+```
+
+**Expected Response**
+
+```json
+{
+ "error": {
+ "message": "Route /embeddings not allowed",
+ "type": "auth_error",
+ "param": "None",
+ "code": "403"
+ }
+}
+```
+
+
+
+
+
+
+## Spend Tracking
+
+### Custom Tags
Requirements:
@@ -294,7 +584,7 @@ curl -X GET "http://0.0.0.0:4000/spend/tags" \
```
-## Tracking Spend with custom metadata
+### Tracking Spend with custom metadata
Requirements:
@@ -506,278 +796,6 @@ curl -X GET "http://0.0.0.0:4000/spend/logs?request_id=
-
-
-
-**Step 1** Define all Params you want to enforce on config.yaml
-
-This means `["user"]` and `["metadata]["generation_name"]` are required in all LLM Requests to LiteLLM
-
-```yaml
-general_settings:
- master_key: sk-1234
- enforced_params:
- - user
- - metadata.generation_name
-```
-
-
-
-
-```bash
-curl -L -X POST 'http://0.0.0.0:4000/key/generate' \
--H 'Authorization: Bearer sk-1234' \
--H 'Content-Type: application/json' \
--d '{
- "enforced_params": ["user", "metadata.generation_name"]
-}'
-```
-
-
-
-
-**Step 2 Verify if this works**
-
-
-
-
-
-```shell
-curl --location 'http://localhost:4000/chat/completions' \
- --header 'Authorization: Bearer sk-5fmYeaUEbAMpwBNT-QpxyA' \
- --header 'Content-Type: application/json' \
- --data '{
- "model": "gpt-3.5-turbo",
- "messages": [
- {
- "role": "user",
- "content": "hi"
- }
- ]
-}'
-```
-
-Expected Response
-
-```shell
-{"error":{"message":"Authentication Error, BadRequest please pass param=user in request body. This is a required param","type":"auth_error","param":"None","code":401}}%
-```
-
-
-
-
-
-```shell
-curl --location 'http://localhost:4000/chat/completions' \
- --header 'Authorization: Bearer sk-5fmYeaUEbAMpwBNT-QpxyA' \
- --header 'Content-Type: application/json' \
- --data '{
- "model": "gpt-3.5-turbo",
- "user": "gm",
- "messages": [
- {
- "role": "user",
- "content": "hi"
- }
- ],
- "metadata": {}
-}'
-```
-
-Expected Response
-
-```shell
-{"error":{"message":"Authentication Error, BadRequest please pass param=[metadata][generation_name] in request body. This is a required param","type":"auth_error","param":"None","code":401}}%
-```
-
-
-
-
-
-```shell
-curl --location 'http://localhost:4000/chat/completions' \
- --header 'Authorization: Bearer sk-5fmYeaUEbAMpwBNT-QpxyA' \
- --header 'Content-Type: application/json' \
- --data '{
- "model": "gpt-3.5-turbo",
- "user": "gm",
- "messages": [
- {
- "role": "user",
- "content": "hi"
- }
- ],
- "metadata": {"generation_name": "prod-app"}
-}'
-```
-
-Expected Response
-
-```shell
-{"id":"chatcmpl-9XALnHqkCBMBKrOx7Abg0hURHqYtY","choices":[{"finish_reason":"stop","index":0,"message":{"content":"Hello! How can I assist you today?","role":"assistant"}}],"created":1717691639,"model":"gpt-3.5-turbo-0125","object":"chat.completion","system_fingerprint":null,"usage":{"completion_tokens":9,"prompt_tokens":8,"total_tokens":17}}%
-```
-
-
-
-
-
-
-## Control available public, private routes
-
-**Restrict certain endpoints of proxy**
-
-:::info
-
-❓ Use this when you want to:
-- make an existing private route -> public
-- set certain routes as admin_only routes
-
-:::
-
-#### Usage - Define public, admin only routes
-
-**Step 1** - Set on config.yaml
-
-
-| Route Type | Optional | Requires Virtual Key Auth | Admin Can Access | All Roles Can Access | Description |
-|------------|----------|---------------------------|-------------------|----------------------|-------------|
-| `public_routes` | ✅ | ❌ | ✅ | ✅ | Routes that can be accessed without any authentication |
-| `admin_only_routes` | ✅ | ✅ | ✅ | ❌ | Routes that can only be accessed by [Proxy Admin](./self_serve#available-roles) |
-| `allowed_routes` | ✅ | ✅ | ✅ | ✅ | Routes are exposed on the proxy. If not set then all routes exposed. |
-
-`LiteLLMRoutes.public_routes` is an ENUM corresponding to the default public routes on LiteLLM. [You can see this here](https://github.com/BerriAI/litellm/blob/main/litellm/proxy/_types.py)
-
-```yaml
-general_settings:
- master_key: sk-1234
- public_routes: ["LiteLLMRoutes.public_routes", "/spend/calculate"] # routes that can be accessed without any auth
- admin_only_routes: ["/key/generate"] # Optional - routes that can only be accessed by Proxy Admin
- allowed_routes: ["/chat/completions", "/spend/calculate", "LiteLLMRoutes.public_routes"] # Optional - routes that can be accessed by anyone after Authentication
-```
-
-**Step 2** - start proxy
-
-```shell
-litellm --config config.yaml
-```
-
-**Step 3** - Test it
-
-
-
-
-
-```shell
-curl --request POST \
- --url 'http://localhost:4000/spend/calculate' \
- --header 'Content-Type: application/json' \
- --data '{
- "model": "gpt-4",
- "messages": [{"role": "user", "content": "Hey, how'\''s it going?"}]
- }'
-```
-
-🎉 Expect this endpoint to work without an `Authorization / Bearer Token`
-
-
-
-
-
-
-**Successful Request**
-
-```shell
-curl --location 'http://0.0.0.0:4000/key/generate' \
---header 'Authorization: Bearer ' \
---header 'Content-Type: application/json' \
---data '{}'
-```
-
-
-**Un-successfull Request**
-
-```shell
- curl --location 'http://0.0.0.0:4000/key/generate' \
---header 'Authorization: Bearer ' \
---header 'Content-Type: application/json' \
---data '{"user_role": "internal_user"}'
-```
-
-**Expected Response**
-
-```json
-{
- "error": {
- "message": "user not allowed to access this route. Route=/key/generate is an admin only route",
- "type": "auth_error",
- "param": "None",
- "code": "403"
- }
-}
-```
-
-
-
-
-
-
-
-**Successful Request**
-
-```shell
-curl http://localhost:4000/chat/completions \
--H "Content-Type: application/json" \
--H "Authorization: Bearer sk-1234" \
--d '{
-"model": "fake-openai-endpoint",
-"messages": [
- {"role": "user", "content": "Hello, Claude"}
-]
-}'
-```
-
-
-**Un-successfull Request**
-
-```shell
-curl --location 'http://0.0.0.0:4000/embeddings' \
---header 'Content-Type: application/json' \
--H "Authorization: Bearer sk-1234" \
---data ' {
-"model": "text-embedding-ada-002",
-"input": ["write a litellm poem"]
-}'
-```
-
-**Expected Response**
-
-```json
-{
- "error": {
- "message": "Route /embeddings not allowed",
- "type": "auth_error",
- "param": "None",
- "code": "403"
- }
-}
-```
-
-
-
-
-
-
-
-
-
-
## Guardrails - Secret Detection/Redaction
❓ Use this to REDACT API Keys, Secrets sent in requests to an LLM.
diff --git a/enterprise/proxy/enterprise_routes.py b/enterprise/proxy/enterprise_routes.py
new file mode 100644
index 0000000000..1d7a39b0ae
--- /dev/null
+++ b/enterprise/proxy/enterprise_routes.py
@@ -0,0 +1,17 @@
+from fastapi import APIRouter
+from fastapi.responses import Response
+from .utils import _should_block_robots
+
+router = APIRouter()
+
+
+@router.get("/robots.txt")
+async def get_robots():
+ """
+ Block all web crawlers from indexing the proxy server endpoints
+ This is useful for ensuring that the API endpoints aren't indexed by search engines
+ """
+ if _should_block_robots():
+ return Response(content="User-agent: *\nDisallow: /", media_type="text/plain")
+ else:
+ return Response(status_code=404)
\ No newline at end of file
diff --git a/enterprise/proxy/readme.md b/enterprise/proxy/readme.md
new file mode 100644
index 0000000000..9ec611fa83
--- /dev/null
+++ b/enterprise/proxy/readme.md
@@ -0,0 +1,6 @@
+# LiteLLM Proxy Enterprise Features - Readme
+
+## Overview
+
+This directory contains enterprise features used on the LiteLLM proxy.
+
diff --git a/enterprise/proxy/utils.py b/enterprise/proxy/utils.py
new file mode 100644
index 0000000000..c3396964ee
--- /dev/null
+++ b/enterprise/proxy/utils.py
@@ -0,0 +1,27 @@
+from typing import Union, Optional
+from litellm.secret_managers.main import str_to_bool
+
+
+def _should_block_robots():
+ """
+ Returns True if the robots.txt file should block web crawlers
+
+ Controlled by
+
+ ```yaml
+ general_settings:
+ block_robots: true
+ ```
+ """
+ from litellm.proxy.proxy_server import general_settings, premium_user, CommonProxyErrors
+ _block_robots: Union[bool, str] = general_settings.get("block_robots", False)
+ block_robots: Optional[bool] = None
+ if isinstance(_block_robots, bool):
+ block_robots = _block_robots
+ elif isinstance(_block_robots, str):
+ block_robots = str_to_bool(_block_robots)
+ if block_robots is True:
+ if premium_user is not True:
+ raise ValueError(f"Blocking web crawlers is an enterprise feature. {CommonProxyErrors.not_premium_user.value}")
+ return True
+ return False
diff --git a/litellm/proxy/proxy_config.yaml b/litellm/proxy/proxy_config.yaml
index 8ff36b91f5..9e56d69b90 100644
--- a/litellm/proxy/proxy_config.yaml
+++ b/litellm/proxy/proxy_config.yaml
@@ -8,3 +8,4 @@ model_list:
api_version: os.environ/AZURE_RESPONSES_OPENAI_API_VERSION
general_settings:
store_prompts_in_spend_logs: true
+ block_robots: true
diff --git a/litellm/proxy/proxy_server.py b/litellm/proxy/proxy_server.py
index 966d79c607..7feb7f5092 100644
--- a/litellm/proxy/proxy_server.py
+++ b/litellm/proxy/proxy_server.py
@@ -378,13 +378,16 @@ from fastapi.security.api_key import APIKeyHeader
from fastapi.staticfiles import StaticFiles
# import enterprise folder
+enterprise_router = APIRouter()
try:
# when using litellm cli
import litellm.proxy.enterprise as enterprise
+ from enterprise.proxy.enterprise_routes import router as enterprise_router
except Exception:
# when using litellm docker image
try:
import enterprise # type: ignore
+ from enterprise.proxy.enterprise_routes import router as enterprise_router
except Exception:
pass
@@ -810,9 +813,9 @@ model_max_budget_limiter = _PROXY_VirtualKeyModelMaxBudgetLimiter(
dual_cache=user_api_key_cache
)
litellm.logging_callback_manager.add_litellm_callback(model_max_budget_limiter)
-redis_usage_cache: Optional[
- RedisCache
-] = None # redis cache used for tracking spend, tpm/rpm limits
+redis_usage_cache: Optional[RedisCache] = (
+ None # redis cache used for tracking spend, tpm/rpm limits
+)
user_custom_auth = None
user_custom_key_generate = None
user_custom_sso = None
@@ -1138,9 +1141,9 @@ async def update_cache( # noqa: PLR0915
_id = "team_id:{}".format(team_id)
try:
# Fetch the existing cost for the given user
- existing_spend_obj: Optional[
- LiteLLM_TeamTable
- ] = await user_api_key_cache.async_get_cache(key=_id)
+ existing_spend_obj: Optional[LiteLLM_TeamTable] = (
+ await user_api_key_cache.async_get_cache(key=_id)
+ )
if existing_spend_obj is None:
# do nothing if team not in api key cache
return
@@ -2812,9 +2815,9 @@ async def initialize( # noqa: PLR0915
user_api_base = api_base
dynamic_config[user_model]["api_base"] = api_base
if api_version:
- os.environ[
- "AZURE_API_VERSION"
- ] = api_version # set this for azure - litellm can read this from the env
+ os.environ["AZURE_API_VERSION"] = (
+ api_version # set this for azure - litellm can read this from the env
+ )
if max_tokens: # model-specific param
dynamic_config[user_model]["max_tokens"] = max_tokens
if temperature: # model-specific param
@@ -7818,9 +7821,9 @@ async def get_config_list(
hasattr(sub_field_info, "description")
and sub_field_info.description is not None
):
- nested_fields[
- idx
- ].field_description = sub_field_info.description
+ nested_fields[idx].field_description = (
+ sub_field_info.description
+ )
idx += 1
_stored_in_db = None
@@ -8252,3 +8255,4 @@ app.include_router(team_callback_router)
app.include_router(budget_management_router)
app.include_router(model_management_router)
app.include_router(tag_management_router)
+app.include_router(enterprise_router)
diff --git a/tests/litellm/enterprise/test_enterprise_routes.py b/tests/litellm/enterprise/test_enterprise_routes.py
new file mode 100644
index 0000000000..61cea9c4c2
--- /dev/null
+++ b/tests/litellm/enterprise/test_enterprise_routes.py
@@ -0,0 +1,40 @@
+import json
+import os
+import sys
+import unittest.mock as mock
+
+import pytest
+from fastapi.testclient import TestClient
+
+sys.path.insert(
+ 0, os.path.abspath("../../..")
+) # Adds the parent directory to the system path
+
+from enterprise.proxy.enterprise_routes import router
+
+
+@pytest.fixture
+def client():
+ return TestClient(router)
+
+
+def test_robots_when_blocked(client):
+ """Test get_robots returns block instructions when _should_block_robots returns True"""
+ with mock.patch(
+ "enterprise.proxy.enterprise_routes._should_block_robots", return_value=True
+ ):
+ response = client.get("/robots.txt")
+ print("got response", response)
+ print("got response text", response.text)
+ print("got response headers", response.headers)
+ assert response.status_code == 200
+ assert response.text == "User-agent: *\nDisallow: /"
+
+
+def test_robots_when_not_blocked(client):
+ """Test get_robots returns 404 when _should_block_robots returns False"""
+ with mock.patch(
+ "enterprise.proxy.enterprise_routes._should_block_robots", return_value=False
+ ):
+ response = client.get("/robots.txt")
+ assert response.status_code == 404