mirror of
https://github.com/tiennm99/litellm.git
synced 2026-08-23 18:25:22 +00:00
[Feat Security] - Allow blocking web crawlers (#10420)
* security add robots.txt settings to security * block web crawlers * test_enterprise_routes.py * docs proxy enterprise
This commit is contained in:
Ishaan Jaff
@@ -1049,3 +1049,25 @@ export DATABASE_SCHEMA="schema-name" # skip to use the default "public" schema
|
||||
litellm --config /path/to/config.yaml --iam_token_db_auth
|
||||
```
|
||||
|
||||
### ✨ Blocking web crawlers
|
||||
|
||||
Note: This is an [enterprise only feature](https://docs.litellm.ai/docs/enterprise).
|
||||
|
||||
To block web crawlers from indexing the proxy server endpoints, set the `block_robots` setting to `true` in your `litellm_config.yaml` file.
|
||||
|
||||
```yaml showLineNumbers title="litellm_config.yaml"
|
||||
general_settings:
|
||||
block_robots: true
|
||||
```
|
||||
|
||||
#### How it works
|
||||
|
||||
When this is enabled, the `/robots.txt` endpoint will return a 200 status code with the following content:
|
||||
|
||||
```shell showLineNumbers title="robots.txt"
|
||||
User-agent: *
|
||||
Disallow: /
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -43,7 +43,9 @@ Features:
|
||||
- ✅ [Public Model Hub](#public-model-hub)
|
||||
- ✅ [Custom Email Branding](./email.md#customizing-email-branding)
|
||||
|
||||
## Audit Logs
|
||||
## Security
|
||||
|
||||
### Audit Logs
|
||||
|
||||
Store Audit logs for **Create, Update Delete Operations** done on `Teams` and `Virtual Keys`
|
||||
|
||||
@@ -95,7 +97,295 @@ curl --location 'http://0.0.0.0:4000/team/new' \
|
||||
```
|
||||
|
||||
|
||||
## Tracking Spend for Custom Tags
|
||||
### Blocking web crawlers
|
||||
|
||||
To block web crawlers from indexing the proxy server endpoints, set the `block_robots` setting to `true` in your `litellm_config.yaml` file.
|
||||
|
||||
```yaml showLineNumbers title="litellm_config.yaml"
|
||||
general_settings:
|
||||
block_robots: true
|
||||
```
|
||||
|
||||
#### How it works
|
||||
|
||||
When this is enabled, the `/robots.txt` endpoint will return a 200 status code with the following content:
|
||||
|
||||
```shell showLineNumbers title="robots.txt"
|
||||
User-agent: *
|
||||
Disallow: /
|
||||
```
|
||||
|
||||
|
||||
|
||||
### Required Params for LLM Requests
|
||||
Use this when you want to enforce all requests to include certain params. Example you need all requests to include the `user` and `["metadata]["generation_name"]` params.
|
||||
|
||||
|
||||
<Tabs>
|
||||
|
||||
<TabItem value="config" label="Set on Config">
|
||||
|
||||
**Step 1** Define all Params you want to enforce on config.yaml
|
||||
|
||||
This means `["user"]` and `["metadata]["generation_name"]` are required in all LLM Requests to LiteLLM
|
||||
|
||||
```yaml
|
||||
general_settings:
|
||||
master_key: sk-1234
|
||||
enforced_params:
|
||||
- user
|
||||
- metadata.generation_name
|
||||
```
|
||||
</TabItem>
|
||||
|
||||
<TabItem value="key" label="Set on Key">
|
||||
|
||||
```bash
|
||||
curl -L -X POST 'http://0.0.0.0:4000/key/generate' \
|
||||
-H 'Authorization: Bearer sk-1234' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"enforced_params": ["user", "metadata.generation_name"]
|
||||
}'
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
**Step 2 Verify if this works**
|
||||
|
||||
<Tabs>
|
||||
|
||||
<TabItem value="bad" label="Invalid Request (No `user` passed)">
|
||||
|
||||
```shell
|
||||
curl --location 'http://localhost:4000/chat/completions' \
|
||||
--header 'Authorization: Bearer sk-5fmYeaUEbAMpwBNT-QpxyA' \
|
||||
--header 'Content-Type: application/json' \
|
||||
--data '{
|
||||
"model": "gpt-3.5-turbo",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "hi"
|
||||
}
|
||||
]
|
||||
}'
|
||||
```
|
||||
|
||||
Expected Response
|
||||
|
||||
```shell
|
||||
{"error":{"message":"Authentication Error, BadRequest please pass param=user in request body. This is a required param","type":"auth_error","param":"None","code":401}}%
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
|
||||
<TabItem value="bad2" label="Invalid Request (No `metadata` passed)">
|
||||
|
||||
```shell
|
||||
curl --location 'http://localhost:4000/chat/completions' \
|
||||
--header 'Authorization: Bearer sk-5fmYeaUEbAMpwBNT-QpxyA' \
|
||||
--header 'Content-Type: application/json' \
|
||||
--data '{
|
||||
"model": "gpt-3.5-turbo",
|
||||
"user": "gm",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "hi"
|
||||
}
|
||||
],
|
||||
"metadata": {}
|
||||
}'
|
||||
```
|
||||
|
||||
Expected Response
|
||||
|
||||
```shell
|
||||
{"error":{"message":"Authentication Error, BadRequest please pass param=[metadata][generation_name] in request body. This is a required param","type":"auth_error","param":"None","code":401}}%
|
||||
```
|
||||
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="good" label="Valid Request">
|
||||
|
||||
```shell
|
||||
curl --location 'http://localhost:4000/chat/completions' \
|
||||
--header 'Authorization: Bearer sk-5fmYeaUEbAMpwBNT-QpxyA' \
|
||||
--header 'Content-Type: application/json' \
|
||||
--data '{
|
||||
"model": "gpt-3.5-turbo",
|
||||
"user": "gm",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "hi"
|
||||
}
|
||||
],
|
||||
"metadata": {"generation_name": "prod-app"}
|
||||
}'
|
||||
```
|
||||
|
||||
Expected Response
|
||||
|
||||
```shell
|
||||
{"id":"chatcmpl-9XALnHqkCBMBKrOx7Abg0hURHqYtY","choices":[{"finish_reason":"stop","index":0,"message":{"content":"Hello! How can I assist you today?","role":"assistant"}}],"created":1717691639,"model":"gpt-3.5-turbo-0125","object":"chat.completion","system_fingerprint":null,"usage":{"completion_tokens":9,"prompt_tokens":8,"total_tokens":17}}%
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
|
||||
|
||||
### Control available public, private routes
|
||||
|
||||
**Restrict certain endpoints of proxy**
|
||||
|
||||
:::info
|
||||
|
||||
❓ Use this when you want to:
|
||||
- make an existing private route -> public
|
||||
- set certain routes as admin_only routes
|
||||
|
||||
:::
|
||||
|
||||
#### Usage - Define public, admin only routes
|
||||
|
||||
**Step 1** - Set on config.yaml
|
||||
|
||||
|
||||
| Route Type | Optional | Requires Virtual Key Auth | Admin Can Access | All Roles Can Access | Description |
|
||||
|------------|----------|---------------------------|-------------------|----------------------|-------------|
|
||||
| `public_routes` | ✅ | ❌ | ✅ | ✅ | Routes that can be accessed without any authentication |
|
||||
| `admin_only_routes` | ✅ | ✅ | ✅ | ❌ | Routes that can only be accessed by [Proxy Admin](./self_serve#available-roles) |
|
||||
| `allowed_routes` | ✅ | ✅ | ✅ | ✅ | Routes are exposed on the proxy. If not set then all routes exposed. |
|
||||
|
||||
`LiteLLMRoutes.public_routes` is an ENUM corresponding to the default public routes on LiteLLM. [You can see this here](https://github.com/BerriAI/litellm/blob/main/litellm/proxy/_types.py)
|
||||
|
||||
```yaml
|
||||
general_settings:
|
||||
master_key: sk-1234
|
||||
public_routes: ["LiteLLMRoutes.public_routes", "/spend/calculate"] # routes that can be accessed without any auth
|
||||
admin_only_routes: ["/key/generate"] # Optional - routes that can only be accessed by Proxy Admin
|
||||
allowed_routes: ["/chat/completions", "/spend/calculate", "LiteLLMRoutes.public_routes"] # Optional - routes that can be accessed by anyone after Authentication
|
||||
```
|
||||
|
||||
**Step 2** - start proxy
|
||||
|
||||
```shell
|
||||
litellm --config config.yaml
|
||||
```
|
||||
|
||||
**Step 3** - Test it
|
||||
|
||||
<Tabs>
|
||||
|
||||
<TabItem value="public" label="Test `public_routes`">
|
||||
|
||||
```shell
|
||||
curl --request POST \
|
||||
--url 'http://localhost:4000/spend/calculate' \
|
||||
--header 'Content-Type: application/json' \
|
||||
--data '{
|
||||
"model": "gpt-4",
|
||||
"messages": [{"role": "user", "content": "Hey, how'\''s it going?"}]
|
||||
}'
|
||||
```
|
||||
|
||||
🎉 Expect this endpoint to work without an `Authorization / Bearer Token`
|
||||
|
||||
</TabItem>
|
||||
|
||||
<TabItem value="admin_only_routes" label="Test `admin_only_routes`">
|
||||
|
||||
|
||||
**Successful Request**
|
||||
|
||||
```shell
|
||||
curl --location 'http://0.0.0.0:4000/key/generate' \
|
||||
--header 'Authorization: Bearer <your-master-key>' \
|
||||
--header 'Content-Type: application/json' \
|
||||
--data '{}'
|
||||
```
|
||||
|
||||
|
||||
**Un-successfull Request**
|
||||
|
||||
```shell
|
||||
curl --location 'http://0.0.0.0:4000/key/generate' \
|
||||
--header 'Authorization: Bearer <virtual-key-from-non-admin>' \
|
||||
--header 'Content-Type: application/json' \
|
||||
--data '{"user_role": "internal_user"}'
|
||||
```
|
||||
|
||||
**Expected Response**
|
||||
|
||||
```json
|
||||
{
|
||||
"error": {
|
||||
"message": "user not allowed to access this route. Route=/key/generate is an admin only route",
|
||||
"type": "auth_error",
|
||||
"param": "None",
|
||||
"code": "403"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
</TabItem>
|
||||
|
||||
<TabItem value="allowed_routes" label="Test `allowed_routes`">
|
||||
|
||||
|
||||
**Successful Request**
|
||||
|
||||
```shell
|
||||
curl http://localhost:4000/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer sk-1234" \
|
||||
-d '{
|
||||
"model": "fake-openai-endpoint",
|
||||
"messages": [
|
||||
{"role": "user", "content": "Hello, Claude"}
|
||||
]
|
||||
}'
|
||||
```
|
||||
|
||||
|
||||
**Un-successfull Request**
|
||||
|
||||
```shell
|
||||
curl --location 'http://0.0.0.0:4000/embeddings' \
|
||||
--header 'Content-Type: application/json' \
|
||||
-H "Authorization: Bearer sk-1234" \
|
||||
--data ' {
|
||||
"model": "text-embedding-ada-002",
|
||||
"input": ["write a litellm poem"]
|
||||
}'
|
||||
```
|
||||
|
||||
**Expected Response**
|
||||
|
||||
```json
|
||||
{
|
||||
"error": {
|
||||
"message": "Route /embeddings not allowed",
|
||||
"type": "auth_error",
|
||||
"param": "None",
|
||||
"code": "403"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
</TabItem>
|
||||
|
||||
</Tabs>
|
||||
|
||||
## Spend Tracking
|
||||
|
||||
### Custom Tags
|
||||
|
||||
Requirements:
|
||||
|
||||
@@ -294,7 +584,7 @@ curl -X GET "http://0.0.0.0:4000/spend/tags" \
|
||||
```
|
||||
|
||||
|
||||
## Tracking Spend with custom metadata
|
||||
### Tracking Spend with custom metadata
|
||||
|
||||
Requirements:
|
||||
|
||||
@@ -506,278 +796,6 @@ curl -X GET "http://0.0.0.0:4000/spend/logs?request_id=<your-call-id" \ # e.g.:
|
||||
]
|
||||
```
|
||||
|
||||
|
||||
|
||||
## Enforce Required Params for LLM Requests
|
||||
Use this when you want to enforce all requests to include certain params. Example you need all requests to include the `user` and `["metadata]["generation_name"]` params.
|
||||
|
||||
|
||||
<Tabs>
|
||||
|
||||
<TabItem value="config" label="Set on Config">
|
||||
|
||||
**Step 1** Define all Params you want to enforce on config.yaml
|
||||
|
||||
This means `["user"]` and `["metadata]["generation_name"]` are required in all LLM Requests to LiteLLM
|
||||
|
||||
```yaml
|
||||
general_settings:
|
||||
master_key: sk-1234
|
||||
enforced_params:
|
||||
- user
|
||||
- metadata.generation_name
|
||||
```
|
||||
</TabItem>
|
||||
|
||||
<TabItem value="key" label="Set on Key">
|
||||
|
||||
```bash
|
||||
curl -L -X POST 'http://0.0.0.0:4000/key/generate' \
|
||||
-H 'Authorization: Bearer sk-1234' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"enforced_params": ["user", "metadata.generation_name"]
|
||||
}'
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
**Step 2 Verify if this works**
|
||||
|
||||
<Tabs>
|
||||
|
||||
<TabItem value="bad" label="Invalid Request (No `user` passed)">
|
||||
|
||||
```shell
|
||||
curl --location 'http://localhost:4000/chat/completions' \
|
||||
--header 'Authorization: Bearer sk-5fmYeaUEbAMpwBNT-QpxyA' \
|
||||
--header 'Content-Type: application/json' \
|
||||
--data '{
|
||||
"model": "gpt-3.5-turbo",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "hi"
|
||||
}
|
||||
]
|
||||
}'
|
||||
```
|
||||
|
||||
Expected Response
|
||||
|
||||
```shell
|
||||
{"error":{"message":"Authentication Error, BadRequest please pass param=user in request body. This is a required param","type":"auth_error","param":"None","code":401}}%
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
|
||||
<TabItem value="bad2" label="Invalid Request (No `metadata` passed)">
|
||||
|
||||
```shell
|
||||
curl --location 'http://localhost:4000/chat/completions' \
|
||||
--header 'Authorization: Bearer sk-5fmYeaUEbAMpwBNT-QpxyA' \
|
||||
--header 'Content-Type: application/json' \
|
||||
--data '{
|
||||
"model": "gpt-3.5-turbo",
|
||||
"user": "gm",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "hi"
|
||||
}
|
||||
],
|
||||
"metadata": {}
|
||||
}'
|
||||
```
|
||||
|
||||
Expected Response
|
||||
|
||||
```shell
|
||||
{"error":{"message":"Authentication Error, BadRequest please pass param=[metadata][generation_name] in request body. This is a required param","type":"auth_error","param":"None","code":401}}%
|
||||
```
|
||||
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="good" label="Valid Request">
|
||||
|
||||
```shell
|
||||
curl --location 'http://localhost:4000/chat/completions' \
|
||||
--header 'Authorization: Bearer sk-5fmYeaUEbAMpwBNT-QpxyA' \
|
||||
--header 'Content-Type: application/json' \
|
||||
--data '{
|
||||
"model": "gpt-3.5-turbo",
|
||||
"user": "gm",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "hi"
|
||||
}
|
||||
],
|
||||
"metadata": {"generation_name": "prod-app"}
|
||||
}'
|
||||
```
|
||||
|
||||
Expected Response
|
||||
|
||||
```shell
|
||||
{"id":"chatcmpl-9XALnHqkCBMBKrOx7Abg0hURHqYtY","choices":[{"finish_reason":"stop","index":0,"message":{"content":"Hello! How can I assist you today?","role":"assistant"}}],"created":1717691639,"model":"gpt-3.5-turbo-0125","object":"chat.completion","system_fingerprint":null,"usage":{"completion_tokens":9,"prompt_tokens":8,"total_tokens":17}}%
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
|
||||
|
||||
## Control available public, private routes
|
||||
|
||||
**Restrict certain endpoints of proxy**
|
||||
|
||||
:::info
|
||||
|
||||
❓ Use this when you want to:
|
||||
- make an existing private route -> public
|
||||
- set certain routes as admin_only routes
|
||||
|
||||
:::
|
||||
|
||||
#### Usage - Define public, admin only routes
|
||||
|
||||
**Step 1** - Set on config.yaml
|
||||
|
||||
|
||||
| Route Type | Optional | Requires Virtual Key Auth | Admin Can Access | All Roles Can Access | Description |
|
||||
|------------|----------|---------------------------|-------------------|----------------------|-------------|
|
||||
| `public_routes` | ✅ | ❌ | ✅ | ✅ | Routes that can be accessed without any authentication |
|
||||
| `admin_only_routes` | ✅ | ✅ | ✅ | ❌ | Routes that can only be accessed by [Proxy Admin](./self_serve#available-roles) |
|
||||
| `allowed_routes` | ✅ | ✅ | ✅ | ✅ | Routes are exposed on the proxy. If not set then all routes exposed. |
|
||||
|
||||
`LiteLLMRoutes.public_routes` is an ENUM corresponding to the default public routes on LiteLLM. [You can see this here](https://github.com/BerriAI/litellm/blob/main/litellm/proxy/_types.py)
|
||||
|
||||
```yaml
|
||||
general_settings:
|
||||
master_key: sk-1234
|
||||
public_routes: ["LiteLLMRoutes.public_routes", "/spend/calculate"] # routes that can be accessed without any auth
|
||||
admin_only_routes: ["/key/generate"] # Optional - routes that can only be accessed by Proxy Admin
|
||||
allowed_routes: ["/chat/completions", "/spend/calculate", "LiteLLMRoutes.public_routes"] # Optional - routes that can be accessed by anyone after Authentication
|
||||
```
|
||||
|
||||
**Step 2** - start proxy
|
||||
|
||||
```shell
|
||||
litellm --config config.yaml
|
||||
```
|
||||
|
||||
**Step 3** - Test it
|
||||
|
||||
<Tabs>
|
||||
|
||||
<TabItem value="public" label="Test `public_routes`">
|
||||
|
||||
```shell
|
||||
curl --request POST \
|
||||
--url 'http://localhost:4000/spend/calculate' \
|
||||
--header 'Content-Type: application/json' \
|
||||
--data '{
|
||||
"model": "gpt-4",
|
||||
"messages": [{"role": "user", "content": "Hey, how'\''s it going?"}]
|
||||
}'
|
||||
```
|
||||
|
||||
🎉 Expect this endpoint to work without an `Authorization / Bearer Token`
|
||||
|
||||
</TabItem>
|
||||
|
||||
<TabItem value="admin_only_routes" label="Test `admin_only_routes`">
|
||||
|
||||
|
||||
**Successful Request**
|
||||
|
||||
```shell
|
||||
curl --location 'http://0.0.0.0:4000/key/generate' \
|
||||
--header 'Authorization: Bearer <your-master-key>' \
|
||||
--header 'Content-Type: application/json' \
|
||||
--data '{}'
|
||||
```
|
||||
|
||||
|
||||
**Un-successfull Request**
|
||||
|
||||
```shell
|
||||
curl --location 'http://0.0.0.0:4000/key/generate' \
|
||||
--header 'Authorization: Bearer <virtual-key-from-non-admin>' \
|
||||
--header 'Content-Type: application/json' \
|
||||
--data '{"user_role": "internal_user"}'
|
||||
```
|
||||
|
||||
**Expected Response**
|
||||
|
||||
```json
|
||||
{
|
||||
"error": {
|
||||
"message": "user not allowed to access this route. Route=/key/generate is an admin only route",
|
||||
"type": "auth_error",
|
||||
"param": "None",
|
||||
"code": "403"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
</TabItem>
|
||||
|
||||
<TabItem value="allowed_routes" label="Test `allowed_routes`">
|
||||
|
||||
|
||||
**Successful Request**
|
||||
|
||||
```shell
|
||||
curl http://localhost:4000/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer sk-1234" \
|
||||
-d '{
|
||||
"model": "fake-openai-endpoint",
|
||||
"messages": [
|
||||
{"role": "user", "content": "Hello, Claude"}
|
||||
]
|
||||
}'
|
||||
```
|
||||
|
||||
|
||||
**Un-successfull Request**
|
||||
|
||||
```shell
|
||||
curl --location 'http://0.0.0.0:4000/embeddings' \
|
||||
--header 'Content-Type: application/json' \
|
||||
-H "Authorization: Bearer sk-1234" \
|
||||
--data ' {
|
||||
"model": "text-embedding-ada-002",
|
||||
"input": ["write a litellm poem"]
|
||||
}'
|
||||
```
|
||||
|
||||
**Expected Response**
|
||||
|
||||
```json
|
||||
{
|
||||
"error": {
|
||||
"message": "Route /embeddings not allowed",
|
||||
"type": "auth_error",
|
||||
"param": "None",
|
||||
"code": "403"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
</TabItem>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</Tabs>
|
||||
|
||||
## Guardrails - Secret Detection/Redaction
|
||||
❓ Use this to REDACT API Keys, Secrets sent in requests to an LLM.
|
||||
|
||||
|
||||
@@ -0,0 +1,17 @@
|
||||
from fastapi import APIRouter
|
||||
from fastapi.responses import Response
|
||||
from .utils import _should_block_robots
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
@router.get("/robots.txt")
|
||||
async def get_robots():
|
||||
"""
|
||||
Block all web crawlers from indexing the proxy server endpoints
|
||||
This is useful for ensuring that the API endpoints aren't indexed by search engines
|
||||
"""
|
||||
if _should_block_robots():
|
||||
return Response(content="User-agent: *\nDisallow: /", media_type="text/plain")
|
||||
else:
|
||||
return Response(status_code=404)
|
||||
@@ -0,0 +1,6 @@
|
||||
# LiteLLM Proxy Enterprise Features - Readme
|
||||
|
||||
## Overview
|
||||
|
||||
This directory contains enterprise features used on the LiteLLM proxy.
|
||||
|
||||
@@ -0,0 +1,27 @@
|
||||
from typing import Union, Optional
|
||||
from litellm.secret_managers.main import str_to_bool
|
||||
|
||||
|
||||
def _should_block_robots():
|
||||
"""
|
||||
Returns True if the robots.txt file should block web crawlers
|
||||
|
||||
Controlled by
|
||||
|
||||
```yaml
|
||||
general_settings:
|
||||
block_robots: true
|
||||
```
|
||||
"""
|
||||
from litellm.proxy.proxy_server import general_settings, premium_user, CommonProxyErrors
|
||||
_block_robots: Union[bool, str] = general_settings.get("block_robots", False)
|
||||
block_robots: Optional[bool] = None
|
||||
if isinstance(_block_robots, bool):
|
||||
block_robots = _block_robots
|
||||
elif isinstance(_block_robots, str):
|
||||
block_robots = str_to_bool(_block_robots)
|
||||
if block_robots is True:
|
||||
if premium_user is not True:
|
||||
raise ValueError(f"Blocking web crawlers is an enterprise feature. {CommonProxyErrors.not_premium_user.value}")
|
||||
return True
|
||||
return False
|
||||
@@ -8,3 +8,4 @@ model_list:
|
||||
api_version: os.environ/AZURE_RESPONSES_OPENAI_API_VERSION
|
||||
general_settings:
|
||||
store_prompts_in_spend_logs: true
|
||||
block_robots: true
|
||||
|
||||
@@ -378,13 +378,16 @@ from fastapi.security.api_key import APIKeyHeader
|
||||
from fastapi.staticfiles import StaticFiles
|
||||
|
||||
# import enterprise folder
|
||||
enterprise_router = APIRouter()
|
||||
try:
|
||||
# when using litellm cli
|
||||
import litellm.proxy.enterprise as enterprise
|
||||
from enterprise.proxy.enterprise_routes import router as enterprise_router
|
||||
except Exception:
|
||||
# when using litellm docker image
|
||||
try:
|
||||
import enterprise # type: ignore
|
||||
from enterprise.proxy.enterprise_routes import router as enterprise_router
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
@@ -810,9 +813,9 @@ model_max_budget_limiter = _PROXY_VirtualKeyModelMaxBudgetLimiter(
|
||||
dual_cache=user_api_key_cache
|
||||
)
|
||||
litellm.logging_callback_manager.add_litellm_callback(model_max_budget_limiter)
|
||||
redis_usage_cache: Optional[
|
||||
RedisCache
|
||||
] = None # redis cache used for tracking spend, tpm/rpm limits
|
||||
redis_usage_cache: Optional[RedisCache] = (
|
||||
None # redis cache used for tracking spend, tpm/rpm limits
|
||||
)
|
||||
user_custom_auth = None
|
||||
user_custom_key_generate = None
|
||||
user_custom_sso = None
|
||||
@@ -1138,9 +1141,9 @@ async def update_cache( # noqa: PLR0915
|
||||
_id = "team_id:{}".format(team_id)
|
||||
try:
|
||||
# Fetch the existing cost for the given user
|
||||
existing_spend_obj: Optional[
|
||||
LiteLLM_TeamTable
|
||||
] = await user_api_key_cache.async_get_cache(key=_id)
|
||||
existing_spend_obj: Optional[LiteLLM_TeamTable] = (
|
||||
await user_api_key_cache.async_get_cache(key=_id)
|
||||
)
|
||||
if existing_spend_obj is None:
|
||||
# do nothing if team not in api key cache
|
||||
return
|
||||
@@ -2812,9 +2815,9 @@ async def initialize( # noqa: PLR0915
|
||||
user_api_base = api_base
|
||||
dynamic_config[user_model]["api_base"] = api_base
|
||||
if api_version:
|
||||
os.environ[
|
||||
"AZURE_API_VERSION"
|
||||
] = api_version # set this for azure - litellm can read this from the env
|
||||
os.environ["AZURE_API_VERSION"] = (
|
||||
api_version # set this for azure - litellm can read this from the env
|
||||
)
|
||||
if max_tokens: # model-specific param
|
||||
dynamic_config[user_model]["max_tokens"] = max_tokens
|
||||
if temperature: # model-specific param
|
||||
@@ -7818,9 +7821,9 @@ async def get_config_list(
|
||||
hasattr(sub_field_info, "description")
|
||||
and sub_field_info.description is not None
|
||||
):
|
||||
nested_fields[
|
||||
idx
|
||||
].field_description = sub_field_info.description
|
||||
nested_fields[idx].field_description = (
|
||||
sub_field_info.description
|
||||
)
|
||||
idx += 1
|
||||
|
||||
_stored_in_db = None
|
||||
@@ -8252,3 +8255,4 @@ app.include_router(team_callback_router)
|
||||
app.include_router(budget_management_router)
|
||||
app.include_router(model_management_router)
|
||||
app.include_router(tag_management_router)
|
||||
app.include_router(enterprise_router)
|
||||
|
||||
@@ -0,0 +1,40 @@
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import unittest.mock as mock
|
||||
|
||||
import pytest
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
sys.path.insert(
|
||||
0, os.path.abspath("../../..")
|
||||
) # Adds the parent directory to the system path
|
||||
|
||||
from enterprise.proxy.enterprise_routes import router
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def client():
|
||||
return TestClient(router)
|
||||
|
||||
|
||||
def test_robots_when_blocked(client):
|
||||
"""Test get_robots returns block instructions when _should_block_robots returns True"""
|
||||
with mock.patch(
|
||||
"enterprise.proxy.enterprise_routes._should_block_robots", return_value=True
|
||||
):
|
||||
response = client.get("/robots.txt")
|
||||
print("got response", response)
|
||||
print("got response text", response.text)
|
||||
print("got response headers", response.headers)
|
||||
assert response.status_code == 200
|
||||
assert response.text == "User-agent: *\nDisallow: /"
|
||||
|
||||
|
||||
def test_robots_when_not_blocked(client):
|
||||
"""Test get_robots returns 404 when _should_block_robots returns False"""
|
||||
with mock.patch(
|
||||
"enterprise.proxy.enterprise_routes._should_block_robots", return_value=False
|
||||
):
|
||||
response = client.get("/robots.txt")
|
||||
assert response.status_code == 404
|
||||
Reference in New Issue
Block a user