docs guardrail params (#7430)

2026-08-05 02:23:42 +00:00 · 2024-12-26 11:08:47 -08:00
parent c1d6c35aef
commit 12c4e7e695
1 changed files with 111 additions and 0 deletions
@@ -409,6 +409,117 @@ Expected response after running during-guard

 </Tabs>

+## ✨ Pass additional parameters to guardrail
+
+:::info
+
+✨ This is an Enterprise only feature [Contact us to get a free trial](https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat)
+
+:::
+
+
+Use this to pass additional parameters to the guardrail API call. e.g. things like success threshold
+
+1. Use `get_guardrail_dynamic_request_body_params`
+
+`get_guardrail_dynamic_request_body_params` is a method of the `litellm.integrations.custom_guardrail.CustomGuardrail` class that fetches the dynamic guardrail params passed in the request body.
+
+```python
+from typing import Any, Dict, List, Literal, Optional, Union
+import litellm
+from litellm._logging import verbose_proxy_logger
+from litellm.caching.caching import DualCache
+from litellm.integrations.custom_guardrail import CustomGuardrail
+from litellm.proxy._types import UserAPIKeyAuth
+
+class myCustomGuardrail(CustomGuardrail):
+    def __init__(self, **kwargs):
+        super().__init__(**kwargs)
+
+    async def async_pre_call_hook(
+        self,
+        user_api_key_dict: UserAPIKeyAuth,
+        cache: DualCache,
+        data: dict,
+        call_type: Literal[
+            "completion",
+            "text_completion",
+            "embeddings",
+            "image_generation",
+            "moderation",
+            "audio_transcription",
+            "pass_through_endpoint",
+            "rerank"
+        ],
+    ) -> Optional[Union[Exception, str, dict]]:
+        # Get dynamic params from request body
+        params = self.get_guardrail_dynamic_request_body_params(request_data=data)
+        # params will contain: {"success_threshold": 0.9}
+        verbose_proxy_logger.debug("Guardrail params: %s", params)
+        return data
+```
+
+2. Pass parameters in your API requests:
+
+LiteLLM Proxy allows you to pass `guardrails` in the request body, following the [`guardrails` spec](quick_start#spec-guardrails-parameter).
+
+<Tabs>
+<TabItem value="openai" label="OpenAI Python">
+
+```python
+import openai
+client = openai.OpenAI(
+    api_key="anything",
+    base_url="http://0.0.0.0:4000"
+)
+
+response = client.chat.completions.create(
+    model="gpt-3.5-turbo",
+    messages=[{"role": "user", "content": "Write a short poem"}],
+    extra_body={
+        "guardrails": [
+            "custom-pre-guard": {
+                "extra_body": {
+                    "success_threshold": 0.9
+                }
+            }
+        ]
+    }
+)
+```
+</TabItem>
+
+<TabItem value="curl" label="Curl">
+
+```shell
+curl 'http://0.0.0.0:4000/chat/completions' \
+    -H 'Content-Type: application/json' \
+    -d '{
+    "model": "gpt-3.5-turbo",
+    "messages": [
+        {
+            "role": "user",
+            "content": "Write a short poem"
+        }
+    ],
+    "guardrails": [
+        "custom-pre-guard": {
+            "extra_body": {
+                "success_threshold": 0.9
+            }
+        }
+    ]
+}'
+```
+</TabItem>
+</Tabs>
+
+The `get_guardrail_dynamic_request_body_params` method will return:
+```json
+{
+    "success_threshold": 0.9
+}
+```

 ## **CustomGuardrail methods**