docs(vertex.md): add doc on passing base64 pdf to vertex ai

This commit is contained in:
Krrish Dholakia
2024-07-08 10:16:41 -07:00
parent c89aa1e626
commit 323add2e67
+67 -3
View File
@@ -825,9 +825,11 @@ assert isinstance(
Pass any file supported by Vertex AI, through LiteLLM.
<Tabs>
<TabItem value="sdk" label="SDK">
### **Using `gs://`**
```python
from litellm import completion
@@ -840,7 +842,7 @@ response = completion(
{"type": "text", "text": "You are a very professional document summarization specialist. Please summarize the given document."},
{
"type": "image_url",
"image_url": "gs://cloud-samples-data/generative-ai/pdf/2403.05530.pdf",
"image_url": "gs://cloud-samples-data/generative-ai/pdf/2403.05530.pdf", # 👈 PDF
},
],
}
@@ -849,7 +851,41 @@ response = completion(
)
print(response.choices[0])
```
### **using base64**
```python
from litellm import completion
import base64
import requests
# URL of the file
url = "https://storage.googleapis.com/cloud-samples-data/generative-ai/pdf/2403.05530.pdf"
# Download the file
response = requests.get(url)
file_data = response.content
encoded_file = base64.b64encode(file_data).decode("utf-8")
response = completion(
model="vertex_ai/gemini-1.5-flash",
messages=[
{
"role": "user",
"content": [
{"type": "text", "text": "You are a very professional document summarization specialist. Please summarize the given document."},
{
"type": "image_url",
"image_url": f"data:application/pdf;base64,{encoded_file}", # 👈 PDF
},
],
}
],
max_tokens=300,
)
print(response.choices[0])
```
</TabItem>
<TabItem value="proxy" lable="PROXY">
@@ -871,6 +907,7 @@ litellm --config /path/to/config.yaml
3. Test it!
**Using `gs://`**
```bash
curl http://0.0.0.0:4000/v1/chat/completions \
-H "Content-Type: application/json" \
@@ -887,8 +924,8 @@ curl http://0.0.0.0:4000/v1/chat/completions \
},
{
"type": "image_url",
"image_url": "gs://cloud-samples-data/generative-ai/pdf/2403.05530.pdf",
},
"image_url": "gs://cloud-samples-data/generative-ai/pdf/2403.05530.pdf" # 👈 PDF
}
}
]
}
@@ -898,6 +935,33 @@ curl http://0.0.0.0:4000/v1/chat/completions \
```
```bash
curl http://0.0.0.0:4000/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer <YOUR-LITELLM-KEY>" \
-d '{
"model": "gemini-1.5-flash",
"messages": [
{
"role": "user",
"content": [
{
"type": "text",
"text": "You are a very professional document summarization specialist. Please summarize the given document"
},
{
"type": "image_url",
"image_url": "data:application/pdf;base64,{encoded_file}" # 👈 PDF
}
}
]
}
],
"max_tokens": 300
}'
```
</TabItem>
</Tabs>