tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-08-02 18:21:56 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(pricing): gemini-embedding-2 GA cost map, blog, and test (#26391)

Browse Source 
* feat(pricing): gemini-embedding-2 GA cost map, blog, and test

- Add model_prices entries for gemini-embedding-2 (Gemini + Vertex paths)
- Add docs blog gemini_embedding_2_ga with LiteLLM proxy curl examples
- Add test_gemini_embedding_2_ga_in_cost_map in test_utils

Made-with: Cursor

* Fix greptile reviews
This commit is contained in:
Sameer Kankute
2026-04-24 09:28:18 -07:00
committed by GitHub
parent 9dcb2bd528
commit e1466be825
4 changed files with 295 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/my-website/blog/gemini_embedding_2_ga/index.md
+172
View File
@@ -0,0 +1,172 @@
---
slug: gemini_embedding_2_ga
title: "Gemini Embedding 2 (GA): Multimodal Embeddings on LiteLLM"
date: 2026-04-24T10:00:00
authors:
- sameer
description: "Use generally available gemini-embedding-2 for multimodal embeddings on LiteLLM via Gemini API and Vertex AI—the same flows as preview, stable model id."
tags: [gemini, embeddings, multimodal, vertex ai]
hide_table_of_contents: false
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
# Gemini Embedding 2 (GA): Multimodal Embeddings
Litellm now fully supports Gemini Embedding 2 GA.
:::info
For end-to-end behavior, input shapes, and MIME types, see the [Gemini Embedding 2 Preview walkthrough](/blog/gemini_embedding_2_multimodal). This post focuses on **GA naming**, **cost map** coverage.
:::
{/* truncate */}
## Supported Input Types
| Modality | Supported Formats |
|----------|-------------------|
| **Text** | Plain text |
| **Image** | PNG, JPEG |
| **Audio** | MP3, WAV |
| **Video** | MP4, MOV |
| **Documents** | PDF |
## Input Formats
LiteLLM accepts three input formats for multimodal content:
1. **Data URIs** – Base64-encoded inline: `data:image/png;base64,<encoded_data>`
2. **GCS URLs** – Cloud Storage paths (Vertex AI): `gs://bucket/path/to/file.png`
3. **Gemini File References** – Pre-uploaded files (Gemini API): `files/abc123`
## Quick Start
<Tabs>
<TabItem value="gemini" label="Gemini API">
```python
from litellm import embedding
import os
os.environ["GEMINI_API_KEY"] = "your-api-key"
# Text + Image (base64)
response = embedding(
model="gemini/gemini-embedding-2",
input=[
"The food was delicious and the waiter...",
"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAgAAAAIAQMAAAD+wSzIAAAABlBMVEX///+/v7+jQ3Y5AAAADklEQVQI12P4AIX8EAgALgAD/aNpbtEAAAAASUVORK5CYII"
],
)
print(response)
```
</TabItem>
<TabItem value="vertex" label="Vertex AI">
```python
import litellm
from litellm import embedding
litellm.vertex_project = "your-project-id"
litellm.vertex_location = "us-central1"
# Text + Image (GCS URL)
response = embedding(
model="vertex_ai/gemini-embedding-2",
input=[
"Describe this image",
"gs://my-bucket/images/photo.png"
],
)
print(response)
```
</TabItem>
<TabItem value="proxy" label="LiteLLM Proxy">
**1. Config (config.yaml)**
```yaml
model_list:
- model_name: gemini-embedding-2
litellm_params:
model: gemini/gemini-embedding-2
api_key: os.environ/GEMINI_API_KEY
- model_name: vertex-gemini-embedding-2
litellm_params:
model: vertex_ai/gemini-embedding-2
vertex_project: os.environ/VERTEXAI_PROJECT
vertex_location: global
general_settings:
master_key: sk-1234
```
**2. Start proxy**
```bash
litellm --config config.yaml
```
**3. Call embeddings** (OpenAI-compatible **`POST /v1/embeddings`** on the proxy)
```bash
curl -sS -X POST http://localhost:4000/v1/embeddings \
-H "Authorization: Bearer sk-1234" \
-H "Content-Type: application/json" \
-d '{
"model": "gemini-embedding-2",
"input": [
"The food was delicious and the waiter...",
"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAgAAAAIAQMAAAD+wSzIAAAABlBMVEX///+/v7+jQ3Y5AAAADklEQVQI12P4AIX8EAgALgAD/aNpbtEAAAAASUVORK5CYII"
]
}'
```
</TabItem>
</Tabs>
## Input Format Examples
| Format | Example | Provider |
|--------|---------|----------|
| **Data URI** | `data:image/png;base64,...` | Gemini, Vertex AI |
| **GCS URL** | `gs://bucket/path/image.png` | Vertex AI |
| **File reference** | `files/abc123` | Gemini API only |
### Supported MIME Types for Data URIs
- **Images:** `image/png`, `image/jpeg`
- **Audio:** `audio/mpeg`, `audio/wav`
- **Video:** `video/mp4`, `video/quicktime`
- **Documents:** `application/pdf`
### GCS URL MIME Inference
For Vertex AI, MIME types are inferred from file extensions:
- `.png` → `image/png`
- `.jpg` / `.jpeg` → `image/jpeg`
- `.mp3` → `audio/mpeg`
- `.wav` → `audio/wav`
- `.mp4` → `video/mp4`
- `.mov` → `video/quicktime`
- `.pdf` → `application/pdf`
## Optional Parameters
| Parameter | Description | Maps to |
|-----------|-------------|---------|
| `dimensions` | Output embedding size | `outputDimensionality` |
```python
response = embedding(
model="gemini/gemini-embedding-2",
input=["text to embed"],
dimensions=768, # Optional: control output vector size
)
```
litellm/model_prices_and_context_window_backup.json
+46
View File
@@ -15097,6 +15097,21 @@
"source": "https://cloud.google.com/vertex-ai/generative-ai/pricing",
"uses_embed_content": true
},
"gemini-embedding-2": {
"input_cost_per_audio_per_second": 0.00016,
"input_cost_per_image": 0.00012,
"input_cost_per_token": 2e-07,
"input_cost_per_video_per_second": 0.00079,
"litellm_provider": "vertex_ai-embedding-models",
"max_input_tokens": 8192,
"max_tokens": 8192,
"mode": "embedding",
"output_cost_per_token": 0,
"output_vector_size": 3072,
"source": "https://cloud.google.com/vertex-ai/generative-ai/pricing",
"supports_multimodal": true,
"uses_embed_content": true
},
"vertex_ai/gemini-embedding-2-preview": {
"input_cost_per_audio_per_second": 0.00016,
"input_cost_per_image": 0.00012,
@@ -15112,6 +15127,21 @@
"supports_multimodal": true,
"uses_embed_content": true
},
"vertex_ai/gemini-embedding-2": {
"input_cost_per_audio_per_second": 0.00016,
"input_cost_per_image": 0.00012,
"input_cost_per_token": 2e-07,
"input_cost_per_video_per_second": 0.00079,
"litellm_provider": "vertex_ai",
"max_input_tokens": 8192,
"max_tokens": 8192,
"mode": "embedding",
"output_cost_per_token": 0,
"output_vector_size": 3072,
"source": "https://cloud.google.com/vertex-ai/generative-ai/pricing",
"supports_multimodal": true,
"uses_embed_content": true
},
"gemini-flash-experimental": {
"input_cost_per_character": 0,
"input_cost_per_token": 0,
@@ -15153,6 +15183,22 @@
"supports_multimodal": true,
"tpm": 10000000
},
"gemini/gemini-embedding-2": {
"input_cost_per_audio_per_second": 0.00016,
"input_cost_per_image": 0.00012,
"input_cost_per_token": 2e-07,
"input_cost_per_video_per_second": 0.00079,
"litellm_provider": "gemini",
"max_input_tokens": 8192,
"max_tokens": 8192,
"mode": "embedding",
"output_cost_per_token": 0,
"output_vector_size": 3072,
"rpm": 10000,
"source": "https://ai.google.dev/gemini-api/docs/pricing",
"supports_multimodal": true,
"tpm": 10000000
},
"gemini/gemini-1.5-flash": {
"deprecation_date": "2025-09-29",
"input_cost_per_token": 7.5e-08,
model_prices_and_context_window.json
+46
View File
@@ -15111,6 +15111,21 @@
"source": "https://cloud.google.com/vertex-ai/generative-ai/pricing",
"uses_embed_content": true
},
"gemini-embedding-2": {
"input_cost_per_audio_per_second": 0.00016,
"input_cost_per_image": 0.00012,
"input_cost_per_token": 2e-07,
"input_cost_per_video_per_second": 0.00079,
"litellm_provider": "vertex_ai-embedding-models",
"max_input_tokens": 8192,
"max_tokens": 8192,
"mode": "embedding",
"output_cost_per_token": 0,
"output_vector_size": 3072,
"source": "https://cloud.google.com/vertex-ai/generative-ai/pricing",
"supports_multimodal": true,
"uses_embed_content": true
},
"vertex_ai/gemini-embedding-2-preview": {
"input_cost_per_audio_per_second": 0.00016,
"input_cost_per_image": 0.00012,
@@ -15126,6 +15141,21 @@
"supports_multimodal": true,
"uses_embed_content": true
},
"vertex_ai/gemini-embedding-2": {
"input_cost_per_audio_per_second": 0.00016,
"input_cost_per_image": 0.00012,
"input_cost_per_token": 2e-07,
"input_cost_per_video_per_second": 0.00079,
"litellm_provider": "vertex_ai",
"max_input_tokens": 8192,
"max_tokens": 8192,
"mode": "embedding",
"output_cost_per_token": 0,
"output_vector_size": 3072,
"source": "https://cloud.google.com/vertex-ai/generative-ai/pricing",
"supports_multimodal": true,
"uses_embed_content": true
},
"gemini-flash-experimental": {
"input_cost_per_character": 0,
"input_cost_per_token": 0,
@@ -15167,6 +15197,22 @@
"supports_multimodal": true,
"tpm": 10000000
},
"gemini/gemini-embedding-2": {
"input_cost_per_audio_per_second": 0.00016,
"input_cost_per_image": 0.00012,
"input_cost_per_token": 2e-07,
"input_cost_per_video_per_second": 0.00079,
"litellm_provider": "gemini",
"max_input_tokens": 8192,
"max_tokens": 8192,
"mode": "embedding",
"output_cost_per_token": 0,
"output_vector_size": 3072,
"rpm": 10000,
"source": "https://ai.google.dev/gemini-api/docs/pricing",
"supports_multimodal": true,
"tpm": 10000000
},
"gemini/gemini-1.5-flash": {
"deprecation_date": "2025-09-29",
"input_cost_per_token": 7.5e-08,
tests/test_litellm/test_utils.py
+31
View File
@@ -2794,6 +2794,37 @@ def test_model_info_for_openrouter_kimi_k2_5():
print("openrouter kimi-k2.5 model info", model_info)
def test_gemini_embedding_2_ga_in_cost_map():
"""GA gemini-embedding-2 entries align with preview multimodal unit pricing."""
import json
from pathlib import Path
json_path = Path(__file__).parents[2] / "model_prices_and_context_window.json"
with open(json_path) as f:
model_cost = json.load(f)
for key, provider in (
("gemini/gemini-embedding-2", "gemini"),
("vertex_ai/gemini-embedding-2", "vertex_ai"),
("gemini-embedding-2", "vertex_ai-embedding-models"),
):
info = model_cost.get(key)
assert (
info is not None
), f"{key} missing from model_prices_and_context_window.json"
assert info["litellm_provider"] == provider
assert info.get("mode") == "embedding"
assert info.get("supports_multimodal") is True
assert info.get("input_cost_per_token") == 2e-07
assert info.get("input_cost_per_image") == 0.00012
assert info.get("input_cost_per_audio_per_second") == 0.00016
assert info.get("input_cost_per_video_per_second") == 0.00079
if provider in ("vertex_ai-embedding-models", "vertex_ai"):
assert info.get("uses_embed_content") is True, (
f"{key} must have uses_embed_content=true for correct Vertex AI routing"
)
def test_gemini_lyria_3_preview_models_in_cost_map():
import json
from pathlib import Path