diff --git a/docs/my-website/docs/providers/gemini.md b/docs/my-website/docs/providers/gemini.md
index 70988f0b5f..31fcb1e221 100644
--- a/docs/my-website/docs/providers/gemini.md
+++ b/docs/my-website/docs/providers/gemini.md
@@ -1,3 +1,7 @@
+import Image from '@theme/IdealImage';
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
# Gemini - Google AI Studio
## Pre-requisites
@@ -17,6 +21,335 @@ response = completion(
)
```
+## Supported OpenAI Params
+- temperature
+- top_p
+- max_tokens
+- stream
+- tools
+- tool_choice
+- response_format
+- n
+- stop
+
+[**See Updated List**](https://github.com/BerriAI/litellm/blob/1c747f3ad372399c5b95cc5696b06a5fbe53186b/litellm/llms/vertex_httpx.py#L122)
+
+## Passing Gemini Specific Params
+### Response schema
+LiteLLM supports sending `response_schema` as a param for Gemini-1.5-Pro on Google AI Studio.
+
+**Response Schema**
+
+
+
+```python
+from litellm import completion
+import json
+import os
+
+os.environ['GEMINI_API_KEY'] = ""
+
+messages = [
+ {
+ "role": "user",
+ "content": "List 5 popular cookie recipes."
+ }
+]
+
+response_schema = {
+ "type": "array",
+ "items": {
+ "type": "object",
+ "properties": {
+ "recipe_name": {
+ "type": "string",
+ },
+ },
+ "required": ["recipe_name"],
+ },
+ }
+
+
+completion(
+ model="gemini/gemini-1.5-pro",
+ messages=messages,
+ response_format={"type": "json_object", "response_schema": response_schema} # 👈 KEY CHANGE
+ )
+
+print(json.loads(completion.choices[0].message.content))
+```
+
+
+
+
+1. Add model to config.yaml
+```yaml
+model_list:
+ - model_name: gemini-pro
+ litellm_params:
+ model: gemini/gemini-1.5-pro
+ api_key: os.environ/GEMINI_API_KEY
+```
+
+2. Start Proxy
+
+```
+$ litellm --config /path/to/config.yaml
+```
+
+3. Make Request!
+
+```bash
+curl -X POST 'http://0.0.0.0:4000/chat/completions' \
+-H 'Content-Type: application/json' \
+-H 'Authorization: Bearer sk-1234' \
+-D '{
+ "model": "gemini-pro",
+ "messages": [
+ {"role": "user", "content": "List 5 popular cookie recipes."}
+ ],
+ "response_format": {"type": "json_object", "response_schema": {
+ "type": "array",
+ "items": {
+ "type": "object",
+ "properties": {
+ "recipe_name": {
+ "type": "string",
+ },
+ },
+ "required": ["recipe_name"],
+ },
+ }}
+}
+'
+```
+
+
+
+
+**Validate Schema**
+
+To validate the response_schema, set `enforce_validation: true`.
+
+
+
+
+```python
+from litellm import completion, JSONSchemaValidationError
+try:
+ completion(
+ model="gemini/gemini-1.5-pro",
+ messages=messages,
+ response_format={
+ "type": "json_object",
+ "response_schema": response_schema,
+ "enforce_validation": true # 👈 KEY CHANGE
+ }
+ )
+except JSONSchemaValidationError as e:
+ print("Raw Response: {}".format(e.raw_response))
+ raise e
+```
+
+
+
+1. Add model to config.yaml
+```yaml
+model_list:
+ - model_name: gemini-pro
+ litellm_params:
+ model: gemini/gemini-1.5-pro
+ api_key: os.environ/GEMINI_API_KEY
+```
+
+2. Start Proxy
+
+```
+$ litellm --config /path/to/config.yaml
+```
+
+3. Make Request!
+
+```bash
+curl -X POST 'http://0.0.0.0:4000/chat/completions' \
+-H 'Content-Type: application/json' \
+-H 'Authorization: Bearer sk-1234' \
+-D '{
+ "model": "gemini-pro",
+ "messages": [
+ {"role": "user", "content": "List 5 popular cookie recipes."}
+ ],
+ "response_format": {"type": "json_object", "response_schema": {
+ "type": "array",
+ "items": {
+ "type": "object",
+ "properties": {
+ "recipe_name": {
+ "type": "string",
+ },
+ },
+ "required": ["recipe_name"],
+ },
+ },
+ "enforce_validation": true
+ }
+}
+'
+```
+
+
+
+
+LiteLLM will validate the response against the schema, and raise a `JSONSchemaValidationError` if the response does not match the schema.
+
+JSONSchemaValidationError inherits from `openai.APIError`
+
+Access the raw response with `e.raw_response`
+
+
+
+### GenerationConfig Params
+
+To pass additional GenerationConfig params - e.g. `topK`, just pass it in the request body of the call, and LiteLLM will pass it straight through as a key-value pair in the request body.
+
+[**See Gemini GenerationConfigParams**](https://ai.google.dev/api/generate-content#v1beta.GenerationConfig)
+
+
+
+
+```python
+from litellm import completion
+import json
+import os
+
+os.environ['GEMINI_API_KEY'] = ""
+
+messages = [
+ {
+ "role": "user",
+ "content": "List 5 popular cookie recipes."
+ }
+]
+
+completion(
+ model="gemini/gemini-1.5-pro",
+ messages=messages,
+ topK=1 # 👈 KEY CHANGE
+)
+
+print(json.loads(completion.choices[0].message.content))
+```
+
+
+
+
+1. Add model to config.yaml
+```yaml
+model_list:
+ - model_name: gemini-pro
+ litellm_params:
+ model: gemini/gemini-1.5-pro
+ api_key: os.environ/GEMINI_API_KEY
+```
+
+2. Start Proxy
+
+```
+$ litellm --config /path/to/config.yaml
+```
+
+3. Make Request!
+
+```bash
+curl -X POST 'http://0.0.0.0:4000/chat/completions' \
+-H 'Content-Type: application/json' \
+-H 'Authorization: Bearer sk-1234' \
+-d '{
+ "model": "gemini-pro",
+ "messages": [
+ {"role": "user", "content": "List 5 popular cookie recipes."}
+ ],
+ "topK": 1 # 👈 KEY CHANGE
+}
+'
+```
+
+
+
+
+**Validate Schema**
+
+To validate the response_schema, set `enforce_validation: true`.
+
+
+
+
+```python
+from litellm import completion, JSONSchemaValidationError
+try:
+ completion(
+ model="gemini/gemini-1.5-pro",
+ messages=messages,
+ response_format={
+ "type": "json_object",
+ "response_schema": response_schema,
+ "enforce_validation": true # 👈 KEY CHANGE
+ }
+ )
+except JSONSchemaValidationError as e:
+ print("Raw Response: {}".format(e.raw_response))
+ raise e
+```
+
+
+
+1. Add model to config.yaml
+```yaml
+model_list:
+ - model_name: gemini-pro
+ litellm_params:
+ model: gemini/gemini-1.5-pro
+ api_key: os.environ/GEMINI_API_KEY
+```
+
+2. Start Proxy
+
+```
+$ litellm --config /path/to/config.yaml
+```
+
+3. Make Request!
+
+```bash
+curl -X POST 'http://0.0.0.0:4000/chat/completions' \
+-H 'Content-Type: application/json' \
+-H 'Authorization: Bearer sk-1234' \
+-D '{
+ "model": "gemini-pro",
+ "messages": [
+ {"role": "user", "content": "List 5 popular cookie recipes."}
+ ],
+ "response_format": {"type": "json_object", "response_schema": {
+ "type": "array",
+ "items": {
+ "type": "object",
+ "properties": {
+ "recipe_name": {
+ "type": "string",
+ },
+ },
+ "required": ["recipe_name"],
+ },
+ },
+ "enforce_validation": true
+ }
+}
+'
+```
+
+
+
+
## Specifying Safety Settings
In certain use-cases you may need to make calls to the models and pass [safety settigns](https://ai.google.dev/docs/safety_setting_gemini) different from the defaults. To do so, simple pass the `safety_settings` argument to `completion` or `acompletion`. For example:
@@ -91,6 +424,72 @@ assert isinstance(
```
+## JSON Mode
+
+
+
+
+```python
+from litellm import completion
+import json
+import os
+
+os.environ['GEMINI_API_KEY'] = ""
+
+messages = [
+ {
+ "role": "user",
+ "content": "List 5 popular cookie recipes."
+ }
+]
+
+
+
+completion(
+ model="gemini/gemini-1.5-pro",
+ messages=messages,
+ response_format={"type": "json_object"} # 👈 KEY CHANGE
+)
+
+print(json.loads(completion.choices[0].message.content))
+```
+
+
+
+
+1. Add model to config.yaml
+```yaml
+model_list:
+ - model_name: gemini-pro
+ litellm_params:
+ model: gemini/gemini-1.5-pro
+ api_key: os.environ/GEMINI_API_KEY
+```
+
+2. Start Proxy
+
+```
+$ litellm --config /path/to/config.yaml
+```
+
+3. Make Request!
+
+```bash
+curl -X POST 'http://0.0.0.0:4000/chat/completions' \
+-H 'Content-Type: application/json' \
+-H 'Authorization: Bearer sk-1234' \
+-d '{
+ "model": "gemini-pro",
+ "messages": [
+ {"role": "user", "content": "List 5 popular cookie recipes."}
+ ],
+ "response_format": {"type": "json_object"}
+}
+'
+```
+
+
+
# Gemini-Pro-Vision
LiteLLM Supports the following image types passed in `url`
- Images with direct links - https://storage.googleapis.com/github-repo/img/gemini/intro/landmark3.jpg