diff --git a/docs/my-website/docs/completion/document_understanding.md b/docs/my-website/docs/completion/document_understanding.md index 6719169aef..c101aa1aef 100644 --- a/docs/my-website/docs/completion/document_understanding.md +++ b/docs/my-website/docs/completion/document_understanding.md @@ -200,3 +200,92 @@ Expected Response + + +## OpenAI 'file' message type + +This is currently only supported for OpenAI models. + +This will be supported for all providers soon. + + + + +```python +import base64 +from litellm import completion + +with open("draconomicon.pdf", "rb") as f: + data = f.read() + +base64_string = base64.b64encode(data).decode("utf-8") + +completion = completion( + model="gpt-4o", + messages=[ + { + "role": "user", + "content": [ + { + "type": "file", + "file": { + "filename": "draconomicon.pdf", + "file_data": f"data:application/pdf;base64,{base64_string}", + } + }, + { + "type": "text", + "text": "What is the first dragon in the book?", + } + ], + }, + ], +) + +print(completion.choices[0].message.content) +``` + + + + + +1. Setup config.yaml + +```yaml +model_list: + - model_name: openai-model + litellm_params: + model: gpt-4o + api_key: os.environ/OPENAI_API_KEY +``` + +2. Start the proxy + +```bash +litellm --config config.yaml +``` + +3. Test it! + +```bash +curl -X POST 'http://0.0.0.0:4000/chat/completions' \ +-H 'Content-Type: application/json' \ +-H 'Authorization: Bearer sk-1234' \ +-d '{ + "model": "openai-model", + "messages": [ + {"role": "user", "content": [ + { + "type": "file", + "file": { + "filename": "draconomicon.pdf", + "file_data": f"data:application/pdf;base64,{base64_string}", + } + } + ]} + ] +}' +``` + + + \ No newline at end of file diff --git a/docs/my-website/docs/providers/openai.md b/docs/my-website/docs/providers/openai.md index 15661f6521..794f3da647 100644 --- a/docs/my-website/docs/providers/openai.md +++ b/docs/my-website/docs/providers/openai.md @@ -228,6 +228,92 @@ response = completion( ``` +## PDF File Parsing + +OpenAI has a new `file` message type that allows you to pass in a PDF file and have it parsed into a structured output. [Read more](https://platform.openai.com/docs/guides/pdf-files?api-mode=chat&lang=python) + + + + +```python +import base64 +from litellm import completion + +with open("draconomicon.pdf", "rb") as f: + data = f.read() + +base64_string = base64.b64encode(data).decode("utf-8") + +completion = completion( + model="gpt-4o", + messages=[ + { + "role": "user", + "content": [ + { + "type": "file", + "file": { + "filename": "draconomicon.pdf", + "file_data": f"data:application/pdf;base64,{base64_string}", + } + }, + { + "type": "text", + "text": "What is the first dragon in the book?", + } + ], + }, + ], +) + +print(completion.choices[0].message.content) +``` + + + + + +1. Setup config.yaml + +```yaml +model_list: + - model_name: openai-model + litellm_params: + model: gpt-4o + api_key: os.environ/OPENAI_API_KEY +``` + +2. Start the proxy + +```bash +litellm --config config.yaml +``` + +3. Test it! + +```bash +curl -X POST 'http://0.0.0.0:4000/chat/completions' \ +-H 'Content-Type: application/json' \ +-H 'Authorization: Bearer sk-1234' \ +-d '{ + "model": "openai-model", + "messages": [ + {"role": "user", "content": [ + { + "type": "file", + "file": { + "filename": "draconomicon.pdf", + "file_data": f"data:application/pdf;base64,{base64_string}", + } + } + ]} + ] +}' +``` + + + + ## OpenAI Fine Tuned Models | Model Name | Function Call | @@ -449,26 +535,6 @@ response = litellm.acompletion( ) ``` -### Using Helicone Proxy with LiteLLM -```python -import os -import litellm -from litellm import completion - -os.environ["OPENAI_API_KEY"] = "" - -# os.environ["OPENAI_API_BASE"] = "" -litellm.api_base = "https://oai.hconeai.com/v1" -litellm.headers = { - "Helicone-Auth": f"Bearer {os.getenv('HELICONE_API_KEY')}", - "Helicone-Cache-Enabled": "true", -} - -messages = [{ "content": "Hello, how are you?","role": "user"}] - -# openai call -response = completion("gpt-3.5-turbo", messages) -``` ### Using OpenAI Proxy with LiteLLM ```python diff --git a/docs/my-website/release_notes/v1.63.14/index.md b/docs/my-website/release_notes/v1.63.14/index.md index 4fee2e8470..f6ae4a691c 100644 --- a/docs/my-website/release_notes/v1.63.14/index.md +++ b/docs/my-website/release_notes/v1.63.14/index.md @@ -58,13 +58,13 @@ Here's a Demo Instance to test changes: 1. **New LLM Features** - Bedrock: Support bedrock application inference profiles [Docs](https://docs.litellm.ai/docs/providers/bedrock#bedrock-application-inference-profile) - - Infer aws region from bedrock application profile id + - Infer aws region from bedrock application profile id - (`arn:aws:bedrock:us-east-1:...`) - Ollama - support calling via `/v1/completions` - [NEEDS DOCS]https://github.com/BerriAI/litellm/pull/9333 - Bedrock - support `us.deepseek.r1-v1:0` model name [Docs](../../docs/providers/bedrock#supported-aws-bedrock-models) https://github.com/BerriAI/litellm/pull/9363 - OpenRouter - `OPENROUTER_API_BASE` env var support [Docs](../../docs/providers/openrouter.md) -- Azure - add audio model parameter support - https://github.com/BerriAI/litellm/commit/fe24b9d90b95012ac030f6919a766cbeab1b1ae3 +- Azure - add audio model parameter support - [Docs](../../docs/providers/azure#azure-audio-model) - OpenAI - ‘file’ message type support - https://github.com/BerriAI/litellm/commit/12e730885bd3948543dca902293f461c1bc4fb60 - OpenAI - o1-pro Responses API streaming support - https://github.com/BerriAI/litellm/pull/9419 - Passthrough Endpoints - support returning api-base on pass-through endpoints - https://github.com/BerriAI/litellm/pull/9439