mirror of
https://github.com/tiennm99/litellm.git
synced 2026-07-14 01:07:15 +00:00
Teddy Amkie
dcbccd1fea
* docs: Corrected documentation updates from Sept 2025 This PR contains the actual intended documentation changes, properly synced with main: ✅ Real changes applied: - Added AWS authentication link to bedrock guardrails documentation - Updated Vertex AI with Gemini API alternative configuration - Added async_post_call_success_hook code snippet to custom callback docs - Added SSO free for up to 5 users information to enterprise and custom_sso docs - Added SSO free information block to security.md - Added cancel response API usage and curl example to response_api.md - Added image for modifying default user budget via admin UI - Re-ordered sidebars in documentation ❌ Sync issues resolved: - Kept all upstream changes that were added to main after branch diverged - Preserved Provider-Specific Metadata Parameters section that was added upstream - Maintained proper curl parameter formatting (-d instead of -D) This corrects the sync issues from the original PR #14769. * docs: Restore missing files from original PR Added back ~16 missing documentation files that were part of the original PR: ✅ Restored files: - docs/my-website/docs/completion/usage.md - docs/my-website/docs/fine_tuning.md - docs/my-website/docs/getting_started.md - docs/my-website/docs/image_edits.md - docs/my-website/docs/image_generation.md - docs/my-website/docs/index.md - docs/my-website/docs/moderation.md - docs/my-website/docs/observability/callbacks.md - docs/my-website/docs/providers/bedrock.md - docs/my-website/docs/proxy/caching.md - docs/my-website/docs/proxy/config_settings.md - docs/my-website/docs/proxy/db_deadlocks.md - docs/my-website/docs/proxy/load_balancing.md - docs/my-website/docs/proxy_api.md - docs/my-website/docs/rerank.md ✅ Fixed context-caching issue: - Restored provider_specific_params.md to main version (preserving Provider-Specific Metadata Parameters section) - Your original PR didn't intend to modify this file - it was just a sync issue Now includes all ~26 documentation files from the original PR #14769. * docs: Remove files that were deleted in original PR - Removed docs/my-website/docs/providers/azure_ai_img_edit.md (was deleted in original PR) - sdk/headers.md was already not present Now matches the complete intended changes from original PR #14769. * docs: Restore azure_ai_img_edit.md from main - Restored docs/my-website/docs/providers/azure_ai_img_edit.md from main branch - This file should not have been deleted as it was a newer commit - SDK headers file doesn't exist in main (was reverted) and wasn't part of your original changes Fixes the file restoration issues. * docs: Fix vertex.md - preserve context caching from newer commit - Restored vertex.md to main version to preserve context caching content (lines 817-887) - Added back only your intended change: alternative gemini config example - Context caching content from newer commit is now preserved Fixes the vertex.md sync issue where newer content was incorrectly deleted. * docs: Fix providers/bedrock.md - restore deleted content from newer commit - Restored providers/bedrock.md to main version - Preserves 'Usage - Request Metadata' section that was added in newer commit - Your actual intended change was to proxy/guardrails/bedrock.md (authentication tip) which is preserved - Now only has additions, no subtractions as intended Fixes the bedrock.md sync issue. * docs: Restore missing IAM policy section in bedrock.md Added back your intended IAM policy documentation that was lost when restoring main version: ✅ Added IAM AssumeRole Policy section: - Explains requirement for sts:AssumeRole permission - Shows error message example when permission missing - Provides complete IAM policy JSON example - Links to AWS AssumeRole documentation - Clarifies trust policy requirements Now bedrock.md has both: - All newer content preserved (Request Metadata section) - Your intended IAM policy addition restored --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2.8 KiB
2.8 KiB
🔑 LiteLLM Keys (Access Claude-2, Llama2-70b, etc.)
Use this if you're trying to add support for new LLMs and need access for testing. We provide a free $10 community-key for testing all providers on LiteLLM:
usage (community-key)
import os
from litellm import completion
## set ENV variables
os.environ["OPENAI_API_KEY"] = "your-api-key"
os.environ["COHERE_API_KEY"] = "your-api-key"
messages = [{ "content": "Hello, how are you?","role": "user"}]
# openai call
response = completion(model="gpt-3.5-turbo", messages=messages)
# cohere call
response = completion("command-nightly", messages)
Need a dedicated key? Email us @ krrish@berri.ai
Supported Models for LiteLLM Key
These are the models that currently work with the "sk-litellm-.." keys.
For a complete list of models/providers that you can call with LiteLLM, check out our provider list or check out models.litellm.ai
- OpenAI models - OpenAI docs
- gpt-4
- gpt-3.5-turbo
- gpt-3.5-turbo-16k
- Llama2 models - TogetherAI docs
- togethercomputer/llama-2-70b-chat
- togethercomputer/llama-2-70b
- togethercomputer/LLaMA-2-7B-32K
- togethercomputer/Llama-2-7B-32K-Instruct
- togethercomputer/llama-2-7b
- togethercomputer/CodeLlama-34b
- WizardLM/WizardCoder-Python-34B-V1.0
- NousResearch/Nous-Hermes-Llama2-13b
- Falcon models - TogetherAI docs
- togethercomputer/falcon-40b-instruct
- togethercomputer/falcon-7b-instruct
- Jurassic/AI21 models - AI21 docs
- j2-ultra
- j2-mid
- j2-light
- NLP Cloud models - NLPCloud docs
- dolpin
- chatdolphin
- Anthropic models - Anthropic docs
- claude-2
- claude-instant-v1
For OpenInterpreter
This was initially built for the Open Interpreter community. If you're trying to use this feature in there, here's how you can do it:
Note: You will need to clone and modify the Github repo, until this PR is merged.
git clone https://github.com/krrishdholakia/open-interpreter-litellm-fork
To run it do:
poetry build
# call gpt-4 - always add 'litellm_proxy/' in front of the model name
poetry run interpreter --model litellm_proxy/gpt-4
# call llama-70b - always add 'litellm_proxy/' in front of the model name
poetry run interpreter --model litellm_proxy/togethercomputer/llama-2-70b-chat
# call claude-2 - always add 'litellm_proxy/' in front of the model name
poetry run interpreter --model litellm_proxy/claude-2
And that's it!
Now you can call any model you like!
Want us to add more models? Let us know!