tiennm99/litellm
1
0
Fork 0
mirror of https://github.com/tiennm99/litellm.git synced 2026-08-01 16:21:12 +00:00
Code Issues Packages Projects Releases Wiki Activity

Litellm docs readme fixes (#16107)

Browse Source 
* add provider test

* docs readme.md

* docs providers

* order providers

* test_providers_alphabetically_ordered

* docs endpoint

* fix config
This commit is contained in:
Ishaan Jaff
2025-10-30 17:05:32 -07:00
committed by GitHub
parent 40c663439a
commit fd073ee682
3 changed files with 284 additions and 45 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+101 -45
View File
@@ -303,51 +303,107 @@ curl 'http://0.0.0.0:4000/key/generate' \
## Supported Providers ([Docs](https://docs.litellm.ai/docs/providers))
| Provider | `/chat/completions` | `/embeddings` | `/image/generations` |
|-------------------------------------------------------------------------------------|---------------------|---------------|----------------------|
| [OpenAI](https://docs.litellm.ai/docs/providers/openai) | ✅ | ✅ | ✅ |
| [Meta - Llama API](https://docs.litellm.ai/docs/providers/meta_llama) | ✅ | | |
| [Azure](https://docs.litellm.ai/docs/providers/azure) | ✅ | ✅ | ✅ |
| [AI/ML API](https://docs.litellm.ai/docs/providers/aiml) | ✅ | ✅ | ✅ |
| [AWS - Sagemaker](https://docs.litellm.ai/docs/providers/aws_sagemaker) | ✅ | ✅ | |
| [AWS - Bedrock](https://docs.litellm.ai/docs/providers/bedrock) | ✅ | ✅ | |
| [Google - Vertex AI](https://docs.litellm.ai/docs/providers/vertex) | ✅ | ✅ | ✅ |
| [Google - PaLM](https://docs.litellm.ai/docs/providers/palm) | ✅ | | |
| [Google AI Studio - Gemini](https://docs.litellm.ai/docs/providers/gemini) | ✅ | | |
| [Mistral AI API](https://docs.litellm.ai/docs/providers/mistral) | ✅ | ✅ | |
| [Cloudflare AI Workers](https://docs.litellm.ai/docs/providers/cloudflare_workers) | ✅ | | |
| [CompactifAI](https://docs.litellm.ai/docs/providers/compactifai) | ✅ | | |
| [Cohere](https://docs.litellm.ai/docs/providers/cohere) | ✅ | ✅ | |
| [Anthropic](https://docs.litellm.ai/docs/providers/anthropic) | ✅ | | |
| [Empower](https://docs.litellm.ai/docs/providers/empower) | ✅ | | |
| [Huggingface](https://docs.litellm.ai/docs/providers/huggingface) | ✅ | ✅ | |
| [Replicate](https://docs.litellm.ai/docs/providers/replicate) | ✅ | | |
| [Together AI](https://docs.litellm.ai/docs/providers/togetherai) | ✅ | | |
| [OpenRouter](https://docs.litellm.ai/docs/providers/openrouter) | ✅ | | |
| [AI21](https://docs.litellm.ai/docs/providers/ai21) | ✅ | | |
| [Baseten](https://docs.litellm.ai/docs/providers/baseten) | ✅ | | |
| [VLLM](https://docs.litellm.ai/docs/providers/vllm) | ✅ | | |
| [NLP Cloud](https://docs.litellm.ai/docs/providers/nlp_cloud) | ✅ | | |
| [Aleph Alpha](https://docs.litellm.ai/docs/providers/aleph_alpha) | ✅ | | |
| [Petals](https://docs.litellm.ai/docs/providers/petals) | ✅ | | |
| [Ollama](https://docs.litellm.ai/docs/providers/ollama) | ✅ | ✅ | |
| [DeepInfra](https://docs.litellm.ai/docs/providers/deepinfra) | ✅ | | |
| [Perplexity AI](https://docs.litellm.ai/docs/providers/perplexity) | ✅ | | |
| [Groq AI](https://docs.litellm.ai/docs/providers/groq) | ✅ | | |
| [Deepseek](https://docs.litellm.ai/docs/providers/deepseek) | ✅ | | |
| [Anyscale](https://docs.litellm.ai/docs/providers/anyscale) | ✅ | | |
| [IBM - Watsonx.ai](https://docs.litellm.ai/docs/providers/watsonx) | ✅ | ✅ | |
| [Voyage AI](https://docs.litellm.ai/docs/providers/voyage) | | ✅ | |
| [Xinference [Xorbits Inference]](https://docs.litellm.ai/docs/providers/xinference) | | ✅ | |
| [FriendliAI](https://docs.litellm.ai/docs/providers/friendliai) | ✅ | | |
| [Galadriel](https://docs.litellm.ai/docs/providers/galadriel) | ✅ | | |
| [GradientAI](https://docs.litellm.ai/docs/providers/gradient_ai) | ✅ | | |
| [Novita AI](https://novita.ai/models/llm?utm_source=github_litellm&utm_medium=github_readme&utm_campaign=github_link) | ✅ | | |
| [Featherless AI](https://docs.litellm.ai/docs/providers/featherless_ai) | ✅ | | |
| [Nebius AI Studio](https://docs.litellm.ai/docs/providers/nebius) | ✅ | ✅ | |
| [Heroku](https://docs.litellm.ai/docs/providers/heroku) | ✅ | | |
| [OVHCloud AI Endpoints](https://docs.litellm.ai/docs/providers/ovhcloud) | ✅ | | |
| [CometAPI](https://docs.litellm.ai/docs/providers/cometapi) | ✅ | ✅ | ✅ |
| Provider | `/chat/completions` | `/messages` | `/responses` | `/embeddings` | `/image/generations` | `/audio/transcriptions` | `/audio/speech` | `/moderations` | `/batches` | `/rerank` |
|-------------------------------------------------------------------------------------|---------------------|-------------|--------------|---------------|----------------------|-------------------------|-----------------|----------------|-----------|-----------|
| [AI/ML API (`aiml`)](https://docs.litellm.ai/docs/providers/aiml) | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | |
| [AI21 (`ai21`)](https://docs.litellm.ai/docs/providers/ai21) | ✅ | ✅ | ✅ | | | | | | | |
| [AI21 Chat (`ai21_chat`)](https://docs.litellm.ai/docs/providers/ai21) | ✅ | ✅ | ✅ | | | | | | | |
| [Aleph Alpha](https://docs.litellm.ai/docs/providers/aleph_alpha) | ✅ | ✅ | ✅ | | | | | | | |
| [Anthropic (`anthropic`)](https://docs.litellm.ai/docs/providers/anthropic) | ✅ | ✅ | ✅ | | | | | | ✅ | |
| [Anthropic Text (`anthropic_text`)](https://docs.litellm.ai/docs/providers/anthropic) | ✅ | ✅ | ✅ | | | | | | ✅ | |
| [Anyscale](https://docs.litellm.ai/docs/providers/anyscale) | ✅ | ✅ | ✅ | | | | | | | |
| [AssemblyAI (`assemblyai`)](https://docs.litellm.ai/docs/pass_through/assembly_ai) | ✅ | ✅ | ✅ | | | ✅ | | | | |
| [Auto Router (`auto_router`)](https://docs.litellm.ai/docs/proxy/auto_routing) | ✅ | ✅ | ✅ | | | | | | | |
| [AWS - Bedrock (`bedrock`)](https://docs.litellm.ai/docs/providers/bedrock) | ✅ | ✅ | ✅ | ✅ | | | | | | ✅ |
| [AWS - Sagemaker (`sagemaker`)](https://docs.litellm.ai/docs/providers/aws_sagemaker) | ✅ | ✅ | ✅ | ✅ | | | | | | |
| [Azure (`azure`)](https://docs.litellm.ai/docs/providers/azure) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | |
| [Azure AI (`azure_ai`)](https://docs.litellm.ai/docs/providers/azure_ai) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | |
| [Azure Text (`azure_text`)](https://docs.litellm.ai/docs/providers/azure) | ✅ | ✅ | ✅ | | | ✅ | ✅ | ✅ | ✅ | |
| [Baseten (`baseten`)](https://docs.litellm.ai/docs/providers/baseten) | ✅ | ✅ | ✅ | | | | | | | |
| [Bytez (`bytez`)](https://docs.litellm.ai/docs/providers/bytez) | ✅ | ✅ | ✅ | | | | | | | |
| [Cerebras (`cerebras`)](https://docs.litellm.ai/docs/providers/cerebras) | ✅ | ✅ | ✅ | | | | | | | |
| [Clarifai (`clarifai`)](https://docs.litellm.ai/docs/providers/clarifai) | ✅ | ✅ | ✅ | | | | | | | |
| [Cloudflare AI Workers (`cloudflare`)](https://docs.litellm.ai/docs/providers/cloudflare_workers) | ✅ | ✅ | ✅ | | | | | | | |
| [Codestral (`codestral`)](https://docs.litellm.ai/docs/providers/codestral) | ✅ | ✅ | ✅ | | | | | | | |
| [Cohere (`cohere`)](https://docs.litellm.ai/docs/providers/cohere) | ✅ | ✅ | ✅ | ✅ | | | | | | ✅ |
| [Cohere Chat (`cohere_chat`)](https://docs.litellm.ai/docs/providers/cohere) | ✅ | ✅ | ✅ | | | | | | | |
| [CometAPI (`cometapi`)](https://docs.litellm.ai/docs/providers/cometapi) | ✅ | ✅ | ✅ | ✅ | | | | | | |
| [CompactifAI (`compactifai`)](https://docs.litellm.ai/docs/providers/compactifai) | ✅ | ✅ | ✅ | | | | | | | |
| [Custom (`custom`)](https://docs.litellm.ai/docs/providers/custom_llm_server) | ✅ | ✅ | ✅ | | | | | | | |
| [Custom OpenAI (`custom_openai`)](https://docs.litellm.ai/docs/providers/openai_compatible) | ✅ | ✅ | ✅ | | | ✅ | ✅ | ✅ | ✅ | |
| [Dashscope (`dashscope`)](https://docs.litellm.ai/docs/providers/dashscope) | ✅ | ✅ | ✅ | | | | | | | |
| [Databricks (`databricks`)](https://docs.litellm.ai/docs/providers/databricks) | ✅ | ✅ | ✅ | | | | | | | |
| [DataRobot (`datarobot`)](https://docs.litellm.ai/docs/providers/datarobot) | ✅ | ✅ | ✅ | | | | | | | |
| [Deepgram (`deepgram`)](https://docs.litellm.ai/docs/providers/deepgram) | ✅ | ✅ | ✅ | | | ✅ | | | | |
| [DeepInfra (`deepinfra`)](https://docs.litellm.ai/docs/providers/deepinfra) | ✅ | ✅ | ✅ | | | | | | | |
| [Deepseek (`deepseek`)](https://docs.litellm.ai/docs/providers/deepseek) | ✅ | ✅ | ✅ | | | | | | | |
| [ElevenLabs (`elevenlabs`)](https://docs.litellm.ai/docs/providers/elevenlabs) | ✅ | ✅ | ✅ | | | | ✅ | | | |
| [Empower (`empower`)](https://docs.litellm.ai/docs/providers/empower) | ✅ | ✅ | ✅ | | | | | | | |
| [Fal AI (`fal_ai`)](https://docs.litellm.ai/docs/providers/fal_ai) | ✅ | ✅ | ✅ | | ✅ | | | | | |
| [Featherless AI (`featherless_ai`)](https://docs.litellm.ai/docs/providers/featherless_ai) | ✅ | ✅ | ✅ | | | | | | | |
| [Fireworks AI (`fireworks_ai`)](https://docs.litellm.ai/docs/providers/fireworks_ai) | ✅ | ✅ | ✅ | | | | | | | |
| [FriendliAI (`friendliai`)](https://docs.litellm.ai/docs/providers/friendliai) | ✅ | ✅ | ✅ | | | | | | | |
| [Galadriel (`galadriel`)](https://docs.litellm.ai/docs/providers/galadriel) | ✅ | ✅ | ✅ | | | | | | | |
| [GitHub Copilot (`github_copilot`)](https://docs.litellm.ai/docs/providers/github_copilot) | ✅ | ✅ | ✅ | | | | | | | |
| [GitHub Models (`github`)](https://docs.litellm.ai/docs/providers/github) | ✅ | ✅ | ✅ | | | | | | | |
| [Google - PaLM](https://docs.litellm.ai/docs/providers/palm) | ✅ | ✅ | ✅ | | | | | | | |
| [Google - Vertex AI (`vertex_ai`)](https://docs.litellm.ai/docs/providers/vertex) | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | |
| [Google AI Studio - Gemini (`gemini`)](https://docs.litellm.ai/docs/providers/gemini) | ✅ | ✅ | ✅ | | | | | | | |
| [GradientAI (`gradient_ai`)](https://docs.litellm.ai/docs/providers/gradient_ai) | ✅ | ✅ | ✅ | | | | | | | |
| [Groq AI (`groq`)](https://docs.litellm.ai/docs/providers/groq) | ✅ | ✅ | ✅ | | | | | | | |
| [Heroku (`heroku`)](https://docs.litellm.ai/docs/providers/heroku) | ✅ | ✅ | ✅ | | | | | | | |
| [Hosted VLLM (`hosted_vllm`)](https://docs.litellm.ai/docs/providers/vllm) | ✅ | ✅ | ✅ | | | | | | | |
| [Huggingface (`huggingface`)](https://docs.litellm.ai/docs/providers/huggingface) | ✅ | ✅ | ✅ | ✅ | | | | | | ✅ |
| [Hyperbolic (`hyperbolic`)](https://docs.litellm.ai/docs/providers/hyperbolic) | ✅ | ✅ | ✅ | | | | | | | |
| [IBM - Watsonx.ai (`watsonx`)](https://docs.litellm.ai/docs/providers/watsonx) | ✅ | ✅ | ✅ | ✅ | | | | | | |
| [Infinity (`infinity`)](https://docs.litellm.ai/docs/providers/infinity) | | | | ✅ | | | | | | |
| [Jina AI (`jina_ai`)](https://docs.litellm.ai/docs/providers/jina_ai) | | | | ✅ | | | | | | |
| [Lambda AI (`lambda_ai`)](https://docs.litellm.ai/docs/providers/lambda_ai) | ✅ | ✅ | ✅ | | | | | | | |
| [Lemonade (`lemonade`)](https://docs.litellm.ai/docs/providers/lemonade) | ✅ | ✅ | ✅ | | | | | | | |
| [LiteLLM Proxy (`litellm_proxy`)](https://docs.litellm.ai/docs/providers/litellm_proxy) | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | |
| [Llamafile (`llamafile`)](https://docs.litellm.ai/docs/providers/llamafile) | ✅ | ✅ | ✅ | | | | | | | |
| [LM Studio (`lm_studio`)](https://docs.litellm.ai/docs/providers/lm_studio) | ✅ | ✅ | ✅ | | | | | | | |
| [Maritalk (`maritalk`)](https://docs.litellm.ai/docs/providers/maritalk) | ✅ | ✅ | ✅ | | | | | | | |
| [Meta - Llama API (`meta_llama`)](https://docs.litellm.ai/docs/providers/meta_llama) | ✅ | ✅ | ✅ | | | | | | | |
| [Mistral AI API (`mistral`)](https://docs.litellm.ai/docs/providers/mistral) | ✅ | ✅ | ✅ | ✅ | | | | | | |
| [Moonshot (`moonshot`)](https://docs.litellm.ai/docs/providers/moonshot) | ✅ | ✅ | ✅ | | | | | | | |
| [Morph (`morph`)](https://docs.litellm.ai/docs/providers/morph) | ✅ | ✅ | ✅ | | | | | | | |
| [Nebius AI Studio (`nebius`)](https://docs.litellm.ai/docs/providers/nebius) | ✅ | ✅ | ✅ | ✅ | | | | | | |
| [NLP Cloud (`nlp_cloud`)](https://docs.litellm.ai/docs/providers/nlp_cloud) | ✅ | ✅ | ✅ | | | | | | | |
| [Novita AI (`novita`)](https://novita.ai/models/llm?utm_source=github_litellm&utm_medium=github_readme&utm_campaign=github_link) | ✅ | ✅ | ✅ | | | | | | | |
| [Nscale (`nscale`)](https://docs.litellm.ai/docs/providers/nscale) | ✅ | ✅ | ✅ | | | | | | | |
| [Nvidia NIM (`nvidia_nim`)](https://docs.litellm.ai/docs/providers/nvidia_nim) | ✅ | ✅ | ✅ | | | | | | | |
| [OCI (`oci`)](https://docs.litellm.ai/docs/providers/oci) | ✅ | ✅ | ✅ | | | | | | | |
| [Ollama (`ollama`)](https://docs.litellm.ai/docs/providers/ollama) | ✅ | ✅ | ✅ | ✅ | | | | | | |
| [Ollama Chat (`ollama_chat`)](https://docs.litellm.ai/docs/providers/ollama) | ✅ | ✅ | ✅ | | | | | | | |
| [Oobabooga (`oobabooga`)](https://docs.litellm.ai/docs/providers/openai_compatible) | ✅ | ✅ | ✅ | | | ✅ | ✅ | ✅ | ✅ | |
| [OpenAI (`openai`)](https://docs.litellm.ai/docs/providers/openai) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | |
| [OpenAI-like (`openai_like`)](https://docs.litellm.ai/docs/providers/openai_compatible) | | | | ✅ | | | | | | |
| [OpenRouter (`openrouter`)](https://docs.litellm.ai/docs/providers/openrouter) | ✅ | ✅ | ✅ | | | | | | | |
| [OVHCloud AI Endpoints (`ovhcloud`)](https://docs.litellm.ai/docs/providers/ovhcloud) | ✅ | ✅ | ✅ | | | | | | | |
| [Perplexity AI (`perplexity`)](https://docs.litellm.ai/docs/providers/perplexity) | ✅ | ✅ | ✅ | | | | | | | |
| [Petals (`petals`)](https://docs.litellm.ai/docs/providers/petals) | ✅ | ✅ | ✅ | | | | | | | |
| [Predibase (`predibase`)](https://docs.litellm.ai/docs/providers/predibase) | ✅ | ✅ | ✅ | | | | | | | |
| [Recraft (`recraft`)](https://docs.litellm.ai/docs/providers/recraft) | | | | | ✅ | | | | | |
| [Replicate (`replicate`)](https://docs.litellm.ai/docs/providers/replicate) | ✅ | ✅ | ✅ | | | | | | | |
| [Sagemaker Chat (`sagemaker_chat`)](https://docs.litellm.ai/docs/providers/aws_sagemaker) | ✅ | ✅ | ✅ | | | | | | | |
| [Sambanova (`sambanova`)](https://docs.litellm.ai/docs/providers/sambanova) | ✅ | ✅ | ✅ | | | | | | | |
| [Snowflake (`snowflake`)](https://docs.litellm.ai/docs/providers/snowflake) | ✅ | ✅ | ✅ | | | | | | | |
| [Text Completion Codestral (`text-completion-codestral`)](https://docs.litellm.ai/docs/providers/codestral) | ✅ | ✅ | ✅ | | | | | | | |
| [Text Completion OpenAI (`text-completion-openai`)](https://docs.litellm.ai/docs/providers/text_completion_openai) | ✅ | ✅ | ✅ | | | ✅ | ✅ | ✅ | ✅ | |
| [Together AI (`together_ai`)](https://docs.litellm.ai/docs/providers/togetherai) | ✅ | ✅ | ✅ | | | | | | | |
| [Topaz (`topaz`)](https://docs.litellm.ai/docs/providers/topaz) | ✅ | ✅ | ✅ | | | | | | | |
| [Triton (`triton`)](https://docs.litellm.ai/docs/providers/triton-inference-server) | ✅ | ✅ | ✅ | | | | | | | |
| [V0 (`v0`)](https://docs.litellm.ai/docs/providers/v0) | ✅ | ✅ | ✅ | | | | | | | |
| [Vercel AI Gateway (`vercel_ai_gateway`)](https://docs.litellm.ai/docs/providers/vercel_ai_gateway) | ✅ | ✅ | ✅ | | | | | | | |
| [VLLM (`vllm`)](https://docs.litellm.ai/docs/providers/vllm) | ✅ | ✅ | ✅ | | | | | | | |
| [Volcengine (`volcengine`)](https://docs.litellm.ai/docs/providers/volcano) | ✅ | ✅ | ✅ | | | | | | | |
| [Voyage AI (`voyage`)](https://docs.litellm.ai/docs/providers/voyage) | | | | ✅ | | | | | | |
| [WandB Inference (`wandb`)](https://docs.litellm.ai/docs/providers/wandb_inference) | ✅ | ✅ | ✅ | | | | | | | |
| [Watsonx Text (`watsonx_text`)](https://docs.litellm.ai/docs/providers/watsonx) | ✅ | ✅ | ✅ | | | | | | | |
| [xAI (`xai`)](https://docs.litellm.ai/docs/providers/xai) | ✅ | ✅ | ✅ | | | | | | | |
| [Xinference (`xinference`)](https://docs.litellm.ai/docs/providers/xinference) | | | | ✅ | | | | | | |
[**Read the Docs**](https://docs.litellm.ai/docs/)
docs/my-website/docs/proxy/config_settings.md
+4
View File
@@ -361,6 +361,10 @@ router_settings:
| AIOHTTP_TRUST_ENV | Flag to enable aiohttp trust environment. When this is set to True, aiohttp will respect HTTP(S)_PROXY env vars. **Default is False**
| AIOHTTP_TTL_DNS_CACHE | DNS cache time-to-live for aiohttp in seconds. **Default is 300**
| ALLOWED_EMAIL_DOMAINS | List of email domains allowed for access
| APSCHEDULER_COALESCE | Whether to combine multiple pending executions of a job into one. **Default is False**
| APSCHEDULER_MAX_INSTANCES | Maximum number of concurrent instances of each job. **Default is 1**
| APSCHEDULER_MISFIRE_GRACE_TIME | Grace time in seconds for misfired jobs. **Default is 1**
| APSCHEDULER_REPLACE_EXISTING | Whether to replace existing jobs with the same ID. **Default is False**
| ARIZE_API_KEY | API key for Arize platform integration
| ARIZE_SPACE_KEY | Space key for Arize platform
| ARGILLA_BATCH_SIZE | Batch size for Argilla logging
tests/documentation_tests/test_readme_providers.py
+179
View File
@@ -0,0 +1,179 @@
"""
Test to ensure all providers in LlmProviders enum are documented in README.md
"""
import os
import re
from litellm.types.utils import LlmProviders
# Define paths
readme_path = "./README.md"
# Providers that shouldn't be required in README
# (specialized tools, observability, database providers that aren't LLM providers)
EXCLUDED_PROVIDERS = {
"aiohttp_openai", # internal http variant
"langfuse", # observability, not LLM provider
"humanloop", # observability, not LLM provider
"pg_vector", # database, not LLM provider
"dotprompt", # prompt management, not provider
"vertex_ai_beta", # beta variant, not needed in main table
}
def get_enum_providers():
"""
Get all provider values from LlmProviders enum, excluding internal variants.
"""
providers = set()
for provider in LlmProviders:
provider_value = provider.value
if provider_value not in EXCLUDED_PROVIDERS:
providers.add(provider_value)
return providers
def get_readme_providers():
"""
Extract provider slugs from README.md provider table.
Looks for provider slugs in backticks within parentheses, e.g.:
[OpenAI (`openai`)](url) -> extracts "openai"
"""
providers = set()
try:
with open(readme_path, "r", encoding="utf-8") as f:
content = f.read()
# Find the supported providers table
# Look for the section starting with "## Supported Providers"
providers_section = re.search(
r"## Supported Providers.*?\n\n(.*?)(?=\n##|\Z)",
content,
re.DOTALL | re.MULTILINE,
)
if providers_section:
table_content = providers_section.group(1)
# Extract provider slugs from backticks in parentheses: (`slug`)
provider_slug_pattern = re.compile(r"\(`([^`]+)`\)")
matches = provider_slug_pattern.findall(table_content)
providers.update(matches)
else:
raise Exception("Could not find 'Supported Providers' section in README.md")
except FileNotFoundError:
raise Exception(f"README.md not found at {readme_path}")
except Exception as e:
raise Exception(f"Error reading README.md: {e}")
return providers
def get_readme_provider_names():
"""
Extract provider display names from README.md provider table in order.
Returns a list of provider names as they appear in the table.
"""
provider_names = []
try:
with open(readme_path, "r", encoding="utf-8") as f:
content = f.read()
# Find the supported providers table
providers_section = re.search(
r"## Supported Providers.*?\n\n(.*?)(?=\n\[|\n##|\Z)",
content,
re.DOTALL | re.MULTILINE,
)
if providers_section:
table_content = providers_section.group(1)
# Extract provider names from table rows that start with |
# Split by lines and process each line
for line in table_content.split('\n'):
# Only process lines that are table rows (start with |)
if line.strip().startswith('|') and '[' in line:
# Extract provider name from: | [Provider Name (...)](...) |
match = re.search(r'\|\s*\[([^\]]+)\]\(', line)
if match:
provider_name = match.group(1)
# Skip header row and separator row
if provider_name != "Provider" and not provider_name.startswith('-'):
provider_names.append(provider_name)
else:
raise Exception("Could not find 'Supported Providers' section in README.md")
except FileNotFoundError:
raise Exception(f"README.md not found at {readme_path}")
except Exception as e:
raise Exception(f"Error reading README.md: {e}")
return provider_names
def test_all_providers_documented():
"""
Test that all providers in LlmProviders enum are documented in README.md.
Verifies that provider slugs in the enum match the slugs shown in backticks
in the README provider table.
"""
enum_providers = get_enum_providers()
readme_providers = get_readme_providers()
print(f"\nProvider slugs in LlmProviders enum (filtered): {sorted(enum_providers)}")
print(f"\nProvider slugs in README.md: {sorted(readme_providers)}")
# Find undocumented providers
undocumented = enum_providers - readme_providers
if undocumented:
raise AssertionError(
f"\nProvider slugs not documented in README.md: {sorted(undocumented)}\n"
f"Please add these providers to the README.md provider table with their slugs in backticks.\n"
f"Example: [Provider Name (`slug`)](url)"
)
else:
print(f"\n✓ All {len(enum_providers)} provider slugs are documented in README.md")
def test_providers_alphabetically_ordered():
"""
Test that providers in README.md are listed in alphabetical order.
"""
provider_names = get_readme_provider_names()
if not provider_names:
raise AssertionError("No provider names found in README.md")
# Create a sorted version for comparison
sorted_names = sorted(provider_names, key=str.lower)
print(f"\nFound {len(provider_names)} providers in README.md")
# Check if the list is alphabetically ordered
out_of_order = []
for i, (actual, expected) in enumerate(zip(provider_names, sorted_names)):
if actual != expected:
out_of_order.append({
"position": i + 1,
"actual": actual,
"expected": expected
})
if out_of_order:
error_msg = "\nProviders are not in alphabetical order:\n"
for item in out_of_order[:10]: # Show first 10 issues
error_msg += f" Position {item['position']}: Found '{item['actual']}', expected '{item['expected']}'\n"
if len(out_of_order) > 10:
error_msg += f" ... and {len(out_of_order) - 10} more issues\n"
raise AssertionError(error_msg)
else:
print(f"✓ All {len(provider_names)} providers are in alphabetical order")
if __name__ == "__main__":
test_all_providers_documented()
test_providers_alphabetically_ordered()