When calling Vertex AI Model Garden custom endpoints with service account
credentials (instead of ADC), the credentials were not passed to the
PredictionServiceClient, resulting in "default credentials not found" error.
- Move credential loading before cache check to ensure availability
- Pass credentials to sync/async PredictionServiceClient
- Add vertex_credentials param to async_completion and async_streaming
Closes#8597
* fix(anthropic): use dynamic max_tokens based on model
When users don't specify max_tokens in requests to Anthropic models,
LiteLLM now uses the correct max_output_tokens value from the model
pricing JSON instead of a hardcoded 4096.
This fixes truncated responses for Claude 3.5+ models which support
higher output limits (8192 for Claude 3.5, 128k for Claude 3.7, etc.)
Fixes#8835
* fix(anthropic): restore env var support for backwards compatibility
Keep DEFAULT_ANTHROPIC_CHAT_MAX_TOKENS as fallback when model is not
found in JSON, allowing users to configure via environment variable.
Add direct Stability AI REST API support for image generation endpoints.
This enables using Stability's SD3, SD3.5, and Stable Image models via
LiteLLM's OpenAI-compatible interface.
Changes:
- Add STABILITY provider to LlmProviders enum
- Create StabilityImageGenerationConfig with multipart/form-data support
- Add OpenAI size to Stability aspect_ratio mapping
- Register provider in ProviderConfigManager
- Add 9 Stability models to model_prices_and_context_window.json
- Add documentation at docs/providers/stability.md
- Add 25 unit tests
Supported models:
- stability/sd3, sd3-large, sd3-large-turbo, sd3-medium
- stability/sd3.5-large, sd3.5-large-turbo, sd3.5-medium
- stability/stable-image-ultra, stable-image-core
Fixes#15337
Perplexity API returns pre-calculated costs in `usage.cost.total_cost`
that include the `request_cost` (fixed per-request fee). LiteLLM was
ignoring this and calculating costs manually, resulting in ~27x
underreporting (e.g., $0.0002 vs actual $0.006).
Changes:
- Use `usage.cost.total_cost` from Perplexity response when available
- Fall back to manual calculation if cost object not present
- Add tests for both behaviors
- Filter skip_mcp_handler and other internal params in fallback_utils.py before calling acompletion
Fixes issue where internal parameters were being passed to provider APIs causing errors
- Remove deployment field from GCS bucket logger test metadata
Fixes model name mismatch where deployment field was overriding the model in logging
- Update Bedrock Titan test to use non-deprecated model (titan-text-express-v1)
Fixes test failure due to deprecated amazon.titan-text-lite-v1 model
- Add agent_id: null to expected JSON to match actual payload structure
- Fixes test_async_gcs_pub_sub_v1 test failure
- agent_id is an optional field in SpendLogsPayload that is always included (as null when not provided)
- Updated litellm/proxy/schema.prisma to include storage_backend and storage_url columns
- Updated litellm-proxy-extras/litellm_proxy_extras/schema.prisma to include storage_backend and storage_url columns
- Fixes database schema mismatch causing 'storage_backend column does not exist' errors
- Keeps all schema files in sync with root schema.prisma
Sameer Kankute