litellm

mirror of https://github.com/tiennm99/litellm.git synced 2026-07-18 22:17:30 +00:00

Files

T

Tim Elfrink c5eb22381d fix(bedrock): include cacheWriteInputTokens in prompt_tokens calculation

Fixes #15263

This PR fixes the cost calculation for Bedrock Anthropic models with prompt caching.

**Root Cause:**
PR #9838 incorrectly removed adding `cacheWriteInputTokens` to `prompt_tokens`
for Bedrock, based on the assumption that it would cause double counting (similar
to an Anthropic API issue). However, Bedrock's token structure is different:

- **Bedrock API**: `inputTokens`, `cacheReadInputTokens`, and `cacheWriteInputTokens`
  are ALL separate values that should be summed for total input tokens
- **Anthropic API**: Same structure - all three token types are separate

The fix in #9838 was later reverted for Anthropic (correctly re-adding
`cache_creation_input_tokens` to `prompt_tokens`), but Bedrock was never fixed.

**Changes:**
1. Re-add `cacheWriteInputTokens` to `input_tokens` in Bedrock transformation
2. Update test assertions to reflect correct behavior
3. Add regression test for prompt caching cost calculation
4. Fix typo in Anthropic transformation where `cache_creation_tokens` was
   incorrectly set to `cache_read_input_tokens`

**Testing:**
- All existing Bedrock transformation tests pass
- New test validates correct cost calculation with prompt caching
- Verified costs are non-negative and accurate

2025-10-07 20:28:46 +02:00

llm_cost_calc

fix(bedrock): include cacheWriteInputTokens in prompt_tokens calculation

2025-10-07 20:28:46 +02:00

prompt_templates

[Bug]: Fix Mimetype Resolution Error in Bedrock Document Understanding (#13309 )

2025-08-05 17:07:10 -07:00

specialty_caches

[Performance] Reduce complexity of InMemoryCache.evict_cache from O(n*log(n)) to O(log(n)) (#15000 )

2025-09-30 16:49:35 -07:00

__init__.py

[Feat] Add github co-pilot as a new LLM API provider (#12325 )