mirror of
https://github.com/tiennm99/litellm.git
synced 2026-08-03 14:23:28 +00:00
mateo-berri
3fe015ef6c
The cache-marker token bound is computed over the IR, but the inbound parse drops the OpenAI message 'name' field (v1's generateContent transform ignores it on the wire), so name bytes were invisible to the bound while v1's is_prompt_caching_valid_prompt token-counts them: 100 one-char marked messages with 80-char names measured v1=4603 tokens (>= 1024, context cache created) against a v2 bound of 900 (served, uncached). New providers/google_genai/guard.py falls back before parse when any message carries 'name' AND a cache_control marker is present (name bytes cannot be bounded post-IR), registered in _RAW_GUARDS for vertex_ai and gemini; vertex_anthropic needs no row (v1 context caching is gemini-only and the anthropic wire ignores name on both sides). The verifier's attack script now finds no unsound shape. Corpus: a name_beside_marker fallback row (fails with the guard unregistered, verified by stash) and a message_name_without_marker quirk row pinning that both sides drop bare 'name' identically. Differential regenerated: 297 IDENTICAL / 55 FALLBACK / 0 divergent.
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/test_litellm
This folder can only run mock tests.