mirror of
https://github.com/tiennm99/litellm.git
synced 2026-08-02 20:22:10 +00:00
mateo-berri
dc639cd223
The old gate compared CHAR count to gemini's 1024-TOKEN cache minimum on the false premise that one token spans at least one character; BPE tokens floor at one BYTE, so 700 CJK or 400 emoji chars passed the gate while v1's token_counter could reach 1024 and fire the cachedContents network call (silent caching loss + wire-body divergence, flag-on). The gate now computes a conservative upper bound on whatever v1 could count: UTF-8 byte length of every text/thinking/tool_result/tool_use-args surface across the WHOLE request (a superset of v1's continuous cached block, which spans first-to-last marked message inclusive), plus a fixed per-message margin for token_counter's overhead, and fails closed on ANY media block when a marker is present (v1 charges 250 tokens per image at zero text bytes). Quirk rows pin the typed fallback for CJK, emoji, and unmarked-image-beside-marker shapes (all three fail against the old gate), and a sub-limit CJK quirk pins that small international markers still serialize identically to v1.
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/test_litellm
This folder can only run mock tests.