mirror of
https://github.com/tiennm99/litellm.git
synced 2026-07-19 00:19:23 +00:00
ryan-crabbeandGitHub
7a6820defa
* perf: cache _get_relevant_args_to_use_for_logging() as module-level frozenset The set of valid LLM API parameter names for logging was being rebuilt on every request from 8 OpenAI SDK type annotations + set operations. Since these are static TypedDict annotations that never change at runtime, compute once at import time and store as a class-level frozenset. Line profiler: get_standard_logging_model_parameters() dropped from 774ms to 77ms across 12K calls (90% reduction, ~25µs/req saved). * test: add tests for cached ModelParamHelper logging args Verify cached frozenset matches dynamic computation and that prompt content keys (messages, prompt, input) are excluded from logged model parameters.
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/test_litellm
This folder can only run mock tests.