Commit Graph
3 Commits
Author SHA1 Message Date
Ishaan Jaffer 3847a59d79 style: black format test_model_param_helper.py 2026-04-15 18:18:52 -07:00
michelligabriele 85497740ff fix(caching): add Responses API params to cache key allow-list 2026-04-14 05:40:42 +02:00
ryan-crabbeandGitHub 7a6820defa perf: cache _get_relevant_args_to_use_for_logging() at module level (#20077)
* perf: cache _get_relevant_args_to_use_for_logging() as module-level frozenset

The set of valid LLM API parameter names for logging was being rebuilt
on every request from 8 OpenAI SDK type annotations + set operations.
Since these are static TypedDict annotations that never change at
runtime, compute once at import time and store as a class-level
frozenset.

Line profiler: get_standard_logging_model_parameters() dropped from
774ms to 77ms across 12K calls (90% reduction, ~25µs/req saved).

* test: add tests for cached ModelParamHelper logging args

Verify cached frozenset matches dynamic computation and that
prompt content keys (messages, prompt, input) are excluded from
logged model parameters.
2026-02-02 10:54:49 -08:00