Commit Graph

1 Commits

Author SHA1 Message Date
ryan-crabbe 7a6820defa perf: cache _get_relevant_args_to_use_for_logging() at module level (#20077)
* perf: cache _get_relevant_args_to_use_for_logging() as module-level frozenset

The set of valid LLM API parameter names for logging was being rebuilt
on every request from 8 OpenAI SDK type annotations + set operations.
Since these are static TypedDict annotations that never change at
runtime, compute once at import time and store as a class-level
frozenset.

Line profiler: get_standard_logging_model_parameters() dropped from
774ms to 77ms across 12K calls (90% reduction, ~25µs/req saved).

* test: add tests for cached ModelParamHelper logging args

Verify cached frozenset matches dynamic computation and that
prompt content keys (messages, prompt, input) are excluded from
logged model parameters.
2026-02-02 10:54:49 -08:00