Sameer Kankute 4dc645fc33 feat(polling): check rate limits before creating polling ID ...

Move pre-call checks (rate limits, guardrails, budget) to run BEFORE
polling ID creation in the background streaming flow. This prevents the
edge case where a rate-limited request receives a polling ID that
immediately fails.

Changes:
- Add skip_pre_call_logic parameter to base_process_llm_request to allow
  skipping pre-call checks (avoiding double-counting of RPM/parallel requests)
- Run common_processing_pre_call_logic before generating polling ID in the
  responses API endpoint. If rate limits/guardrails fail, return error
  immediately without creating a polling ID
- Background streaming task passes skip_pre_call_logic=True to avoid re-running
  pre-call checks that were already done before polling ID creation
- Add tests verifying skip_pre_call_logic parameter works correctly

Fixes the edge case where polling_via_cache would return a polling ID
for a request that immediately fails due to rate limiting.

2026-03-19 13:59:59 +05:30

..

endpoints.py

feat(polling): check rate limits before creating polling ID

2026-03-19 13:59:59 +05:30