fix: update success-event handler to track tokens for deployment-default limits

async_log_success_event only updated the per-model cache counter when
model_rpm_limit / model_tpm_limit were present in key metadata or
model_max_budget was set. For the new deployment-default path
(default_api_key_tpm_limit / default_api_key_rpm_limit), none of those
conditions held, so current_tpm stayed at zero and tpm enforcement was
never applied across multiple requests.

Extend the guard condition to also trigger when the model group has a
deployment-default tpm or rpm limit, and import the two helpers at
module level.

Co-Authored-By: Claude (claude-sonnet-4-6) <noreply@anthropic.com>
This commit is contained in:
Ephrim Stanley
2026-03-19 01:59:41 -04:00
co-authored by Claude
parent b90f520748
commit 48cb4a8343
@@ -14,6 +14,8 @@ from litellm.integrations.custom_logger import CustomLogger
from litellm.litellm_core_utils.core_helpers import _get_parent_otel_span_from_kwargs
from litellm.proxy._types import CommonProxyErrors, CurrentItemRateLimit, UserAPIKeyAuth
from litellm.proxy.auth.auth_utils import (
_get_deployment_default_rpm_limit,
_get_deployment_default_tpm_limit,
get_key_model_rpm_limit,
get_key_model_tpm_limit,
)
@@ -546,6 +548,8 @@ class _PROXY_MaxParallelRequestsHandler(CustomLogger):
"model_rpm_limit" in user_api_key_metadata
or "model_tpm_limit" in user_api_key_metadata
or user_api_key_model_max_budget is not None
or _get_deployment_default_tpm_limit(model_group) is not None
or _get_deployment_default_rpm_limit(model_group) is not None
)
):
request_count_api_key = (