mirror of
https://github.com/tiennm99/litellm.git
synced 2026-07-14 15:05:48 +00:00
Andrew Maguire
bd15250960
Fixes #16486 ## Problem Callbacks configured via litellm.success_callback (e.g., PostHog, LangSmith) were not being invoked for litellm.acompletion() in short-lived scripts. The callbacks worked correctly for synchronous completions but async completions would queue callbacks that were lost when the script exited before GLOBAL_LOGGING_WORKER could process them. Root cause: asyncio.run() closes the event loop immediately after the async function completes, preventing the background worker from processing queued callbacks. ## Solution Implemented a two-level atexit handler approach: 1. GLOBAL_LOGGING_WORKER atexit handler (logging_worker.py): - Processes remaining callbacks from queue before exit - Creates new event loop to run pending coroutines synchronously - Applies time and iteration limits to prevent blocking shutdown 2. Integration-specific atexit handlers (posthog.py as example): - Flushes internal queue to external service - Uses synchronous HTTP client for reliable delivery - Each integration needs its own handler due to varying sync APIs ## Changes - litellm/litellm_core_utils/logging_worker.py: - Added _flush_on_exit() method - Registered atexit handler in __init__ - Processes up to MAX_ITERATIONS_TO_CLEAR_QUEUE events - Time-limited to MAX_TIME_TO_CLEAR_QUEUE seconds - litellm/integrations/posthog.py: - Added _flush_on_exit() method - Registered atexit handler in __init__ - Groups events by credentials for batch sending - Uses sync_client for reliable HTTP delivery - tests/logging_callback_tests/test_posthog.py: - Added test_async_callback_atexit_handler_exists() - Added test_posthog_atexit_flushes_internal_queue() - Added test_sync_callback_not_affected_by_atexit() ## Testing - All existing tests pass - Manual end-to-end testing confirms fix: - Async events now arrive in PostHog - Sync events continue working (no regression) - Unit tests verify atexit handlers registered and functional ## Impact - Fixes async callback delivery for ALL integrations using GLOBAL_LOGGING_WORKER - No breaking changes - only adds missing functionality - Sync path unchanged - no performance impact