Commit Graph

3 Commits

Author SHA1 Message Date
Andrew Maguire bd15250960 fix: Add atexit handlers to flush callbacks for async completions (#16487)
Fixes #16486

## Problem
Callbacks configured via litellm.success_callback (e.g., PostHog, LangSmith)
were not being invoked for litellm.acompletion() in short-lived scripts.
The callbacks worked correctly for synchronous completions but async
completions would queue callbacks that were lost when the script exited
before GLOBAL_LOGGING_WORKER could process them.

Root cause: asyncio.run() closes the event loop immediately after the
async function completes, preventing the background worker from processing
queued callbacks.

## Solution
Implemented a two-level atexit handler approach:

1. GLOBAL_LOGGING_WORKER atexit handler (logging_worker.py):
   - Processes remaining callbacks from queue before exit
   - Creates new event loop to run pending coroutines synchronously
   - Applies time and iteration limits to prevent blocking shutdown

2. Integration-specific atexit handlers (posthog.py as example):
   - Flushes internal queue to external service
   - Uses synchronous HTTP client for reliable delivery
   - Each integration needs its own handler due to varying sync APIs

## Changes
- litellm/litellm_core_utils/logging_worker.py:
  - Added _flush_on_exit() method
  - Registered atexit handler in __init__
  - Processes up to MAX_ITERATIONS_TO_CLEAR_QUEUE events
  - Time-limited to MAX_TIME_TO_CLEAR_QUEUE seconds

- litellm/integrations/posthog.py:
  - Added _flush_on_exit() method
  - Registered atexit handler in __init__
  - Groups events by credentials for batch sending
  - Uses sync_client for reliable HTTP delivery

- tests/logging_callback_tests/test_posthog.py:
  - Added test_async_callback_atexit_handler_exists()
  - Added test_posthog_atexit_flushes_internal_queue()
  - Added test_sync_callback_not_affected_by_atexit()

## Testing
- All existing tests pass
- Manual end-to-end testing confirms fix:
  - Async events now arrive in PostHog
  - Sync events continue working (no regression)
- Unit tests verify atexit handlers registered and functional

## Impact
- Fixes async callback delivery for ALL integrations using GLOBAL_LOGGING_WORKER
- No breaking changes - only adds missing functionality
- Sync path unchanged - no performance impact
2025-11-11 19:12:53 -08:00
Carlos Marchal dd560f792e feat: posthog per request api key 2025-10-09 18:32:11 +02:00
Carlos Marchal e168161e64 Feat/add posthog observability (#14610)
* feat: add posthog observability

* docs: add posthog logging docs

* docs: posthog integration in proxy mode
2025-09-17 08:24:04 -07:00