* add openssl in apk install in runtime stage in dockerfile.non_rootdocker-compose logs -f litellm
* Improve Docker-compose.yaml for local debugging
---------
Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com>
* fix(guardrails): Fix PANW Prisma AIRS post-call hook method name
- Changed async_post_call_hook to async_post_call_success_hook to match proxy calling convention
- Added event_hook parameter to initialization to ensure proper hook registration
- Fixes post-call response scanning for PANW Prisma AIRS guardrails
Resolves issue where post-call hooks were not being invoked due to method name mismatch.
* Update PANW Prisma AIRS tests to use correct method name
* fix(router.py): add acompletion_streaming_iterator inside router
allows router to catch errors mid-stream for fallbacks
Work for https://github.com/BerriAI/litellm/issues/6532
* fix(router.py): working mid-stream fallbacks
* fix(router.py): more iterations
* fix(router.py): working mid-stream fallbacks with fallbacks set on router
* fix(router.py): pass prior content back in new request as assistant prefix message
* fix(router.py): add a system prompt to help guide non-prefix supporting models to use the continued text correctly
* fix(common_utils.py): support converting `prefix: true` for non-prefix supporting models
* fix: reduce LOC in function
* test(test_router.py): add unit tests for new function
* test: add basic unit test
* fix(router.py): ensure return type of fallback stream is compatible with CustomStreamWrapper
prevent client code from breaking
* fix: cleanup
* test: update test
* fix: fix linting error
* fix(router.py): return combined usage
ensures accurate usage tracking on clientside for stream w/ fallbacks
* fix(router.py): add acompletion_streaming_iterator inside router
allows router to catch errors mid-stream for fallbacks
Work for https://github.com/BerriAI/litellm/issues/6532
* fix(router.py): working mid-stream fallbacks
* fix(router.py): more iterations
* fix(router.py): working mid-stream fallbacks with fallbacks set on router
* fix(router.py): pass prior content back in new request as assistant prefix message
* fix(router.py): add a system prompt to help guide non-prefix supporting models to use the continued text correctly
* fix(common_utils.py): support converting `prefix: true` for non-prefix supporting models
* fix: reduce LOC in function
* test(test_router.py): add unit tests for new function
* test: add basic unit test
* fix(router.py): ensure return type of fallback stream is compatible with CustomStreamWrapper
prevent client code from breaking
* fix: cleanup
* test: update test
* fix: fix linting error
* fix(router.py): safe deep copy kwargs
OTEL adds a parent_otel_span which cannot be deepcopied
* fix: use safe deep copy in other places as well
* test: add script to check and ban copy.deepcopy of kwargs
enforce safe_deep_copy usage
* build(ui/): new component for adding model group alias on UI
* fix(proxy_server.py): support updating model_group_alias via /config/update
allows ui component to work
* fix(router.py): update model_group_alias in router settings based on db value
* fix: fix code qa error
The test was failing because it was trying to patch MAX_LANGFUSE_INITIALIZED_CLIENTS
at the wrong path. The constant is imported from litellm.constants into the langfuse
module namespace, so we need to use patch.object on the imported module reference.
Changes:
- Import langfuse module explicitly for patching
- Use patch.object instead of patch string path
- This fixes the AttributeError that was causing CI failures
* ensure original client is disconnected when re-creating
* test_recreate_prisma_client_successful_disconnect
* test_recreate_prisma_client_successful_disconnect
* disable background health checks for specific models
* test_background_health_check_skip_disabled_models
* Disable Background Health Checks For Specific Models
Krrish Dholakia