mirror of
https://github.com/tiennm99/litellm.git
synced 2026-07-16 06:16:57 +00:00
Igal BoxermanandGitHub
6bb63525db
* fix(generic-guardrail-api): fix SerializationIterator error on multimodal requests
When sending multimodal messages (with images) through the Generic Guardrail API,
the `model_dump()` call fails with "Object of type SerializationIterator is not
JSON serializable" error.
Root cause: The `ChatCompletionAssistantMessage` type defines `content` as an
`Iterable` (not just `List`), and Pydantic's `model_dump()` creates a
`SerializationIterator` for iterables which is not JSON serializable.
Fix: Use `model_dump(mode="json")` which properly converts all iterables to
lists and ensures all complex objects are JSON serializable.
* fix(guardrails): pass tools (function definitions) to guardrail inputs
The unified guardrail handler was not passing the `tools` parameter
(function definitions) from the request to the guardrail inputs.
This meant guardrails could not inspect or validate tool definitions.
Added extraction of `data.get("tools")` and inclusion in the
GenericGuardrailAPIInputs passed to `apply_guardrail()`.
* test(guardrails): add tests for tools passed to guardrail
Added tests verifying that tools (function definitions) are correctly
passed to guardrails in the unified guardrail handler:
- test_tools_passed_to_guardrail
- test_multiple_tools_passed_to_guardrail
- test_no_tools_in_request
- test_tools_and_tool_calls_both_passed
30 lines
716 B
YAML
30 lines
716 B
YAML
model_list:
|
|
- model_name: gpt-4
|
|
litellm_params:
|
|
model: openai/gpt-4
|
|
api_key: os.environ/OPENAI_API_KEY
|
|
|
|
- model_name: gpt-4o
|
|
litellm_params:
|
|
model: openai/gpt-4o
|
|
api_key: os.environ/OPENAI_API_KEY
|
|
|
|
- model_name: gpt-3.5-turbo
|
|
litellm_params:
|
|
model: openai/gpt-3.5-turbo
|
|
api_key: os.environ/OPENAI_API_KEY
|
|
|
|
guardrails:
|
|
- guardrail_name: thisispillar
|
|
litellm_params:
|
|
guardrail: generic_guardrail_api
|
|
mode: [pre_call, post_call]
|
|
api_base: os.environ/PILLAR_API_BASE
|
|
api_key: os.environ/PILLAR_API_KEY
|
|
default_on: true
|
|
additional_provider_specific_params:
|
|
plr_evidence: true
|
|
|
|
general_settings:
|
|
master_key: sk-1234
|