- Drop multimodal content for non-OpenAI-family providers (Google/Anthropic
raised on OpenAI image_url parts) so multimodal requests degrade to text
instead of returning 500.
- Stateless continuations (no conversation_id) default save_conversation to
False, avoiding an orphan conversation with an empty question on every tool
round (stateful continuations still default True).
- Only strip leaked reasoning from content for structured requests
(response_format / json_schema / json_object); legitimate answers that mention
the marker text are no longer corrupted.
- Forward sampling params (temperature, max_tokens, ...) on continuation turns.
- An explicit json_object request clears an agent-configured json_schema so it
isn't silently overridden.
- Drop max_tokens when max_completion_tokens is also sent (OpenAI rejects both).
- Don't build a chatcmpl-None completion id from the placeholder "None" id event.
OpenAI-compatible clients send multimodal user turns as a `content` array of
typed parts. translate_request previously assigned the array straight to the
question, breaking the string-only retrieval / token-budgeting / history paths
(HTTP 500). Now:
- content_to_text() extracts text from content arrays for the question,
history and system prompt, so the string paths work unchanged.
- The full content array (text + image_url parts) is preserved as
`multimodal_content`, threaded to the agent and emitted as the final user
message so images reach the model. Token budgeting uses the text only.
The content array (incl. image_url) now reaches the LLM call intact; images
render for vision-capable models. A text-only upstream model will reject the
image_url variant, as expected.
- Stateless tool continuation. OpenAI-compatible clients (opencode, etc.)
resend the full messages array — system, user, assistant(tool_calls),
tool(results) — but no conversation_id, so the prior
"conversation_id required for tool continuation" 400 broke every tool call.
When no conversation_id is present, rebuild the agent + pending tool calls +
tool results directly from the resent messages
(StreamProcessor.build_continuation_from_messages) instead of loading
server-side pending_tool_state, and call gen_continuation.
- Forward OpenAI sampling params (temperature, max_tokens,
max_completion_tokens, top_p, frequency_penalty, presence_penalty, stop,
seed) from the request to the LLM gen call; the agent otherwise uses its
configured defaults.
Make the OpenAI-compatible Chat Completions endpoint honor per-request
Structured Outputs and keep it OpenAI-compatible.
- translate_request now forwards the request's `response_format` (json_schema)
or a `response_schema` convenience field to the agent as its json_schema,
overriding the agent-configured schema for that request.
- Honor `response_format.json_schema.strict` (default true); strict:false
passes the schema through without forcing additionalProperties:false /
all-required (OpenAI's lenient mode).
- Support `response_format {"type":"json_object"}` via the provider's native
JSON mode.
- Keep `content` clean: some models echo their reasoning into content as
stringified `{'type': 'thought', ...}` reprs when response_format is set.
Strip those from content and reroute them to `reasoning_content` (OpenAI
never puts reasoning in content), for both streaming and non-streaming.
Flow: translate_request -> StreamProcessor._configure_agent ->
Agent.json_schema / json_schema_strict / json_object -> _llm_gen ->
prepare_structured_output_format(schema, strict).
* feat: implement WorkflowAgent and GraphExecutor for workflow management and execution
* refactor: workflow schemas and introduce WorkflowEngine
- Updated schemas in `schemas.py` to include new agent types and configurations.
- Created `WorkflowEngine` class in `workflow_engine.py` to manage workflow execution.
- Enhanced `StreamProcessor` to handle workflow-related data.
- Added new routes and utilities for managing workflows in the user API.
- Implemented validation and serialization functions for workflows.
- Established MongoDB collections and indexes for workflows and related entities.
* refactor: improve WorkflowAgent documentation and update type hints in WorkflowEngine
* feat: workflow builder and managing in frontend
- Added new endpoints for workflows in `endpoints.ts`.
- Implemented `getWorkflow`, `createWorkflow`, and `updateWorkflow` methods in `userService.ts`.
- Introduced new UI components for alerts, buttons, commands, dialogs, multi-select, popovers, and selects.
- Enhanced styling in `index.css` with new theme variables and animations.
- Refactored modal components for better layout and styling.
- Configured TypeScript paths and Vite aliases for cleaner imports.
* feat: add workflow preview component and related state management
- Implemented WorkflowPreview component for displaying workflow execution.
- Created WorkflowPreviewSlice for managing workflow preview state, including queries and execution steps.
- Added WorkflowMiniMap for visual representation of workflow nodes and their statuses.
- Integrated conversation handling with the ability to fetch answers and manage query states.
- Introduced reusable Sheet component for UI overlays.
- Updated Redux store to include workflowPreview reducer.
* feat: enhance workflow execution details and state management in WorkflowEngine and WorkflowPreview
* feat: enhance workflow components with improved UI and functionality
- Updated WorkflowPreview to allow text truncation for better display of long names.
- Enhanced BaseNode with connectable handles and improved styling for better visibility.
- Added MobileBlocker component to inform users about desktop requirements for the Workflow Builder.
- Introduced PromptTextArea component for improved variable insertion and search functionality, including upstream variable extraction and context addition.
* feat(workflow): add owner validation and graph version support
* fix: ruff lint
---------
Co-authored-by: Alex <a@tushynski.me>
* feat: Implement model registry and capabilities for multi-provider support
- Added ModelRegistry to manage available models and their capabilities.
- Introduced ModelProvider enum for different LLM providers.
- Created ModelCapabilities dataclass to define model features.
- Implemented methods to load models based on API keys and settings.
- Added utility functions for model management in model_utils.py.
- Updated settings.py to include provider-specific API keys.
- Refactored LLM classes (Anthropic, OpenAI, Google, etc.) to utilize new model registry.
- Enhanced utility functions to handle token limits and model validation.
- Improved code structure and logging for better maintainability.
* feat: Add model selection feature with API integration and UI component
* feat: Add model selection and default model functionality in agent management
* test: Update assertions and formatting in stream processing tests
* refactor(llm): Standardize model identifier to model_id
* fix tests
---------
Co-authored-by: Alex <a@tushynski.me>
Alex