Commit Graph
2 Commits
Author SHA1 Message Date
viettranx 32ced98f6a fix(tools): add extractor chain retry/timeout, fix openai_compat media routing
- ExtractorChain now supports per-entry retry (max_retries) and chain-level
  timeout (context.WithTimeout), matching MediaProviderChain pattern
- Low quality content breaks out of retry (not transient), errors retry
- Fix ResolveProviderType: skip generic "openai_compat" DB type so
  OpenRouter routes via name-based inference to correct /chat/completions
  endpoint instead of falling through to /images/generations (caused 404)
- Remove dead "openai_compat" entry from dbTypeToMediaType map
- Seed data: defuddle default max_retries=2
2026-03-20 22:26:48 +07:00
viettranx 38dfcf8bb0 feat(tools): add Defuddle extractor chain for web_fetch (#296)
Add Cloudflare Worker (fetch.goclaw.sh) as primary markdown extractor
with waterfall fallback to built-in HTML→Markdown converter.

Architecture:
- ExtractorChain pattern with quality gate (min 100 chars, 10 words)
- Settings stored in builtin_tools DB table (not config.json5)
- ResolveExtractorChain reads chain from context per-request
- InProcessExtractor delegates to fetchRawContent (full SSRF + domain
  policy checks on redirects)
- DefuddleExtractor with configurable base_url + timeout
- Seed default chain [defuddle, html-to-markdown] for new deployments
- Backfill migration for existing deployments

Web UI:
- Dedicated DnD extractor chain form on builtin tools page
- Drag-and-drop ordering, enable/disable per extractor
- Timeout + base URL config for Defuddle
- i18n support (en/vi/zh)
2026-03-20 14:21:46 +07:00