viettranx 2e869f4ece feat(kg): add tsvector FTS, entity deduplication, and embedding fixes ...

- Replace ILIKE search with PostgreSQL tsvector/GIN full-text search
  for KG entities (migration 000031)
- Add entity deduplication system with dual-threshold strategy:
  auto-merge at 0.98+ similarity with Jaro-Winkler name match,
  flag 0.90-0.98 as candidates for manual review
- Add ScanDuplicates for bulk on-demand duplicate detection
- Add MergeEntities with advisory lock and tenant-scoped relation
  re-pointing (delete-then-update to avoid ON CONFLICT on UPDATE)
- Wire dedup inline after KG extraction pipeline
- Fix BackfillKGEmbeddings: was failing silently due to
  context.Background() missing tenant_id; now runs cross-tenant
- Fix BackfillKGEmbeddings: break on error → continue with failed
  ID tracking and max consecutive error cap
- Add EmbedEntity helper; UpsertEntity now generates embeddings
  in background goroutine
- Add HTTP endpoints: POST /kg/dedup/scan, GET /kg/dedup,
  POST /kg/merge, POST /kg/dedup/dismiss
- Add web UI: Dedup dialog with Scan All button, side-by-side
  entity comparison, merge/dismiss actions
- Add Jaro-Winkler similarity algorithm with 34 unit tests
- Update IngestExtraction to return upserted entity IDs
- Bump RequiredSchemaVersion to 31

2026-03-29 12:15:21 +07:00

..

extractor_prompt.go

refactor(kg): improve extraction prompt with few-shot example and controlled vocabulary

2026-03-12 18:54:14 +07:00

extractor_test.go

fix(kg): sanitize LLM JSON output before parsing (#167)

2026-03-14 16:22:48 +07:00

extractor.go

fix(kg): sanitize LLM JSON output before parsing (#167)

2026-03-14 16:22:48 +07:00

similarity_test.go

feat(kg): add tsvector FTS, entity deduplication, and embedding fixes

2026-03-29 12:15:21 +07:00

similarity.go

feat(kg): add tsvector FTS, entity deduplication, and embedding fixes

2026-03-29 12:15:21 +07:00