- Replace ILIKE search with PostgreSQL tsvector/GIN full-text search
for KG entities (migration 000031)
- Add entity deduplication system with dual-threshold strategy:
auto-merge at 0.98+ similarity with Jaro-Winkler name match,
flag 0.90-0.98 as candidates for manual review
- Add ScanDuplicates for bulk on-demand duplicate detection
- Add MergeEntities with advisory lock and tenant-scoped relation
re-pointing (delete-then-update to avoid ON CONFLICT on UPDATE)
- Wire dedup inline after KG extraction pipeline
- Fix BackfillKGEmbeddings: was failing silently due to
context.Background() missing tenant_id; now runs cross-tenant
- Fix BackfillKGEmbeddings: break on error → continue with failed
ID tracking and max consecutive error cap
- Add EmbedEntity helper; UpsertEntity now generates embeddings
in background goroutine
- Add HTTP endpoints: POST /kg/dedup/scan, GET /kg/dedup,
POST /kg/merge, POST /kg/dedup/dismiss
- Add web UI: Dedup dialog with Scan All button, side-by-side
entity comparison, merge/dismiss actions
- Add Jaro-Winkler similarity algorithm with 34 unit tests
- Update IngestExtraction to return upserted entity IDs
- Bump RequiredSchemaVersion to 31
viettranx
2e869f4ece