Commit Graph
32 Commits
Author SHA1 Message Date
tiennm99 c8a1359987 feat(xlsxread): stage 1 reader — calamine sheet enumeration and header skip 2026-05-19 15:09:51 +07:00
tiennm99 b886f38df6 feat(xlsxread): stage 0 scaffold with pinned deps and clap CLI skeleton 2026-05-19 15:09:45 +07:00
tiennm99 77a3393898 chore(ci): bump node to 24 2026-05-13 10:52:39 +07:00
tiennm99 90cac022e7 fix(ci): approve better-sqlite3 build via pnpm-workspace.yaml allowBuilds 2026-05-13 10:39:17 +07:00
tiennm99 69ebb7ac17 fix(ci): bump Node.js to 22 required by pnpm@11.1.1 2026-05-13 10:33:59 +07:00
tiennm99 48ee0724b7 fix(ci): remove pnpm version override conflicting with packageManager 2026-05-13 10:28:59 +07:00
tiennm99 42d5f8d276 chore: migrate from npm to pnpm
Replace package-lock.json with pnpm-lock.yaml, add packageManager field,
update package.json scripts from npm run to pnpm, and update GitHub Actions
workflow to use pnpm/action-setup@v4 with frozen-lockfile installs.
2026-05-13 10:20:29 +07:00
tiennm99 3a1e2857ca docs: note data/ source URL; mark data-old/ and data-old2/ origin unrecorded 2026-04-14 23:30:12 +07:00
tiennm99 ddb1a63fb7 docs: translate narrative prose to English in README and docs/ 2026-04-14 23:28:55 +07:00
tiennm99 d14de5e1a6 docs: expand README, add architecture / deployment / data-pipeline docs
- README: live-site URLs, features, scripts, project layout, quickstart
- docs/system-architecture.md: data flow, 3-variant deploy mechanism,
  schema, parse-quirk matrix, score-tier model, admission-block model,
  frontend key behaviors (deep-link, share, keyboard shortcuts)
- docs/data-pipeline.md: source Excel shape, score regex, why three
  parsers, overflow-sheet gotcha, audit tooling, refresh flow
- docs/deployment-guide.md: CI workflow, adding a new variant, rollback,
  GH Pages file-size note
- docs/README.md: index for the docs dir
2026-04-14 23:25:51 +07:00
tiennm99 5cae275e22 tweak: Long An best-block preset → top 10 instead of top 100 2026-04-14 23:19:36 +07:00
tiennm99 da4c017e96 feat: full 49-block admission catalog + winning-block label in top-100 query
- admission-blocks.js: expand from 8 core blocks to all 49 official 2017
  blocks (A00-A11, B00-B08, C00-C20, D01-D15) the DB schema can support.
  Detail card now shows every qualifying block, sorted best-first
- custom-query: rewrite "Top 100 Long An" preset using UNION ALL +
  ROW_NUMBER() OVER PARTITION BY so we can label each row with the
  winning block code, not just the score
2026-04-14 23:17:58 +07:00
tiennm99 655d8f8cfc feat: add 'Top 100 điểm khối cao nhất - Long An' SQL preset
Computes each student's max admission-block sum across 49 official 2017
blocks (A00-A11, B00-B08, C00-C20, D01-D15, restricted to subjects
present in our DB — no Đức/Nhật languages). One row per student, sorted
by their personal best, top 100.

SQL note: SQLite's MAX(x,y,...) scalar returns NULL if ANY arg is NULL.
Each block expression is wrapped in COALESCE(..., -1); NULLIF(..., -1)
restores NULL only for students with zero computable blocks.
2026-04-14 23:11:01 +07:00
tiennm99 b7ef3aa332 style: TFT rarity-ladder score tiers (6 levels, unique hues)
Old palette reused gold for both "Trung bình" and "Xuất sắc" — confusing.
Replaces with League of Legends / TFT rarity ladder where each rank gets
a distinct hue that also communicates relative rarity.

Tiers:
  ≤ 1    Điểm liệt   (common)    white / gray
  < 5    Chưa đạt    (uncommon)  green
  5-6.5  Trung bình  (rare)      blue
  6.5-8  Khá         (epic)      purple
  8-9    Giỏi        (legendary) gold
  9-10   Xuất sắc    (prismatic) multi-color gradient

- admission-blocks.js: scoreTier now returns 6 keys (common..prismatic)
  plus the "điểm liệt" tier (≤1) that didn't exist before
- student-detail.jsx: TIER_LEGEND updated to 6 entries with ranges
- App.css: --tier-{common,uncommon,rare,epic,legendary,prismatic}-*
  tokens, light + dark variants each AA-contrast verified; applied to
  .score-cell, .score-tile, .tier-legend-item
2026-04-14 23:03:08 +07:00
tiennm99 98eabce01f tweak: reduce Long An khối A preset from top 100 to top 10 2026-04-14 22:51:48 +07:00
tiennm99 95be7e479d feat: deep-link, share, score legend, web font, grouped SQL presets
- App.jsx owns query state, syncs to URL via ?q= (works for SBD or name);
  hydrates initial search from URL on DB ready
- Loading panel now shows DB size (~47 MB) and one-time download note
- Footer reports total student count from the live DB
- Keyboard shortcut '/' focuses the search input when not already typing
- StudentDetail: "Chia sẻ" button uses Web Share API or clipboard with a
  formatted multi-line summary including a deep-link URL
- StudentDetail: inline tier legend (▽ ○ ◆ ★ ✦) with score ranges
- ScoreTable cells now use background tint matching detail tiles
- CustomQuery: presets grouped into 4 categories; auto-runs PRAGMA
  table_info(student) on first DB ready so the tab opens with the schema
  visible instead of a blank textarea
- SearchForm accepts controlled value prop for URL hydration
- Add Be Vietnam Pro web font for consistent Vietnamese diacritic rendering
2026-04-14 22:44:00 +07:00
tiennm99 f1169d3d28 feat: Long An SQL presets, clickable search examples
- custom-query: two new presets filtered by Long An (SBD prefix 49):
  Top 10 điểm Toán, Top 100 khối A (Toán+Lý+Hóa)
- search-form: show clickable example pills ("49008235" /
  "Nguyễn Minh Tiến") next to the empty-state hint; clicking fills
  the input and triggers the debounced live search
- App.css: .example-btn pill styling matching preset buttons
2026-04-14 22:21:54 +07:00
tiennm99 33937231a6 chore: remove completed static-score-lookup plan (fully implemented) 2026-04-14 22:13:20 +07:00
tiennm99 622e7383ef feat: build 3 parallel site variants, one per dataset
Sites:
  /thptqg2017/      → data/      861,068 students (baotintuc.vn .xls)
  /thptqg2017/old/  → data-old/  847,348 students (original xlsx)
  /thptqg2017/old2/ → data-old2/ 679,764 students (update/ overrides)

Build pipeline:
- scripts/build-lib.js: shared schema, score regex, toAscii, insert
  builder — keeps the 3 DBs drop-in compatible for the same frontend
- scripts/build-database.js: multi-sheet walk for baotintuc .xls (HN/HCM
  overflow past the 65k-per-sheet .xls cap)
- scripts/build-database-old.js: single-sheet xlsx + strict numeric SBD
  guard (rejects the header-row leak present in data-old)
- scripts/build-database-old2.js: multi-sheet + blank-row pre-filter for
  the 54 update files (HCM overflow present, HN not in this subset)
- Each build prints a source-vs-DB audit; verified 0 parse errors and
  exact row-count match across all 3 datasets

Frontend plumbing:
- vite.config.js: VARIANT env selects base path, publicDir, outDir
- package.json: build:db:*, build:db:all, build:old, build:old2,
  build:all
- .github/workflows/deploy.yml: CI builds all 3 DBs + sites
- .gitignore: cover public-old/ and public-old2/ outputs
- eslint.config.js: add Node globals for config/script files
2026-04-14 22:12:25 +07:00
tiennm99 0cbd822f9f chore: preserve prior update/ overrides in data-old2/
Restore the 54 corrected-export files that previously lived in
data/raw/update/ (removed in 718e2e9). Kept alongside data-old/ for
historical reference; not consumed by build pipeline.
2026-04-14 21:46:42 +07:00
tiennm99 8f86e4dc3b feat: refresh data from baotintuc.vn source, fix overflow sheet loss
Dataset update:
- Crawl all 63 .xls province files from baotintuc.vn CDN (original source)
- Old xlsx dataset moved to data-old/ for reference
- Net: +13,719 students (Hà Nội +7,275, HCM +6,445) — the old .xls → xlsx
  conversion silently dropped rows beyond the 65,536 per-sheet cap
- Also removes 1 bogus header row that had leaked into the old DB
- 100% identical scores on the 847,348 SBDs present in both datasets

Build pipeline:
- build-database.js: iterate ALL sheets per workbook (fixes the overflow
  loss) and accept .xls in addition to .xlsx

Audit tooling:
- scripts/crawl-baotintuc.js: idempotent 63-province downloader
- scripts/diff-datasets.js: compares two DBs by SBD set and per-column
  score deltas
2026-04-14 21:42:29 +07:00
tiennm99 d8dc1e0955 feat: live search, student detail card, score tier color coding
- SearchForm: 300ms debounced live search, visible label, aria-live mode
  hint, inline clear (×) button, minimum-length guards (3 digits / 2 chars)
- StudentDetail: rich single-result card with copy-SBD action, subject
  tiles with 5-tier color+icon coding, admission-block totals (A/A1/B/C/
  D1-D4) computed only when the student has all 3 required subject scores
- ScoreTable: per-cell tier color coding with tooltip
- App.jsx: switches to detail card when exactly 1 result; onClear wired
- App.css: semantic token system with light + dark (prefers-color-scheme)
  variants, :focus-visible ring, reduced-motion fallback, desktop-first
  breakpoints; removes hardcoded hex across components
- admission-blocks.js: pure data + calculator, no UI coupling
2026-04-14 21:21:46 +07:00
tiennm99 718e2e9117 refactor: flatten data layout to data/, drop update/ overrides
- Move 63 Excel files from data/raw/ to data/ (single flat dir)
- Remove all 53 files in data/raw/update/: verified identical SBD
  coverage to raw/ (847349 rows either way), so they added no new
  students — only potential score corrections that can be reintroduced
  later if source is recovered
- Update build-database.js to read data/ directly
- Add scripts/audit-row-counts.js: compares source row count to DB row
  count to verify zero-loss parsing
- Point check-duplicates.js at new data/ location
2026-04-14 21:02:47 +07:00
tiennm99 f10046f63d chore: remove duplicate Excel files, add md5 audit script
- Drop 10_LamDong_GNFT (1) and 2.BacKan_YQNX(1): identical row content to
  siblings (Excel metadata differs but file size & sheet rows match)
- Add scripts/check-duplicates.js to detect byte-identical and row-identical
  files across data/raw and data/raw/update
2026-04-14 20:49:41 +07:00
tiennm99 52cc7ac2d0 feat: diacritics-insensitive search and SQL query tab
- Add ho_ten_ascii column + index for accent-folded name search
- Parse Tiếng Pháp/Nga/Trung scores (recovers ~2k students' foreign-language data)
- Loosen score regex to accept integer values (e.g. KHTN: 4)
- App.jsx: 3-mode search (SBD / ASCII / Vietnamese) and Tra cứu/SQL tabs
- New custom-query component: 8 presets adapted to 2017 schema, read-only safety, auto-LIMIT
- ScoreTable: render new foreign-lang columns, hide columns null across all results
2026-04-14 20:44:43 +07:00
tiennm99 4474547433 refactor: remove Java code, move web app to project root
- Remove Gradle build, Java sources, Hibernate config, old database.sqlite
- Move Excel data files from src/main/resources/raw/ to data/raw/
- Move Vite+React app from web/ to project root
- Merge package.json into single root-level config
- Update build script paths and CI workflow accordingly
2026-04-13 00:06:22 +07:00
tiennm99 a5637fb66e feat: add implementation plans and Vite scaffold files 2026-04-13 00:00:43 +07:00
tiennm99 1cf65be51c feat: add static score lookup site with Node.js DB builder
- Node script parses 119 Excel files into SQLite (847K students)
- Vite + React frontend with sql.js for client-side querying
- Search by exam ID (số báo danh) or student name
- Gzipped DB (36MB) with download progress bar
- GitHub Actions workflow for GitHub Pages deployment
2026-04-12 23:54:06 +07:00
tiennm99 4d20456e32 [Add] logics to insert from excel files into sqlite database 2023-11-13 23:22:59 +07:00
tiennm99 56eff2681a [Add] lib to create database 2023-10-30 07:35:55 +07:00
tiennm99 628e538ae0 [Add] init with idea & add Excel sources 2023-10-29 06:55:42 +07:00
Tien Nguyen MinhandGitHub a0eeada7fb Initial commit 2023-10-29 06:18:07 +07:00