tiennm99
6ff2ed99ec
The repo held two near-duplicate projects. 2016/ and 2017/ each carried their
own React frontend, their own copy of the same Rust crate, and their own
package manager setup. 2016/tools/sync-from-thptqg2017.sh existed purely to
copy the parser source between them.
New layout:
index.html + src/ the 2017 frontend, now the only one
data/<id>/ 2016, 2017, 2017-old, 2017-old2
parser/ the single Rust crate, configs renamed to <id>.toml
docs/ both projects' docs, 2016 copies suffixed -2016-legacy
pending the merge pass
<id> is now one identifier end to end: data/<id>/ feeds parser/configs/<id>.toml
and produces db/<id>.db.gz.
pnpm gives way to npm. pnpm-workspace.yaml existed only to whitelist
better-sqlite3's native build, which npm permits by default, so it has no
equivalent and is simply gone. Lockfiles cannot be converted; package-lock.json
is generated fresh. The migration direction is safe — pnpm's strict layout
forbids phantom dependencies, so anything that resolved under pnpm resolves
under npm's flat tree.
Adds parser/scripts/build-db.js and src/datasets.js: the four dataset IDs are
declared once and read by both the build tooling and (from the next phase) the
frontend.
Follow-on fixes the move made necessary:
- eslint's Node-globals override pointed at scripts/, now parser/scripts/
- crawl-baotintuc.js wrote to <root>/data, now data/2017
- golden tests loaded configs by their old thptqg*-data.toml names
Drops the #[ignore]d Rust-vs-Node golden test. It shelled out to pnpm to run
scripts/build-database.js, a file removed when the parser was ported to Rust,
so it could never pass. check-duplicates.js and diff-datasets.js were already
broken before this change and are annotated as such rather than half-fixed.
63 Rust tests pass and clippy is clean from the new location.
3.0 KiB
Codebase Summary
Directory layout
thptqg2016/
├── data/ # Source Excel files (~100, mixed formats)
├── scripts/
│ └── build-database.js # Parse Excel → SQLite (build-time, Node + better-sqlite3)
├── public/
│ └── thptqg2016.db # Generated DB, gzipped during CI
├── src/
│ ├── main.jsx # React entry
│ ├── App.jsx # Root: tabs, lookup logic, useSqlite wiring
│ ├── App.css / index.css # Design tokens, dark mode, a11y styles
│ ├── hooks/
│ │ └── use-sqlite.js # Fetch .db.gz + decompress + init sql.js
│ └── components/
│ ├── search-form.jsx # Input for exam ID / full name
│ ├── score-table.jsx # Result table for lookups
│ └── custom-query.jsx # SQL editor + presets + result grid
├── .github/workflows/deploy.yml # CI: build db → gzip → vite build → Pages
├── vite.config.js # base: "/thptqg2016/"
└── eslint.config.js
Key modules
scripts/build-database.js
Build-time only. Reads every .xlsx/.xls in data/, detects the header format (three variants), parses the DIEM_THI string via regex or separate score columns, normalizes gender, derives a diacritics-stripped ho_ten_ascii column for accent-insensitive search, and inserts into SQLite with three indexes (ho_ten, ho_ten_ascii, ten_cum_thi).
src/hooks/use-sqlite.js
Streams .db.gz with download progress, decompresses via DecompressionStream("gzip"), loads sql.js (WASM served from the sql.js.org CDN), and returns { db, loading, error, progress }.
src/App.jsx
Two tabs: Lookup and Custom SQL. Lookup auto-detects exam IDs (regex ^[A-Z]{2,4}\d+$) vs names and picks one of three query paths: exact exam ID / ASCII LIKE / original + ASCII LIKE. Capped at 100 rows.
src/components/custom-query.jsx
Whitelists leading keywords (SELECT, PRAGMA, EXPLAIN, WITH), auto-appends LIMIT 1000 when missing, measures performance.now() execution time, and ships 7 preset analytics queries.
student table schema
so_bao_danh TEXT PRIMARY KEY -- exam ID
ho_ten TEXT NOT NULL -- full name
ho_ten_ascii TEXT NOT NULL -- diacritics stripped, lowercased
ngay_sinh TEXT -- date of birth
ten_cum_thi TEXT -- exam cluster name
gioi_tinh TEXT -- "Nam" | "Nữ" | NULL
toan, ngu_van, vat_ly, hoa_hoc, -- REAL (nullable) subject scores
sinh_hoc, lich_su, dia_ly,
tieng_anh, tieng_phap, tieng_duc,
tieng_nhat, tieng_trung
Indexes: idx_ho_ten, idx_ho_ten_ascii, idx_ten_cum_thi.
Conventions
- JS/JSX filenames: kebab-case (e.g.,
search-form.jsx,use-sqlite.js) - React components:
PascalCasenamed exports - UI strings: Vietnamese (target audience)
- Code comments: English; explain why, not what