tiennm99
2c24943cc6
Three reversals had landed without the documentation following them, so the docs described a pipeline that compresses its output, a schema with three secondary indexes, and a browser that re-downloads the file on every visit. None of those are true any more. - Compression: the assembler stopped producing .gz when the databases began shipping as .sqlite3. The deployment guide's "why no uncompressed database can ship" section explained a guard that now exists for the opposite reason — to keep .db, .gz and journals out, so .sqlite3 stays the only name. - Indexes: the architecture printed a DDL with three CREATE INDEX statements and a paragraph on the partial one. schema.go carries none. - Persistence: "the download is repeated every visit ... has not been done" was listed as an open risk after db-cache.js closed it. Replaced with the ETag flow, the offline fallback, and the risks that did replace it. Measured both transfers rather than scaling one from the other, which would have been wrong: 2016 is 142 MB stored and 31 MB delivered, 2017 is 119 MB and 36 MB. The smaller database is the larger download, so neither figure follows from the stored size. Also corrects a CHUNK_BYTES reference to a module that no longer exists, the 238-289 MB per-dataset figure, two paths to web/src/lib/datasets.js, and the CI step list, which omitted npm test and the post-deploy header check. The two code comments that said the same outdated things go with them.
2.7 KiB
Project Overview
Goal
A public lookup tool for Vietnam's National High School Graduation Exam scores, running entirely in the browser and hosted for free on GitHub Pages. Covers the 2016 and 2017 exams — 1.7 million candidates across two datasets.
Scope
- Lookup by exam ID or full name, with Vietnamese diacritics handled
- Read-only SQL queries against a single
studenttable - Admission-block (khối thi) totals computed per candidate
- Static datasets — both exams are long over and the data is frozen
Target users
- Former candidates checking their scores
- Education researchers and data journalists running aggregate statistics
- Developers exploring SQL against a real-world dataset
Constraints
- Zero backend. The browser downloads the whole database (142 MB for 2016, 119 MB for 2017; about 31 MB and 36 MB on the wire, since the host gzips it) and queries it in memory. The copy is kept on the device, so the transfer is paid once rather than once per visit.
- Read-only.
INSERT/UPDATE/DELETEare rejected, so nobody is misled into thinking edits persist. The file is fetched, never written. - Row caps. 100 rows for lookups, 1000 for custom SQL, to prevent browser hangs.
- Vietnamese-first UI. App labels and data are Vietnamese; documentation is English.
Datasets
| id | Exam | Candidates | Notes |
|---|---|---|---|
2016 |
2016 | 877,460 | 119 files, four column layouts |
2017 |
2017 | 861,068 | current generation of three publications |
Two further 2017 datasets (2017-old, 2017-old2) were kept alongside these
because the three publications disagreed. They have been removed; git history
still has them.
Both datasets now have a crawler source. 2017 comes from the baotintuc.vn CDN,
which is still live. 2016 comes from an aggregator article on
dtnt.bacninh.edu.vn, also still online. Both datasets have been crawled
successfully, so either can be rebuilt from source — see
data-pipeline for the full article URLs.
History
Each year began as a standalone repository (thptqg2016, thptqg2017), merged
here with full history. They initially kept separate frontends and separate
copies of the same Rust parser, synchronised by hand. That duplication was
removed: there is now one frontend, one parser, and one canonical schema, with
per-dataset differences confined to one small config file and one registry
entry each.
The unification also fixed a latent data-loss bug — neither year's parser
config listed the complete set of subjects, so 1,691 candidates were missing
their foreign-language score. See data-pipeline.md.
Status
Stable, data frozen. Work is limited to UX polish and keeping the pipeline maintainable.