mirror of
https://github.com/tiennm99/thptqg.git
synced 2026-08-13 21:21:49 +00:00
tiennm99
c359a0b444
The repository now reads as the pipeline it is: crawler fetches, parser converts, assembler verifies and publishes, with data/ and web/ as the stores they hand work through. go-parser is renamed parser now that there is no other. The assembler replaces build-db.js and assemble-site.js. It compiles the parser, builds and verifies each database, compresses it, runs the Vite build and assembles _site — one command, and the only place that knows the order. It also closes a real hole: nothing previously asserted that a database reached the site. An empty staging directory assembled happily, so every page rendered, every query 404d and CI stayed green. The row-count and size guards could not catch that, since they only run when a database was built at all. Removing Node from the root forced the dataset list out of web/src/datasets.js, which the assembler cannot import. datasets.json is now the registry both sides read — JSON because Go and the browser both parse it without a dependency — while presentation stays in the web app, keyed by id and cross-checked against the registry so a half-added dataset fails instead of half-working. Guards verified by making each one fail: a missing database, and an expected row count one higher than the truth.
32 lines
1.0 KiB
JSON
32 lines
1.0 KiB
JSON
{
|
|
"_comment": [
|
|
"The dataset registry: the one place every stage agrees on what exists.",
|
|
"",
|
|
" crawler/ fills data/<id>/",
|
|
" parser/ reads data/<id>/ with configs/<id>.yml, writes <id>.db",
|
|
" assembler/ verifies, compresses and publishes it as db/<id>.db.gz",
|
|
" web/ serves it at /thptqg/<id>/",
|
|
"",
|
|
"It is JSON rather than a module because Go and the Vite app both read it,",
|
|
"and JSON is the only format both parse without a dependency. Presentation",
|
|
"(titles, labels, SQL presets) stays in web/src/datasets.js keyed by id;",
|
|
"that file fails loudly if the two lists disagree.",
|
|
"",
|
|
"expectedRows is exact, not approximate. The inputs are frozen historical",
|
|
"exam results, so a deviation of even one row means something changed that",
|
|
"nobody intended, and the assembler refuses to publish."
|
|
],
|
|
"datasets": [
|
|
{
|
|
"id": "2016",
|
|
"expectedRows": 877461,
|
|
"dbSizeMb": 44
|
|
},
|
|
{
|
|
"id": "2017",
|
|
"expectedRows": 861068,
|
|
"dbSizeMb": 48
|
|
}
|
|
]
|
|
}
|