mirror of
https://github.com/tiennm99/thptqg.git
synced 2026-10-04 12:13:27 +00:00
tiennm99
b38965dc4b
Every .ts becomes .js, every lang="ts" becomes lang-less, and the type declarations go with them: types.ts held nothing but types, so it is deleted outright. Tooling follows. typescript, svelte-check, typescript-eslint and @types/sql.js are uninstalled; tsconfig.json becomes jsconfig.json, which still extends the generated SvelteKit config so $lib and $app resolve in an editor; `npm run lint` is now ESLint alone, and CI's comment about it covering the type check goes too. What this gives up, stated plainly: a mistyped column name like row.nguvan used to fail the build and now renders blank, and the datasets.json-to-CONTENT cross-check is back to throwing at module load rather than at compile time. The runtime guard for the latter is still there and still throws loudly. Two mechanical notes. The svelte/no-navigation-without-resolve rule started flagging the footer's source link, which points at an off-site article — without type information the rule can no longer tell an external URL from a route, so that one line carries a disable comment. And Vitest's include pattern had to follow the tests to .js. Lint, 25 tests and the build all pass.
101 lines
3.9 KiB
Markdown
101 lines
3.9 KiB
Markdown
# thptqg
|
|
|
|
Tra cứu điểm thi THPT Quốc gia — exam-score lookup for Vietnam's national high
|
|
school graduation exam. Client-side SQL over a SQLite database read in place by
|
|
HTTP range request, built
|
|
from the published `.xls`/`.xlsx` score files by the Go `parser` module. Where
|
|
those files come from: [data pipeline](./docs/data-pipeline.md#sources).
|
|
|
|
Live at **[tiennm99.github.io/thptqg](https://tiennm99.github.io/thptqg/)**.
|
|
|
|
| Dataset | Exam | Candidates | Site |
|
|
| --- | --- | --- | --- |
|
|
| `2016` | 2016 | 877,460 | [/2016/](https://tiennm99.github.io/thptqg/2016/) |
|
|
| `2017` | 2017 | 861,068 | [/2017/](https://tiennm99.github.io/thptqg/2017/) |
|
|
|
|
Two earlier 2017 publications (`2017-old`, `2017-old2`) were kept for a while
|
|
because they disagreed with the current one. They have been removed; they remain
|
|
in git history.
|
|
|
|
## Layout
|
|
|
|
The repository is one directory per pipeline stage, plus the two stores they
|
|
pass between them.
|
|
|
|
```
|
|
crawler/ Go — re-fetches the source spreadsheets → data/
|
|
parser/ Go — Excel to SQLite data/ → .db
|
|
assembler/ Go — verifies, compresses, builds, assembles .db + web/ → _site/
|
|
web/ npm — the frontend, one SvelteKit app for every dataset
|
|
data/<id>/ raw Excel files, one directory per dataset
|
|
datasets.json the registry: which datasets exist, and their expected size
|
|
docs/ architecture, data pipeline, deployment
|
|
```
|
|
|
|
Each stage runs on its own and hands its output to the next through the stores.
|
|
`web/` is the only npm project; the three stages are independent Go modules.
|
|
|
|
`datasets.json` is the contract between them. It is JSON because Go and the web
|
|
app both read it and neither needs a dependency to do so; presentation stays in
|
|
`web/src/lib/datasets.js`, keyed by id, which fails loudly if the two disagree.
|
|
|
|
The dataset id is one identifier end to end:
|
|
|
|
```
|
|
data/2017/ → parser/configs/2017.yml → db/2017.sqlite3 → /thptqg/2017/
|
|
```
|
|
|
|
## Build
|
|
|
|
```bash
|
|
(cd web && npm ci)
|
|
go -C assembler run ./cmd/assemble # databases, then the site, into _site/
|
|
npx serve _site
|
|
```
|
|
|
|
That one command compiles the parser, builds and verifies each database against
|
|
its registry row count, compresses it, builds the web app and assembles `_site` —
|
|
refusing to continue if a database is short, an artifact looks truncated, or one
|
|
is missing altogether. Sub-steps when iterating:
|
|
|
|
```bash
|
|
go -C assembler run ./cmd/assemble db 2017 # one database
|
|
go -C assembler run ./cmd/assemble site # web build and _site only
|
|
go -C assembler run ./cmd/assemble verify A B # compare two sets of databases
|
|
(cd web && npm run dev) # the app against staged databases
|
|
```
|
|
|
|
The source spreadsheets are committed, so a crawl is only needed to refresh
|
|
them:
|
|
|
|
```bash
|
|
go -C crawler run ./cmd/crawl 2016
|
|
go -C crawler run ./cmd/crawl 2017
|
|
```
|
|
|
|
Each reads the download links out of the article that published the dataset, so
|
|
no link list is kept in the repository. Crawling is idempotent — files already
|
|
present are skipped — and is never part of the build.
|
|
|
|
Pushing to `main` runs the same steps in
|
|
`.github/workflows/deploy-pages.yml` and publishes to GitHub Pages.
|
|
|
|
## Adding a dataset
|
|
|
|
1. Put the Excel files in `data/<id>/`
|
|
2. Add `parser/configs/<id>.yml` — sheet mode, column indices, validation
|
|
guards. No SQL; the schema is canonical.
|
|
3. Add an entry to `datasets.json` with its expected row count and size
|
|
4. Add the matching presentation to `CONTENT` in `web/src/lib/datasets.js`
|
|
|
|
Everything else follows: the assembler, the router and the hub all read the
|
|
registry, and the UI adapts to whichever columns the dataset fills. Steps 3 and 4
|
|
check each other, so forgetting either one fails rather than half-working.
|
|
|
|
## Docs
|
|
|
|
See [`docs/`](./docs/) — [overview](./docs/project-overview.md),
|
|
[architecture](./docs/system-architecture.md),
|
|
[data pipeline](./docs/data-pipeline.md),
|
|
[deployment](./docs/deployment-guide.md).
|