diff --git a/2016/README.md b/2016/README.md index a86d382..6fcd788 100644 --- a/2016/README.md +++ b/2016/README.md @@ -1,31 +1,42 @@ # thptqg2016 -Tra cứu điểm thi THPT Quốc gia 2016 — 877.461 thí sinh toàn quốc. +Lookup tool for Vietnam's 2016 National High School Graduation Exam (THPT Quốc gia) scores — 877,461 candidates nationwide. -Hỗ trợ truy vấn SQL tùy chỉnh trực tiếp trên trình duyệt. +Fully static app running entirely in the browser (SQLite via `sql.js`). No backend, no query logging. -## Tính năng +## Features -- **Tra cứu nhanh** theo số báo danh hoặc họ tên -- **Truy vấn SQL tùy chỉnh** với 7 mẫu truy vấn có sẵn -- **Dữ liệu đầy đủ**: điểm 12 môn, cụm thi, giới tính -- Ngoại ngữ: Tiếng Anh, Pháp, Đức, Nhật, Trung -- Chạy hoàn toàn trên trình duyệt (SQLite via sql.js) +- **Quick lookup** by exam ID (`số báo danh`) or full name (diacritics-insensitive) +- **Custom read-only SQL** (SELECT / PRAGMA / EXPLAIN / WITH) with 7 built-in preset queries +- **Complete data**: 12 subjects (Math, Literature, Physics, Chemistry, Biology, History, Geography, English, French, German, Japanese, Chinese), exam cluster, date of birth, gender +- Safety caps: 100 rows for lookup, 1000 rows for custom SQL +- Dark mode, `Ctrl+Enter` shortcut to run queries ## Demo -https://tiennm99.github.io/thptqg2016/ + -## Phát triển +## Development ```bash npm install -npm run build:db # Tạo database từ file Excel -npm run dev # Chạy dev server +npm run build:db # Parse data/*.xlsx → public/thptqg2016.db +npm run dev # Vite dev server +npm run build # Production bundle → dist/ +npm run lint # ESLint ``` -## Công nghệ +The GitHub Actions workflow (`.github/workflows/deploy.yml`) builds the DB, gzips it, and deploys to GitHub Pages on every push to `main`. -React · Vite · sql.js · GitHub Pages +## Tech stack -**Nguồn**: Sưu tầm từ trang báo thời đó · Dữ liệu chỉ mang tính tham khảo +React 19 · Vite · sql.js (WASM) · better-sqlite3 (build-time only) · GitHub Pages + +## Documentation + +- [Project overview (PDR)](./docs/project-overview-pdr.md) +- [Codebase summary](./docs/codebase-summary.md) +- [System architecture](./docs/system-architecture.md) +- [Deployment guide](./docs/deployment-guide.md) + +**Source**: Collected from news sites at the time · Data is for reference only. diff --git a/2016/docs/codebase-summary.md b/2016/docs/codebase-summary.md new file mode 100644 index 0000000..29626e7 --- /dev/null +++ b/2016/docs/codebase-summary.md @@ -0,0 +1,63 @@ +# Codebase Summary + +## Directory layout + +``` +thptqg2016/ +├── data/ # Source Excel files (~100, mixed formats) +├── scripts/ +│ └── build-database.js # Parse Excel → SQLite (build-time, Node + better-sqlite3) +├── public/ +│ └── thptqg2016.db # Generated DB, gzipped during CI +├── src/ +│ ├── main.jsx # React entry +│ ├── App.jsx # Root: tabs, lookup logic, useSqlite wiring +│ ├── App.css / index.css # Design tokens, dark mode, a11y styles +│ ├── hooks/ +│ │ └── use-sqlite.js # Fetch .db.gz + decompress + init sql.js +│ └── components/ +│ ├── search-form.jsx # Input for exam ID / full name +│ ├── score-table.jsx # Result table for lookups +│ └── custom-query.jsx # SQL editor + presets + result grid +├── .github/workflows/deploy.yml # CI: build db → gzip → vite build → Pages +├── vite.config.js # base: "/thptqg2016/" +└── eslint.config.js +``` + +## Key modules + +### `scripts/build-database.js` +Build-time only. Reads every `.xlsx/.xls` in `data/`, detects the header format (three variants), parses the `DIEM_THI` string via regex or separate score columns, normalizes gender, derives a diacritics-stripped `ho_ten_ascii` column for accent-insensitive search, and inserts into SQLite with three indexes (`ho_ten`, `ho_ten_ascii`, `ten_cum_thi`). + +### `src/hooks/use-sqlite.js` +Streams `.db.gz` with download progress, decompresses via `DecompressionStream("gzip")`, loads `sql.js` (WASM served from the `sql.js.org` CDN), and returns `{ db, loading, error, progress }`. + +### `src/App.jsx` +Two tabs: **Lookup** and **Custom SQL**. Lookup auto-detects exam IDs (regex `^[A-Z]{2,4}\d+$`) vs names and picks one of three query paths: exact exam ID / ASCII LIKE / original + ASCII LIKE. Capped at 100 rows. + +### `src/components/custom-query.jsx` +Whitelists leading keywords (`SELECT`, `PRAGMA`, `EXPLAIN`, `WITH`), auto-appends `LIMIT 1000` when missing, measures `performance.now()` execution time, and ships 7 preset analytics queries. + +## `student` table schema + +```sql +so_bao_danh TEXT PRIMARY KEY -- exam ID +ho_ten TEXT NOT NULL -- full name +ho_ten_ascii TEXT NOT NULL -- diacritics stripped, lowercased +ngay_sinh TEXT -- date of birth +ten_cum_thi TEXT -- exam cluster name +gioi_tinh TEXT -- "Nam" | "Nữ" | NULL +toan, ngu_van, vat_ly, hoa_hoc, -- REAL (nullable) subject scores +sinh_hoc, lich_su, dia_ly, +tieng_anh, tieng_phap, tieng_duc, +tieng_nhat, tieng_trung +``` + +Indexes: `idx_ho_ten`, `idx_ho_ten_ascii`, `idx_ten_cum_thi`. + +## Conventions + +- JS/JSX filenames: **kebab-case** (e.g., `search-form.jsx`, `use-sqlite.js`) +- React components: `PascalCase` named exports +- UI strings: Vietnamese (target audience) +- Code comments: English; explain *why*, not *what* diff --git a/2016/docs/deployment-guide.md b/2016/docs/deployment-guide.md new file mode 100644 index 0000000..87e4a03 --- /dev/null +++ b/2016/docs/deployment-guide.md @@ -0,0 +1,50 @@ +# Deployment Guide + +## Automatic (recommended) + +Push to `main` → GitHub Actions builds and deploys to GitHub Pages automatically. + +Workflow: `.github/workflows/deploy.yml` + +CI steps: +1. `npm ci` +2. `npm run build:db` — generate `public/thptqg2016.db` from `data/*.xlsx` +3. `gzip -k -9 public/thptqg2016.db` — max compression, keep original +4. `npm run build` — Vite bundles `dist/` +5. `rm -f dist/thptqg2016.db` — ship only the gzipped copy +6. `actions/upload-pages-artifact@v3` + `actions/deploy-pages@v4` + +One-time setup: **Settings → Pages → Source: GitHub Actions**. + +## Manual (local verification) + +```bash +npm install +npm run build:db +gzip -k -9 public/thptqg2016.db # Linux/macOS; on Windows use 7zip or WSL +npm run build +rm dist/thptqg2016.db # optional, shrinks artifact +npm run preview # serve dist/ locally +``` + +Open . + +## Base path + +`vite.config.js` sets `base: "/thptqg2016/"`. If you fork under a different repo name, update this to match `` so assets resolve correctly on GitHub Pages. + +## Updating data + +1. Add the new Excel file to `data/`. +2. If its header is unfamiliar, open `scripts/build-database.js` and extend `detectFormat()` or the `processSeparateScoresRow` / `processMappedRow` helpers. +3. Run `npm run build:db` locally to check row counts and error skips. +4. Commit + push → CI redeploys. + +## Troubleshooting + +| Symptom | Typical cause | +|---------|---------------| +| Blank page, 404 on assets | `base` in `vite.config.js` doesn't match the repo name | +| `Failed to fetch database: 404` | gzip step skipped, or `.db.gz` removed from `dist/` | +| WASM fails to load | `sql.js.org` blocked / offline — self-host `sql-wasm.wasm` in `public/` and update `SQL_WASM_URL` in `use-sqlite.js` | +| Missing rows after build | Excel file has an unknown header — check console for `Failed to read` or `errorCount` | diff --git a/2016/docs/project-overview-pdr.md b/2016/docs/project-overview-pdr.md new file mode 100644 index 0000000..5891649 --- /dev/null +++ b/2016/docs/project-overview-pdr.md @@ -0,0 +1,32 @@ +# Project Overview — thptqg2016 + +## Goal + +Provide a public lookup tool for Vietnam's 2016 National High School Graduation Exam scores (877,461 candidates), running entirely on the client, hosted for free on GitHub Pages. + +## Scope + +- Lookup by exam ID or full name (with Vietnamese diacritics handling) +- Read-only SQL queries against a single `student` table +- Static dataset — no updates (the 2016 exam is long over) + +## Target users + +- Former 2016 candidates checking their scores +- Education researchers / data journalists running aggregate stats +- Developers exploring SQL on a real-world dataset + +## Constraints + +- **Zero backend**: the full DB (tens of MB gzipped) is downloaded to the browser +- **Read-only**: INSERT/UPDATE/DELETE rejected to avoid the illusion that user edits persist +- **Row caps**: 100 rows (lookup), 1000 rows (SQL) to prevent browser hangs +- **Vietnamese-first UI**: app labels and data are Vietnamese; documentation is English + +## Data sources + +Excel files (`.xlsx`/`.xls`) collected from newspapers and exam clusters in 2016, stored in `data/`. One file per cluster, with several different column layouts (see `scripts/build-database.js`). + +## Status + +Stable. Data is frozen. Recent work focuses on UX polish (dark mode, accessibility, diacritics-insensitive search). diff --git a/2016/docs/system-architecture.md b/2016/docs/system-architecture.md new file mode 100644 index 0000000..4c98b2d --- /dev/null +++ b/2016/docs/system-architecture.md @@ -0,0 +1,83 @@ +# System Architecture + +## Overview + +A **static serverless** design: the entire dataset is packaged into a single SQLite file, gzip-compressed, and served as a static asset via GitHub Pages. The browser downloads it, decompresses it, and queries it in-process using `sql.js` (SQLite compiled to WebAssembly). + +``` +┌─────────────────┐ build ┌──────────────────────┐ +│ data/*.xlsx │ ───────────▶ │ scripts/ │ +│ (mixed formats)│ │ build-database.js │ +└─────────────────┘ │ (Node + xlsx + │ + │ better-sqlite3) │ + └──────────┬───────────┘ + │ + ▼ + ┌──────────────────────┐ + │ public/thptqg2016.db │ + └──────────┬───────────┘ + │ gzip -9 (CI) + ▼ + ┌──────────────────────┐ + │ dist/thptqg2016.db.gz│ + │ dist/assets/* │ ◀── Vite build + └──────────┬───────────┘ + │ upload-pages-artifact + ▼ + GitHub Pages CDN + │ + ▼ + ┌──────────────────────────────────┐ + │ Browser │ + │ ┌────────────────────────────┐ │ + │ │ useSqlite hook │ │ + │ │ fetch(.db.gz) + stream │ │ + │ │ DecompressionStream gzip │ │ + │ │ sql.js WASM (from CDN) │ │ + │ └─────────────┬──────────────┘ │ + │ ▼ │ + │ ┌────────────────────────────┐ │ + │ │ React UI │ │ + │ │ - SearchForm / ScoreTable │ │ + │ │ - CustomQuery (SQL editor)│ │ + │ └────────────────────────────┘ │ + └──────────────────────────────────┘ +``` + +## Build-time data flow + +1. Developer drops Excel files into `data/`. +2. `npm run build:db` reads every file; for each one: + - Detects the header row against a `KNOWN_HEADERS` set. + - Picks a format: `separate-scores` (one column per subject) vs. `mapped` (single `DIEM_THI` string). + - Parses each row into a canonical 18-column record. + - `INSERT OR REPLACE` into SQLite (primary key = `so_bao_danh` handles duplicates). +3. `VACUUM` shrinks the file. +4. CI runs `gzip -k -9` → `.db.gz`. + +## Runtime flow + +1. Page loads → React mounts → `useSqlite("thptqg2016.db.gz")`. +2. Streaming fetch with a progress bar (driven by `Content-Length`). +3. `DecompressionStream("gzip")` decompresses on the fly. +4. `sql.js` loads its WASM from `https://sql.js.org/dist/sql-wasm.wasm`. +5. `new SQL.Database(Uint8Array)` — DB is now in RAM. +6. Each search / query → `db.prepare()` + `stmt.step()` loop → render. + +## Design decisions + +| Concern | Choice | Rationale | +|---------|--------|-----------| +| Storage | Static SQLite file | No backend needed; dataset is frozen | +| Compression | gzip in CI, `DecompressionStream` in browser | Native browser API; no extra library | +| WASM hosting | `sql.js.org` CDN | Smaller self-hosted artifact | +| Diacritics search | Pre-computed `ho_ten_ascii` column | `LOWER(REPLACE(...))` at query time defeats the index | +| SQL safety | Leading-keyword allowlist | `sql.js` is in-memory so writes don't persist, but the allowlist prevents user confusion | +| Row caps | 100 (lookup), 1000 (SQL) | Keep DOM render sizes reasonable | + +## Risks and limitations + +- **DB size**: tens of MB gzipped — slow links have a visible wait; mitigated by the progress bar. +- **Browser memory**: the full DB lives in RAM; older mobile devices may OOM. +- **Dependency on `sql.js.org`**: if that CDN is unreachable, WASM fails to load. +- **Excel format drift**: a new source file with an unseen header layout needs a new branch in `detectFormat()`.