diff --git a/plans/260726-0952-lol-leaguepedia-score-enrichment/phase-01-feasibility-gate.md b/plans/260726-0952-lol-leaguepedia-score-enrichment/phase-01-feasibility-gate.md new file mode 100644 index 0000000..abe5df5 --- /dev/null +++ b/plans/260726-0952-lol-leaguepedia-score-enrichment/phase-01-feasibility-gate.md @@ -0,0 +1,100 @@ +--- +phase: 1 +title: "Feasibility Gate" +status: pending +effort: "S" +priority: P1 +dependencies: [] +--- + +# Phase 1: Feasibility Gate + +## Overview + +Answer the questions research could not, because Fandom rate-limited then refused the connection. **This phase can cancel the plan.** No production code until every check below passes. + +## Why This Exists + +Research (`plans/reports/260726-0941-lol-score-source-alternatives.md`) established that Leaguepedia is the only viable free source with independent ingestion, and that its `MatchSchedule` table exposes `Team1Score`/`Team2Score`/`Winner`. It could **not** establish that Leaguepedia actually has the LEC/LCS scores lolesports is missing, nor how fast its editors are. Those are the load-bearing assumptions. + +Leaguepedia certainly *covers* LEC/LCS/LPL/LCK — it is the reference wiki. The open risks are latency, join viability, and access. + +## Checks + +Run against the known-bad matches (lolesports has these as `completed` with `0-0`): + +| startTime (UTC) | league | teams | +|---|---|---| +| 2026-07-25T14:30Z | lec | G2 Esports vs Team Vitality | +| 2026-07-25T17:30Z | lec | Movistar KOI vs Karmine Corp | +| 2026-07-25T19:00Z | cblol-brazil | LOUD vs paiN Gaming | +| 2026-07-25T20:00Z | lcs | FlyQuest vs LYON | +| 2026-07-25T23:00Z | lcs | Dignitas vs Sentinels | + +Baseline query (proven to work during research): + +```bash +curl -s -G "https://lol.fandom.com/api.php" \ + -A "miti99bot/0.1 (+https://github.com/tiennm99/miti99bot)" \ + --data-urlencode "action=cargoquery" \ + --data-urlencode "format=json" \ + --data-urlencode "tables=MatchSchedule=MS" \ + --data-urlencode "fields=MS.DateTime_UTC,MS.Team1,MS.Team2,MS.Team1Score,MS.Team2Score,MS.Winner,MS.BestOf,MS.OverviewPage" \ + --data-urlencode "where=MS.DateTime_UTC >= '2026-07-25 14:00:00' AND MS.DateTime_UTC < '2026-07-26 00:00:00'" \ + --data-urlencode "order_by=MS.DateTime_UTC" \ + --data-urlencode "limit=100" +``` + +### Gate 1 — Data exists (BLOCKING) + +- [ ] At least 4 of the 5 matches above return a row with non-empty `Team1Score`/`Team2Score` +- [ ] Scores are plausible for the Bo3 (2-0/2-1, not 0-0) +- [ ] For `Movistar KOI vs Karmine Corp`, score is `2-0` or `0-2` — VOD evidence from the debug session showed exactly 2 games played, game 3 `unneeded` + +**If fewer than 4 of 5 have scores → STOP. Cancel plan.** Leaguepedia is no faster than Riot and the enrichment buys nothing. + +### Gate 2 — Name join works (BLOCKING) + +Compare Leaguepedia `Team1`/`Team2` against Riot `Team.Name` for the same matches. + +- [ ] Riot `Team.Name` values match Leaguepedia `Team1`/`Team2` exactly, OR differ only by case/punctuation/whitespace +- [ ] Record every mismatch verbatim — these define the normalization rules for Phase 3 +- [ ] Confirm no two matches in a single day-window share the same normalized name pair (would make the join ambiguous) + +Known naming risk to check specifically: `paiN Gaming` (Riot code `PAIN`) and `LYON` — irregular capitalization. + +**If names diverge structurally (e.g. Leaguepedia uses `Movistar KOI (European Team)` disambiguators) → note it; Phase 3 gains a disambiguator-stripping rule. Not fatal, but scope grows.** + +### Gate 3 — Editorial latency (INFORMATIONAL, shapes value) + +- [ ] Note wall-clock lag between match end and Leaguepedia having the score, for at least 2 matches +- [ ] If Leaguepedia is consistently *slower* than Riot's backfill, the feature is near-worthless even if Gates 1-2 pass → report and ask before continuing + +### Gate 4 — Access viability (BLOCKING) + +- [ ] A single cold query succeeds without prior warmup (research only succeeded on retry 3 of 3 after bursts) +- [ ] Determine whether the throttle is per-IP burst-based or a hard quota — space 3 queries 60s apart and confirm all 3 succeed +- [ ] Confirm no API key or login is required for `MatchSchedule` reads + +**If a lone spaced query cannot reliably succeed → STOP or pivot to Liquipedia LPDB** (free tier, needs key + manual approval; repo is public so it likely qualifies). + +### Gate 5 — Licensing + +- [ ] Confirm CC-BY-SA 3.0 attribution requirement and decide placement (Phase 4 handles it — likely a line in `/lol` help or the digest footer) + +## Deliverable + +Append findings to `plans/reports/260726-0941-lol-score-source-alternatives.md` (do not create a second report — the unresolved-questions section there is exactly what this phase closes). Record: raw JSON for one match, the exact name pairs observed, throttle behaviour, and a GO/NO-GO. + +## Success Criteria + +- [ ] Gates 1, 2, 4 pass → GO, proceed to Phase 2 +- [ ] Gate 3 measured and acceptable +- [ ] Any gate fails → NO-GO recorded with evidence; plan marked `cancelled`; report back before writing code +- [ ] Normalization rules for Phase 3 written down from real observed data, not guessed + +## Risk Assessment + +**Biggest risk: doing Phases 2-4 first and discovering Leaguepedia is also stale.** That is the entire reason this phase is a hard gate rather than advisory. + +Second risk: verifying against *this* stall only. One stall is a sample of one. If Gate 3 shows Leaguepedia leading Riot by hours here, that is suggestive, not proof of steady-state behaviour. Note the caveat rather than over-claiming. diff --git a/plans/260726-0952-lol-leaguepedia-score-enrichment/phase-02-cargo-client.md b/plans/260726-0952-lol-leaguepedia-score-enrichment/phase-02-cargo-client.md new file mode 100644 index 0000000..2125e27 --- /dev/null +++ b/plans/260726-0952-lol-leaguepedia-score-enrichment/phase-02-cargo-client.md @@ -0,0 +1,137 @@ +--- +phase: 2 +title: "Cargo Client" +status: pending +effort: "M" +priority: P2 +dependencies: [1] +--- + +# Phase 2: Cargo Client + +## Overview + +A minimal read-only client for Leaguepedia's Cargo API returning finished-match scores in a UTC time window. Mirrors the existing `Client` shape in `api_client.go` so tests can inject an `httptest.Server`. + +## Requirements + +Functional: +- Fetch `MatchSchedule` rows for `[from, to)` UTC +- Return team names, series scores, Bo count, scheduled time +- Skip rows without a usable score (unplayed / in-progress future matches) + +Non-functional: +- Disabled by default; single env var enables +- Bounded timeout, no retries (a miss is not worth a second request against a throttling host) +- Descriptive User-Agent with contact URL — required by wiki API etiquette +- Never returns an error the caller must surface to the user; caller treats all failures as "no data" + +## Architecture + +New file `internal/modules/lol/leaguepedia_client.go`. + +```go +// leaguepediaEnabledEnv gates the whole feature. Absent or not "1"/"true" → no +// requests are ever made. +const leaguepediaEnabledEnv = "LOL_LEAGUEPEDIA_ENABLED" + +const ( + leaguepediaURL = "https://lol.fandom.com/api.php" + // leaguepediaUserAgent identifies the bot per wiki API etiquette; Fandom + // throttles anonymous traffic aggressively and an honest UA with contact + // info is the minimum courtesy for a free source. + leaguepediaUserAgent = "miti99bot/0.1 (+https://github.com/tiennm99/miti99bot)" + leaguepediaTimeout = 6 * time.Second + // leaguepediaRowLimit caps a day/week window. A dense week across all + // leagues (incl. amateur tiers we filter out) can exceed 100 rows. + leaguepediaRowLimit = 500 +) + +// SeriesResult is one finished series as Leaguepedia records it. Team names are +// full names ("Movistar KOI"), matching Riot's Team.Name — see Phase 3 join. +type SeriesResult struct { + StartTime time.Time + Team1 string + Team2 string + Team1Wins int + Team2Wins int +} + +type leaguepediaClient struct { + HTTP *http.Client + URL string // test override; empty → leaguepediaURL +} + +// leaguepediaEnabled reports whether the operator opted in. +func leaguepediaEnabled() bool + +// FetchResults returns finished series in [from, to). Returns (nil, nil) when +// disabled — callers treat empty and error identically, so there is no +// "disabled" error to special-case. +func (c *leaguepediaClient) FetchResults(ctx context.Context, from, to time.Time) ([]SeriesResult, error) +``` + +### Cargo response shape + +Rows arrive wrapped, with **spaces** in field names (Cargo replaces `_`): + +```json +{"cargoquery":[{"title":{ + "DateTime UTC":"2026-07-25 17:30:00", + "Team1":"Movistar KOI","Team2":"Karmine Corp", + "Team1Score":"2","Team2Score":"0", + "Winner":"1","BestOf":"3"}}]} +``` + +Two decoding traps, both must be handled: +- `DateTime UTC` and `Team1Score` are **strings**, not numbers → decode into `string` and `strconv.Atoi` +- Errors come back HTTP 200 with `{"error":{"info":"You've exceeded your rate limit..."}}` → must check for an `error` key, not just the status code + +Timestamp format is `2006-01-02 15:04:05`, no zone suffix, always UTC. + +## Related Code Files + +- Create: `internal/modules/lol/leaguepedia_client.go` +- Create: `internal/modules/lol/leaguepedia_client_test.go` +- Reference (patterns to copy, do not modify): `internal/modules/lol/api_client.go` (Client/HTTP/URL shape, `truncate` for log bounding), `internal/modules/misc/wheelofnames_api_client.go:62` (env const + `os.Getenv`) + +## Implementation Steps + +1. Add env gate `leaguepediaEnabled()` — `strings.TrimSpace(os.Getenv(...))` matched against `"1"`/`"true"`, so an empty or garbage value is safely off +2. Add `httpClient()`/`baseURL()` helpers mirroring `api_client.go:133-145` +3. Build the query with `url.Values` — `action=cargoquery`, `format=json`, `tables=MatchSchedule=MS`, the field list, `where` bounded by `from`/`to`, `order_by`, `limit` +4. Set `User-Agent` and `Accept`; issue GET with the request context +5. Decode into a struct with an `Error *struct{ Info string }` field; if present, log at warn with `truncate(...)` and return an error +6. Map rows → `[]SeriesResult`, skipping any row where either score fails to parse or `Team1Wins == 0 && Team2Wins == 0` (unplayed — same "absent vs zero" trap as the original bug; do not import a fabricated 0-0 from a *second* source) +7. Parse `DateTime UTC` with `time.ParseInLocation(..., time.UTC)`; skip unparseable rows + +## Tests + +All via `httptest.Server`, no network: + +- [ ] Happy path: 3 rows → 3 `SeriesResult` with correct scores and UTC times +- [ ] Disabled env → returns `(nil, nil)` and **makes zero HTTP requests** (assert with a request counter on the test server) +- [ ] Rate-limit body (`HTTP 200` + `{"error":{"info":"..."}}`) → error, no partial results +- [ ] Non-2xx status → error +- [ ] Malformed JSON → error +- [ ] Row with `Team1Score:""` → skipped, siblings still returned +- [ ] Row with `Team1Score:"0", Team2Score:"0"` → skipped (unplayed guard) +- [ ] Row with unparseable `DateTime UTC` → skipped, siblings returned +- [ ] Request assertions: `where` contains both bounds, UA is set, `limit` present +- [ ] Context cancellation propagates + +## Success Criteria + +- [ ] `go test ./internal/modules/lol/ -run Leaguepedia` green +- [ ] Zero network access in tests +- [ ] Zero HTTP requests when disabled +- [ ] No change to any existing file — this phase is purely additive +- [ ] `go vet ./...` clean + +## Risk Assessment + +**Silent schema drift.** Cargo field names with spaces are easy to typo into always-empty strings, which would look like "Leaguepedia has no data" rather than a bug. Mitigate by asserting exact parsed values in the happy-path test using a fixture captured verbatim in Phase 1, not hand-written JSON. + +**Importing another fabricated 0-0.** Step 6's guard is the whole point — this plan exists because a 0-0 was trusted once already. + +**Throttling in production.** Out of scope here (client makes one request when asked); Phase 4 owns call-site frequency. diff --git a/plans/260726-0952-lol-leaguepedia-score-enrichment/phase-03-score-join.md b/plans/260726-0952-lol-leaguepedia-score-enrichment/phase-03-score-join.md new file mode 100644 index 0000000..6ae0e93 --- /dev/null +++ b/plans/260726-0952-lol-leaguepedia-score-enrichment/phase-03-score-join.md @@ -0,0 +1,110 @@ +--- +phase: 3 +title: "Score Join" +status: pending +effort: "M" +priority: P2 +dependencies: [2] +--- + +# Phase 3: Score Join + +## Overview + +Pure functions that decide which lolesports events need a score, and merge Leaguepedia rows into them. No I/O — this is the correctness core and must be exhaustively unit-tested. + +## Requirements + +- Identify events needing enrichment: `State == "completed" && !scoreIsPublished(t1, t2)` +- Match each to at most one `SeriesResult` +- Fill `GameWins` + `Outcome` so `formatEventLine` renders `✅` naturally, with the winner bolded +- Refuse ambiguous matches rather than guess +- Leave everything else byte-identical + +## Architecture + +New file `internal/modules/lol/score_enrich.go`. Reuses `scoreIsPublished` from `format.go` — that predicate already encodes exactly "has upstream published a usable score", so the trigger needs no new logic. + +```go +// needsScore reports whether an event is a finished series with no published +// score — the only shape this enrichment may touch. +func needsScore(e ScheduleEvent) bool + +// normalizeTeamName folds the cosmetic differences between Riot's Team.Name and +// Leaguepedia's Team1/Team2 into a comparable key. Rules are derived from the +// real pairs recorded in Phase 1, not invented here. +func normalizeTeamName(s string) string + +// mergeScores returns events with missing scores filled from results. Input is +// never mutated; a copy is returned. Unmatched, ambiguous, and already-scored +// events pass through untouched. Reports how many were filled, for logging. +func mergeScores(events []ScheduleEvent, results []SeriesResult) ([]ScheduleEvent, int) +``` + +### Match key + +`(normalizeTeamName(team1), normalizeTeamName(team2), UTC date of StartTime)` + +Date rather than exact instant: Leaguepedia records *scheduled* time, which can drift from Riot's by minutes when a broadcast slips. Same-day plus an exact team pair is specific enough — two teams do not play the same opponent twice in a day in these leagues. + +Orientation: try `(t1, t2)`; if no hit, try `(t2, t1)` and **swap the scores when applying**. Riot and Leaguepedia do not guarantee the same side ordering. Getting this backwards silently inverts every result, so it needs its own test. + +### Ambiguity rule + +Build the index as `map[key][]SeriesResult`. Apply only when `len(candidates) == 1`. Two or more → skip and log at warn. A wrong score is worse than `score pending`; that is the premise of the whole fix. + +### Applying a result + +```go +// Winner gets outcome "win", loser "loss", so formatEventLine bolds correctly. +// Outcome must be set: scoreIsPublished keys off it, so a fill that set only +// GameWins would still render as "score pending". +``` + +Edge case — a genuine draw or an unresolved series cannot occur here, because Phase 2 already dropped `0-0` rows. If both wins are equal and non-zero (a real Bo2 1-1), set both outcomes to `"loss"`... **decide during implementation**: LEC/LCS/LPL Bo2 splits exist. Safer: if `Team1Wins == Team2Wins`, fill `GameWins` on both and set both `Outcome` to `"tie"`. `formatEventLine` bolds only on `== "win"`, so a tie renders `✅ A 1–1 B` unbolded, which is correct. Add a test. + +## Related Code Files + +- Create: `internal/modules/lol/score_enrich.go` +- Create: `internal/modules/lol/score_enrich_test.go` +- Reference (do not modify): `internal/modules/lol/format.go` (`scoreIsPublished`, `declaredOutcome`, `seriesWins`) + +## Implementation Steps + +1. `needsScore` — thin wrapper over `State == "completed"` + `!scoreIsPublished(...)`; guard `len(Teams) >= 2` first (`format.go:126-131` tolerates short slices, so this must too) +2. `normalizeTeamName` — lowercase, trim, collapse internal whitespace, strip punctuation. Add a disambiguator strip (`" (…Team)"` suffix) **only if Phase 1 observed one** +3. Build the candidate index from `results` +4. Copy the events slice; deep-copy only the `Match.Teams` of events being modified (`Team.Result` is a pointer — mutating a shared `*TeamResult` would corrupt the caller's data and, worse, the cached payload) +5. For each event where `needsScore`, look up forward then reversed; apply on a unique hit +6. Return the copy plus a fill count + +## Tests + +- [ ] Fills a `{"outcome":null,"gameWins":0}` event → renders `✅ MKOI 2–0 KC` with `MKOI` via `formatEventLine` +- [ ] Reversed orientation: Leaguepedia has `KC vs MKOI 0-2` → still yields `MKOI 2–0 KC`, winner bolded correctly +- [ ] Already-scored event is **never** touched, even when a conflicting Leaguepedia row exists +- [ ] `unstarted` and `inProgress` events untouched +- [ ] No matching row → unchanged, still `☑️ … score pending` +- [ ] Two candidates for one key → unchanged + fill count 0 +- [ ] Name normalization: `"paiN Gaming"` vs `"PaiN Gaming"`, extra whitespace, punctuation +- [ ] Different UTC date, same teams → no match +- [ ] Tie (`1-1` Bo2) → both filled, neither bolded +- [ ] Event with <2 teams → no panic +- [ ] **Aliasing**: input events unchanged after `mergeScores` (assert the original `*TeamResult` pointers still read `Outcome == ""`) +- [ ] Empty results / empty events → no-op +- [ ] Schedule fields (`StartTime`, `Strategy.Count`, `BlockName`, `League`) identical before/after + +## Success Criteria + +- [ ] All tests green; existing `format_test.go` untouched and still passing +- [ ] `mergeScores` is pure — no I/O, no globals, no clock +- [ ] Input slice and its pointees provably unmutated +- [ ] Ambiguity and orientation both covered by dedicated tests + +## Risk Assessment + +**Pointer aliasing is the sharpest hazard.** `Team.Result` is `*TeamResult`, and enriched events may be the same objects written into the Mongo cache by `GetEventsWithFallback`. In-place mutation would persist a Leaguepedia-derived score into the lolesports cache, making a later cache read indistinguishable from real upstream data. Step 4 + the aliasing test exist for this. + +**Silent orientation inversion** would report every score backwards while looking perfectly healthy. Dedicated test. + +**Name drift over a season.** Rebrands mid-split break the join. Failure mode is benign (falls back to `score pending`), but log unmatched events at debug so it is diagnosable. diff --git a/plans/260726-0952-lol-leaguepedia-score-enrichment/phase-04-wire-in.md b/plans/260726-0952-lol-leaguepedia-score-enrichment/phase-04-wire-in.md new file mode 100644 index 0000000..6e91830 --- /dev/null +++ b/plans/260726-0952-lol-leaguepedia-score-enrichment/phase-04-wire-in.md @@ -0,0 +1,114 @@ +--- +phase: 4 +title: "Wire-in" +status: pending +effort: "S" +priority: P2 +dependencies: [3] +--- + +# Phase 4: Wire-in + +## Overview + +Connect client + join to the four commands and the daily push, in one shared place. Add the CC-BY-SA attribution and document the env var. + +## Requirements + +- Every render path benefits: `/lol`, `/lol_tomorrow`, `/lol_this_week`, `/lol_next_week`, daily push +- **Zero** extra HTTP requests when nothing needs enrichment +- No new user-visible error path — enrichment failure is invisible except for the unchanged `☑️ score pending` +- One integration point, not five (DRY) + +## Architecture + +`replyForRange` (`handlers.go:96-124`) already funnels all four commands through fetch → filter → render. `runDailyPush` (`cron.go:174-253`) duplicates that sequence. So there are exactly **two** call sites. + +Insert between filter and render: + +```go +filtered := FilterMajor(events) +filtered = s.enrichScores(ctx, filtered, from, to) // new +text = renderDay(filtered, from, emptyLine) +``` + +```go +// enrichScores fills scores lolesports has not published, from Leaguepedia. +// Best-effort by design: any failure returns events unchanged so the digest +// still sends. Makes no request when nothing needs enrichment. +func (s *state) enrichScores(ctx context.Context, events []ScheduleEvent, from, to time.Time) []ScheduleEvent { + if !leaguepediaEnabled() || !slices.ContainsFunc(events, needsScore) { + return events + } + results, err := s.leaguepedia.FetchResults(ctx, from, to) + if err != nil { + log.Warn("lol_leaguepedia_fetch_fail", "err", err) + return events + } + merged, filled := mergeScores(events, results) + log.Info("lol_leaguepedia_enriched", "filled", filled, "candidates", ...) + return merged +} +``` + +The `ContainsFunc(events, needsScore)` short-circuit is what keeps steady-state cost at zero — on a normal day no event needs a score and no request is made. + +`state` gains `leaguepedia *leaguepediaClient`, wired in `New` (`lol.go:17-22`) beside `client: &Client{}`. + +### Attribution + +CC-BY-SA 3.0 obliges crediting Leaguepedia when its data is displayed. Cheapest honest placement: a footer line appended **only when at least one score was filled**, so unaffected digests stay clean. + +``` +Scores via Leaguepedia (CC BY-SA 3.0) +``` + +This makes attribution conditional on actual use — decide during implementation whether `renderDay`/`renderWeek` take a footer argument or the caller appends. Prefer the caller appending, to keep the renderers pure. + +## Related Code Files + +- Modify: `internal/modules/lol/handlers.go` (one call in `replyForRange`) +- Modify: `internal/modules/lol/cron.go` (one call in `runDailyPush`) +- Modify: `internal/modules/lol/lol.go` (wire `leaguepedia` into `state`) +- Modify: `docs/deploy-coolify-selfhosted.md` (add `LOL_LEAGUEPEDIA_ENABLED` to the env table at line ~28, `Required: optional`) +- Modify: `.env.example` (the docs table links to it, so both must stay in sync) +- Create: `internal/modules/lol/score_enrich_integration_test.go` (or extend `handlers_test.go`) + +## Implementation Steps + +1. Add `leaguepedia *leaguepediaClient` to `state` (`handlers.go:17-29`); wire in `New` +2. Add `enrichScores` method — put it in `score_enrich.go` next to `mergeScores`, not in `handlers.go`, keeping handlers thin +3. Call it in `replyForRange` after `FilterMajor` +4. Call it in `runDailyPush` after `FilterMajor`, **before** `claimDailyPush` — a Leaguepedia hiccup must not consume the day's idempotency claim, matching the existing rationale at `cron.go:195-197` +5. Conditional attribution footer when `filled > 0` +6. Document the env var; note default-off and the rate-limit rationale + +## Tests + +Existing `handlers_test.go` already injects a fake lolesports server; extend that harness with a fake Cargo server. + +- [ ] `/lol` with an unscored event + Leaguepedia having it → reply contains `✅` and the real score +- [ ] Same, Leaguepedia 500 → reply contains `☑️ … score pending`, **no error to user** +- [ ] Same, env disabled → `☑️ … score pending`, zero Cargo requests +- [ ] All events already scored → zero Cargo requests (assert counter == 0) +- [ ] Daily push enriches, and a Cargo failure still sends the digest +- [ ] Daily push: Cargo failure does **not** consume the push claim (second run still sends) +- [ ] Attribution footer present only when a score was filled +- [ ] Week view enriches across multiple days in one Cargo request (assert counter == 1) + +## Success Criteria + +- [ ] `go test ./...` fully green +- [ ] `go vet ./...` clean; changed files gofmt-clean modulo the repo's pre-existing CRLF +- [ ] Feature off by default — behaviour byte-identical to `570ee94` with the env var unset +- [ ] Zero Cargo requests in the steady state +- [ ] Attribution shown when data is used +- [ ] Manual check with the var set against the real API, confirming a real score renders + +## Risk Assessment + +**Regression surface is the four commands + daily push.** Mitigated by the default-off flag: with it unset the diff is inert, so a bad merge cannot degrade production until someone opts in. + +**Latency.** Adds up to `leaguepediaTimeout` (6s) to a `/lol` reply when a gap exists. Telegram tolerates this, and the short-circuit means it only happens on stall days. If it becomes annoying, cache — deliberately deferred per `plan.md` (the TTL index's partial filter is scoped to `matches:`, so a new prefix would never expire and needs its own index). + +**Push ordering.** Step 4 exists because enriching after the claim would let a Cargo failure burn the day's claim and silently skip the digest. diff --git a/plans/260726-0952-lol-leaguepedia-score-enrichment/plan.md b/plans/260726-0952-lol-leaguepedia-score-enrichment/plan.md new file mode 100644 index 0000000..8c19cbb --- /dev/null +++ b/plans/260726-0952-lol-leaguepedia-score-enrichment/plan.md @@ -0,0 +1,89 @@ +--- +title: "LoL Leaguepedia score enrichment" +description: "Fill missing series scores from Leaguepedia when lolesports marks a match completed but publishes no result" +status: pending +priority: P2 +branch: "main" +tags: [lol, external-api, enrichment] +blockedBy: [] +blocks: [] +created: "2026-07-26T02:54:48.516Z" +createdBy: "ck:plan" +source: skill +--- + +# LoL Leaguepedia score enrichment + +## Overview + +lolesports flips `event.state` to `completed` off the broadcast timeline but fills `result.outcome`/`result.gameWins` from a separate per-game ingestion path. When that path stalls, finished matches carry `{"outcome": null, "gameWins": 0}`. Commit `570ee94` made this render honestly as `☑️ … score pending`; this plan restores a real score by filling the gap from Leaguepedia. + +**Additive only.** lolesports stays authoritative for schedule, times, Bo count, block names, and any score it *does* publish. Leaguepedia is consulted solely for events where `scoreIsPublished(t1, t2) == false`. Every failure path — disabled, throttled, network error, ambiguous match, no row — falls through to today's `☑️ … score pending`. + +Evidence base: `plans/reports/260726-0941-lol-score-source-alternatives.md`. + +## Key Design Decisions + +**Join on team name + date, not team codes.** Riot's payload already carries `Team.Name` (`"Movistar KOI"`), and Leaguepedia's `MatchSchedule.Team1`/`Team2` are also full names. No code-mapping table, no `Teams` table query, no per-split `OverviewPage` string to maintain. This removes the largest maintenance risk identified in research. + +**Query by UTC time window, not by tournament.** One Cargo request per render covers every league in the window. Avoids `OverviewPage` naming (`LEC/2026 Season/Split 3`), which drifts every split. + +**No new persistence.** The existing TTL index (`startup.go:38-52`) has a partial filter scoped to `_id` in `[matches:, matches;)`, so any new cache prefix would never expire. Enrichment runs inline per render; the daily push is 1 request/day and `/lol*` commands are user-triggered and low-volume. Revisit only if rate limiting proves to bite. + +**Off by default.** Anonymous Fandom Cargo throttled hard during research (1 success in ~15 burst attempts, then connection refused). Ship behind `LOL_LEAGUEPEDIA_ENABLED`, matching the env-const + `os.Getenv` pattern in `internal/modules/misc/wheelofnames_api_client.go:62`. + +## Phases + +| Phase | Name | Status | +|-------|------|--------| +| 1 | [Feasibility Gate](./phase-01-feasibility-gate.md) | Pending | +| 2 | [Cargo Client](./phase-02-cargo-client.md) | Pending | +| 3 | [Score Join](./phase-03-score-join.md) | Pending | +| 4 | [Wire-in](./phase-04-wire-in.md) | Pending | + +Phase 1 is a **kill switch**, not a formality. It is cheap (a handful of curl calls) and can invalidate Phases 2-4 entirely. Do not write production code before it passes. + +## Architecture + +``` +replyForRange (handlers.go) ─┐ + ├─→ GetEventsWithFallback ─→ FilterMajor ─→ enrichScores ─→ render +runDailyPush (cron.go) ─┘ │ + │ only if any event has + │ state==completed && !scoreIsPublished + ▼ + leaguepediaClient.FetchResults(from, to) + │ + ┌──────────────┴──────────────┐ + 1 match found 0 or 2+ / error + │ │ + fill GameWins+Outcome leave untouched + ▼ ▼ + ✅ MKOI 2–0 KC ☑️ MKOI vs KC · score pending +``` + +Invariants (assert in tests): +- Never overwrites a score lolesports published +- Never mutates a non-`completed` event +- Never changes schedule fields (`StartTime`, `Strategy`, `BlockName`, `League`) +- Any error → input events returned unchanged, digest still sends + +## Acceptance Criteria + +- [ ] `✅ MKOI 2–0 KC · Bo3 (Week 1)` renders for a lolesports-unscored match that Leaguepedia has +- [ ] `☑️ … score pending` still renders when Leaguepedia lacks it, errors, is throttled, or is disabled +- [ ] Scores lolesports *does* publish are never altered +- [ ] Enrichment never blocks or fails a digest — no new error path reaches the user +- [ ] Zero extra HTTP requests when every completed event already has a score +- [ ] `go test ./...` green; no network access in tests (`httptest` only) +- [ ] Feature off by default; single env var flips it + +## Dependencies + +No cross-plan dependencies. All 8 existing plans are `completed` and scoped to the `stock` module. + +External: `lol.fandom.com` Cargo API (CC-BY-SA 3.0 — attribution obligation, see Phase 4). + +## Open Questions + +Tracked in Phase 1; all must be answered before Phase 2 starts. diff --git a/plans/reports/260726-0941-lol-score-source-alternatives.md b/plans/reports/260726-0941-lol-score-source-alternatives.md new file mode 100644 index 0000000..3f7afcf --- /dev/null +++ b/plans/reports/260726-0941-lol-score-source-alternatives.md @@ -0,0 +1,80 @@ +# Research: Alternative LoL Result Source (restore real scores) + +Conducted 2026-07-26 09:41 ICT. Trigger: lolesports marks matches `completed` without publishing `result.gameWins`/`outcome`, so the digest now shows `☑️ … score pending` (commit `570ee94`) instead of a score. + +Goal: source that has real series scores for finished matches, so `✅ MKOI 2–0 KC` returns. + +## Executive Summary + +**Any wrapper of lolesports inherits the bug.** That eliminates most "alternative LoL API" search hits — they proxy the same upstream store. Verified empirically: `getCompletedEvents` returns the identical `MKOI 0-0 KC`, so the gap is store-wide at Riot, not endpoint-specific. + +Only genuinely independent ingestion helps: community wikis (Leaguepedia, Liquipedia) or commercial providers (PandaScore, Abios, GRID). + +**Recommendation: don't build it yet.** The schedule half of lolesports is reliable; only results are broken, and they self-heal. A second provider means a new HTTP dep + team-name→code mapping + cache + failure handling — real complexity for a cosmetic win on an intermittent upstream failure. Current behavior is honest. Wait, measure how often/long Riot stalls, and build the enrichment only if chronic. + +If it does prove chronic → **Leaguepedia Cargo, as opportunistic enrichment, not migration.** + +## Empirical Findings (this session) + +| Probe | Result | +|---|---| +| `getSchedule` | 6 events `completed` w/ `{"outcome":null,"gameWins":0}`; clean cutoff — all ≤`2026-07-25T11:30Z` scored, all ≥`14:30Z` not | +| `getEventDetails` (match `115548681803406271`) | games 1&2 `state:unstarted` but **7 VODs each**; game 3 `unstarted`/0 VODs → played + broadcast, results never ingested | +| `getEventDetails` (scored control) | games `completed`/8 VODs, game 3 `unneeded` | +| `getCompletedEvents?tournamentId=115548681802226458` | **same staleness** — `MKOI 0-0 KC`. Rules out a free same-provider fix | +| `getStandings` | active stage returns `Playoffs` only; no useful Week-1 series data | +| Leaguepedia Cargo `MatchSchedule` | API works, returns populated `Team1Score`/`Team2Score`/`Winner` for finished matches. **Heavily throttled anonymously — 1 success in ~15 attempts** | + +Notable: the one successful Cargo query returned scored results for matches at `2026-07-25 12:00–14:00Z` — inside the window Riot has failed on (cutoff `11:30Z`). Suggestive that Leaguepedia is ahead of Riot there, but those were amateur/university leagues with different editors, so **not** proof for LEC. + +## Provider Comparison + +| Provider | Cost | Auth | Independent ingestion? | Verdict | +|---|---|---|---|---| +| **Leaguepedia** (Fandom Cargo) | free | none | ✅ community-edited | **Best fit.** `MatchSchedule.Team1Score/Team2Score/Winner/BestOf`. Harsh anon rate limit; uses team *names* not codes; CC-BY-SA attribution required | +| **Liquipedia** LPDB | free tier | API key | ✅ | 60 req/hr. Free tier needs non-commercial + **open-sourced project** + custom UA + attribution. Repo is public → likely qualifies. Must apply for key | +| **PandaScore** | free tier 1k req/hr (schedules+results); paid from €400/mo/game | key | ✅ | Free tier may suffice for a daily digest. Verify results are in free scope — pricing implies historical is paid. Betting-adjacent ToS restrictions on stats plans | +| **Abios / GRID / Sportradar** | $2k–10k/mo | key | ✅ | Absurd for a Telegram bot. **Rule out** | +| lolesportsapi.com | free 500/mo | key | ❌ wraps lolesports | **Useless** — inherits the bug | +| Pupix/lol-esports-api | free | — | ❌ wrapper | **Useless** | +| Apify LoL scraper | paid compute | token | ❌ scrapes lolesports | **Useless** | + +## Recommended Design (if/when built) + +Enrichment layer, not a migration. lolesports stays authoritative for schedule, times, Bo count, block names — all correct today. + +``` +GetEventsWithFallback (lolesports) → events + └─ for events where state==completed && !scoreIsPublished(t1,t2): + └─ 1 batched Leaguepedia Cargo query for the day window + ├─ hit → fill gameWins + outcome → renders ✅ with real score + └─ miss → unchanged ☑️ … score pending +``` + +Properties: +- `scoreIsPublished` (already shipped) is the exact trigger — no new detection logic +- 1 extra request per digest, only when a gap exists; cache alongside existing `cacheRecord` +- Degrades to current behavior on any failure — strictly additive, no regression path +- Never overwrites a Riot-published score; fills gaps only + +Work required: +1. Team identity mapping — Leaguepedia returns `Movistar KOI`, we display `MKOI`. Cargo `Teams` table has a `Short` field; needs a cached lookup or a static map for the ~12 allowlisted leagues +2. `OverviewPage` naming per league/split (e.g. `LEC/2026 Season/Split 3`) — brittle, changes each split. Alternatively match on team names + UTC timestamp, which avoids page naming entirely +3. Rate-limit discipline — cache aggressively, custom UA w/ contact, single daily query +4. CC-BY-SA attribution somewhere user-visible + +## Cost/Benefit + +Against: new external dep on a community wiki (editorial lag, schema drift, throttling); name→code mapping is ongoing maintenance; ~150–250 LOC + tests; benefit is cosmetic — current output is already truthful. + +For: restores the informative digest; useful independently if Riot's stalls become routine. + +**Verdict: defer.** Cheapest next step is observation, not code — the current fix already fails safe. + +## Unresolved Questions + +1. **Could not verify** Leaguepedia holds the specific missing LEC/LCS scores (`MKOI vs KC`, `FLY vs LYON`, `DIG vs SEN`) — rate-limited on every targeted attempt; WebFetch of the wiki page returned HTTP 402. **This is the single blocking unknown**; retry the Cargo query later before committing to Leaguepedia. +2. Is Riot's stall a one-off or recurring? No history collected. Determines whether any of this is worth building. +3. Does PandaScore's free tier actually include completed-match results, or schedules only? Pricing page ambiguous. +4. Does Liquipedia grant LPDB keys to a public hobby Telegram bot? Terms say non-commercial + open-source, which fits, but approval is manual. +5. Leaguepedia editorial latency for major leagues specifically — unmeasured. If editors lag Riot, the enrichment adds nothing.