mirror of
https://github.com/tiennm99/claude-status-webhook.git
synced 2026-04-17 11:20:30 +00:00
feat: notify admin via Telegram on webhook errors
Send error details to ADMIN_CHAT_ID (optional secret) whenever the statuspage webhook handler hits an error path. Uses waitUntil() so the notification never blocks the 200 response.
This commit is contained in:
@@ -22,6 +22,7 @@ No linter configured.
|
|||||||
|
|
||||||
- `BOT_TOKEN` — Telegram bot token
|
- `BOT_TOKEN` — Telegram bot token
|
||||||
- `WEBHOOK_SECRET` — Secret token in Statuspage webhook URL path
|
- `WEBHOOK_SECRET` — Secret token in Statuspage webhook URL path
|
||||||
|
- `ADMIN_CHAT_ID` — Telegram chat ID to receive webhook error notifications (optional)
|
||||||
|
|
||||||
## Architecture
|
## Architecture
|
||||||
|
|
||||||
@@ -39,7 +40,7 @@ Cloudflare Workers with two entry points exported from `src/index.js`:
|
|||||||
|
|
||||||
### Data Flow
|
### Data Flow
|
||||||
|
|
||||||
1. **Statuspage → Worker**: Webhook POST → verify URL secret (timing-safe via `crypto-utils.js`) → parse incident/component event → filter subscribers by type + component → `sendBatch` to CF Queue
|
1. **Statuspage → Worker**: Webhook POST → verify URL secret (timing-safe via `crypto-utils.js`) → parse incident/component event → filter subscribers by type + component → `sendBatch` to CF Queue. On any error, admin is notified via Telegram (`admin-notifier.js`, non-blocking via `waitUntil`)
|
||||||
2. **Queue → Telegram**: Consumer processes batches of 30 → `sendMessage` via `telegram-api.js` helper → auto-removes blocked subscribers (403/400), retries on 429
|
2. **Queue → Telegram**: Consumer processes batches of 30 → `sendMessage` via `telegram-api.js` helper → auto-removes blocked subscribers (403/400), retries on 429
|
||||||
3. **User → Bot**: Telegram webhook → grammY handles `/help`, `/start`, `/stop`, `/status`, `/subscribe`, `/history`, `/uptime` commands → reads/writes KV
|
3. **User → Bot**: Telegram webhook → grammY handles `/help`, `/start`, `/stop`, `/status`, `/subscribe`, `/history`, `/uptime` commands → reads/writes KV
|
||||||
|
|
||||||
|
|||||||
@@ -55,6 +55,10 @@ npx wrangler secret put BOT_TOKEN
|
|||||||
|
|
||||||
npx wrangler secret put WEBHOOK_SECRET
|
npx wrangler secret put WEBHOOK_SECRET
|
||||||
# Choose a random secret string for the Statuspage webhook URL
|
# Choose a random secret string for the Statuspage webhook URL
|
||||||
|
|
||||||
|
npx wrangler secret put ADMIN_CHAT_ID
|
||||||
|
# (Optional) Your Telegram chat ID — receive webhook error alerts via Telegram
|
||||||
|
# Use the bot's /info command to find your chat ID
|
||||||
```
|
```
|
||||||
|
|
||||||
### 5. Deploy
|
### 5. Deploy
|
||||||
|
|||||||
@@ -63,6 +63,7 @@ A middleware in `index.js` normalizes double slashes in URL paths (Statuspage oc
|
|||||||
| `kv-store.js` | ~140 | KV key management, subscriber CRUD, filtered listing |
|
| `kv-store.js` | ~140 | KV key management, subscriber CRUD, filtered listing |
|
||||||
| `status-fetcher.js` | ~120 | Statuspage API client, HTML formatters, status helpers |
|
| `status-fetcher.js` | ~120 | Statuspage API client, HTML formatters, status helpers |
|
||||||
| `crypto-utils.js` | ~12 | Timing-safe string comparison (SHA-256 hashed) |
|
| `crypto-utils.js` | ~12 | Timing-safe string comparison (SHA-256 hashed) |
|
||||||
|
| `admin-notifier.js` | ~25 | Sends webhook error alerts to admin via Telegram (non-blocking) |
|
||||||
| `telegram-api.js` | ~9 | Telegram Bot API URL builder |
|
| `telegram-api.js` | ~9 | Telegram Bot API URL builder |
|
||||||
|
|
||||||
## KV Storage
|
## KV Storage
|
||||||
@@ -103,6 +104,7 @@ Binding: `claude-status` queue
|
|||||||
Enabled via `wrangler.jsonc` `observability` config. Automatic — no code changes required.
|
Enabled via `wrangler.jsonc` `observability` config. Automatic — no code changes required.
|
||||||
|
|
||||||
- **Logs**: All `console.log`/`console.error` calls, request metadata, exceptions. Persisted with invocation logs enabled. Free tier: 200k logs/day, 3-day retention.
|
- **Logs**: All `console.log`/`console.error` calls, request metadata, exceptions. Persisted with invocation logs enabled. Free tier: 200k logs/day, 3-day retention.
|
||||||
|
- **Admin alerts**: Webhook errors are forwarded to admin Telegram chat in real-time (requires `ADMIN_CHAT_ID` secret)
|
||||||
- **Traces**: Automatic instrumentation of fetch calls, KV reads, queue operations. Persisted.
|
- **Traces**: Automatic instrumentation of fetch calls, KV reads, queue operations. Persisted.
|
||||||
- **Sampling**: 100% (`head_sampling_rate: 1`) for both logs and traces — reduce for high-volume scenarios
|
- **Sampling**: 100% (`head_sampling_rate: 1`) for both logs and traces — reduce for high-volume scenarios
|
||||||
- **Dashboard**: Cloudflare Dashboard → Workers → Observability
|
- **Dashboard**: Cloudflare Dashboard → Workers → Observability
|
||||||
@@ -110,9 +112,10 @@ Enabled via `wrangler.jsonc` `observability` config. Automatic — no code chang
|
|||||||
## Security
|
## Security
|
||||||
|
|
||||||
- **Statuspage webhook always-200**: Handler always returns HTTP 200 (even on errors) to prevent Statuspage from removing the webhook subscription. Errors are logged, not surfaced as HTTP status codes.
|
- **Statuspage webhook always-200**: Handler always returns HTTP 200 (even on errors) to prevent Statuspage from removing the webhook subscription. Errors are logged, not surfaced as HTTP status codes.
|
||||||
|
- **Admin error alerts**: All webhook errors send a Telegram notification to `ADMIN_CHAT_ID` (if set) via `admin-notifier.js`. Uses `waitUntil()` to avoid blocking the response. Silently fails if `ADMIN_CHAT_ID` is not configured.
|
||||||
- **Statuspage webhook auth**: URL path secret validated with timing-safe SHA-256 comparison
|
- **Statuspage webhook auth**: URL path secret validated with timing-safe SHA-256 comparison
|
||||||
- **Telegram webhook**: Registered via `setup-bot.js` — Telegram only sends to the registered URL
|
- **Telegram webhook**: Registered via `setup-bot.js` — Telegram only sends to the registered URL
|
||||||
- **No secrets in code**: `BOT_TOKEN` and `WEBHOOK_SECRET` stored as Cloudflare secrets
|
- **No secrets in code**: `BOT_TOKEN`, `WEBHOOK_SECRET`, and `ADMIN_CHAT_ID` stored as Cloudflare secrets
|
||||||
- **HTML injection**: All user/external strings passed through `escapeHtml()` before Telegram HTML rendering
|
- **HTML injection**: All user/external strings passed through `escapeHtml()` before Telegram HTML rendering
|
||||||
|
|
||||||
## Dependencies
|
## Dependencies
|
||||||
|
|||||||
32
src/admin-notifier.js
Normal file
32
src/admin-notifier.js
Normal file
@@ -0,0 +1,32 @@
|
|||||||
|
import { telegramUrl } from "./telegram-api.js";
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Send an error notification to the admin via Telegram.
|
||||||
|
* Silently fails — admin alerts must never break the main flow.
|
||||||
|
* @param {object} env - Worker env bindings
|
||||||
|
* @param {string} reason - Short error reason
|
||||||
|
* @param {Record<string, unknown>} [details] - Extra context to include
|
||||||
|
*/
|
||||||
|
export async function notifyAdmin(env, reason, details = {}) {
|
||||||
|
if (!env.ADMIN_CHAT_ID || !env.BOT_TOKEN) return;
|
||||||
|
|
||||||
|
const text = [
|
||||||
|
`<b>[Webhook Error]</b> ${reason}`,
|
||||||
|
`<pre>${JSON.stringify(details, null, 2)}</pre>`,
|
||||||
|
].join("\n");
|
||||||
|
|
||||||
|
try {
|
||||||
|
await fetch(telegramUrl(env.BOT_TOKEN, "sendMessage"), {
|
||||||
|
method: "POST",
|
||||||
|
headers: { "Content-Type": "application/json" },
|
||||||
|
body: JSON.stringify({
|
||||||
|
chat_id: env.ADMIN_CHAT_ID,
|
||||||
|
text,
|
||||||
|
parse_mode: "HTML",
|
||||||
|
disable_web_page_preview: true,
|
||||||
|
}),
|
||||||
|
});
|
||||||
|
} catch {
|
||||||
|
// Swallow — never let admin notification break the webhook
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,6 +1,7 @@
|
|||||||
import { getSubscribersByType } from "./kv-store.js";
|
import { getSubscribersByType } from "./kv-store.js";
|
||||||
import { humanizeStatus, escapeHtml } from "./status-fetcher.js";
|
import { humanizeStatus, escapeHtml } from "./status-fetcher.js";
|
||||||
import { timingSafeEqual } from "./crypto-utils.js";
|
import { timingSafeEqual } from "./crypto-utils.js";
|
||||||
|
import { notifyAdmin } from "./admin-notifier.js";
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Format incident event as Telegram HTML message
|
* Format incident event as Telegram HTML message
|
||||||
@@ -42,13 +43,17 @@ export async function handleStatuspageWebhook(c) {
|
|||||||
// Validate URL secret (timing-safe)
|
// Validate URL secret (timing-safe)
|
||||||
// Guard against misconfigured deploy (undefined env var)
|
// Guard against misconfigured deploy (undefined env var)
|
||||||
if (!c.env.WEBHOOK_SECRET) {
|
if (!c.env.WEBHOOK_SECRET) {
|
||||||
console.error(JSON.stringify({ event: "webhook_error", reason: "WEBHOOK_SECRET not configured" }));
|
const reason = "WEBHOOK_SECRET not configured";
|
||||||
|
console.error(JSON.stringify({ event: "webhook_error", reason }));
|
||||||
|
c.executionCtx.waitUntil(notifyAdmin(c.env, reason));
|
||||||
return c.text("OK", 200);
|
return c.text("OK", 200);
|
||||||
}
|
}
|
||||||
|
|
||||||
const secret = c.req.param("secret");
|
const secret = c.req.param("secret");
|
||||||
if (!await timingSafeEqual(secret, c.env.WEBHOOK_SECRET)) {
|
if (!await timingSafeEqual(secret, c.env.WEBHOOK_SECRET)) {
|
||||||
console.error(JSON.stringify({ event: "webhook_error", reason: "invalid_secret" }));
|
const reason = "invalid_secret";
|
||||||
|
console.error(JSON.stringify({ event: "webhook_error", reason }));
|
||||||
|
c.executionCtx.waitUntil(notifyAdmin(c.env, reason));
|
||||||
return c.text("OK", 200);
|
return c.text("OK", 200);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -57,13 +62,18 @@ export async function handleStatuspageWebhook(c) {
|
|||||||
try {
|
try {
|
||||||
body = await c.req.json();
|
body = await c.req.json();
|
||||||
} catch {
|
} catch {
|
||||||
console.error(JSON.stringify({ event: "webhook_error", reason: "invalid_json" }));
|
const reason = "invalid_json";
|
||||||
|
console.error(JSON.stringify({ event: "webhook_error", reason }));
|
||||||
|
c.executionCtx.waitUntil(notifyAdmin(c.env, reason));
|
||||||
return c.text("OK", 200);
|
return c.text("OK", 200);
|
||||||
}
|
}
|
||||||
|
|
||||||
const eventType = body?.meta?.event_type;
|
const eventType = body?.meta?.event_type;
|
||||||
if (!eventType) {
|
if (!eventType) {
|
||||||
console.error(JSON.stringify({ event: "webhook_error", reason: "missing_event_type" }));
|
const reason = "missing_event_type";
|
||||||
|
const details = { keys: Object.keys(body || {}), meta: body?.meta ?? null };
|
||||||
|
console.error(JSON.stringify({ level: "error", event: "webhook_error", reason, ...details }));
|
||||||
|
c.executionCtx.waitUntil(notifyAdmin(c.env, reason, details));
|
||||||
return c.text("OK", 200);
|
return c.text("OK", 200);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -73,21 +83,27 @@ export async function handleStatuspageWebhook(c) {
|
|||||||
let category, html, componentName;
|
let category, html, componentName;
|
||||||
if (eventType.startsWith("incident.")) {
|
if (eventType.startsWith("incident.")) {
|
||||||
if (!body.incident) {
|
if (!body.incident) {
|
||||||
console.error(JSON.stringify({ event: "webhook_error", reason: "missing_incident_data", eventType }));
|
const reason = "missing_incident_data";
|
||||||
|
console.error(JSON.stringify({ event: "webhook_error", reason, eventType }));
|
||||||
|
c.executionCtx.waitUntil(notifyAdmin(c.env, reason, { eventType }));
|
||||||
return c.text("OK", 200);
|
return c.text("OK", 200);
|
||||||
}
|
}
|
||||||
category = "incident";
|
category = "incident";
|
||||||
html = formatIncidentMessage(body.incident);
|
html = formatIncidentMessage(body.incident);
|
||||||
} else if (eventType.startsWith("component.")) {
|
} else if (eventType.startsWith("component.")) {
|
||||||
if (!body.component) {
|
if (!body.component) {
|
||||||
console.error(JSON.stringify({ event: "webhook_error", reason: "missing_component_data", eventType }));
|
const reason = "missing_component_data";
|
||||||
|
console.error(JSON.stringify({ event: "webhook_error", reason, eventType }));
|
||||||
|
c.executionCtx.waitUntil(notifyAdmin(c.env, reason, { eventType }));
|
||||||
return c.text("OK", 200);
|
return c.text("OK", 200);
|
||||||
}
|
}
|
||||||
category = "component";
|
category = "component";
|
||||||
componentName = body.component.name || null;
|
componentName = body.component.name || null;
|
||||||
html = formatComponentMessage(body.component, body.component_update);
|
html = formatComponentMessage(body.component, body.component_update);
|
||||||
} else {
|
} else {
|
||||||
console.error(JSON.stringify({ event: "webhook_error", reason: "unknown_event_type", eventType }));
|
const reason = "unknown_event_type";
|
||||||
|
console.error(JSON.stringify({ event: "webhook_error", reason, eventType }));
|
||||||
|
c.executionCtx.waitUntil(notifyAdmin(c.env, reason, { eventType }));
|
||||||
return c.text("OK", 200);
|
return c.text("OK", 200);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -106,7 +122,9 @@ export async function handleStatuspageWebhook(c) {
|
|||||||
return c.text("OK", 200);
|
return c.text("OK", 200);
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
// Catch-all: log error but still return 200 to prevent Statuspage from removing us
|
// Catch-all: log error but still return 200 to prevent Statuspage from removing us
|
||||||
console.error(JSON.stringify({ event: "webhook_error", reason: "unexpected", error: err.message }));
|
const reason = "unexpected";
|
||||||
|
console.error(JSON.stringify({ event: "webhook_error", reason, error: err.message }));
|
||||||
|
c.executionCtx.waitUntil(notifyAdmin(c.env, reason, { error: err.message }));
|
||||||
return c.text("OK", 200);
|
return c.text("OK", 200);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -41,4 +41,5 @@
|
|||||||
// Secrets (set via `wrangler secret put`):
|
// Secrets (set via `wrangler secret put`):
|
||||||
// BOT_TOKEN - Telegram bot token
|
// BOT_TOKEN - Telegram bot token
|
||||||
// WEBHOOK_SECRET - Statuspage webhook URL secret
|
// WEBHOOK_SECRET - Statuspage webhook URL secret
|
||||||
|
// ADMIN_CHAT_ID - Telegram chat ID to receive error notifications
|
||||||
}
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user