Repository navigation
Alerts: troubleshooting checklist for a missing alert - #812
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Automations to automatically generate PRs for you. |
Add a missing-alerts checklist item for channel types the Hobby plan doesn't send to, and a warning on the groups page that a group alert override with no alert channels alerts no one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
hlenke
left a comment
There was a problem hiding this comment.
Ready to merge · Every claim in the new troubleshooting section and the group warning matches the alerting and webapp source; one optional wording tweak inline.
Nothing blocks this PR. CI passes (docs-gate, preview-checks, static-docs-checks, Mintlify link-rot and spellcheck), it's mergeable, and main hasn't changed either file since the merge base.
Placement: the new ## Troubleshooting missing alerts section sits at the end of communicate/alerts/overview.mdx, after "Testing your alerts". That's the right spot: it doesn't push down the page's main content.
Overlap: #814 also edits platform/groups.mdx, but in a different hunk ("Mixing Checks and Monitors in a Group", around line 167; this PR adds the Warning under "Retries & Alerting", around line 128). They don't conflict and can land in either order.
Inline: 0 blockers, 0 important, 1 wording (optional).
What I checked
- Read the whole diff from merge base
e0e0277f:communicate/alerts/overview.mdx(+26) andplatform/groups.mdx(+4). Checked the heading outline of both pages at the PR head. gh pr checks 812: all pass.gh pr view: MERGEABLE / CLEAN.git log e0e0277f..origin/main: no new commits onmain.pr_impact.py: only an info note that/communicate/alerts/overviewis a hub page. The PR changes no slugs or anchors.- Anchors exist on
main:configuration#escalation-configuration("## Escalation Configuration"),configuration#configuration-inheritance("### Configuration Inheritance"), andmaintenance-windows/overview#silence-alerts("#### Silence alerts"). - Hobby channel allowlist: monorepo
apps/backend/api/daemons/alerting-daemon/src/helpers.ts(origin/main@84921d2637). On Hobby, only email and Slack/Slack App channels, plain webhooks, and Microsoft Teams, Discord and Telegram webhooks pass the plan filter. Every other channel is dropped before notifications are created. This matches line 126. - Notification events and degraded recovery: the same file sends a recovery from degraded only to channels with both degraded and recovery turned on. This matches line 125.
- Channel selection:
AlertingDaemon.tsapplies the plan filter, then the event filter, to the check's and group's subscriptions. - UI labels in
apps/webapp/src: "Notification events" (PageAlertChannelBuilder.i18n.ts); "a check degrades" (channel builder); "Delivery status" and "Notification result" (AlertNotificationsDetail.vue); "Retries & Alerting" (group builder tabs and check builders); "Group override" + "for" + section title, which renders as "Group override for alert settings" (OverrideByGroupCheckbox); "Use the global account notification settings" (ResourceAlertSettingsGlobalSelector.vue); "Muted" (i18n/en.ts); and the "N sent" count in the check overview alerts row (CheckOverviewAlertsRowLogsLink.vue). - Group override defaults: the webapp's alert-settings unit tests cover the reset to defaults when the group override is turned on. The PR's own end-to-end run (described in its body) showed that this selects the global settings with no group channels.
- Local app unavailable (BUSY), so I verified against source. I didn't rerun the PR author's end-to-end scenarios.
What and why
Users whose check failed without a notification have no single place that tells them what to look at. The alerts overview explains states, thresholds and testing, the configuration page explains settings, and the notification log page explains deliveries, but nothing walks through "why didn't I get this alert?".
This adds a Troubleshooting missing alerts section to
communicate/alerts/overview.mdx, right after "Testing your alerts". It starts from the check's Alerts table (which shows whether Checkly generated an alert and how many notifications went out) and splits the causes into three cases:-): not subscribed (and group override uses the group's channels), the channel's notification events, muted check or group, a maintenance window silencing alerts, a channel subscribed after the failure started;It links to the existing configuration, maintenance window and notification log pages instead of repeating them. No existing text changed.
Two follow-ups:
platform/groups.mdxgets a warning: turning on Group override for alert settings selects Use the global account notification settings, but the group's alert channels still replace the checks' own, so a group without channels alerts no one.How each claim was verified
Local stack at monorepo
origin/main(e086a77755): API, webapp, scheduler, results daemon, alerting daemon and the Go runner, so checks really ran. Each scenario used its own API check and webhook channels pointing at a local listener; the check's target returned 200 or 500 on demand. "Alerts table" =check_alertsrows and the check overview page; "notifications" = notification log rows, the overview's Notifications column and requests the listener received.alerts/notifications?alertNotificationId=…-when nothing was sentALERT_FAILURE, one deliveryuseGlobalAlertSettingstrue, no group subscriptions). A failing check in that group, subscribed to its own webhook channel: failure alert generated, no notification log rows, nothing at the listenerWhat couldn't be tested
127.0.0.1for the local listener. Nothing else was changed.Left out on purpose: how recoveries reach channels that didn't get the failure notification (treated as a bug).
🤖 Generated with Claude Code