Notification groups
Bundle channels into routes so changing your on-call rotation is one edit, not fifty.
A group is a named bundle of channels:
"Backend on-call" = Slack #backend-alerts
+ email backend@company.com
+ browser push to the duty engineer"Backend on-call" = Slack #backend-alerts
+ email backend@company.com
+ browser push to the duty engineerPoint fifty monitors at that group and changing the on-call rotation is one edit. Point them at channels individually and it is fifty.
When a group is paged, every channel in it fires in parallel.
Create one#
- Open Settings → Notification Groups → New Group.
- Name it after the audience or the severity:
Backend on-call,Critical infra,Low priority. - Tick the channels it should contain.
- Save.
Then open any monitor, go to Notifications, and assign the group. A monitor can hold several groups. Every channel in every assigned group fires, and Drumbeats removes duplicates when a channel appears in more than one.
Name groups after who gets woken up#
A group called Critical infra tells you something. A group called Cron monitors does not, because the monitor type has nothing to do with who should respond.
| Monitor | Group | Fans out to |
|---|---|---|
| Nightly backup | DBA on-call | Slack #dba, email dba@company.com |
| Invoice generator | Finance on-call | Slack #finance, email finance@company.com |
| Queue worker | Backend on-call | Slack #backend-alerts, push to the duty engineer |
| Health check | Low priority | Slack #monitoring |
Layer groups instead of duplicating them#
Two groups on one monitor compose. Both fire, so you can keep a base group for everyday visibility and add a severity group on top without rebuilding the channel list:
Payment processor → "Backend on-call" (Slack + email)
→ "Critical infra" (PagerDuty)Payment processor → "Backend on-call" (Slack + email)
→ "Critical infra" (PagerDuty)Skip the group for a one-off destination#
A monitor can also point at channels directly, without a group. Both lists fire.
Use this sparingly. A direct channel is invisible to anyone reading the group list later, so it is the routing that gets forgotten during a handover. Reach for it only when exactly one monitor needs exactly one destination and a group would be ceremony.
What happens when it breaks#
| Situation | What you get |
|---|---|
| Monitor has no group and no direct channel | Nothing. The incident is recorded and nobody is paged |
| Group exists but holds no channels | Nothing. Same outcome |
| One channel in the group fails, others succeed | The working channels get the alert. The failed one is not retried |
| Every channel in the group fails | Drumbeats retries the delivery for about a minute, then stops |
How you get alerted during setup#
Add a quiet channel to a new group while you are wiring it up, such as a personal Telegram chat or a #dev-alerts Slack channel. Trigger a real failure with a manual failure ping and confirm the whole path works before the group is attached to anything that pages a human. Remove the quiet channel once you trust it.
Next#
Channels for setting up the endpoints a group bundles. Alert logic for what decides a group is paged at all. REST API notification groups for managing these from code.