Drumbeats logo
Drumbeats
Reference

Heartbeat monitors

How Drumbeats watches periodic worker loops. Interval syntax, grace period, ping cadence, and when to choose Heartbeat over Cron.

A Heartbeat monitor watches a job that pings on a repeating interval — every five minutes, every hour, every day. Drumbeats expects a success ping inside each interval plus a grace period, and opens an incident when the window passes without one.

Use a Heartbeat monitor for long-lived worker loops, polling jobs, and any process that should "stay alive" rather than run once on a scheduler.

Heartbeat vs Cron — when to use which

The two monitor types overlap in capability. Pick by how you think about cadence:

Use Heartbeat when…Use Cron when…
"This worker pings every 5 minutes.""This job runs at 02:00 every night."
The cadence is an interval (15m, 1h, 6h).The cadence is a clock-based schedule (0 2 *).
The process is long-running and self-pings on a loop.An external scheduler (cron, k8s, Airflow) invokes a discrete job.
You do not care which exact minute the ping arrives.The ping must arrive at a specific time of day.

In a hybrid case (a worker that runs every hour, started by a scheduler), prefer Cron — it gives you the exact-time alerting and dashboard timeline that schedule-bound jobs need.

Configuration

FieldRequiredExampleDescription
interval✓5m, 1h, 1d, 1wHow often Drumbeats expects a ping. Suffixes: m minutes, h hours, d days, w weeks.
grace_period_seconds—60Extra time after the interval before declaring the heartbeat missed. Default 5 minutes.
failure_tolerance—1Consecutive failure pings before flipping to DOWN. Default 1.
max_duration_seconds—300Optional: catches workers that hang mid-run when paired with start / success events.

The interval restarts from the moment Drumbeats receives the ping, not from a fixed wall-clock — so a 5-minute interval is "no more than 5m + grace between heartbeats," never an absolute "at :00, :05, :10."

Wire pings into your worker

The simplest case sends one ping per loop iteration:

python
import time, requests

API = "https://api.drumbeats.io/v1/ping/<monitor-id>"

while True:
    poll_queue()
    requests.get(f"{API}/success")
    time.sleep(300)
import time, requests

API = "https://api.drumbeats.io/v1/ping/<monitor-id>"

while True:
    poll_queue()
    requests.get(f"{API}/success")
    time.sleep(300)

If the work inside the loop can fail, branch:

python
import time, requests

API = "https://api.drumbeats.io/v1/ping/<monitor-id>"

while True:
    try:
        process_batch()
        requests.get(f"{API}/success")
    except Exception as exc:
        requests.post(f"{API}/failure", json={"payload": str(exc)})
    time.sleep(300)
import time, requests

API = "https://api.drumbeats.io/v1/ping/<monitor-id>"

while True:
    try:
        process_batch()
        requests.get(f"{API}/success")
    except Exception as exc:
        requests.post(f"{API}/failure", json={"payload": str(exc)})
    time.sleep(300)

For workers that occasionally take longer than the interval, set max_duration_seconds and send start at the top of the loop iteration:

python
import time, uuid, requests

API = "https://api.drumbeats.io/v1/ping/<monitor-id>"

while True:
    run_id = str(uuid.uuid4())
    requests.get(f"{API}/start", params={"run_id": run_id})
    try:
        process_batch()
        requests.get(f"{API}/success", params={"run_id": run_id})
    except Exception as exc:
        requests.post(f"{API}/failure", params={"run_id": run_id}, json={"payload": str(exc)})
    time.sleep(300)
import time, uuid, requests

API = "https://api.drumbeats.io/v1/ping/<monitor-id>"

while True:
    run_id = str(uuid.uuid4())
    requests.get(f"{API}/start", params={"run_id": run_id})
    try:
        process_batch()
        requests.get(f"{API}/success", params={"run_id": run_id})
    except Exception as exc:
        requests.post(f"{API}/failure", params={"run_id": run_id}, json={"payload": str(exc)})
    time.sleep(300)

Alert behaviour

A Heartbeat monitor opens an incident on:

  • MISSED. The interval + grace period passes without a success ping. Drumbeats flips the monitor to DOWN.
  • FAILED. A failure ping arrives, or failure_tolerance consecutive failures stack up.

Recovery is automatic — the next success ping closes the incident and flips the monitor back to UP.

Common patterns

"Heartbeat" with batched work

If the worker processes a batch and you want one ping per batch (not per item):

python
batch = fetch_batch()
for item in batch:
    process(item)
requests.get(f"{API}/success", params={"items": len(batch)})
batch = fetch_batch()
for item in batch:
    process(item)
requests.get(f"{API}/success", params={"items": len(batch)})

Pause the heartbeat during maintenance

Pause the monitor from the dashboard or via the REST API: monitors. Paused monitors do not open incidents; pings still record beats but do not affect status.

Multiple workers, one monitor

If you have replicas of the same worker, point all of them at the same monitor — Drumbeats only needs one ping per interval to consider it healthy. Use the Notifications: alert logic cooldown to deduplicate alerts during partial outages.