Cron monitors

Watch a job that runs on a fixed schedule, and get paged when a run is missed, fails, or hangs.

One crontab line, monitored:

bash
0 2 * * * /usr/bin/backup.sh \
  && curl -sf https://api.drumbeats.io/v1/ping/<monitor-id>/success \
  || curl -sf https://api.drumbeats.io/v1/ping/<monitor-id>/failure
0 2 * * * /usr/bin/backup.sh \
  && curl -sf https://api.drumbeats.io/v1/ping/<monitor-id>/success \
  || curl -sf https://api.drumbeats.io/v1/ping/<monitor-id>/failure

Drumbeats works out when the next run is due from your schedule, timezone, and grace period. If success does not arrive by then, it opens an incident.

Use a Cron monitor when a scheduler drives the work on a known timetable: system cron, Kubernetes CronJob, Airflow, GitLab schedules, Render cron jobs.

Configure the monitor#

bash
curl -X POST https://api.drumbeats.io/v1/monitors \
  -H "X-API-Key: $DRUMBEATS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "project_id": "<project-id>",
    "name": "Nightly backup",
    "type": "JOB_CRON",
    "schedule": "0 2 * * *",
    "timezone": "Europe/Amsterdam",
    "grace_period_seconds": 300,
    "max_duration_seconds": 1800
  }'
curl -X POST https://api.drumbeats.io/v1/monitors \
  -H "X-API-Key: $DRUMBEATS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "project_id": "<project-id>",
    "name": "Nightly backup",
    "type": "JOB_CRON",
    "schedule": "0 2 * * *",
    "timezone": "Europe/Amsterdam",
    "grace_period_seconds": 300,
    "max_duration_seconds": 1800
  }'
FieldRequiredDefaultWhat it does
typeyesJOB_CRON for this monitor type
scheduleyesStandard five-field cron expression. Six-field with seconds is rejected
timezonenoUTCAny IANA timezone. The schedule is read in this zone
grace_period_secondsno300How long Drumbeats waits past the scheduled time before calling the run missed
schedule_toleranceno1Missed runs in a row before the monitor flips DOWN
failure_toleranceno1Failure pings in a row before the monitor flips DOWN
max_duration_secondsnononeA run that starts and does not finish inside this window is recorded as hung
min_duration_secondsnononeA run that finishes faster than this opens a DURATION_LOW warning. Must be smaller than max_duration_seconds
alert_surge_thresholdno10Consecutive alerts before Drumbeats pauses paging for this monitor

Free accounts cannot schedule a monitor to fire more often than every 60 seconds. Paid plans cannot go below 30 seconds. A tighter schedule is rejected with a 400.

Wire the ping into crontab#

The one-liner at the top of this page is enough for missed-run and failure detection. Add a start ping when you also want duration tracking, hung-run detection, and the job's output stored on the run:

bash
RUN_ID="backup-$(date +%s)"
API="https://api.drumbeats.io/v1/ping/<monitor-id>"

curl -sf "$API/start?run_id=$RUN_ID"

OUTPUT=$(/usr/bin/backup.sh 2>&1)
STATUS=$?

if [ $STATUS -eq 0 ]; then
  EVENT=success
else
  EVENT=failure
fi

curl -sf -X POST "$API/$EVENT?run_id=$RUN_ID" \
  -H "Content-Type: application/json" \
  -d "$(jq -n --arg p "$OUTPUT" '{payload: $p}')"
RUN_ID="backup-$(date +%s)"
API="https://api.drumbeats.io/v1/ping/<monitor-id>"

curl -sf "$API/start?run_id=$RUN_ID"

OUTPUT=$(/usr/bin/backup.sh 2>&1)
STATUS=$?

if [ $STATUS -eq 0 ]; then
  EVENT=success
else
  EVENT=failure
fi

curl -sf -X POST "$API/$EVENT?run_id=$RUN_ID" \
  -H "Content-Type: application/json" \
  -d "$(jq -n --arg p "$OUTPUT" '{payload: $p}')"

run_id is optional on Cron monitors. Send it anyway. It makes hung-run detection exact and lets you read that run's output from the dashboard during triage.

What happens when it breaks#

SituationIncident eventMonitor statusHow fast
No success by the scheduled time plus grace_period_secondsMISSEDDOWN after schedule_tolerance missesAt the end of the grace period
A failure ping, or a non-zero exit code pingFAILEDDOWN after failure_tolerance failuresImmediately
start sent, no finish inside max_duration_secondsFAILEDDOWNWhen the window closes
Run finishes slower than max_duration_secondsDURATION_HIGHStays UP. Warning onlyOn the finish ping
Run finishes faster than min_duration_secondsDURATION_LOWStays UP. Warning onlyOn the finish ping

A job that finishes far too quickly usually means it exited early, so DURATION_LOW is worth setting on jobs with a stable runtime.

How you get alerted#

The incident pages every notification group assigned to the monitor. Channels fire in parallel. The message names the monitor, the incident event, and the time the run was expected, and links to the incident timeline.

Recovery is automatic. A success ping inside the next expected window resolves the incident, flips the monitor to UP, sends a recovery message with the outage duration, and resets the miss and failure counters.

Handle the awkward cases#

The job sometimes decides not to run#

Do not skip the ping when a feature flag short-circuits the job. Drumbeats reads silence as a missed run. Send success with a payload that says why instead:

bash
if should_skip; then
  curl -sf -X POST "$API/success?run_id=$RUN_ID" \
    -H "Content-Type: application/json" \
    -d '{"payload": "skipped: feature flag off"}'
  exit 0
fi
if should_skip; then
  curl -sf -X POST "$API/success?run_id=$RUN_ID" \
    -H "Content-Type: application/json" \
    -d '{"payload": "skipped: feature flag off"}'
  exit 0
fi

Daylight saving shifts the wall clock#

Set timezone and Drumbeats follows the transition. 0 2 * in Europe/Amsterdam stays 02:00 local time all year. Leave it at UTC and the job will look an hour late twice a year.

Several crontab lines ping one monitor#

Drumbeats computes the next expected run from whichever line ran most recently, which gets confusing fast. Give each schedule its own monitor. The dashboard then shows each one separately and you can tune grace periods independently.

Next#

Heartbeat monitors cover jobs that loop on an interval instead of a clock. Scheduled pings is the full endpoint reference for the pings on this page. Alert logic explains the tolerance fields in the table above.