Shell integration

Wire Drumbeats into shell scripts and system cron, avoiding the traps that silently report success.

bash
0 2 * * * /usr/bin/backup.sh \
  && curl -sf --max-time 3 https://api.drumbeats.io/v1/ping/<monitor-id>/success \
  || curl -sf --max-time 3 https://api.drumbeats.io/v1/ping/<monitor-id>/failure
0 2 * * * /usr/bin/backup.sh \
  && curl -sf --max-time 3 https://api.drumbeats.io/v1/ping/<monitor-id>/success \
  || curl -sf --max-time 3 https://api.drumbeats.io/v1/ping/<monitor-id>/failure

&& sends success on a zero exit, || sends failure on anything else. Bash is assumed, but this works in zsh, dash, and busybox. The only hard requirement is curl.

Always use these curl flags#

FlagWhy
-sSilences the progress meter. Cron mails any output to root, so without this every run generates mail
-fMakes curl exit non-zero on an HTTP error, so a 500 from the ping API does not read as success in your chain
--max-time 3Caps the request. Without it, a hung connection stalls the cron slot

A helper you can source#

Past a couple of jobs, put the IDs in one file and source it:

/usr/local/lib/drumbeats.sh
#!/usr/bin/env bash
# Usage: drumbeats_run <monitor-key> <command...>

DRUMBEATS_BASE="${DRUMBEATS_BASE_URL:-https://api.drumbeats.io/v1}"

declare -A DRUMBEATS_MONITORS=(
  [daily_backup]="11111111-2222-3333-4444-555555555555"
  [hourly_sync]="66666666-7777-8888-9999-aaaaaaaaaaaa"
  [newsletter_send]="bbbbbbbb-cccc-dddd-eeee-ffffffffffff"
)

# Best effort. Never fails the caller.
_drumbeats_ping() {
  local monitor=$1 event=$2 run_id=$3 payload=$4
  local url="${DRUMBEATS_BASE}/ping/${monitor}/${event}?run_id=${run_id}"

  if [ -z "$payload" ]; then
    curl -sf --max-time 3 "$url" >/dev/null 2>&1 || true
  else
    curl -sf --max-time 3 -X POST "$url" \
      -H 'Content-Type: application/json' \
      -d "$(jq -n --arg p "$payload" '{payload: $p}')" >/dev/null 2>&1 || true
  fi
}

drumbeats_run() {
  local key=$1
  shift

  local monitor=${DRUMBEATS_MONITORS[$key]}
  if [ -z "$monitor" ]; then
    echo "drumbeats: unknown monitor key '$key'" >&2
    return 2
  fi

  local run_id="${key}-$(date +%s)-$RANDOM"
  _drumbeats_ping "$monitor" "start" "$run_id" ""

  local output status
  output=$("$@" 2>&1)
  status=$?

  if [ $status -eq 0 ]; then
    _drumbeats_ping "$monitor" "success" "$run_id" ""
  else
    # Keep the tail. The error is at the end, and Drumbeats truncates
    # from the front if you exceed the plan limit.
    _drumbeats_ping "$monitor" "failure" "$run_id" "$(echo "$output" | tail -c 20000)"
  fi

  return $status
}
#!/usr/bin/env bash
# Usage: drumbeats_run <monitor-key> <command...>

DRUMBEATS_BASE="${DRUMBEATS_BASE_URL:-https://api.drumbeats.io/v1}"

declare -A DRUMBEATS_MONITORS=(
  [daily_backup]="11111111-2222-3333-4444-555555555555"
  [hourly_sync]="66666666-7777-8888-9999-aaaaaaaaaaaa"
  [newsletter_send]="bbbbbbbb-cccc-dddd-eeee-ffffffffffff"
)

# Best effort. Never fails the caller.
_drumbeats_ping() {
  local monitor=$1 event=$2 run_id=$3 payload=$4
  local url="${DRUMBEATS_BASE}/ping/${monitor}/${event}?run_id=${run_id}"

  if [ -z "$payload" ]; then
    curl -sf --max-time 3 "$url" >/dev/null 2>&1 || true
  else
    curl -sf --max-time 3 -X POST "$url" \
      -H 'Content-Type: application/json' \
      -d "$(jq -n --arg p "$payload" '{payload: $p}')" >/dev/null 2>&1 || true
  fi
}

drumbeats_run() {
  local key=$1
  shift

  local monitor=${DRUMBEATS_MONITORS[$key]}
  if [ -z "$monitor" ]; then
    echo "drumbeats: unknown monitor key '$key'" >&2
    return 2
  fi

  local run_id="${key}-$(date +%s)-$RANDOM"
  _drumbeats_ping "$monitor" "start" "$run_id" ""

  local output status
  output=$("$@" 2>&1)
  status=$?

  if [ $status -eq 0 ]; then
    _drumbeats_ping "$monitor" "success" "$run_id" ""
  else
    # Keep the tail. The error is at the end, and Drumbeats truncates
    # from the front if you exceed the plan limit.
    _drumbeats_ping "$monitor" "failure" "$run_id" "$(echo "$output" | tail -c 20000)"
  fi

  return $status
}
bash
0 2 * * * . /usr/local/lib/drumbeats.sh && drumbeats_run daily_backup /usr/bin/backup.sh
0 2 * * * . /usr/local/lib/drumbeats.sh && drumbeats_run daily_backup /usr/bin/backup.sh

The wrapper captures stdout and stderr together and attaches the last 20 KB to the failure ping. That text renders on the incident timeline, which usually means you never have to open /var/log/cron.

Build the JSON with jq. A stack trace containing a quote or a newline breaks hand-rolled interpolation, and it breaks precisely on the runs where you needed the payload.

Watch out for these#

Cover every exit path with a trap#

For scripts with several exits, set a trap that reports failure and clear it just before the success ping:

bash
trap '_drumbeats_ping "$MONITOR" failure "$RUN_ID" ""' EXIT

do_the_work

trap - EXIT
_drumbeats_ping "$MONITOR" success "$RUN_ID" ""
trap '_drumbeats_ping "$MONITOR" failure "$RUN_ID" ""' EXIT

do_the_work

trap - EXIT
_drumbeats_ping "$MONITOR" success "$RUN_ID" ""

The trap fires on any exit you did not plan for, including an exit 1 buried in a function.

What happens when it breaks#

SituationWhat Drumbeats seesWhat to do
The command exits non-zerofailure with the captured outputNothing. The helper handles it
set -e aborts before the pingNothing until the window closesCapture the status instead of relying on the exit
SIGTERM from a systemd timeoutA start with no finishSet max_duration_seconds on the monitor
SIGKILL, or the OOM killerSame. No trap runs on SIGKILLSame. Server-side detection is the only thing that catches this
The host reboots mid-runSameSame

How you get alerted#

A failure opens a FAILED incident once failure_tolerance is reached and pages every notification group on the monitor. The captured stderr appears as a preview in the alert and in full on the timeline.

Next#

Exit codes for the shortest possible wiring. Scheduled pings for more cron patterns. Monitor types if you have not picked one yet. Production hardening for the language-agnostic rules. Alternatives if you are still choosing a vendor.