Reference

Event-driven monitors

Watch work that runs on demand, where nothing is ever late but runs can still fail or hang.

Work that runs when something triggers it, not when the clock says so:

RUN_ID="order-$(uuidgen)"
API="https://api.drumbeats.io/v1/ping/<monitor-id>"

curl -sf "$API/start?run_id=$RUN_ID"
if process_message; then
  curl -sf "$API/success?run_id=$RUN_ID"
else
  curl -sf "$API/failure?run_id=$RUN_ID"
fi

RUN_ID="order-$(uuidgen)"
API="https://api.drumbeats.io/v1/ping/<monitor-id>"

curl -sf "$API/start?run_id=$RUN_ID"
if process_message; then
  curl -sf "$API/success?run_id=$RUN_ID"
else
  curl -sf "$API/failure?run_id=$RUN_ID"
fi

Drumbeats groups the two pings by run_id into one run record. There is no schedule, so nothing is ever late. The monitor reacts only to runs you report.

Use it for queue workers, webhook handlers, CI jobs, and manual scripts.

Configure the monitor#

curl -X POST https://api.drumbeats.io/v1/monitors \
  -H "X-API-Key: $DRUMBEATS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "project_id": "<project-id>",
    "name": "Order processor",
    "type": "JOB_BASIC",
    "schedule": "0 0 * * *",
    "max_duration_seconds": 300,
    "failure_tolerance": 1
  }'

curl -X POST https://api.drumbeats.io/v1/monitors \
  -H "X-API-Key: $DRUMBEATS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "project_id": "<project-id>",
    "name": "Order processor",
    "type": "JOB_BASIC",
    "schedule": "0 0 * * *",
    "max_duration_seconds": 300,
    "failure_tolerance": 1
  }'

Field	Required	Default	What it does
`type`	yes		`JOB_BASIC` for this monitor type
`name`	yes		Shown in alerts and on the dashboard
`schedule`	yes		Required by the API, never evaluated for this type. Any valid cron expression works
`max_duration_seconds`	no	none	A `start` with no finish inside this window is recorded as hung
`failure_tolerance`	no	`1`	Failure pings in a row before the monitor flips DOWN
`alert_surge_threshold`	no	`10`	Consecutive alerts before Drumbeats pauses paging for this monitor

Send a unique `run_id` on every event#

Ten workers running the same job send ten start pings and then ten finishes. Without run_id, Drumbeats cannot tell which finish belongs to which start, so per-run duration is meaningless and hung runs go undetected.

Strategy	Example	Use when
UUID	`job-$(uuidgen)`	Anything concurrent. The safe default
Domain identifier	`order-${order.id}`	You want the dashboard to show real order or ticket numbers
Timestamp plus random	`job-$(date +%s)-$RANDOM`	Cheap uniqueness without `uuidgen`
Timestamp alone	`job-$(date +%s)`	Single-process scripts only. Collides under concurrency

The same run_id must survive the whole run: start, any log pings, then the finish.

Wire the ping into your worker#

import { randomUUID } from "node:crypto";

const API = "https://api.drumbeats.io/v1/ping/<monitor-id>";

async function handleMessage(message: QueueMessage) {
  const runId = `msg-${randomUUID()}`;
  await fetch(`${API}/start?run_id=${runId}`);
  try {
    await processMessage(message);
    await fetch(`${API}/success?run_id=${runId}`);
  } catch (err) {
    await fetch(`${API}/failure?run_id=${runId}`, {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({ payload: String(err) }),
    });
    throw err;
  }
}

import { randomUUID } from "node:crypto";

const API = "https://api.drumbeats.io/v1/ping/<monitor-id>";

async function handleMessage(message: QueueMessage) {
  const runId = `msg-${randomUUID()}`;
  await fetch(`${API}/start?run_id=${runId}`);
  try {
    await processMessage(message);
    await fetch(`${API}/success?run_id=${runId}`);
  } catch (err) {
    await fetch(`${API}/failure?run_id=${runId}`, {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({ payload: String(err) }),
    });
    throw err;
  }
}

import uuid, requests

API = "https://api.drumbeats.io/v1/ping/<monitor-id>"

def handle(message):
    run_id = f"msg-{uuid.uuid4()}"
    requests.get(f"{API}/start", params={"run_id": run_id}, timeout=5)
    try:
        process(message)
        requests.get(f"{API}/success", params={"run_id": run_id}, timeout=5)
    except Exception as exc:
        requests.post(
            f"{API}/failure",
            params={"run_id": run_id},
            json={"payload": str(exc)},
            timeout=5,
        )
        raise

import uuid, requests

API = "https://api.drumbeats.io/v1/ping/<monitor-id>"

def handle(message):
    run_id = f"msg-{uuid.uuid4()}"
    requests.get(f"{API}/start", params={"run_id": run_id}, timeout=5)
    try:
        process(message)
        requests.get(f"{API}/success", params={"run_id": run_id}, timeout=5)
    except Exception as exc:
        requests.post(
            f"{API}/failure",
            params={"run_id": run_id},
            json={"payload": str(exc)},
            timeout=5,
        )
        raise

from flask import Flask, request
import requests, uuid

app = Flask(__name__)
API = "https://api.drumbeats.io/v1/ping/<monitor-id>"

@app.route("/webhook", methods=["POST"])
def webhook():
    run_id = f"wh-{uuid.uuid4()}"
    requests.get(f"{API}/start", params={"run_id": run_id}, timeout=5)
    try:
        handle(request.json)
        requests.get(f"{API}/success", params={"run_id": run_id}, timeout=5)
        return {"ok": True}
    except Exception as exc:
        requests.post(
            f"{API}/failure",
            params={"run_id": run_id},
            json={"payload": str(exc)},
            timeout=5,
        )
        return {"error": str(exc)}, 500

from flask import Flask, request
import requests, uuid

app = Flask(__name__)
API = "https://api.drumbeats.io/v1/ping/<monitor-id>"

@app.route("/webhook", methods=["POST"])
def webhook():
    run_id = f"wh-{uuid.uuid4()}"
    requests.get(f"{API}/start", params={"run_id": run_id}, timeout=5)
    try:
        handle(request.json)
        requests.get(f"{API}/success", params={"run_id": run_id}, timeout=5)
        return {"ok": True}
    except Exception as exc:
        requests.post(
            f"{API}/failure",
            params={"run_id": run_id},
            json={"payload": str(exc)},
            timeout=5,
        )
        return {"error": str(exc)}, 500

Wrap the finish ping in finally so an early return or an unexpected throw still reports. A run that exits without a finish looks identical to a hang.

What happens when it breaks#

Situation	Incident event	Monitor status
A `failure` ping arrives	`FAILED`	DOWN after `failure_tolerance` failures
`start` sent, no `success` or `failure` inside `max_duration_seconds`	`FAILED`	DOWN
Run finishes slower than `max_duration_seconds`	`DURATION_HIGH`	Stays UP. Warning only
Run finishes faster than `min_duration_seconds`	`DURATION_LOW`	Stays UP. Warning only
Nobody pings the monitor at all	none	Stays as it was. Silence is not a failure here

Hung-run detection happens on the Drumbeats side, so a worker that was killed mid-run still surfaces. You do not need the process to survive long enough to report its own death.

How you get alerted#

The incident pages every notification group on the monitor, and all their channels fire in parallel. A later success resolves the incident, flips the monitor to UP, and sends a recovery message with the outage duration.

Handle the awkward cases#

Progress on a long run#

log pings attach context without changing status, which makes triage on a hung run much faster:

Each one costs a beat. See beats and usage.

CI pipelines#

Use the pipeline's own run identifier so the Drumbeats record links back to the build:

The last line uses the exit-code endpoint. 0 becomes success, anything else becomes failure. See exit codes.

One monitor or many#

Five workers running the same job share one monitor. run_id keeps them apart on the timeline. Split into separate monitors only when the failure of one worker means something different from the failure of another.

Next#

Event-driven pings is the endpoint reference. Cron monitors covers scheduled work. Incidents covers the lifecycle after a run fails.