Check-in

What a check-in can carry

The simplest check-in only says 'done'. Most job monitoring services accept more than that:

• Start and finish. One check-in when the job starts and one when it ends give you the run time, and show runs that start but never finish.
• Success or failure. A separate failure check-in, or the job's exit code, lets the monitor alert straight away instead of waiting for a deadline to pass.
• A run ID. When runs can overlap, an ID ties each finish to the right start.
• Output. A short piece of log output attached to a failure saves a trip to the server.

Not a network ping

This kind of ping has nothing to do with the ping command, which sends ICMP packets to see whether a host is reachable. A check-in is an HTTPS request from your job to the monitor. Because the job makes an outbound request, it works from behind firewalls and NAT, and the monitor never needs access to your servers.

Example

Check-ins around one run of a job

PING="https://api.drumbeats.io/v1/ping/<monitor-id>"
RUN_ID="report-$(date +%s)"

curl -fsS -m 10 "$PING/start?run_id=$RUN_ID"
if ./build-report.sh; then
  curl -fsS -m 10 "$PING/success?run_id=$RUN_ID"
else
  curl -fsS -m 10 "$PING/failure?run_id=$RUN_ID"
fi

Three requests, one run. The monitor learns when the run started, how long it took and whether it worked. -m 10 stops a slow network from holding up the job.

Common mistakes

• Letting the check-in break the job. If the monitor is slow or unreachable, a check-in with no timeout can hang your job. Set a short timeout, and do not let a failed check-in fail the job itself.
• Checking in from the wrong place. A check-in sent at the top of a script, or from a background timer, keeps arriving while the real work is broken. Send it after the work is done.
• Treating the URL as public. The check-in URL is the credential for that monitor. Anyone who has it can send check-ins, so keep it out of public repositories and shared logs.
• Retrying for too long. One or two retries with a short delay are enough. A check-in that arrives an hour late no longer tells the monitor anything useful.

Related terms

• HeartbeatA signal a running process sends at a regular interval to show it is still alive and working.
• Dead man's switchA mechanism that acts when an expected signal stops arriving. In monitoring, silence is the alert.
• Missed runA run that was expected but never reported in, so it left no error behind.
• Silent failureA failure that raises no error and no alert, so it is only found when someone misses the result.

All glossary terms