Three Lines of Bash Stand Between Your Cron Job and a Silent Outage
Level: Intermediate · Time: 6 min · Outcome: A hardened, ShellCheck-clean cron wrapper generated for you
Three separate incidents taught me the same lesson over about a year. A */1 rsync stacked six copies on a slow disk and drove a box to load 41. A mysqldump hung on a database lock and didn't run for nine days without a single alert. A deploy raced a database's boot and paged me at 11pm for a migration that wasn't broken. Different commands, different failures — but the same root shape every time: a script that was fine when I ran it by hand fell over the first time conditions weren't ideal and nobody was watching.
The fixes are individually small. A cron job that can't take down a server needs three guards: a lock so it can't overlap itself, a timeout so it can't hang forever, and a bounded retry so a transient blip doesn't kill the run. The problem isn't knowing that. It's that composing the three correctly by hand — getting the flock file descriptor right, the timeout SIGKILL grace right, the retry quoting right, and not mangling a pipeline in the process — is exactly the kind of fiddly that produces a wrapper that looks right and silently isn't.
So I built a generator that does the composition: the Hardened Cron Wrapper Generator.
What it actually does
You paste in the command your cron job runs, toggle the guards you want, set the schedule, and it emits two things: a complete wrapper script and the crontab line that calls it. The generated bash follows a strict standard — set -euo pipefail, CHECK/CROSS defined before use, comments that explain why each line is there — and it's ShellCheck-clean, which is the part that matters when you're pasting something into a root crontab.
The toggles map exactly to the three failure modes:
Lock (flock) stops overlap. You pick the behavior: Skip if busy (-n) for frequent jobs where a missed run is harmless because the next one catches up, or Wait then give up (-w) for jobs that must eventually run but can tolerate a short queue. The tool puts the lock file under /run/lock — a tmpfs cleared on reboot — not /tmp, where temp-cleaners can delete a lock mid-run and let a second copy through. That single path choice is a bug people ship constantly.
Bound the runtime (timeout) stops hangs. You set the max runtime and a SIGKILL grace period. The grace exists because timeout first sends a polite SIGTERM, and a process wedged in uninterruptible I/O — a dead NFS mount, a held database lock — physically can't act on SIGTERM. The grace is how long to wait before sending SIGKILL, which the kernel enforces unconditionally. The tool sets a sane default; you tune it for jobs you know touch slow storage.
Retry transient failures rides out the blip. You set max attempts and a base delay, and the backoff doubles each round up to a 30-second cap, with jitter so parallel callers don't retry in lockstep and re-overwhelm a recovering service.
The part that makes the output trustworthy
Two design decisions in the tool are worth calling out because they're the ones I'd get wrong by hand at 11pm.
First, the crontab line changes shape depending on what you enable. Lock and timeout are simple enough to inline directly on the crontab line — flock and timeout in front of the command, a one-liner. But retry and email-on-failure need a bash function and a mailx call, which can't live on a crontab line. So when you turn either of those on, the output switches from an inline crontab one-liner to a full wrapper script plus a crontab line that calls it. The tool makes that decision for you instead of generating something that won't run.
Second, it's explicit about pipelines. timeout and the retry loop bound the single process they launch — so a raw cmd1 | cmd2 would only be partly guarded. The tool surfaces this: for a pipeline or multiple commands, you wrap them in bash -c '...' yourself, and it tells you so, rather than silently generating a wrapper that bounds half your command.
# A pipeline must be wrapped so timeout/retry bound the whole thing, not just curl
bash -c 'curl -fsS https://api.example.com | jq .status'
It also handles the thing underneath all three guards that people forget: logging. Cron throws stdout and stderr away by default, so a job can fail every night for a month and the only signal is the absence of whatever it was supposed to produce. The tool's timestamped-logging toggle redirects output to a file you control, and the email-alert toggle sends a mailx failure notice after all retries are exhausted — so the job fails loudly instead of disappearing.
Why generate it instead of memorizing it
Because the failure mode of hand-writing these is that the wrapper looks correct and isn't. The flock that used /tmp. The timeout with no -k that left the wedged process alive. The retry loop that retried a 404 forever. Each of those is a one-character or one-decision mistake, and each one only shows up the night it matters. A generator that bakes the right defaults in — and explains them inline in the output so you can read what you're shipping — removes that whole class of error.
Build your wrapper here: https://bashsnippets.xyz/tools/cron-wrapper-generator
The three snippets it composes, if you want the full reasoning behind each guard, are flock, timeout, and retry with backoff. Build the schedule itself in the Cron Job Builder, and read the whole decision in Bash Scripts That Survive Cron. The rest of the library is at https://bashsnippets.xyz
Originally published at bashsnippets.xyz