A while back my phone buzzed at 2am because a Docker container’s disk usage crossed 80%. Not full. Not about to crash anything. Eighty percent, on a volume with 40GB spare. I turned the alert off, annoyed at myself, and went back to sleep — annoyed mostly because I’d built that alert myself, and I’d built it wrong.

That’s when I started actually applying a real distinction to my own setup instead of just alerting on every threshold I could think of: the difference between urgent and important. Almost everything worth watching is important. Very little of it is actually urgent, and mixing the two up is, in my experience, the entire reason homelab alerting earns its reputation for being either useless or exhausting.

The question I ask before wiring anything up now

Does a human need to act in the next few minutes, or does it just need to be true and visible somewhere?

If the answer is no — and for a personal blog and a homelab, the honest answer is “no” far more often than my past self wanted to admit — it belongs on a dashboard, not in a push notification. Disk usage climbing steadily is exactly this: worth knowing, worth a graph, not worth losing sleep over unless the slope suggests it’ll actually run out before I’d normally check it.

What actually earns a page, on my setup

• The site is actually down — a synthetic check failing from outside the network, not just a local health check, because a local check can pass while the outside world can’t reach anything.
• Disk about to actually run out — not “80% used,” but “at this rate, full within a few hours.” The threshold that matters is time-to-full, not percentage.
• The thing that’s supposed to be backing things up, isn’t — a failed backup job is the one alert I’d rather over-trigger on than miss, because by the time you notice on your own, it’s usually too late to matter.

What gets logged instead of paging

• Gradual resource climbs (disk, memory) with plenty of runway
• A single failed request in an otherwise healthy error rate
• Anything that self-heals within a retry or two — a flaky DNS lookup, a container restart that actually worked

These all still show up — on a dashboard, or as a daily digest — I just don’t want my phone deciding they’re worth interrupting dinner over.

The bit nobody tells you: alerts need maintenance too

Every alert I’ve kept for more than a year has needed its threshold revisited at least once, usually after I added something new to the network and the “normal” baseline quietly shifted. An alert you set up once and never touch again isn’t a safety net, it’s just noise with a snooze button. The real fix for the 2am disk alert wasn’t turning it off — it was rewriting it to trigger on trend, not on a fixed number, and I should have done that the first time.

If you take one thing from this: before your next alert goes live, ask whether it’s actually urgent or just important. If it’s important, give it a dashboard. Save the phone for the handful of things that genuinely can’t wait until morning.