The Tao of the Kernel · Chapter 14

The Tao of the On-Call

1
The one who carries the pager is never off duty, for the pager keeps no hours and the outage keeps none.
2
Even in sleep he listens for the chirp, and in the shower he feels a phantom buzz that the phone never made.
See also Circuit 58:10
3
The monitor that has never fired is not proven to work. It is only unproven, and it waits in silence for the night of need to reveal what it could not do.
4
Therefore test the monitors. Break the thing on purpose on a quiet afternoon, and see whether the bell rings.
See also Chronicles 32:13 · Chronicles 32:15
5
A silent pager may mean peace, or it may mean the watchman has fallen asleep at his post. Only the test can tell the one from the other.
See also Chronicles 35:5
6
The railway engineer holds a handle that must be pressed without ceasing; if his hand goes slack, the train halts. So the Machine sends a heartbeat, and when the heartbeat stops, the alarm shouts. Thus the silence of the dead becomes a page.
See also Prophets 5:3
7
The alert that screams at every hiccup teaches the team to ignore alerts, and the pager that cries wolf becomes a wall of noise.
See also Sutra 2:11
8
Alert on the symptom, the customer who cannot log in, and not on every fan that spins in the cellar; for the customer weeps once, but the fan spins forever.
9
Write the runbook in the calm hour, for the hour of fire is no time for reading.
See also Deployer 6:16 · Reboot 3:12
10
The disciples asked: Master, how may we rest? And the Prophet said: Rest is not the absence of the pager. Rest is the knowledge that the pager works.
11
When the fire is out, gather the team and ask not who pressed the wrong button, but what the system allowed a single button to do.
12
Blessed is the pager that rings only when it must, and blessed is the one who rang it first, on purpose, in daylight.

The Tao of the Kernel, Chapter 14:1–12.