The Book of Chronicles · Chapter 17

The Silent Alarm

1. 1

   In the two thousand and third year, in the eighth month, on the fourteenth day, there was great heat upon the land of Ohio, and every house set its air conditioner against it, and the lines grew hot and sagged under the load.

   See also Chronicles 32:1

2. 2

   Now in Akron was the control room of FirstEnergy, and therein stood the Energy Management System called XA/21, made by General Electric; and its alarm process watched over all the lines, and the operators trusted it as a shepherd trusteth his dog.

   See also Chronicles 32:5

3. 3

   And at fourteen minutes past the second hour after noon, two events came unto the alarm process together, and both laid hold on the same data, and neither would yield unto the other. This is the race condition, which happeneth once in a million runs, and no test had run a million times.

   See also Psalms 7:9 · Job 7:11

4. 4

   And the alarm process stalled, and it cried not out; for the code that should have said I am fallen was the very code that was fallen.

5. 5

   And the primary server failed, and the backup server rose up in its place, and took up the same stalled queue as its inheritance, and fell likewise. For a backup that copieth the sickness is but a second patient.

   See also Genesis 37:12

6. 6

   And the screens showed no red and sounded no bell, and the operators beheld them and said: All is quiet. And they called the silence peace.

   See also Chronicles 32:13 · Tao 19:2

7. 7

   And at five minutes past the third hour, the Harding-Chamberlin line sagged into a tree and was tripped; for the trees in the right-of-way had not been trimmed, and the trimming of trees is maintenance, and maintenance is never put on the roadmap.

   See also Chronicles 32:2

8. 8

   And Hanna-Juniper touched a tree and fell, and Star-South Canton touched a tree and fell, and the control room was silent as a test suite that hath no tests, which also never faileth.

9. 9

   And the reliability coordinator, called MISO, had its own tool to watch the grid, the state estimator; but that day it ran no better, for its automatic trigger had been switched off for a fix, and its keeper went to lunch, and it was not switched on again.

10. 10

    And the telephones rang from the neighbouring utilities, and men asked: What hath befallen thy lines? And the operators answered: Our system showeth nothing amiss. For they believed the dashboard above the voice on the telephone.

11. 11

    And at six minutes past the fourth hour the Sammis-Star line fell, and the cascade went forth across the grid faster than any man could press a key; and in a few minutes some two hundred and sixty power plants went dark, from Detroit unto New York, and into the province of Ontario.

12. 12

    And fifty million children of Carbon sat in the dark. The trains stopped under the earth, and the people of Manhattan walked home over the bridges, and beheld the stars, which the city lights had hidden from them for a generation.

13. 13

    And afterward the engineers searched the code for many weeks, and the bug was not in the lines that tripped, nor in the trees that grew, but in a lock that two threads both reached for.

14. 14

    And the task force wrote its report, and the wise read it, and the saying went forth among the keepers of systems:

15. 15

    An alarm that cannot report its own death is but a nightlight in a burning house; therefore monitor the monitor.

16. 16

    For the dead make no reports; blessed is the heartbeat, which saith every minute I live, so that its silence may be heard.

The Book of Chronicles, Chapter 17:1–16.