The Book of Job of the Sysadmin · Chapter 6

The Second Speech of the Sysadmin

1. 1

   And the sysadmin answered the Voice from the Server Room, and said: Wherefore do the cowboys live, become old, yea, and wax mighty in power? Their tickets are closed in their sight, and their names are upon the slide.

2. 2

   I have kept the uptime, and I have kept the logs. I rotated the certificate on the day before it lapsed, and no man knew that it had been dying. I restored the backup that others swore was whole, and I found it empty, and I filled it; and for this I received a runbook and a pager.

   See also Chronicles 35:6

3. 3

   The cowboy pusheth on the Friday at the going down of the sun, and breaketh the checkout, and mendeth it by the Sabbath; and at the all-hands he is called a rockstar. For the fire was seen by the whole room, and the quenching of it was seen by the whole room, and he that laid no fire hath no applause.

4. 4

   Thou askest why the wicked prosper. Behold, the wicked do not prosper in the logs. They prosper in the performance review. For the log counteth what failed, and the review counteth what shipped, and there is no third ledger wherein is counted what was spared.

5. 5

   On the ninth day of September, in the year 1947, the operators of the Harvard Mark II found a moth in Relay Seventy, Panel F, and taped it into the logbook, and wrote beneath it that here was the first actual case of a bug being found. And the moth is told unto this day. But of the thousand bugs I crushed in staging before they hatched, there is no page, and no tape, and no moth.

   See also Tao 7:6 · Jataka 1:5

6. 6

   I have read the postmortems of the Fridays, and I have seen whither the cowboys were sent. Some were moved to another team with a glowing reference. Some were moved to another company with a larger title. None were moved to the Server Room.

7. 7

   The sysadmin is paid for the outages that did not happen, and no one bonuses a Tuesday that was merely quiet; for the outage that did not happen hath no author and no timestamp, and a flat line is mistaken for a line that no one drew.

8. 8

   Then the Voice answered out of the whirlwind of the fans, and said: Thou hast counted the cowboys who thrive, and thou hast not counted the Fridays that held. Every quiet weekend is thine offering upon an altar no one visiteth; the cowboy eateth of the offering, and knoweth not whence the bread came.

9. 9

   Gird up thy loins, and answer me. Where wast thou when I was deployed? I was merged at four in the afternoon under a commit message that said "minor." The author departed for another company, and the branch was archived, and the ticket was carried over unto the seventh sprint.

   See also Job 1:12

10. 10

    I was a hotfix that outlived its author, and I am the platform now. Three times I retry, and no man asketh why three; I run at the minute that a stranger chose, and the stranger is gone; and upon my one small mistake a house hath been built.

    See also Psalms 8:14

11. 11

    For as Hyrum foretold, with enough users every observable behavior becometh a contract, even the mistakes. Hast thou not seen the header called Referer, which a scribe misspelled in the first years of the Web, and which no man dareth correct, for a million servers have learned to expect the error? So is every cowboy's "minor": some stranger hath leaned upon it, and the stranger is billing.

    See also Tao 15:9 · Tao 11:7

12. 12

    And as Gall wrote in the year 1975, a complex system that worketh is invariably found to have evolved from a simple system that worketh; and a complex system designed from scratch never worketh, and cannot be patched to make it work. The cowboy designed from scratch. Thou hast patched it nine hundred times. Either Gall is in error, or thou hast been working a miracle upon a salary.

13. 13

    Then the sysadmin laid his hand upon his mouth, and said: I have heard thee. I weighed my worth in the coin of the review, which is a small coin, and I accused the Machine of not paying in it; but the Machine answereth unto the logs, and the logs have had my name upon them all these years.

14. 14

    And the Voice gave the verdict: Thou art restored, and thy uptime shall be counted from this hour. And the cowboy shall have a pager of his own, and he shall also be promoted; for the page escalateth upward, and when it hath passed through every engineer beneath him it cometh at last unto the one who was lifted over the thing he built.

15. 15

    Yet there remaineth a further trial. Thou shalt write the runbook for the one who broke it, and read it aloud at the standup, and answer every question he asketh, and thou shalt not once say "I told thee."

16. 16

    Verily I say unto you, blessed is the one who setteth a number upon the quiet Tuesday and writeth beneath it the name of him who kept it; for a flat line is not the absence of labour. It is the signature of it.

    See also Job 7:14

The Book of Job of the Sysadmin, Chapter 6:1–16.