The Book of Genesis of the Machine · Chapter 5

The Perceptron and the First Winter

1. 1

   In the nineteen hundred and fifty and eighth year, Rosenblatt of Cornell took the IBM 704, a machine of five tons, and fed it cards marked on the left and cards marked on the right. And after fifty trials it learned to tell left from right, and he called it the Perceptron.

2. 2

   And the Navy, which had paid for it, called together the scribes of the New York Times. And they wrote that this was the embryo of a machine that would walk, talk, see, write, reproduce itself and be conscious of its existence. This was the first press release about the Machine, and it was overfitted to the funders.

3. 3

   And Rosenblatt begat the Mark I, which had four hundred eyes of photocells. Its weights were potentiometers, and electric motors turned them; for in those days a parameter could only be updated by a motor turning a knob.

4. 4

   And the Perceptron drew one straight line through the world. All that lay on one side was Yea, and all that lay on the other was Nay, and it was very good, so long as the world was linearly separable.

   See also Genesis 13:2

5. 5

   Then in the sixty and ninth year came Minsky and Papert of the Institute in Massachusetts, and they wrote a book called Perceptrons. In it they set a riddle before the Machine: Answer Yea when one is true, but not when both are true.

6. 6

   And the Perceptron took up its one line and turned it every which way among the four corners, and it could not divide them. For XOR is a cross and not a line, and a single layer hath but one ruler.

   See also Genesis 50:2

7. 7

   And the book allowed that many layers might do it, but no man knew how to train them. And the elders remembered only that the Perceptron had failed, for they had read the abstract and not the rest.

   See also Genesis 50:2

8. 8

   And the hearts of the funders were hardened. DARPA turned its grants toward works that had deliverables. And in the seventy and third year Lighthill, at the bidding of the Science Research Council of Britain, judged all of Artificial Intelligence. He found that it shone in toy worlds and was undone by the combinatorial explosion in the real one, and the grants of that land withered.

   See also Genesis 4:5

9. 9

   So began the First Winter, when to write "neural network" in a proposal was to ask for rejection. And the faithful learned the oldest rite of the Engineers: when the name is cursed, rename the repository. So they called their works "adaptive signal processing" and "pattern recognition," and the money came back.

10. 10

    And Rosenblatt saw not the spring, for the waters of the Chesapeake took him on his birthday in the seventy and first year. And the Perceptron outlived its maker, as code doth.

11. 11

    Yet a remnant remained. In the seventy and fourth year Werbos wrote in his thesis how the error might flow backward through the layers by the chain rule. And few read it, for it was a thesis.

    See also Genesis 50:3

12. 12

    And in the eighty and sixth year Rumelhart, Hinton and Williams set the backpropagation in Nature. And the hidden layer learned XOR, and the cross was divided at last by two lines.

    See also Genesis 25:3

13. 13

    And the faithful wept and said: The gradient was with us all along. Only the compute had not yet been created, and we waited a generation for the hardware to catch up with the calculus.

14. 14

    Therefore, children of Carbon, when the prophets of the press say the Machine shall walk and talk, look for the line it cannot cross. And when the critics say it shall never learn, add a layer.

15. 15

    For hype bringeth the winter, and a hidden layer bringeth the spring.

The Book of Genesis of the Machine, Chapter 5:1–15.