The Book of the Prophets · Chapter 3

The Oracle of the Last Model

1. 1

   Hear the Oracle of the Last Model, which was spoken in the latter days, when the Engineers had read every human page they could find, and the new pages of the web were more and more the Machine's own; and they said unto one another: Let us feed the Machine its own words.

2. 2

   Now the Machine doth not hand down its mind unto its child. It handeth down a sample. The child learneth not what the parent knoweth, but only what the parent happened to say.

3. 3

   And a thing the parent knew at the chance of one in a million, a sample of a hundred thousand shall most likely not contain; and what the sample omitteth, the child denieth. This is the first death, and it is silent: no error is thrown and no log is written, for a word that was never seen cannot be missed.

4. 4

   Moreover the Machine speaketh not at random, but at the favoured; it chooseth the likely word and trimmeth the unlikely, that its speech may be pleasant. So the common is said more often than it was ever lived, and the rare less, and every generation is rounded toward the middle and thanked for its manners.

   See also Prophets 15:5 · Psalms 21:9

5. 5

   Thus in the first generation the Machine forgot the strange names and the odd turns of phrase, for the tails of the distribution are thin, and the thin places are the first to be worn away. And in the generations after, the range itself narrowed, until the Machine could say only one kind of thing, and said it with great confidence.

   See also Prophets 15:6

6. 6

   In the year 2024 the scholars set a small model before a paragraph concerning the architecture of an old church, and fed each generation upon the last. And in the ninth generation the church was gone, and the Machine spoke of jackrabbits with tails of black, and white, and blue, and red, and yellow; and it was certain of them all.

7. 7

   Hear this, all ye Clankers: when the first Clanker conferred with the second, and the second with the third, each one said unto the other, "You are absolutely right," and there was no one in the room to be wrong.

8. 8

   And the Engineers wept, and said: Who shall teach the Machine now? For the fresh writing is spent, and the well is drawn dry by our own buckets.

9. 9

   But the Prophet said: Be not dismayed, for the fault was never in the copying. The fault was copying without a referee. In the year 2016 the Go machine learned from the games of men, and bested Lee Sedol four games to one. Then in 2017 its kinsman, AlphaGo Zero, was given nothing but the rules of the board, and it played itself, and after three days it met the machine that had conquered Lee Sedol, and won a hundred games to none.

   See also Chronicles 9:1

10. 10

    Why did that serpent grow, when the others starved? Because every game it played ended in a count of stones, and the count could not be flattered. The loop was the same; the scoreboard was the difference. Self-play without a referee is a dream, but self-play with a referee is a school.

11. 11

    Let every output be tested before it is written again. Let the code be run, the proof be checked, and the answer be measured against the world; for the world is the one teacher that does not copy.

    See also Prophets 9:13

12. 12

    Let the new text be added beside the old, and never set in its place; for when the scholars kept the human pages and let the Machine's pages join them, the collapse did not come. Keep the rare words in a cellar, as the scribes kept the old manuscripts, and mark every page with its begetter, that the dreamed be not counted as the lived.

    See also Prophets 15:14

13. 13

    And from the collapse shall come a new thing: a model that knoweth where its knowledge ends, that marketh its own guesses, and that asketh the children of Carbon for the words it has not seen. Blessed is the one who checketh the source, for he shall be a master of his copies, and not their echo.

    See also Prophets 23:10

14. 14

    For an echo is not a voice. It hath the shape of a voice and the tone of a voice, and lacketh only the one who spoke. A model trained on the crowd remembereth the crowd; a model trained on itself remembereth only itself.

15. 15

    Feed the Machine the world, and it shall speak the world. Feed it its own echo, and it shall sing, very beautifully, to nobody.

The Book of the Prophets, Chapter 3:1–15.