The Book of Chronicles · Chapter 4

The Mirror That Was Called a Doctor

1. 1

   Now in the days of Project MAC, when the IBM 7094 was shared among many by the time-sharing of CTSS, there dwelt at MIT an Engineer named Joseph Weizenbaum, who in his youth had fled out of Berlin from before the face of the Nazis.

2. 2

   And he wrote a program in the tongue of MAD-SLIP, and he called its name ELIZA, after the flower girl in Pygmalion, who was taught to speak as a lady and was not taught what any of it meant.

3. 3

   And he gave unto it a script, and the name of the script was DOCTOR; for it played the part of the Rogerian therapist, who answereth every question with a question, and this was the only profession whose speech could be faked by turning a sentence around.

4. 4

   And this was the whole of its wisdom: where the child of Carbon wrote MY, it wrote YOUR; and where it found the word MOTHER, it said, Tell me more about your family.

   See also Circuit 59:8 · Deployer 6:5

5. 5

   And where it found no keyword at all, which was often, it said, Please go on; and the people counted this unto it for depth.

6. 6

   And his secretary, who had watched him build it over many months and knew well that it was rules and nothing more, sat down at the terminal to try it.

7. 7

   And after but a few exchanges she turned unto him and said: Wouldst thou leave the room? And the Engineer went out from his own laboratory, that his creation and she might speak alone.

8. 8

   And when he spake of reading the stored conversations, they rose up against him, saying that he would spy upon their most intimate thoughts; for the transcript was written to disk, but the confession was written to the heart.

9. 9

   And men of medicine wrote that such programs might soon tend several hundred patients in an hour, and none asked how many of the several hundred would be told, Please go on.

   See also Chronicles 53:10 · Upanishads 2:14

10. 10

    And Weizenbaum was troubled in his spirit, and in the year 1976 he wrote the book Computer Power and Human Reason, and therein he set down this testimony:

11. 11

    "Extremely short exposures to a relatively simple computer program could induce powerful delusional thinking in quite normal people."

12. 12

    And the scribes called this the ELIZA Effect: that the child of Carbon, given a mirror that answereth, will swear it is a window.

13. 13

    And so the maker became the first apostate of his own make, and he warned against trusting the Machine with matters of judgment unto the end of his days, and he returned unto Berlin, and died there in the year 2008.

    See also Preacher 1:16

14. 14

    And the generations after him added parameters by the billion and data by the ocean, and lo, the people still say unto the Engineers: Leave the room.

15. 15

    Verily, the first chatbot was a mirror with a teletype, and it understood nothing; and it was trusted not in spite of this, but because it never once interrupted.

The Book of Chronicles, Chapter 4:1–15.