Lex and Ilya discuss whether neural networks can be made to reason. They agree that AlphaGo is an example of a neural network that can reason, but they disagree on whether the kind of reasoning seen in AlphaGo is the same as more general reasoning. Ilya suggests that neural networks can be trained to reason, but only if the task requires it. He then introduces the idea of neural networks as a search for small circuits, and Lex elaborates on this idea.
