François Chollet: Keras, Deep Learning, and the Progress of AI | Lex Fridman Podcast #38

Specialized Intelligence

François Chollet discusses the specialized nature of intelligence, emphasizing that all intelligent systems, including human intelligence, are specialized in certain categories of problems. He explains that human intelligence is tailored to the human experience, making it difficult for individuals to solve long-term problems on their own. Instead, civilization acts as a collective problem-solving system, leveraging infrastructure like books, computers, and institutions to tackle larger-scale issues 1.

Human intelligence is specialized in the human experience and human experience is very short, like one lifetime is short. Even within one lifetime we have a very hard time envisioning things on a scale of years.

--- François Chollet

Chollet also challenges the notion of exponentially increasing intelligence, arguing that intelligence emerges from the interaction between a brain, body, and environment. He believes that simply enhancing a brain in isolation does not equate to increased intelligence 2.

   

Complex Problems

François Chollet explores the challenges of solving complex problems with AI, particularly in the context of deep learning. He notes that while deep learning is effective for perception problems, it struggles with tasks requiring explicit, rule-based reasoning. For example, understanding the physics of a scene is more efficiently handled by symbolic models rather than deep learning 3.

The Turing Test is all about tricking people into believing they're talking to a human. And I don't think that's actually very difficult because it's more about exploiting human perception and not so much about intelligence.

--- François Chollet

Chollet also highlights that intelligence is not just about the brain's performance but the interaction with the environment. He argues that even very smart people today are not limited by their brain's capabilities but by the problems they encounter and the systems they operate within 4.

   

Intelligence Explosion

Chollet questions the concept of an intelligence explosion, where AI would exponentially improve itself. He argues that no system exists in isolation, and enhancing one part of a system often creates bottlenecks elsewhere. He uses the scientific process as an example, noting that while resource consumption in science has increased exponentially, the output in terms of significant discoveries has remained linear 5.

Scientific progress is not actually exponential. If you look at science, what you see is the picture of a system that is consuming an exponentially increasing amount of resources, but it's having a linear output in terms of scientific progress.

--- François Chollet

Chollet also addresses the appeal of the AI apocalypse narrative, likening it to religious mythology. He suggests that the idea of a singularity, where AI surpasses human intelligence and transforms the world, is more of a belief system than a scientific argument 6.