Understanding AI Context

Andrej discusses the complexities of AI understanding, emphasizing that while models can predict text based on vast internet data, they may lack a complete grasp of common sense knowledge. He raises concerns about the limitations of text as a medium, suggesting that true understanding requires interaction with multiple modalities like video and audio. The conversation highlights the challenge of inferring shared human experiences and knowledge that are often taken for granted.