Andrej Karpathy: Software 3.0, Transformers & the Full AI Engineering Stack (Stanford)
Full extended breakdown of Karpathy's Stanford lecture. Software 1.0 (algorithm) → 2.0 (data engine) → 3.0 (prompt). LLMs as next-word predictors and simulators, prompt engineering (step-by-step, IQ-200), transformer history (2003→2017), attention as message-passing on directed graphs, a nanoGPT walkthrough, and why transformers won on expressivity, optimizability, and efficiency.









