I read every major CS paper of the last 100 years...
📚 Main topics
- Turing and the limits of algorithmsAlan Turing formalized the idea of a general-purpose computing machine and proved that no program can determine whether every other program will halt. His work also showed that some mathematical problems cannot be solved algorithmically. 1:02
- Information, perceptrons, and AI’s first winterClaude Shannon described information in terms of bits and uncertainty. Frank Rosenblatt’s perceptron sparked early excitement about machine learning, but limitations of single-layer perceptrons led to an AI winter. 2:05
- Distributed systems and backpropagationLeslie Lamport’s happened-before relation orders events by causality when computers have no shared clock. Backpropagation later made it possible to train multilayer neural networks by passing errors backward through their layers. 4:09
- Search, deep learning, and transformersPageRank used links to rank web pages, while AlexNet showed how data, GPUs, and deep learning could dramatically improve image classification. The Transformer architecture then enabled models to process relationships between words in parallel. 6:45
- Scaling language modelsOpenAI’s GPT-3 paper demonstrated that very large language models could perform varied tasks from prompts, helping fuel the current AI boom. 8:50
✨ Key takeaways
- A chain of ideas built modern computingThe video traces how foundational papers on computation, information, learning, distributed systems, search, and neural network architectures contributed to today’s AI. 9:53
- Data and compute matterBackpropagation had been known for years, but it became far more effective when larger datasets and faster hardware became available. 6:14
- Scale can produce surprising abilitiesGPT-3’s size and training data allowed it to translate, summarize, and write code without being specifically trained for each task. 9:21
🧠 Lessons learned
- Limits can be productiveThe perceptron’s weaknesses helped reveal the importance of multilayer networks, even though methods to train them took longer to develop. 3:38
- Prediction links early information theory to AIShannon’s work on uncertainty and predicting the next letter echoes the next-token prediction used by modern language models. 2:05
- Infrastructure enables progressDistributed coordination, large datasets, and scalable architectures all helped turn theoretical ideas into practical systems. 4:09
🏁 Conclusion/next steps
- A century of foundationsThe video summarizes the progression from Turing’s abstract machine to modern AI systems, emphasizing how each generation built on earlier ideas. 9:53
- The next-word prediction connectionDespite their apparent sophistication, chatbots generate text by predicting the next token—a concept that connects back to Shannon’s work on information. 9:21
webdev
app development
lesson
tutorial
computer science
cs
computer science papers
On Computable Numbers
Entscheidungsproblem
alan turing
Claude Shannon
A Mathematical Theory of Communication
The Perceptron
Perceptrons
Leslie Lamport
geoffrey hinton
Larry Page
Sergey Brin
Sutskever
ilya sutskever
ai
artificial intelligence
coding
programming
developing
history of computers
Transcript excerpt
0:00 The year was 1936. Alan Turing asked a simple question, "Can machines think?" Actually, no, that's not right. What he really asked was something way more boring, "Can every mathematical problem be solved by an algorithm?" Surprisingly, he proved the answer is no, but in the process, he accidentally invented the computer. Then 12 years later in 1948, another legend shows up named Claude Shannon, and he reduced all human communication down to ones and zeros, casually inventing the bit like it was no big deal. One thing led to another and now in 2026, we have
0:31 18-year-olds in hoodies typing import torch into Python files and cashing billion-dollar checks from venture capitalist boomers. But reaching this point has been underpinned by a century-long chain reaction of computer science papers written mostly by dead people much smarter than us. In today's video, we'll look at 10 of the most important scientific papers in the history of computer science and how they changed the world for better or worse. Our story begins nearly a century ago when mathematician David Hilbert asked the field's biggest flexive of a question, "Is there a universal
🔒 The full, searchable transcript is available with Pro.
Questions & Answers
Common questions about this video
What did Alan Turing prove in his 1936 paper on computable numbers?
Turing showed that no algorithm can decide every mathematical problem, and proved that a general program cannot always determine whether another program will halt or run forever. To explain this, he defined the abstract Turing machine. 1:02
How did Claude Shannon define and measure information?
Shannon treated information in terms of how surprising a message is, measuring it in bits. He showed that information can be represented as a stream of ones and zeros, and used entropy to describe uncertainty. 2:05
Why did the perceptron face criticism, and what was the important idea hidden in that criticism?
Marvin Minsky and Seymour Papert showed that a single-layer perceptron could not learn exclusive OR, contributing to an AI funding collapse. But their work also noted that stacking perceptrons could solve the limitation; researchers still needed a way to train those layers. 3:06
What is Lamport's “happened-before” relation used for?
It orders events in distributed systems by causality instead of relying on a shared wall-clock time. This lets computers use logical clocks to coordinate event order. 4:09
How does backpropagation train a neural network?
The network processes data forward, measures its error, then sends that error backward through the layers using the chain rule to adjust the weights. Repeating this process helps the network learn. 5:43
How did PageRank change web search?
Instead of ranking pages mainly by how often they contained a word, PageRank treated links as votes and gave more weight to votes from trustworthy pages. The algorithm helped Google organize a huge collection of web pages. 6:45
What innovation did the Transformer introduce?
The Transformer replaced sequential token-by-token processing with attention, allowing words to consider other words in the text at the same time. This made the architecture better suited to scaling large language models. 8:19
What was the central idea behind OpenAI's GPT-3 paper?
OpenAI scaled a Transformer to 175 billion parameters and trained it on a vast dataset, betting that useful abilities would emerge at sufficient scale. GPT-3 could perform tasks such as translation, summarization, and code writing without being specifically trained for each one. 8:50
Related summaries
Meta's new model wants "deep access" to your personal life...
The most trusted code on Earth is being rewritten in Rust
Google’s AI endgame is here… everything you missed at I/O 2026
I finally found a use case for OpenClaw…
Cursor ditches VS Code, but not everyone is happy...
