10 Papers That Shaped Computer Science and AI

The central idea

A century of research built today’s AI step by step: Turing defined computation, Shannon formalized information, neural-network researchers developed learning methods, and later work supplied the data, hardware, and architectures that made large language models possible.

The papers and their impact

1. Alan Turing — “On Computable Numbers” (1936)
2. Introduced the theoretical Turing machine, an abstract model of a computer.
3. Proved there is no algorithm that can determine whether every possible program will halt or run forever.

4. This showed that some mathematical problems cannot be solved algorithmically.

5. Claude Shannon — “A Mathematical Theory of Communication” (1948)

6. Defined information in measurable terms, using the bit as its basic unit.
7. Showed that information can be represented as ones and zeros, regardless of what a message means.

8. Applied entropy to measure uncertainty and developed ideas central to compression and prediction.

9. Frank Rosenblatt — Perceptron research (late 1950s)

10. Proposed a simple learning system inspired by neurons: it adjusts weighted inputs to classify patterns.

11. The perceptron helped establish the idea of machine learning, though early predictions about its capabilities proved overblown.

12. Marvin Minsky and Seymour Papert — Perceptrons (1969)

13. Demonstrated that a single-layer perceptron cannot solve problems such as exclusive OR (XOR).
14. This helped dampen enthusiasm and funding for neural networks.

15. The work also highlighted that multiple layers could overcome these limits, though effective training methods were not yet available.

16. Leslie Lamport — “Time, Clocks, and the Ordering of Events in a Distributed System” (1978)

17. Showed why computers in a distributed system cannot rely on a shared, perfectly synchronized clock.
18. Introduced the happened-before relation and logical clocks, which order events by causality.

19. These ideas underpin distributed databases and large-scale computing systems.

20. David Rumelhart, Geoffrey Hinton, and Ronald Williams — backpropagation research (1986)

21. Explained how to train multilayer neural networks: send inputs forward, measure the error, then propagate it backward using calculus to adjust weights.
22. Made it possible for networks to learn useful internal features and solve problems that single-layer systems could not.

23. Backpropagation remains fundamental to modern neural networks.

24. Larry Page and Sergey Brin — “The Anatomy of a Large-Scale Hypertextual Web Search Engine” (1998)

25. Described PageRank, which ranks web pages partly by treating links as votes weighted by the authority of the linking page.

26. Led to Google and helped organize the web into a vast collection of accessible text—valuable data for later AI systems.

27. Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton — AlexNet / ImageNet paper (2012)

28. Combined a large labeled image dataset with a deep convolutional neural network trained on consumer GPUs.
29. AlexNet sharply reduced image-recognition errors in the ImageNet competition.

30. Its success showed that deep learning could work remarkably well with enough data and computing power.

31. Ashish Vaswani and colleagues — “Attention Is All You Need” (2017)

32. Introduced the transformer, which uses attention to let a model relate words across a sequence rather than process them strictly one at a time.

33. Transformers are both powerful and scalable, and became the foundation of today’s large language models.

34. OpenAI — “Language Models Are Few-Shot Learners” (2020)

    • Presented GPT-3, a transformer with 175 billion parameters trained on a large text dataset.
    • Showed that a sufficiently large language model could perform tasks such as translation, summarization, and coding from prompts, without being specifically trained for each task.
    • This scaling approach helped spark the current wave of generative AI and led toward ChatGPT.

The overall progression

• Turing: defined a theoretical machine.
• Shannon: quantified information.
• Rosenblatt: introduced a learning neuron.
• Hinton and colleagues: enabled multilayer learning.
• Distributed-systems research: helped coordinate large computing systems.
• Google and ImageNet researchers: supplied data and demonstrated the power of deep learning.
• Transformers and GPT-3: scaled language modeling into today’s generative AI.

The video’s closing comparison is that modern chatbots still rely on a core idea familiar from Shannon’s work: predicting what comes next, now applied to tokens at enormous scale.

Sponsor mention

The video also promotes Coder Agents, a self-hosted coding-agent platform for teams that want control over their AI workflows and development environments. It supports different language models and is aimed particularly at organizations with strict security or regulatory requirements.