AlphaGo vs AlphaGo Zero: The Clash of the Titans of AI

The Story of the Encounter AlphaGo vs AlphaGo Zero — Historic match: AI against the grand master.
Image source: astronoo.com (new window) — AI-generated image, public domain.

Scientific summary

Astronoo's article compares AlphaGo (2016) and AlphaGo Zero (2017), two artificial intelligence systems developed by DeepMind for the game of Go. AlphaGo used supervised learning on millions of human games, combined with reinforcement learning. AlphaGo Zero revolutionized this approach by learning from scratch, solely with the rules of the game and by playing against itself. In 40 days, AlphaGo Zero surpassed all previous versions, beating AlphaGo Master 100 to 0. This demonstration proved that AI can achieve superperformance without relying on human data or strategies, paving the way for applications in various fields such as structural biology (AlphaFold) or energy optimization.

What is the difference between AlphaGo and AlphaGo Zero and why is their encounter historical?

AlphaGo (2016) was the first AI program to beat a world champion of Go, Lee Sedol, using supervised learning on millions of human games, combined with reinforcement learning against itself. AlphaGo Zero (2017) revolutionized this approach by learning from scratch, without any human data: only the rules of the game and by playing millions of games against itself. In 40 days, AlphaGo Zero surpassed all previous versions of AlphaGo, beating it 100 to 0 in test matches. This victory demonstrated that AI can achieve superperformance without being limited by human data or strategies.

Also worth exploring

Continue your exploration of the Universe with these topics:

The Story of the Encounter AlphaGo vs AlphaGo Zero

A turning point in artificial intelligence

The story of the encounter between AlphaGo and AlphaGo Zero marks a significant turning point in the field of artificial intelligence (AI). These two systems, developed by DeepMind, a subsidiary of Google, have not only pushed the boundaries of what was thought possible for machines, but have also opened new perspectives for machine learning and AI in general.

The context of the competition

The comparison between AlphaGo and AlphaGo Zero did not take place during a public match, but in a laboratory environment at DeepMind, based in London. The researchers used simulations and benchmarks to objectively measure the performance of the two systems, demonstrating the superiority of AlphaGo Zero.

AlphaGo: The Precursor

The first victory against a world champion

AlphaGo, launched in 2016, made a splash by beating Lee Sedol, one of the best Go players in the world, in a series of highly publicized matches. Go, a complex strategy game originating in China, was considered one of the last bastions of human superiority in strategy games. The victory of AlphaGo demonstrated that machines could not only match, but also surpass humans in tasks requiring advanced intelligence and strategy.

AlphaGo's architecture: supervision and reinforcement

AlphaGo used a combination of supervised learning and reinforcement learning techniques. It was trained on millions of Go games played by humans, then refined its skills by playing against itself. This approach allowed AlphaGo to develop innovative strategies and surprise even human experts.

AlphaGo Zero: The Innovation

Learning from scratch (tabula rasa)

AlphaGo Zero, introduced in 2017, represented an even more impressive advance. Unlike AlphaGo, AlphaGo Zero was not trained on Go games played by humans. Instead, it learned to play Go from scratch, using only the rules of the game and playing millions of games against itself. This approach, known as reinforcement learning from scratch, allowed AlphaGo Zero to develop entirely new strategies and surpass AlphaGo in just 40 days of training.

More efficient and faster training

AlphaGo Zero used a single neural network, unlike AlphaGo which used two (one for policy, one for value). This simplification, combined with increased computing power, allowed AlphaGo Zero to achieve a higher level of play in record time.

The Historical Encounter

The theoretical match: AlphaGo Master vs AlphaGo Zero

The theoretical encounter between AlphaGo and AlphaGo Zero was a key moment to assess the progress made in the field of AI. These matches took place in a laboratory environment at DeepMind, based in London. They were conducted for research purposes and did not take place in front of an audience or in a competitive setting. The results showed an overwhelming domination by AlphaGo Zero, with a win ratio of 100 to 0 against AlphaGo Master (the most advanced version of AlphaGo that had beaten Lee Sedol).

Lessons from this confrontation

This victory demonstrated that AI systems could learn and improve autonomously, without needing human data to guide them. It also proved that pure reinforcement learning, without human supervision, could lead to more creative and effective strategies than those developed by humans.

Implications and Perspectives

A break with human data

The victory of AlphaGo Zero had profound implications for AI. It showed that machine learning systems could be even more powerful and efficient when they were freed from the constraints of human data. The victory of AlphaGo Zero reinforced China's determination to invest massively in AI research and development. The Chinese government has already implemented several initiatives to become a global leader in this field. For example, the "Made in China 2025" plan includes ambitious goals for AI, and Chinese companies like Baidu, Alibaba, and Tencent are investing billions in AI projects.

Applications beyond the game of Go

The principle of reinforcement learning from scratch has been extended to other fields: solving protein folding problems (AlphaFold), optimizing energy consumption in data centers, discovering new materials, or developing strategies in chemistry and fundamental physics. This approach consists of defining rules and a reward function, then letting the AI explore and optimize by itself.

Conclusion

A pivotal moment in the history of AI

The theoretical encounter between AlphaGo and AlphaGo Zero will remain in the annals of AI history as a pivotal moment. It not only demonstrated the impressive capabilities of machine learning systems, but also opened new perspectives for the future of AI.

Towards increasingly autonomous AI

AlphaGo Zero proved that AI can achieve superperformance without relying on human expertise, paving the way for AI systems capable of solving complex problems in areas where human data is scarce or non-existent.

References

FAQ: Everything you need to know about AlphaGo, AlphaGo Zero and their revolution

Why is the game of Go such a difficult challenge for AI?

Go is a strategy game with vertiginous combinatorial complexity. While chess has about 10¹²⁰ possible positions, Go has about 10¹⁷⁰ (more than the number of atoms in the observable universe). This complexity makes the "brute force" approach (exhaustive exploration of all moves) impossible. For a long time, Go was considered one of the last bastions of human superiority over machines, because it requires strategic intuition, position evaluation, and a global vision that are difficult to program.

What is the fundamental difference between AlphaGo and AlphaGo Zero?

The key difference lies in their training data. AlphaGo was trained in two phases: first by supervised learning on millions of human games (to imitate the best players), then by reinforcement learning by playing against itself. AlphaGo Zero eliminated the first phase: it learned from scratch (tabula rasa), knowing only the rules of the game. It played millions of games against itself, discovering innovative strategies that humans had never imagined. This approach not only surpassed AlphaGo, but also produced a more "creative" and efficient game.

What are the implications of AlphaGo Zero's victory beyond the game of Go?

AlphaGo Zero's victory has major implications: it demonstrates that AI can achieve superperformance without relying on potentially biased or suboptimal human data. This principle of "reinforcement learning from scratch" has been extended to other fields: solving protein folding problems (AlphaFold), optimizing energy consumption in data centers, discovering new materials, or developing strategies in chemistry and fundamental physics. The method consists of defining rules and a reward function, then letting the AI explore and optimize by itself.

What has been the impact of AlphaGo and AlphaGo Zero on AI research?

AlphaGo and AlphaGo Zero have had a considerable impact on AI research. They demonstrated the power of deep reinforcement learning and inspired much research in this field. They also proved that it is possible to develop AI systems capable of solving complex problems without requiring prior human expertise, paving the way for applications in areas where human data is scarce or expensive to obtain.

Can AlphaGo Zero be applied to other games or problems?

Yes, the principle of AlphaGo Zero has been applied to other games, notably chess and shogi, with equally impressive results: AlphaZero (the generalized version) beat the best chess and shogi programs in a few hours of training. Beyond games, the principles of AlphaGo Zero are used to solve complex scientific problems, such as predicting protein structures (AlphaFold) or optimizing complex systems.

What is AlphaFold and what is its connection to AlphaGo Zero?

AlphaFold is an AI system developed by DeepMind to predict the 3D structure of proteins from their amino acid sequence. It uses principles similar to those of AlphaGo Zero: reinforcement learning and autonomous exploration of the solution space. AlphaFold revolutionized structural biology by solving a problem that had remained unsolved for 50 years. It is considered one of the most important applications of the techniques developed with AlphaGo Zero.