Generative AIs, such as GPT-3, Copilot, or Gemini, are trained on massive datasets, comprising billions of texts, images, music, or videos. These deep neural networks, composed of millions, even billions of parameters, raise a fascinating question: how can the increase in these parameters give rise to a form of intelligence?
Parameters are the internal variables of an AI model, automatically adjusted during training. For example, OpenAI's GPT-3 contains 175 billion parameters, while DALL-E has 12 billion. These parameters determine the model's ability to learn and produce coherent results. The higher the number of parameters, the more the model can capture complex nuances in the data.
However, there is a limit: overfitting. When the number of parameters exceeds the available data, the model loses its ability to generalize. To avoid this, it is necessary to balance the model's size and the amount of training data.
Artificial neural networks (ANNs) work by performing simple calculations: each neuron combines its inputs, applies an activation function, and passes the result to the next layer. Yet, from this basic mathematical process emerges the ability to generate coherent and contextually relevant sentences.
AI does not understand the meaning of words, but it predicts the next word or "token" based on learned probabilities. Despite this, it gives the impression of understanding context, intent, and language nuances. This emergence of "intelligent" behavior from simple rules is one of the most intriguing aspects of AI.
Emergence is a scientific concept describing the appearance of new properties from the interaction of simple components. For example, after the Big Bang, matter emerged from pure energy. Similarly, life emerges from the complex interaction of chemical molecules.
In 2017, with the advent of GPT-2, researchers observed a phase transition: beyond a certain threshold of data and parameters, AI model performance improved dramatically. This transition marked the emergence of sophisticated cognitive abilities, such as generating human-like text.
AI is evolving at a breakneck pace. With increasing model complexity, new properties could emerge, such as creativity, understanding of reality, or even a form of consciousness. These advances could revolutionize our understanding of intelligence, both artificial and biological.
In parallel, AI could help unlock the mysteries of the human brain. By studying how artificial neural networks learn and evolve, scientists could better understand the mechanisms of human cognition.
"Chance is the god of inventors." - Pierre Dac