An artificial neural network is a computer system composed of mathematical processing units (artificial neurons) organized into interconnected layers. Inspired by the human brain, it learns from data by adjusting its "synaptic weights" to perform tasks such as image recognition or language processing, without explicit programming of all the rules.
To understand how artificial neural networks work, it is necessary to grasp the meaning of artificial intelligence (AI). AI involves creating computer systems capable of automating tasks without human intervention, as well as learning, adapting, improving, communicating, and especially making decisions. In this sense, AI seeks to replicate aspects of human intelligence.
The definition of intelligence is highly debatable, but the meaning that interests us here is that of functions controlled by the brain. Indeed, whether mechanical like walking or cognitive like decision-making, the architecture of these functions remains the same in all areas of our brain. Furthermore, all functions are of equal importance, without distinction between those considered noble or less noble. They are all performed with the same complexity of calculations, through a vast network of billions of interconnected neurons that exchange information. Information is exchanged via electrical and chemical signals.
In the human brain, the process of communication between neurons is as follows:
- When electrical signals, initiated at the neuron's cell membrane, reach a critical threshold, they trigger a brief electrical impulse called an "action potential." Action potentials travel along the axon to the neuron's synapse.
- At the synapse (points of contact between neurons), action potentials trigger the release of chemical molecules called "neurotransmitters."
- Neurotransmitters then bind to receptors located on the membrane of the postsynaptic neuron (the receiving neuron). This chemical binding triggers an electrical response in the postsynaptic neuron.
- The postsynaptic neuron integrates all the inputs it receives from the emitting neurons, and if the critical threshold is reached, it generates its own action potential, which propagates along its own axon, thus continuing the transmission of information in the network.
Depending on the types of neurotransmitters released, presynaptic activity can have an excitatory or inhibitory effect on the postsynaptic neuron's electrical activity. This makes the triggering of an action potential more or less likely.
This complex process is the basis of brain function and human cognition. It is from this process that researchers developed an artificial intelligence model, initially very simple and then increasingly sophisticated as technology advanced.
The artificial neuron is the basic unit of an artificial neural network. An artificial neural network consists of a succession of interconnected neuron layers, where each layer takes its inputs from the outputs of the previous layer.
Artificial neurons are not computer bits (0 or 1), but rather mathematical abstractions (numbers, operations, functions, equations, matrices, sets, probabilities, etc.). In other words, they are processing units that perform mathematical operations on the data presented to them. They do not store data, unlike computer bits, which are the basis of digital information storage.
In an artificial neural network, each neuron is characterized by an activity level captured by a variable called the "activation potential." The neuron's synapse is characterized by another variable called the "synaptic weight."
- The activation potential represents the electrical state of the postsynaptic neuron at a given time. It is calculated by summing the input signals from presynaptic neurons, each signal being modulated by the corresponding synaptic weight. The activation potential can vary continuously within a range from negative to positive values, depending on the intensity of the incoming signals.
- Synaptic weights determine whether a synaptic connection is excitatory, inhibitory, or null. Weights modulate the impact of input signals on the activation potential. Positive weights increase activity, negative weights reduce it, and zero weights have no effect.
The activation potential results from the combination of input signals weighted by synaptic weights. This potential is then subjected to an activation function, which introduces non-linearity and determines whether the postsynaptic neuron generates a response (action potential) or not. Ultimately, these mechanisms allow the neuron to process information and respond adaptively to stimuli. The functioning of these variables is fundamental for modeling neuron behavior, both in biological and artificial neural networks.
Imagine an artificial neural network used for image classification. This network has a postsynaptic neuron that receives connections from three presynaptic neurons. Each of these three presynaptic neurons is associated with a specific feature of the image the network is analyzing, such as the presence of vertical lines, horizontal lines, and curves.
The postsynaptic neuron has an initial activation potential of 0.
When the three presynaptic neurons send their signals, each signal is multiplied by the synaptic weight associated with the corresponding connection. Assume the synaptic weights are as follows:
- Synaptic weight for the vertical lines feature: +0.5
- Synaptic weight for the horizontal lines feature: -0.3
- Synaptic weight for the curves feature: +0.2
The signals from the three presynaptic neurons are weighted by their respective synaptic weights and summed.
If we have the following signals:
- Signal for vertical lines: 1
- Signal for horizontal lines: 0.5
- Signal for curves: 0.8
The activation potential would be calculated as follows:
Activation potential = (1 * 0.5) + (0.5 * (-0.3)) + (0.8 * 0.2) = 0.5 - 0.15 + 0.16 = 0.51
If the activation potential exceeds a defined threshold (e.g., 0), the postsynaptic neuron will generate an action potential, indicating that the desired feature has been detected in the image.
In this example, synaptic weights play a crucial role in determining the relative importance of each image feature. The input signals weighted by synaptic weights are used to calculate the activation potential, which, if it exceeds the threshold, will trigger the response of the postsynaptic neuron. This allows the neural network to make decisions based on the features detected in the image.
The hardware infrastructure of an artificial neural network is not biological; it is the same as that of classical computing (microprocessors, graphics cards, etc.).
The software infrastructure of an artificial neural network is different. Machine learning algorithms learn from data and adjust their behavior based on the examples provided, whereas traditional programming algorithms are based on static explicit instructions that do not change on their own. In this sense, AI is a revolution because to statically write the rules of ChatGPT 3.5 with its 175 billion parameters, it would have taken thousands of years.
A neural network is organized into layers where each artificial neuron (mathematical function) receives inputs, performs calculations on these inputs, and generates an output. The first layer is the input layer, which receives raw data (text, digital image, or other collected data). Behind it, there are one or more hidden layers (not accessible from the outside), followed by the output layer, which produces predictions.
To make a prediction, data is propagated from the input layer to the output layer. Each neuron sums its weighted inputs, applies an activation function, and passes the result to the next layer.
Activation functions introduce non-linearities into the network. This means that the relationship between quantities is not a constant proportion but a probability. This is what gives neural networks their ability to solve a variety of problems, from image recognition to machine translation, including natural language modeling.
After making a prediction, the network compares its own results to the correct labels to measure the error or difference between the two. The correct labels are an essential component of the training set for a supervised learning model. They are provided for each example in the training set to enable the model to learn to make accurate predictions.
In the next step, the backpropagation algorithm adjusts the network's weights (internal parameters that determine how neurons respond to inputs). This allows it to find the values that minimize the model's error. This process is repeated until the network reaches a satisfactory performance level.
In the network, there are also hyperparameters to adjust, such as the learning rate, the batch size used, the network architecture, the choice of activation function in the layers, etc.
After evaluating the model for each combination, researchers choose the hyperparameters that give the best performance on the validation data.
In the end, the trained model is evaluated on new, unseen data.
Suppose we have a data center capable of providing us with 100,000 different 28x28 grayscale images representing handwritten digits from 0 to 9.
Our neural network will have an input layer configured to the size of the images (28x28 neurons), one or more hidden layers, and an output layer with 10 neurons (since there are 10 possible digits: 0 to 9). Each neuron in the output layer represents the probability that the image corresponds to a particular digit.
The weights of the connections between neurons are initially set to random values.
The raw data, for example, the digital image of the number 3, is introduced into the input layer.
To analyze local regions of the image, convolution filters slide over the image to extract a map of hierarchical visual features. The first layers detect edges, while higher layers detect more complex patterns.
Data propagates through the network by following weighted connections and applying activation functions. At each layer, mathematical operations are performed to obtain an output.
The output layer produces scores for each digit (0-9). A function transforms these scores into probabilities. The digit with the highest probability will be the network's prediction.
Next, the network will compare its prediction with the actual label of the image.
A cost function measures the divergence between the model's predictions and the actual labels.
To do this, the error is propagated backward through the network. The network adjusts its parameters (weights and biases) in each layer to minimize the error, using algorithms like gradient descent.
This process is repeated over a very large number of training images. The network will adjust its parameters in each iteration to improve its classification of handwritten digits. Once the network is trained, it is tested on a separate dataset to evaluate its performance.
An artificial neural network is inspired by the human brain: interconnected neurons exchange signals whose impact is modulated by synaptic weights. In its artificial version, each neuron is a mathematical function that calculates a weighted sum of its inputs, applies an activation function, and transmits the result.
Learning involves automatically adjusting synaptic weights from examples, via a process of backpropagation and optimization (gradient descent). It is this ability to learn without explicit programming of rules that distinguishes AI from traditional computing.
In practice, the network is organized into layers (input, hidden, output), and after iterative training on labeled data, it becomes capable of making predictions on new data (image recognition, language processing, etc.).
A biological neuron exchanges information via electrical and chemical signals (neurotransmitters). The artificial neuron is a mathematical abstraction (numbers, functions) that performs calculations. Unlike computer bits, it does not store data but processes information via an activation potential and synaptic weights.
Learning happens in three main steps: first, the network makes a prediction by propagating input data through its layers. Then, it compares its prediction to the correct label via a cost function. Finally, a backpropagation algorithm adjusts the synaptic weights to minimize the error, repeating this process over many examples.
Activation functions introduce non-linearities into the network. This means that the relationship between quantities is not a constant proportion but a probability. This characteristic allows neural networks to solve complex problems such as image recognition, machine translation, or natural language modeling.
AI Tools: How to Choose? 
Generative AI vs AGI: Where does imitation end and consciousness begin? 
Artificial networks vs biological networks: Two systems, one common architecture 
Human Brain and Artificial Intelligences: Similarities and Differences 
AlphaGo vs AlphaGo Zero: A Revolution in Artificial Intelligence 
The Next Step for Intelligent Machines 
The First Step Towards the Emergence of Life 
From Biological Neuron to Formal Neuron: Simplifying the Brain 
Artificial intelligence: the explosion of gigantism 
When AI models train on their own data, they go mad! 
Emergence of artificial intelligence: Illusion of intelligence or intelligence? 
How do machines understand, interpret and generate language in a similar way to humans? 
How does an artificial neural network work?