Layers and Networks

By James

The brain is the center of our memory, our perception—our cognition. It is responsible for every feeling, every undertaking, every thought within our lives and that of many other species. However, what remains at its core is the ability to connect, associate one with the other, and use these connections between many to allow us to act, learn, and feel.

In recent years, people have attempted to replicate that capability of the brain in computers, from the earliest neural networks to present-day trillion-parameter LLMs. Similar to the brain, they function through learning patterns and relationships between the data that has been fed to them, generating outputs powering apps such as ChatGPT. This poses the question: how do Neural Networks work, and why are they important to know.

How Neural Networks Work

To understand how Neural Networks (which will be called NNs sometimes for conciseness) work, you can start with the first association that Neural Networks are similar to brains, simulated on a device. Just as you might be learning about NNs while reading this article—to satisfy curiosity or for any other reason, NNs themselves learn from data to complete a task and produce an output.

Neural Networks have come a long way. Their concept was developed in 1943 by Warren McCulloch and Walter Pitts, describing how neurons in brains might work through modeling their activity in a network of electrical circuits. Fast-forward to the 21st century. A lot has changed. Research about NNs has snowballed far beyond electrical circuits and into digit recognition (MNIST aka Modified National Institute of Standards and Technology database), image recognition (AlexNet), and very notably, text generation (GPT, Claude, LLaMA…). However, to truly understand NNs, you need to focus on their core, and how they operate on the inside.

This diagram shows a Neural Network with one input layer, two hidden layers, and one output layer, while its circles represent artificial neurons. The most fundamental component of a Neural Network is an artificial neuron. Similar to how the human neuron transmits electrical signals to produce a thought or movement, an artificial neuron transmits data throughout the neural network, resulting in an output of a decision or classification.

In a Neural Network, inputs first pass through an input layer where their features such as pixel values or other numerical representations are relayed by neurons to the hidden layers.

In hidden layers, features from the input layers are then mathematically transformed by neurons and mathematical functions known as weights and biases, adjustable values learned by the network through data. In addition, activation functions are also added to aid in the mathematical operations, allowing for the network to learn complex patterns, modeling them with curves and boundaries outside the scope of simpler math. Each hidden layer passes its transformed features to the next, until reaching the output layer.

The output layer of a Neural Network is its final portion, where its neurons apply activation functions on the transformed features to produce the network's output. The number of neurons normally corresponds to the amount of possible outcomes. In the case of a neural network outputting a Yes/No decision, there may be 2 neurons in its output layer where one represents "Yes" and another represents "No".

To further illustrate Neural Networks, we can return to MNIST. MNIST is a dataset of handwritten digits meant for AI models to be trained on it to recognize numerical handwriting.

A Neural Network can be trained on it, learning associations and differentiating between digits. After the NN learns the MNIST data, it can predict digits from photos, starting from the photo being split into features such as pixel darkness values. Those features are then manipulated in the hidden layers to make associations and find a potential digit in the input. Lastly, after passing through all the hidden layers, the features are finally sent to the output layer for final transformation and after that, the NN produces its output of a detected digit.

This is only one of many examples where Neural Networks are implemented, which have now become integrated in even the most complex models aiding doctors in disease diagnoses or powering popular apps such as ChatGPT.

Why Neural Networks

Neural Networks have made their way into virtually every industry, device—and this article. Especially in recent years, NNs have grown at fast pace, serving as the underlying backbone of all new models, whether small ones made as passion projects, or complex trillion-parameter models (think Gemini 2.5, Claude 4, or the soon-to-be-released GPT-5). Knowledge of AI's foundational components, especially Neural Networks is critical to get ahead of what's going on with it and its new advancements.

There is a lot more to Neural Networks than what this article discusses, but this should give basic overview of what's behind, around, and about them.