What is deep learning? Deep learning is the form of machine learning behind most of the AI tools you use at work, such as ChatGPT, Claude and speech-to-text for taking minutes. The technique learns patterns from large numbers of examples using networks made up of many layers.
A deep learning model learns by repeatedly comparing its predictions with the correct answer and adjusting its internal settings accordingly.
Classic machine learning works with features a person chooses in advance, whereas deep learning extracts those features from raw data itself. That is why deep learning can process raw text, images and speech without anyone describing the features beforehand.
Within artificial intelligence, deep learning is the layer below machine learning and the foundation of language models. If you work with it in an office, you get convincing output that is sometimes wrong, and that limitation determines how you check the output.
What is deep learning?
Deep learning is a form of machine learning in which a model learns from examples through a network with many successive computational layers. Each layer extracts a slightly more abstract pattern from the data than the layer before it.
In 2015, Yann LeCun, Yoshua Bengio and Geoffrey Hinton described deep learning in Nature as models with multiple processing layers. Those layers learn representations of data at different levels of abstraction. In image recognition, the first layers detect edges and patches of colour, later layers shapes, and the final layers complete objects.
The word deep in deep learning refers to the number of layers, not to depth of insight or understanding. A deep learning model recognises patterns in data without understanding what those patterns mean.
How does deep learning work?
Deep learning works by having a model process a very large number of examples and adjust its settings by a small amount after each mistake. Those settings are called weights: numbers that determine how much each signal counts.
With labelled data, the model receives the correct answer with each example, for instance an email labelled spam or not spam. Training follows a fixed cycle of predicting, measuring the error and adjusting the weights. After enough rounds, the model recognises the pattern in examples it has never processed before.
The building block of deep learning is a neural network, with layers of computational units that pass signals to each other. Deep learning uses such networks with many hidden layers between the input and the output.
What is the difference between deep learning and machine learning?
The difference between deep learning and machine learning lies mainly in who decides what the model pays attention to. In classic machine learning, a person chooses the features in advance, such as the amount, the date and the sender of an invoice.
A deep learning model extracts those features from raw data itself, such as the pixels of a scan or the words in a text. The table below sets out the five main differences side by side.
| Aspect | Classic machine learning | Deep learning |
|---|---|---|
| Features | A person chooses in advance which features count | The model learns the features from the data itself |
| Type of data | Mainly structured data in tables | Also unstructured data: text, images and sound |
| Amount of data | Works with smaller datasets | Needs large numbers of examples |
| Computing power | Runs on an ordinary processor | Needs a lot of computing power, usually on GPUs |
| Explainability | The decision can usually be traced | The decision is hard to trace |
Classic machine learning remains the logical choice for structured questions with a clear outcome, such as a credit score or a revenue forecast. Deep learning comes into its own with text, images and speech, where nobody can describe the features well in advance.
How do AI and machine learning relate to deep learning?
AI and machine learning relate to deep learning as three nested concepts: deep learning sits within machine learning, and machine learning sits within AI. Artificial intelligence is the whole field of systems that perform tasks that would normally require human intelligence.
Machine learning is the branch in which systems learn from data instead of following fixed rules. Within that branch, deep learning is the approach that uses networks with many layers.
Language models are the best-known application of deep learning. A language model, also called an LLM, is a deep learning model trained on huge amounts of text that predicts the most likely continuation word by word. Generative AI, which creates text, images or sound, runs on the same technique.
Each concept sits within the one below it.
Where do you come across deep learning in your work?
Deep learning sits behind most of the AI tools you use at work, often without you noticing. Four applications come up most often in office work:
- AI chatbots: ChatGPT from OpenAI, Claude from Anthropic and Gemini from Google run on large language models, the best-known form of deep learning.
- Speech recognition: minute-taking tools turn the recording of a meeting into text with models trained on large amounts of spoken language.
- Translation: translation tools translate whole sentences in context, rather than word by word with a fixed word list.
- Document recognition: scanning and accounting software reads amounts, dates and names from invoices, receipts and scanned letters.
Almost every large language model is built on the transformer, an architecture that Ashish Vaswani and colleagues described in 2017 in the paper Attention Is All You Need. The T in GPT stands for that transformer.
What are the limitations of deep learning?
Deep learning has four limitations that directly affect working with AI tools. The first is explainability: a deep learning model gives an outcome without showing which steps led to it.
The second is dependence on training data. Whatever is missing or skewed in the data, the model misses or distorts, and that leads to bias in things like recruitment texts or risk estimates.
The third is convincing but wrong output. A language model predicts the most likely continuation and does not check whether that continuation is correct, which leads to AI hallucination in the form of made-up sources, legal provisions or figures.
The fourth is the computing power required, which means only large players can afford to train the biggest models. For professionals, this leads to one working rule: deep learning delivers a draft, and the checking stays with you.
Is ChatGPT a form of deep learning?
Yes, ChatGPT is a form of deep learning, because the service runs on OpenAI’s GPT models: large neural networks with many layers, trained on huge amounts of text. The chat interface is the layer through which you work with that model.
Is deep learning the same as generative AI?
No, deep learning is not the same as generative AI, because generative AI is one application of deep learning. Generative AI creates new text, images or sound.
Deep learning is also used for recognition and prediction, such as speech recognition and fraud detection, where nothing new is created.
Why does deep learning need so much data and computing power?
Deep learning needs a lot of data and computing power because a model has millions to billions of weights. Each of those weights is only set correctly after a very large number of examples. AlexNet, the network that won the ImageNet competition in 2012, already had 60 million parameters, the adjustable numbers that include the weights.
Every training round requires large numbers of calculations at the same time. Graphics processing units, or GPUs, perform such calculations in parallel, which is why deep learning training runs on GPUs or similar chips.
When did deep learning take off?
Deep learning took off in 2012, when the AlexNet network won the ImageNet image recognition competition. According to the official results, AlexNet achieved a top-5 error rate of just over 15 per cent, against just over 26 per cent for the runner-up. A top-5 error rate shows how often the correct label was not among the model’s five best guesses.
In 2019, Yoshua Bengio, Geoffrey Hinton and Yann LeCun received the 2018 Turing Award for their breakthroughs in deep neural networks. In 2024, John Hopfield and Geoffrey Hinton received the Nobel Prize in Physics for discoveries that enable machine learning with artificial neural networks.
Frequently asked questions about deep learning
Beyond how deep learning works, many questions concern the data and software that go with it.
Does deep learning always need labelled data?
No, deep learning does not always need labelled data, because language models first learn from large amounts of text without labels. In 2018, OpenAI described a two-stage approach: first train a transformer on a great deal of data without supervision, then fine-tune it on much smaller labelled datasets. InstructGPT added a step with human feedback, in which reviewers ranked the model’s answers.
What is a convolutional neural network?
A convolutional neural network, or CNN, is a deep learning network for images that uses convolutional layers to recognise local patterns, such as edges and shapes. AlexNet, the network that won the ImageNet competition in 2012, consisted of five convolutional layers, some of which were followed by so-called max-pooling layers. That is why the title of the AlexNet paper refers to deep convolutional neural networks.
Which software do you use to build a deep learning model?
You build a deep learning model with a framework such as PyTorch or TensorFlow. PyTorch is an open source project under the PyTorch Foundation, which is hosted by the Linux Foundation. TensorFlow describes itself as an end-to-end platform for building and deploying machine learning models. To work with ChatGPT or Claude in an office, you do not need such a framework.
How do you learn to work with AI tools that run on deep learning?
You learn to work with AI tools that run on deep learning by understanding what such a tool does and where your own checking begins. To use ChatGPT, speech recognition or image recognition responsibly, you do not build neural networks, but you do need to know the difference between a pattern and a fact.
LearnLLM covers those basics in the AI Literacy Course, with practical assignments based on your own work, ending with a certificate of completion.
