What Is Deep Learning?

Deep learning is a branch of machine learning concerned with training neural networks that contain many successive layers of nonlinear transformations, enabling automatic extraction of hierarchical feature representations from raw data. By stacking layers, these systems learn progressively more abstract descriptions of input signals without requiring hand-engineered features, which has made deep learning the dominant approach in tasks involving images, audio, text, and sequential sensor data. The field draws on optimization theory, numerical linear algebra, and statistical learning theory, and its practical success is closely tied to the availability of large labeled datasets and parallel computing hardware, particularly graphics processing units.

Early multilayer perceptrons proved difficult to train because gradient signals diminish exponentially with depth (the vanishing gradient problem), a barrier that slowed progress through the 1990s. Renewed interest came with Geoff Hinton's work on deep belief networks around 2006, which showed that deep networks could be pre-trained layer by layer before fine-tuning. The decisive turning point was the 2012 ImageNet competition, where a deep convolutional network described in Communications of the ACM reduced the top-5 error rate to 15.3%, more than 10 percentage points below the previous best and signaling a step change in the field.

Architectural Families

Deep learning encompasses several canonical network architectures, each suited to different data structures. Convolutional neural networks (CNNs) apply learned filter banks across spatial or temporal dimensions, exploiting local structure and translation invariance, making them the standard choice for image analysis tasks. Recurrent neural networks (RNNs), and in particular long short-term memory (LSTM) networks, process variable-length sequences by maintaining a hidden state that propagates context across time steps. Transformer architectures, introduced in 2017 with the paper "Attention Is All You Need," replace sequential recurrence with multi-head self-attention, allowing parallel processing of entire sequences and scaling to models with hundreds of billions of parameters. Generative architectures, including variational autoencoders and generative adversarial networks, are trained to model the data distribution rather than a discriminative mapping. Research on learning hierarchical representations in deep networks has characterized why these architectures benefit from depth, showing that intermediate layers add representational flexibility that shallow networks with the same parameter count cannot replicate.

Supervised and Unsupervised Learning

Deep learning can be applied under several learning paradigms. In supervised learning, the network is trained by minimizing a loss function that compares its predictions to ground-truth labels over a large labeled dataset, using stochastic gradient descent with backpropagation. In unsupervised learning, the network learns structure without labels, typically by reconstructing its inputs (autoencoders), modeling the data density (generative models), or forming clusters. Self-supervised learning, which generates supervision signals from the data itself (for example, predicting masked image patches or next tokens in a sequence), has become a major paradigm for pre-training large foundation models that are later fine-tuned for specific tasks. Feature extraction from pre-trained deep networks has proven broadly useful, enabling high accuracy on new tasks with limited labeled data through transfer learning.

Applications

Deep learning has applications in a wide range of fields, including:

• Image classification and object detection in medical imaging, autonomous vehicles, and satellite analysis
• Image and video segmentation for surgical guidance and scene understanding
• Natural language processing, including machine translation, question answering, and large language models
• Speech recognition and speaker verification
• Drug discovery and protein structure prediction, as demonstrated by the AlphaFold system from DeepMind published in Nature
• Recommendation systems and anomaly detection in industrial sensor data