What Are Systolic Arrays?

Systolic arrays are homogeneous networks of tightly coupled processing elements arranged in a regular geometric pattern, each performing a simple computation and passing results to neighboring elements in a rhythmic, clock-synchronized flow. The name draws an analogy to the rhythmic pumping of the heart: data "pulses" through the array in a regular cadence, with each processing element consuming input, operating on it, and passing the result onward without stalling the pipeline. H.T. Kung and Charles Leiserson introduced the concept in a 1979 technical report at Carnegie Mellon University, motivated by the constraints of very-large-scale integration (VLSI) fabrication, where communication bandwidth between distant components is more expensive than local computation.

The core insight behind systolic arrays is that many computationally intensive algorithms, particularly those involving dense linear algebra, can be decomposed into fine-grained operations that map naturally onto a mesh or linear chain of identical processing elements. By keeping data moving locally between neighbors rather than routing it through shared memory or a central bus, systolic arrays minimize memory access bottlenecks and achieve high throughput with modest hardware complexity. H.T. Kung was later elected to the U.S. National Academy of Engineering specifically for introducing systolic computation and its contributions to VLSI design, as noted in his faculty biography at Harvard's School of Engineering.

Architecture and Data Flow

A systolic array consists of a grid of processing elements (PEs), each connected only to its immediate neighbors. Data enters from the edges of the array and propagates inward according to a fixed schedule determined by the algorithm being implemented. In a matrix multiplication array, for example, elements of the input matrices flow through the grid at fixed velocities, and each PE accumulates a partial dot product as the streams pass through it. At the end of the computation cycle, the result matrix has been constructed without any element visiting more than a few neighbors, achieving a high degree of parallelism through spatial distribution rather than fast interconnects.

Pipeline processing is a related architectural strategy: where a pipeline serializes a sequence of stages along a single data path, a systolic array applies the same pipelining discipline simultaneously across many parallel paths arranged in two dimensions. This distinction allows systolic arrays to achieve both spatial and temporal parallelism, making them far more efficient than single-pipeline designs for problems with sufficient data reuse.

Signal Processing and Matrix Computations

The original motivation for systolic arrays was digital signal processing (DSP) and linear algebra, where operations such as discrete Fourier transforms, convolutions, LU decomposition, and matrix-vector products dominate computational cost. These operations involve highly regular access patterns and dense arithmetic that map directly onto the fixed topology of a systolic array. Because each PE only needs to communicate with its nearest neighbors, the design scales well in VLSI without requiring long, power-hungry global interconnects. The full derivation of systolic architectures for linear algebra and signal processing is detailed in the original 1979 CMU technical report by Kung and Leiserson.

Systolic Arrays in Modern Hardware Accelerators

The concept re-emerged prominently with the rise of deep learning. Matrix multiplication is the dominant operation in neural network inference and training, and systolic arrays are ideally suited to it. Google's Tensor Processing Unit (TPU), introduced in 2016, implements a large systolic array at its core to execute matrix multiplications with high energy efficiency. The TPU architecture demonstrated that Kung and Leiserson's 1979 design principle remained practically relevant decades later and became a widely studied reference for AI chip design, as described in publications from Google Research.

Applications

Systolic arrays have applications across a range of computation-intensive domains, including:

• Deep learning inference and training in neural network accelerators
• Real-time digital signal processing for radar and communications
• Dense linear algebra in scientific simulation and computational fluid dynamics
• Image and video processing pipelines
• Cryptographic computations, including those used in hardware security modules