What Is High Performance Computing?

High performance computing (HPC) is the practice of aggregating computational power far beyond what a single desktop or server can deliver, in order to solve problems that are either too large, too time-sensitive, or too data-intensive for conventional systems. HPC systems range from multi-node clusters running commodity processors to tightly coupled supercomputers with hundreds of thousands of processor cores, high-bandwidth interconnects, and parallel storage subsystems capable of moving terabytes per second. The field draws on parallel algorithms, computer architecture, networking, and systems software, and its scale is tracked through benchmarks such as the High Performance Linpack (HPL) test that underlies the TOP500 list of the world's most powerful systems, published twice yearly since 1993.

The discipline has roots in vector supercomputing of the 1970s and the message-passing workstation clusters of the 1990s. Contemporary HPC systems are predominantly heterogeneous, pairing many-core CPUs with GPU accelerators, and the first exascale machines (systems capable of 10^18 floating-point operations per second) began entering service at U.S. Department of Energy national laboratories in 2022.

Parallel Architectures

HPC systems are classified by their memory organization. In shared-memory architectures, multiple processor cores access a single address space, enabling low-latency communication through ordinary reads and writes but introducing contention at the memory bus. In distributed-memory architectures, each node has private memory, and processes coordinate only by passing messages explicitly over a high-speed network fabric such as InfiniBand or a proprietary interconnect. Modern supercomputers are hybrid: each node is itself a shared-memory multiprocessor, while nodes communicate over a distributed-memory interconnect. GPU-accelerated nodes add a third tier, with GPU on-chip memory accessed by thousands of narrow compute cores in parallel. The interconnect topology, whether fat-tree, dragonfly, or torus, strongly influences scaling efficiency at node counts in the hundreds of thousands.

Programming Models

The dominant programming interfaces for HPC are MPI (Message Passing Interface) and OpenMP, often used together in a hybrid style. MPI governs communication between distributed-memory processes, providing portable point-to-point and collective operations that work across any network fabric. OpenMP adds thread-level parallelism within a shared-memory node through compiler directives that parallelize loops and task regions with minimal source changes. CUDA and HIP extend this model to GPU threads, which execute the same instruction stream across thousands of cores simultaneously. A comparative analysis of MPI, OpenMP, and CUDA demonstrates how these models address fundamentally different granularities of parallelism and are typically composed rather than substituted for one another. The choice of programming model shapes the algorithm design, the memory access pattern, and the communication overhead that limit scaling at large node counts.

Performance Benchmarking and Scaling

Measuring HPC performance requires more than raw floating-point rates. The HPL benchmark stresses dense linear algebra and reflects peak throughput, while HPCG (High Performance Conjugate Gradient) captures the irregular, sparse memory access patterns typical of real scientific applications. The OpenMP Architecture Review Board and the MPI Forum publish the standards that these workloads exercise. Amdahl's Law and Gustafson's Law bound the speedup achievable by adding processors, with Amdahl's framing relevant when the serial fraction of a code is fixed and Gustafson's framing relevant when the problem size grows with the machine. Strong scaling tests measure how fast a fixed problem can be solved on more cores; weak scaling tests measure how the solution time behaves as both problem size and core count grow proportionally.

Applications

High performance computing has applications across a range of fields, including:

• Computational fluid dynamics for aerospace and automotive design
• Climate and weather modeling at regional and global scales
• Molecular dynamics and quantum chemistry simulation in drug discovery
• Seismic processing and reservoir modeling in petroleum exploration
• Large-scale machine learning training for neural network models
• Nuclear stockpile stewardship and fusion energy research at national laboratories