What Are Graphical Models?

Graphical models are a family of statistical frameworks that use graphs to represent the joint probability distribution over a set of random variables. Each node in the graph corresponds to a variable, and the edges encode the conditional independence structure: variables that are not connected in the graph are assumed to be conditionally independent given the values of certain neighboring variables. This graph-based encoding allows complex high-dimensional distributions to be specified compactly and to be processed by efficient algorithms that exploit the factored structure of the joint distribution.

The field draws from probability theory, graph theory, and information theory. It provides a common mathematical language for models that were historically developed separately, including hidden Markov models, Kalman filters, factor graphs, and neural network architectures with latent variables. The standard reference for the field, the MIT Press textbook by Daphne Koller and Nir Friedman on Probabilistic Graphical Models, organizes the discipline around three problems: representation, inference, and learning.

Directed Graphical Models

Directed graphical models, also known as Bayesian networks or belief networks, represent conditional dependencies through a directed acyclic graph (DAG). A directed edge from variable A to variable B indicates that A is a direct probabilistic cause of B in the model; the joint distribution factorizes as a product of conditional distributions, one per node given its parents in the graph. The DAG structure encodes a set of conditional independence relations through a criterion called d-separation: two variables are independent given a third set if all paths between them in the graph are blocked by that set. Bayesian networks have been applied to medical diagnosis, since a directed graph naturally represents causal relationships such as disease causing symptoms, and to document modeling and speech recognition through hidden Markov models, which are special cases of directed graphical models with a chain topology. A Stanford overview of graphical models in structured relational learning describes how directed models generalize to relational domains.

Undirected Graphical Models

Undirected graphical models, also called Markov random fields (MRFs) or Markov networks, use undirected edges and define the joint distribution through a set of potential functions over cliques in the graph. Clique potentials encode the compatibility between variable configurations within a group; the joint distribution is proportional to the product of all clique potentials, normalized by a partition function that sums over all configurations. Undirected models are well suited for problems where dependencies are symmetric and no natural causal direction exists. Computer vision applications such as image segmentation and stereo reconstruction use MRFs with grid-structured graphs, where each pixel forms a node and potentials couple adjacent pixels to enforce spatial smoothness. The Stanford CS228 course notes on undirected graphical models describe how common energy minimization objectives in vision can be interpreted as MAP inference in an MRF.

Inference and Learning

Inference in graphical models means computing marginal probabilities or the most probable assignment of variables given observations. Exact inference algorithms such as variable elimination and the junction tree algorithm exploit the graph structure to perform inference efficiently for tree-structured graphs or graphs with small treewidth. For graphs with large treewidth, exact inference is intractable in general, and approximation methods dominate: loopy belief propagation applies the sum-product algorithm to cyclic graphs and often converges to useful approximations in practice, while variational methods and Markov chain Monte Carlo provide alternative families of approximations. Learning graphical models from data involves two distinct tasks: parameter estimation, fitting the potential values given a known graph structure, and structure learning, identifying the graph topology that best explains the observed data.

Applications

Graphical models have applications in a range of fields, including:

• Medical diagnosis and clinical decision support through Bayesian belief networks
• Image segmentation and stereo reconstruction in computer vision
• Natural language processing, including part-of-speech tagging and parsing
• Robotics, for simultaneous localization and mapping (SLAM)
• Genetics and genomics, for modeling gene regulatory networks
• Speech recognition through hidden Markov models