What Is Multi-Label Classification?

Multi-label classification, also written multilabel classification, is the supervised learning task of assigning each instance a subset of labels drawn from a fixed label set, rather than exactly one label. A news article can be tagged politics and economics at once, a photograph can contain a dog and a bicycle, and a patient record can carry several concurrent diagnoses. This differs from multi-class classification, where the classes are mutually exclusive and a prediction picks exactly one, and from multi-output problems, where several separate target variables each have their own class space.

The formal difference has practical consequences. With L labels the output space contains 2^L possible label sets, so enumerating outcomes is infeasible beyond small L. Label frequencies are usually highly imbalanced, with a few common labels and a long tail of rare ones, and labels are rarely independent: certain combinations co-occur far more often than chance would predict, while others are mutually exclusive in practice. A classification algorithm that ignores that dependence structure discards information that improves accuracy.

Problem Transformation Methods

One family converts the multi-label problem into tasks that existing single-label learners can solve. Binary relevance trains one independent binary classifier per label and unions the positive predictions. It is simple, parallelizes cleanly, and scales linearly in the number of labels, but by construction it cannot represent correlation between labels. Label powerset takes the opposite view, treating each observed label combination as a single class in a multi-class problem. It captures dependence exactly but suffers from combination sparsity, since many valid subsets never appear in training data.

Classifier chains sit between the two. The labels are ordered, and each classifier in the chain receives the original features augmented with the predicted values of all preceding labels, so dependence is modeled at close to binary relevance cost. The classifier chains method for multi-label classification introduced this construction along with an ensemble variant that averages over random label orders, which reduces sensitivity to a poorly chosen chain sequence. A later review of classifier chains and their theoretical basis analyzes the method as an approximation to the joint label distribution and examines when probabilistic and Monte Carlo inference over the chain outperform the greedy pass.

Algorithm Adaptation and Neural Approaches

A second family modifies a learning algorithm directly to emit label sets. ML-kNN adapts nearest neighbor prediction using maximum a posteriori estimation over the label counts among neighbors. Decision tree variants replace the splitting criterion with a multi-label entropy measure. Ranking approaches such as rank-SVM optimize a pairwise loss that penalizes ranking an irrelevant label above a relevant one, then apply a learned threshold to convert the ranking into a set.

Deep networks handle the task naturally by placing a sigmoid unit on each output and training under binary cross-entropy, which is binary relevance in a shared representation. Sequence models go further by generating labels one at a time conditioned on those already emitted. The sequence generation model for multi-label classification frames prediction as decoding with attention over the input, capturing correlations that an independent sigmoid layer cannot.

Evaluation

Accuracy is ambiguous when a prediction can be partly right. Subset accuracy, which demands an exact match of the whole label set, is strict and often uninformative. Hamming loss averages per-label error and is lenient toward missing rare labels. Micro-averaged F1 weights each label instance equally and is dominated by frequent labels, while macro-averaged F1 weights each label equally and exposes tail performance. Reporting several of these together is standard practice.

Applications

Multi-label classification has applications across many domains, including:

• Text categorization, document tagging, and email routing
• Image and video annotation with multiple objects or scene attributes
• Music and audio tagging by genre, mood, and instrumentation
• Functional genomics, including gene and protein function prediction
• Medical coding, where a record maps to several diagnosis codes
• Recommender systems and content moderation with overlapping categories