What Is NSL-KDD?

NSL-KDD is a benchmark data set for evaluating network intrusion detection systems, built as a cleaned and rebalanced revision of the KDD Cup 1999 data and commonly expanded in the literature as the Network Security Laboratory Knowledge Discovery in Databases set. It was released in 2009 by Mahbod Tavallaee, Ebrahim Bagheri, Wei Lu, and Ali Ghorbani at the University of New Brunswick, alongside a paper analyzing the statistical defects of the original collection. Each record describes a single network connection using 41 features plus a label, and the labels resolve either to normal traffic or to one of four attack families: denial of service, probe, remote to local, and user to root.

The data set is a supervised classification benchmark rather than a packet capture. Its features were derived from the 1998 DARPA intrusion detection evaluation traffic and fall into three groups: basic connection attributes such as duration, protocol, and byte counts; content features extracted from the payload, such as failed login attempts; and traffic features computed over a two-second window or over the last hundred connections to the same host or service.

Why the Original KDD Cup Data Needed Revision

The KDD Cup 1999 set contained enormous numbers of duplicate records, and the duplication was not evenly distributed across classes. Frequent denial of service records appeared many times over, while the rare and more damaging remote to local and user to root categories appeared only a handful of times. Any classifier trained on that distribution drifts toward the frequent classes and reports inflated accuracy while missing exactly the attacks that matter most. The NSL-KDD description from the Canadian Institute for Cybersecurity records the scale of the problem: removing duplicates cut the attack records in the training portion by more than 93 percent and reduced the test portion by roughly 75 percent overall.

Construction and Structure

Building NSL-KDD involved two operations. Duplicate records were removed from both the training and test partitions so that no single connection pattern could dominate the learned decision boundary. Then the surviving records were resampled by difficulty level, where difficulty was estimated from how many of a panel of standard learners classified each record correctly. Records that every learner handled easily were retained in inverse proportion to their frequency, which spreads the benchmark across a wider range of classification hardness. The result is a set small enough to run in full without subsampling, which removes a major source of incomparability between published results. Distributions include a standard training file, a full test file, and a harder test subset containing records that most baseline classifiers misclassified.

Use and Known Limitations

NSL-KDD became the default first benchmark for machine learning approaches to intrusion detection, appearing in evaluations of support vector machines, random forests, deep autoencoders, and recurrent networks. Comparative work such as an analysis of data mining techniques across KDD Cup 99, NSL-KDD, and UNSW-NB15 uses it as a common reference point. Its limitations are equally well documented. The underlying traffic dates from the late 1990s, so it contains none of the encrypted transport, cloud service patterns, or modern attack techniques that dominate current networks, and the synthetic background traffic differs statistically from real enterprise capture. A survey of benchmarking data sets for anomaly-based network intrusion detection catalogs successor collections including UNSW-NB15, CICIDS2017, and CSE-CIC-IDS2018 that were designed to address these gaps.

Applications

NSL-KDD has applications in a range of research and teaching contexts, including:

• Benchmarking supervised classifiers for network intrusion detection
• Evaluating anomaly detection and one-class learning methods
• Feature selection and dimensionality reduction studies
• Class imbalance and resampling research
• Adversarial machine learning and data augmentation experiments
• Coursework and laboratory exercises in network security