What Are Large Language Models?

Large language models, commonly abbreviated LLMs, are neural networks with very large parameter counts trained on broad text corpora to predict tokens in sequence, and thereby to generate, summarize, translate, and classify natural language. They belong to the wider field of artificial intelligence and grew directly out of statistical language modeling, where the task has always been to assign probabilities to sequences of words. What separates a large language model from earlier n-gram or recurrent approaches is scale: parameter counts in the billions, training corpora measured in trillions of tokens, and an architecture that parallelizes well enough to make such training practical.

The models are trained by self-supervision. No human labels are required, because the objective is derived from the text itself, usually by predicting the next token given everything before it. After this pretraining phase the model holds a broad statistical representation of language and of the facts and patterns embedded in its training data, which can then be adapted to specific tasks through fine-tuning, instruction tuning, or prompting alone.

Transformer Architecture

Nearly every current large language model rests on the transformer, introduced in the 2017 paper Attention Is All You Need, which replaced recurrence and convolution with self-attention. Self-attention computes, for each token, a weighted combination of representations of every other token in the context, with weights derived from learned query, key, and value projections. Because these operations are matrix multiplications over the whole sequence at once, training parallelizes across accelerators in a way recurrent networks never could. The cost is that attention scales quadratically with sequence length, and surveys of long-context transformer design catalog the sparse, linear, and recurrent-memory variants developed to extend usable context windows without that quadratic blowup.

Generative Pre-Trained Transformers and Scaling

The generative pre-trained transformer, or GPT, is the decoder-only family that dominates current practice. A decoder-only model uses causal masking so that each position attends only to earlier positions, which makes autoregressive generation natural at inference time. Empirical scaling relationships observed across these models tie loss to parameter count, dataset size, and training compute in a predictable way, and that predictability is what justified the enormous training runs of the past several years. Capabilities such as in-context learning, where a model performs a task from examples supplied in the prompt without any weight update, appear as model scale crosses certain thresholds rather than improving smoothly from the start.

Alignment, Evaluation, and Risk

A pretrained model predicts plausible continuations, which is not the same as being helpful, accurate, or safe. Post-training methods including supervised instruction tuning and reinforcement learning from human feedback shape the model toward preferred behavior, and retrieval augmentation grounds outputs in documents retrieved at query time rather than in parameters alone. Evaluation remains difficult, since benchmark contamination and the open-ended nature of generation both complicate measurement. Confabulation, in which a model produces fluent but false statements, is a persistent failure mode, and the NIST AI Risk Management Framework and its generative AI profile organize this and related concerns, including data privacy, information integrity, and value chain security, into categories that developers and deployers can govern, map, measure, and manage.

Applications

Large language models have applications in a range of fields, including:

• Conversational assistants and customer support automation
• Software development, including code generation, review, and migration
• Document summarization and information extraction from unstructured text
• Machine translation and cross-lingual retrieval
• Text recognition pipelines, where a model corrects and interprets noisy transcribed output
• Scientific literature search and structured data extraction from published papers
• Agent frameworks that call external tools and APIs on a user's behalf