What Is Federated Learning?

Federated learning, also described as collaborative learning, is a machine learning setting in which many clients train a shared model under the coordination of a central server while their training data stays where it was generated. Each round, the server distributes the current model parameters, each participating client computes an update on its local data, and only those updates are returned and combined. The raw records never move. The technique was introduced by Google researchers in 2016 in a paper on communication-efficient learning of deep networks from decentralized data, which proposed the federated averaging algorithm that still anchors most deployments.

The setting differs from conventional distributed training in several ways that shape its algorithms. Client data is not independent and identically distributed, since each phone, hospital, or bank holds a sample reflecting its own users and practices. Client populations are massive and only intermittently available, participation is unreliable, and the network link is usually the scarce resource rather than compute. A widely cited survey of advances and open problems in federated learning, assembled by authors from more than two dozen institutions, catalogs these constraints and the research agenda they define.

Cross-Device and Cross-Silo Settings

Practitioners distinguish two regimes. In the cross-device setting, clients are millions of consumer endpoints such as smartphones or embedded sensors, each holding a small and highly skewed dataset, dropping in and out of training as battery and connectivity allow. Only a small fraction participates in any round, and a client typically contributes once. In the cross-silo setting, clients number in the tens or low hundreds and are institutions: hospitals, banks, manufacturers, or national statistics offices. Silo clients are reliably available, hold large local datasets, keep persistent state across rounds, and are usually bound by contract and regulation rather than by consumer privacy expectations. Algorithms tuned for one regime often perform poorly in the other, particularly with respect to how much local computation is done before an update is sent.

Aggregation and Optimization

Federated averaging has each selected client run several passes of stochastic gradient descent locally, then the server takes a weighted mean of the resulting parameters. Doing multiple local steps saves communication rounds but causes client drift when local objectives disagree, so later methods add a proximal penalty, server-side momentum, or control variates that correct for the divergence. Statistical heterogeneity also raises a fairness question, because a single global model can serve high-volume clients well and rare ones poorly. Personalization strategies respond by fine-tuning a global model locally, by learning a shared representation with per-client heads, or by clustering clients with similar distributions.

Privacy and Security

Keeping data local reduces exposure but does not by itself guarantee privacy, since model updates can leak information about the examples that produced them. Two defenses are standard. Secure aggregation, described in work on practical secure aggregation for federated learning on user-held data, uses cryptographic masking so the server learns only the sum of many client updates and never an individual one. Differential privacy adds calibrated noise and clips update norms so that no single record measurably changes the trained model. A joint NIST and United Kingdom blog series on privacy-preserving federated learning surveys how these techniques combine and where they still fall short, including poisoning attacks in which malicious clients submit crafted updates to degrade or backdoor the model.

Applications

Federated learning has applications in a range of fields, including:

• Mobile keyboards, where next-word prediction and vocabulary discovery train on typing histories held on device
• Clinical research, where hospitals build shared diagnostic models without pooling patient records
• Financial services, for fraud and anti-money-laundering models trained across institutions
• Industrial and automotive fleets, where vehicles and machines share learned behavior models
• Telecommunications and edge networks, for traffic prediction and resource allocation at base stations