Content addressable storage
What Is Content Addressable Storage?
Content addressable storage (CAS) is a storage paradigm in which data is identified and retrieved by a cryptographic hash derived from its content rather than by a location, filename, or path assigned by the file system. In a conventional location-addressed system, retrieving a file requires knowing where it is stored. In a content-addressed system, retrieving a file requires knowing what it contains, expressed as a fixed-length digest produced by a hash function such as SHA-256. Because identical content always produces the same hash, the identifier is intrinsic to the data itself and does not change when data is moved, copied, or replicated across systems.
CAS originates from work in distributed computing and cryptographic data structures in the 1990s, with significant theoretical grounding in Merkle trees and the concept of content-derived identifiers. It is foundational to several major modern systems: the Git version control system, widely used for software source code management, is implemented as a content-addressable object store. The InterPlanetary File System (IPFS), described in a 2014 paper by Juan Benet, extends content addressing to a global peer-to-peer network using cryptographic content identifiers (CIDs) and a distributed hash table for routing.
Hash-Based Addressing
The central mechanism in content addressable storage is the cryptographic hash function. When data is written to a CAS store, the system computes a hash of the content and uses that hash as the storage key. The properties of cryptographic hash functions that make them suitable for this purpose are determinism, collision resistance, and avalanche effect: the same input always produces the same output, different inputs with overwhelming probability produce different outputs, and a small change to the input produces an entirely different output.
SHA-256, a member of the SHA-2 family standardized by NIST, is the hash function most commonly used in production CAS implementations. Git uses SHA-1 for legacy compatibility but has been transitioning toward SHA-256 for new repositories. IPFS uses multihash encoding, which prepends the hash function identifier and digest length to the hash value, allowing different hash algorithms to coexist within the same system. The NIST guidelines on cryptographic hash standards provide the authoritative specifications for the algorithms used in these systems.
Deduplication and Integrity
Because identical content maps to a single hash, content addressable storage inherently provides data deduplication at the block or object level. Two files with identical content, regardless of their names or locations in any directory hierarchy, resolve to the same storage object. This property reduces storage consumption in systems with significant redundancy, such as backup repositories, container image registries, and code hosting platforms where many repositories share common library dependencies.
Integrity verification is a direct consequence of hash-based addressing. Retrieving a content-addressed object and recomputing its hash allows a client to confirm that the received data matches what was originally stored, without relying on the storage system to report corruption. This is particularly valuable in distributed systems where objects traverse multiple network nodes and storage devices: any modification in transit or at rest is immediately detectable.
Distributed Systems and Peer-to-Peer Applications
Content addressing is well suited to distributed systems where no central authority controls data placement. In a distributed CAS network, a node can request an object by its content hash without knowing which node holds it, and any node that possesses the object can respond. This decoupling of content identity from physical location enables the kind of resilient, redundant storage described in research on IPFS and decentralized cloud storage.
Git repositories function as local content-addressed object stores and become distributed when pushed to remote services or replicated across developer workstations. Each commit, tree, and blob object is stored under its SHA hash, and the integrity of the full history can be verified by any holder of the repository.
Applications
Content addressable storage is deployed across a range of technical domains, including:
- Version control systems for software source code, where Git uses CAS as its object model
- Container image registries, where image layers are deduplicated across repositories by content hash
- Distributed and peer-to-peer file systems, including IPFS, for censorship-resistant data distribution
- Backup and archival storage, where deduplication reduces the volume of data retained
- Content delivery and blockchain systems requiring tamper-evident data verification