Hypercore
Hypercore (formerly Dat) is a protocol for distributing and synchronizing append-only logs. It combines content addressing with mutable, real-time data streams, and is the foundation of the Holepunch ecosystem.
Append-Only Logs
A Hypercore is a single-writer append-only log:
Each entry is signed by the author's Ed25519 identity key.
A Merkle tree over entries enables efficient verification of any subset—you can prove entry #1000 is authentic without downloading entries #1-999 (sparse replication).
The log is identified by the public key; readers verify integrity without trusting the source.
Entries are immutable once appended. The log head is mutable (the latest signed entry pointer).
This structure makes Hypercore inherently offline-first: the author appends locally, and peers sync when connected.
Higher-Level Data Structures
Hyperdrive: a POSIX-like filesystem built on Hypercores. Each file change appends to the log; directory listings are derived from the log.
Hyperbee: a B-tree key-value store over a Hypercore, supporting sorted queries, range scans, and efficient diffs between versions.
Autobase: multi-writer collaboration by merging multiple single-writer Hypercores with causal ordering, approaching CRDT-like semantics. This bridges the single-writer limitation.
Hyperswarm
Hyperswarm is the networking layer (analogous to libp2p but more focused):
A Kademlia DHT specialized for topic-based peer discovery. Peers announce and lookup by topic hash.
Built-in UDP and TCP hole punching for NAT traversal, with relay fallback.
Noise protocol encryption for authenticated connections, binding each connection to a key pair.
Holepunch and Keet
Holepunch is the company/platform building on Hypercore:
Keet: a P2P video/chat/file-sharing application built entirely on Hypercore and Hyperswarm, with no servers. Demonstrates the edge-first pattern in practice: all data lives on user devices.
Pear Runtime: a platform for building P2P applications on Hypercore primitives.
Dat Legacy
Hypercore descends from the Dat protocol, originally designed for scientific data sharing and archiving. The lineage emphasizes:
Data integrity and versioning via content addressing.
Efficient sync of large datasets with sparse replication (download only what you need).
Unix-philosophy simplicity compared to IPFS's broader scope.
How Hypercore Connects to Everything Else
Hypercore's append-only logs are a foundation for State Sync (log shipping, anti-entropy, Autobase multi-writer). It shares the log-centric philosophy of SSB. Hyperswarm competes with libp2p as a P2P networking stack. Security relies on single-writer signing and Merkle verification. Privacy benefits from the offline-first model (less network exposure). Storage supports sparse replication and efficient disk layout. Compliance faces the tension between append-only logs and data erasure (though Hypercore supports truncation).
Do you like what you are reading? Subscribe to receive updates.
Unsubscribe anytime