Content Addressability

Law 3 requires that all artifacts are BLAKE3 hash-addressed. This chapter explains the mechanics.

ContentHash

#![allow(unused)]
fn main() {
#[derive(Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord)]
pub struct ContentHash([u8; 32]);

impl ContentHash {
    pub fn of(data: &[u8]) -> Self {
        Self(*blake3::hash(data).as_bytes())
    }

    pub fn as_bytes(&self) -> &[u8; 32] { &self.0 }
    pub fn as_hex(&self) -> String { hex::encode(self.0) }
    pub fn from_bytes(bytes: [u8; 32]) -> Self { Self(bytes) }
}
}

ContentHash is a Copy type — 32 bytes, passed by value. It implements Ord for use in BTreeMap keys.

Why BLAKE3

BLAKE3 is faster than SHA-256 with equivalent security properties. For a system that hashes every blob, every record, and every envelope, throughput matters.

Storage Layout

FileSystemBlobStore uses a git-style two-level directory structure:

blobs/
  af/
    1349b9f5f9a1a6a0404dea36dcc949...    # Full hash as filename
  c7/
    d2fe1a6b...

The first two hex characters of the hash form the directory name. This prevents any single directory from accumulating too many entries.

Structural Deduplication

When BlobStore::put() is called, it computes the BLAKE3 hash and checks if that blob already exists:

#![allow(unused)]
fn main() {
async fn put(&self, data: Bytes) -> Result<ContentHash, StorageError> {
    let hash = ContentHash::of(&data);
    if self.exists(&hash).await? {
        return Ok(hash); // Already stored — no-op
    }
    // Atomic write: temp file + rename
    self.write_blob(&hash, &data).await?;
    Ok(hash)
}
}

Identical content maps to the same hash and is stored exactly once.

Integrity Verification

Every read verifies the hash of the retrieved data:

#![allow(unused)]
fn main() {
async fn get(&self, hash: &ContentHash) -> Result<Bytes, StorageError> {
    let data = self.read_blob(hash).await?;
    let actual = ContentHash::of(&data);
    if actual != *hash {
        return Err(StorageError::IntegrityError {
            expected: *hash,
            actual,
        });
    }
    Ok(data)
}
}

Tampering is detectable by any reader at any time.

WARC Record Hashing

Every WarcRecord carries its content hash in the Palimpsest-Content-Hash header:

WARC/1.1
WARC-Type: response
Palimpsest-Content-Hash: blake3:af1349b9f5f9a1a6a0404dea36dcc949...
Content-Length: 4096

[payload bytes]

RecordId is also derived from the content hash, not from random UUIDs:

#![allow(unused)]
fn main() {
pub fn from_content(content_hash: &ContentHash, record_type: &RecordType) -> Self {
    // Deterministic UUID v5 from hash + type
}
}