Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Commit & PR Conventions

Commit Messages

Format:

<type>(<scope>): <description>

<body — explains WHY, not what>

Types: feat, fix, refactor, test, docs, perf, chore

Scope: the crate name in parens: feat(frontier):, fix(envelope):, refactor(storage):

Examples:

feat(frontier): add crawl resumption via frontier save/load

Enables stopping and restarting crawls without losing state.
The frontier serializes its complete state (host queues, seen set,
politeness timestamps) to JSON and restores it on load.

fix(fetch): strip script/style content before link extraction

Link extraction was producing junk URLs from minified JavaScript.
Stripping <script> and <style> tags before scanning eliminates
false positives without affecting real links.

Breaking changes: prefix the body with BREAKING:

Special requirements:

  • Commits touching fetch/artifact/replay must include a replay fidelity test
  • Commits touching frontier/envelope must include a determinism test

Pull Requests

  • One concern per PR. No bundled drive-bys.
  • Must include tests exercising the invariant being changed
  • Benchmark before/after for performance-sensitive paths
  • cargo clippy -- -D warnings and cargo test must pass
  • New dependencies require justification in the PR description

Dependency Policy

Minimize external dependencies — every dep is attack surface.

Approved:

  • tokio — async runtime
  • reqwest/hyper — HTTP
  • serde + serde_json — serialization
  • blake3 — content hashing
  • chrono — temporal types
  • tracing — structured observability

Forbidden in core crates:

  • rand — use CrawlSeed for all randomness
  • anyhow/eyre — use typed PalimpsestError

Process:

  • Pin all versions in Cargo.lock (committed)
  • Run cargo audit before merging new deps
  • No build scripts that download or execute external code