The diff that answers
“what actually matters?”
Point wtd at a directory of configs, contracts, or JSON. It finds the consensus, ranks the drift, flags the outlier — and shows the evidence behind every claim.
curl -fsSL https://raw.githubusercontent.com/copyleftdev/whatthediff/main/install.sh | sh
$ wtd configs/ WhatTheDiff — corpus analysis Corpus: 5 artifacts · 12 distinct primitives · 34 observations Consensus universal 2 (present in all 5 artifacts) majority 5 unique 5 consensus core: 7 primitives Drift (0 = pure consensus) 0.727 configs/svc-d.yaml ⚠ OUTLIER 0.375 configs/svc-c.yaml 0.000 configs/svc-a.yaml Evidence — unique primitives configs/svc-d.yaml (4 unique) kv admin_backdoor=enabled (line 7) kv tls=false (line 5) kv db.host=10.9.9.9 (line 3)
Files in. Meaning out.
wtd never compares raw text. Every artifact is decomposed into
primitives — stable semantic facts like db.port=5432 —
each hashed with BLAKE3, so key order, whitespace, quoting and comments never register as difference.
Facts every artifact agrees on. Your ground truth.
k = NThe consensus core — held by most, missing from a few.
2k > NFactions — and wtd names them: groups sharing the same deviations, each with its signature.
1 < k, 2k ≤ NFacts found in exactly one artifact. Where outliers hide.
k = 1Evidence, not vibes.
Meaning over syntax
JSON parsed leaf-by-leaf, YAML and configs as key paths, Markdown by structure. Reordering keys or reformatting a file changes nothing in the report.
Every claim is inspectable
Each observation keeps its full occurrence list — artifact and line. The report never asserts anything you can't open a file and check.
Deterministic to the byte
No AI in the engine, no randomness, no hidden state. Same corpus in, byte-identical report out — property-tested across 565 seeded iterations.
Fast at any scale
~23 µs per file with flat, linear scaling — measured, not estimated, from 1,000 to 200,000 artifacts. Written in Zig with streaming-first design.
One small binary
~300 KB, statically linked, zero dependencies. Fully static musl builds on Linux — drop it in a container, a CI step, or an air-gapped box.
AI-ready evidence graph
wtd --json emits the full evidence graph (wtd.report.v1) —
a stable contract for pipelines, dashboards, or an LLM that explains
why a file drifted without ever inventing facts.
Measured, and honest about the ceiling.
The benchmark plants known rogues in generated corpora, then fails unless wtd flags exactly the planted set. Speed without correctness doesn't count.
| files | planted rogues | wall time | per file | detection |
|---|---|---|---|---|
| 1,000 | 20 | 0.02 s | 20 µs | exact · 0 false positives |
| 10,000 | 200 | 0.18 s | 18 µs | exact · 0 false positives |
| 50,000 | 1,000 | 0.93 s | 19 µs | exact · 0 false positives |
| 200,000 | 4,000 | 3.88 s | 19 µs | exact · 0 false positives |
| 1,000,000 | 20,000 | 21.8 s | 22 µs | exact · 0 false positives |
ReleaseFast build, 2026-07-07 (v0.5.0 streaming evidence store). File contents live in a per-artifact arena reset after each file, so resident memory scales with distinct facts, not corpus bytes — 1M files run in 3.8 GB RSS (2.56M distinct primitives, 41.8M observations). Reproduce any row with scripts/bench.sh.
It reads binaries too — like SSDeep, but it explains itself.
Point wtd at a folder of executables and every binary is cut into content-defined chunks — the same content-triggered piecewise hashing at the heart of SSDeep/CTPH. Each chunk is a primitive, so the same consensus, drift and faction engine clusters executables by shared code — with the exact chunks and byte offsets as evidence.
$ wtd ./samples --factions Factions (groups deviating from consensus in the same way) faction of 3 · cohesion 1.00 members: mathapp-v1, mathapp-v2, mathapp-v3 shared: chunk f722a9b73035213b… (3/3 members) faction of 3 · cohesion 1.00 members: textproc-v1, textproc-v2, textproc-v3 shared: chunk 2123887eae9ddcfe… (3/3 members)
Six stripped ELF binaries, source deleted —
two families of three variant builds each, clustered correctly from the bytes alone.
Unlike SSDeep's pairwise 0–100 score you get family clustering, shared-vs-unique regions,
and wtd ask "which are variants of
the same program?". ELF, PE, Mach-O and Wasm are recognized, so a lone PE among ELF
files is an outlier before chunk analysis even matters.
Same engine, pointed at secrets: --keys-only
The faction engine doesn't care what it's clustering. Point it at a folder of
.env files with --keys-only and it compares them by key schema,
not values — dropping the value from every key=value pair and hashing raw lines, so
no secret ever enters the output. It's the "which environment is missing a key,
which profiles share an auth shape?" audit, safe to run on real credentials.
$ wtd ./environments --keys-only --factions Factions (groups sharing the same key schema) faction of 3 · cohesion 1.00 members: prod.env, staging.env, dev.env shared: kv DATABASE_URL (3/3 members) shared: kv CLIENT_ID (3/3 members) shared: kv CLIENT_SECRET (3/3 members) faction of 2 · cohesion 1.00 members: db-primary.env, db-replica.env shared: kv USER_NAME (2/2 members) shared: kv PASS (2/2 members) — key names only; every value dropped before the report is built.
One line. Any OS. Checksum-verified.
Linux x86_64 · aarch64, fully static
curl -fsSL https://raw.githubusercontent.com/\ copyleftdev/whatthediff/main/install.sh | sh
Installs to /usr/local/bin or ~/.local/bin.
macOS Intel · Apple Silicon
curl -fsSL https://raw.githubusercontent.com/\ copyleftdev/whatthediff/main/install.sh | sh
Same script — it detects your OS and architecture.
Windows x86_64 · aarch64
irm https://raw.githubusercontent.com/\ copyleftdev/whatthediff/main/install.ps1 | iex
PowerShell — installs and adds wtd to your user PATH.
SHA256SUMS before installing — and refuse on mismatch.
Pin a version with WTD_VERSION=v1.0.0, or grab binaries yourself from
GitHub Releases.
Questions, answered plainly.
How is this different from diff or git diff?
Those compare two files line-by-line and answer “what changed?”. wtd compares N files by meaning and answers “what actually matters?” — consensus, drift, outliers, and the evidence for each. Key order, whitespace and comments never show up as differences.
Does it use AI?
Not in the engine. Comparison, consensus, drift and outlier detection are fully deterministic — byte-identical output for the same corpus. The optional wtd ask command sends the engine's evidence to an LLM (Anthropic API, OpenRouter, or a local model via Ollama) that explains it with file-and-line citations — it can never invent a finding, because the evidence block is all it sees.
What formats does it understand?
JSON (fully parsed), YAML-style configs, XML (attributes, entities, CDATA), PDF (text extraction, FlateDecode included), binaries/executables (SSDeep-class fuzzy analysis — clusters executables into families by shared content-defined chunks, ELF/PE/Mach-O/Wasm detection), INI/env/TOML-style key-value files, Markdown, and any text file as a fallback. Formats unify: the same fact in JSON, YAML, INI, or XML hashes to the same identity — even an XML attribute matches a JSON nested key. Malformed input degrades gracefully to line-based comparison instead of failing.
How does outlier detection work?
Every primitive held by a strict majority of files forms the consensus core. A file's drift is 1 − Jaccard(its primitives, core); anything past mean + 1.5σ is flagged. In benchmarks with planted rogues, detection is exact at every corpus size — 4,000 of 4,000 found at 200k files, zero false positives.
Can it compare credential or .env profiles safely?
Yes — --keys-only compares by
structure, not content: it drops the value from every key=value
primitive and hashes raw lines, so no secret enters the report. Point it at a folder of
.env files to find schema drift —
which environment is missing a key, which profiles share an auth shape — without exposing a
single value. Shell export KEY=… is
normalized to KEY.
Can it compare binaries or executables?
Yes — see Beyond text above. Executables are cut into content-defined chunks (the technique inside SSDeep/CTPH fuzzy hashing), so the same faction engine clusters them into families of variants. Unlike SSDeep's pairwise score you get clustering, the shared and unique regions with byte offsets, and an AI explanation. ELF, PE, Mach-O and Wasm formats and architectures are detected.
Can I trust the binaries?
Releases are built in a public GitHub Actions workflow from tagged source, and every asset ships with a SHA256SUMS file. The install scripts verify checksums before installing and refuse on mismatch. It's also ~1,900 lines of Zig — small enough to read.
What's on the roadmap?
Semantic source-code extractors. Already shipped: SSDeep-class binary/executable fuzzy analysis (v0.8.0 — content-defined chunking clusters executables into families, validated on real compiled ELF variants), the PDF text extractor (v0.7.0), the XML extractor (v0.6.0 — attributes unify with child elements), the streaming evidence store (v0.5.0 — one million files in 21.8 s and 3.8 GB RSS, detection still exact), wtd ask (v0.2.0, the AI adapter over the evidence graph), cross-format canonical unification (v0.3.0 — the same fact in JSON, YAML, or INI hashes to the same identity), and faction detection (v0.4.0 — groups deviating from consensus together, with their defining signatures).