drift

drift detects and quantifies structural, type, and distributional changes between two JSON documents. It answers the question every engineer asks when something breaks: what changed?

Not what changed in the values — what changed in the shape, types, and statistical behavior of the data.

Usage

vajra drift <baseline> <candidate> [flags]

Arguments:

Flags:

Population-Level Comparison

When --group-by is specified, drift partitions records by the field value and computes pairwise drift between all groups. Instead of comparing two documents, you compare two (or more) subpopulations within the same dataset.

vajra drift prs.ndjson --group-by '$.author_type'

Drift Report (grouped by $.author_type)
Groups: bot (412 records), human (835 records)

Pairwise drift: bot vs human
  Structural similarity: 0.91 (Jaccard)

  Distribution shifts:
    $.files_changed              JSD: 0.42 (high)
      bot:   median 1.0, p95 3.0
      human: median 4.0, p95 18.0

    $.review_comments            JSD: 0.38 (moderate)
      bot:   median 0.0, p95 1.0
      human: median 2.0, p95 8.0

  Overall severity: HIGH (significant distributional divergence)

This is useful for comparing behavioral subgroups — bot vs. human PRs, different teams, production vs. staging, before vs. after a policy change — without needing separate files.

Drift Dimensions

Structural Drift

Path set symmetric difference:

added_paths   = paths(candidate) \ paths(baseline)
removed_paths = paths(baseline) \ paths(candidate)

New fields appearing. Old fields disappearing. The most visible form of schema evolution.

Type Drift

For each path present in both documents, the dominant type is compared. Any path where the type changed (e.g., string to number, array to object) is flagged.

Distributional Drift

Jensen-Shannon Divergence (JSD) measures how much value distributions shifted between baseline and candidate:

JSD(P || Q) = 0.5 * KL(P || M) + 0.5 * KL(Q || M)

where M = 0.5 * (P + Q).

JSD is symmetric, always finite, bounded to [0, 1], and its square root is a proper metric.

For numeric paths, Vajra instead computes the 1D Wasserstein distance (earth mover’s distance), which captures how far values moved, not just that they moved.

value is not comparable across metrics — rank by effect_size

Each drift carries both a value and an effect_size, and they answer different questions.

value is in the metric’s own units. JSD is bounded to [0,1], but Wasserstein is in the units of the underlying field, so the two cannot be ranked against each other. On a real corpus, sorting by value produces this:

$ vajra drift features.ndjson --group-by '$.label' --format json --quiet \
    | jq '.pairwise_drift[0].distributional_drifts | sort_by(-.value)'
  value=1321026.74   WassersteinDistance   $[*].total_bytes
  value=20085.70     WassersteinDistance   $[*].file_bytes
  value=15075.44     WassersteinDistance   $[*].ast_nodes

total_bytes leads by six orders of magnitude for one reason only: it is measured in bytes. A boolean path that separates the two populations almost perfectly reports a value of 0.64 and sorts near the bottom.

effect_size is unit-free and bounded to [0,1], so the same list ranks usefully:

    | jq '.pairwise_drift[0].distributional_drifts | sort_by(-.effect_size)'
  effect=1.0000   value=1.0000        JSD           $[*].label
  effect=0.6435   value=0.6435        Wasserstein   $[*].has_repository
  effect=0.6352   value=17.4676       Wasserstein   $[*].pj_distinct_paths

How it is computed:

Both are 0 for identical distributions and 1 for maximal separation, so they order together. They are not the same statistic — treat effect_size as a magnitude for ranking, not as an estimate of one specific quantity. When you need the real-world size of a shift (“payloads grew by 1.3 MB”), read value.

Note that Cliff’s delta relates to the AUC of the two samples as |delta| = 2 * |AUC - 0.5|, so it is directly comparable to a separation score computed from ranks.

Thresholds are still in raw units. A path is only reported when its value exceeds a per-metric threshold (JSD > 0.05, Wasserstein > 0.1). Because the Wasserstein threshold is in field units, a byte-scale path clears it on almost any change while a small-scale path may not. effect_size fixes ranking, not filtering.

Drift Classification

Each drifted path receives a classification:

Severity Scoring

The overall drift severity is a weighted sum of drift dimensions, tuned by the active profile:

• Auditor profiles weight subtractive drift highest (missing data is critical for compliance)
• Engineer profiles weight type-mutative drift highest (breaking changes)
• Fraud profiles weight distributional drift highest (behavioral shifts)

Example: Text Output

vajra drift yesterday.json today.json

Drift Report: yesterday.json -> today.json
Structural similarity: 0.94 (Jaccard)

Added paths (2):
  $.response.metadata.processing_flags    [array of strings]
  $.response.metadata.api_version         [string]

Removed paths (0): none

Type changes (1):
  $.response.items[*].quantity            string -> number (clean type migration)

Distribution shifts (1):
  $.response.items[*].status              JSD: 0.34 (moderate)
    before: {"active": 0.82, "pending": 0.15, "error": 0.03}
    after:  {"active": 0.61, "pending": 0.12, "error": 0.27}
    note: "error" rate increased 9x

Null rate changes (0): none

Overall severity: MEDIUM (structural additions + significant distribution shift)

Example: JSON Output

vajra drift baseline.json candidate.json --format json --quiet

{
  "added_paths": [],
  "removed_paths": [],
  "type_changes": [],
  "structural_similarity": 1.0,
  "severity": "High",
  "distributional_drifts": [
    {
      "path": "$[*].latency_ms",
      "metric": "WassersteinDistance",
      "value": 188.25,
      "effect_size": 1.0
    },
    {
      "path": "$[*].status",
      "metric": "JensenShannonDivergence",
      "value": 0.75,
      "effect_size": 0.75
    }
  ]
}

Read both numbers together. latency_ms moved 188 ms (value) and the two samples do not overlap at all (effect_size 1.0). status has an effect_size equal to its value, because for categorical paths the effect size is the JSD.

With --group-by, the same structure appears once per population pair under pairwise_drift, alongside group_sizes and groups.

Example: Medical Claim Drift

vajra drift baseline_claim.json updated_claim.json --profile auditor

Drift Report: baseline_claim.json -> updated_claim.json
Structural similarity: 0.87 (Jaccard)

Added paths (3):
  $.claims[*].service_lines[*].modifier_codes     [array of strings]
  $.claims[*].rendering_provider                   [object]
  $.claims[*].rendering_provider.npi               [string]

Removed paths (1):
  $.claims[*].provider.taxonomy                    [string]
    ** SUBTRACTIVE: field present in baseline, absent in candidate **

Type changes (0): none

Distribution shifts (2):
  $.claims[*].service_lines[*].status              JSD: 0.22
    before: {"adjudicated": 0.85, "pending": 0.15}
    after:  {"adjudicated": 0.64, "pending": 0.21, "denied": 0.15}
    note: new value "denied" appeared

  $.claims[*].service_lines[*].charge_amount       Wasserstein: 125.40
    before: median 285.00, p95 890.00
    after:  median 410.00, p95 1350.00
    note: charges shifted upward

Overall severity: HIGH (subtractive drift in auditor profile)

The auditor profile flags the removed taxonomy path as high severity because subtractive drift — data that was present and is now absent — is the most dangerous form of schema evolution for compliance.

When to Use It

• API version migration. Compare the response shape before and after a deploy.
• Vendor data monitoring. Compare this week’s feed to last week’s. Detect undocumented schema changes before they break your pipeline.
• Regulatory compliance. Prove that the data structure has not drifted outside acceptable bounds.
• CI integration. Gate deploys on drift severity. If drift exceeds a threshold, fail the build and require review.

Pairs Well With

• fingerprint — quick structural same-or-different check before detailed drift analysis
• inspect — understand each document’s structure before comparing
• anomalies — drift detects changes between versions; anomalies detect deviations within a version
• essence — drift observations feed into essence generation when a baseline is provided

Comparing Two Releases: --tree

drift compares two documents. --tree answers a different question: what structurally changed between two releases of the same thing?

vajra drift ./pkg-1.0.0 ./pkg-1.0.1 --tree \
  --input-format source --lang javascript --format json --quiet

{
  "baseline_files": 61,
  "candidate_files": 61,
  "summary": { "added": 0, "removed": 0, "changed": 59, "unchanged": 2 },
  "total_node_delta": -391898,
  "files": [
    { "path": "package/lib/log.js", "change": "changed",
      "baseline_nodes": 6714, "candidate_nodes": 789, "node_delta": -5925 }
  ],
  "errors": []
}

Files are matched on their path relative to each root, so the differently-named extraction directories of two tarballs line up rather than reporting every file as both added and removed. Comparison is by structural shape, so reformatting and identifier renaming do not register — but an added branch, a new call, or an injected payload does.

Why change, not state

Every signal that reads an artifact’s presentation — does it declare a repository, does it have a description, does it look mature — is blind to a compromised established package, because a hijacked real package presents perfectly. What distinguishes it is not its state but its change: version N looked one way, version N+1 grew a payload.

Measured on two adjacent releases of a real npm package flagged as malicious, against a normal patch release of a legitimate one:

Three orders of magnitude apart. A patch release that rewrites the structure of 97% of its files is not a normal patch, whichever direction the change runs — in that case the earlier version was obfuscated and the later one was not, which is equally worth knowing.

Read both columns together: summary says how much of the release moved, total_node_delta says in which direction and by how much. A large positive delta concentrated in one or two files is the signature of an injected payload; a large delta spread across nearly every file is a build-pipeline change, such as minification or obfuscation being switched on or off.

Limits

• Only files the format selector accepts are compared — .json by default, source files under --input-format source. A payload dropped in a .sh or a binary is invisible here.
• Parse failures are reported, not hidden. Deeply nested obfuscated code can exceed the tree-sitter recursion limit; such files land in errors and are counted as changed rather than silently treated as unchanged.
• Shape equality is not semantic equality. Two files with the same shape can behave differently — a changed string literal is a value change, and this compares structure. Pair with --format json on the individual files when a shape match needs confirming.