Browser Sandbox
Isolation Model
Headless Chrome runs in a sandboxed process with strict isolation:
- No persistent storage — each page load starts from a clean browser context. No cookies, localStorage, or IndexedDB carry over between pages.
- Controlled network — the browser communicates only through the fetch engine’s controlled proxy. Direct network access is blocked.
- Disabled exfiltration — WebRTC, geolocation, notifications, and clipboard APIs are disabled to prevent data leakage.
Timeout Enforcement
Every page load has a hard timeout (default: 30 seconds). If the page does not complete loading within the timeout, the browser process is killed.
Determinism Overrides
Before any page scripts execute, Palimpsest injects JavaScript overrides seeded from CrawlSeed:
// Time is frozen and advances deterministically
Date.now = function() { return 1700000000000 + (__date_offset += 1); };
// Math.random is seeded (xorshift)
Math.random = function() { /* seeded PRNG */ };
// performance.now advances in fixed increments
performance.now = function() { return (__perf_offset += 0.1); };
This prevents JavaScript on the page from introducing non-determinism. Same seed = same execution.
CDP Stealth Mode
When stealth: true is set on BrowserFetchConfig, a comprehensive anti-detection suite is applied on top of the determinism overrides.
Chrome Launch Hardening
--disable-blink-features=AutomationControlled
--disable-component-extensions-with-background-pages
--no-first-run
--no-default-browser-check
17 Stealth Evasion Patches
All patches injected via Page.addScriptToEvaluateOnNewDocument before navigation:
| Patch | Purpose |
|---|---|
| navigator.webdriver | Set to false (configurable via WebdriverValue enum) |
| window.chrome | Full Chrome object mock (app, csi, loadTimes, runtime) |
| navigator.plugins | 3 plugins matching real Chrome |
| navigator.mimeTypes | PDF + NaCl mime types |
| navigator.permissions | Fix Notification permission inconsistency |
| navigator.languages | ["en-US", "en"] |
| navigator.hardwareConcurrency | 8 cores |
| navigator.deviceMemory | 8 GB |
| WebGL vendor/renderer | Intel UHD Graphics 630 |
| Canvas fingerprint | Seeded sub-pixel noise |
| Window/screen dimensions | Match viewport + chrome UI offset |
| AudioContext | Seeded oscillator noise |
| ClientRect | Seeded sub-pixel noise |
| sourceURL markers | Strip automation stack traces |
| navigator.userAgent | Consistent with HTTP User-Agent header |
| navigator.maxTouchPoints | 0 |
Determinism Guarantee
All noise patches (canvas, audio, ClientRect) use deterministic xorshift PRNGs with sub-seeds derived from CrawlSeed. Same seed = same noise = same fingerprint. This is Law 1 compliant.
Verified Results
Tested against 5 public bot detection sites:
- Rebrowser Bot Detector: 10/10 pass
- Sannysoft: 55/56 pass (only PluginArray prototype)
- FingerprintJS BotD: No bot verdict
- CreepJS: No hard failures
Sub-Resource Capture
Chrome DevTools Protocol (CDP) network event listeners capture all sub-resources:
Network.requestWillBeSent— records every outgoing requestNetwork.responseReceived— captures response metadataNetwork.getResponseBody— retrieves response body for each sub-resource
Each sub-resource is recorded as a separate WARC record with its own ContentHash, and the full dependency graph is stored in the resource-graph record.