Repeat-user stickiness
100%
across three 1M-assignment policies
Experiment-system evaluation
Multivariate assignment, event integrity, and inference calibration.
VariantGrid treats an experiment as a versioned decision contract. It combines typed multivariate design, deterministic assignment, event-integrity gates, calibrated inference, and an operator workflow that withholds conclusions when the evidence is compromised.
A/A false-positive rate
5.13%
10,000 seeded simulations at a 5% nominal level
Interval coverage
94.87%
against a 95% nominal target
Interactive system
The embedded sandbox exposes the system snapshot, deterministic assignment path, adversarial event cases, and experiment workflow.
Prefer a full-screen workspace? Open the embedded system directly.
Decision path
Each stage leaves evidence for the next one. A failed integrity check interrupts the path before an effect estimate can become a recommendation.
1. DefineHypothesis, metrics, states, constraints
2. FreezeImmutable version and allocation policy
3. ObserveAssignment, exposure, outcome events
4. GateSRM, integrity, timing, guardrails
5. DecideEffect, uncertainty, frozen record
Measured results
| Result | Observed | Scope |
|---|---|---|
| Assignment reliability | ≤0.0402 percentage-point drift; 1.458 μs median sampled p95 | Local in-process Python benchmark; hardware-sensitive |
| Inference calibration | 5.13% false positives; 94.87% interval coverage | Fixed-seed A/A simulations |
| Adversarial event integrity | Duplicate, missing, late, reordered, and cross-version cases passed | Local SQLite semantics, not distributed durability |
Methods and artifacts
Raw benchmark results Policies, trials, stickiness, drift, and latency measurements.
Assignment method Hashing contract, weighted allocation, version isolation, and parity checks.
Inference method Estimands, intervals, sequential correction, guardrails, and assumptions.
Event handling Exposure semantics, retries, ordering, version isolation, and reconciliation.
Scope and reproduction
PYTHONPATH=src python3 scripts/verify_release.py
This command stages a clean copy and runs the test, benchmark, and adversarial-integrity suites. The resulting artifacts establish deterministic local assignment, calibrated seeded simulation, adversarial SQLite event handling, and a reproducible operator workflow. They do not establish distributed durability, production uptime, customer adoption, or causal business impact.