REGENIE Parity Suite
| Status | Applies to | Owner |
|---|---|---|
| Three blocking gates | Full 1KG chromosome 22 Step 2 workflows as of 2026-07-20 | Correctness maintainers |
The parity suite compares production g results with independently generated
upstream REGENIE v4.1 outputs. Earlier g output may be used as a secondary
regression diagnostic, never as the sole oracle for a compatibility claim. A
workflow becomes release-blocking only when its metadata gate_status is
blocking.
The machine-readable record is tests/parity/golden_metadata.json. It pins the
reference version, exact commands, artifact hashes, row counts, supported native
CLI configuration, hashes for every workflow input, and statistic-specific
tolerances.
Commands
The metadata and comparison-helper tests are login-node safe:
just test-local-focused
The three release-blocking quantitative, binary score-only, and binary
approximate-Firth runs are serialized GPU workloads and belong on landau:
just slurm-gpu-test-parity-required
The required recipe selects all workflows marked parity_blocking and sets
G_REGENIE_PARITY_REQUIRE_DATA=1, so missing fixture or oracle files fail
loudly. The full-data gate does not run on GitHub-hosted runners because the
protected fixture is unavailable there.
GWAS_ENGINE_DATA_DIR can point at a fixture tree outside the repository's
ignored data/ directory. G_REGENIE_PARITY_DEVICE may override the recorded
gpu device only for a deliberate diagnostic run on an appropriate allocation.
just test-parity is useful for an explicitly requested local diagnostic: it
skips workflows with missing data. Do not run it on a login node merely because
the fixture happens to be mounted there.
Golden Workflows
The quantitative oracle was generated with:
regenie --step 1 --bed data/1kg_chr22_full --phenoFile data/pheno_cont.txt --covarFile data/covariates.txt --qt --force-step1 --bsize 1000 --out data/baselines/regenie_step1_qt
regenie --step 2 --bgen data/1kg_chr22_full.bgen --sample data/1kg_chr22_full.sample --ref-first --phenoFile data/pheno_cont.txt --covarFile data/covariates.txt --qt --bsize 400 --pred data/baselines/regenie_step1_qt_pred.list --out data/baselines/regenie_step2_qt
Its 418,943-row output is
data/baselines/regenie_step2_qt_phenotype_continuous.regenie with SHA-256
0c4782540b992d9f2163e2d1732ea0a9781e1816b23d80b8c893c3ad4ffab7b0.
The binary score-only oracle was generated on hilbert with REGENIE v4.1. The
log records a start time of Mon Jul 20 13:45:00 2026 and an end time of
Mon Jul 20 13:45:10 2026; REGENIE did not record the time zone.
regenie --step 1 --bed data/1kg_chr22_full --phenoFile data/pheno_bin.txt --covarFile data/covariates.txt --bt --cc12 --force-step1 --bsize 1000 --out data/baselines/regenie_step1
regenie --step 2 --bgen data/1kg_chr22_full.bgen --sample data/1kg_chr22_full.sample --ref-first --phenoFile data/pheno_bin.txt --covarFile data/covariates.txt --bt --cc12 --bsize 400 --pred data/baselines/regenie_step1_pred.list --out data/baselines/regenie_step2_score_only
Its 418,943-row output is
data/baselines/regenie_step2_score_only_phenotype_binary.regenie with SHA-256
ba7278541d211a8ca446f5af3d45beba06030ad40f8124651db3038c196dac33.
The pinned log SHA-256 is
c4002866c86dd67ebe23fcb563f17488635b59547cc30baa3a8566730e2e0e5b.
This workflow is release-blocking.
The binary approximate-Firth oracle was generated with:
regenie --step 1 --bed data/1kg_chr22_full --phenoFile data/pheno_bin.txt --covarFile data/covariates.txt --bt --cc12 --force-step1 --bsize 1000 --out data/baselines/regenie_step1
regenie --step 2 --bgen data/1kg_chr22_full.bgen --sample data/1kg_chr22_full.sample --ref-first --phenoFile data/pheno_bin.txt --covarFile data/covariates.txt --bt --cc12 --firth --approx --bsize 400 --pred data/baselines/regenie_step1_pred.list --out data/baselines/regenie_step2
Its 418,943-row output is
data/baselines/regenie_step2_phenotype_binary.regenie with SHA-256
0b9dc124525b6fec63e1b0d3f446263c05f690862235bd84f51b1b3c77b6ed72.
The pinned log records 17,938 corrections and zero correction failures.
Both binary gates were qualified on landau against the pinned full
chromosome oracle. The qualified source is commit
68f831f9ba51e28140b281c786555e3af6c36d4f; the pre-commit report identifies
its frozen parent 6ad69f40607c83fe95edb99d217e6195457975b1 and source-diff SHA-256
8bf2ccce77b82dc793826048a259e2f4abfba9c99235ac623a9978d23a2b8fd7.
JAX and jaxlib were 0.11.0, and the native-library SHA-256 was
8265e3ad6f5a59cf607941ec67c78f5529b56e82cf1f9caca9a8d43e129b378c.
The score-only maximum absolute differences were
8.72401046753124e-6 (BETA), 9.53237915035654e-6 (SE),
5.738945007305318e-5 (CHISQ), and 1.5566101074115934e-5
(LOG10P). Approximate-Firth maxima were 8.72401046753124e-6,
9.53237915035654e-6, 8.694763183569876e-5, and
1.5566101074115934e-5, respectively. Both workflows matched all 418,943
variant keys, sample counts, schemas, and significance classifications.
Score-only produced no corrections or failures; approximate Firth matched the
expected 17,938 corrections and zero failures. Qualification reports are
generated under ignored results/ paths and are not committed artifacts.
Before launching g, the suite verifies SHA-256 for the BGEN, Oxford sample
file, phenotype, covariates, Step-1 prediction list, and every .loco file
referenced by that list, as well as the upstream output and log. Referenced
paths must resolve within the configured data root. Approximate-Firth
correction and failure counts are parsed from the hash-pinned log; metadata is
checked against that parsed summary, and the production aggregate is compared
with the parsed values.
Comparison Contract
Production is invoked through g._core.cli.run with 16,384-variant chunks,
eight output writers, telemetry off, and direct Parquet parts. The test reads
*.run/parts/*.parquet; it does not require the removed finalization path.
Rows are joined one-to-one by the composite key
(CHROM, GENPOS, ID, ALLELE0, ALLELE1). This is required because IDs repeat in
the full chromosome fixture. Both source tables and the joined result must have
exactly 418,943 unique rows.
For finite values the assertion is strictly:
abs(g_value - regenie_value) < absolute_tolerance
NaN, positive-infinity, and negative-infinity masks must match exactly. N
must match exactly. The p < 0.05 and p < 5e-8 classifications derived from
LOG10P must also match exactly.
| Statistic | Quantitative | Binary score-only | Binary approximate-Firth |
|---|---|---|---|
BETA |
1.0e-3 |
1.0e-3 |
2.0e-3 |
SE |
1.0e-3 |
1.0e-3 |
1.0e-3 |
CHISQ |
1.5e-2 |
2.0e-2 |
3.0e-3 |
LOG10P |
1.5e-2 |
2.0e-2 |
1.0e-3 |
These tolerances are exclusive bounds. A difference equal to the tolerance fails.
Upstream REGENIE's binary result table labels every row as TEST=ADD and does
not identify successful approximate-Firth rows individually. Consequently, the
external contract can assert the exact aggregate correction/failure counts from
the pinned log, but not a per-row upstream correction mask. g method/status
labels are still checked for valid combinations and consistent failure masks.
Qualification Reports
After a completed full comparison, the suite writes one JSON artifact to:
results/parity/qualification/<workflow>/<UTC timestamp>_<process ID>.json
results/ is ignored. Set G_REGENIE_PARITY_REPORT_DIRECTORY to use another
ignored or temporary root. The report records the git commit and dirty-state
hashes, native-library and lockfile hashes, JAX/jaxlib versions, configured
device and TOML hash, input and reference hashes, individual and aggregate
Parquet hashes, output schema and column order, run-manifest/metadata hashes,
row/correction counts, and every observed maximum absolute difference with its
exclusive tolerance. Failed qualifications retain the assertion message in the
report before pytest re-raises the failure.