Released evidence resource
OmicsEdge Evidence Atlas
Public sequencing benchmarks hold more information than a final accuracy number. The Evidence Atlas preserves the benchmark evidence itself: every run-scoped event, both of its observations, and where each one came from.
Released v1.0.0 Released 2026-10-01 · formal citation name: OmicsEdge Benchmark Evidence Registry v1.0.0
- Release
- v1.0.0 · web bundle schema 1.0.0 · canonical schema 1.0.0
- Samples
- HG002, HG003, HG004 (Genome in a Bottle Ashkenazi trio)
- Technologies
- Illumina (NovaSeq 6000) · Oxford Nanopore (PromethION R9.4)
- Assembly
- GRCh38 · registry coordinates 0-based half-open; source VCF positions 1-based
- Variant classes
- Germline SNVs and small indels
- Benchmark
- hap.py 0.3.15 with the vcfeval 3.12.1 engine, against GIAB v4.2.1 truth sets
- Sources
- Public precisionFDA Truth Challenge V2 call sets: one Illumina and one Oxford Nanopore submission per sample
- Coverage
- HG002: Phase 1 proof of concept on chr20 (3,602 events). HG003 and HG004: Phase 2 panel of 50 frozen 25 kb windows across 19 chromosomes (7,951 events)
Why it exists
A different question from a benchmark leaderboard.
Most variant benchmarks answer: how well did this call set perform? The Evidence Atlas asks a narrower and more inspectable question:
“What empirical benchmark evidence exists for this variant, under which conditions, and exactly where did it come from?”
Earlier descriptions of this work spoke of a future sequencing-evidence model. What has been released is the layer any such model would need first: the evidence itself, preserved without being collapsed into a score. The Atlas is a provenance-rich evidence resource. It is not a predictive model, and no predictive model has been released.
Entity model
Biological variant and benchmark event stay separate.
A biological variant can accumulate evidence from many benchmark runs without pretending those runs are the same. Each event keeps its truth-side and query-side observations exactly as the comparator recorded them.
Identity states, exactly as released
| State | Meaning | Events |
|---|---|---|
EXACT_NORMALIZED_ALLELE | A biological variant identity was established by the frozen exact normalized-allele procedure. | 11,223 |
UNRESOLVED | Evaluated in Phase 2D, but no exact biological variant identity could be established. No identity is guessed. | 60 |
NOT_EVALUATED_FOR_CROSS_RUN_IDENTITY | Preserved in Phase 1A but outside the Phase 1B cross-run candidate set. This is not the same as unresolved. | 270 |
Scientific boundaries
What v1 deliberately does not do.
These are the release's own scientific_boundary_flags, read from its frozen manifest, not disclaimers added afterwards.
- No caller ranking
caller_ranking_disabled: true - No clinical interpretation
clinical_interpretation_disabled: true - No consensus generation
consensus_generation_disabled: true - No per-variant genomic-context labels
per_variant_genomic_context_disabled: true - No reliability scoring
reliability_scoring_disabled: true - No representation-equivalence inference
representation_equivalence_disabled: true - No technology ranking
technology_ranking_disabled: true - No trust scoring
trust_scoring_disabled: true - No predictive model, diagnosis or pathogenicity interpretation
A TP, FP or FN in the Atlas is a recorded outcome of one benchmark comparison under named conditions. It is not an OmicsEdge trust score and not an independent truth judgment. GIAB v4.2.1 is the comparator reference, not perfect biological truth.
Reproducibility and provenance
Frozen, checksummed and verifiable.
- Sources and the Phase 2 interval panel were frozen before any benchmark outcome was inspected.
- 20 / 20 pre-specified Phase 2 release-validation checks passed, and the release archive was independently extracted and verified. These checks validated release, protocol and data-integrity criteria; they are not clinical validation and not predictive-model performance metrics.
- The browser bundle is a deterministic transformation of the canonical export: two independent builds are byte-identical.
- This website serves that bundle unmodified. Unit tests verify the served bundle byte-for-byte against the source release and fail if any file differs.
01 · Source benchmark
Truth set artifact-truth-hg003-grch38-v421
- query call set: artifact-query-60z59-hg003
- benchmark regions: artifact-phase2-frozen-panel
02 · Version
hap.py 0.3.15 · vcfeval 3.12.1
- runtime image d63b963a6cb0…
- reference: artifact-reference-grch38-no-alt
03 · Sample
HG003
- experiment exp-pfda-v2-60z59-hg003
04 · Platform
Illumina NovaSeq 6000
- pipeline RN-Illumina · 35x
05 · Event
chr8:3,989,216 T>C (1-based, GRCh38)
- event-run-phase2c-hg003-60z59-grch38-v421-chr8-record-1226
- benchmark run run-phase2c-hg003-60z59-grch38-v421
06 · Observations
truth TP · query TP · GT 0/1 · match gm
- obs-run-phase2c-hg003-60z59-grch38-v421-1226-truth
- obs-run-phase2c-hg003-60z59-grch38-v421-1226-query
07 · Atlas record
EXACT_NORMALIZED_ALLELE → biological variant
- event-variant-link:sha256:7bb5…
- variant:sha256:00e903d4b12125d074b…
- 4 linked events across 2 samples and 2 technologies
Cite and reuse
How to cite the release.
Pathak, P. (2026). OmicsEdge Benchmark Evidence Registry (Version 1.0.0) [Computer software]. Zenodo. https://doi.org/10.5281/zenodo.23085673
- DOI
- 10.5281/zenodo.23085673
- Release
- GitHub release v1.0.0
- Code
- github.com/omicsedgebio/omicsedge-benchmark-evidence · MIT (original OmicsEdgeBio code)
- Data licence
- No separate licence is asserted for the evidence records; MIT covers OmicsEdgeBio code only.
- Data terms
- Benchmark inputs, truth resources and derived records originate from public third-party resources (GIAB, NIST, precisionFDA Truth Challenge V2) and remain subject to their original terms.
- Programme
- Internally tracked as OmicsEdge research Project 003; independent of the other OmicsEdge research programmes.
Technical provenance
- Final archive
- results/phase2e/omicsedge_phase2_final_results.tar.gz
- Archive SHA-256
- 186cbed74fa718a728f7a24b43d58118a308165fae385eb95cc4cb4444788ca3
- Canonical manifest
- 19b3e0f397c4dbe03c419e3da9e58f0ad865ee4faf3df5421c1163cfb113c310
- Search index
- 5aa6736173ed1782850d8833908d9bf8a2970a3e58825e8fbf65843390cd547a
- Shard index
- 763ac095782b5c3e9c524aa7b4bf5ce8ee2761d1045b29ec46fbe9d88e5502ca
- Largest shard
- 504,402 bytes
- Served from
- /data/evidence-atlas/v1/