Released evidence resource

OmicsEdge Evidence Atlas

Public sequencing benchmarks hold more information than a final accuracy number. The Evidence Atlas preserves the benchmark evidence itself: every run-scoped event, both of its observations, and where each one came from.

Released v1.0.0 Released 2026-10-01 · formal citation name: OmicsEdge Benchmark Evidence Registry v1.0.0

11,553benchmark eventsrun-scoped
23,106observationstruth and query side
4,278biological variantsexact identity
256evidence shardsstatic, checksummed
Release
v1.0.0 · web bundle schema 1.0.0 · canonical schema 1.0.0
Samples
HG002, HG003, HG004 (Genome in a Bottle Ashkenazi trio)
Technologies
Illumina (NovaSeq 6000) · Oxford Nanopore (PromethION R9.4)
Assembly
GRCh38 · registry coordinates 0-based half-open; source VCF positions 1-based
Variant classes
Germline SNVs and small indels
Benchmark
hap.py 0.3.15 with the vcfeval 3.12.1 engine, against GIAB v4.2.1 truth sets
Sources
Public precisionFDA Truth Challenge V2 call sets: one Illumina and one Oxford Nanopore submission per sample
Coverage
HG002: Phase 1 proof of concept on chr20 (3,602 events). HG003 and HG004: Phase 2 panel of 50 frozen 25 kb windows across 19 chromosomes (7,951 events)

Why it exists

A different question from a benchmark leaderboard.

Most variant benchmarks answer: how well did this call set perform? The Evidence Atlas asks a narrower and more inspectable question:

“What empirical benchmark evidence exists for this variant, under which conditions, and exactly where did it come from?”

Earlier descriptions of this work spoke of a future sequencing-evidence model. What has been released is the layer any such model would need first: the evidence itself, preserved without being collapsed into a score. The Atlas is a provenance-rich evidence resource. It is not a predictive model, and no predictive model has been released.

Entity model

Biological variant and benchmark event stay separate.

A biological variant can accumulate evidence from many benchmark runs without pretending those runs are the same. Each event keeps its truth-side and query-side observations exactly as the comparator recorded them.

VARIANTEVENT_VARIANT_LINKEVENTOBSERVATIONwith EXPERIMENT, BENCHMARK_RUN, SOURCE_ARTIFACT and PROVENANCE_LINK

Identity states, exactly as released

StateMeaningEvents
EXACT_NORMALIZED_ALLELEA biological variant identity was established by the frozen exact normalized-allele procedure.11,223
UNRESOLVEDEvaluated in Phase 2D, but no exact biological variant identity could be established. No identity is guessed.60
NOT_EVALUATED_FOR_CROSS_RUN_IDENTITYPreserved in Phase 1A but outside the Phase 1B cross-run candidate set. This is not the same as unresolved.270

Scientific boundaries

What v1 deliberately does not do.

These are the release's own scientific_boundary_flags, read from its frozen manifest, not disclaimers added afterwards.

  • No caller ranking caller_ranking_disabled: true
  • No clinical interpretation clinical_interpretation_disabled: true
  • No consensus generation consensus_generation_disabled: true
  • No per-variant genomic-context labels per_variant_genomic_context_disabled: true
  • No reliability scoring reliability_scoring_disabled: true
  • No representation-equivalence inference representation_equivalence_disabled: true
  • No technology ranking technology_ranking_disabled: true
  • No trust scoring trust_scoring_disabled: true
  • No predictive model, diagnosis or pathogenicity interpretation

A TP, FP or FN in the Atlas is a recorded outcome of one benchmark comparison under named conditions. It is not an OmicsEdge trust score and not an independent truth judgment. GIAB v4.2.1 is the comparator reference, not perfect biological truth.

Reproducibility and provenance

Frozen, checksummed and verifiable.

  • Sources and the Phase 2 interval panel were frozen before any benchmark outcome was inspected.
  • 20 / 20 pre-specified Phase 2 release-validation checks passed, and the release archive was independently extracted and verified. These checks validated release, protocol and data-integrity criteria; they are not clinical validation and not predictive-model performance metrics.
  • The browser bundle is a deterministic transformation of the canonical export: two independent builds are byte-identical.
  • This website serves that bundle unmodified. Unit tests verify the served bundle byte-for-byte against the source release and fail if any file differs.
  1. 01 · Source benchmark

    Truth set artifact-truth-hg003-grch38-v421

    • query call set: artifact-query-60z59-hg003
    • benchmark regions: artifact-phase2-frozen-panel
  2. 02 · Version

    hap.py 0.3.15 · vcfeval 3.12.1

    • runtime image d63b963a6cb0…
    • reference: artifact-reference-grch38-no-alt
  3. 03 · Sample

    HG003

    • experiment exp-pfda-v2-60z59-hg003
  4. 04 · Platform

    Illumina NovaSeq 6000

    • pipeline RN-Illumina · 35x
  5. 05 · Event

    chr8:3,989,216 T>C (1-based, GRCh38)

    • event-run-phase2c-hg003-60z59-grch38-v421-chr8-record-1226
    • benchmark run run-phase2c-hg003-60z59-grch38-v421
  6. 06 · Observations

    truth TP · query TP · GT 0/1 · match gm

    • obs-run-phase2c-hg003-60z59-grch38-v421-1226-truth
    • obs-run-phase2c-hg003-60z59-grch38-v421-1226-query
  7. 07 · Atlas record

    EXACT_NORMALIZED_ALLELE → biological variant

    • event-variant-link:sha256:7bb5…
    • variant:sha256:00e903d4b12125d074b…
    • 4 linked events across 2 samples and 2 technologies
A real record, end to end. The variant the frozen release selects in its own deterministic product query. Every line is read from the Evidence Atlas v1 bundle. Open it in the Evidence Explorer.

Cite and reuse

How to cite the release.

Pathak, P. (2026). OmicsEdge Benchmark Evidence Registry (Version 1.0.0) [Computer software]. Zenodo. https://doi.org/10.5281/zenodo.23085673

DOI
10.5281/zenodo.23085673
Release
GitHub release v1.0.0
Code
github.com/omicsedgebio/omicsedge-benchmark-evidence · MIT (original OmicsEdgeBio code)
Data licence
No separate licence is asserted for the evidence records; MIT covers OmicsEdgeBio code only.
Data terms
Benchmark inputs, truth resources and derived records originate from public third-party resources (GIAB, NIST, precisionFDA Truth Challenge V2) and remain subject to their original terms.
Programme
Internally tracked as OmicsEdge research Project 003; independent of the other OmicsEdge research programmes.
Technical provenance
Final archive
results/phase2e/omicsedge_phase2_final_results.tar.gz
Archive SHA-256
186cbed74fa718a728f7a24b43d58118a308165fae385eb95cc4cb4444788ca3
Canonical manifest
19b3e0f397c4dbe03c419e3da9e58f0ad865ee4faf3df5421c1163cfb113c310
Search index
5aa6736173ed1782850d8833908d9bf8a2970a3e58825e8fbf65843390cd547a
Shard index
763ac095782b5c3e9c524aa7b4bf5ce8ee2761d1045b29ec46fbe9d88e5502ca
Largest shard
504,402 bytes
Served from
/data/evidence-atlas/v1/