Files
Eric Coissac 0134c69cf4
ci.yml / build (pull_request) Successful in 3m40s
Release / create-release (push) Successful in 1m7s
Release / build-macos-arm64 (push) Successful in 1m52s
Release / build-linux-x86_64 (push) Successful in 6m41s
docs: clarify VCF pseudo-alignment format and update obikmer dependency
Update the VCF export documentation to reflect that `-` represents an overlapping deletion (extra allele `*`) and introduce the `<ABS>` symbolic allele for absent families. Add a new dedicated documentation file for the VCF pseudo-alignment format, remove obsolete descriptions from phylo.md, and fix benchmark scripts to handle errors and directory creation correctly.

- Update `obikmer` dependency from 1.4.1 to 1.4.2 in Cargo.toml and Cargo.lock
- Add `UserDocMD/formats/vcf.md` describing VCF pseudo-alignment structure, alleles, and genotypes
- Remove obsolete VCF descriptions from `UserDocMD/usage/phylo.md` and update ploidy/free-loss notes
- Update `DevDocMD/theory/evolutionary_distances.md` to clarify `-` as overlapping deletion and `<ABS>` for others
- Fix `benchmark/index_one_count.sh` to create index directory and check read command errors
- Fix `benchmark/index_one_presence.sh` to create index directory and check simulation errors
- Update `src/obikphylo/src/siblings/algorithms/alignment_vcf.rs` to use `<ABS>` for absent families instead of `*`
2026-11-09 20:59:28 +11:00
..

Benchmark pipeline

Requires GNU Make ≥ 4.3 (grouped targets &:). On macOS use gmake.

gmake all          # full pipeline
gmake simulate     # simulation only
gmake reference    # reference kmer sets only

All generated and downloaded artifacts live under run/ (see Directory layout), so the whole tree is gitignored with a single benchmark/run/ entry.

Pipeline overview

flowchart TD
    GENOMES["run/genomes/*.fna.gz"]
    BIN["obikmer binary"]

    GENOMES --> simulate
    simulate --> simdata[("run/simulated_data/")]

    simdata --> reference
    reference --> refnpz[("run/reference_index/*.npz")]

    subgraph presence ["Presence track"]
        simdata  --> index_presence
        BIN      --> index_presence
        index_presence --> pres_done[("run/specimen_index_presence/")]
        index_presence --> pres_istats[("run/stats/indexing_presence/")]
        pres_istats --> aggregate_index_presence

        pres_done --> merge_presence
        BIN       --> merge_presence
        merge_presence --> gpres[("run/global_index_presence/")]

        refnpz    --> verify_presence
        pres_done --> verify_presence
        verify_presence --> vpres_stats[("run/stats/verify_presence/")]
        vpres_stats --> aggregate_verify_presence

        gpres --> filter_presence
        BIN   --> filter_presence
        filter_presence --> spec_pres[("run/specific_index_presence/")]
        filter_presence --> spec_pres_stats[("run/stats/specific_kmer_presence/")]
        spec_pres_stats --> aggregate_filter_presence

        refnpz --> verify_merge_presence
        gpres  --> verify_merge_presence
        verify_merge_presence --> vmp[("run/stats/verify_merge_presence/")]
    end

    subgraph count ["Count track"]
        simdata --> index_count
        BIN     --> index_count
        index_count --> count_done[("run/specimen_index_count/")]
        index_count --> count_istats[("run/stats/indexing_count/")]
        count_istats --> aggregate_index_count

        count_done --> merge_count
        BIN        --> merge_count
        merge_count --> gcount[("run/global_index_count/")]

        refnpz     --> verify_count
        count_done --> verify_count
        verify_count --> vcount_stats[("run/stats/verify_count/")]
        vcount_stats --> aggregate_verify_count

        gcount --> filter_count
        BIN    --> filter_count
        filter_count --> spec_count[("run/specific_index_count/")]
        filter_count --> spec_count_stats[("run/stats/specific_kmer_count/")]
        spec_count_stats --> aggregate_filter_count

        refnpz --> verify_merge_count
        gcount --> verify_merge_count
        verify_merge_count --> vmc[("run/stats/verify_merge_count/")]
    end

    subgraph query ["Query track (2 specimens: E. coli + archaeon)"]
        GENOMES --> simulate_query
        simulate_query --> qdata[("run/query_data/")]

        gpres --> pack_dense_presence
        BIN   --> pack_dense_presence
        pack_dense_presence --> gpresd[("run/global_index_presence_dense/")]

        count_done --> pack_dense_count
        BIN        --> pack_dense_count
        pack_dense_count --> gcountd[("run/global_index_count_dense/")]

        qdata  --> query_presence_dense
        gpresd --> query_presence_dense
        query_presence_dense --> qpd[("run/query_presence_dense/")]
        qpd --> aggregate_query_presence_dense

        qdata  --> query_presence_sparse
        gpres  --> query_presence_sparse
        query_presence_sparse --> qps[("run/query_presence_sparse/")]
        qps --> aggregate_query_presence_sparse

        qdata   --> query_count_dense
        gcountd --> query_count_dense
        query_count_dense --> qcd[("run/query_count_dense/")]
        qcd --> aggregate_query_count_dense

        qdata  --> query_count_sparse
        gcount --> query_count_sparse
        query_count_sparse --> qcs[("run/query_count_sparse/")]
        qcs --> aggregate_query_count_sparse

        qpd --> verify_query
        qps --> verify_query
        verify_query --> vq_stats[("run/stats/verify_query/")]
        vq_stats --> aggregate_verify_query
    end

    aggregate_verify_presence         --> all
    aggregate_verify_count            --> all
    vmp                               --> all
    vmc                               --> all
    aggregate_query_presence_dense    --> all
    aggregate_query_presence_sparse   --> all
    aggregate_query_count_dense       --> all
    aggregate_query_count_sparse      --> all
    aggregate_verify_query            --> all
    all -. "$(MAKE) re-eval" .-> aggregate_filter_presence
    all -. "$(MAKE) re-eval" .-> aggregate_filter_count

Steps

Target Script Description
simulate simulate.sh Simulate sequencing reads from the reference genomes
reference build_reference.sh Build reference kmer sets (.npz) from simulation truth
index_presence index_one_presence.sh Index each specimen (presence mode)
index_count index_one_count.sh Index each specimen (count mode)
aggregate_index_presence aggregate_stats.sh Aggregate per-specimen indexing stats (presence)
aggregate_index_count aggregate_stats.sh Aggregate per-specimen indexing stats (count)
merge_presence merge_presence.sh Merge all specimen presence indexes into a global index
merge_count merge_count.sh Merge all specimen count indexes into a global index
verify_presence verify_one_presence.sh Verify each specimen presence index against reference
verify_count verify_one_count.sh Verify each specimen count index against reference
aggregate_verify_presence aggregate_stats.sh Aggregate per-specimen verification stats (presence)
aggregate_verify_count aggregate_stats.sh Aggregate per-specimen verification stats (count)
filter_presence filter_one_presence.sh Extract species-specific presence indexes from global index
filter_count filter_one_count.sh Extract species-specific count indexes from global index
aggregate_filter_presence aggregate_stats.sh Aggregate species-specific kmer stats (presence)
aggregate_filter_count aggregate_stats.sh Aggregate species-specific kmer stats (count)
verify_merge_presence verify_merge_presence.sh Verify global presence index against all reference sets
verify_merge_count verify_merge_count.sh Verify global count index against all reference sets
simulate_query simulate_query_one.sh Simulate a fixed-size (100k pairs) read set per query specimen
pack_dense_presence pack_dense.sh presence Build global_index_presence_dense/ from global_index_presence/
pack_dense_count pack_dense.sh count Build global_index_count_dense/ from the per-specimen count sources
query_presence_dense query_one.sh presence dense Query against the dense presence global index
query_presence_sparse query_one.sh presence sparse Query against the sparse (as-merged) presence global index
query_count_dense query_one.sh count dense Query against the dense count global index
query_count_sparse query_one.sh count sparse Query against the sparse (as-merged) count global index
aggregate_query_presence_dense aggregate_stats.sh Aggregate dense presence-query wall/RSS stats
aggregate_query_presence_sparse aggregate_stats.sh Aggregate sparse presence-query wall/RSS stats
aggregate_query_count_dense aggregate_stats.sh Aggregate dense count-query wall/RSS stats
aggregate_query_count_sparse aggregate_stats.sh Aggregate sparse count-query wall/RSS stats
verify_query verify_query_one.sh Diff dense vs sparse presence-query output per specimen (regression check)
aggregate_verify_query aggregate_stats.sh Aggregate dense/sparse query regression stats

Directory layout

benchmark/
└── run/                                 # everything generated/downloaded — gitignored as a whole
    ├── genomes/                         # input reference genomes (.fna.gz), downloaded by downloads.sh
    ├── simulated_data/                  # generated by simulate
    │   └── <species>/<specimen>/
    ├── query_data/                      # generated by simulate_query (2 specimens, fixed 100k pairs)
    │   └── <species>/<specimen>/
    ├── reference_index/                 # reference kmer sets (.npz)
    ├── reference_dist/                  # reference pairwise distance matrices
    ├── obikmer_dist/                    # obikmer phylo distance matrices (presence/, count/)
    ├── specimen_index_presence/         # per-specimen presence indexes
    ├── specimen_index_count/            # per-specimen count indexes
    ├── global_index_presence/           # merged global presence index (sparse, as merged)
    ├── global_index_presence_dense/     # global presence index, dense-repacked (query benchmark)
    ├── global_index_count/              # merged global count index (sparse, as merged)
    ├── global_index_count_dense/        # global count index, dense-repacked (query benchmark)
    ├── specific_index_presence/         # species-specific presence indexes
    ├── specific_index_count/            # species-specific count indexes
    ├── query_presence_dense/            # query output against global_index_presence_dense
    ├── query_presence_sparse/           # query output against global_index_presence
    ├── query_count_dense/               # query output against global_index_count_dense
    ├── query_count_sparse/              # query output against global_index_count
    └── stats/                           # all benchmark statistics
        ├── indexing_presence/
        ├── indexing_count/
        ├── verify_presence/
        ├── verify_count/
        ├── specific_kmer_presence/
        ├── specific_kmer_count/
        ├── verify_merge_presence/
        ├── verify_merge_count/
        ├── pack_dense_presence/
        ├── pack_dense_count/
        ├── query_presence_dense/
        ├── query_presence_sparse/
        ├── query_count_dense/
        ├── query_count_sparse/
        ├── dist_comparison/
        └── verify_query/