OBIKmers/obikmer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 34 Wiki Activity

feat: enable index resumption and enforce directory creation

Browse Source 
The command now supports reopening existing indexes instead of failing when the output file exists. Control flow branches between opening an existing index and constructing a new one, moving configuration setup exclusively to the creation path. Directory existence is enforced upfront with proper I/O error propagation. The --force flag retains its original semantics by removing the target directory before proceeding with a fresh build.
This commit is contained in:
Eric Coissac
2026-08-21 05:06:38 +02:00
parent abc51c2add
commit 5c1584967f
161 changed files with 5274 additions and 845 deletions
Download Patch File Download Diff File Expand all files Collapse all files
DevDoc/architecture/index_architecture.refs/index.html
+34 -6
View File
@@ -727,14 +727,14 @@
<li class="md-nav__item">
<a href="../../implementation/obilayeredmap/" class="md-nav__link">
<a href="../../implementation/layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -972,6 +972,34 @@
<li class="md-nav__item">
<a href="../../implementation/partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
@@ -1242,10 +1270,10 @@
<h1 id="coverage-architectureindex_architecturemd">Coverage: architecture/index_architecture.md</h1>
<h2 id="code-couvert">Code couvert</h2>
<ul>
<li><code>obilayeredmap/src/layer.rs</code> — Layer<D>, trait LayerData, modes () / PersistentCompactIntMatrix / PersistentBitMatrix</li>
<li><code>obilayeredmap/src/mphf_layer.rs</code> — MphfLayer, EvidenceKind (Exact / Approx), LayerEvidence enum</li>
<li><code>obilayeredmap/src/map.rs</code> — LayeredMap<D></li>
<li><code>obilayeredmap/src/meta.rs</code> — LayerMeta, PartitionMeta</li>
<li><code>obikindex/src/layer/typed_layer.rs</code> — Layer<D>, trait LayerData, modes () / PersistentCompactIntMatrix / PersistentBitMatrix</li>
<li><code>obikindex/src/layer/mphf_layer.rs</code> — MphfLayer, EvidenceKind (Exact / Approx), LayerEvidence enum</li>
<li><code>obikindex/src/layer/map.rs</code> — LayeredMap<D></li>
<li><code>obikindex/src/layer/meta.rs</code> — LayerMeta, PartitionMeta</li>
<li><code>obikindex/src/meta.rs</code> — IndexConfig (kmer_size, n_bits, with_counts, evidence, block_bits), IndexMeta</li>
<li><code>obikindex/src/index.rs</code> — KmerIndex, build_layers</li>
<li><code>obicompactvec/src/</code> — PersistentCompactIntMatrix, PersistentBitMatrix (DataStore implementations)</li>
DevDoc/architecture/index_architecture/index.html
+30 -2
View File
@@ -731,14 +731,14 @@
<li class="md-nav__item">
<a href="../../implementation/obilayeredmap/" class="md-nav__link">
<a href="../../implementation/layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -976,6 +976,34 @@
<li class="md-nav__item">
<a href="../../implementation/partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
DevDoc/architecture/numa_partition_runner/index.html
+30 -2
View File
@@ -729,14 +729,14 @@
<li class="md-nav__item">
<a href="../../implementation/obilayeredmap/" class="md-nav__link">
<a href="../../implementation/layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -974,6 +974,34 @@
<li class="md-nav__item">
<a href="../../implementation/partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
DevDoc/architecture/numa_worker_pools/index.html
+32 -4
View File
@@ -731,14 +731,14 @@
<li class="md-nav__item">
<a href="../../implementation/obilayeredmap/" class="md-nav__link">
<a href="../../implementation/layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -976,6 +976,34 @@
<li class="md-nav__item">
<a href="../../implementation/partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
@@ -1532,7 +1560,7 @@
<td>Replace <code>available_parallelism()</code> with per-NUMA core count for spawn criterion</td>
</tr>
<tr>
<td><code>obikpartitionner/src/merge_layer.rs</code></td>
<td><code>obikindex/src/partition/merge_layer.rs</code></td>
<td>No change — <code>merge_partition</code> already works inside any Rayon context</td>
</tr>
<tr>
@@ -1540,7 +1568,7 @@
<td>No change — <code>par_iter</code> and <code>current_num_threads</code> are pool-context-aware</td>
</tr>
<tr>
<td><code>obikpartitionner/src/partition.rs</code></td>
<td><code>obikindex/src/partition/partition.rs</code></td>
<td>No change — same reason</td>
</tr>
</tbody>
DevDoc/architecture/query.refs/index.html
+31 -3
View File
@@ -727,14 +727,14 @@
<li class="md-nav__item">
<a href="../../implementation/obilayeredmap/" class="md-nav__link">
<a href="../../implementation/layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -972,6 +972,34 @@
<li class="md-nav__item">
<a href="../../implementation/partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
@@ -1243,7 +1271,7 @@
<h2 id="code-couvert">Code couvert</h2>
<ul>
<li><code>obikmer/src/cmd/query.rs</code> — commande query, format de sortie</li>
<li><code>obikpartitionner/src/query_layer.rs</code> — routage de la requête à travers les partitions</li>
<li><code>obikindex/src/partition/query_layer.rs</code> — routage de la requête à travers les partitions</li>
<li><code>obiread/src/lib.rs</code> — lecture des séquences d'entrée pour la requête</li>
</ul>
<h2 id="notes">Notes</h2>
DevDoc/architecture/query/index.html
+40 -12
View File
@@ -727,14 +727,14 @@
<li class="md-nav__item">
<a href="../../implementation/obilayeredmap/" class="md-nav__link">
<a href="../../implementation/layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -972,6 +972,34 @@
<li class="md-nav__item">
<a href="../../implementation/partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
@@ -1527,7 +1555,7 @@
</ul>
<hr />
<h2 id="algorithm">Algorithm</h2>
<p>The query follows the same superkmer-based partitioning strategy used at indexing time. Everything below happens inside <code>process_chunk</code> (<code>query.rs</code>); there is no separate per-stage function, but the internal data flow is staged: k-mer-level dereplication, a two-part MPHF/column-major matrix lookup (<code>obikpartitionner::query_partition_with</code>), and a sparse Findere pass, each producing sparse intermediate structures rather than one dense allocation for the whole chunk.</p>
<p>The query follows the same superkmer-based partitioning strategy used at indexing time. Everything below happens inside <code>process_chunk</code> (<code>query.rs</code>); there is no separate per-stage function, but the internal data flow is staged: k-mer-level dereplication, a two-part MPHF/column-major matrix lookup (<code>obikindex::partition::query_partition_with</code>), and a sparse Findere pass, each producing sparse intermediate structures rather than one dense allocation for the whole chunk.</p>
<div class="highlight"><pre><span></span><code>for each chunk of sequences (parallel workers via obipipeline, one call to process_chunk):
build QueryBatch (QueryBatch::from_records):
decompose all sequences into superkmers (SuperKmerIter) — construction only,
@@ -1602,9 +1630,9 @@
<p>The <code>-z</code> CLI option overrides the index metadata value. A higher z increases stringency (lower FP, some true positives may be discarded at sequence ends); a lower z increases sensitivity.</p>
<hr />
<h2 id="layer-lookup-mphflayerfind">Layer lookup: <code>MphfLayer::find</code></h2>
<p><code>MphfLayer::open(dir, mode: &amp;IndexMode)</code> receives the mode from <code>PartitionMeta</code> — no per-layer file is read. The caller (<code>QueryLayer</code>) never chooses the dispatch path: it is fixed at open time by <code>LayerEvidence</code>. See <a href="../../implementation/obilayeredmap/">obilayeredmap</a> for the full <code>find</code> / <code>find_strict</code> API.</p>
<p><code>MphfLayer::open(dir, mode: &amp;IndexMode)</code> receives the mode from <code>PartitionMeta</code> — no per-layer file is read. The caller (<code>QueryLayer</code>) never chooses the dispatch path: it is fixed at open time by <code>LayerEvidence</code>. See <a href="../../implementation/layer_tier/">obikindex::layer</a> for the full <code>find</code> / <code>find_strict</code> API.</p>
<h3 id="querylayer-variant-selection"><code>QueryLayer</code> variant selection</h3>
<p><code>QueryLayer::open</code> (<code>obikpartitionner/src/query_layer.rs:28-45</code>) only ever returns two variants — <code>Presence</code> or <code>Count</code>, checked in this order:</p>
<p><code>QueryLayer::open</code> (<code>obikindex/src/partition/query_layer.rs:28-45</code>) only ever returns two variants — <code>Presence</code> or <code>Count</code>, checked in this order:</p>
<table>
<thead>
<tr>
@@ -1774,7 +1802,7 @@ Fix direction: restructure <code>query</code>'s pipe with an initial <code>Flat<
<p><strong>3. Chunk-size memory formula ignores <code>n_genomes</code>.</strong>
<code>chunk_bytes = available_memory_bytes() / (n_workers * 16)</code> (<code>query.rs:407-414</code>) assumes a fixed ~8–16× overhead per raw input byte. But <code>KmerResults::new</code> (<code>query.rs:165-179</code>) allocates <code>data: Vec&lt;u32&gt;</code> sized <code>total_kmers_in_chunk × n_genomes</code> — dense, <strong>for every k-mer position in the chunk, hit or not</strong> — plus <code>win_min</code> and (with <code>--detail</code>) <code>cov</code>, same scaling. Real per-chunk memory is <code>O(n_genomes)</code>, not constant; the formula doesn't know <code>n_genomes</code> at all. This is the direct cause of the OOM kill on indexes with many reference genomes.</p>
<p><strong>4. MPHF lookup and matrix-row fetch are fused, not staged.</strong>
<code>QueryLayer::find_into</code> (<code>obikpartitionner/src/query_layer.rs:48-67</code>) does the MPHF <code>find</code> <em>and</em> the <code>fill_row</code> matrix read in one call per k-mer, inside a single-threaded loop (<code>query_partition_with</code>). There is no separation between "is this k-mer indexed" (cheap, <code>O(1)</code>, independent of <code>n_genomes</code>) and "what are its per-genome values" (the expensive, <code>n_genomes</code>-scaling part).</p>
<code>QueryLayer::find_into</code> (<code>obikindex/src/partition/query_layer.rs:48-67</code>) does the MPHF <code>find</code> <em>and</em> the <code>fill_row</code> matrix read in one call per k-mer, inside a single-threaded loop (<code>query_partition_with</code>). There is no separation between "is this k-mer indexed" (cheap, <code>O(1)</code>, independent of <code>n_genomes</code>) and "what are its per-genome values" (the expensive, <code>n_genomes</code>-scaling part).</p>
<p><strong>5. Dereplication should happen at k-mer granularity, directly — not via an intermediate superkmer-level dedup.</strong>
<code>QueryBatch::from_records</code> currently dereplicates at the <em>superkmer</em> level (<code>HashMap&lt;RoutableSuperKmer, Vec&lt;SKDesc&gt;&gt;</code>, <code>query.rs:112</code>). This misses redundancy between k-mers shared by <em>different</em> superkmers (read overlaps, repeats, a SNP splitting an otherwise-identical run). Superkmer <em>construction</em> (<code>SuperKmerIter</code>) stays mandatory — it is the mechanism that computes minimizers/partition routing, not an optional dedup layer — but the dedup structure built on top of it should key directly on <code>CanonicalKmer</code>, in the same pass: <code>HashMap&lt;CanonicalKmer, Vec&lt;(seq_idx, pos)&gt;&gt;</code>. This also means the MPHF <code>find</code> itself runs once per <strong>distinct</strong> k-mer instead of once per occurrence — a win independent of the matrix-fetch cost below.</p>
<p><strong>6. Stage 1 output: bucket confirmed hits by layer, keyed by MPHF slot.</strong>
@@ -1806,12 +1834,12 @@ Stage 2's output should be <code>HashMap&lt;genome_idx, Vec&lt;(seq_idx, positio
<p>None of this is implemented yet — parked here as a coherent roadmap while the design is discussed further. Suggested dependency order: (1) I/O parallelism → (3) genome-aware chunk sizing → (4)–(9) staged/k-mer-deduped/NUMA-aware-partition-and-column-major/sparse query engine (larger refactor, biggest structural payoff — reuses <code>PartitionRunner</code> rather than inventing a new parallelism mechanism) → (2) parallel gzip (separate, orthogonal, tracked in chunkreader.md) → secondary diagnostics patterns.</p>
<hr />
<h2 id="implementation-plan">Implementation plan</h2>
<p>Concrete, phased translation of the roadmap above. Phases 0–2 are small, independent, low-risk, and each individually testable against current <code>query</code> output — land them first, in order, and measure on the reference 192-core/8-NUMA machine before deciding whether phases 3–5 (the staged/sparse engine, the larger structural payoff) are still worth their cost. Phases 3–5 are one coordinated change spanning <code>obikmer</code>, <code>obikpartitionner</code>, and <code>obicompactvec</code> — they should not be split across releases mid-way, because the intermediate state (e.g. k-mer-level dedup feeding the old dense <code>KmerResults</code>) has no correctness or performance benefit on its own. Phase 6 is unrelated to phases 0–5 and can happen any time, independently, if <code>rapidgzip-rs</code> is validated (see <a href="../../implementation/chunkreader/#future-work--parallel-gzip-decompression-in-xopen">chunkreader.md</a>).</p>
<p>Concrete, phased translation of the roadmap above. Phases 0–2 are small, independent, low-risk, and each individually testable against current <code>query</code> output — land them first, in order, and measure on the reference 192-core/8-NUMA machine before deciding whether phases 3–5 (the staged/sparse engine, the larger structural payoff) are still worth their cost. Phases 3–5 are one coordinated change spanning <code>obikmer</code>, <code>obikindex::partition</code>, and <code>obicompactvec</code> — they should not be split across releases mid-way, because the intermediate state (e.g. k-mer-level dedup feeding the old dense <code>KmerResults</code>) has no correctness or performance benefit on its own. Phase 6 is unrelated to phases 0–5 and can happen any time, independently, if <code>rapidgzip-rs</code> is validated (see <a href="../../implementation/chunkreader/#future-work--parallel-gzip-decompression-in-xopen">chunkreader.md</a>).</p>
<p>Instrumentation is deliberately sequenced <em>before</em> the I/O fix (reordering the roadmap's own listed order), because every later phase's justification rests on a measurement ("to be measured, not assumed" appears throughout the roadmap above) — without it, phases 3–5 would be undertaken on faith.</p>
<p>Performance measurement on the reference 192-core/8-NUMA machine is done by the project owner, not from this development environment (macOS, 16 cores — <code>PartitionRunner</code>'s NUMA pinning is Linux-only, so even phase 4's mechanism can't be functionally exercised for its actual purpose here). Each phase below is therefore written to be <em>self-measuring</em>: the debug-level logging it adds must be enough, on its own, to judge whether that phase's algorithmic choice paid off from a cluster run's logs, without needing to attach a profiler.</p>
<h3 id="conventions-applied-to-every-phase-below">Conventions applied to every phase below</h3>
<p><strong>Debug logging.</strong> Every phase that changes an algorithmic choice (not phase 0, which <em>is</em> the logging) adds <code>tracing::debug!</code>/<code>trace!</code> at points that let a cluster run's logs answer "did this help": counts, ratios, and timings that quantify the specific claim that phase makes — e.g. phase 3 must log how many MPHF <code>find</code> calls were saved by k-mer-level dedup (the whole justification for that phase), phase 4 must log per-column scan timings, phase 5 must log actual retained-memory / sparsity ratios achieved. Prefer one structured <code>debug!</code> per chunk (fields, not prose) over free-text — the cluster logs will be the only evidence available for judging these choices, so they need to be grep/awk-able, not just readable.</p>
<p><strong>Unit tests.</strong> This project's convention (<code>obiread</code>, <code>obikseq</code>, <code>obidebruinj</code>, <code>obicompactvec</code>, <code>obilayeredmap</code>, <code>obiskio</code>, <code>obifastwrite</code>) is <code>#[cfg(test)] #[path = "tests/&lt;name&gt;.rs"] mod tests;</code> at the bottom of the source file, with the actual test code in a sibling <code>src/tests/&lt;name&gt;.rs</code>. Neither <code>obikmer</code> nor <code>obikpartitionner</code> (the two crates phases 3 and 5 touch most) currently have a <code>src/tests/</code> directory at all — this needs creating, following the existing pattern exactly, not inventing a new one.</p>
<p><strong>Unit tests.</strong> This project's convention (<code>obiread</code>, <code>obikseq</code>, <code>obidebruinj</code>, <code>obicompactvec</code>, <code>obikindex::layer</code>, <code>obiskio</code>, <code>obifastwrite</code>) is <code>#[cfg(test)] #[path = "tests/&lt;name&gt;.rs"] mod tests;</code> at the bottom of the source file, with the actual test code in a sibling <code>src/tests/&lt;name&gt;.rs</code>. Neither <code>obikmer</code> nor <code>obikindex::partition</code> (the two crates phases 3 and 5 touch most) currently have a <code>src/tests/</code> directory at all — this needs creating, following the existing pattern exactly, not inventing a new one.</p>
<p><strong>Workflow (<code>jj</code>).</strong> Work happens in a fresh <code>jj</code> commit, easy to abandon. <code>jj new</code> between phases is reasonable where it helps isolate a phase for review, but only when the working copy compiles at that point (project convention) — phase 3's internal sub-steps (batch dedup change, then <code>query_layer.rs</code> split, then the new return shape) will likely not each compile independently since they're one coupled change, so treat "commit boundary" and "plan phase boundary" as related but not forced to match 1:1; use judgement per phase rather than mechanically splitting on every bullet.</p>
<h3 id="phase-0-instrumentation-prerequisite-for-measuring-every-later-phase">Phase 0 — Instrumentation (prerequisite for measuring every later phase)</h3>
<p><strong>Goal</strong>: make core utilization, throughput, and per-stage timing visible on a real run, so phases 1–5 can be justified with numbers instead of assumption.</p>
@@ -1854,26 +1882,26 @@ Stage 2's output should be <code>HashMap&lt;genome_idx, Vec&lt;(seq_idx, positio
<li><code>obikmer/src/cmd/query.rs</code>:</li>
<li>Replace <code>QueryBatch::from_records</code>'s dedup map (<code>HashMap&lt;RoutableSuperKmer, Vec&lt;SKDesc&gt;&gt;</code>, current <code>query.rs:112</code>) with a per-partition <code>HashMap&lt;CanonicalKmer, Vec&lt;(seq_idx: u32, pos: u32)&gt;&gt;</code>, built in the same <code>SuperKmerIter</code> pass: superkmer construction and partition routing (<code>part_idx</code> from the superkmer's minimizer hash) are unchanged, only the granularity of what gets deduplicated changes — each <code>CanonicalKmer</code> within a superkmer is inserted individually instead of the whole superkmer being the dedup key.</li>
<li><strong>Verified</strong>: <code>CanonicalKmer</code> (<code>obikseq/src/kmer.rs:390</code>, <code>pub type CanonicalKmer = CanonicalKmerOf&lt;KLen&gt;</code>) — the underlying <code>CanonicalKmerOf&lt;L&gt;</code> derives <code>Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash</code> (<code>kmer.rs:269</code>). Usable as a <code>HashMap</code>/<code>HashSet</code> key as-is, no change needed.</li>
<li><code>obikpartitionner/src/query_layer.rs</code>:</li>
<li><code>obikindex/src/partition/query_layer.rs</code>:</li>
<li>Split <code>QueryLayer::find_into</code> (<code>query_layer.rs:48-67</code>) into two methods: <code>find_slot(&amp;self, kmer: CanonicalKmer) -&gt; Option&lt;usize&gt;</code> (MPHF only, no matrix touch) and keep <code>fill_row</code> as-is for phase 4 to call later.</li>
<li>Replace <code>query_partition_with</code>'s inner loop (<code>query_layer.rs:103-113</code>) with a version that, for each unique <code>CanonicalKmer</code>, calls <code>find_slot</code> across the partition's layers (stopping at first hit, same as today), and instead of immediately filling a row, records <code>(layer_idx, slot)</code>.</li>
<li>New return shape for the partition-level query, replacing today's <code>on_hit(sk_idx, kmer_idx, row)</code> callback: <code>HashMap&lt;layer_idx, HashMap&lt;slot, Vec&lt;(seq_idx, pos)&gt;&gt;&gt;</code> (roadmap point 6) — built directly from the k-mer dedup map's <code>Vec&lt;(seq_idx,pos)&gt;</code> values, keyed by the resolved slot instead of the k-mer.</li>
<li><strong>This phase alone has no throughput benefit yet</strong> (matrix fetch still happens, just deferred) beyond the k-mer-level dedup itself (fewer MPHF calls when queries have overlapping/repeated k-mers) — its purpose is to produce the input phase 4 needs. Land phase 3+4 together, not phase 3 alone, per the "don't split 3–5 across releases" note above.</li>
<li>Log, per chunk: total k-mer occurrences vs. unique <code>CanonicalKmer</code> count (the dedup ratio — the entire justification for this phase) and the resulting MPHF <code>find</code> call count. If the dedup ratio is close to <code>1.0</code> on real query data (little redundancy), that's the cluster run telling us this phase wasn't worth it — the logging needs to be able to say that, not just confirm the happy path.</li>
<li><strong>Unit tests</strong>: create <code>obikmer/src/cmd/tests/query.rs</code> (new <code>src/tests/</code> dir for this crate, following the project's <code>#[cfg(test)] #[path = "tests/query.rs"] mod tests;</code> convention) and <code>obikpartitionner/src/tests/query_layer.rs</code> (likewise new for this crate). Cover: the k-mer-level dedup map construction on synthetic sequences with known repeated/overlapping k-mers (assert unique-kmer count and occurrence lists); the <code>find_slot</code>/bucket-by-layer-and-slot construction against a small hand-built <code>QueryLayer</code> fixture, asserting the <code>(layer_idx, slot, seq_idx, pos)</code> tuples match what the old per-occurrence loop would have produced.</li>
<li><strong>Unit tests</strong>: create <code>obikmer/src/cmd/tests/query.rs</code> (new <code>src/tests/</code> dir for this crate, following the project's <code>#[cfg(test)] #[path = "tests/query.rs"] mod tests;</code> convention) and <code>obikindex/src/partition/tests/query_layer.rs</code> (likewise new for this crate). Cover: the k-mer-level dedup map construction on synthetic sequences with known repeated/overlapping k-mers (assert unique-kmer count and occurrence lists); the <code>find_slot</code>/bucket-by-layer-and-slot construction against a small hand-built <code>QueryLayer</code> fixture, asserting the <code>(layer_idx, slot, seq_idx, pos)</code> tuples match what the old per-occurrence loop would have produced.</li>
</ul>
<h3 id="phase-4-column-major-matrix-fetch-roadmap-points-78-implemented-numa-parallelism-deferred">Phase 4 — Column-major matrix fetch (roadmap points 7–8) — implemented, NUMA parallelism deferred</h3>
<p><strong>Goal (revised during implementation)</strong>: replace <code>fill_row</code>-per-hit (row-major, worst-case mmap locality) with a column-major scan. <code>PartitionRunner</code> turned out to be the wrong mechanism for this at this call granularity — see below; the column-major fetch itself is implemented and validated, without it.</p>
<p><strong>What shipped:</strong>
- <code>obicompactvec</code>: the per-column accessors this phase needed <strong>already existed</strong> — <code>PersistentCompactIntMatrix::col_view(c)</code> and <code>PersistentBitMatrix::col_view(c)</code> are public, and <code>IntSliceView::get(slot)</code>/<code>BitSliceView::get(slot)</code> are public — the original plan underestimated how much of this plumbing the pairwise-distance code (<code>dump</code>/<code>select</code>/<code>stats</code>) had already required. The one real gap: <code>PersistentBitMatrix::col_view()</code> panics on the <code>Implicit</code> variant (the documented mono-genome fast path, <code>bitmatrix.rs</code>). Added <code>PersistentBitMatrix::get(c, slot) -&gt; u32</code> (<code>bitmatrix.rs</code>), a non-panicking column-major point lookup that returns <code>1</code> for <code>Implicit</code> regardless of <code>c</code> — the smallest surface needed, not a new <code>col_get</code> API from scratch.
- <code>obikpartitionner/src/query_layer.rs</code>: <code>query_partition_with</code> is now two explicit stages, matching roadmap points 6–8: <strong>stage 1</strong> (MPHF-only, per unique k-mer, bucket hits by <code>(layer_idx, slot)</code>, emits <code>QueryHit::Found</code>) then <strong>stage 2</strong> (per layer with ≥1 hit, column-major: for each genome column <code>g</code> in <code>0..layer.n_cols().min(n_genomes)</code>, scan that layer's bucketed slots and call <code>col_value(g, slot)</code>, emitting <code>QueryHit::Value(descs, g, value)</code> on nonzero). <code>QueryHit</code> is a single enum delivered through one <code>FnMut(QueryHit)</code> callback — an earlier two-closure design (<code>on_found</code> + <code>on_value</code>) didn't borrow-check, since the caller's single mutable accumulator (<code>KmerResults</code>) can't be captured by two separate <code>FnMut</code> closures passed to the same call.
- <code>obikindex/src/partition/query_layer.rs</code>: <code>query_partition_with</code> is now two explicit stages, matching roadmap points 6–8: <strong>stage 1</strong> (MPHF-only, per unique k-mer, bucket hits by <code>(layer_idx, slot)</code>, emits <code>QueryHit::Found</code>) then <strong>stage 2</strong> (per layer with ≥1 hit, column-major: for each genome column <code>g</code> in <code>0..layer.n_cols().min(n_genomes)</code>, scan that layer's bucketed slots and call <code>col_value(g, slot)</code>, emitting <code>QueryHit::Value(descs, g, value)</code> on nonzero). <code>QueryHit</code> is a single enum delivered through one <code>FnMut(QueryHit)</code> callback — an earlier two-closure design (<code>on_found</code> + <code>on_value</code>) didn't borrow-check, since the caller's single mutable accumulator (<code>KmerResults</code>) can't be captured by two separate <code>FnMut</code> closures passed to the same call.
- <code>obikmer/src/cmd/query.rs</code>: <code>KmerResults::set</code> (row-major, whole-row-at-once) replaced by <code>mark_found</code> (stage 1: flag a position as indexed, independent of any genome's value) and <code>set_one</code> (stage 2: write one genome's value at one position). <code>QueryStats</code> extended with <code>n_columns_scanned</code>/<code>n_col_get_calls</code>, logged per chunk.
- Total <code>get()</code>-equivalent calls are unchanged from the row-major version (<code>n_hits × n_cols</code> in the worst case, confirmed by <code>n_col_get_calls</code> in the debug log) — the win is locality (sequential access within one layer's column at a time, across <code>mmap</code>'d regions, instead of jumping across all columns per hit), exactly as predicted.</p>
<p><strong>What did not ship, and why — <code>PartitionRunner</code> is architecturally the wrong tool here:</strong>
Reading <code>obikindex/src/numa.rs</code>'s actual <code>run()</code> body (not just its doc comments) shows every call spawns a timer thread <strong>plus one OS thread per worker slot on every NUMA node</strong> (<code>std::thread::scope</code> + one <code>s.spawn()</code> per node per <code>max_workers</code>) — on the 192-core/8-NUMA reference machine, that's on the order of 190+ fresh OS threads spawned <strong>per call</strong>. This is fine for its actual, established usage in this codebase (<code>merge.rs</code>, <code>index.rs</code>'s <code>build_layers</code>): one <code>PartitionRunner::new()</code> + one <code>run()</code> call per command invocation, amortised over a batch of ~256 long-running partitions. It is not fine for <code>query</code>'s call pattern: <code>query_partition_with</code> runs once per <code>(chunk, partition)</code>, potentially thousands of times per second — spawning ~190 OS threads that often to scan a handful of genome columns would very likely cost far more than the row-major approach it's meant to replace. This is exactly the "resolve empirically, don't assume" composition risk the roadmap flagged, just resolved by reading the mechanism's actual cost before wiring it in, rather than by measuring a regression on the cluster after the fact.
The column-major loop in stage 2 is therefore a <strong>plain sequential loop</strong> for now — it captures the whole, provable locality win (roadmap point 8's actual claim) without adding any parallelism mechanism. Genome-column-level parallelism (point 8's "bonus" axis) and partition-level parallelism (point 7) are both deferred — not abandoned. Candidates for a follow-up, once there's a concrete profiling need: (a) <code>rayon</code>'s already-warm global pool (<code>into_par_iter()</code>) for the column axis specifically — cheap to invoke repeatedly since it doesn't spawn threads per call, though it's the same "naive rayon" pattern <code>numa_worker_pools.md</code> warns about for a <em>different</em> workload (random pointer-chasing over large hash maps); a column scan's access pattern (sequential reads within one <code>mmap</code>'d region) has a different contention profile and hasn't been shown to have the same problem — needs its own measurement, not an assumption either way; (b) restructuring so <code>PartitionRunner</code> is invoked once per whole <code>query</code> run (or per large batch of chunks) rather than per <code>(chunk, partition)</code>, amortising its spawn cost the way <code>merge</code>/<code>build_layers</code> do — a bigger structural change than this phase's scope.
- Log (implemented): <code>QueryStats::n_columns_scanned</code>/<code>n_col_get_calls</code>, folded into the existing per-chunk <code>debug!("k-mer dedup + column-major fetch", ...)</code> line (<code>query.rs</code>) alongside phase 3's dedup counters.
- <strong>Unit tests</strong>: extended <code>obikpartitionner/src/tests/query_layer.rs</code> (phase 3's file) — <code>query_partition_with</code>'s empty/missing-index paths updated for the new <code>QueryStats</code> fields and single-callback signature.
- <strong>Unit tests</strong>: extended <code>obikindex/src/partition/tests/query_layer.rs</code> (phase 3's file) — <code>query_partition_with</code>'s empty/missing-index paths updated for the new <code>QueryStats</code> fields and single-callback signature.
- <strong>Validation performed</strong>: full workspace build + <code>cargo test --workspace</code>, zero failures. Functional validation against real indexes: (1) a single-genome index — output byte-identical to pre-phase-4 (same <code>kmer_count</code>/<code>kmer_strict_matches</code> on every record); (2) the existing 20-genome <code>benchmark/global_index_presence</code> index — runs correctly, <code>n_hits=0</code> for an unrelated query (expected: no shared k-mers between a plant read and a bacterial reference set), no panics, confirming the <code>Implicit</code>/multi-column bounds logic doesn't crash on a real multi-genome, mixed-format index; (3) <strong>the critical correctness case</strong>: built two single-sequence-pair test genomes, merged into one 2-genome index, queried with reads from both — reads from <code>genomeA</code> matched <strong>only</strong> <code>genomeA</code> (<code>kmer_count</code> identical to the pre-dedup occurrence count, zero leakage into <code>genomeB</code>'s column) and vice versa. This is the test that would have caught a column-index mixup, an off-by-one in <code>n_cols</code>, or cross-genome bleed from the stage-1/stage-2 split — it passed cleanly.
- <strong>Not yet done</strong>: the microbenchmark comparing column-major vs. the old row-major access pattern's wall time / page-fault counters on a large-<code>n_genomes</code> layer — needs a realistically large multi-genome index and, for the page-fault counters specifically, Linux (not available from this development environment). Left for cluster validation alongside phases 1–3's own pending measurements.</p>
<h3 id="phase-5-sparse-findere-rework-roadmap-point-9">Phase 5 — Sparse Findere rework (roadmap point 9)</h3>
DevDoc/architecture/rebuild_filter/index.html
+32 -4
View File
@@ -727,14 +727,14 @@
<li class="md-nav__item">
<a href="../../implementation/obilayeredmap/" class="md-nav__link">
<a href="../../implementation/layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -972,6 +972,34 @@
<li class="md-nav__item">
<a href="../../implementation/partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
@@ -1417,9 +1445,9 @@ IO pattern: sequential read of each source column → random write into new colu
</table>
<h2 id="files-to-modify">Files to modify</h2>
<ul>
<li><code>src/obikpartitionner/src/rebuild_layer.rs</code> — <code>rebuild_partition</code> and <code>iter_src_layers</code></li>
<li><code>src/obikindex/src/partition/rebuild_layer.rs</code> — <code>rebuild_partition</code> and <code>iter_src_layers</code></li>
<li>Possibly <code>src/obicompactvec/</code> — add column iterator API if not already present</li>
<li><code>src/obilayeredmap/</code> — check if per-column sequential access is exposed on <code>SrcLayerData</code></li>
<li><code>src/obikindex/src/layer/</code> — check if per-column sequential access is exposed on <code>SrcLayerData</code></li>
</ul>
<h2 id="open-questions">Open questions</h2>
<ul>
DevDoc/architecture/sequences/invariant.refs/index.html
+30 -2
View File
@@ -727,14 +727,14 @@
<li class="md-nav__item">
<a href="../../../implementation/obilayeredmap/" class="md-nav__link">
<a href="../../../implementation/layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -972,6 +972,34 @@
<li class="md-nav__item">
<a href="../../../implementation/partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
DevDoc/architecture/sequences/invariant/index.html
+31 -3
View File
@@ -9,7 +9,7 @@
<link rel="prev" href="../../../implementation/benchmark_query_testing/">
<link rel="prev" href="../../../implementation/partition_layer_cache/">
<link rel="next" href="../../index_architecture/">
@@ -731,14 +731,14 @@
<li class="md-nav__item">
<a href="../../../implementation/obilayeredmap/" class="md-nav__link">
<a href="../../../implementation/layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -976,6 +976,34 @@
<li class="md-nav__item">
<a href="../../../implementation/partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
DevDoc/architecture/siblings/index.html
+176 -40
View File
@@ -731,14 +731,14 @@
<li class="md-nav__item">
<a href="../../implementation/obilayeredmap/" class="md-nav__link">
<a href="../../implementation/layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -976,6 +976,34 @@
<li class="md-nav__item">
<a href="../../implementation/partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
@@ -1328,6 +1356,17 @@
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#implemented-2026-08-20" class="md-nav__link">
<span class="md-ellipsis">
Implemented (2026-08-20)
</span>
</a>
</li>
</ul>
@@ -1602,6 +1641,17 @@
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#implemented-2026-08-20" class="md-nav__link">
<span class="md-ellipsis">
Implemented (2026-08-20)
</span>
</a>
</li>
</ul>
@@ -1637,7 +1687,7 @@ one-way function, not an invertible bijection with a stored inverse. Any
method that reconstructs a kmer from a bare slot number is wrong by
construction, regardless of the mechanism used (MPHF re-hash, or evidence
decode + direct unitig read). See <code>MphfLayer::kmer_at</code>
(<code>obilayeredmap/src/mphf_layer.rs</code>) — flagged for removal, currently called
(<code>obikindex/src/layer/mphf_layer.rs</code>) — flagged for removal, currently called
from <code>obikphylo/siblings/build.rs</code> and <code>family_scan.rs</code> (since removed — see
"Pending work" status below).</p>
<h2 id="two-pipelines-never-mixed">Two pipelines, never mixed</h2>
@@ -1700,7 +1750,7 @@ partition is unknown) and must keep going through
<code>index()</code>.</p>
<h2 id="pending-work-done">Pending work — done</h2>
<p>The plan above shipped: <code>obikphylo</code> (a new crate — phylo-domain extension
traits over <code>obikindex::KmerIndex</code>/<code>obilayeredmap::Layer&lt;D&gt;</code>, replacing the
traits over <code>obikindex::KmerIndex</code>/<code>obikindex::layer::Layer&lt;D&gt;</code>, replacing the
old <code>obikindex::siblings</code> module) builds and reads the annex purely in
iteration order (<code>SiblingLayerExt::iter_siblings</code>/<code>iter_minorants</code>, both with
batch variants, mirroring <code>Layer&lt;D&gt;</code>'s own <code>KmerIter</code>/<code>KmerBatchIter</code>
@@ -2268,13 +2318,13 @@ BRWT-style column-correlation exploitation.</p>
<p><code>PersistentSparseBitMatrix</code> went from a validated but unused type to a
real, selectable on-disk format:</p>
<ul>
<li><strong>Generic <code>Layer&lt;D&gt;</code></strong>: <code>obilayeredmap::Layer&lt;D&gt;</code>'s presence-only methods
<li><strong>Generic <code>Layer&lt;D&gt;</code></strong>: <code>obikindex::layer::Layer&lt;D&gt;</code>'s presence-only methods
(<code>n_cols</code>, <code>sub_matrix</code>, <code>fill_sub_matrix</code>) are generic over any
<code>D: LayerData&lt;Item = Box&lt;[bool]&gt;&gt; + BinaryMatrix</code>, not hardcoded to
<code>PersistentBitMatrix</code> — <code>PersistentSparseBitMatrix</code> implements
<code>LayerData</code> (<code>open</code>/<code>read</code>) the same way. <code>find_slot</code>/<code>index_batch</code> were
already generic over any <code>D: LayerData</code>, so they needed no change.
Verified by <code>obilayeredmap</code>'s
Verified by <code>obikindex::layer</code>'s
<code>presence_layer_generic_over_sparse_matches_dense</code> test: build a dense
presence layer, convert it to sparse via <code>build_from_dense</code>, open both
as <code>Layer&lt;PersistentBitMatrix&gt;</code>/<code>Layer&lt;PersistentSparseBitMatrix&gt;</code> on
@@ -2284,7 +2334,7 @@ real, selectable on-disk format:</p>
<code>AtomicUsize</code>s in test builds, not thread-local, so a test using a
different <code>k</code> races every other test in the same crate binary; a k=11
version of this test passed alone but failed under the full
<code>obilayeredmap</code> suite for exactly that reason before being fixed.)</li>
<code>obikindex::layer</code> suite for exactly that reason before being fixed.)</li>
<li><strong><code>obikphylo::siblings::cache::Mat</code></strong> gained a third variant,
<code>SparsePresence(Layer&lt;PersistentSparseBitMatrix&gt;)</code>, alongside <code>Count</code>
and <code>Presence</code> — every method (<code>find_slot</code>, <code>index_batch</code>,
@@ -2314,7 +2364,7 @@ real, selectable on-disk format:</p>
dense-path test (<code>sibling_annex_one_sibling_each</code>) exactly — proves the
sparse format round-trips through the real build pipeline
(<code>PartitionCache</code> sparse-detection included), not just the
<code>obicompactvec</code>/<code>obilayeredmap</code> unit layers below it.</li>
<code>obicompactvec</code>/<code>obikindex::layer</code> unit layers below it.</li>
</ul>
<p>Full workspace <code>cargo test</code> (all crates, unit + doc tests) green after
this change.</p>
@@ -2360,7 +2410,7 @@ this change.</p>
(activation: either given; defaults <code>1.0</code>/<code>0.5</code> for whichever is unset).</li>
</ul>
<p>Full workspace <code>cargo test</code> green after this change (167 unit tests in
<code>obicompactvec</code>+<code>obilayeredmap</code>+<code>obikphylo</code> alone, plus every other
<code>obicompactvec</code>+<code>obikindex::layer</code>+<code>obikphylo</code> alone, plus every other
crate's suite, no regressions).</p>
<p><strong>Still open, not part of this change</strong> (per "Correction to the 'single
pass' framing" above): <code>--raw-snp-distance</code>/<code>--raw-snp-counts</code> (the
@@ -2380,7 +2430,7 @@ reproducible across two runs with warm disk cache), the opposite of
<code>pack --sparse</code>'s stated intent ("faster for single-row access... like
query").</p>
<p><strong>Root cause, read from source, not measured in isolation:</strong>
<code>KmerPartition::query_partition_with</code> (<code>obikpartitionner/src/query_layer.rs:155-220</code>)
<code>KmerPartition::query_partition_with</code> (<code>obikindex/src/partition/query_layer.rs:155-220</code>)
is architecturally column-major: stage 2 walks <code>for g in 0..n_cols { for
slot in hit_slots { layer.col_value(g, slot) } }</code>, documented (correctly)
as the right locality strategy for the packed/columnar formats, where
@@ -2407,45 +2457,131 @@ consumes as <code>QueryHit::Value</code> — not a materialized sub-matrix.</p>
<p><strong>Proposed primitive</strong> (design only, not implemented — explicit ask: keep
count matrices <em>not excluded</em>, even though effort right now is
presence/absence only):</p>
<div class="highlight"><pre><span></span><code><span class="sd">/// Visit every nonzero cell among `slots`. Order unspecified.</span>
<span class="k">fn</span><span class="w"> </span><span class="nf">for_each_nonzero</span><span class="p">(</span><span class="o">&amp;</span><span class="bp">self</span><span class="p">,</span><span class="w"> </span><span class="n">slots</span><span class="p">:</span><span class="w"> </span><span class="kp">&amp;</span><span class="p">[</span><span class="kt">usize</span><span class="p">],</span><span class="w"> </span><span class="n">f</span><span class="p">:</span><span class="w"> </span><span class="nc">impl</span><span class="w"> </span><span class="nb">FnMut</span><span class="p">(</span><span class="kt">usize</span><span class="w"> </span><span class="cm">/*idx into slots*/</span><span class="p">,</span><span class="w"> </span><span class="kt">usize</span><span class="w"> </span><span class="cm">/*col*/</span><span class="p">,</span><span class="w"> </span><span class="kt">u32</span><span class="w"> </span><span class="cm">/*value*/</span><span class="p">));</span>
<p>Not a closure-driven <code>for_each</code> — a real <code>Iterator</code>, one concrete struct
per matrix format, so the traversal state (current position in the sorted
slot list, current column, permutation, sparse-row decode cursor…) lives
in named struct fields instead of being threaded implicitly through
recursion or a captured closure. RPITIT (stable since 1.75, and this
workspace is edition 2024) means the trait method can return it without
naming or boxing the concrete type:</p>
<div class="highlight"><pre><span></span><code><span class="sd">/// Yields every nonzero cell among `slots`, in implementation-defined order.</span>
<span class="k">fn</span><span class="w"> </span><span class="nf">nonzero_iter</span><span class="o">&lt;&#39;</span><span class="na">a</span><span class="o">&gt;</span><span class="p">(</span><span class="o">&amp;&#39;</span><span class="na">a</span><span class="w"> </span><span class="bp">self</span><span class="p">,</span><span class="w"> </span><span class="n">slots</span><span class="p">:</span><span class="w"> </span><span class="kp">&amp;</span><span class="o">&#39;</span><span class="na">a</span><span class="w"> </span><span class="p">[</span><span class="kt">usize</span><span class="p">])</span><span class="w"> </span><span class="p">-&gt;</span><span class="w"> </span><span class="nc">impl</span><span class="w"> </span><span class="nb">Iterator</span><span class="o">&lt;</span><span class="n">Item</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="p">(</span><span class="kt">usize</span><span class="p">,</span><span class="w"> </span><span class="kt">usize</span><span class="p">,</span><span class="w"> </span><span class="kt">u32</span><span class="p">)</span><span class="o">&gt;</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="o">&#39;</span><span class="na">a</span><span class="p">;</span>
<span class="c1">// item: (idx into `slots`, col, value)</span>
</code></pre></div>
<p>This is the one foundational primitive per format — both <code>fill_sub_matrix</code>
and a <code>for_each</code>-style callback become trivial, free consumers of it
(<code>.for_each(f)</code> is already <code>Iterator::for_each</code> from std; <code>fill_sub_matrix</code>
becomes "drain the iterator, scatter into <code>out[][]</code>"), instead of two
independently-maintained traversals that can silently diverge (see the bug
below — this is exactly how it happened).</p>
<ul>
<li>On <code>PersistentSparseBitMatrix</code>: native override, one pass per slot via
the existing (currently private) <code>for_each_genome_in_row</code> — O(Σ row
nnz), zero n_cols-wide allocation. This is the whole point: expose code
that already exists rather than write anything new for the sparse side.</li>
<li>On <code>PersistentBitMatrix::{Packed,Columnar,Implicit}</code>: <strong>provided
default</strong>, derived from <code>fill_sub_matrix</code> (materialize, then filter to
true cells) — reuses the already-optimal column-major/mmap path for
those formats, no new code needed there either.</li>
<li><strong>On <code>PersistentCompactIntMatrix</code> (counts)</strong>: same provided-default
treatment, derived from its own existing <code>fill_sub_matrix</code> (<code>u32</code>-typed
already, <code>intmatrix.rs:387</code>) — not hand-optimized (no sparse count
format exists — "Explicitly deferred" per <code>traits.rs:9-12</code>), but not
excluded either: it gets a working, not-pathological implementation for
free today, on the same trait, ready for a native override the day a
sparse count format lands. This is why the signature carries <code>u32</code>
rather than <code>bool</code> — presence is <code>0</code>/<code>1</code>, counts are <code>u32</code>, one trait
covers both without a bool/u32 split forcing counts out of the design.</li>
<li><strong>On <code>PersistentSparseBitMatrix</code></strong>: the struct is nearly free to write —
it wraps the existing (currently private) <code>for_each_genome_in_row</code>
per-row decode, advancing to the next <code>slots</code> entry on exhaustion. O(Σ
row nnz), zero <code>n_cols</code>-wide allocation.</li>
<li><strong>On <code>PersistentBitMatrix::{Packed,Columnar}</code></strong>: revised — cheaper than
first thought, by reusing the same split already used for
<code>fill_matrix</code>'s own implementation instead of hand-writing a resumable
state machine at the matrix level. The base-vector layer
(<code>BitSliceView</code>, <code>views.rs</code>) already separates the two concerns:
<code>fill_batch_sorted</code> (<code>views.rs:55-60</code>, sorted-slot batch lookup) and a
genuine per-bit <code>Iterator</code> (<code>BitSliceIter</code>, <code>views.rs:94+</code>) sit side by
side there, one level below the matrix. Adding a
"positions among <code>sorted_slots</code> where the bit is set" iterator at that
same vector level is a <code>filter</code> over the existing <code>get()</code> — no new
state machine, since <code>std::iter::Filter</code> already <em>is</em> one, generated by
the compiler:
<div class="highlight"><pre><span></span><code><span class="k">fn</span><span class="w"> </span><span class="nf">nonzero_among_sorted</span><span class="o">&lt;&#39;</span><span class="na">s</span><span class="o">&gt;</span><span class="p">(</span><span class="o">&amp;&#39;</span><span class="na">s</span><span class="w"> </span><span class="bp">self</span><span class="p">,</span><span class="w"> </span><span class="n">sorted_slots</span><span class="p">:</span><span class="w"> </span><span class="kp">&amp;</span><span class="o">&#39;</span><span class="na">s</span><span class="w"> </span><span class="p">[</span><span class="kt">usize</span><span class="p">])</span><span class="w"> </span><span class="p">-&gt;</span><span class="w"> </span><span class="nc">impl</span><span class="w"> </span><span class="nb">Iterator</span><span class="o">&lt;</span><span class="n">Item</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="kt">usize</span><span class="o">&gt;</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="o">&#39;</span><span class="na">s</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">sorted_slots</span><span class="p">.</span><span class="n">iter</span><span class="p">().</span><span class="n">copied</span><span class="p">().</span><span class="n">filter</span><span class="p">(</span><span class="k">move</span><span class="w"> </span><span class="o">|&amp;</span><span class="n">slot</span><span class="o">|</span><span class="w"> </span><span class="bp">self</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="n">slot</span><span class="p">))</span>
<span class="p">}</span>
</code></pre></div>
The matrix-level <code>nonzero_iter</code> then composes these per column with
<code>flat_map</code> over <code>0..n_cols</code> (each column's hits, tagged with <code>c</code>,
slot mapped back through the sort permutation <code>fill_batch</code>/
<code>fill_batch_sorted</code> already carry) — again a combinator chain, not a
hand-rolled struct. Same algorithm, same mmap/sort locality as today's
<code>fill_sub_matrix</code>; just assembled from <code>std</code> iterator adaptors instead
of a loop body writing into a buffer, mirroring the vector/matrix split
the codebase already uses for <code>fill_batch_sorted</code> rather than
introducing a new shape.</li>
<li><strong><code>Implicit</code></strong>: trivial (<code>slots.iter().map(|&amp;i| (i, 0, 1))</code>, one column,
always present).</li>
<li><strong>On <code>PersistentCompactIntMatrix</code> (counts)</strong>: same treatment as
<code>Packed</code>/<code>Columnar</code> — no sparse count format exists yet ("Explicitly
deferred" per <code>traits.rs:9-12</code>), so no native low-effort case the way
<code>Sparse</code> has one, but not excluded either: the iterator's <code>Item</code> is
already <code>(usize, usize, u32)</code>, not <code>bool</code>, specifically so presence
(<code>0</code>/<code>1</code>) and counts (arbitrary <code>u32</code>) share one primitive instead of a
bool/u32 split forcing counts out of the design. Ready for a native
sparse-count struct later without a signature change.</li>
</ul>
<p>Would let <code>query_partition_with</code>'s stage 2 collapse to one
<code>layer.matrix().for_each_nonzero(&amp;hit_slots, |i, g, v| on_event(...))</code>
call per layer, format-agnostic, with each backend's existing (or
default-derived) implementation deciding the actual access pattern.</p>
<p><strong>Separately, an existing bug in the generic path</strong> (found while tracing
this, itself not yet fixed): <code>PersistentBitMatrix::fill_sub_matrix</code>
(<code>persistent.rs:190-215</code>, the enum wrapper backing <code>BinaryMatrix</code>'s
default trait impl) does <em>not</em> delegate to
<code>for (i, g, v) in layer.matrix().nonzero_iter(&amp;hit_slots) { on_event(...) }</code>
per layer, format-agnostic, each backend's struct deciding the actual
traversal.</p>
<p><strong>This also closes the existing dispatch bug for free, by construction</strong>:
<code>PersistentBitMatrix::fill_sub_matrix</code> (<code>persistent.rs:190-215</code>, the enum
wrapper backing <code>BinaryMatrix</code>'s trait impl) today does <em>not</em> delegate to
<code>PersistentSparseBitMatrix::fill_sub_matrix</code> for <code>Self::Sparse</code> — it
reimplements the same naive per-(column, slot) <code>fill_row_bool</code> loop
instead, bypassing the efficient native method one file over
(<code>sparse.rs:249-258</code>). <code>obikphylo::siblings::cache::Mat</code>
(<code>cache.rs:138-145</code>) independently built its own parallel enum wrapper
that dispatches correctly — a sign this was worked around rather than
fixed at the source. Any future <code>for_each_nonzero</code> work should fix this
dispatch too (or route through it), rather than adding a third
independently-dispatching wrapper.</p>
that dispatches correctly — evidence this was worked around rather than
fixed at the source: two hand-written traversals for the same format,
free to drift apart, and they did. If <code>fill_sub_matrix</code> itself is
rewritten as "drain <code>nonzero_iter</code>, scatter into <code>out[][]</code>", there is only
one traversal per format left to get right — the bug class doesn't just
get fixed once, it stops being possible to reintroduce.</p>
<h2 id="implemented-2026-08-20">Implemented (2026-08-20)</h2>
<p>Built as designed above, with one deviation from the original sketch:
<code>nonzero_iter</code> ended up <code>Box&lt;dyn Iterator&lt;...&gt;&gt;</code>, not a bare <code>impl
Iterator</code>, because <code>Columnar</code>/<code>Packed</code>/<code>Sparse</code>/<code>Implicit</code> are genuinely
different concrete types and this method isn't on a trait (kept off
<code>BinaryMatrix</code> deliberately — that trait is used as <code>dyn BinaryMatrix</code> in
<code>tests/sparse.rs</code>, and RPITIT methods aren't dyn-compatible). One <code>Box</code>
per <code>nonzero_iter</code> call, not per cell — negligible next to what it
replaces.</p>
<ul>
<li><code>BitSliceView::nonzero_among_sorted</code> / <code>IntSliceView::nonzero_among_sorted</code>
(<code>obicompactvec/src/views.rs</code>): the vector-level <code>filter</code>/<code>filter_map</code>
primitive, exactly as sketched — no new state machine, <code>std</code>'s own.</li>
<li><code>PersistentSparseBitMatrix::nonzero_iter</code> (<code>bitmatrix/sparse.rs</code>): native,
<code>std::iter::from_fn</code> over one buffered row at a time via the existing
<code>for_each_genome_in_row</code> — no <code>n_cols</code>-wide allocation, ever.</li>
<li><code>PersistentBitMatrix::nonzero_iter</code> (<code>bitmatrix/persistent.rs</code>): dispatches
to the above for <code>Sparse</code>; for <code>Columnar</code>/<code>Packed</code>, loops columns,
collects each column's <code>nonzero_among_sorted</code> hits via <code>.extend()</code> (not
<code>flat_map</code> — a <code>flat_map</code> closure can't lazily return something
borrowing its own captured sort permutation across separate calls
without either boxing per-column or fighting the borrow checker; eager
collection into one <code>Vec</code> sidesteps it, at zero cost since
<code>fill_sub_matrix</code> already fully materialized anyway). <code>Implicit</code> trivial.</li>
<li><code>PersistentBitMatrix::fill_sub_matrix</code> and <code>sub_matrix</code> rewritten to
drain <code>nonzero_iter</code> — the dispatch bug is gone because there is now
only one traversal per format, not because the old one was patched.
<code>PersistentCompactIntMatrix::nonzero_iter</code> added the same way (counts
not excluded, per the earlier ask) — no native low-effort case, since no
sparse count format exists, but on the same primitive, ready for one.</li>
<li><code>KmerPartition::query_partition_with</code> (<code>obikindex/src/partition/query_layer.rs</code>):
stage 2's column-major <code>for g { for slot { col_value } }</code> replaced by one
<code>layer.nonzero_iter(&amp;slot_list)</code> call per layer, format-agnostic.</li>
<li>Tests: <code>nonzero_iter_matches_dense</code>, <code>nonzero_iter_matches_row</code>, and —
the one that actually targets the dispatch bug rather than each type's
own correctness — <code>enum_wrapper_dispatches_to_native_sparse</code> (builds
<code>PersistentBitMatrix::Sparse(...)</code> directly, not through <code>open</code>, since
<code>open</code> only auto-detects <code>Sparse</code> from a <code>presence/</code> dir layout).
<code>cargo test --workspace</code>: green, no regressions.</li>
</ul>
<p><strong>Measured</strong>: re-ran the <code>benchmark/</code> query branch (100k reads × 2
specimens, same setup as the original finding). Correctness still 0
mismatches. The dense/sparse performance gap is gone — previously sparse
~30-50% slower than dense, reproducibly; now within ~1-3% either way
(7.42s dense vs 7.60s sparse for <code>Escherichia_coli--K-12_MG1655</code>; 5.25s vs
5.30s for <code>Saccharolobus_islandicus--M.16.4</code>) — noise-level, not a
systematic gap. <code>pack --sparse</code>'s claimed query win isn't confirmed
outright by this (sparse should arguably now <em>beat</em> dense on truly sparse
real data, not just tie), but the pathological regression is fixed.</p>