OBIKmers/obikmer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 34 Wiki Activity

docs: document partition runner deadlock fix and select CLI updates

Browse Source 
Update documentation to reflect architectural fixes in the NUMA partition runner and implementation changes in the merge and select modules. The partition runner now wraps task execution in `catch_unwind`, emitting panic events that count toward completion thresholds to prevent controller deadlocks, while re-raising the first payload on return. CLI documentation for select has been updated to mandate output directories, introduce default hard-linking with automatic fallback, and document presence aggregation optimizations. Known issues such as self-merge crashes and sparse-packed index gaps are now documented as failing fast due to these improvements.
This commit is contained in:
Eric Coissac
2026-08-28 23:25:46 +02:00
parent 6b0c0867cc
commit 579cfd8752
6 changed files with 243 additions and 93 deletions
Download Patch File Download Diff File Expand all files Collapse all files
DevDoc/architecture/numa_partition_runner/index.html
+39 -1
View File
@@ -1367,6 +1367,17 @@
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#panic-propagation-fixed-2026-08-28" class="md-nav__link">
<span class="md-ellipsis">
Panic propagation (fixed, 2026-08-28)
</span>
</a>
</li>
<li class="md-nav__item">
@@ -1553,6 +1564,17 @@
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#panic-propagation-fixed-2026-08-28" class="md-nav__link">
<span class="md-ellipsis">
Panic propagation (fixed, 2026-08-28)
</span>
</a>
</li>
<li class="md-nav__item">
@@ -1836,11 +1858,27 @@ dormant workers have been woken and grows every node by the same amount per
step, capped by that node's remaining dormant workers and by the run's total
budget (<code>n_total</code>) — balance across nodes is now guaranteed by construction,
not incidental to channel implementation details.</p>
<h2 id="panic-propagation-fixed-2026-08-28">Panic propagation (fixed, 2026-08-28)</h2>
<p>A worker whose <code>f(i)</code> panicked used to never send its <code>WorkerEvent::Completed</code>
— the controller's <code>while completed &lt; n_total</code> loop then waited forever for
an event that partition could no longer produce, since other live
workers/timer threads kept <code>event_rx</code> open. Discovered via <code>obikselect</code>
panicking on an unimplemented <code>Sparse</code> matrix case (see <code>implementation/select.md</code>):
the process hung indefinitely instead of erroring out.</p>
<p>Fixed: each worker wraps <code>f(i)</code> in <code>std::panic::catch_unwind</code> and sends a new
<code>WorkerEvent::Panicked(i, payload)</code> instead of silently dropping the
partition. The controller counts it toward <code>completed</code> (unblocking the loop)
and keeps the first payload seen; once <code>run</code> returns, <code>panic::resume_unwind</code>
re-raises it on the caller's thread — the original message/backtrace still
surfaces, from the right place, instead of a silent deadlock. Takes priority
over a plain <code>Err</code> (a panic means a bug, not a normal typed failure).</p>
<h2 id="open-questions">Open questions</h2>
<ul>
<li>
<p><strong>Error handling</strong>: <code>run</code> currently returns the first error; remaining errors
are dropped. A <code>Vec&lt;E&gt;</code> return would give complete diagnostics.</p>
are dropped. A <code>Vec&lt;E&gt;</code> return would give complete diagnostics. (Panics are
now caught and re-raised individually — see above — this only concerns
plain <code>Err</code> results.)</p>
</li>
<li>
<p><strong><code>INITIAL_DIVISOR</code> / <code>GROWTH_DIVISOR</code> tuning</strong>: currently <code>4</code> and <code>8</code>
DevDoc/implementation/merge/index.html
+30
View File
@@ -1034,6 +1034,17 @@
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#known-issue-not-yet-fixed-2026-08-28" class="md-nav__link">
<span class="md-ellipsis">
Known issue (not yet fixed, 2026-08-28)
</span>
</a>
</li>
</ul>
@@ -1598,6 +1609,17 @@
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#known-issue-not-yet-fixed-2026-08-28" class="md-nav__link">
<span class="md-ellipsis">
Known issue (not yet fixed, 2026-08-28)
</span>
</a>
</li>
</ul>
@@ -1795,6 +1817,14 @@ spectrums/
&lt;label&gt;.json ← one file per genome, rebuilt from all sources
index.meta ← complete genome list + evidence kind written at bootstrap
</code></pre></div>
<p><code>mphf.bin</code>/<code>unitigs.bin</code>/<code>evidence.bin</code>/<code>unitigs.bin.idx</code>/<code>fingerprint.bin</code>/<code>layer_meta.json</code> marked "unchanged" above are hard-linked from the base source's own files during the bootstrap copy (2026-08-28), not copied — <code>merge_partition</code> never rewrites them for pre-existing layers, only the <code>presence</code>/<code>counts</code> subdirectory gets widened in place, so only that subdirectory is a real, independent copy. Falls back to a real copy per file if linking itself fails (different filesystems). Verified: source files are byte-identical (checksummed) before/after a normal merge.</p>
<h2 id="known-issue-not-yet-fixed-2026-08-28">Known issue (not yet fixed, 2026-08-28)</h2>
<p>Merging an index against itself — literally the same directory passed twice as separate source arguments (e.g. <code>obikmer merge -o out IDX IDX --rename-duplicates</code>) — panics deep in the MPHF's rank-select structure (<code>common_traits::select_in_word</code>, <code>assertion failed: rank &lt; self.count_ones()</code>), inside <code>MphfLayer::find</code> called from <code>merge_partition</code>'s "is this source kmer already in dst" check against the bootstrap-copied <code>dst_layers</code>. Root cause not identified; ruled out so far:</p>
<ul>
<li><strong>Not an empty-new-layer issue</strong>: a source contributing zero new kmers (verified both as the sole additional source and as a third, fully-redundant source in a 3-way merge) is handled correctly — no layer is created for it, no crash, exit 0.</li>
<li><strong>Not caused by the hard-link change above</strong>: verified via checksum that a normal (two distinct sources) merge leaves every source file byte-identical.</li>
</ul>
<p>Only reproduces when <code>sources[0]</code> and <code>sources[1]</code> are the exact same on-disk path opened as two independent <code>KmerIndex</code> handles — an artificial scenario (nobody merges an index with itself intentionally), deprioritized rather than investigated further for now. Does terminate cleanly rather than hang, since <code>PartitionRunner</code>'s panic propagation fix (see <code>architecture/numa_partition_runner.md</code>) now surfaces it as a normal process panic/exit 101 instead of a silent deadlock.</p>
DevDoc/implementation/select/index.html
+88 -56
View File
@@ -1119,17 +1119,6 @@
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#in-place-keep-only-group-a" class="md-nav__link">
<span class="md-ellipsis">
In-place: keep only group A
</span>
</a>
</li>
<li class="md-nav__item">
@@ -1157,6 +1146,23 @@
</span>
</a>
<nav class="md-nav" aria-label="Implementation notes">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#known-gap-not-yet-fixed-2026-08-28" class="md-nav__link">
<span class="md-ellipsis">
Known gap (not yet fixed, 2026-08-28)
</span>
</a>
</li>
</ul>
</nav>
</li>
</ul>
@@ -1638,17 +1644,6 @@
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#in-place-keep-only-group-a" class="md-nav__link">
<span class="md-ellipsis">
In-place: keep only group A
</span>
</a>
</li>
<li class="md-nav__item">
@@ -1676,6 +1671,23 @@
</span>
</a>
<nav class="md-nav" aria-label="Implementation notes">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#known-gap-not-yet-fixed-2026-08-28" class="md-nav__link">
<span class="md-ellipsis">
Known gap (not yet fixed, 2026-08-28)
</span>
</a>
</li>
</ul>
</nav>
</li>
</ul>
@@ -1728,23 +1740,32 @@ are preserved unchanged; only the data matrices are rewritten.</p>
<hr />
<h2 id="synopsis">Synopsis</h2>
<div class="highlight"><pre><span></span><code>obikmer<span class="w"> </span><span class="k">select</span><span class="w"> </span>&lt;input-index&gt;
<span class="w"> </span><span class="o">{</span><span class="w"> </span>--output<span class="w"> </span>&lt;dir&gt;<span class="w"> </span><span class="p">|</span><span class="w"> </span>--in-place<span class="w"> </span><span class="o">}</span>
<span class="w"> </span>--output<span class="w"> </span>&lt;dir&gt;
<span class="w"> </span><span class="o">[</span>--group<span class="w"> </span>&lt;name&gt;:&lt;pred&gt;<span class="w"> </span>...<span class="o">]</span>
<span class="w"> </span><span class="o">[</span>--group-op<span class="w"> </span>&lt;name&gt;:&lt;op&gt;<span class="w"> </span>...<span class="o">]</span>
<span class="w"> </span><span class="o">[</span>--aggregate-by<span class="w"> </span>&lt;key&gt;<span class="w"> </span><span class="o">]</span>
<span class="w"> </span><span class="o">[</span>--aggregate-op<span class="w"> </span>&lt;op&gt;<span class="w"> </span><span class="o">]</span>
<span class="w"> </span><span class="o">[</span>--select<span class="w"> </span>&lt;col1,col2,...&gt;<span class="w"> </span><span class="o">]</span>
<span class="w"> </span><span class="o">[</span>--presence-threshold<span class="w"> </span>&lt;N&gt;<span class="w"> </span><span class="o">]</span>
<span class="w"> </span><span class="o">[</span>--force-copy<span class="w"> </span><span class="o">]</span>
</code></pre></div>
<hr />
<h2 id="output-destination">Output destination</h2>
<p>Exactly one of <code>--output</code> or <code>--in-place</code> must be specified.</p>
<p><strong><code>--output &lt;dir&gt;</code></strong> — writes a new index to <code>&lt;dir&gt;</code>. The source index is
unchanged. The MPHF and unitig files are copied; only the data matrices are
rewritten with the new column layout.</p>
<p><strong><code>--in-place</code></strong> — rewrites the data matrices of the source index directly.
Removed or replaced columns are lost. The operation writes to temporary files
first, then renames atomically, so an interrupted run leaves the index intact.</p>
<p><code>--output &lt;dir&gt;</code> is required — <code>select</code> always writes a new index; there is no
<code>--in-place</code> mode (2026-08-28: never implemented, removed from the design). The
source index is unchanged.</p>
<p>Each layer's kmer-identity files (<code>mphf.bin</code>/<code>unitigs.bin</code>/<code>evidence.bin</code>/
<code>unitigs.bin.idx</code>/<code>fingerprint.bin</code>/<code>layer_meta.json</code>) are never rewritten by a
column projection/aggregation, so they are hard-linked into the output rather
than copied — no extra disk for them even on a large index. Falls back to a
real copy automatically if linking fails (different filesystems); <code>--force-copy</code>
forces a real copy always, for an output that must survive independently of the
source on disk (a hard link shares the same inode — rewriting one path outside
<code>select</code> itself would affect the other). Only the <code>presence</code>/<code>counts</code>
subdirectory is ever a genuinely new, independent file.</p>
<p>To replace an index with a selected version of itself, select to a temporary
directory and swap it in (<code>rm -rf INDEX &amp;&amp; mv INDEX.tmp INDEX</code>) — the case
<code>--in-place</code> used to cover.</p>
<hr />
<h2 id="defining-output-columns">Defining output columns</h2>
<h3 id="named-groups-group">Named groups — <code>--group</code></h3>
@@ -1908,9 +1929,6 @@ is &gt; N.</p>
<div class="highlight"><pre><span></span><code>obikmer<span class="w"> </span><span class="k">select</span><span class="w"> </span>myindex<span class="w"> </span>--output<span class="w"> </span>out<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>--select<span class="w"> </span><span class="s2">&quot;Betula_nana--TROM-V-149986,Betula_nana--AG-P04-25-01&quot;</span>
</code></pre></div>
<h3 id="in-place-keep-only-group-a">In-place: keep only group A</h3>
<div class="highlight"><pre><span></span><code>obikmer<span class="w"> </span><span class="k">select</span><span class="w"> </span>myindex<span class="w"> </span>--in-place<span class="w"> </span>--group<span class="w"> </span><span class="s2">&quot;A:group=A&quot;</span><span class="w"> </span>--select<span class="w"> </span><span class="s2">&quot;A&quot;</span>
</code></pre></div>
<h3 id="compose-with-filter">Compose with filter</h3>
<div class="highlight"><pre><span></span><code><span class="c1"># Step 1: keep only B. nana-specific k-mers</span>
obikmer<span class="w"> </span>filter<span class="w"> </span>myindex<span class="w"> </span>--output<span class="w"> </span>filtered<span class="w"> </span><span class="se">\</span>
@@ -1921,32 +1939,46 @@ obikmer<span class="w"> </span><span class="k">select</span><span class="w"> </s
</code></pre></div>
<hr />
<h2 id="implementation-notes">Implementation notes</h2>
<p><code>select</code> does not rebuild the MPHF. The 256 partitions are processed in parallel
(rayon), each writing its output independently; results require no synchronisation
because every partition owns a distinct set of files.</p>
<p>For each layer in each partition:</p>
<p><code>select</code> does not rebuild the MPHF. Every partition is processed independently
(<code>PartitionRunner</code>), each writing its own output layers; no cross-partition
synchronisation is needed.</p>
<p>For each layer in each partition (<code>obikselect::select_layer::select_partition</code>):</p>
<ol>
<li>The slot count <code>n</code> is read by opening the source data matrix.</li>
<li>A new data matrix is built with M columns (M = number of output columns).</li>
<li>For each slot <code>s</code> in <code>0..n</code>:</li>
<li><code>old_row = matrix.fill_row(s)</code> — reads the original <code>N</code>-column row without allocating.</li>
<li>For each output column <code>j</code>:<ul>
<li><code>new_row[j] = aggregate(op, old_row[group_indices])</code>.</li>
<li>Pass-through columns are represented as single-element groups with the
default operator (<code>any</code> for presence, <code>sum</code> for count) — same code path.</li>
</ul>
</li>
<li>The new row is written slot by slot into each column builder.</li>
<li>All plain files in the source layer directory (<code>mphf.bin</code>, <code>unitigs.bin</code>,
evidence files, <code>layer_meta.json</code>) are copied verbatim; only the <code>presence/</code>
or <code>counts/</code> subdirectory is rewritten.</li>
<li><code>copy_layer_files</code> hard-links the source layer's kmer-identity files
(<code>mphf.bin</code>/<code>unitigs.bin</code>/<code>evidence.bin</code>/<code>unitigs.bin.idx</code>/
<code>fingerprint.bin</code>/<code>layer_meta.json</code>) into the destination — never a real
copy unless linking fails or <code>--force-copy</code> is given.</li>
<li>A new data matrix is built with M columns (M = number of output columns),
under a fresh <code>presence/</code>/<code>counts/</code> subdirectory (never touching the
source's own).</li>
<li><strong>Presence source (2026-08-28: <code>batch_presence_counts</code>)</strong>: one shared pass
over the source bit matrix computes every output group's presence count at
once — row-major native for a <code>Sparse</code> source (<code>for_each_genome_in_row</code>,
which has no column representation to read a <code>col_view</code> from at all — the
reason this replaced the old per-group loop, not just an optimisation of
it), deduplicated column-major (one <code>col_view</code> per <em>distinct referenced
column</em>, not per group) for <code>Columnar</code>/<code>Packed</code>. Every <code>AggOp</code> for a bit
matrix is then a cheap derivation of that one count vector (<code>sum</code> = the
count itself, <code>any</code>/<code>max</code> = <code>count ≥ 1</code>, <code>all</code>/<code>min</code> = <code>count == group
size</code>, <code>none</code> = <code>count == 0</code>) — see
<code>obikselect::select_layer::agg_result_from_count</code>.</li>
<li><strong>Count source</strong>: unchanged, one <code>col_view</code>-driven pass per output column
via <code>MatrixGroupOps</code> — <code>sum</code>/<code>min</code>/<code>max</code> are genuine per-value reductions
for a count matrix, not derivable from a single presence count the way
they are for a bit matrix.</li>
<li><code>index.meta</code> is rewritten with the new genome list and updated <code>with_counts</code>.</li>
</ol>
<p><strong><code>--in-place</code> write strategy:</strong> new data is written to a temporary sibling
directory (<code>presence_new/</code> or <code>counts_new/</code>); on success the old directory is
removed and the temporary one is renamed into place. An interrupted run leaves
at most one stale <code>*_new/</code> directory; the original data is intact until the
rename step.</p>
<h3 id="known-gap-not-yet-fixed-2026-08-28">Known gap (not yet fixed, 2026-08-28)</h3>
<p>Step 4 above still panics (<code>col_view() not available on Sparse
PersistentCompactIntMatrix</code>) if the source is a <strong>count</strong> index packed
sparse — <code>batch_presence_counts</code>' row-major treatment was only ported to the
bit-matrix (<code>Presence</code>) case, since that was the one actually blocking a real
benchmark run. <code>select</code>/<code>filter</code> on a sparse-packed count index still hits
this; the fix would follow the same shape (a <code>PersistentSparseCompactIntMatrix</code>
row-major decode, analogous to <code>for_each_genome_in_row</code>), just not done. Since
<code>obisys::numa::runner::PartitionRunner</code>'s panic-propagation fix (see
<code>architecture/numa_partition_runner.md</code>), this at least fails fast (process
panic, exit 101) instead of hanging.</p>
DevDocMD/architecture/numa_partition_runner.md
+20 -1
View File
@@ -304,10 +304,29 @@ step, capped by that node's remaining dormant workers and by the run's total
budget (`n_total`) — balance across nodes is now guaranteed by construction,
not incidental to channel implementation details.
## Panic propagation (fixed, 2026-08-28)
A worker whose `f(i)` panicked used to never send its `WorkerEvent::Completed`
— the controller's `while completed < n_total` loop then waited forever for
an event that partition could no longer produce, since other live
workers/timer threads kept `event_rx` open. Discovered via `obikselect`
panicking on an unimplemented `Sparse` matrix case (see `implementation/select.md`):
the process hung indefinitely instead of erroring out.
Fixed: each worker wraps `f(i)` in `std::panic::catch_unwind` and sends a new
`WorkerEvent::Panicked(i, payload)` instead of silently dropping the
partition. The controller counts it toward `completed` (unblocking the loop)
and keeps the first payload seen; once `run` returns, `panic::resume_unwind`
re-raises it on the caller's thread — the original message/backtrace still
surfaces, from the right place, instead of a silent deadlock. Takes priority
over a plain `Err` (a panic means a bug, not a normal typed failure).
## Open questions
- **Error handling**: `run` currently returns the first error; remaining errors
are dropped. A `Vec<E>` return would give complete diagnostics.
are dropped. A `Vec<E>` return would give complete diagnostics. (Panics are
now caught and re-raised individually — see above — this only concerns
plain `Err` results.)
- **`INITIAL_DIVISOR` / `GROWTH_DIVISOR` tuning**: currently `4` and `8`
(start at 1/4 of a node's cores, grow by 1/8 per step), chosen to fix an
DevDocMD/implementation/merge.md
+11
View File
@@ -194,3 +194,14 @@ spectrums/
<label>.json ← one file per genome, rebuilt from all sources
index.meta ← complete genome list + evidence kind written at bootstrap
```
`mphf.bin`/`unitigs.bin`/`evidence.bin`/`unitigs.bin.idx`/`fingerprint.bin`/`layer_meta.json` marked "unchanged" above are hard-linked from the base source's own files during the bootstrap copy (2026-08-28), not copied — `merge_partition` never rewrites them for pre-existing layers, only the `presence`/`counts` subdirectory gets widened in place, so only that subdirectory is a real, independent copy. Falls back to a real copy per file if linking itself fails (different filesystems). Verified: source files are byte-identical (checksummed) before/after a normal merge.
## Known issue (not yet fixed, 2026-08-28)
Merging an index against itself — literally the same directory passed twice as separate source arguments (e.g. `obikmer merge -o out IDX IDX --rename-duplicates`) — panics deep in the MPHF's rank-select structure (`common_traits::select_in_word`, `assertion failed: rank < self.count_ones()`), inside `MphfLayer::find` called from `merge_partition`'s "is this source kmer already in dst" check against the bootstrap-copied `dst_layers`. Root cause not identified; ruled out so far:
- **Not an empty-new-layer issue**: a source contributing zero new kmers (verified both as the sole additional source and as a third, fully-redundant source in a 3-way merge) is handled correctly — no layer is created for it, no crash, exit 0.
- **Not caused by the hard-link change above**: verified via checksum that a normal (two distinct sources) merge leaves every source file byte-identical.
Only reproduces when `sources[0]` and `sources[1]` are the exact same on-disk path opened as two independent `KmerIndex` handles — an artificial scenario (nobody merges an index with itself intentionally), deprioritized rather than investigated further for now. Does terminate cleanly rather than hang, since `PartitionRunner`'s panic propagation fix (see `architecture/numa_partition_runner.md`) now surfaces it as a normal process panic/exit 101 instead of a silent deadlock.
DevDocMD/implementation/select.md
+55 -35
View File
@@ -23,28 +23,37 @@ are preserved unchanged; only the data matrices are rewritten.
```sh
obikmer select <input-index>
{ --output <dir> | --in-place }
--output <dir>
[--group <name>:<pred> ...]
[--group-op <name>:<op> ...]
[--aggregate-by <key> ]
[--aggregate-op <op> ]
[--select <col1,col2,...> ]
[--presence-threshold <N> ]
[--force-copy ]
```
---
## Output destination
Exactly one of `--output` or `--in-place` must be specified.
`--output <dir>` is required — `select` always writes a new index; there is no
`--in-place` mode (2026-08-28: never implemented, removed from the design). The
source index is unchanged.
**`--output <dir>`** — writes a new index to `<dir>`. The source index is
unchanged. The MPHF and unitig files are copied; only the data matrices are
rewritten with the new column layout.
Each layer's kmer-identity files (`mphf.bin`/`unitigs.bin`/`evidence.bin`/
`unitigs.bin.idx`/`fingerprint.bin`/`layer_meta.json`) are never rewritten by a
column projection/aggregation, so they are hard-linked into the output rather
than copied — no extra disk for them even on a large index. Falls back to a
real copy automatically if linking fails (different filesystems); `--force-copy`
forces a real copy always, for an output that must survive independently of the
source on disk (a hard link shares the same inode — rewriting one path outside
`select` itself would affect the other). Only the `presence`/`counts`
subdirectory is ever a genuinely new, independent file.
**`--in-place`** — rewrites the data matrices of the source index directly.
Removed or replaced columns are lost. The operation writes to temporary files
first, then renames atomically, so an interrupted run leaves the index intact.
To replace an index with a selected version of itself, select to a temporary
directory and swap it in (`rm -rf INDEX && mv INDEX.tmp INDEX`) — the case
`--in-place` used to cover.
---
@@ -186,12 +195,6 @@ obikmer select myindex --output out \
--select "Betula_nana--TROM-V-149986,Betula_nana--AG-P04-25-01"
```
### In-place: keep only group A
```sh
obikmer select myindex --in-place --group "A:group=A" --select "A"
```
### Compose with filter
```sh
@@ -207,28 +210,45 @@ obikmer select filtered --output final --aggregate-by site
## Implementation notes
`select` does not rebuild the MPHF. The 256 partitions are processed in parallel
(rayon), each writing its output independently; results require no synchronisation
because every partition owns a distinct set of files.
`select` does not rebuild the MPHF. Every partition is processed independently
(`PartitionRunner`), each writing its own output layers; no cross-partition
synchronisation is needed.
For each layer in each partition:
For each layer in each partition (`obikselect::select_layer::select_partition`):
1. The slot count `n` is read by opening the source data matrix.
2. A new data matrix is built with M columns (M = number of output columns).
3. For each slot `s` in `0..n`:
- `old_row = matrix.fill_row(s)` — reads the original `N`-column row without allocating.
- For each output column `j`:
- `new_row[j] = aggregate(op, old_row[group_indices])`.
- Pass-through columns are represented as single-element groups with the
default operator (`any` for presence, `sum` for count) — same code path.
- The new row is written slot by slot into each column builder.
4. All plain files in the source layer directory (`mphf.bin`, `unitigs.bin`,
evidence files, `layer_meta.json`) are copied verbatim; only the `presence/`
or `counts/` subdirectory is rewritten.
1. `copy_layer_files` hard-links the source layer's kmer-identity files
(`mphf.bin`/`unitigs.bin`/`evidence.bin`/`unitigs.bin.idx`/
`fingerprint.bin`/`layer_meta.json`) into the destination — never a real
copy unless linking fails or `--force-copy` is given.
2. A new data matrix is built with M columns (M = number of output columns),
under a fresh `presence/`/`counts/` subdirectory (never touching the
source's own).
3. **Presence source (2026-08-28: `batch_presence_counts`)**: one shared pass
over the source bit matrix computes every output group's presence count at
once — row-major native for a `Sparse` source (`for_each_genome_in_row`,
which has no column representation to read a `col_view` from at all — the
reason this replaced the old per-group loop, not just an optimisation of
it), deduplicated column-major (one `col_view` per *distinct referenced
column*, not per group) for `Columnar`/`Packed`. Every `AggOp` for a bit
matrix is then a cheap derivation of that one count vector (`sum` = the
count itself, `any`/`max` = `count ≥ 1`, `all`/`min` = `count == group
size`, `none` = `count == 0`) — see
`obikselect::select_layer::agg_result_from_count`.
4. **Count source**: unchanged, one `col_view`-driven pass per output column
via `MatrixGroupOps` — `sum`/`min`/`max` are genuine per-value reductions
for a count matrix, not derivable from a single presence count the way
they are for a bit matrix.
5. `index.meta` is rewritten with the new genome list and updated `with_counts`.
**`--in-place` write strategy:** new data is written to a temporary sibling
directory (`presence_new/` or `counts_new/`); on success the old directory is
removed and the temporary one is renamed into place. An interrupted run leaves
at most one stale `*_new/` directory; the original data is intact until the
rename step.
### Known gap (not yet fixed, 2026-08-28)
Step 4 above still panics (`col_view() not available on Sparse
PersistentCompactIntMatrix`) if the source is a **count** index packed
sparse — `batch_presence_counts`' row-major treatment was only ported to the
bit-matrix (`Presence`) case, since that was the one actually blocking a real
benchmark run. `select`/`filter` on a sparse-packed count index still hits
this; the fix would follow the same shape (a `PersistentSparseCompactIntMatrix`
row-major decode, analogous to `for_each_genome_in_row`), just not done. Since
`obisys::numa::runner::PartitionRunner`'s panic-propagation fix (see
`architecture/numa_partition_runner.md`), this at least fails fast (process
panic, exit 101) instead of hanging.