OBIKmers/obikmer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 34 Wiki Activity

Implement SNP distance models with gamma correction and PHYLIP output

Browse Source 
Renames the CLI flag from --metric to --distance and introduces eight closed-form SNP distance models with optional Jin-Nei gamma correction. Integrates the ndarray crate for matrix operations and adds relaxed PHYLIP output formatting. Updates architecture and theory documentation to cover the new sparse matrix variants, algorithmic fixes, and distance metric implementations.
This commit is contained in:
Eric Coissac
2026-08-28 21:54:19 +02:00
parent 0b40d2d0da
commit 4f34a646c5
17 changed files with 2199 additions and 73 deletions
Download Patch File Download Diff File Expand all files Collapse all files
DevDoc/implementation/partition_layer_cache/index.html
+881 -1
View File
@@ -1213,6 +1213,122 @@
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#6-done-2026-08-21-counter-third-algorithm-extracted-the-same-way-as-dereplicator" class="md-nav__link">
<span class="md-ellipsis">
(6) done (2026-08-21): Counter — third algorithm, extracted the same way as Dereplicator
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#7-done-2026-08-21-layerbuilder-fourth-and-last-pipeline-algorithm" class="md-nav__link">
<span class="md-ellipsis">
(7) done (2026-08-21): LayerBuilder — fourth and last pipeline algorithm
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#8-design-agreed-not-yet-implemented-2026-08-21-obikalgorithmalgorithm-trait-obikindexerextensions-privatepublic-extension-trait-split-kmerlayer-rename" class="md-nav__link">
<span class="md-ellipsis">
(8) design agreed, not yet implemented (2026-08-21): obikalgorithm::Algorithm trait + obikindexer::extensions — private/public extension-trait split, KmerLayer rename
</span>
</a>
<nav class="md-nav" aria-label="(8) design agreed, not yet implemented (2026-08-21): obikalgorithm::Algorithm trait + obikindexer::extensions — private/public extension-trait split, KmerLayer rename">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#why-this-came-up" class="md-nav__link">
<span class="md-ellipsis">
Why this came up
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#the-general-pattern-not-obikindexer-specific" class="md-nav__link">
<span class="md-ellipsis">
The general pattern (not obikindexer-specific)
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#concretely-next-to-implement-two-items-in-order" class="md-nav__link">
<span class="md-ellipsis">
Concretely, next to implement (two items, in order)
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#9-done-2026-08-21-obikindexerextensionsprivatebuilder-item-1-above-implemented" class="md-nav__link">
<span class="md-ellipsis">
(9) done (2026-08-21): obikindexer::extensions::PrivateBuilder — item 1 above, implemented
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#10-done-2026-08-21-obikindexindexbuilder-the-public-counterpart-same-session" class="md-nav__link">
<span class="md-ellipsis">
(10) done (2026-08-21): obikindex::IndexBuilder — the public counterpart, same session
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#11-done-2026-08-21-kmerindexindexmeta-made-fully-stateless-indexstate-moved-off-sentinel-files" class="md-nav__link">
<span class="md-ellipsis">
(11) done (2026-08-21): KmerIndex/IndexMeta made fully stateless, IndexState moved off sentinel files
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#12-done-2026-08-21-obikalgorithmalgorithm-the-shared-trait-resumed-and-closed-in-one-session" class="md-nav__link">
<span class="md-ellipsis">
(12) done (2026-08-21): obikalgorithm::Algorithm — the shared trait, resumed and closed in one session
</span>
</a>
</li>
<li class="md-nav__item">
@@ -1719,6 +1835,122 @@
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#6-done-2026-08-21-counter-third-algorithm-extracted-the-same-way-as-dereplicator" class="md-nav__link">
<span class="md-ellipsis">
(6) done (2026-08-21): Counter — third algorithm, extracted the same way as Dereplicator
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#7-done-2026-08-21-layerbuilder-fourth-and-last-pipeline-algorithm" class="md-nav__link">
<span class="md-ellipsis">
(7) done (2026-08-21): LayerBuilder — fourth and last pipeline algorithm
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#8-design-agreed-not-yet-implemented-2026-08-21-obikalgorithmalgorithm-trait-obikindexerextensions-privatepublic-extension-trait-split-kmerlayer-rename" class="md-nav__link">
<span class="md-ellipsis">
(8) design agreed, not yet implemented (2026-08-21): obikalgorithm::Algorithm trait + obikindexer::extensions — private/public extension-trait split, KmerLayer rename
</span>
</a>
<nav class="md-nav" aria-label="(8) design agreed, not yet implemented (2026-08-21): obikalgorithm::Algorithm trait + obikindexer::extensions — private/public extension-trait split, KmerLayer rename">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#why-this-came-up" class="md-nav__link">
<span class="md-ellipsis">
Why this came up
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#the-general-pattern-not-obikindexer-specific" class="md-nav__link">
<span class="md-ellipsis">
The general pattern (not obikindexer-specific)
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#concretely-next-to-implement-two-items-in-order" class="md-nav__link">
<span class="md-ellipsis">
Concretely, next to implement (two items, in order)
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#9-done-2026-08-21-obikindexerextensionsprivatebuilder-item-1-above-implemented" class="md-nav__link">
<span class="md-ellipsis">
(9) done (2026-08-21): obikindexer::extensions::PrivateBuilder — item 1 above, implemented
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#10-done-2026-08-21-obikindexindexbuilder-the-public-counterpart-same-session" class="md-nav__link">
<span class="md-ellipsis">
(10) done (2026-08-21): obikindex::IndexBuilder — the public counterpart, same session
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#11-done-2026-08-21-kmerindexindexmeta-made-fully-stateless-indexstate-moved-off-sentinel-files" class="md-nav__link">
<span class="md-ellipsis">
(11) done (2026-08-21): KmerIndex/IndexMeta made fully stateless, IndexState moved off sentinel files
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#12-done-2026-08-21-obikalgorithmalgorithm-the-shared-trait-resumed-and-closed-in-one-session" class="md-nav__link">
<span class="md-ellipsis">
(12) done (2026-08-21): obikalgorithm::Algorithm — the shared trait, resumed and closed in one session
</span>
</a>
</li>
<li class="md-nav__item">
@@ -1853,6 +2085,21 @@ The underlying module boundary and its rationale (Layer tier / Partition
tier / Index tier, each depending only downward) are unchanged; only the
crate-vs-module packaging changed. See <a href="../layer_tier/">obikindex::layer</a>
for the current module doc.</p>
<p><strong>Superseded, second event, same day (2026-08-21):</strong> <code>obikpartitionner</code>
and <code>obikderep</code>, the two algorithm crates, are also gone — but unlike
<code>obikpartition</code>/<code>obilayeredmap</code> above, they were <strong>not</strong> folded into
<code>obikindex</code>. They were first (mistakenly) merged into <code>obikindex</code> as an
<code>algorithms</code> submodule, then corrected into a new sibling crate,
<strong><code>obikindexer</code></strong>, holding <code>obikindexer::algorithms::{partitionner,
dereplicator}</code> and depending on <code>obikindex</code> — never the reverse, same
dependency direction <code>obikpartitionner</code>/<code>obikderep</code> already had. Read
<code>obikpartitionner::X</code> as <code>obikindexer::algorithms::partitionner::X</code> and
<code>obikderep::X</code> as <code>obikindexer::algorithms::dereplicator::X</code> throughout
what follows. The distinction the mistake surfaced, worth keeping: data
crates (<code>obikindex</code>, holding the <code>index</code>/<code>partition</code>/<code>layer</code> model) merge
naturally into one crate as submodules; algorithm crates that operate on
that model from outside stay separate, so the dependency only ever runs
one way.</p>
<p>Status (2026-08-20, latest pass): (1) done — <code>obilayeredmap::Layer</code>
exists, <code>Mat</code> is gone. (1b) done — <code>Layer::Empty</code>, the first non-ready
state, added (panics on every read method). (2a) done — the
@@ -1872,7 +2119,24 @@ turned out to be bigger than <code>KmerPartition</code> alone: <code>Layer</code
constructors don't self-name either. Full redesign of both, agreed in
detail, session ended (budget) before implementation — see "(5) design
agreed" below; <strong>read it before touching <code>KmerPartition</code>/<code>Layer</code>
signatures</strong>, the shape is fully specified. Earlier mix-up, for
signatures</strong>, the shape is fully specified. (6) done — <code>Counter</code>, a third
algorithm, extracted from <code>PartitionRouter</code> the same way <code>Dereplicator</code>
was in (4). (7) done — <code>LayerBuilder</code>, the fourth and last pipeline
algorithm; the indexing pipeline is now fully decomposed into
<code>obikindexer::algorithms::{partitionner, dereplicator, counter,
layer_builder}</code>. (8) design agreed, item 1 done in (9) —
<code>obikindexer::extensions::PrivateBuilder</code>, private, six construction-only
<code>KmerIndex</code> methods moved out. (10) done, same session — <code>obikindex::
IndexBuilder</code>, public, the four maintenance methods
(<code>clear_output_for_create</code>/<code>create_skeleton</code>/<code>finalize_indexed</code>/<code>state</code>)
shared with <code>merge</code>/<code>select</code>/<code>rebuild</code>/<code>reindex</code>. Item 2 from (8)
(<code>obikalgorithm::Algorithm</code>) done in (12) — new crate, <code>type Output</code> +
<code>fn run(&amp;mut self) -&gt; SKResult&lt;Self::Output&gt;</code>, <code>on_progress</code> moved off
<code>run()</code>'s signature entirely into a per-algorithm <code>.on_progress(...)</code>
setter. Note: <code>Layer</code> renamed
<code>KmerLayer</code> (2026-08-21, outside this conversation). (5) itself still not
implemented, still first on the "order of remaining work" list. Earlier
mix-up, for
context: an earlier
version of this doc used the name <code>KmerPartition</code> (singular) for what was
actually the <em>collection</em> type (later renamed <code>KmerPartitions</code>, later
@@ -2515,6 +2779,622 @@ itself or a new crate.</p>
examples were judged not enough to be sure of the shape (<code>Fn+Sync</code> vs
<code>FnMut</code> callback bound already diverged between the two that exist).</li>
</ol>
<h2 id="6-done-2026-08-21-counter-third-algorithm-extracted-the-same-way-as-dereplicator">(6) done (2026-08-21): <code>Counter</code> — third algorithm, extracted the same way as <code>Dereplicator</code></h2>
<p>Between (5) and this, the user did a session of their own crate
restructuring (see the two "Superseded" notes at the top of this file):
<code>obikpartition</code>/<code>obilayeredmap</code> folded into <code>obikindex</code> as submodules
(<code>obikindex::partition</code>, <code>obikindex::layer</code>), and <code>obikpartitionner</code>/
<code>obikderep</code> merged into one sibling crate, <code>obikindexer</code>, holding
<code>obikindexer::algorithms::{partitionner, dereplicator}</code>. (5)'s design
(<code>Layer</code>/<code>KmerPartition</code> self-naming by number, <code>PartitionRouter</code>'s
<code>&amp;mut</code> → <code>&amp;</code> fix) was <strong>not</strong> part of that — pure crate/module packaging,
confirmed by reading the actual code (<code>Layer::open</code>/<code>create</code> still take an
external <code>dir: &amp;Path</code>, <code>KmerPartition</code> still eagerly opens all layers,
<code>PartitionRouter</code> still holds <code>&amp;mut KmerIndex</code>). (5) remains exactly as
specified, not yet implemented.</p>
<p>This step: <code>count_kmer</code> (still living on <code>PartitionRouter</code>, per (4)'s own
"still not done" note) extracted into <code>obikindexer::algorithms::counter::
Counter</code>, mirroring <code>Dereplicator</code> exactly — third data point for the
eventual <code>obikalgorithm</code> trait, still not extracted (still only 3 examples
with 2 different callback bounds; holding off per (5)'s "order of
remaining work").</p>
<div class="highlight"><pre><span></span><code><span class="k">pub</span><span class="w"> </span><span class="k">struct</span><span class="w"> </span><span class="nc">Counter</span><span class="o">&lt;&#39;</span><span class="na">a</span><span class="o">&gt;</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">index</span><span class="p">:</span><span class="w"> </span><span class="kp">&amp;</span><span class="o">&#39;</span><span class="na">a</span><span class="w"> </span><span class="nc">KmerIndex</span><span class="p">,</span>
<span class="w"> </span><span class="n">n_partitions</span><span class="p">:</span><span class="w"> </span><span class="kt">usize</span><span class="p">,</span>
<span class="w"> </span><span class="n">keep_partial</span><span class="p">:</span><span class="w"> </span><span class="kt">bool</span><span class="p">,</span>
<span class="p">}</span>
<span class="k">impl</span><span class="o">&lt;&#39;</span><span class="na">a</span><span class="o">&gt;</span><span class="w"> </span><span class="n">Counter</span><span class="o">&lt;&#39;</span><span class="na">a</span><span class="o">&gt;</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">pub</span><span class="w"> </span><span class="k">fn</span><span class="w"> </span><span class="nf">new</span><span class="p">(</span><span class="n">index</span><span class="p">:</span><span class="w"> </span><span class="kp">&amp;</span><span class="o">&#39;</span><span class="na">a</span><span class="w"> </span><span class="nc">KmerIndex</span><span class="p">)</span><span class="w"> </span><span class="p">-&gt;</span><span class="w"> </span><span class="nc">Self</span><span class="p">;</span>
<span class="w"> </span><span class="k">pub</span><span class="w"> </span><span class="k">fn</span><span class="w"> </span><span class="nf">keep_partial</span><span class="p">(</span><span class="k">mut</span><span class="w"> </span><span class="bp">self</span><span class="p">,</span><span class="w"> </span><span class="n">v</span><span class="p">:</span><span class="w"> </span><span class="kt">bool</span><span class="p">)</span><span class="w"> </span><span class="p">-&gt;</span><span class="w"> </span><span class="nc">Self</span><span class="p">;</span><span class="w"> </span><span class="c1">// setter, mirrors PartitionRouter&#39;s style; defaults to false</span>
<span class="w"> </span><span class="k">pub</span><span class="w"> </span><span class="k">fn</span><span class="w"> </span><span class="nf">run</span><span class="p">(</span><span class="o">&amp;</span><span class="bp">self</span><span class="p">,</span><span class="w"> </span><span class="n">on_progress</span><span class="p">:</span><span class="w"> </span><span class="nb">Option</span><span class="o">&lt;</span><span class="k">impl</span><span class="w"> </span><span class="nb">Fn</span><span class="p">(</span><span class="n">Progress</span><span class="p">)</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="nb">Sync</span><span class="o">&gt;</span><span class="p">)</span><span class="w"> </span><span class="p">-&gt;</span><span class="w"> </span><span class="nc">SKResult</span><span class="o">&lt;</span><span class="n">KmerSpectrum</span><span class="o">&gt;</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div>
<p>Same shape as <code>Dereplicator</code> throughout: <code>Fn(Progress) + Sync</code> (not
<code>FnMut</code>) since counting is also a parallel <code>par_iter</code> over partitions, an
<code>AtomicU64</code> position counter incremented from inside the parallel closure
so progress reports arrive in real time rather than bursting at the end
once <code>.collect()</code> finishes, <code>total: Some(n_partitions)</code> (known up front).
<code>KmerSpectrum</code> (the <code>{f0, f1, counts}</code> aggregate) moved from
<code>partitionner::router</code> to <code>counter</code>, since it's <code>Counter::run</code>'s return
value now, not <code>PartitionRouter</code>'s. <code>count.rs</code>/<code>kmer_sort.rs</code> moved
verbatim from <code>partitionner/</code> to <code>counter/</code> (unchanged bodies — only
<code>count_kmer</code> itself, <code>KmerSpectrum</code>, and the imports they pulled in were
removed from <code>router.rs</code>).</p>
<p>One divergence from <code>Dereplicator</code>: a <code>keep_partial</code> setter exists (no
equivalent on <code>Dereplicator</code>, which has no setters at all) — a real,
already-present parameter (<code>keep_intermediate</code> at the CLI), not a
speculative addition.</p>
<p><code>count_kmer</code>'s three former callers (<code>obikmer::cmd::index</code>, <code>obikphylo</code>'s
test harness, <code>obikindexer::algorithms::partitionner</code>'s own
<code>pipeline_counts</code> test helper) all updated to <code>Counter::new(&amp;idx).
run(...)</code> — the last one simplified further: it used to read back
<code>kmer_spectrum_raw.json</code> from disk after calling <code>count_partition</code>
directly (white-box), now it just uses the <code>KmerSpectrum</code> <code>Counter::run</code>
already returns.</p>
<p>Full workspace suite green (<code>cargo check --workspace --all-targets</code> +
<code>cargo test --workspace</code>, exit code 0), plus an end-to-end CLI smoke test
against real FASTA data (scatter → dereplicate → count → index-build →
query) — required every time per (3)'s lesson, and it earned its keep
again: the very first smoke-test query returned zero matches, which
looked like a regression until traced to the query sequence itself being
low-complexity ("GGCCCCCCACG", six same-base runs) and rejected by
<em>query's own</em> default entropy threshold — nothing to do with this change.
Re-tested with a different substring, confirmed working (kmer found,
count matched the index).</p>
<p>Still not done: (5) (<code>Layer</code>/<code>KmerPartition</code> redesign, <code>PartitionRouter</code>'s
<code>&amp;mut</code>→<code>&amp;</code>), the future cache crate, <code>build_layers</code> (still a <code>KmerIndex</code>
inherent method, not an algorithm), and <code>obikalgorithm</code> itself.</p>
<h2 id="7-done-2026-08-21-layerbuilder-fourth-and-last-pipeline-algorithm">(7) done (2026-08-21): <code>LayerBuilder</code> — fourth and last pipeline algorithm</h2>
<p>Closes out the indexing pipeline: <code>build_layers</code>/<code>build_index_layer</code>
(the last stage still living as <code>KmerIndex</code> inherent methods, flagged as
inconsistent since (6)) extracted into <code>obikindexer::algorithms::
layer_builder::LayerBuilder</code>, same two-phase shape as the other three.</p>
<div class="highlight"><pre><span></span><code><span class="k">pub</span><span class="w"> </span><span class="k">struct</span><span class="w"> </span><span class="nc">LayerBuilder</span><span class="o">&lt;&#39;</span><span class="na">a</span><span class="o">&gt;</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="n">index</span><span class="p">:</span><span class="w"> </span><span class="kp">&amp;</span><span class="o">&#39;</span><span class="na">a</span><span class="w"> </span><span class="nc">KmerIndex</span><span class="p">,</span>
<span class="w"> </span><span class="n">n_partitions</span><span class="p">:</span><span class="w"> </span><span class="kt">usize</span><span class="p">,</span>
<span class="w"> </span><span class="n">min_abundance</span><span class="p">:</span><span class="w"> </span><span class="kt">u32</span><span class="p">,</span>
<span class="w"> </span><span class="n">max_abundance</span><span class="p">:</span><span class="w"> </span><span class="nb">Option</span><span class="o">&lt;</span><span class="kt">u32</span><span class="o">&gt;</span><span class="p">,</span>
<span class="w"> </span><span class="n">keep_intermediate</span><span class="p">:</span><span class="w"> </span><span class="kt">bool</span><span class="p">,</span>
<span class="p">}</span>
<span class="k">impl</span><span class="o">&lt;&#39;</span><span class="na">a</span><span class="o">&gt;</span><span class="w"> </span><span class="n">LayerBuilder</span><span class="o">&lt;&#39;</span><span class="na">a</span><span class="o">&gt;</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">pub</span><span class="w"> </span><span class="k">fn</span><span class="w"> </span><span class="nf">new</span><span class="p">(</span><span class="n">index</span><span class="p">:</span><span class="w"> </span><span class="kp">&amp;</span><span class="o">&#39;</span><span class="na">a</span><span class="w"> </span><span class="nc">KmerIndex</span><span class="p">)</span><span class="w"> </span><span class="p">-&gt;</span><span class="w"> </span><span class="nc">Self</span><span class="p">;</span>
<span class="w"> </span><span class="k">pub</span><span class="w"> </span><span class="k">fn</span><span class="w"> </span><span class="nf">min_abundance</span><span class="p">(</span><span class="k">mut</span><span class="w"> </span><span class="bp">self</span><span class="p">,</span><span class="w"> </span><span class="n">v</span><span class="p">:</span><span class="w"> </span><span class="kt">u32</span><span class="p">)</span><span class="w"> </span><span class="p">-&gt;</span><span class="w"> </span><span class="nc">Self</span><span class="p">;</span>
<span class="w"> </span><span class="k">pub</span><span class="w"> </span><span class="k">fn</span><span class="w"> </span><span class="nf">max_abundance</span><span class="p">(</span><span class="k">mut</span><span class="w"> </span><span class="bp">self</span><span class="p">,</span><span class="w"> </span><span class="n">v</span><span class="p">:</span><span class="w"> </span><span class="nb">Option</span><span class="o">&lt;</span><span class="kt">u32</span><span class="o">&gt;</span><span class="p">)</span><span class="w"> </span><span class="p">-&gt;</span><span class="w"> </span><span class="nc">Self</span><span class="p">;</span>
<span class="w"> </span><span class="k">pub</span><span class="w"> </span><span class="k">fn</span><span class="w"> </span><span class="nf">keep_intermediate</span><span class="p">(</span><span class="k">mut</span><span class="w"> </span><span class="bp">self</span><span class="p">,</span><span class="w"> </span><span class="n">v</span><span class="p">:</span><span class="w"> </span><span class="kt">bool</span><span class="p">)</span><span class="w"> </span><span class="p">-&gt;</span><span class="w"> </span><span class="nc">Self</span><span class="p">;</span>
<span class="w"> </span><span class="k">pub</span><span class="w"> </span><span class="k">fn</span><span class="w"> </span><span class="nf">run</span><span class="p">(</span><span class="o">&amp;</span><span class="bp">self</span><span class="p">,</span><span class="w"> </span><span class="n">on_progress</span><span class="p">:</span><span class="w"> </span><span class="nb">Option</span><span class="o">&lt;</span><span class="k">impl</span><span class="w"> </span><span class="nb">FnMut</span><span class="p">(</span><span class="n">Progress</span><span class="p">)</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="nb">Send</span><span class="o">&gt;</span><span class="p">)</span><span class="w"> </span><span class="p">-&gt;</span><span class="w"> </span><span class="nc">SKResult</span><span class="o">&lt;</span><span class="kt">usize</span><span class="o">&gt;</span><span class="p">;</span><span class="w"> </span><span class="c1">// returns total kmers built</span>
<span class="p">}</span>
</code></pre></div>
<p><strong>Different from all three prior extractions in one respect, deliberately
not "fixed" to match them</strong>: the actual per-partition construction logic
(De Bruijn graph from dereplicated superkmers + provisional counts →
unitigs → MPHF → matrix) stayed put as <code>KmerIndex::build_index_layer</code>/
<code>remove_build_artifacts</code> (both already <code>pub</code>) — not moved into
<code>obikindexer</code>. Checked first: unlike <code>dereplicate_partition</code>/
<code>count_partition</code> (which only ever had one caller), <code>build_index_layer</code>
depends on several <code>obikindex</code>-internal helpers (<code>graph_pipeline::
{write_graph_as_unitigs, materialize_layer}</code>, <code>common::olm_to_sk</code>) that
are <code>pub(crate)</code> and shared with <code>merge</code>/<code>select</code>/<code>rebuild</code>'s own
layer-construction paths — moving <code>build_index_layer</code> out would have
meant either exporting that internal surface just for this one algorithm
or duplicating it. Neither was needed: <code>build_index_layer</code>/
<code>remove_build_artifacts</code> were <em>already</em> public <code>KmerIndex</code> methods, so
<code>LayerBuilder</code>'s job is purely the orchestration around them (scheduling,
config, progress) — the exact same "algorithm calls already-public
<code>KmerIndex</code> primitives" shape <code>PartitionRouter</code>/<code>Dereplicator</code>/<code>Counter</code>
already have, just at a coarser grain for this one stage. This is the
"is the producer's API actually deficient?" check from
[[feedback_no_spaghetti_petits_pois]] applied and answered "no" — not
skipped.</p>
<p><strong>Two more real divergences, both forced by <code>PartitionRunner</code>, not
arbitrary:</strong>
- Uses <code>obikindex::PartitionRunner</code> (NUMA-aware scheduler, already
<code>pub use</code>d from <code>obikindex</code>) instead of plain <code>rayon::into_par_iter</code>
like <code>Dereplicator</code>/<code>Counter</code> — matches what <code>build_layers</code> already used
before extraction; this stage is more CPU/memory-intensive per partition
(graph construction) than scatter/dereplicate/count.
- Callback bound is <code>FnMut(Progress) + Send</code> — a third variant, not
matching either prior shape. <code>PartitionRunner::run</code>'s <code>on_done</code> is
invoked from its own single controller thread (never concurrently, so
no <code>Sync</code> needed, unlike <code>Dereplicator</code>/<code>Counter</code>'s <code>Fn + Sync</code>), but
that controller thread is itself <code>std::thread::scope</code>-spawned, so the
closure still has to be <code>Send</code> to cross into it — caught immediately by
the compiler (<code>cannot be sent between threads safely</code>) when <code>Send</code> was
first omitted, not a design guess. <code>obikalgorithm</code>'s eventual shared
trait now has three real callback-bound data points to reconcile
(<code>FnMut</code> alone for <code>PartitionRouter::run</code>'s sequential loop, <code>FnMut +
Send</code> here, <code>Fn + Sync</code> for <code>Dereplicator</code>/<code>Counter</code>'s <code>rayon</code>
<code>par_iter</code>), not two.</p>
<p><code>KmerIndex::build_layers</code> deleted outright (<code>KmerIndex</code> stays a pure data
structure — no compute orchestration methods, consistent with <code>dereplicate</code>/
<code>count_kmer</code>'s removal in (4)/(6)). New <code>KmerIndex::mark_indexed()</code> added,
symmetric to <code>mark_scattered</code>/<code>mark_counted</code>, replacing the inline
<code>touch(SENTINEL_INDEXED)</code> that used to live inside <code>build_layers</code>.
<code>Stage::start("index")</code>/<code>rep.push(...)</code> and the <code>progress_bar</code>/
<code>"{n} total kmers indexed"</code> log line both moved to <code>cmd/index/mod.rs</code>,
same pattern as (3)/(4)/(6) — <code>LayerBuilder</code> renders nothing itself, just
reports <code>Progress</code>.</p>
<p>All callers updated: <code>cmd/index/mod.rs</code> (Stage 3), <code>obikphylo</code>'s test
harness (also gained a <code>mark_indexed()</code> call it was missing — harmless
before since nothing checked <code>IndexState::Indexed</code> in that test, but now
correct).</p>
<p>Full workspace suite green (<code>cargo check --workspace --all-targets</code> +
<code>cargo test --workspace</code>, exit code 0), plus the end-to-end CLI smoke test
(<code>scripts/smoke_test_index.sh</code>, built earlier specifically so this
verification step is a one-liner from now on) — 870 kmers indexed, query
round-trip confirmed, same numbers as (6).</p>
<p><strong>The indexing pipeline is now fully decomposed</strong>: <code>obikindexer::
algorithms::{partitionner, dereplicator, counter, layer_builder}</code>, each a
<code>new</code>/(setters)/<code>run</code> algorithm operating on a <code>&amp;KmerIndex</code> (or <code>&amp;mut</code> for
<code>PartitionRouter</code>, not yet fixed — see (5)), <code>KmerIndex</code> itself holding no
pipeline-orchestration logic anymore. Still not done: (5), the future
cache crate, <code>obikalgorithm</code> (now unblocked — three real callback-bound
variants observed, worth revisiting whether a single trait can express
all three or whether that's itself the answer: it can't, and the trait
should not force it).</p>
<h2 id="8-design-agreed-not-yet-implemented-2026-08-21-obikalgorithmalgorithm-trait-obikindexerextensions-privatepublic-extension-trait-split-kmerlayer-rename">(8) design agreed, not yet implemented (2026-08-21): <code>obikalgorithm::Algorithm</code> trait + <code>obikindexer::extensions</code> — private/public extension-trait split, <code>KmerLayer</code> rename</h2>
<p>Session note: <code>Layer</code> was renamed <code>KmerLayer</code> (user, outside this
conversation, alongside other naming homogenisation with <code>KmerIndex</code>/
<code>KmerPartition</code>) — every reference to <code>Layer</code> in this doc from before
2026-08-21 means today's <code>obikindex::layer::KmerLayer</code>.</p>
<h3 id="why-this-came-up">Why this came up</h3>
<p>Verifying "does <code>cmd/index</code> now rest entirely on the algorithm structs"
(it doesn't quite — see below) led to sorting <code>KmerIndex</code>'s own methods by
a criterion the user was explicit is <strong>semantic, not mechanical</strong>: "les
méthodes qui, sémantiquement, n'ont pas d'intérêt hors de la construction
de l'index" (methods that have no semantic interest outside index
construction) — not "methods only called from <code>cmd/index</code> today," which
a grep could answer but would miss methods construction-adjacent code
elsewhere (<code>merge</code>/<code>select</code>/<code>rebuild</code>/<code>reindex</code>) also depends on for the
same reason.</p>
<p><strong>Checked, not assumed</strong> (grepped every call site before classifying):</p>
<ul>
<li><strong>Construction-only, real candidates for a private extension trait</strong>:
<code>KmerIndex::{mark_scattered, mark_counted, mark_indexed, write_spectrum,
build_index_layer, remove_build_artifacts, clear_output_for_create,
create_skeleton, finalize_indexed, state}</code>. The last four are called
from <code>merge.rs</code>/<code>select.rs</code>/<code>rebuild.rs</code>/<code>reindex.rs</code> too (as
precondition checks — "is my source <code>Indexed</code>?" — or shared
skeleton/finalize machinery), not just from the 4-stage pipeline — so
this extension trait's scope is "construction of any kind," not
narrowly "the initial build pipeline."</li>
<li><strong>Looked construction-only by name, checked, and kept on <code>KmerIndex</code></strong>:
<code>layer_unitigs_path</code> (unitigs are the only way to recover a built
index's kmer sequences — read by <code>rebuild_layer.rs</code> and others, well
beyond construction — see [[project_unitigs_always_kept]]),
<code>pack_matrices</code> (re-runnable maintenance on an already-finished index
via <code>obikmer pack</code>, not just a pipeline step), <code>upgrade_layer_meta</code>
(migration, runnable on any existing index at any time).</li>
</ul>
<h3 id="the-general-pattern-not-obikindexer-specific">The general pattern (not obikindexer-specific)</h3>
<p><code>KmerIndex</code>/<code>KmerPartition</code>/<code>KmerLayer</code> stay generic, in <code>obikindex</code> —
every domain-specific consumer crate gets to attach its own extension
trait(s), of two kinds:</p>
<ul>
<li><strong>Private</strong> (<code>pub(crate)</code>, invisible outside the defining crate) — for
plumbing only that crate's own algorithms need. <code>obikindexer</code> gets
exactly one of these (see below); no public counterpart makes sense for
it — "l'index est tellement central que le second trait n'a pas
vraiment d'intérêt" for construction specifically: nothing external
should ever want to call <code>mark_scattered</code> or <code>build_index_layer</code>.</li>
<li><strong>Public</strong> — for a genuinely reusable domain extension. The user's own
example, found while discussing this, not hypothetical: <code>obikindex/src/
index/distance.rs</code> (phylogenetic distance metrics) is currently an
<code>impl KmerIndex</code> block <strong>inside <code>obikindex</code> itself</strong> — under this
principle it should be a public extension trait owned by <code>obikphylo</code>
instead (distance metrics are a phylo concept, <code>obikindex</code> has no more
business defining them than <code>obikindex::layer</code> has defining
"family"/"minorant", the reasoning <code>SiblingLayerExt</code> already followed
for <code>KmerLayer</code> — see <code>obikphylo/src/siblings/iter.rs</code>). <strong>Explicitly
deferred</strong> — noted here so it isn't lost, not part of this round.</li>
<li>The future cache-manager crate (still blocked on (5), see above) will
add its own <strong>public</strong> extension trait mirroring part of <code>KmerIndex</code>'s/
<code>KmerPartition</code>'s own read API in cached form (e.g. a cached
<code>.partition(i)</code> that doesn't re-touch disk) — same pattern, third data
point once built.</li>
</ul>
<h3 id="concretely-next-to-implement-two-items-in-order">Concretely, next to implement (two items, in order)</h3>
<ol>
<li><strong><code>obikindexer::extensions</code></strong> — a private (<code>pub(crate)</code>) extension
trait, most likely named something like <code>IndexBuildExt</code> (final name
not yet chosen), implemented for <code>KmerIndex</code>, carrying the ten methods
listed above, moved out of <code>obikindex::index::{kmer_index,
index_layer}</code>. Every algorithm in <code>obikindexer::algorithms::*</code> that
currently calls <code>idx.mark_scattered()</code>/etc. keeps the same call syntax
(extension trait methods are called the same way as inherent ones,
just need the trait in scope) — <code>cmd/index/mod.rs</code> itself would need
<code>use obikindexer::extensions::IndexBuildExt;</code> (or the module re-exports
it) to keep compiling, since it's the one place outside <code>obikindexer</code>'s
own algorithms that currently calls <code>mark_scattered</code>/<code>write_spectrum</code>/
<code>mark_counted</code>/<code>mark_indexed</code> directly. <strong>Not yet decided</strong>: exact
trait name, whether it's one trait or split further (e.g. sentinel
marking vs. skeleton/finalize machinery), and whether <code>merge</code>/<code>select</code>/
<code>rebuild</code>/<code>reindex</code> (not yet extracted into algorithms themselves) move
onto it now too or keep calling the soon-to-be-inherent-no-longer
methods some other way in the meantime — <strong>ask before implementing</strong>,
this changes the blast radius significantly (4 more <code>obikindex</code>
internal files depend on <code>clear_output_for_create</code>/<code>create_skeleton</code>/
<code>finalize_indexed</code>/<code>state</code>).</li>
<li><strong><code>obikalgorithm::Algorithm</code> trait</strong> — new crate, the shared trait
<code>obikpartitionner</code>→<code>obikindexer</code> merge (session start of 2026-08-21)
and (6)/(7) were deliberately building toward, now with four real
<code>new</code>/(setters)/<code>run</code> examples and three distinct callback-bound
shapes to reconcile (plain <code>FnMut</code> for <code>PartitionRouter</code>, <code>FnMut +
Send</code> for <code>LayerBuilder</code>, <code>Fn + Sync</code> for <code>Dereplicator</code>/<code>Counter</code> —
see (7)). Exact shape not yet drafted in this doc — do that as its own
design pass before coding, same discipline as everything above.</li>
</ol>
<p>Both items: <strong>design only, nothing implemented yet</strong> — this section is
the record to resume from, not a plan already executed.</p>
<h2 id="9-done-2026-08-21-obikindexerextensionsprivatebuilder-item-1-above-implemented">(9) done (2026-08-21): <code>obikindexer::extensions::PrivateBuilder</code> — item 1 above, implemented</h2>
<p>Renamed from <code>IndexBuilder</code> to <code>PrivateBuilder</code> immediately after (same
session), freeing the name <code>IndexBuilder</code> for (10)'s public trait — read
<code>IndexBuilder</code> below as <code>PrivateBuilder</code> throughout this section.</p>
<p>Scoped down from (8)'s six-method list to the concrete set that's
genuinely movable without further ripple — checked, not assumed, before
writing anything:</p>
<div class="highlight"><pre><span></span><code><span class="k">pub</span><span class="p">(</span><span class="k">crate</span><span class="p">)</span><span class="w"> </span><span class="k">trait</span><span class="w"> </span><span class="n">PrivateBuilder</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">fn</span><span class="w"> </span><span class="nf">mark_scattered</span><span class="p">(</span><span class="o">&amp;</span><span class="k">mut</span><span class="w"> </span><span class="bp">self</span><span class="p">)</span><span class="w"> </span><span class="p">-&gt;</span><span class="w"> </span><span class="nc">OKIResult</span><span class="o">&lt;</span><span class="p">()</span><span class="o">&gt;</span><span class="p">;</span>
<span class="w"> </span><span class="k">fn</span><span class="w"> </span><span class="nf">mark_counted</span><span class="p">(</span><span class="o">&amp;</span><span class="bp">self</span><span class="p">)</span><span class="w"> </span><span class="p">-&gt;</span><span class="w"> </span><span class="nc">OKIResult</span><span class="o">&lt;</span><span class="p">()</span><span class="o">&gt;</span><span class="p">;</span>
<span class="w"> </span><span class="k">fn</span><span class="w"> </span><span class="nf">mark_indexed</span><span class="p">(</span><span class="o">&amp;</span><span class="bp">self</span><span class="p">)</span><span class="w"> </span><span class="p">-&gt;</span><span class="w"> </span><span class="nc">OKIResult</span><span class="o">&lt;</span><span class="p">()</span><span class="o">&gt;</span><span class="p">;</span>
<span class="w"> </span><span class="k">fn</span><span class="w"> </span><span class="nf">write_spectrum</span><span class="p">(</span><span class="o">&amp;</span><span class="bp">self</span><span class="p">,</span><span class="w"> </span><span class="n">f0</span><span class="p">:</span><span class="w"> </span><span class="kt">u64</span><span class="p">,</span><span class="w"> </span><span class="n">f1</span><span class="p">:</span><span class="w"> </span><span class="kt">u64</span><span class="p">,</span><span class="w"> </span><span class="n">counts</span><span class="p">:</span><span class="w"> </span><span class="kp">&amp;</span><span class="nc">BTreeMap</span><span class="o">&lt;</span><span class="kt">u32</span><span class="p">,</span><span class="w"> </span><span class="kt">u64</span><span class="o">&gt;</span><span class="p">)</span><span class="w"> </span><span class="p">-&gt;</span><span class="w"> </span><span class="nc">OKIResult</span><span class="o">&lt;</span><span class="p">()</span><span class="o">&gt;</span><span class="p">;</span>
<span class="w"> </span><span class="k">fn</span><span class="w"> </span><span class="nf">build_index_layer</span><span class="p">(</span><span class="o">&amp;</span><span class="bp">self</span><span class="p">,</span><span class="w"> </span><span class="n">i</span><span class="p">:</span><span class="w"> </span><span class="kt">usize</span><span class="p">,</span><span class="w"> </span><span class="n">min_ab</span><span class="p">:</span><span class="w"> </span><span class="kt">u32</span><span class="p">,</span><span class="w"> </span><span class="n">max_ab</span><span class="p">:</span><span class="w"> </span><span class="nb">Option</span><span class="o">&lt;</span><span class="kt">u32</span><span class="o">&gt;</span><span class="p">,</span><span class="w"> </span><span class="n">with_counts</span><span class="p">:</span><span class="w"> </span><span class="kt">bool</span><span class="p">,</span><span class="w"> </span><span class="n">mode</span><span class="p">:</span><span class="w"> </span><span class="kp">&amp;</span><span class="nc">IndexMode</span><span class="p">,</span><span class="w"> </span><span class="n">block_bits</span><span class="p">:</span><span class="w"> </span><span class="kt">u8</span><span class="p">)</span><span class="w"> </span><span class="p">-&gt;</span><span class="w"> </span><span class="nb">Result</span><span class="o">&lt;</span><span class="kt">usize</span><span class="p">,</span><span class="w"> </span><span class="n">SKError</span><span class="o">&gt;</span><span class="p">;</span>
<span class="w"> </span><span class="k">fn</span><span class="w"> </span><span class="nf">remove_build_artifacts</span><span class="p">(</span><span class="o">&amp;</span><span class="bp">self</span><span class="p">,</span><span class="w"> </span><span class="n">i</span><span class="p">:</span><span class="w"> </span><span class="kt">usize</span><span class="p">);</span>
<span class="p">}</span>
<span class="k">impl</span><span class="w"> </span><span class="n">PrivateBuilder</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="n">KmerIndex</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="o">..</span><span class="p">.</span><span class="w"> </span><span class="p">}</span>
</code></pre></div>
<p>All six moved bodily out of <code>obikindex::index::{kmer_index, index_layer}</code>
into <code>obikindexer::extensions</code> (new module, <code>pub(crate)</code>) — <code>index_layer.rs</code>
is now empty and deleted outright.
<code>clear_output_for_create</code>/<code>create_skeleton</code>/<code>finalize_indexed</code>/<code>state</code>
stayed inherent on <code>KmerIndex</code>, per (8)'s reasoning: <code>merge</code>/<code>select</code>/
<code>rebuild</code>/<code>reindex</code> — living <em>inside</em> <code>obikindex</code> itself — call them too,
and <code>obikindex</code> can never depend on <code>obikindexer</code> to reach a trait defined
there. Moving those four is real future work (extract
merge/select/rebuild/reindex into algorithms first), not part of this
step.</p>
<p><strong>One new, small, deliberate API widening in <code>obikindex</code></strong>: <code>build_index_layer</code>
depends on three helpers that were <code>pub(crate)</code> to <code>obikindex</code>
(<code>graph_pipeline::{write_graph_as_unitigs, materialize_layer}</code>,
<code>common::olm_to_sk</code>) — widened to <code>pub</code> (re-exported from <code>obikindex</code>'s
crate root) so <code>obikindexer</code> could reach them. This is exactly the
"enrich shared/lower-level APIs instead of ad hoc local code" call the
project's own rules ask for, made explicitly rather than routed around:
three functions, already generically written (no rewrite needed), now
serve a second caller instead of being duplicated.</p>
<p><strong>Why the trait had to be defined in <code>obikindexer</code>, not <code>obikindex</code></strong>:
Rust's orphan rule — implementing a trait for a foreign type requires
either the trait or the type to be local to the current crate. <code>KmerIndex</code>
is foreign to <code>obikindexer</code>, so the trait must be the local half; if it
were defined in <code>obikindex</code> instead, <code>pub(crate)</code> there would make it
invisible to <code>obikindexer</code> too (crate-private means private to <em>that</em>
crate, not "private except to one named dependent") — the opposite of
what was wanted.</p>
<p><strong>A real design decision made while wiring callers up, not a mechanical
rename</strong>: <code>PrivateBuilder</code> being genuinely <code>pub(crate)</code> to <code>obikindexer</code>
means <code>obikmer::cmd::index</code> (a different crate) can no longer call
<code>mark_scattered</code>/<code>mark_counted</code>/<code>mark_indexed</code>/<code>write_spectrum</code> directly —
it never could have, once privacy was real rather than aspirational. Each
algorithm now marks its own completion as part of <code>run()</code>/<code>close()</code>
instead of leaving it to the caller:
- <code>PartitionRouter::close()</code> (not <code>run()</code>) calls <code>mark_scattered()</code> —
<code>close()</code>, not <code>run()</code>, is the actual shared completion point between
the file-driven <code>run()</code> path and the manual <code>write</code>/<code>write_batch</code>+
<code>close()</code> path low-level callers (tests) use; putting it in <code>run()</code>
alone would have silently skipped marking for every caller that never
calls <code>run()</code>. <code>run()</code> already calls <code>self.close()</code> at its own end, so
this covers both paths through one line, not two.
- <code>Counter::run</code> calls <code>write_spectrum</code> then <code>mark_counted</code> before
returning.
- <code>LayerBuilder::run</code> calls <code>mark_indexed</code> before returning.</p>
<p><code>cmd/index/mod.rs</code> lost all four direct calls (<code>mark_scattered</code>/
<code>write_spectrum</code>/<code>mark_counted</code>/<code>mark_indexed</code>) — each stage's <code>if
idx.state() &lt; IndexState::X { ... }</code> block is now purely "run the
algorithm," no separate bookkeeping call after it. Confirms, precisely
this time (checked by re-reading the whole file, not assumed): <code>cmd/index</code>
now rests on the four algorithms for every read/write of pipeline state
except <code>KmerIndex::{exists, create, state, n_partitions}</code>, which are
genuinely index-identity concerns, not construction bookkeeping — the
original question this whole design pass started from.</p>
<p>Same fix applied to <code>obikphylo</code>'s test harness (its four explicit
<code>mark_*</code>/<code>write_spectrum</code> calls removed, relying on the algorithms now
doing it themselves) — <code>obikindexer::algorithms::partitionner</code>'s own
<code>pipeline_counts</code> test needed no change (never called <code>mark_*</code> directly).</p>
<p>Full workspace suite green (<code>cargo check --workspace --all-targets</code> +
<code>cargo test --workspace</code>, exit code 0), plus the CLI smoke test — 870
kmers, same as (6)/(7).</p>
<p>Still not done at the time of writing: item 2 from (8) (<code>obikalgorithm::
Algorithm</code>), (5), the future cache crate, the <code>distance.rs</code> →
<code>obikphylo</code> relocation (noted in (8), explicitly deferred), and
extracting <code>merge</code>/<code>select</code>/<code>rebuild</code>/<code>reindex</code> into algorithms.</p>
<h2 id="10-done-2026-08-21-obikindexindexbuilder-the-public-counterpart-same-session">(10) done (2026-08-21): <code>obikindex::IndexBuilder</code> — the public counterpart, same session</h2>
<p>Immediate correction to (9): the private trait built there was renamed
<code>PrivateBuilder</code> (freeing the name), and the four methods (9) had left
inherent on <code>KmerIndex</code> — <code>clear_output_for_create</code>/<code>create_skeleton</code>/
<code>finalize_indexed</code>/<code>state</code> — got their own trait after all: <strong><code>IndexBuilder</code></strong>,
public, defined in <code>obikindex</code> itself (not <code>obikindexer</code>):</p>
<div class="highlight"><pre><span></span><code><span class="k">pub</span><span class="w"> </span><span class="k">trait</span><span class="w"> </span><span class="n">IndexBuilder</span><span class="p">:</span><span class="w"> </span><span class="nb">Sized</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">fn</span><span class="w"> </span><span class="nf">clear_output_for_create</span><span class="o">&lt;</span><span class="n">P</span><span class="p">:</span><span class="w"> </span><span class="nb">AsRef</span><span class="o">&lt;</span><span class="n">Path</span><span class="o">&gt;&gt;</span><span class="p">(</span><span class="n">output</span><span class="p">:</span><span class="w"> </span><span class="nc">P</span><span class="p">,</span><span class="w"> </span><span class="n">force</span><span class="p">:</span><span class="w"> </span><span class="kt">bool</span><span class="p">)</span><span class="w"> </span><span class="p">-&gt;</span><span class="w"> </span><span class="nc">OKIResult</span><span class="o">&lt;</span><span class="p">()</span><span class="o">&gt;</span><span class="p">;</span>
<span class="w"> </span><span class="k">fn</span><span class="w"> </span><span class="nf">create_skeleton</span><span class="o">&lt;</span><span class="n">P</span><span class="p">:</span><span class="w"> </span><span class="nb">AsRef</span><span class="o">&lt;</span><span class="n">Path</span><span class="o">&gt;&gt;</span><span class="p">(</span><span class="n">output</span><span class="p">:</span><span class="w"> </span><span class="nc">P</span><span class="p">,</span><span class="w"> </span><span class="n">meta</span><span class="p">:</span><span class="w"> </span><span class="kp">&amp;</span><span class="nc">IndexMeta</span><span class="p">)</span><span class="w"> </span><span class="p">-&gt;</span><span class="w"> </span><span class="nc">OKIResult</span><span class="o">&lt;</span><span class="bp">Self</span><span class="o">&gt;</span><span class="p">;</span>
<span class="w"> </span><span class="k">fn</span><span class="w"> </span><span class="nf">finalize_indexed</span><span class="o">&lt;</span><span class="n">P</span><span class="p">:</span><span class="w"> </span><span class="nb">AsRef</span><span class="o">&lt;</span><span class="n">Path</span><span class="o">&gt;&gt;</span><span class="p">(</span><span class="n">output</span><span class="p">:</span><span class="w"> </span><span class="nc">P</span><span class="p">,</span><span class="w"> </span><span class="n">rep</span><span class="p">:</span><span class="w"> </span><span class="kp">&amp;</span><span class="nc">mut</span><span class="w"> </span><span class="n">Reporter</span><span class="p">)</span><span class="w"> </span><span class="p">-&gt;</span><span class="w"> </span><span class="nc">OKIResult</span><span class="o">&lt;</span><span class="bp">Self</span><span class="o">&gt;</span><span class="p">;</span>
<span class="w"> </span><span class="k">fn</span><span class="w"> </span><span class="nf">state</span><span class="p">(</span><span class="o">&amp;</span><span class="bp">self</span><span class="p">)</span><span class="w"> </span><span class="p">-&gt;</span><span class="w"> </span><span class="nc">IndexState</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">impl</span><span class="w"> </span><span class="n">IndexBuilder</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="n">KmerIndex</span><span class="w"> </span><span class="p">{</span><span class="w"> </span><span class="o">..</span><span class="p">.</span><span class="w"> </span><span class="p">}</span>
</code></pre></div>
<p>User's framing: these four are "maintenance", not "scientific computation
on an index" — a different kind of non-generic-ness than (9)'s six
(<code>mark_*</code>/<code>write_spectrum</code>/<code>build_index_layer</code>/<code>remove_build_artifacts</code>,
exclusive to the 4-stage pipeline). Maintenance is used more broadly
(<code>merge</code>/<code>select</code>/<code>rebuild</code>/<code>reindex</code>), so it gets a real, public trait —
not folded back into <code>KmerIndex</code>'s inherent surface, and not private
either.</p>
<p><strong>Where it lives, and why that's not arbitrary</strong>: (9) needed the orphan
rule to force its trait into <code>obikindexer</code>, to achieve genuine
crate-private visibility. Here the requirement is the opposite:
<code>merge.rs</code>/<code>select.rs</code>/<code>rebuild.rs</code>/<code>reindex.rs</code> — the trait's own
heaviest users — live <em>inside</em> <code>obikindex</code>. A trait they need to reach
must be local to <code>obikindex</code> (or a crate <code>obikindex</code> itself depends on,
which doesn't exist for this). So <code>IndexBuilder</code> lives in a new
<code>obikindex/src/index/builder.rs</code>, <code>pub trait</code> (no orphan-rule tension at
all here — both trait and type are local to the same crate), re-exported
from <code>obikindex</code>'s crate root alongside <code>PrivateBuilder</code>'s sibling
<code>obikindexer::extensions::PrivateBuilder</code> staying where it is. Two
traits, two crates, two different reasons, not a contradiction.</p>
<p><strong>Blast radius, all inside <code>obikindex</code> plus one external crate</strong>: every
internal caller of these four methods needs the trait imported now that
they're no longer inherent — <code>merge.rs</code>, <code>select.rs</code>, <code>rebuild.rs</code>,
<code>reindex.rs</code> (<code>use crate::index::builder::IndexBuilder;</code>) and, externally,
<code>obikmer::cmd::index::mod</code> (<code>use obikindex::IndexBuilder;</code>, for the three
<code>idx.state() &lt; IndexState::X</code> resumability checks). Call syntax at every
site is unchanged (<code>KmerIndex::create_skeleton(...)</code>,
<code>self.state()</code>) — only trait-in-scope requirements are new, which is
exactly the point: same ergonomics, less surface baked into <code>KmerIndex</code>
itself.</p>
<p>Verification went one step further than (9): beyond
<code>cargo check --workspace --all-targets</code> + <code>cargo test --workspace</code> +
<code>scripts/smoke_test_index.sh</code> (all green, 870 kmers again), ran
<code>obikmer merge</code> end to end on two freshly built indexes (exercises
<code>clear_output_for_create</code>/<code>finalize_indexed</code> directly, the two methods
<code>scripts/smoke_test_index.sh</code> itself never touches) — exit 0, <code>pack</code>
stage completed. Test suite alone would not have caught a regression
here: no existing test builds two real indexes and merges them through
the CLI.</p>
<p><code>KmerIndex</code> itself now carries only: identity/config accessors
(<code>root_path</code>/<code>meta</code>/<code>kmer_size</code>/<code>n_bits</code>/<code>evidence_mode</code>/<code>genomes</code>/...),
path resolution (<code>partition_dir</code>/<code>index_dir</code>/<code>layer_dir</code>/
<code>partition_meta</code>/<code>n_layers</code>), and a few index-maintenance operations not
yet sorted into either trait (<code>layer_unitigs_path</code>, <code>pack_matrices</code>,
<code>upgrade_layer_meta</code> — see (8)'s "tested and discarded" list; still
correctly inherent, not construction-only by the semantic criterion) —
<code>create</code>/<code>open</code>/<code>exists</code> (identity, can't be trait methods needing <code>Self</code>
before one exists) round that out.</p>
<h2 id="11-done-2026-08-21-kmerindexindexmeta-made-fully-stateless-indexstate-moved-off-sentinel-files">(11) done (2026-08-21): <code>KmerIndex</code>/<code>IndexMeta</code> made fully stateless, <code>IndexState</code> moved off sentinel files</h2>
<p>Triggered mid-discussion of <code>obikalgorithm::Algorithm</code> (still not started —
see "Still not done" below): user asked why <code>PartitionRouter::new</code> still
took <code>&amp;mut KmerIndex</code> at all, and questioned whether <code>mark_scattered</code>
belonged in the algorithm or in <code>cmd/index</code>. Investigation found <code>&amp;mut</code>
had become <em>newly</em> necessary since (9) — <code>mark_scattered</code> was mutating
<code>self.meta.genomes</code> in memory so <code>Counter</code>'s later <code>write_spectrum</code> call
(same <code>idx</code> instance) would see the derived label. User's resolution: the
"disk is truth, stateless" principle already agreed for <code>KmerPartition</code>/
<code>Layer</code> in (5) (still unimplemented for those two) should extend to
<code>KmerIndex</code> itself — move <code>IndexState</code> (<code>Empty</code>/<code>Scattered</code>/<code>Counted</code>/
<code>Indexed</code>) off the three sentinel files (<code>scatter.done</code>/<code>count.done</code>/
<code>index.done</code>, detected by existence) and into a field of <code>index.meta</code>'s
own JSON, so the <code>mark_*</code> calls become plain disk writes an algorithm can
legitimately make on <code>&amp;self</code> — no in-memory mutation left to protect.</p>
<p><strong>Shape of <code>IndexMeta</code>, per the user's explicit spec</strong>: one JSON file per
index (<code>index.meta</code>), one <code>IndexMeta</code> instance per index, held and
returned as <code>Arc&lt;IndexMeta&gt;</code> (not <code>&amp;IndexMeta</code>) by <code>KmerIndex::meta()</code>.
<code>config</code> (<code>kmer_size</code>/<code>minimizer_size</code>/<code>n_bits</code>/<code>with_counts</code>/<code>evidence</code>/
<code>block_bits</code>) is fixed at construction, cached as a <code>pub</code> field (getter
kept alongside, for symmetry) — "les champs constants restent des champs
de la structure", read once, never re-read from disk. <code>genomes</code> and
<code>state</code> are the opposite: no in-memory cache at all, every accessor
(<code>genomes()</code>, <code>state()</code>) re-reads <code>index.meta</code> from disk, every mutator
(<code>push_genome</code>/<code>rename_genome</code>/<code>set_genomes</code>/<code>set_state</code>/<code>mark_scattered</code>/
<code>mark_counted</code>/<code>mark_indexed</code>) does a full read-modify-write of the same
file. An internal <code>std::sync::RwLock&lt;()&gt;</code> is held across each
read-modify-write sequence (not just the write) so two callers sharing
the same <code>Arc&lt;IndexMeta&gt;</code> can't lose an update to each other — this is
<em>not</em> a cross-process lock (that's <code>obisys::DirLock</code>, already held by
<code>cmd/index</code> for the whole build); it only serialises access through one
shared in-process instance.</p>
<p><strong>Construction, chain-of-responsibility style, matching (5)'s pattern</strong>:
<code>IndexMeta::create(&amp;KmerIndex, config, genomes)</code> / <code>IndexMeta::open(&amp;KmerIndex)</code>
ask the index for its own root path rather than taking one directly. Since
<code>KmerIndex::create</code> doesn't have a complete <code>KmerIndex</code> yet to hand in
(it's what's being built), added lower-level <code>pub(crate)</code> path-based
primitives <code>create_at(&amp;Path, ...)</code> / <code>open_at(&amp;Path)</code> that <code>KmerIndex::create</code>/
<code>open</code> and <code>builder.rs</code>'s <code>create_skeleton</code> call directly, bypassing the
convenience wrappers for that one bootstrap case.</p>
<p><strong>The one deliberate exception</strong>: <code>select_in_place</code> and <code>reindex</code>
genuinely rewrite <code>config</code> after an index already exists (output
type/evidence mode changes in place) — contradicting "config never
changes" for the general case. Resolved with a separate, explicitly
rare-labelled <code>IndexMeta::rewrite_config(config, genomes)</code> (preserves
<code>state</code>, overwrites everything else); callers refresh their own cached
<code>Arc&lt;IndexMeta&gt;</code> afterward (<code>self.meta = Arc::new(IndexMeta::open(self)?)</code>)
since <code>IndexMeta</code> has no way to reach back into whichever <code>KmerIndex</code>
holds it.</p>
<p><strong>Consequence confirmed, not just hoped for</strong>: with <code>mark_scattered</code> no
longer touching anything in memory, <code>PartitionRouter</code> genuinely never
needs <code>&amp;mut KmerIndex</code> — <code>PartitionRouter&lt;'a&gt; { index: &amp;'a KmerIndex }</code>,
<code>new(&amp;'a KmerIndex)</code>. This is effectively the <code>PartitionRouter</code> half of
(5)'s "order of remaining work" item done as a side effect; <code>KmerPartition</code>/
<code>Layer</code> themselves are still unimplemented for (5).</p>
<p><strong>Blast radius — much larger than (9)/(10), touched nearly every crate</strong>:
every <code>.meta().genomes</code>/<code>.meta.genomes</code> field access became a fallible
<code>.genomes()?</code> method call (<code>genomes</code> reads <code>io::Result&lt;Vec&lt;GenomeInfo&gt;&gt;</code>
now, not a field), and <code>.meta_mut()</code> was removed outright (no more direct
field mutation from outside <code>IndexMeta</code>). Fixed across:
- <code>obikindex</code> internals: <code>meta.rs</code>/<code>state.rs</code>/<code>kmer_index.rs</code>/<code>builder.rs</code>
(full rewrites), <code>reindex.rs</code>/<code>select.rs</code> (switched to <code>rewrite_config</code>),
<code>merge.rs</code> (heaviest single file — genome counts precomputed once per
source into a <code>Vec&lt;Vec&lt;GenomeInfo&gt;&gt;</code> up front rather than re-reading
<code>index.meta</code> from disk repeatedly through the function, sentinel write
replaced with <code>dst2.meta.mark_indexed()</code>), <code>stats.rs</code>, <code>distance.rs</code>,
<code>dump.rs</code>, <code>predicate.rs</code> (its <code>IndexMeta</code>-inherent <code>matching_genome_indices</code>/
<code>build_group_filter</code> now read genomes fresh internally), <code>mod.rs</code>/<code>lib.rs</code>
(sentinel constant re-exports removed — <code>IndexState</code> no longer has
<code>SENTINEL_*</code>/<code>detect()</code> at all).
- <code>obikindexer::extensions::PrivateBuilder</code>: <code>mark_scattered</code> signature
dropped <code>&amp;mut self</code> → <code>&amp;self</code>; the four <code>mark_*</code>/<code>write_spectrum</code> bodies
became one-line delegations to <code>self.meta().mark_*()</code>.
- <code>obikphylo::siblings</code>: <code>alignment.rs</code>/<code>cardinality.rs</code>/<code>distance.rs</code>/
<code>entropy.rs</code>/<code>sankoff_bundle.rs</code>/<code>stats.rs</code>/<code>tests.rs</code> — all had
<code>self.meta().genomes.len()</code>-shaped reads, mechanically fixed to
<code>.genomes().map_err(OKIError::Io)?.len()</code> (tests: <code>.unwrap()</code>).
- <code>obikmer::cmd::*</code>: <code>annotate</code> (rewrote its rename path to load genomes
once, mutate the in-memory <code>Vec</code>, then <code>idx.meta().set_genomes(...)</code>
instead of <code>meta_mut()</code>), <code>filter</code>/<code>pack</code>/<code>dump</code>/<code>unitig</code>/<code>merge</code>/<code>select</code>/
<code>phylo</code> (fetch-once-then-use pattern for genome counts/labels),
<code>utils/maintenance.rs</code> (<code>run_rename</code> now calls the pre-existing
<code>IndexMeta::rename_genome</code>, dropping its own hand-rolled field mutation
entirely), <code>index/mod.rs</code> (three <code>idx.state() &lt; IndexState::X</code>
resumability checks needed a fallible read — factored into a small
<code>current_state(&amp;KmerIndex) -&gt; IndexState</code> helper rather than repeating
the same <code>unwrap_or_else</code> three times), <code>query/*</code> (<code>emit_batch</code>'s
signature changed from <code>&amp;IndexMeta</code> to <code>&amp;[GenomeInfo]</code>, and <code>genomes</code> is
now fetched once in <code>run()</code> and threaded down through <code>process_chunk</code>
as <code>Arc&lt;Vec&lt;GenomeInfo&gt;&gt;</code> rather than re-reading <code>index.meta</code> from disk
on every chunk — a deliberate deviation from the "always re-read"
default, justified because this is a genuine per-chunk hot path, unlike
every other call site touched in this pass).
- One <code>&amp;IndexMeta</code>-vs-<code>Arc&lt;IndexMeta&gt;</code> argument-type mismatch pattern
recurred at several CLI call sites (<code>build_filters</code>/<code>build_specs</code>/
<code>emit_batch</code>'s original signature) — resolved via <code>Arc</code>'s deref
coercion (<code>&amp;idx.meta()</code> coerces to <code>&amp;IndexMeta</code>) rather than changing
every downstream signature to accept <code>Arc&lt;IndexMeta&gt;</code>.</p>
<p><strong>Verification</strong>: <code>cargo check --workspace --all-targets</code> and
<code>cargo test --workspace</code> both green (0 failures) after the full
propagation, <code>scripts/smoke_test_index.sh</code> green (870 kmers, same as every
prior round), plus a manual CLI run of <code>index</code> (×2) → <code>merge</code> → <code>select</code>
→ <code>reindex</code> → <code>utils --new-label</code> (rename) → <code>utils --stats</code>, all exit 0,
confirming the four most-affected commands (the ones (10)'s verification
already flagged as under-covered by the automated test suite) still work
end to end against the new <code>Arc&lt;IndexMeta&gt;</code>/on-disk-<code>IndexState</code> shape.</p>
<p>Still not done: (5)'s <code>KmerPartition</code>/<code>Layer</code> self-naming redesign itself
(only the <code>PartitionRouter</code>-<code>&amp;mut</code>-removal piece landed, as a side
effect); the <code>distance.rs</code> → <code>obikphylo</code> relocation ((9), explicitly
deferred); the future cache-manager crate. (8)'s <code>obikalgorithm::
Algorithm</code> trait, resumed and closed in (12) below.</p>
<h2 id="12-done-2026-08-21-obikalgorithmalgorithm-the-shared-trait-resumed-and-closed-in-one-session">(12) done (2026-08-21): <code>obikalgorithm::Algorithm</code> — the shared trait, resumed and closed in one session</h2>
<p>Resumed (8)'s point 2 through a point-by-point discussion of what's
actually common across the four pipeline algorithms, now that (11) made
<code>KmerIndex</code> itself immutable everywhere. Four sub-points, each closed
before moving to the next:</p>
<p><strong>1. Receiver (<code>&amp;self</code> vs <code>&amp;mut self</code>)</strong> — investigated whether (11)'s
removal of <code>&amp;mut KmerIndex</code> also removed the need for <code>PartitionRouter::
run</code> to take <code>&amp;mut self</code>. It didn't: <code>PartitionRouter</code> holds real
per-run state of its own (<code>writers: Vec&lt;Option&lt;SKFileWriter&gt;&gt;</code>, open file
handles, purely in-process RAM — confirmed by checking where <code>writers</code> is
stored, nothing to do with <code>KmerIndex</code>/disk truth), unrelated to the
index. First proposal (wrap <code>writers</code> in <code>RefCell</code> so all four could
share a uniform <code>&amp;self</code>) was retracted on pushback: manufacturing
interior mutability with runtime borrow checks to satisfy a cosmetic
uniformity that Rust doesn't even require is over-engineering — a trait
method's receiver must match exactly across implementors, but nothing
stops that shared receiver from being <code>&amp;mut self</code> with three of the four
implementations simply not using the mutability. Settled: trait declares
<code>&amp;mut self</code>; <code>Dereplicator</code>/<code>Counter</code>/<code>LayerBuilder</code> (previously <code>&amp;self</code>)
now also take <code>&amp;mut self</code>, unused.</p>
<p><strong>2. <code>path_source</code> as a <code>PartitionRouter</code> setter, not a <code>run()</code> param</strong> —
added a <code>files: Option&lt;Box&lt;dyn Iterator&lt;Item = PathBuf&gt; + Send&gt;&gt;</code> field +
<code>.files(impl Iterator&lt;Item = PathBuf&gt; + Send + 'static) -&gt; Self</code> setter
(boxed rather than a generic type parameter on <code>PartitionRouter&lt;'a&gt;</code>:
negligible cost — one <code>PathBuf</code> per input <em>file</em>, not per k-mer — for a
much more usable type when passing the builder around). <code>run</code> now does
<code>self.files.take().ok_or_else(...)</code>, erroring if <code>.files(...)</code> was never
called, instead of taking <code>path_source</code> as a parameter.</p>
<p><strong>3. Unifying the three progress-callback bound shapes</strong> — reopened, then
resolved differently than (8) originally framed it. First proposal
(force everything to <code>FnMut(Progress) + Send</code>) was rejected on the same
principle as point 1: <code>Dereplicator</code>/<code>Counter</code>'s <code>Fn(Progress) + Sync</code>
isn't arbitrary — their callback is invoked concurrently from multiple
rayon worker threads, and <code>FnMut</code> requires exclusive access, so forcing
it would mean wrapping the callback in a <code>Mutex</code> for zero benefit at the
one real call site (<code>pb.inc(1)</code>, already thread-safe). The actual
resolution: move <code>on_progress</code> off <code>run()</code>'s signature entirely, onto a
per-algorithm <code>.on_progress(...)</code> setter — same treatment as point 2's
<code>path_source</code> — so each algorithm keeps its own bound (<code>PartitionRouter</code>:
<code>FnMut(Progress) + 'a</code>, sequential, no <code>Send</code> needed; <code>LayerBuilder</code>:
<code>FnMut(Progress) + Send + 'a</code>, crosses into <code>PartitionRunner</code>'s
<code>thread::scope</code>-spawned controller thread once; <code>Dereplicator</code>/<code>Counter</code>:
<code>Fn(Progress) + Sync + 'a</code>, invoked concurrently from rayon workers).
This <em>also</em> dissolves the original problem <code>run()</code> had: once the
callback isn't part of <code>run</code>'s signature at all, there's nothing left to
unify there, and point 4 (below) becomes trivial.</p>
<p><strong>4. <code>Output</code> as an associated type, <code>Error</code> fixed</strong> — trivial once (3)
moved the callback out: <code>Error</code> was already uniform (all four return
<code>obiskio::SKResult&lt;T&gt;</code> = <code>Result&lt;T, SKError&gt;</code>), only <code>Output</code> varied
(<code>()</code>/<code>()</code>/<code>KmerSpectrum</code>/<code>usize</code>). First cut reused <code>obiskio::SKResult</code>
directly as the trait's return type — <strong>caught and corrected the same
session</strong>: <code>SKError</code> enumerates I/O-specific cases (<code>BadMagic</code>/
<code>Truncated</code>/<code>Compression</code>/...), meaningless at the level of a generic
"algorithm" abstraction, and borrowing it made <code>obikalgorithm</code> — meant to
be minimal and neutral — depend on a low-level I/O crate purely to reuse
its error type. Textbook instance of the "petits pois" failure mode
(patch around a convenient existing type instead of asking what this
crate should actually own). Fixed to a genuinely generic, boxed error
type owned by <code>obikalgorithm</code> itself:</p>
<div class="highlight"><pre><span></span><code><span class="c1">// obikalgorithm — no dependency on obiskio or any other crate</span>
<span class="k">pub</span><span class="w"> </span><span class="k">type</span><span class="w"> </span><span class="nc">Error</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="nb">Box</span><span class="o">&lt;</span><span class="k">dyn</span><span class="w"> </span><span class="n">std</span><span class="p">::</span><span class="n">error</span><span class="p">::</span><span class="n">Error</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="nb">Send</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="nb">Sync</span><span class="o">&gt;</span><span class="p">;</span>
<span class="k">pub</span><span class="w"> </span><span class="k">type</span><span class="w"> </span><span class="nb">Result</span><span class="o">&lt;</span><span class="n">T</span><span class="o">&gt;</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">std</span><span class="p">::</span><span class="n">result</span><span class="p">::</span><span class="nb">Result</span><span class="o">&lt;</span><span class="n">T</span><span class="p">,</span><span class="w"> </span><span class="n">Error</span><span class="o">&gt;</span><span class="p">;</span>
<span class="k">pub</span><span class="w"> </span><span class="k">trait</span><span class="w"> </span><span class="n">Algorithm</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="k">type</span><span class="w"> </span><span class="nc">Output</span><span class="p">;</span>
<span class="w"> </span><span class="k">fn</span><span class="w"> </span><span class="nf">run</span><span class="p">(</span><span class="o">&amp;</span><span class="k">mut</span><span class="w"> </span><span class="bp">self</span><span class="p">)</span><span class="w"> </span><span class="p">-&gt;</span><span class="w"> </span><span class="nb">Result</span><span class="o">&lt;</span><span class="bp">Self</span><span class="p">::</span><span class="n">Output</span><span class="o">&gt;</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div>
<p>Any concrete error (<code>SKError</code>, <code>std::io::Error</code>, ...) converts
automatically via <code>?</code>, through <code>std</code>'s own blanket <code>impl&lt;E: Error + Send
+ Sync&gt; From&lt;E&gt; for Box&lt;dyn Error + Send + Sync&gt;</code> — no custom <code>From</code> impl
needed, no dependency on the crate that defines the concrete error type.
The four algorithms' <code>run</code> bodies needed no change beyond the signature's
return type (every existing <code>?</code> on an <code>SKError</code>-returning subcall keeps
compiling, converting through the same blanket impl at the boundary).</p>
<p><code>PartitionRouter</code>/<code>Dereplicator</code>/<code>Counter</code>/<code>LayerBuilder</code> each <code>impl
Algorithm for X&lt;'_&gt; { type Output = ...; fn run(&amp;mut self) -&gt; obikalgorithm::Result&lt;...&gt; { ... } }</code>
— the old inherent <code>run</code> methods were removed outright (not kept as
duplicates), so callers now <code>use obikalgorithm::Algorithm;</code> to call
<code>.run()</code>. Every field-lifetime-bound boxed callback (<code>Box&lt;dyn
FnMut(Progress) + 'a&gt;</code> etc.) is tied to the algorithm's own <code>'a</code> (the
<code>&amp;'a KmerIndex</code> lifetime already on the struct), not <code>'static</code> — avoids
forcing callers' progress closures to <code>move</code>-capture (and therefore clone
or <code>Arc</code>-wrap) local state like <code>TracedBar</code>/EMA-rate accumulators that
they'd otherwise want to keep using by reference after <code>run()</code> returns.</p>
<p><strong>Why a new crate, not a submodule of <code>obikindexer</code></strong>: <code>obikmer::cmd::
index::mod</code> and <code>obikphylo</code>'s own test helpers both need to call <code>.run()</code>
on these algorithms, so the trait has to be reachable from outside
<code>obikindexer</code> — putting it in <code>obikindexer</code> itself would work file-wise
but conflates "the trait every algorithm implements" with "one crate's
particular four implementations of it", the same reasoning that already
separated <code>obikindex</code> (data model) from <code>obikindexer</code> (algorithms
operating on it). <code>obikalgorithm</code> has <strong>no dependencies at all</strong> (see
above); <code>obikindexer</code>, <code>obikmer</code>, and <code>obikphylo</code> (dev-dependency, for
its test helper) all depend on it.</p>
<p><strong>Blast radius</strong>: <code>obikindexer</code>'s four algorithm modules (struct field +
setter + trait impl each); <code>obikmer::cmd::index::mod</code> (three call sites:
<code>.on_progress(cb)</code> before <code>.run()</code>, unqualified now that the trait is in
scope); <code>obikindexer::algorithms::partitionner::tests</code> and <code>obikphylo::
siblings::tests</code> (both had direct <code>.run(None::&lt;fn(Progress)&gt;)</code>-shaped
calls needing the same treatment). New <code>obikalgorithm</code> crate registered
in the workspace <code>Cargo.toml</code>, depended on by <code>obikindexer</code>/<code>obikmer</code>
(regular) and <code>obikphylo</code> (dev).</p>
<p><strong>Verification</strong>: <code>cargo check --workspace --all-targets</code> and <code>cargo test
--workspace</code> both green (0 failures), <code>scripts/smoke_test_index.sh</code> green
(870 kmers, same as every prior round) — this round didn't repeat the
manual <code>merge</code>/<code>select</code>/<code>reindex</code> CLI exercise from (11), since nothing
in this pass touched those commands' code paths (only the four pipeline
algorithms and <code>cmd::index</code>, already covered by the smoke test). Reverified
after the <code>obiskio</code>-dependency fix above (same three checks, still green,
<code>obikalgorithm/Cargo.toml</code> now has zero <code>[dependencies]</code>).</p>
<p>Still not done: (5)'s <code>KmerPartition</code>/<code>Layer</code> self-naming redesign; the
<code>distance.rs</code> → <code>obikphylo</code> relocation ((9), explicitly deferred); the
future cache-manager crate (mentioned in (8) as a later, mirrored
extension-trait exercise, not started).</p>
<h2 id="the-problem">The problem</h2>
<p>Reading a layer's data (MPHF + matrix) is not free: <code>MphfLayer::open</code> mmaps
<code>mphf.bin</code> plus (<code>evidence.bin</code>/<code>fingerprint.bin</code> + <code>unitigs.bin</code>), and the