Extract indexing stage into LayerBuilder with NUMA-aware scheduling

Introduce a dedicated LayerBuilder struct to orchestrate parallel layer 0 construction across partitions. The fluent API supports configurable abundance thresholds and intermediate artifact retention. Parallel execution is delegated to a NUMA-aware PartitionRunner scheduler, while progress reporting and stage timing are shifted to the command layer. A new mark_indexed method cleanly separates state tracking from orchestration by generating a completion sentinel.

src/obikindexer/src/algorithms/layer_builder/mod.rs

@@ -0,0 +1,109 @@
+//! Layer construction — the last stage of the indexing pipeline: turns
+//! each partition's dereplicated superkmers + provisional counts (from
+//! [`crate::algorithms::counter`]) into the real layer 0 (`mphf.bin` +
+//! `unitigs.bin` + matrix). See
+//! `DevDocMD/implementation/partition_layer_cache.md`.
+
+use obikindex::{KmerIndex, PartitionRunner};
+use obiskio::SKResult;
+use obisys::Progress;
+
+/// Builds every partition's real layer 0 in parallel.
+///
+/// Two-phase construction, same shape as the other pipeline algorithms:
+/// `new` (no disk access), optional setters (`min_abundance`/
+/// `max_abundance`/`keep_intermediate`), then `run`. The actual
+/// per-partition construction (De Bruijn graph, unitigs, MPHF, matrix)
+/// stays a `KmerIndex` method (`build_index_layer`) — unlike
+/// `Dereplicator`/`Counter`, it depends on several `obikindex`-internal
+/// helpers (`graph_pipeline`, `common::olm_to_sk`) already shared with
+/// `merge`/`select`/`rebuild`'s own layer-construction paths, so it
+/// belongs there, not duplicated or newly exported just for this
+/// algorithm. `LayerBuilder`'s own job is the orchestration around it:
+/// scheduling, configuration, and progress reporting.
+pub struct LayerBuilder<'a> {
+    index: &'a KmerIndex,
+    n_partitions: usize,
+    min_abundance: u32,
+    max_abundance: Option<u32>,
+    keep_intermediate: bool,
+}
+
+impl<'a> LayerBuilder<'a> {
+    pub fn new(index: &'a KmerIndex) -> Self {
+        Self {
+            index,
+            n_partitions: index.n_partitions(),
+            min_abundance: 1,
+            max_abundance: None,
+            keep_intermediate: false,
+        }
+    }
+
+    /// Minimum kmer abundance (inclusive) — kmers below this are filtered
+    /// out. Default 1 (no filtering).
+    pub fn min_abundance(mut self, v: u32) -> Self {
+        self.min_abundance = v;
+        self
+    }
+
+    /// Maximum kmer abundance (inclusive) — kmers above this are filtered
+    /// out. Default `None` (no ceiling).
+    pub fn max_abundance(mut self, v: Option<u32>) -> Self {
+        self.max_abundance = v;
+        self
+    }
+
+    /// Keep each partition's build artifacts (dereplicated superkmers,
+    /// `mphf1.bin`, `counts1.bin`) after their layer is built, instead of
+    /// deleting them (default: `false`).
+    pub fn keep_intermediate(mut self, v: bool) -> Self {
+        self.keep_intermediate = v;
+        self
+    }
+
+    /// Build every partition's layer 0 in parallel via `PartitionRunner`
+    /// (NUMA-aware scheduling — this stage is more CPU/memory-intensive
+    /// per partition than scatter/dereplicate/count, unlike those it
+    /// doesn't use plain rayon). Returns the total number of kmers built
+    /// across all partitions.
+    ///
+    /// `on_progress`, when set, is called once per completed partition —
+    /// `FnMut + Send`, not `Fn + Sync`: `PartitionRunner::run` calls
+    /// `on_done` from its own single controller thread, not concurrently
+    /// from workers, so (unlike `Dereplicator`/`Counter`) there's no need
+    /// for a `Sync` bound — but that controller runs in a spawned scoped
+    /// thread, so the closure itself still has to cross a thread boundary
+    /// once, hence `Send`. `total: Some(n_partitions)` — known up front.
+    pub fn run(&self, mut on_progress: Option<impl FnMut(Progress) + Send>) -> SKResult<usize> {
+        let with_counts = self.index.with_counts();
+        let evidence = self.index.evidence_mode().clone();
+        let block_bits = self.index.block_bits();
+        let min_ab = self.min_abundance;
+        let max_ab = self.max_abundance;
+        let mut total_kmers: usize = 0;
+        let mut done: u64 = 0;
+
+        let order: Vec<usize> = (0..self.n_partitions).collect();
+        let runner = PartitionRunner::new();
+        runner.run(
+            &order,
+            |i| self.index.build_index_layer(i, min_ab, max_ab, with_counts, &evidence, block_bits),
+            |_i, n_kmers, _elapsed| {
+                total_kmers += n_kmers;
+                done += 1;
+                if let Some(cb) = on_progress.as_mut() {
+                    cb(Progress { position: done, total: Some(self.n_partitions as u64) });
+                }
+            },
+        )?;
+
+        if !self.keep_intermediate {
+            for i in 0..self.n_partitions {
+                self.index.remove_build_artifacts(i);
+            }
+        }
+
+        Ok(total_kmers)
+    }
+}

src/obikindexer/src/algorithms/mod.rs

@@ -4,8 +4,11 @@
 //! algorithm is its own submodule: [`partitionner`] (routing raw super-kmers
 //! into partitions), [`dereplicator`] (deduplicating a partition's raw
 //! super-kmers), [`counter`] (counting unique canonical kmers from a
-//! partition's dereplicated super-kmers) — the three run in that order.
+//! partition's dereplicated super-kmers), [`layer_builder`] (turning
+//! dereplicated super-kmers + counts into the real layer 0) — the four run
+//! in that order.
 
 pub mod counter;
 pub mod dereplicator;
+pub mod layer_builder;
 pub mod partitionner;