large refactoring

This commit is contained in:
Eric Coissac
2026-08-17 09:28:53 +02:00
parent 49e66f16a2
commit 7bac0f3850
236 changed files with 77584 additions and 3733 deletions
+3 -3
View File
@@ -76,7 +76,7 @@ const FAMILY_BATCH: usize = 65536;
/// Restricts [`scan_layer_families`] to a subset of a layer's families,
/// identified by their iteration-order index — see
/// `docmd/architecture/siblings.md`, "`--subsample`/`--shannon`", for why:
/// `DevDocMD/architecture/siblings.md`, "`--subsample`/`--shannon`", for why:
/// bounding which families actually pay the expensive cross-partition
/// resolution cost is what makes a sampled run cheap on an index far larger
/// than the sample itself. `All` (the default for every pre-existing
@@ -145,7 +145,7 @@ struct SourceBatch {
start_family_idx: usize,
/// One entry per minorant family in this batch, straight from
/// `iter_minorants_batch` — `order` (iteration-order index, not an MPHF
/// slot; see `docmd/architecture/siblings.md`), `kmer`, and `mask`
/// slot; see `DevDocMD/architecture/siblings.md`), `kmer`, and `mask`
/// already carried through, no second annex read.
entries: Vec<SiblingEntry>,
_permit: ThrottleGuard,
@@ -207,7 +207,7 @@ pub(super) fn scan_layer_families(
// Streamed straight from `iter_minorants_batch` (zips this layer's own
// `iter_kmers()` with the annex, both in iteration order — never an
// MPHF slot; see `docmd/architecture/siblings.md`) — never collected
// MPHF slot; see `DevDocMD/architecture/siblings.md`) — never collected
// into a `Vec` first: a layer can hold billions of k-mers, so
// materialising every minorant family up front is exactly the memory
// blowup an earlier version of this traversal was rewritten to avoid