perf: enable zero-allocation queries and memory-mapped indexes

Introduce zero-allocation row extraction and query result buffers across `obicompactvec` and `obikpartitionner` to eliminate per-kmer heap allocations. Replace in-memory MPHF deserialization with memory-mapped, zero-copy views to reduce runtime memory footprint. Add configurable I/O chunking, a RAM-aware `--chunk-size` parameter, and system memory monitoring via the new `sysinfo` dependency. Re-export `PreloadedIndex` for external consumers.
This commit is contained in:
Eric Coissac
2026-06-03 09:39:49 +02:00
parent 1661dd6b1c
commit de1a41810a
11 changed files with 403 additions and 274 deletions
+8
View File
@@ -33,6 +33,14 @@ impl PersistentCompactIntMatrix {
self.cols.iter().map(|c| c.get(slot)).collect()
}
/// Fill `buf[i]` with `col_i[slot]`, without allocating.
/// `buf` must have length ≥ `self.n_cols()`.
pub fn fill_row(&self, slot: usize, buf: &mut [u32]) {
for (c, col) in self.cols.iter().enumerate() {
buf[c] = col.get(slot);
}
}
// ── Distance matrices ─────────────────────────────────────────────────────
pub fn bray_dist_matrix(&self) -> Array2<f64> {