feat: introduce obikpartition crate for kmer partition layer lookup

Introduces the `obikpartition` crate containing the `KmerPartition` struct to manage open layers in sequential order. The `open` constructor eagerly initializes layers under a specified directory, while the `find` method returns the index of the first layer containing a given k-mer. Dependencies are strictly scoped to `obilayeredmap` and `obikseq`, with unit tests validating layer ingestion and single-kmer lookup behavior. Batch lookups and further migration are deferred to a subsequent phase.
This commit is contained in:
Eric Coissac
2026-08-20 20:47:20 +02:00
parent c4b69e1af5
commit 164e879585
6 changed files with 250 additions and 5 deletions
@@ -2,9 +2,11 @@
Status (2026-08-20, latest pass): (1) done — `obilayeredmap::Layer`
exists, `Mat` is gone. (1b) done — `Layer::Empty`, the first non-ready
state, added (panics on every read method). (2) `KmerPartition` + a
multi-partition cache — **not started**, precisely specified below after
a real mix-up: an earlier
state, added (panics on every read method). (2a) done — the
`obikpartition` crate and `KmerPartition` itself exist (`open`/`n_layers`/
`layer`/`layers`/`find`). (2b) — migrating `PartitionCache`/`QueryLayer`
onto it — **not started**, deliberately deferred. Earlier mix-up, for
context: an earlier
version of this doc used the name `KmerPartition` (singular) for what was
actually the *collection* type (later renamed `KmerPartitions`, later
merged into `KmerIndex` — see "Major restructuring" below), and never
@@ -252,7 +254,66 @@ Full workspace suite green (`cargo check --workspace --all-targets` then
Still deferred, per explicit instruction: `build_mphf()`/
`build_unitigs()`/`build_evidence()` to progress `Empty` further, and (2)
`KmerPartition` itself — unchanged from above.
`KmerPartition` itself — unchanged from above (see "(2a) done" below,
added next).
## (2a) done (2026-08-20): `obikpartition` crate + `KmerPartition`
Built exactly the shape "Definitions" (top of file) specifies, nothing
more — deliberately scoped down from the full "Direction agreed" plan
below: only steps 1–2 (`open`/`n_layers`/`layer`/`layers`/`find`), not 3–4
(migrating `PartitionCache`/`QueryLayer` onto it), per explicit
instruction to implement `KmerPartition` first and decide the wiring
("comment on branche tout ça dans la construction") separately, later.
```rust
pub struct KmerPartition {
layers: Vec<obilayeredmap::Layer>,
}
impl KmerPartition {
pub fn open(index_dir: &Path, mode: &IndexMode, n_layers: usize, with_counts: bool) -> OLMResult<Self>;
pub fn n_layers(&self) -> usize;
pub fn layer(&self, i: usize) -> &Layer;
pub fn layers(&self) -> &[Layer];
pub fn find(&self, kmer: CanonicalKmer) -> Option<usize>;
}
```
`open` takes `index_dir`/`mode`/`n_layers`/`with_counts` as plain
arguments — no reach-back into `KmerIndex` (would need `obikpartition →
obikindex`, the wrong direction) — and builds each layer's path via
`obilayeredmap::layer_dir(index_dir, l)`, the same shared naming
primitive `KmerIndex::layer_dir` itself delegates to, not a second copy of
the `layer_N` convention. `find` mirrors `PartitionCache::find`'s
semantics (first layer that carries the kmer wins) but doesn't yet cover
`find_presence_batch`/`find_presence_batch_fast` — those exist only to
serve `PartitionCache`, so they're part of the (2b) migration, not this
step; building them now against the current sibling-specific tuple shape
`(CanonicalKmer, usize, u8, u8)` would either bake phylo vocabulary
(`family_idx`, `base`) into `obikpartition` or require deciding a generic
payload shape — a real design fork, deferred to when (2b) is actually
tackled rather than guessed at here.
Crate deps: `obikseq`, `obilayeredmap` only (dev-deps add `obiskio`,
`obicompactvec`, `tempfile` for tests) — matches the "Definitions"
constraint (`obikpartition` depends on `obilayeredmap` and below, never
`obikindex`/`obikpartitionner`/`obikphylo`). Registered as a new workspace
member (`src/Cargo.toml`). 3 new tests (`open_reads_every_layer_in_order`,
`find_reports_the_first_layer_that_carries_the_kmer`,
`find_returns_none_for_an_absent_kmer`). Full workspace suite green
(`cargo check --workspace --all-targets` then `cargo test --workspace`)
after.
Still not done: (2b) — migrating `obikphylo::siblings::cache::
PartitionCache` (currently `Vec<Vec<Layer>>`) and
`obikindex::query_layer::QueryLayer` (currently uncached, bypasses `Layer`
entirely) onto `KmerPartition`/`Vec<KmerPartition>`; deciding whether that
collection lives in `obikpartition` or `obikindex`; deciding the
batch-lookup surface's exact shape (generic payload vs. as-is sibling
tuple moved in wholesale); `scan_layer_families`'s still-independent
`PartitionMeta::load` (see "Remaining instance…" below) — all explicitly
deferred to whenever wiring is tackled next.
## The problem