feat: introduce obikpartition crate for kmer partition layer lookup
Introduces the `obikpartition` crate containing the `KmerPartition` struct to manage open layers in sequential order. The `open` constructor eagerly initializes layers under a specified directory, while the `find` method returns the index of the first layer containing a given k-mer. Dependencies are strictly scoped to `obilayeredmap` and `obikseq`, with unit tests validating layer ingestion and single-kmer lookup behavior. Batch lookups and further migration are deferred to a subsequent phase.
This commit is contained in:
@@ -2,9 +2,11 @@
|
||||
|
||||
Status (2026-08-20, latest pass): (1) done — `obilayeredmap::Layer`
|
||||
exists, `Mat` is gone. (1b) done — `Layer::Empty`, the first non-ready
|
||||
state, added (panics on every read method). (2) — `KmerPartition` + a
|
||||
multi-partition cache — **not started**, precisely specified below after
|
||||
a real mix-up: an earlier
|
||||
state, added (panics on every read method). (2a) done — the
|
||||
`obikpartition` crate and `KmerPartition` itself exist (`open`/`n_layers`/
|
||||
`layer`/`layers`/`find`). (2b) — migrating `PartitionCache`/`QueryLayer`
|
||||
onto it — **not started**, deliberately deferred. Earlier mix-up, for
|
||||
context: an earlier
|
||||
version of this doc used the name `KmerPartition` (singular) for what was
|
||||
actually the *collection* type (later renamed `KmerPartitions`, later
|
||||
merged into `KmerIndex` — see "Major restructuring" below), and never
|
||||
@@ -252,7 +254,66 @@ Full workspace suite green (`cargo check --workspace --all-targets` then
|
||||
|
||||
Still deferred, per explicit instruction: `build_mphf()`/
|
||||
`build_unitigs()`/`build_evidence()` to progress `Empty` further, and (2)
|
||||
— `KmerPartition` itself — unchanged from above.
|
||||
— `KmerPartition` itself — unchanged from above (see "(2a) done" below,
|
||||
added next).
|
||||
|
||||
## (2a) done (2026-08-20): `obikpartition` crate + `KmerPartition`
|
||||
|
||||
Built exactly the shape "Definitions" (top of file) specifies, nothing
|
||||
more — deliberately scoped down from the full "Direction agreed" plan
|
||||
below: only steps 1–2 (`open`/`n_layers`/`layer`/`layers`/`find`), not 3–4
|
||||
(migrating `PartitionCache`/`QueryLayer` onto it), per explicit
|
||||
instruction to implement `KmerPartition` first and decide the wiring
|
||||
("comment on branche tout ça dans la construction") separately, later.
|
||||
|
||||
```rust
|
||||
pub struct KmerPartition {
|
||||
layers: Vec<obilayeredmap::Layer>,
|
||||
}
|
||||
|
||||
impl KmerPartition {
|
||||
pub fn open(index_dir: &Path, mode: &IndexMode, n_layers: usize, with_counts: bool) -> OLMResult<Self>;
|
||||
pub fn n_layers(&self) -> usize;
|
||||
pub fn layer(&self, i: usize) -> &Layer;
|
||||
pub fn layers(&self) -> &[Layer];
|
||||
pub fn find(&self, kmer: CanonicalKmer) -> Option<usize>;
|
||||
}
|
||||
```
|
||||
|
||||
`open` takes `index_dir`/`mode`/`n_layers`/`with_counts` as plain
|
||||
arguments — no reach-back into `KmerIndex` (would need `obikpartition →
|
||||
obikindex`, the wrong direction) — and builds each layer's path via
|
||||
`obilayeredmap::layer_dir(index_dir, l)`, the same shared naming
|
||||
primitive `KmerIndex::layer_dir` itself delegates to, not a second copy of
|
||||
the `layer_N` convention. `find` mirrors `PartitionCache::find`'s
|
||||
semantics (first layer that carries the kmer wins) but doesn't yet cover
|
||||
`find_presence_batch`/`find_presence_batch_fast` — those exist only to
|
||||
serve `PartitionCache`, so they're part of the (2b) migration, not this
|
||||
step; building them now against the current sibling-specific tuple shape
|
||||
`(CanonicalKmer, usize, u8, u8)` would either bake phylo vocabulary
|
||||
(`family_idx`, `base`) into `obikpartition` or require deciding a generic
|
||||
payload shape — a real design fork, deferred to when (2b) is actually
|
||||
tackled rather than guessed at here.
|
||||
|
||||
Crate deps: `obikseq`, `obilayeredmap` only (dev-deps add `obiskio`,
|
||||
`obicompactvec`, `tempfile` for tests) — matches the "Definitions"
|
||||
constraint (`obikpartition` depends on `obilayeredmap` and below, never
|
||||
`obikindex`/`obikpartitionner`/`obikphylo`). Registered as a new workspace
|
||||
member (`src/Cargo.toml`). 3 new tests (`open_reads_every_layer_in_order`,
|
||||
`find_reports_the_first_layer_that_carries_the_kmer`,
|
||||
`find_returns_none_for_an_absent_kmer`). Full workspace suite green
|
||||
(`cargo check --workspace --all-targets` then `cargo test --workspace`)
|
||||
after.
|
||||
|
||||
Still not done: (2b) — migrating `obikphylo::siblings::cache::
|
||||
PartitionCache` (currently `Vec<Vec<Layer>>`) and
|
||||
`obikindex::query_layer::QueryLayer` (currently uncached, bypasses `Layer`
|
||||
entirely) onto `KmerPartition`/`Vec<KmerPartition>`; deciding whether that
|
||||
collection lives in `obikpartition` or `obikindex`; deciding the
|
||||
batch-lookup surface's exact shape (generic payload vs. as-is sibling
|
||||
tuple moved in wholesale); `scan_layer_families`'s still-independent
|
||||
`PartitionMeta::load` (see "Remaining instance…" below) — all explicitly
|
||||
deferred to whenever wiring is tackled next.
|
||||
|
||||
## The problem
|
||||
|
||||
|
||||
Reference in New Issue
Block a user