refactor: rename obikpartitionner crate to obikpartition

Update Cargo manifests, dependency paths, and Rust imports across the workspace to reflect the `obikpartitionner` to `obikpartition` rename. Synchronize architecture and implementation documentation with the new module structure. Fix minor syntax issues in test assertions to ensure compilation compatibility. No behavioral or API changes are introduced.
This commit is contained in:
Eric Coissac
2026-08-20 16:41:22 +02:00
parent f9ef6b8391
commit 9379bbaad8
63 changed files with 80 additions and 80 deletions
+2 -2
View File
@@ -303,7 +303,7 @@ This parameter has no effect on presence/absence indexes (where values are alrea
## Implementation
- **`obikpartitionner::filter::GroupQuorumFilter`** — implements `KmerFilter`
- **`obikpartition::filter::GroupQuorumFilter`** — implements `KmerFilter`
using pre-computed ingroup and outgroup index vectors. The heavy logic
(predicate parsing, three-value evaluation, genome classification) happens
once before any iteration; each k-mer row evaluation is a simple index
@@ -314,7 +314,7 @@ This parameter has no effect on presence/absence indexes (where values are alrea
`UnitigArgs`. `FilterArgs::build_filters()` returns a ready-to-use filter
list.
- **`obikpartitionner::KmerPartition::iter_partition_kmers`** — accepts
- **`obikpartition::KmerPartition::iter_partition_kmers`** — accepts
`filters: &[Box<dyn KmerFilter>]` and applies them per-kmer before invoking
the callback. `filter`, `dump`, and `unitig` all go through this single
entry point.
+2 -2
View File
@@ -4,8 +4,8 @@
## Code couvert
- `obikindex/src/merge.rs``KmerIndex::merge()`, validation de compatibilité d'évidence, `validate_evidence_compat()`
- `obikpartitionner/src/merge_layer.rs``merge_partition()`, construction de la nouvelle layer, paramètre `block_bits`
- `obikpartitionner/src/rebuild_layer.rs``rebuild_partition()`, paramètre `block_bits`
- `obikpartition/src/merge_layer.rs``merge_partition()`, construction de la nouvelle layer, paramètre `block_bits`
- `obikpartition/src/rebuild_layer.rs``rebuild_partition()`, paramètre `block_bits`
- `obilayeredmap/src/layer.rs``Layer::append_genome_column()` (PersistentCompactIntMatrix et PersistentBitMatrix)
- `obicompactvec/src/intmatrix.rs``append_column` pour PersistentCompactIntMatrix
- `obicompactvec/src/bitmatrix.rs``append_column` pour PersistentBitMatrix
+2 -2
View File
@@ -6,7 +6,7 @@ Kmer indexing per partition proceeds in two phases. The separation is necessary
### Phase 1 — provisional MPHF + kmer spectrum
Implemented in `obikpartitionner::KmerPartition::count_kmer()``count_partition()`.
Implemented in `obikpartition::KmerPartition::count_kmer()``count_partition()`.
1. **External sort**: read the dereplicated superkmer file; extract the raw `u64` canonical kmer value for every kmer of every superkmer. Sort in RAM-bounded chunks (adaptive budget: 40% of available RAM ÷ n_threads, minimum 1 M kmers per chunk), then k-way merge with inline dedup. Result: `sorted_unique.bin` — a flat array of f0 distinct sorted `u64` values. Exact kmer count f0 is known at this point.
2. **Build provisional MPHF** (ptr_hash, same configuration as phase 2) over `sorted_unique.bin` using `new_from_par_iter`. Delete `sorted_unique.bin` immediately after. Persist to `mphf1.bin`.
@@ -148,7 +148,7 @@ MphfLayer::build_approx_evidence(dir, b, z)
There is no `build_evidence` dispatch wrapper. Callers choose the appropriate post-hoc build directly.
In `obikpartitionner`, `build_index_layer` receives `block_bits: u8` from `IndexConfig::block_bits` and forwards it directly to `Layer::build` and `Layer::build_approx_evidence`.
In `obikpartition`, `build_index_layer` receives `block_bits: u8` from `IndexConfig::block_bits` and forwards it directly to `Layer::build` and `Layer::build_approx_evidence`.
### Membership verification
+1 -1
View File
@@ -4,7 +4,7 @@
## Code couvert
- `obilayeredmap/src/mphf_layer.rs` — type Mphf (PtrHash + CubicEps + CachelineEfVec + Xx64), construction en 2 passes, `build()`, `build_exact_evidence()`, `build_approx_evidence()`, `build_evidence()`
- `obikpartitionner/src/index_layer.rs``build_index_layer()` avec passage de `block_bits`
- `obikpartition/src/index_layer.rs``build_index_layer()` avec passage de `block_bits`
## Notes
+1 -1
View File
@@ -9,5 +9,5 @@
## Notes
Document stable (librairie générique, peu de risque de dérive).
Vérifier si `obipipeline` est toujours utilisé dans la phase scatter de `obikpartitionner`
Vérifier si `obipipeline` est toujours utilisé dans la phase scatter de `obikpartition`
ou s'il a été remplacé par Rayon dans certains chemins.
@@ -16,7 +16,7 @@ explicitly, since two of the names below are misleading.
but it holds the *whole* multi-partition structure below — the name
suggests "one partition", the value is all of them.
- **Partition, the collection (not one partition)** =
`obikpartitionner::KmerPartition`. Despite the singular name, this owns
`obikpartition::KmerPartition`. Despite the singular name, this owns
*every* partition of the index: `root_path`, `n_partitions`, and
per-partition accessors that all take an explicit index `i`
(`partition_dir(i)`, `index_dir(i)`, `layer_dir(i, l)`,
@@ -95,7 +95,7 @@ auto-detection" turns up three unrelated implementations:
|---|---|---|---|
| `Layer<D>` | `obilayeredmap` | one layer, `D` fixed at compile time | held alive by whoever owns the `Layer`, no policy of its own |
| `Mat` | `obikphylo::siblings::cache` | one layer, format resolved per instance from an enum of 3 `Layer<D>` variants | yes, via `PartitionCache` |
| `QueryLayer` | `obikpartitionner::query_layer` | one layer, `(MphfLayer, PersistentBitMatrix\|PersistentCompactIntMatrix)` pair, bypasses `Layer<D>` entirely | **no** — opened fresh inside `query_partition_with` on every call |
| `QueryLayer` | `obikpartition::query_layer` | one layer, `(MphfLayer, PersistentBitMatrix\|PersistentCompactIntMatrix)` pair, bypasses `Layer<D>` entirely | **no** — opened fresh inside `query_partition_with` on every call |
`query_partition_with` is `obikmer query`'s normal query path — the one
most exposed to repeated cross-partition lookups — and it is the one with
@@ -164,7 +164,7 @@ Established in discussion, not yet coded:
`open_data`/`layer_dir` — see git history around 2026-08-20). One
`LayeredMap<D>` (or its heterogeneous-`D` successor) = one partition.
- Nobody currently owns "the collection of partitions" as reusable state.
`obikpartitionner::KmerPartition` is the closest candidate — it already
`obikpartition::KmerPartition` is the closest candidate — it already
owns `n_partitions()`/`part_dir(i)` — but today it is a pure
config/path resolver, not a state holder: `dereplicate`, `count_kmer`,
`obikindex`'s `distance.rs`/`stats.rs`, and
@@ -182,9 +182,9 @@ Established in discussion, not yet coded:
2. A cache spanning multiple partitions, keeping (2)'s layer type open per
partition per layer, for the whole lifetime of a long-running command —
generalising `PartitionCache` minus its sibling-specific parts
(`fast_mode`, `find_presence_batch`) — living in `obikpartitionner`
(`fast_mode`, `find_presence_batch`) — living in `obikpartition`
(owner of the partition dimension) and built on top of (1).
3. `obikpartitionner::query_partition_with` — the normal query path,
3. `obikpartition::query_partition_with` — the normal query path,
currently uncached — becomes a consumer of (2), not just
`obikphylo::PartitionCache`.
@@ -198,7 +198,7 @@ real number behind it for this codebase's scale).
Groundwork for (1)/(2), landed ahead of the design itself:
- `KmerPartition` (`obikpartitionner`) gained `partition_dir`/`index_dir`/
- `KmerPartition` (`obikpartition`) gained `partition_dir`/`index_dir`/
`layer_dir` as the single source of truth for a partition's on-disk
layout, replacing per-module duplicated `const INDEX_SUBDIR: &str =
"index"` (7 copies) and ad hoc path joins — including one found
@@ -266,7 +266,7 @@ storage decisions, now a *third* copy of the same logic to keep in sync)
`Mat::open` already reads the result back in the sibling-annex path.
Stale comment, not corrected.
- **Retracted (2026-08-20)**: an earlier pass through this doc claimed
`obikpartitionner::query_layer::QueryLayer::open` had no sparse-format
`obikpartition::query_layer::QueryLayer::open` had no sparse-format
detection and would silently corrupt reads on a `pack --sparse`d layer.
False — `PersistentBitMatrix` (`obicompactvec::bitmatrix::persistent`)
is a 4-way enum (`Columnar`/`Packed`/`Sparse`/`Implicit`), not 3-way as
+4 -4
View File
@@ -3,12 +3,12 @@
## Code couvert
- `obikpartitionner/src/partition.rs` — estimation des paramètres (phase 0)
- `obikpartition/src/partition.rs` — estimation des paramètres (phase 0)
- `obiskbuilder/src/iter.rs` — scatter : filtre entropie, extraction superkmers, routage partition (phase 1)
- `obikpartitionner/src/filter.rs` — déduplication bucket-sort (phase 2)
- `obikpartitionner/src/kmer_sort.rs` — tri externe + agrégation de comptages (phase 3)
- `obikpartition/src/filter.rs` — déduplication bucket-sort (phase 2)
- `obikpartition/src/kmer_sort.rs` — tri externe + agrégation de comptages (phase 3)
- `obidebruinj/src/debruijn.rs` — graphe De Bruijn, extraction des unitigs (phase 5)
- `obikpartitionner/src/index_layer.rs` — construction MPHF + évidence (phase 6), paramètre `block_bits`
- `obikpartition/src/index_layer.rs` — construction MPHF + évidence (phase 6), paramètre `block_bits`
- `obikindex/src/index.rs``build_layers()`, `dereplicate_and_count()`
## Notes