refactor: rename obikpartitionner crate to obikpartition
Update Cargo manifests, dependency paths, and Rust imports across the workspace to reflect the `obikpartitionner` to `obikpartition` rename. Synchronize architecture and implementation documentation with the new module structure. Fix minor syntax issues in test assertions to ensure compilation compatibility. No behavioral or API changes are introduced.
This commit is contained in:
@@ -303,7 +303,7 @@ This parameter has no effect on presence/absence indexes (where values are alrea
|
||||
|
||||
## Implementation
|
||||
|
||||
- **`obikpartitionner::filter::GroupQuorumFilter`** — implements `KmerFilter`
|
||||
- **`obikpartition::filter::GroupQuorumFilter`** — implements `KmerFilter`
|
||||
using pre-computed ingroup and outgroup index vectors. The heavy logic
|
||||
(predicate parsing, three-value evaluation, genome classification) happens
|
||||
once before any iteration; each k-mer row evaluation is a simple index
|
||||
@@ -314,7 +314,7 @@ This parameter has no effect on presence/absence indexes (where values are alrea
|
||||
`UnitigArgs`. `FilterArgs::build_filters()` returns a ready-to-use filter
|
||||
list.
|
||||
|
||||
- **`obikpartitionner::KmerPartition::iter_partition_kmers`** — accepts
|
||||
- **`obikpartition::KmerPartition::iter_partition_kmers`** — accepts
|
||||
`filters: &[Box<dyn KmerFilter>]` and applies them per-kmer before invoking
|
||||
the callback. `filter`, `dump`, and `unitig` all go through this single
|
||||
entry point.
|
||||
|
||||
@@ -4,8 +4,8 @@
|
||||
## Code couvert
|
||||
|
||||
- `obikindex/src/merge.rs` — `KmerIndex::merge()`, validation de compatibilité d'évidence, `validate_evidence_compat()`
|
||||
- `obikpartitionner/src/merge_layer.rs` — `merge_partition()`, construction de la nouvelle layer, paramètre `block_bits`
|
||||
- `obikpartitionner/src/rebuild_layer.rs` — `rebuild_partition()`, paramètre `block_bits`
|
||||
- `obikpartition/src/merge_layer.rs` — `merge_partition()`, construction de la nouvelle layer, paramètre `block_bits`
|
||||
- `obikpartition/src/rebuild_layer.rs` — `rebuild_partition()`, paramètre `block_bits`
|
||||
- `obilayeredmap/src/layer.rs` — `Layer::append_genome_column()` (PersistentCompactIntMatrix et PersistentBitMatrix)
|
||||
- `obicompactvec/src/intmatrix.rs` — `append_column` pour PersistentCompactIntMatrix
|
||||
- `obicompactvec/src/bitmatrix.rs` — `append_column` pour PersistentBitMatrix
|
||||
|
||||
@@ -6,7 +6,7 @@ Kmer indexing per partition proceeds in two phases. The separation is necessary
|
||||
|
||||
### Phase 1 — provisional MPHF + kmer spectrum
|
||||
|
||||
Implemented in `obikpartitionner::KmerPartition::count_kmer()` → `count_partition()`.
|
||||
Implemented in `obikpartition::KmerPartition::count_kmer()` → `count_partition()`.
|
||||
|
||||
1. **External sort**: read the dereplicated superkmer file; extract the raw `u64` canonical kmer value for every kmer of every superkmer. Sort in RAM-bounded chunks (adaptive budget: 40% of available RAM ÷ n_threads, minimum 1 M kmers per chunk), then k-way merge with inline dedup. Result: `sorted_unique.bin` — a flat array of f0 distinct sorted `u64` values. Exact kmer count f0 is known at this point.
|
||||
2. **Build provisional MPHF** (ptr_hash, same configuration as phase 2) over `sorted_unique.bin` using `new_from_par_iter`. Delete `sorted_unique.bin` immediately after. Persist to `mphf1.bin`.
|
||||
@@ -148,7 +148,7 @@ MphfLayer::build_approx_evidence(dir, b, z)
|
||||
|
||||
There is no `build_evidence` dispatch wrapper. Callers choose the appropriate post-hoc build directly.
|
||||
|
||||
In `obikpartitionner`, `build_index_layer` receives `block_bits: u8` from `IndexConfig::block_bits` and forwards it directly to `Layer::build` and `Layer::build_approx_evidence`.
|
||||
In `obikpartition`, `build_index_layer` receives `block_bits: u8` from `IndexConfig::block_bits` and forwards it directly to `Layer::build` and `Layer::build_approx_evidence`.
|
||||
|
||||
### Membership verification
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
## Code couvert
|
||||
|
||||
- `obilayeredmap/src/mphf_layer.rs` — type Mphf (PtrHash + CubicEps + CachelineEfVec + Xx64), construction en 2 passes, `build()`, `build_exact_evidence()`, `build_approx_evidence()`, `build_evidence()`
|
||||
- `obikpartitionner/src/index_layer.rs` — `build_index_layer()` avec passage de `block_bits`
|
||||
- `obikpartition/src/index_layer.rs` — `build_index_layer()` avec passage de `block_bits`
|
||||
|
||||
## Notes
|
||||
|
||||
|
||||
@@ -9,5 +9,5 @@
|
||||
## Notes
|
||||
|
||||
Document stable (librairie générique, peu de risque de dérive).
|
||||
Vérifier si `obipipeline` est toujours utilisé dans la phase scatter de `obikpartitionner`
|
||||
Vérifier si `obipipeline` est toujours utilisé dans la phase scatter de `obikpartition`
|
||||
ou s'il a été remplacé par Rayon dans certains chemins.
|
||||
|
||||
@@ -16,7 +16,7 @@ explicitly, since two of the names below are misleading.
|
||||
but it holds the *whole* multi-partition structure below — the name
|
||||
suggests "one partition", the value is all of them.
|
||||
- **Partition, the collection (not one partition)** =
|
||||
`obikpartitionner::KmerPartition`. Despite the singular name, this owns
|
||||
`obikpartition::KmerPartition`. Despite the singular name, this owns
|
||||
*every* partition of the index: `root_path`, `n_partitions`, and
|
||||
per-partition accessors that all take an explicit index `i`
|
||||
(`partition_dir(i)`, `index_dir(i)`, `layer_dir(i, l)`,
|
||||
@@ -95,7 +95,7 @@ auto-detection" turns up three unrelated implementations:
|
||||
|---|---|---|---|
|
||||
| `Layer<D>` | `obilayeredmap` | one layer, `D` fixed at compile time | held alive by whoever owns the `Layer`, no policy of its own |
|
||||
| `Mat` | `obikphylo::siblings::cache` | one layer, format resolved per instance from an enum of 3 `Layer<D>` variants | yes, via `PartitionCache` |
|
||||
| `QueryLayer` | `obikpartitionner::query_layer` | one layer, `(MphfLayer, PersistentBitMatrix\|PersistentCompactIntMatrix)` pair, bypasses `Layer<D>` entirely | **no** — opened fresh inside `query_partition_with` on every call |
|
||||
| `QueryLayer` | `obikpartition::query_layer` | one layer, `(MphfLayer, PersistentBitMatrix\|PersistentCompactIntMatrix)` pair, bypasses `Layer<D>` entirely | **no** — opened fresh inside `query_partition_with` on every call |
|
||||
|
||||
`query_partition_with` is `obikmer query`'s normal query path — the one
|
||||
most exposed to repeated cross-partition lookups — and it is the one with
|
||||
@@ -164,7 +164,7 @@ Established in discussion, not yet coded:
|
||||
`open_data`/`layer_dir` — see git history around 2026-08-20). One
|
||||
`LayeredMap<D>` (or its heterogeneous-`D` successor) = one partition.
|
||||
- Nobody currently owns "the collection of partitions" as reusable state.
|
||||
`obikpartitionner::KmerPartition` is the closest candidate — it already
|
||||
`obikpartition::KmerPartition` is the closest candidate — it already
|
||||
owns `n_partitions()`/`part_dir(i)` — but today it is a pure
|
||||
config/path resolver, not a state holder: `dereplicate`, `count_kmer`,
|
||||
`obikindex`'s `distance.rs`/`stats.rs`, and
|
||||
@@ -182,9 +182,9 @@ Established in discussion, not yet coded:
|
||||
2. A cache spanning multiple partitions, keeping (2)'s layer type open per
|
||||
partition per layer, for the whole lifetime of a long-running command —
|
||||
generalising `PartitionCache` minus its sibling-specific parts
|
||||
(`fast_mode`, `find_presence_batch`) — living in `obikpartitionner`
|
||||
(`fast_mode`, `find_presence_batch`) — living in `obikpartition`
|
||||
(owner of the partition dimension) and built on top of (1).
|
||||
3. `obikpartitionner::query_partition_with` — the normal query path,
|
||||
3. `obikpartition::query_partition_with` — the normal query path,
|
||||
currently uncached — becomes a consumer of (2), not just
|
||||
`obikphylo::PartitionCache`.
|
||||
|
||||
@@ -198,7 +198,7 @@ real number behind it for this codebase's scale).
|
||||
|
||||
Groundwork for (1)/(2), landed ahead of the design itself:
|
||||
|
||||
- `KmerPartition` (`obikpartitionner`) gained `partition_dir`/`index_dir`/
|
||||
- `KmerPartition` (`obikpartition`) gained `partition_dir`/`index_dir`/
|
||||
`layer_dir` as the single source of truth for a partition's on-disk
|
||||
layout, replacing per-module duplicated `const INDEX_SUBDIR: &str =
|
||||
"index"` (7 copies) and ad hoc path joins — including one found
|
||||
@@ -266,7 +266,7 @@ storage decisions, now a *third* copy of the same logic to keep in sync)
|
||||
`Mat::open` already reads the result back in the sibling-annex path.
|
||||
Stale comment, not corrected.
|
||||
- **Retracted (2026-08-20)**: an earlier pass through this doc claimed
|
||||
`obikpartitionner::query_layer::QueryLayer::open` had no sparse-format
|
||||
`obikpartition::query_layer::QueryLayer::open` had no sparse-format
|
||||
detection and would silently corrupt reads on a `pack --sparse`d layer.
|
||||
False — `PersistentBitMatrix` (`obicompactvec::bitmatrix::persistent`)
|
||||
is a 4-way enum (`Columnar`/`Packed`/`Sparse`/`Implicit`), not 3-way as
|
||||
|
||||
@@ -3,12 +3,12 @@
|
||||
|
||||
## Code couvert
|
||||
|
||||
- `obikpartitionner/src/partition.rs` — estimation des paramètres (phase 0)
|
||||
- `obikpartition/src/partition.rs` — estimation des paramètres (phase 0)
|
||||
- `obiskbuilder/src/iter.rs` — scatter : filtre entropie, extraction superkmers, routage partition (phase 1)
|
||||
- `obikpartitionner/src/filter.rs` — déduplication bucket-sort (phase 2)
|
||||
- `obikpartitionner/src/kmer_sort.rs` — tri externe + agrégation de comptages (phase 3)
|
||||
- `obikpartition/src/filter.rs` — déduplication bucket-sort (phase 2)
|
||||
- `obikpartition/src/kmer_sort.rs` — tri externe + agrégation de comptages (phase 3)
|
||||
- `obidebruinj/src/debruijn.rs` — graphe De Bruijn, extraction des unitigs (phase 5)
|
||||
- `obikpartitionner/src/index_layer.rs` — construction MPHF + évidence (phase 6), paramètre `block_bits`
|
||||
- `obikpartition/src/index_layer.rs` — construction MPHF + évidence (phase 6), paramètre `block_bits`
|
||||
- `obikindex/src/index.rs` — `build_layers()`, `dereplicate_and_count()`
|
||||
|
||||
## Notes
|
||||
|
||||
Reference in New Issue
Block a user