Replaces lifetime-bound references with runtime reference counting across multiple crates. This enables safe concurrent access across parallel workers without explicit cloning or manual lifetime management. Introduces the `query` and `utils` CLI commands in obikmer2, along with supporting modules for batch processing, sparse indexing, sliding-window findere logic, and output formatting. Updates dependency manifests and aligns test suites with the new ownership model.
Adds the `pack` subcommand to `obikmer2` for persisting presence and count matrices in sparse or dense formats. Introduces the `compact_layer` module in `obikrebuild` to merge multiple partition layers into a single layer in-place via atomic operations. Updates dependencies across `obikdump`, `obikmer2`, and `obikrebuild` to resolve new local crate references. Shifts the architecture from multi-layer accumulation to in-place compaction, removing legacy rebuild modules.
Introduces a new unitig extraction feature that reads k-mers from an index, filters them by group metadata, and partitions the computation using rayon. The implementation constructs per-partition de Bruijn graphs, merges them, computes node degrees, and writes the resulting sequences as FASTA. A corresponding CLI command is added to obikmer2 to expose this functionality with configurable filtering and progress reporting.
Introduces a new `dump` CLI command to export index k-mers as a CSV table. Adds a reusable `GroupFilterArgs` struct for ingroup/outgroup metadata-predicate quorum filtering with configurable count and fraction thresholds. Refactors the filter command to use explicit flags and a `GenomeSelector`-based pipeline, improving error handling and predicate application order. Exposes `IndexDump` from the `obikdump` crate and updates local dependencies accordingly.
Move the partition iterator implementation from obikdump to obikfilter, updating imports and exposing the type publicly. Add obikidxcache as a local dependency for obikfilter. In obikindexer, expose the new build_layer_from_kmers function to enable shared graph-building logic across pipelines without code duplication.
Move partition iteration logic to obikdump, introducing a FilteredPartitionIter trait over IndexCache for batch-oriented scanning with configurable data retrieval and early termination. Consolidate ingroup and outgroup index storage in GroupQuorumFilter into a unified Selection struct driven by predicate matching. Update dependency manifests to include obikidxcache, rayon, and obikentropy, and remove the deprecated dump_layer module while adjusting public API re-exports.
This commit partitions the obikindex crate into multiple focused subcrates (obikfilter, obikmerge, obikquery, obikrebuild, obikselect, obikstats, obikdump, and obikidxcache) to reduce coupling and clarify module boundaries. It standardizes error handling across the workspace using OKIError and OKIResult, updates index APIs to support lazy, disk-backed partition access, and migrates NUMA system utilities to a new obisys crate. All modifications are structural, focusing on dependency graph expansion, import path updates, and API surface reorganization without altering core runtime behavior.
Eric Coissac