Commit Graph
6 Commits
Author SHA1 Message Date
Eric Coissac 76cbd3a886 Centralize partition metadata access and add layer introspection APIs
Replaced scattered direct metadata loading with centralized instance methods on `KmerPartition` to guarantee consistent error mapping and legacy recovery. Introduced `StorageKind`, `LayerContent`, and `EvidenceKind` enums alongside lightweight disk-probe methods that inspect file presence without opening heavy data structures. Updated callers across the index, partitioner, and phylo modules to use the new partition API, and added unit tests validating the introspection behavior.
2026-08-20 15:29:53 +02:00
Eric Coissac f7ebc7a1ab Centralize partition directory resolution and add caching design spec
Introduce a design specification outlining performance bottlenecks in layer data access and an agreed-upon implementation direction for caching. Refactor the codebase to centralize index and layer directory path resolution within the partition object, replacing manual string joining and external helper functions with dedicated accessor methods.
2026-08-20 15:10:57 +02:00
Eric Coissac 89ea077456 Add benchmark pipeline for dense and sparse query testing
Introduces a complete query benchmark track to evaluate performance and verify consistency between dense and sparse index formats. Adds scripts to simulate fixed-size paired-end reads, pack a sparse presence index, execute queries in both modes, and capture wall time and RSS metrics. Includes a verification step that compares outputs by read ID to ensure content identity across parallel processing. Updates build configuration, documentation, and ignore patterns to support the new pipeline for two microbial specimens.
2026-08-20 13:59:12 +02:00
Eric Coissac 5a9d903e51 chore: update dependencies and adapt to updated crate APIs
Bumps core dependencies including ndarray, rand, hashbrown, niffler, ureq, sysinfo, indicatif, lru, and remove_dir_all. Adapts source code to accommodate breaking changes by migrating RNG initialization, adjusting HTTP response handling, and replacing the fs4 crate with standard library file locking. Adds a planning document for query benchmarking and sparse index regression tests.
2026-08-20 13:45:41 +02:00
Eric Coissac 0da725ffe9 refine k-mer index architecture documentation and remove obsolete spec
Introduces raw mapping and iteration APIs that bypass membership checks, clarifies variant-specific storage layouts and auto-detection logic, and documents optimized batch access patterns with caller-provided buffers. Removes the outdated obicompactvector_reflexion.md specification to consolidate architectural details into current implementation docs.
2026-08-20 13:20:29 +02:00
Eric Coissac 7bac0f3850 large refactoring 2026-08-17 09:28:53 +02:00