Commit Graph
8 Commits
Author SHA1 Message Date
Eric Coissac 8296e3544c Simplify Hybrid layer implementation and update dependencies
Release / create-release (push) Successful in 2m31s
Release / build-linux-x86_64 (push) Successful in 9m24s
Release / build-macos-arm64 (push) Successful in 2m5s
ci.yml / build (pull_request) Successful in 3m51s
Dropped fields related to Hybrid layers for simplification and removed the unused `ahash` dependency. Updated dependency versions for `obidebruinj`, `obikmer`, and related crates, and cleaned up temporary artifacts.
2026-10-15 12:23:15 +02:00
Eric Coissac 12afbe819c Refactor MPHF layer logic and evidence handling
This commit implements core lookup functionality for MPHF layers, including `hash` and `find_strict` methods. It introduces support for different evidence types (Exact, Hybrid, Approximate) within layers, clarifying lookup behavior and improving partition calculation logic in the cache. Additionally, dead code is removed and dependencies are updated.
2026-10-15 12:22:45 +02:00
Eric Coissac f9edefd135 Refactor query system for performance and memory reduction
This change introduces dynamic chunk sizing based on input size estimates, ensuring memory constraints are respected while allowing for smaller chunks that yield significant speed improvements. Additionally, index structures were optimized, resulting in drastic memory footprint reduction and faster query execution times.
2026-10-15 12:21:38 +02:00
Eric Coissac 025c40c0db Refactor chunking logic to enforce 128 k-mer limit
This change reduces the maximum number of k-mers per chunk from 256 to 128. This modification requires updating the data model to handle sequences exceeding this limit by splitting them into overlapping chunks, ensuring no k-mers are lost. Extensive testing has been added to validate the new chunking logic and k-mer constraints.
2026-10-15 12:20:36 +02:00
Eric Coissac 8468319a63 Refactor unitig detection and graph traversal logic
Introduce new hashing mechanisms and support for directional unitig checks. Fix fragmentation issues and improve performance metrics through optimized graph construction and new benchmarking utilities.
2026-10-15 12:19:24 +02:00
Eric Coissac d32d86193f Refactor index metadata loading to use buffered I/O
This change replaces unbuffered file reads with buffered I/O operations to improve performance during index metadata loading. This includes implementing new methods for reading index metadata files and wrapping file handles in `BufReader` to optimize subsequent reads.
2026-10-15 12:17:51 +02:00
Eric Coissac 093d81c359 Introduce memory mapping for index files and sparse matrices
This change implements memory-mapped views for index readers and sparse matrices, enabling shared page-cache access. It also includes performance analysis confirming that the default chunk size cap negatively impacts parallelism for small inputs.
2026-10-15 12:17:09 +02:00
Eric Coissac fe38424682 Implement batch lookup and chunked data management
Introduces support for batched MPHF lookups, chunked data handling via `SlottedKmers`, and a comprehensive benchmarking suite to measure performance across the indexing and querying pipeline.

This includes new layer functionality for batch operations, chunking logic in the query layer, and scripts to measure wall time and resource usage for various commands. Documentation related to specific implementation details has also been cleaned up.
2026-10-15 12:14:51 +02:00