Commit Graph
5 Commits
Author SHA1 Message Date
Eric Coissac 025c40c0db Refactor chunking logic to enforce 128 k-mer limit
This change reduces the maximum number of k-mers per chunk from 256 to 128. This modification requires updating the data model to handle sequences exceeding this limit by splitting them into overlapping chunks, ensuring no k-mers are lost. Extensive testing has been added to validate the new chunking logic and k-mer constraints.
2026-10-15 12:20:36 +02:00
Eric Coissac 8468319a63 Refactor unitig detection and graph traversal logic
Introduce new hashing mechanisms and support for directional unitig checks. Fix fragmentation issues and improve performance metrics through optimized graph construction and new benchmarking utilities.
2026-10-15 12:19:24 +02:00
Eric Coissac d32d86193f Refactor index metadata loading to use buffered I/O
This change replaces unbuffered file reads with buffered I/O operations to improve performance during index metadata loading. This includes implementing new methods for reading index metadata files and wrapping file handles in `BufReader` to optimize subsequent reads.
2026-10-15 12:17:51 +02:00
Eric Coissac 093d81c359 Introduce memory mapping for index files and sparse matrices
This change implements memory-mapped views for index readers and sparse matrices, enabling shared page-cache access. It also includes performance analysis confirming that the default chunk size cap negatively impacts parallelism for small inputs.
2026-10-15 12:17:09 +02:00
Eric Coissac fe38424682 Implement batch lookup and chunked data management
Introduces support for batched MPHF lookups, chunked data handling via `SlottedKmers`, and a comprehensive benchmarking suite to measure performance across the indexing and querying pipeline.

This includes new layer functionality for batch operations, chunking logic in the query layer, and scripts to measure wall time and resource usage for various commands. Documentation related to specific implementation details has also been cleaned up.
2026-10-15 12:14:51 +02:00