feat: implement persistent layered index and chunked binary format
Introduce the `obilayeredmap` specification and persistent MPHF-based index architecture for incremental multi-dataset indexing. Implement chunked binary serialization with a fixed `u8` k-mer count limit (256) and overlapping super-kmer segments. Add memory-mapped I/O and a companion `.idx` index file for allocation-free, O(1) unitig access. Update MkDocs navigation, enhance the k-mer comparison script, and add comprehensive tests for serialization, partitioning, and file I/O pipelines.
This commit is contained in:
1 parent
8c17bf958b
commit
5169f65dc9
24 files changed
+1342
-382
No files matched your search
@@ -44,6 +44,7 @@ nav:
|
||||
- On-disk storage: implementation/storage.md
|
||||
- MPHF selection: implementation/mphf.md
|
||||
- Unitig evidence encoding: implementation/unitig_evidence.md
|
||||
- obilayeredmap crate: implementation/obilayeredmap.md
|
||||
- Architecture:
|
||||
- Sequences: architecture/sequences/invariant.md
|
||||
|
||||
|
||||
Reference in new issue
Block a user