feat: implement persistent layered index and chunked binary format

Introduce the `obilayeredmap` specification and persistent MPHF-based index architecture for incremental multi-dataset indexing. Implement chunked binary serialization with a fixed `u8` k-mer count limit (256) and overlapping super-kmer segments. Add memory-mapped I/O and a companion `.idx` index file for allocation-free, O(1) unitig access. Update MkDocs navigation, enhance the k-mer comparison script, and add comprehensive tests for serialization, partitioning, and file I/O pipelines.
2026-05-09 17:20:08 +08:00
parent 8c17bf958b
commit 5169f65dc9
24 changed files with 1342 additions and 382 deletions
@@ -44,6 +44,7 @@ nav:
      - On-disk storage: implementation/storage.md
      - MPHF selection: implementation/mphf.md
      - Unitig evidence encoding: implementation/unitig_evidence.md
+      - obilayeredmap crate: implementation/obilayeredmap.md
  - Architecture:
      - Sequences: architecture/sequences/invariant.md