Files
obikmer/UserDocMD/theory/indexing_architecture.md
T
Eric Coissac 9da0876ee6 Refactor indexing logic using hash-based minimizer selection.
Updates indexing architecture to use hashing for minimizer distribution and introduces a new hash-based selection mechanism. Additionally, updates documentation references and implements a new script for converting MkDocs into a flat wiki structure.
2026-10-13 08:34:51 +02:00

1.7 KiB

Partitioning and indexing architecture

An index is split into a fixed number of partitions, each handling an independent, disjoint slice of the kmer space. Partitioning keeps the working set of each stage small enough to process efficiently and enables parallel construction and querying.

Routing

The canonical minimizer of a super-kmer (see Minimizer selection) is hashed to produce a $p$-bit routing value that selects the destination partition:

canonical minimizer → hash(minimizer) → p-bit value → partition index

Within a partition, kmers are indexed as plain values via a minimal perfect hash function (see On-disk storage); the minimizer plays no further role once a super-kmer has reached its partition.

Parameter guidance

Even though H already makes minimizer values well-distributed (see Minimizer selection), choosing p well below the number of bits available in the minimizer (2m) leaves a comfortable entropy margin, provided the number of distinct minimizers actually observed is much larger than the number of partitions.

Minimizer size m Minimizer bits (2m) Typical partition-index bits p Partitions
9 18 6–8 64–256
11 22 8–10 256–1 024
13 26 10–12 1 024–4 096
15 30 10–14 1 024–16 384

The number of partitions must satisfy p \le 2m, and in practice p is chosen well below that bound to leave a comfortable entropy margin. For k=31, m=13, p=10 (1024 partitions), partition load is well balanced on real genomic data.