Table of Contents
Partitioning and indexing architecture
An index is split into a fixed number of partitions, each handling an independent, disjoint slice of the kmer space. Partitioning keeps the working set of each stage small enough to process efficiently and enables parallel construction and querying.
Routing
The canonical minimizer of a super-kmer (see Minimizer selection) is hashed to produce a $p$-bit routing value that selects the destination partition:
canonical minimizer → hash(minimizer) → p-bit value → partition index
Within a partition, kmers are indexed as plain values via a minimal perfect hash function (see On-disk storage); the minimizer plays no further role once a super-kmer has reached its partition.
Parameter guidance
Even though H already makes minimizer values well-distributed (see Minimizer selection), choosing p well below the number of bits available in the minimizer (2m) leaves a comfortable entropy margin, provided the number of distinct minimizers actually observed is much larger than the number of partitions.
Minimizer size m |
Minimizer bits (2m) |
Typical partition-index bits p |
Partitions |
|---|---|---|---|
| 9 | 18 | 6–8 | 64–256 |
| 11 | 22 | 8–10 | 256–1 024 |
| 13 | 26 | 10–12 | 1 024–4 096 |
| 15 | 30 | 10–14 | 1 024–16 384 |
The number of partitions must satisfy p \le 2m, and in practice p is chosen well below that bound to leave a comfortable entropy margin. For k=31, m=13, p=10 (1024 partitions), partition load is well balanced on real genomic data.
Wiki sidebar
Theory
Kmer indexing
- DNA encoding
- Kmers
- Minimizer selection
- Super-kmers
- Partitioning and indexing architecture
- Low-complexity kmer filter
Phylogeny
Kmer-based
SNP-based
Usage
- superkmer
- index
- merge
- filter
- select
- query
- dump
- annotate
- phylo
- unitig
- estimate
- convert
- utils
- pack
- Predicates and taxonomy paths