From 331b9d330da56c3c2aac26a50eee9314e24ee883 Mon Sep 17 00:00:00 2001 From: Eric Coissac Date: Sat, 12 Sep 2026 12:57:04 +0200 Subject: [PATCH] Introduce super-kmers and hash-based minimizer selection Defines super-kmers as the fundamental unit of work, including the definition of canonical super-kmers. Implements a hash-based strategy for selecting minimizers, replacing or augmenting standard lexicographic ordering. --- theory-kmers_and_superkmers.md | 18 ++++++++++- theory-minimizer_selection.md | 56 +++++++++++++++++++++++++++------- 2 files changed, 62 insertions(+), 12 deletions(-) diff --git a/theory-kmers_and_superkmers.md b/theory-kmers_and_superkmers.md index 44e39df..dacb1fd 100644 --- a/theory-kmers_and_superkmers.md +++ b/theory-kmers_and_superkmers.md @@ -11,7 +11,7 @@ A **kmer** is a DNA subsequence of fixed length $k$. Two constraints apply to $k A **super-kmer** is a maximal run of consecutive, overlapping kmers from a read that share the same canonical minimizer (see [Minimizer selection](theory-minimizer_selection)). Each kmer in the run overlaps the next by $k-1$ nucleotides. A super-kmer is capped at 256 nucleotides; a longer run is split at that boundary. -For a random minimizer of length $m$ over kmers of length $k$, the expected length of a super-kmer is approximately (@Zheng2020-ji; @Golan2025-xf): +For a random minimizer of length $m$ over kmers of length $k$, the expected length of a super-kmer is approximately (Golan & Shur 2025; Zheng *et al.* 2020): $$L_{\text{nt}} \approx \frac{k-m+2}{2} + k - 1$$ @@ -22,3 +22,19 @@ For $k=31$, $m=13$ this is about 40 nucleotides; in practice super-kmers rarely A **canonical super-kmer** is the lexicographic minimum of a super-kmer and its reverse complement. When a read and its reverse complement are both encountered, they produce super-kmers that are reverse complements of each other; both reduce to the same canonical super-kmer, so a genomic region is represented once regardless of which strand was read. Super-kmers are the unit of work used throughout construction and querying: sequences are decomposed into super-kmers first, and every downstream step (partition routing, deduplication, counting) operates on them rather than on individual kmers. + +
+ +
+ +Golan, S. & Shur, A.M. (2025). [Expected density of random minimizers](https://doi.org/10.1007/978-3-031-82670-2\_25). In: *Lecture notes in computer science*, Lecture notes in computer science. Springer Nature Switzerland, Cham, pp. 347–360. + +
+ +
+ +Zheng, H., Kingsford, C. & Marçais, G. (2020). [Improved design and analysis of practical minimizers](https://doi.org/10.1093/bioinformatics/btaa472). *Bioinformatics (Oxford, England)*, 36, i119–i127. + +
+ +
diff --git a/theory-minimizer_selection.md b/theory-minimizer_selection.md index 13d19ad..48104ba 100644 --- a/theory-minimizer_selection.md +++ b/theory-minimizer_selection.md @@ -8,7 +8,7 @@ The minimizer partitions a sequence into super-kmers: maximal runs of overlappin ## Hash-based ("random") minimizer -`obikmer` selects minimizers by hash order rather than plain lexicographic order. Ordering m-mers lexicographically on their 2-bit encoding systematically favors AT-rich m-mers (an all-A m-mer always encodes to 0), which causes low-complexity regions to dominate as minimizers and produces unbalanced partitions (@Zheng2020-ji; @Zheng2021-cc; @Pan2024-hb; @Kille2023-px; @Golan2025-xf). +`obikmer` selects minimizers by hash order rather than plain lexicographic order. Ordering m-mers lexicographically on their 2-bit encoding systematically favors AT-rich m-mers (an all-A m-mer always encodes to 0), which causes low-complexity regions to dominate as minimizers and produces unbalanced partitions (Golan & Shur 2025; Kille *et al.* 2023; Pan & Reinert 2024; Zheng *et al.* 2020, 2021). Instead, a well-distributed hash function $H$ is applied to the canonical (lexicographically minimal) form of each m-mer, and the m-mer with the smallest $H$ value wins. Because $H$ is a bijection with good avalanche properties, every distinct m-mer in a window has an equal chance of holding the minimum hash value, independent of its nucleotide composition. @@ -24,7 +24,7 @@ The hash function is a 64-bit mixing function (splitmix64-style finalizer) appli $$H(x) = \text{mix64}(x \oplus s)$$ -``` +``` text H(x): x ← x ⊕ s x ← x ⊕ (x >> 30) @@ -36,15 +36,15 @@ H(x): The choice of $s$ is not arbitrary. Low-complexity m-mers (homopolymers, short tandem repeats) are disproportionately abundant in real genomes; if one of them happened to be the perpetual argmin of $H$ — as the all-A m-mer is when $s = 0$, since $\text{mix64}(0) = 0$ is a fixed point — it would win far more windows than the composition-uniform behavior established above predicts, not because $H$ favors it, but because that pathological input keeps recurring in real sequence data. Exhaustive checks confirm that, with this seed, the argmin is never a homopolymer or any periodic repeat, for every tested $m$: -| $m$ | argmin (canonical) | decoded sequence | minimal period | -|---|---|---|---| -| 3 | 16 | `CAA` | 3 | -| 5 | 78 | `ACATG` | 5 | -| 7 | 5512 | `CCCGAGA` | 7 | -| 9 | 108760 | `CGGGATCGA` | 9 | -| 11 | 179014 | `AAGGTGTCACG` | 11 | -| 13 | 33759044 | `GAAATACTTCACA` | 13 | -| 15 | 29869313 | `AACTACTTACCAAAC` | 15 | +| $m$ | argmin (canonical) | decoded sequence | minimal period | +|-----|--------------------|-------------------|----------------| +| 3 | 16 | `CAA` | 3 | +| 5 | 78 | `ACATG` | 5 | +| 7 | 5512 | `CCCGAGA` | 7 | +| 9 | 108760 | `CGGGATCGA` | 9 | +| 11 | 179014 | `AAGGTGTCACG` | 11 | +| 13 | 33759044 | `GAAATACTTCACA` | 13 | +| 15 | 29869313 | `AACTACTTACCAAAC` | 15 | If the minimum period length is found to be $m$ as observed in the above table, then the sequence is aperiodic, as no shorter period can be identified. @@ -55,3 +55,37 @@ A super-kmer's partition is the low $p$ bits of its minimizer's hash: $$\text{partition} = H(\text{minimizer}) \bmod 2^p$$ See [Partitioning and indexing architecture](theory-indexing_architecture) for more details. + +
+ +
+ +Golan, S. & Shur, A.M. (2025). [Expected density of random minimizers](https://doi.org/10.1007/978-3-031-82670-2\_25). In: *Lecture notes in computer science*, Lecture notes in computer science. Springer Nature Switzerland, Cham, pp. 347–360. + +
+ +
+ +Kille, B., Garrison, E., Treangen, T.J. & Phillippy, A.M. (2023). [Minmers are a generalization of minimizers that enable unbiased local jaccard estimation](https://doi.org/10.1093/bioinformatics/btad512). *Bioinformatics (Oxford, England)*, 39. + +
+ +
+ +Pan, C. & Reinert, K. (2024). [A simple refined DNA minimizer operator enables 2-fold faster computation](https://doi.org/10.1093/bioinformatics/btae045). *Bioinformatics (Oxford, England)*, 40. + +
+ +
+ +Zheng, H., Kingsford, C. & Marçais, G. (2020). [Improved design and analysis of practical minimizers](https://doi.org/10.1093/bioinformatics/btaa472). *Bioinformatics (Oxford, England)*, 36, i119–i127. + +
+ +
+ +Zheng, H., Kingsford, C. & Marçais, G. (2021). [Sequence-specific minimizers via polar sets](https://doi.org/10.1093/bioinformatics/btab313). *Bioinformatics (Oxford, England)*, 37, i187–i195. + +
+ +