Files
obikmer/DevDocMD/index.md
T
Eric Coissac c93616b432 feat: add threshold-based Hamming distance and refine phylo documentation
Add `threshold_hamming_dist_matrix` to calculate k-mers present in exactly one genome using union minus intersection. Update `DistanceArgs` and `distance.rs` to support this metric on count indexes via `--presence-threshold`, removing the previous restriction that hamming was only for presence/absence indexes.

- Add `threshold_hamming_dist_matrix` method to `CountPartials` in `src/obikmer/src/traits.rs`
- Implement new test `threshold_hamming_counts_kmers_present_in_one_genome_only` in `src/obicompactvec/src/tests/intmatrix.rs`
- Update `DistanceArgs` in `src/obikmer/src/cmd/phylo/args/distance.rs` to clarify threshold effects on Jaccard/Mash/Hamming
- Modify `distance` function in `src/obikphylo/src/distance.rs` to support Hamming on count indexes and use pre-computed bit structures
- Update `obicompactvec.md` documentation with new function details and partial result explanations
- Refine `index.md` descriptions for `phylo`, `unitig`, and `select` options
2026-11-19 07:11:57 +01:00

5.1 KiB

obikmer

obikmer is a Rust tool for manipulation, counting, indexing, and set operations on DNA sequences represented as kmer sets.

Subcommands

Subcommand Purpose
superkmer Extract super-kmers from a sequence file and write to stdout
index Build a complete genome index (scatter → dereplicate → count → layered MPHF)
merge Merge multiple built indexes into one
filter Apply row-level selection (σ) to an index: retain only k-mers matching the ingroup/outgroup predicates. Output is a new single-layer index — compaction is a consequence, not the goal. Supports the shared kmer filtering system
query Query an index with sequences and annotate matches
dump Dump all indexed k-mers as CSV (kmer + per-genome counts or presence); supports the shared kmer filtering system; --head N limits output to the first N k-mers
annotate Add or update genome metadata from a CSV file; or dump metadata as CSV
phylo Genome-vs-genome distances, trees and SNP-based pseudo-alignments. --distance selects either a whole-genome distance computed from k-mer content (jaccard, mash, hamming, bray-curtis, relfreq-bray-curtis, euclidean, relfreq-euclidean, hellinger, hellinger-euclidean; --presence-threshold N converts counts to presence for Jaccard/Mash/Hamming on count indexes) or a snp-* distance (snp-raw, snp-jc, snp-k2p, snp-k81, snp-f81, snp-t92, snp-tn93, snp-tv; optional --gamma-shape) computed from central-position SNPs of k-mer families. --nj/--upgma build trees (Jaccard if no --distance); --shared-kmers writes the shared-k-mer count matrix on its own. SNP analyses read the sibling annex, built on demand (--sibling-annex forces it), with site selection/sampling options (--n-sites, --quartet, --pair-balance, --entropy, --free-loss, --no-ambiguity/--only-diploid, --session), a pseudo-alignment export (--pseudo-alignment, --fasta, --vcf) and Sankoff cost calibration with TNT/PhyG/IQ-TREE exports. See evolutionary distances
unitig Build a global de Bruijn graph across all partitions and enumerate its unitigs as FASTA; supports the shared kmer filtering system
select Project and/or aggregate genome columns into a new or in-place index; the column-axis counterpart of filter (see select)
estimate Estimate approximate-index parameters (z, evidence bits, FP rates) before indexing
convert Convert an index's evidence representation (exact / approximate / hybrid), in place
utils Miscellaneous index utilities: --new-label NEW=OLD renames a genome label; --upgrade-index adds missing layer_meta.json to old indexes
pack Pack per-column matrix files into single-file format to reduce query I/O

Constraints

  • Target scale: individual genome datasets, tens of Gbases
  • Maximum efficiency in computation, memory, and disk usage
  • k odd, k ∈ [11, 31], fixed at runtime; kmer fits in a u64 (2 bits/base)
  • Canonical form: min(kmer, revcomp(kmer)) reduces strand-symmetric space by half
  • Input formats for index/superkmer: FASTA (.fa, .fasta), FASTQ (.fq, .fastq), GenBank flat file (.gb, .gbk, .gbff), all optionally gzip-compressed; directories expanded recursively; streaming stdin via -
  • Input formats for query: FASTA, FASTQ, optionally gzip-compressed; streaming stdin via -

Parameter constraints (enforced at CLI)

All constraints below are checked by CommonArgs::validate() at the start of superkmer and index. Invalid values exit immediately with an error.

Parameter Constraint Reason
k (--kmer-size) odd even k allows palindromic k-mers: kmer == revcomp(kmer), breaking the canonical form invariant
k (--kmer-size) k ∈ [11, 31] k > 31 overflows u64 at 2 bits/base; k < 11 gives insufficient specificity
m (--minimizer-size) odd same palindrome argument as k
m (--minimizer-size) 3 ≤ m ≤ k−1 minimizer must be strictly shorter than the kmer
z (-z, Findere, index --approx only) z ≤ k−1 effective indexed kmer size is k−z+1; z ≥ k would make it ≤ 0

Genome label constraints

Genome labels are arbitrary Unicode strings with the following restrictions:

Character Forbidden Reason
/ yes filesystem path separator
= yes --new-label parser separator
\0 yes null byte
\n \r \t yes break CSV output
spaces allowed use shell quoting: --new-label 'new label=old label'

Empty labels are also rejected. Labels derived automatically from the index directory name (when --label is omitted) are not validated since they come from the filesystem and are already safe.

Priority operations

  • Kmer counting (frequencies)
  • Fast search / query
  • Set operations: union, intersection, difference