feat: add threshold-based Hamming distance and refine phylo documentation
Add `threshold_hamming_dist_matrix` to calculate k-mers present in exactly one genome using union minus intersection. Update `DistanceArgs` and `distance.rs` to support this metric on count indexes via `--presence-threshold`, removing the previous restriction that hamming was only for presence/absence indexes. - Add `threshold_hamming_dist_matrix` method to `CountPartials` in `src/obikmer/src/traits.rs` - Implement new test `threshold_hamming_counts_kmers_present_in_one_genome_only` in `src/obicompactvec/src/tests/intmatrix.rs` - Update `DistanceArgs` in `src/obikmer/src/cmd/phylo/args/distance.rs` to clarify threshold effects on Jaccard/Mash/Hamming - Modify `distance` function in `src/obikphylo/src/distance.rs` to support Hamming on count indexes and use pre-computed bit structures - Update `obicompactvec.md` documentation with new function details and partial result explanations - Refine `index.md` descriptions for `phylo`, `unitig`, and `select` options
This commit is contained in:
1 parent
c4eacb3f72
commit
c93616b432
6 files changed
+33
-13
No files matched your search
@@ -509,6 +509,7 @@ Provided finalisations:
|
||||
| `hellinger_dist_matrix()` | `√partial_hellinger[i,j] / √2` |
|
||||
| `hellinger_euclidean_dist_matrix()` | `√partial_hellinger[i,j]` |
|
||||
| `threshold_mash_dist_matrix(k, t)` | Mash distance, derived from `threshold_jaccard_dist_matrix(t)` — no separate partial |
|
||||
| `threshold_hamming_dist_matrix(t)` | `union[i,j] − inter[i,j]` (k-mers present in exactly one genome) — no separate partial |
|
||||
|
||||
### BitPartials
|
||||
|
||||
|
||||
+1
-1
@@ -13,7 +13,7 @@
|
||||
| `query` | Query an index with sequences and annotate matches |
|
||||
| `dump` | Dump all indexed k-mers as CSV (kmer + per-genome counts or presence); supports the shared [kmer filtering](implementation/filtering.md) system; `--head N` limits output to the first N k-mers |
|
||||
| `annotate` | Add or update genome metadata from a CSV file; or dump metadata as CSV |
|
||||
| `phylo` | Genome-vs-genome distances, trees and SNP-based pseudo-alignments. `--distance` selects either a whole-genome distance computed from k-mer content (`jaccard`, `mash`, `hamming`, `bray-curtis`, `relfreq-bray-curtis`, `euclidean`, `relfreq-euclidean`, `hellinger`, `hellinger-euclidean`; `--presence-threshold N` converts counts to presence for Jaccard/Mash on count indexes; `hamming` is currently refused on count indexes) or a `snp-*` distance (`snp-raw`, `snp-jc`, `snp-k2p`, `snp-k81`, `snp-f81`, `snp-t92`, `snp-tn93`, `snp-tv`; optional `--gamma-shape`) computed from central-position SNPs of k-mer families. `--nj`/`--upgma` build trees (Jaccard if no `--distance`); `--shared-kmers` writes the shared-k-mer count matrix on its own. SNP analyses read the sibling annex, built on demand (`--sibling-annex` forces it), with site selection/sampling options (`--n-sites`, `--quartet`, `--pair-balance`, `--entropy`, `--free-loss`, `--no-ambiguity`/`--only-diploid`, `--session`), a pseudo-alignment export (`--pseudo-alignment`, `--fasta`, `--vcf`) and Sankoff cost calibration with TNT/PhyG/IQ-TREE exports. See [evolutionary distances](theory/evolutionary_distances.md) |
|
||||
| `phylo` | Genome-vs-genome distances, trees and SNP-based pseudo-alignments. `--distance` selects either a whole-genome distance computed from k-mer content (`jaccard`, `mash`, `hamming`, `bray-curtis`, `relfreq-bray-curtis`, `euclidean`, `relfreq-euclidean`, `hellinger`, `hellinger-euclidean`; `--presence-threshold N` converts counts to presence for Jaccard/Mash/Hamming on count indexes) or a `snp-*` distance (`snp-raw`, `snp-jc`, `snp-k2p`, `snp-k81`, `snp-f81`, `snp-t92`, `snp-tn93`, `snp-tv`; optional `--gamma-shape`) computed from central-position SNPs of k-mer families. `--nj`/`--upgma` build trees (Jaccard if no `--distance`); `--shared-kmers` writes the shared-k-mer count matrix on its own. SNP analyses read the sibling annex, built on demand (`--sibling-annex` forces it), with site selection/sampling options (`--n-sites`, `--quartet`, `--pair-balance`, `--entropy`, `--free-loss`, `--no-ambiguity`/`--only-diploid`, `--session`), a pseudo-alignment export (`--pseudo-alignment`, `--fasta`, `--vcf`) and Sankoff cost calibration with TNT/PhyG/IQ-TREE exports. See [evolutionary distances](theory/evolutionary_distances.md) |
|
||||
| `unitig` | Build a global de Bruijn graph across all partitions and enumerate its unitigs as FASTA; supports the shared [kmer filtering](implementation/filtering.md) system |
|
||||
| `select` | Project and/or aggregate genome columns into a new or in-place index; the column-axis counterpart of `filter` (see [select](implementation/select.md)) |
|
||||
| `estimate` | Estimate approximate-index parameters (z, evidence bits, FP rates) before indexing |
|
||||
|
||||
@@ -249,6 +249,20 @@ fn partial_threshold_jaccard_consistent() {
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn threshold_hamming_counts_kmers_present_in_one_genome_only() {
|
||||
// At threshold 2 the genomes (columns) hold the k-mers (rows):
|
||||
// c0 = {r0, r2}, c1 = {r1}, c2 = {r1}.
|
||||
let (_d, m) = make_matrix(&[&[3, 0, 2], &[1, 2, 0], &[0, 3, 1]]);
|
||||
let h = m.threshold_hamming_dist_matrix(2);
|
||||
let expected = [[0, 3, 3], [3, 0, 0], [3, 0, 0]];
|
||||
for i in 0..3 {
|
||||
for j in 0..3 {
|
||||
assert_eq!(h[[i, j]], expected[i][j], "[{i},{j}]");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn single_col_distance_matrix_is_zero() {
|
||||
let (_d, m) = make_matrix(&[&[1, 2, 3]]);
|
||||
|
||||
@@ -129,6 +129,16 @@ pub trait CountPartials: ColumnWeights {
|
||||
jaccard_to_mash(&self.threshold_jaccard_dist_matrix(threshold), k)
|
||||
}
|
||||
|
||||
/// Hamming distance on presence/absence: the number of k-mers present
|
||||
/// (count ≥ `threshold`) in exactly one of the two genomes, i.e. the union
|
||||
/// minus the intersection that [`partial_threshold_jaccard`] already gives.
|
||||
///
|
||||
/// [`partial_threshold_jaccard`]: Self::partial_threshold_jaccard
|
||||
fn threshold_hamming_dist_matrix(&self, threshold: u32) -> Array2<u64> {
|
||||
let (inter, union) = self.partial_threshold_jaccard(threshold);
|
||||
union - inter
|
||||
}
|
||||
|
||||
fn relfreq_bray_dist_matrix(&self) -> Array2<f64> {
|
||||
let global = self.col_weights();
|
||||
let mut m = self.partial_relfreq_bray(&global).mapv(|v| 1.0 - v);
|
||||
|
||||
@@ -21,8 +21,7 @@ pub struct DistanceArgs {
|
||||
/// mash Mash distance, a divergence estimate derived
|
||||
/// from the Jaccard index
|
||||
/// hamming number of k-mers present in one genome and
|
||||
/// absent from the other (only for indexes
|
||||
/// without counts)
|
||||
/// absent from the other
|
||||
/// The others compare k-mer counts, and need an index that stores them:
|
||||
/// bray-curtis Bray-Curtis dissimilarity on k-mer counts
|
||||
/// relfreq-bray-curtis Bray-Curtis dissimilarity on counts divided
|
||||
@@ -56,9 +55,9 @@ pub struct DistanceArgs {
|
||||
|
||||
/// Minimum k-mer count to count as present
|
||||
///
|
||||
/// Converts counts into presence/absence for the Jaccard and Mash
|
||||
/// distances and for the shared k-mer counts, when the index stores
|
||||
/// counts: a k-mer is present in a genome only when its count reaches
|
||||
/// Converts counts into presence/absence for the Jaccard, Mash and
|
||||
/// Hamming distances and for the shared k-mer counts, when the index
|
||||
/// stores counts: a k-mer is present in a genome only when its count reaches
|
||||
/// this threshold. It has no effect on indexes without counts, nor on
|
||||
/// the distances that use the counts themselves.
|
||||
#[arg(long, default_value = "1")]
|
||||
|
||||
@@ -214,8 +214,8 @@ impl Metrics for IndexCache {
|
||||
let bit = AggregatedBit { cache: self, n_genomes };
|
||||
|
||||
// One construction decides both what the index can serve and how:
|
||||
// the six count-based distances need a count index, Hamming needs a
|
||||
// presence/absence one, Jaccard and Mash work on either.
|
||||
// the six count-based distances need a count index; Jaccard, Mash and
|
||||
// Hamming work on either (on counts, through the presence threshold).
|
||||
let matrix = match (use_counts, metric) {
|
||||
(true, BrayCurtis) => count.bray_dist_matrix(),
|
||||
(true, RelfreqBrayCurtis) => count.relfreq_bray_dist_matrix(),
|
||||
@@ -225,11 +225,7 @@ impl Metrics for IndexCache {
|
||||
(true, HellingerEuclidean) => count.hellinger_euclidean_dist_matrix(),
|
||||
(true, Jaccard) => count.threshold_jaccard_dist_matrix(presence_threshold),
|
||||
(true, Mash) => count.threshold_mash_dist_matrix(kmer_size, presence_threshold),
|
||||
(true, Hamming) => {
|
||||
return Err(OKIError::InvalidInput(
|
||||
"Hamming is only available for presence/absence indexes".into(),
|
||||
));
|
||||
}
|
||||
(true, Hamming) => count.threshold_hamming_dist_matrix(presence_threshold).mapv(|v| v as f64),
|
||||
(false, Jaccard) => bit.jaccard_dist_matrix(),
|
||||
(false, Mash) => bit.mash_dist_matrix(kmer_size),
|
||||
(false, Hamming) => bit.hamming_dist_matrix().mapv(|v| v as f64),
|
||||
|
||||
Reference in new issue
Block a user