feat: add VCF export and refine alignment output infrastructure
Introduce `--vcf` and `--fasta` options for phylo commands, allowing users to export pseudo-alignment data as VCF or FASTA. Implement comprehensive VCF logic including stable virtual coordinates, canonical ID generation, and flexible genotype representation based on ambiguity settings. Refactor internal alignment structures to track k-mer families and site loci, ensuring consistent data flow from scanning through to output generation. - Add `src/obikmer/src/cmd/phylo/alignment_out.rs` with `create` and `write_alignment_outputs` to handle file creation and dispatching of FASTA/VCF formats based on `PhyloArgs`. - Add `src/obikphylo/src/siblings/algorithms/alignment_vcf.rs` implementing VCF-specific logic: `write_vcf`, `alignment`, `cell_alleles`, `majority_base`, and site construction via `site` using fixed $K=11$. Includes helpers for IUPAC state inversion, genotype ordering, and stable coordinate generation. - Update `src/obikmer/src/cmd/phylo/args.rs` to add `fasta` and `vcf` flags and a new `alignment_output()` method that returns true if any output format is requested. Add `pseudo_alignment` field documentation. - Modify `src/obikmer/src/cmd/phylo/mod.rs` to use `alignment_output()` for determining alignment needs, update error messages for `--n-sites`, and import `write_alignment_outputs` from the new module. - Refactor `src/obikphylo/src/siblings/algorithms/alignment.rs` by adding `SiteLocus` struct (partition, layer, family_idx, family_id) and updating `SnpAlignment` and `AlignmentReducer` to track sites. Update `new`, `finish`, and `reduce` to populate site information from `SurvivingFamily`. - Update `src/obikphylo/src/siblings/algorithms/iqtree.rs`: `alignment` now returns `SnpAlignment` with empty sites; update `drop_ascertainment_noninformative` and `recode_symbols_as_absent` to handle the new `sites` field. - Update `src/obikphylo/src/siblings/algorithms/family_scan.rs`: `GeneratedBatch` now includes a `kmers` field; `scan_layer_families` accepts `CanonicalKmer` and passes it to closures instead of masks. - Update `src/obikphylo/src/siblings/algorithms/subsample.rs`: `SurvivingFamily` gains a `kmer` field; `sample_layer` closure accepts and uses this k-mer. - Update `src/obikphylo/src/siblings/algorithms/snp_distance.rs`: `tally_2genomes` initializes `SurvivingFamily` with explicit `CanonicalKmer` values. - Update `src/obikphylo/src/siblings/algorithms/stats.rs`: `sibling_annex_stats` signature adds `_kmer` parameter. - Update `src/obikphylo/src/siblings/algorithms/family_annexes.rs`: closures in `ensure_layer_family_annexes` and `ensure_layer_pair_weight_annexes` now accept `_kmer`. - Update `src/obikphylo/src/siblings/algorithms/pairwise.rs`: `ambiguous_vs_single` initializes k-mer using `CanonicalKmer::from_raw_unchecked(0)`. - Add `iupac_state` function to `src/obikphylo/src/siblings/algorithms/masking.rs` mapping DNA characters to binary codes. - Update `src/obikphylo/src/siblings/algorithms/mod.rs` to export `SiteLocus` and include new `alignment_vcf` module. - Add `family_id` and `write_family_id_ascii` methods to `src/obikseq/src/kmer.rs` for stable k-mer family identification. - Add tests `family_id_partitions_kmers_into_families` and `family_id_ascii_masks_centre` to `src/obikseq/src/tests/kmer.rs`. - Update `src/obikphylo/src/siblings/mod.rs` imports to include new algorithm exports.
This commit is contained in:
1 parent
ee62b131f9
commit
b132f0847f
20 files changed
+558
-37
No files matched your search
@@ -2873,6 +2873,57 @@ user, no external consumers to break). Scoped to the distance matrix only:
|
||||
`--shared-kmers` and `--raw-snp-counts` are counts, not distances, and keep
|
||||
their existing CSV-only format.
|
||||
|
||||
## VCF export of the pseudo-alignment
|
||||
|
||||
`phylo --vcf` writes the drawn sites of `--pseudo-alignment` as a VCF
|
||||
(`--fasta`/`--vcf` choose the formats, FASTA is the default, both can be
|
||||
combined). A family has no genomic coordinate, so coordinates are virtual and
|
||||
derived from where the scan finds the family: `CHROM = obikmer_<partition>_<layer>`,
|
||||
`POS = family_idx + 1` (`scan_layer_families`' iteration-order index inside
|
||||
that layer). Stable for a given index + annex, sorted by construction because
|
||||
`on_family` is replayed in `family_idx` order. A global dense rank or a bit
|
||||
packing of (layer, partition, idx) into one `POS` was rejected: it needs
|
||||
per-layer minorant counts and can exceed VCF's 2^31-1 bound.
|
||||
|
||||
`ID` is `CanonicalKmer::family_id()`: centre forced to `A` on the
|
||||
k-mer and on its revcomp, each canonicalised, minimum of the two (centre
|
||||
printed as `N`). Index-independent, hence the key for merging VCFs from
|
||||
different indexes. No `INFO/LOCUS` duplicate (dropped: same string as `ID`, costly
|
||||
at 5M loci); `QUAL`/`FILTER`/`INFO` are all `.`. An alternative defined as the minimum of
|
||||
`central_canonical_neighbors()` was dropped: it canonicalises 4 k-mers where
|
||||
this one needs 2, with no other benefit. `family_id` idempotence and
|
||||
family-wide equality are tested exhaustively for k = 3, 5, 7.
|
||||
|
||||
`SurvivingFamily` now carries the minorant k-mer (the scan callback passes it),
|
||||
and `SnpAlignment` a `SiteLocus` per column. Alleles are the unambiguous cells
|
||||
of the IUPAC matrix, in the FASTA's convention (central base of the member's
|
||||
canonical form, not necessarily the `ID` k-mer's strand).
|
||||
|
||||
Absence handling. `masked_state` keeps absence (`m == 0`, `-`) as a real state
|
||||
unless `--free-loss`, where it becomes missing (`?`). A first version mapped
|
||||
`-` and `?` both to `.`: on a toy 4-genome index half of the lines then had
|
||||
`ALT = .` (presence/absence-only sites, selected by `survives_masking`), i.e.
|
||||
polymorphic sites rendered as monomorphic. Now `-` is the extra allele `*`
|
||||
(VCF >= 4.2 "allele missing due to upstream deletion"), after the bases; `?`
|
||||
and IUPAC ambiguity codes stay `.` (haploid GT cannot express a genome
|
||||
carrying several forms; superseded below). The writer needs no knowledge of
|
||||
`--free-loss`: `-` simply never occurs with it. A reference-genome option
|
||||
(`--vcf-ref`) was implemented then dropped: `REF` is always the per-site
|
||||
majority base (alphabetical on ties).
|
||||
|
||||
Ploidy. VCF allows any ploidy in `GT` (number of alleles written, variable
|
||||
per sample/site), but an obikmer cell only says *which* forms a genome
|
||||
carries, not their multiplicity (`A/A/G` and `A/G/G` are both `R`), nor
|
||||
whether several forms mean heterozygosity or paralogues. Compromise chosen:
|
||||
`--no-ambiguity` => haploid (ambiguous cells are already `?`); otherwise
|
||||
diploid, single base `X` -> `X/X`, two-base code -> `X/Y`, `-` -> `*/*`, and
|
||||
three/four-base codes or `?` -> `./.`. `REF` is the majority allele counting
|
||||
each copy once (so a het contributes 1+1, a homozygote 2). Counting copies
|
||||
for a real dosage (polyploids) would need k-mer counts (`--with-counts`),
|
||||
not used by the pseudo-alignment scan today. Toy index: default mode 187
|
||||
lines, no `ALT = .`; `--no-ambiguity` 93 lines (the sites whose polymorphism
|
||||
was only ambiguity codes disappear, since those cells became `?`).
|
||||
|
||||
## References
|
||||
|
||||
The Mash mutation-rate model this discussion contrasts with:
|
||||
|
||||
Reference in new issue
Block a user