refactor: replace no_ambiguity bool with AmbiguityPolicy enum
Replace the binary `no_ambiguity` flag with an `AmbiguityPolicy` enum (`Keep`, `Forbid`, `OnlyDiploid`) to support multiple ambiguity conventions. This change updates CLI arguments, internal logic, and persistent storage formats to distinguish between keeping all genomes, forbidding ambiguous ones, or restricting analysis to diploid sites only. - Add `src/obikphylo/src/siblings/ambiguity.rs` defining the `AmbiguityPolicy` enum with variants `Keep`, `Forbid`, and `OnlyDiploid`, including methods for header flags and determining if a genome is dropped. - Update `PhyloArgs` in `args.rs` to derive `AmbiquityPolicy` from CLI flags and construct `AnnexSelection` for sibling annexes. - Modify `SamplingSpec` in `subsample.rs` to store `ambiguity: AmbiguityPolicy` instead of `no_ambiguity`, updating masking and scalar value logic accordingly. - Update `SiblingExt` in `sibling_ext.rs` so `build_sibling_annex` accepts an `AnnexSelection` to build only requested derived annexes, and `family_overlap` uses the specific policy. - Refactor `FamilyOverlap` in `family_overlap.rs` to store and use `AmbiguityPolicy`, replacing boolean checks with policy-based carrier determination. - Update `FamilyScalarAnnex` in `family_scalar_annex.rs` to store `AmbiguityPolicy` in headers, enabling distinct file names for each policy variant. - Modify `quartet_annex.rs` to generate distinct file names based on `AmbiguityPolicy`, ensuring no overlap between Keep, Forbid, and OnlyDiploid annexes. - Update `family_annexes.rs` to use `AnnexSelection` for building entropy, quartet, and pair weight annexes under specific policies instead of a single boolean convention. - Refactor algorithm modules (`entropy.rs`, `masking.rs`, `pair_weight.rs`, `quartet.rs`, `snp_distance.rs`) to accept `AmbiguityPolicy` arguments, replacing boolean checks with policy-specific logic (e.g., `OnlyDiploid` masking). - Update documentation in `phylo.md` and `evolutionary_distances.md` to describe the new `--only-diploid` option, the behavior of `--sibling-annex` with specific policies, and the separate storage of entropies per convention.
This commit is contained in:
1 parent
b55123a376
commit
5368c98875
21 files changed
+477
-278
No files matched your search
@@ -48,7 +48,7 @@ See [Central-position SNP model](../theory/phylogeny/snp_based/central_snp_model
|
||||
|
||||
| Option | Description |
|
||||
|---|---|
|
||||
| `--sibling-annex` | Build (or rebuild) the sibling-count/minorant annex, then every annex derived from it (entropy, quartet, pair weight, family overlap, under both `--no-ambiguity` conventions). Every option in this section needs the sibling annex, and builds it by itself, under an exclusive lock on the index, if it is missing (as it is after `migrate-mphf`); running `--sibling-annex` first does the same work once, which is the safe way to start several `phylo` runs on the same index at the same time |
|
||||
| `--sibling-annex` | Build (or rebuild) the sibling-count/minorant annex. Rebuilding it makes the annexes derived from it obsolete, so they are dropped; then only the derived annexes that the other options of the same command read are rebuilt, each under the `--no-ambiguity` / `--only-diploid` convention given: `--entropy` the entropy annex, `--quartet` the quartet annex, `--pair-balance` the pair-weight and family-overlap annexes, `--family-overlap` or `--min-shared-family` the family-overlap annex. For example `--sibling-annex --quartet --only-diploid` prepares exactly the quartet annex of that convention, and `--sibling-annex` alone prepares none. An annex that is not prepared is built later, on demand, by the computation that reads it. Every option in this section needs the sibling annex, and builds it by itself, under an exclusive lock on the index, if it is missing (as it is after `migrate-mphf`); running `--sibling-annex` first does the same work once, which is the safe way to start several `phylo` runs on the same index at the same time |
|
||||
| `--sibling-stats` | Write `<prefix>_siblings.csv`: the family-size distribution, per genome and globally |
|
||||
| `--sibling-hist` | Print the global family-size histogram (1-4 members) only |
|
||||
| `--family-overlap` | Write `<prefix>_family_overlap.csv`: for every genome pair, how many variable families both genomes carry a call for |
|
||||
@@ -146,7 +146,7 @@ With `--n-sites all` (or, for a `snp-*` `--distance` value, without `--n-sites`)
|
||||
|
||||
A family carried by fewer than four genomes, or whose genomes all share one state except a single variant, fits every tree topology equally well and so tells a tree nothing: to separate two pairs of genomes from each other (`AB|CD`) it takes at least two distinct states, each carried by at least two genomes. `--quartet` restricts the draw to the families that show this pattern. Absence is never a state here: a genome that does not carry the family does not count, with or without `--free-loss`. A genome carrying several members of the family counts as a carrier of its own combination of bases, unless `--no-ambiguity` is given, in which case it is left out, as in the alignment.
|
||||
|
||||
The restriction is applied before the draw, from a persisted annex (one bit per variable family, built the first time `--quartet` is used with a given `--no-ambiguity` convention, in one pass shared with the other annexes, and built for both conventions by `--sibling-annex`), then checked exactly on each family drawn, taking `--exclude-genome` into account. The quota given to `--n-sites` is therefore spent only on usable families.
|
||||
The restriction is applied before the draw, from a persisted annex (one bit per variable family, built the first time `--quartet` is used with a given `--no-ambiguity` / `--only-diploid` convention, in one pass shared with the other annexes, or ahead of time by `--sibling-annex --quartet`), then checked exactly on each family drawn, taking `--exclude-genome` into account. The quota given to `--n-sites` is therefore spent only on usable families.
|
||||
|
||||
### Describing a draw: `--sample-stats`, `--sample-plan`
|
||||
|
||||
@@ -170,7 +170,7 @@ A call is any state other than the empty one: absence is never a call, with or w
|
||||
|
||||
Two genomes that are distantly related share few families, so a uniform draw covers such a pair with few sites, far fewer than a close pair: the pairs that matter for the deep branches of a tree are the worst represented. `--pair-balance` compensates. Each family is weighted by the mean, over the pairs of genomes carrying it, of 1 divided by the number of families that pair shares across the whole index (the matrix `--family-overlap` writes): a family carried by a rarely-sharing pair weighs more. It is the mean, not the sum, so a family carried by most genomes — in skimming data often a high-copy region — is not favoured merely for covering many pairs.
|
||||
|
||||
Each layer's share of the draw is proportional to the total weight of its families, and a family's chance of being drawn is proportional to its own weight. The weights are persisted (one value per variable family, built the first time `--pair-balance` is used with a given `--no-ambiguity` convention, after the family-overlap matrix it depends on, and by `--sibling-annex` for both conventions). They combine with `--entropy` and `--quartet`: the weight multiplies the entropy kernel, and `--quartet` restricts the families first.
|
||||
Each layer's share of the draw is proportional to the total weight of its families, and a family's chance of being drawn is proportional to its own weight. The weights are persisted (one value per variable family, built the first time `--pair-balance` is used with a given `--no-ambiguity` / `--only-diploid` convention, after the family-overlap matrix it depends on, or ahead of time by `--sibling-annex --pair-balance`). They combine with `--entropy` and `--quartet`: the weight multiplies the entropy kernel, and `--quartet` restricts the families first.
|
||||
|
||||
### `--entropy MU` / `--entropy-sd SIGMA`: biasing the sample toward informative families
|
||||
|
||||
@@ -178,7 +178,7 @@ By default, `--n-sites N` draws families uniformly. With `--entropy`/`--entropy-
|
||||
|
||||
The first `phylo` run on a given index that uses `--entropy`/`--entropy-sd` pays a one-time extra cost (every candidate family's entropy is computed once and saved alongside the index); later runs, even with different `MU`/`SIGMA`, reuse that saved data.
|
||||
|
||||
The entropy that weights the draw follows `--no-ambiguity`, so that it describes the same genomes as the alignment. Without the flag, every genome carrying the family counts, and a genome carrying several members of it is a state of its own (`entropy15`). With the flag, only the genomes carrying exactly one member count, since the others become missing data (`entropy4`). The two entropies are saved separately: switching `--no-ambiguity` builds the other one once, without discarding the first, and `--sibling-annex` builds both.
|
||||
The entropy that weights the draw follows `--no-ambiguity` and `--only-diploid`, so that it describes the same genomes as the alignment. By default, every genome carrying the family counts, and a genome carrying several members of it is a state of its own (`entropy15`). With `--no-ambiguity`, only the genomes carrying exactly one member count, since the others become missing data (`entropy4`). With `--only-diploid`, the genomes carrying one or two members count and those carrying three or four become missing data. The three entropies are saved separately: switching convention builds the missing one once, without discarding the others, and `--sibling-annex --entropy` builds the one of the convention given.
|
||||
|
||||
## Sankoff calibration and phylogenetic exports
|
||||
|
||||
|
||||
Reference in new issue
Block a user