refactor: treat absence as state unless --free-loss and update quartet annexes
Refactor the handling of genome absence to be a distinct state unless `--free-loss` is specified. This change introduces `QuartetConvention` to pair ambiguity policies with the free-loss flag, updating file naming conventions and logic across sampling, masking, and stats modules. - Update `--exclude-genome` description in `pair_coverage_sampling.md` to clarify absence as a state unless `--free-loss` is specified. - Move multi-member genome handling convention from `pair_coverage_sampling.md` to file name conventions. - Clarify persisted annex structure in `pair_coverage_sampling.md` as `PersistentBitVec` per non-monomorphic family with counting convention carried by file name. - Fix initial version's absence handling in `has_bipartition`, the annex, and `--sample-stats` to correctly count index 0 when absence is a state. - Replace entropy-only builder with `AnnexRequest` and `ensure_layer_family_annexes`. - Consolidate sampling parameters into a single `SamplingSpec`. - Update family constraint description in `sampling_design.md`. - Modify uniform draws and `--quartet` removal text in `sampling_design.md`. - Clarify state counting in tests for `--free-loss` option in `sampling_design.md`. - State that absence forms a pair without `--free-loss` but never with it in `sampling_design.md`. - Update `--quartet` option to require at least two distinct states, each carried by at least two genomes. - Define absence as a state unless `--free-loss` is given for family carrier definition. - Clarify `--pair-balance` favors families covering pairs with fewest shared families. - Modify logic for families fitting every tree topology equally well in `phylo.md`. - Change `--quartet` restriction logic to apply after pre-checks involving `--no-ambiguity` and `--exclude-genome`. - Update `obikmer` dependency version from 1.4.4 to 1.4.5 in `Cargo.lock` and `Cargo.toml`. - Modify `annex_selection` in `args/mod.rs` to derive `free_loss`, `entropy`, `quartet`, and `overlap` from `SamplingSpec`. - Update `SamplingArgs::quartet` documentation in `sampling.rs` to describe state determination based on `--free-loss`. - Add `free_loss` field to `AnnexSelection` struct in `family_annexes.rs`. - Change `AnnexRequest` to use `Vec<QuartetConvention>` instead of `Vec<AmbiguityPolicy>`. - Update `request` logic in `family_annexes.rs` to conditionally create `QuartetConvention` based on `self.quartet`. - Modify `dedup` signature in `family_annexes.rs` to accept iterator of type `T: Ord`. - Update `ensure_layer_family_annexes` in `family_annexes.rs` to use `missing_files` and `QuartetConvention`. - Add `missing_files` definition to `family_annexes.rs`. - Change `has_bipartition` in `masking.rs` to filter counts where count is `>= 2`. - Update `is_quartet_informative` in `masking.rs` to accept `free_loss` boolean parameter. - Modify `two_pairs_of_distinct_states_is_informative` and `singleton_variant_is_not_informative` in `masking.rs` to test both `free_loss` policies. - Add `under_free_loss_absence_is_missing_and_forms_no_pair` and `without_free_loss_absence_is_a_state_that_can_form_a_pair` definitions to `masking.rs`. - Update `no_ambiguity_drops_multi_member_genomes_from_the_count` in `masking.rs` to check `is_quartet_informative` under `free_loss=true`. - Add `the_annex_rule_and_the_exact_check_agree_on_the_same_family` definition to `masking.rs`. - Remove `absent_genomes_are_not_a_state` definition from `masking.rs`. - Update `restrict_to_quartet` in `quartet.rs` to accept `QuartetConvention` instead of `AmbiquityPolicy`. - Modify `add_site` in `sample_stats.rs` to handle state 0 as a state participating in bipartition. - Add `absence_as_a_state_makes_a_site_informative` definition to `sample_stats.rs`. - Add `quartet_convention` function to `SamplingSpec` in `subsample.rs`. - Update `sample_index` in `subsample.rs` to use `spec.quartet_convention()` for `AnnexRequest`. - Update `restrict_to_quartet` call in `subsample.rs` to use `spec.quartet_convention()`. - Add comment regarding `--no-ambiguity` counting convention in `helpers.rs`. - Import `QUARTET_ANNEX_FILE_NAMES` and `QuartetConvention` from `quartet_annex` in `mod.rs`. - Add `QuartetConvention` struct with `policy` and `free_loss` fields to `quartet_annex.rs`. - Modify `quartet_annex_file_name` in `quartet_annex.rs` to take `QuartetConvention`. - Add `the_conventions_never_share_a_file` function to `quartet_annex.rs`. - Remove `the_policies_never_share_a_file` definition from `quartet_annex.rs`.
This commit is contained in:
1 parent
c93616b432
commit
5db7b440da
15 files changed
+218
-101
No files matched your search
@@ -226,7 +226,9 @@ Design decision (user): the way genomes are counted depends on `--no-ambiguity`,
|
||||
|
||||
## Implemented: `--quartet` (2026-10-13)
|
||||
|
||||
`--quartet` keeps only the families with at least two called states each carried by at least two genomes (`masking::is_quartet_informative` for the raw masks, `survives_quartet` for the exact check on masked states and `--exclude-genome`). Absence never counts as a state; with `--no-ambiguity`, multi-member genomes are left out of the count. The persisted annex is a plain `PersistentBitVec` (one bit per non-monomorphic family, compact ordinal) — no ad hoc format; the `--no-ambiguity` convention is carried by the file name (`quartet.pbiv`, `quartet.noamb.pbiv`) because that header has no room for a flag. It is a safe pre-filter: excluding genomes can only destroy a bipartition, never create one.
|
||||
`--quartet` keeps only the families with at least two called states each carried by at least two genomes (`masking::is_quartet_informative` for the raw masks, `survives_quartet` for the exact check on masked states and `--exclude-genome`). Genomes are read through `masked_state`, exactly as in the alignment: absence is a state of its own unless `--free-loss` (then it is missing and counts for nothing), and with `--no-ambiguity` multi-member genomes are left out of the count. The persisted annex is a plain `PersistentBitVec` (one bit per non-monomorphic family, compact ordinal) — no ad hoc format; its counting convention (`QuartetConvention`: ambiguity policy × `--free-loss`) is carried by the file name (`quartet.pbiv`, `quartet.noamb.pbiv`, `quartet.diploid.pbiv` for `--free-loss`; `quartet.abs.*.pbiv` for absence-as-a-state) because that header has no room for a flag.
|
||||
|
||||
*Correction (2026-11-19).* The first version never counted absence, whatever `--free-loss` said, which contradicted the alignment: without `--free-loss` a genome lacking the family has the state `∅`, so `A A ∅ ∅` separates two pairs yet was rejected. Fixed in `has_bipartition` (index 0 is now counted, and is only ever populated when absence is a state), in the annex (one file per `QuartetConvention`) and in `--sample-stats` (informative sites follow the same rule). The old file names keep the `--free-loss` semantics, which is exactly the old rule, so annexes already built with `--free-loss` stay valid; the sampling measured below used `--free-loss` and is unaffected. It is a safe pre-filter: excluding genomes can only destroy a bipartition, never create one.
|
||||
|
||||
Plumbing: `AnnexRequest` (which annexes, per convention) and `ensure_layer_family_annexes` (one shared scan for entropy, quartet and overlap) replace the entropy-only builder; the sampling parameters travel as one `SamplingSpec { free_loss, no_ambiguity, quartet, entropy_bias }` instead of a growing positional list.
|
||||
|
||||
|
||||
Reference in new issue
Block a user