Commit Graph
8 Commits
Author SHA1 Message Date
Eric Coissac 5db7b440da refactor: treat absence as state unless --free-loss and update quartet annexes
ci.yml / build (pull_request) Successful in 4m33s
Release / create-release (push) Successful in 1m56s
Release / build-linux-x86_64 (push) Successful in 11m18s
Release / build-macos-arm64 (push) Successful in 2m27s
Refactor the handling of genome absence to be a distinct state unless `--free-loss` is specified. This change introduces `QuartetConvention` to pair ambiguity policies with the free-loss flag, updating file naming conventions and logic across sampling, masking, and stats modules.

- Update `--exclude-genome` description in `pair_coverage_sampling.md` to clarify absence as a state unless `--free-loss` is specified.
- Move multi-member genome handling convention from `pair_coverage_sampling.md` to file name conventions.
- Clarify persisted annex structure in `pair_coverage_sampling.md` as `PersistentBitVec` per non-monomorphic family with counting convention carried by file name.
- Fix initial version's absence handling in `has_bipartition`, the annex, and `--sample-stats` to correctly count index 0 when absence is a state.
- Replace entropy-only builder with `AnnexRequest` and `ensure_layer_family_annexes`.
- Consolidate sampling parameters into a single `SamplingSpec`.
- Update family constraint description in `sampling_design.md`.
- Modify uniform draws and `--quartet` removal text in `sampling_design.md`.
- Clarify state counting in tests for `--free-loss` option in `sampling_design.md`.
- State that absence forms a pair without `--free-loss` but never with it in `sampling_design.md`.
- Update `--quartet` option to require at least two distinct states, each carried by at least two genomes.
- Define absence as a state unless `--free-loss` is given for family carrier definition.
- Clarify `--pair-balance` favors families covering pairs with fewest shared families.
- Modify logic for families fitting every tree topology equally well in `phylo.md`.
- Change `--quartet` restriction logic to apply after pre-checks involving `--no-ambiguity` and `--exclude-genome`.
- Update `obikmer` dependency version from 1.4.4 to 1.4.5 in `Cargo.lock` and `Cargo.toml`.
- Modify `annex_selection` in `args/mod.rs` to derive `free_loss`, `entropy`, `quartet`, and `overlap` from `SamplingSpec`.
- Update `SamplingArgs::quartet` documentation in `sampling.rs` to describe state determination based on `--free-loss`.
- Add `free_loss` field to `AnnexSelection` struct in `family_annexes.rs`.
- Change `AnnexRequest` to use `Vec<QuartetConvention>` instead of `Vec<AmbiguityPolicy>`.
- Update `request` logic in `family_annexes.rs` to conditionally create `QuartetConvention` based on `self.quartet`.
- Modify `dedup` signature in `family_annexes.rs` to accept iterator of type `T: Ord`.
- Update `ensure_layer_family_annexes` in `family_annexes.rs` to use `missing_files` and `QuartetConvention`.
- Add `missing_files` definition to `family_annexes.rs`.
- Change `has_bipartition` in `masking.rs` to filter counts where count is `>= 2`.
- Update `is_quartet_informative` in `masking.rs` to accept `free_loss` boolean parameter.
- Modify `two_pairs_of_distinct_states_is_informative` and `singleton_variant_is_not_informative` in `masking.rs` to test both `free_loss` policies.
- Add `under_free_loss_absence_is_missing_and_forms_no_pair` and `without_free_loss_absence_is_a_state_that_can_form_a_pair` definitions to `masking.rs`.
- Update `no_ambiguity_drops_multi_member_genomes_from_the_count` in `masking.rs` to check `is_quartet_informative` under `free_loss=true`.
- Add `the_annex_rule_and_the_exact_check_agree_on_the_same_family` definition to `masking.rs`.
- Remove `absent_genomes_are_not_a_state` definition from `masking.rs`.
- Update `restrict_to_quartet` in `quartet.rs` to accept `QuartetConvention` instead of `AmbiquityPolicy`.
- Modify `add_site` in `sample_stats.rs` to handle state 0 as a state participating in bipartition.
- Add `absence_as_a_state_makes_a_site_informative` definition to `sample_stats.rs`.
- Add `quartet_convention` function to `SamplingSpec` in `subsample.rs`.
- Update `sample_index` in `subsample.rs` to use `spec.quartet_convention()` for `AnnexRequest`.
- Update `restrict_to_quartet` call in `subsample.rs` to use `spec.quartet_convention()`.
- Add comment regarding `--no-ambiguity` counting convention in `helpers.rs`.
- Import `QUARTET_ANNEX_FILE_NAMES` and `QuartetConvention` from `quartet_annex` in `mod.rs`.
- Add `QuartetConvention` struct with `policy` and `free_loss` fields to `quartet_annex.rs`.
- Modify `quartet_annex_file_name` in `quartet_annex.rs` to take `QuartetConvention`.
- Add `the_conventions_never_share_a_file` function to `quartet_annex.rs`.
- Remove `the_policies_never_share_a_file` definition from `quartet_annex.rs`.
2026-11-19 07:12:36 +01:00
Eric Coissac 0e40e83fd1 Refactor probability formula notation
Update the mathematical formula for $P(X \geq r)$ to explicitly use summation notation and clearly format the probability expression.
2026-10-15 12:00:51 +02:00
Eric Coissac f29dcbcfb8 Implement advanced sampling and family annex logic
Release / create-release (push) Successful in 2m35s
Release / build-linux-x86_64 (push) Successful in 8m19s
Release / build-macos-arm64 (push) Successful in 2m3s
ci.yml / build (pull_request) Successful in 4m8s
Introduces complex sampling mechanisms including quartet filtering, pair balance weighting, and entropy bias. Enforces the `--no-ambiguity` convention for family counting and updates annex storage for scalar data.

This includes new logic for calculating family weights based on overlap, quota-based layer determination, and filtering families based on quartet informativeness. Updates also include argument parsing for `--n-sites` and improved handling of family overlap files.
2026-10-13 08:57:18 +02:00
Eric Coissac 0d12494f1a Refactor indexing, sampling, and entropy documentation
This commit introduces new theoretical frameworks for k-mer indexing, pair coverage sampling, and sibling partition unification. It also implements new algorithms for calculating family entropy and statistical tests for overlap analysis. Documentation is updated across the board to reflect these new features and theoretical concepts.
2026-10-13 08:54:21 +02:00
Eric Coissac 0d1ecac233 Refactor k-mer indexing, enhance session features, and update documentation
This commit introduces significant refinements across the codebase. Core indexing logic is updated with hard constraints on k-mer length, statistical modeling clarifications, and the implementation of the Karlin & Ost theory for chance-collision statistics. Session management features are enhanced with robust memoization capabilities, atomic file operations, per-key locking, and memory mapping support. Additionally, documentation is updated, including a new citation style, and new mathematical notebook support is added.
2026-10-13 08:41:42 +02:00
Eric Coissac e715528d34 Refactor documentation for kmer indexing and phylogenetic models
This commit updates documentation across the board, introducing definitions for kmer constraints, super-kmers, k-mer set distances, SNP models, and sampling design principles. It also clarifies the index construction pipeline and the underlying mathematical models used for genome pair calibration.
2026-10-13 08:38:49 +02:00
Eric Coissac 9da0876ee6 Refactor indexing logic using hash-based minimizer selection.
Updates indexing architecture to use hashing for minimizer distribution and introduces a new hash-based selection mechanism. Additionally, updates documentation references and implements a new script for converting MkDocs into a flat wiki structure.
2026-10-13 08:34:51 +02:00
Eric Coissac 6acafa7f2c docs: add obikmer user guide and MkDocs build configuration
Introduces a comprehensive documentation set covering theoretical foundations, CLI usage, installation, and system architecture. Adds MkDocs configuration and Makefile targets to generate, serve with live reload, and clean the documentation site. Includes citation styles and bibliography files for academic references.
2026-08-13 17:19:01 +02:00