OBIKmers/obikmer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 27 Wiki Activity

feat: introduce _iqtree_states.csv for compact symbol mapping

Browse Source 
Generates a new CSV output that maps IQ-TREE's compact state symbols to canonical states alongside full-precision empirical frequencies. Updates documentation to clarify that state frequencies sum to 1.0 by design and documents conditional behavior under `--free-loss`. Includes unit tests verifying absent state exclusion, frequency summation, and CSV structure. Also restricts entropy annex resolution to non-monomorphic minorants to eliminate redundant per-genome checks.
This commit is contained in:
Eric Coissac
2026-08-17 09:38:26 +02:00
parent c6cfdac043
commit 9654201885
6 changed files with 230 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
DevDoc/architecture/siblings/index.html
+18 -3
View File
@@ -2051,13 +2051,28 @@ but indexed by <code>family_idx</code> (every minorant of the layer, monomorphic
included — the same numbering <code>Selection</code>/<code>scan_layer_families</code> already
use), one <code>f32</code> entropy15 value per entry, <code>-1.0</code> sentinel for monomorphic/
not-yet-computed. First use of <code>--entropy</code>/<code>--entropy-sd</code> on an index
pays a one-time cost (<code>ensure_entropy_annexes</code> in <code>entropy.rs</code>: a full,
unsampled <code>Selection::All</code> scan, resolving every non-monomorphic
minorant's <code>genome_mask</code> once to compute and persist its entropy) — every
pays a one-time cost (<code>ensure_entropy_annexes</code> in <code>entropy.rs</code>) — every
later run (any <code>μ</code>/<code>σ</code>, any command) reads the file positionally, no
re-scan, restoring the usual <code>Selection::Some</code> "skip resolving excluded
families" speedup that a naive "weigh during the resolving scan" design
would have permanently forfeited.</p>
<p><strong>Bug found and fixed (2026-08-15): <code>ensure_entropy_annexes</code> scanned with
<code>Selection::All</code> instead of bounding to non-monomorphic minorants.</strong>
Monomorphism (<code>family_size() &lt; 2</code>) is knowable directly from the annex
bits alone, no per-genome resolution needed — but the original
implementation called <code>scan_layer_families</code> with <code>Selection::All</code>
anyway, so <code>fill_sub_matrix_carries</code> (the expensive per-genome
resolution) ran for <em>every</em> minorant, ~98% of which are monomorphic
(measured elsewhere in this doc) and had their <code>genome_mask</code> immediately
discarded once the callback checked <code>family_size() &lt; 2</code>. Fixed by adding
[<code>subsample::non_monomorphic_selection_layer</code>] — a cheap, annex-only,
non-sampling pass (same shape as <code>reservoir_sample_layer</code>, but keeping
every non-monomorphic minorant's <code>family_idx</code> instead of a bounded
reservoir) — and passing <code>Selection::Some(&amp;eligible)</code> instead of
<code>Selection::All</code>, so the expensive resolution now runs only for the ~2%
of minorants that can actually produce a real entropy value. A
<code>debug_assert!(mask.family_size() &gt;= 2, ...)</code> inside the
<code>scan_layer_families</code> callback guards the invariant.</p>
<p><strong>Resolved</strong>: the existing hard "non-monomorphic minorant" eligibility
filter stays a hard gate upstream of the Gaussian weighting — only
qualifying families ever get a stored entropy value or a weighted draw.</p>