feat: introduce _iqtree_states.csv for compact symbol mapping
Generates a new CSV output that maps IQ-TREE's compact state symbols to canonical states alongside full-precision empirical frequencies. Updates documentation to clarify that state frequencies sum to 1.0 by design and documents conditional behavior under `--free-loss`. Includes unit tests verifying absent state exclusion, frequency summation, and CSV structure. Also restricts entropy annex resolution to non-monomorphic minorants to eliminate redundant per-genome checks.
This commit is contained in:
@@ -887,6 +887,17 @@
|
||||
</ul>
|
||||
</nav>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#_iqtree_statescsv-compact-symbol-traceability-2026-08-15" class="md-nav__link">
|
||||
<span class="md-ellipsis">
|
||||
|
||||
_iqtree_states.csv: compact-symbol traceability (2026-08-15)
|
||||
|
||||
</span>
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
@@ -2095,6 +2106,17 @@
|
||||
</ul>
|
||||
</nav>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#_iqtree_statescsv-compact-symbol-traceability-2026-08-15" class="md-nav__link">
|
||||
<span class="md-ellipsis">
|
||||
|
||||
_iqtree_states.csv: compact-symbol traceability (2026-08-15)
|
||||
|
||||
</span>
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
@@ -4048,6 +4070,42 @@ with a different taxon count crashes with <code>Assertion 'i >= 0 &&
|
||||
the old leaf count onto the new, smaller alignment. Not an <code>obikmer</code> bug;
|
||||
avoid by using a fresh <code>--prefix</code> (or <code>-redo</code>) whenever the underlying
|
||||
alignment's taxon set changes, never <code>--undo</code> across them.</p>
|
||||
<h2 id="_iqtree_statescsv-compact-symbol-traceability-2026-08-15"><code>_iqtree_states.csv</code>: compact-symbol traceability (2026-08-15)</h2>
|
||||
<p>User-reported suspicion: <code>--iqtree --free-loss</code> state frequencies "don't
|
||||
sum to 1". Verified both by code trace and unit test
|
||||
(<code>obikmer/src/cmd/phylo/iqtree.rs</code>, <code>free_loss_excludes_absent_state_and_freq_sums_to_one</code>)
|
||||
— <strong>not a bug</strong>: <code>compact_alphabet</code>'s counting loop <code>continue</code>s on
|
||||
<code>free_loss && b == b'-'</code> <em>before</em> recoding/counting, so the absent state
|
||||
never enters <code>occurs</code>/<code>counts</code>, and <code>freq[i] = counts[old]/total</code> sums to
|
||||
1.0 by construction (<code>total</code> is defined as the sum over exactly the same
|
||||
states). Confirmed against real data the user provided: a pasted
|
||||
<code>_iqtree.model</code> frequency line summed to <code>1.000001</code> — a 6-decimal
|
||||
print-rounding artifact (<code>format!("{p:.6}")</code> × 15 values), not a
|
||||
computation error; IQ-TREE's own 3-decimal-rounded report of the same
|
||||
values summed to exactly <code>1.000</code>.</p>
|
||||
<p>Investigating that report surfaced a real, separate gap: nothing mapped
|
||||
<code>_iqtree.model</code>/<code>_iqtree.fasta</code>'s compact state symbols (<code>0-9A-F</code>, IQ-TREE
|
||||
renumbers away unused states from the full 16, see this module's own
|
||||
"<code>--iqtree</code>... state count" discussion) back to the canonical 16-state
|
||||
alphabet (<code>STATE_SYMBOL</code>, <code>sankoff.rs</code>) — so a pattern like "compact state
|
||||
0 has zero exchangeability with every other state" (<code>R(a,b) =
|
||||
exp(-cost(a,b)) = 0</code> for an entire row/column) couldn't be traced back to
|
||||
which real state that was, or whether it was expected (a state combination
|
||||
that was simply never observed alongside anything else in the calibration
|
||||
data, giving <code>cost = -ln(0) = +∞</code>) or a genuine problem.</p>
|
||||
<p><strong>Fix</strong>: <code>write_iqtree_states_csv</code> (<code>iqtree.rs</code>) writes
|
||||
<code><prefix>_iqtree_states.csv</code> — one row per surviving state,
|
||||
<code>iqtree_symbol,canonical_symbol,frequency</code>, frequency at full <code>f64</code>
|
||||
precision (not the model file's truncated 6 decimals). Written alongside
|
||||
<code>_iqtree.model</code>/<code>_iqtree.fasta</code> from the same <code>CompactAlphabet</code> both
|
||||
already use, so there is no risk of the three files disagreeing. Covered
|
||||
by <code>states_csv_maps_compact_symbols_back_to_canonical_ones</code>.</p>
|
||||
<p>The zero-exchangeability pattern itself (state 0 in the user's report,
|
||||
frequency 3.26%, <code>R=0</code> with every other state) is not yet explained —
|
||||
plausibly a genuinely unobserved transition in the calibration
|
||||
(<code>cardinality_transitions</code>/<code>composition_transitions</code> count <code>0</code> for every
|
||||
pair involving it), which is a legitimate, if numerically extreme, result
|
||||
of <code>-ln(0)</code>, not necessarily a bug — not investigated further.</p>
|
||||
<h2 id="references">References</h2>
|
||||
<p>The Mash mutation-rate model this discussion contrasts with:
|
||||
(Fan <em>et al.</em> 2015; Marbl Lab 2026)<sup id="fnref:Mash-distances-doc"><a class="footnote-ref" href="#fn:Mash-distances-doc">1</a></sup> <sup id="fnref:Fan2015-mash-formula"><a class="footnote-ref" href="#fn:Fan2015-mash-formula">2</a></sup>.</p>
|
||||
|
||||
Reference in New Issue
Block a user