feat: introduce _iqtree_states.csv for compact symbol mapping
Generates a new CSV output that maps IQ-TREE's compact state symbols to canonical states alongside full-precision empirical frequencies. Updates documentation to clarify that state frequencies sum to 1.0 by design and documents conditional behavior under `--free-loss`. Includes unit tests verifying absent state exclusion, frequency summation, and CSV structure. Also restricts entropy annex resolution to non-monomorphic minorants to eliminate redundant per-genome checks.
This commit is contained in:
@@ -2056,6 +2056,47 @@ the old leaf count onto the new, smaller alignment. Not an `obikmer` bug;
|
||||
avoid by using a fresh `--prefix` (or `-redo`) whenever the underlying
|
||||
alignment's taxon set changes, never `--undo` across them.
|
||||
|
||||
## `_iqtree_states.csv`: compact-symbol traceability (2026-08-15)
|
||||
|
||||
User-reported suspicion: `--iqtree --free-loss` state frequencies "don't
|
||||
sum to 1". Verified both by code trace and unit test
|
||||
(`obikmer/src/cmd/phylo/iqtree.rs`, `free_loss_excludes_absent_state_and_freq_sums_to_one`)
|
||||
— **not a bug**: `compact_alphabet`'s counting loop `continue`s on
|
||||
`free_loss && b == b'-'` *before* recoding/counting, so the absent state
|
||||
never enters `occurs`/`counts`, and `freq[i] = counts[old]/total` sums to
|
||||
1.0 by construction (`total` is defined as the sum over exactly the same
|
||||
states). Confirmed against real data the user provided: a pasted
|
||||
`_iqtree.model` frequency line summed to `1.000001` — a 6-decimal
|
||||
print-rounding artifact (`format!("{p:.6}")` × 15 values), not a
|
||||
computation error; IQ-TREE's own 3-decimal-rounded report of the same
|
||||
values summed to exactly `1.000`.
|
||||
|
||||
Investigating that report surfaced a real, separate gap: nothing mapped
|
||||
`_iqtree.model`/`_iqtree.fasta`'s compact state symbols (`0-9A-F`, IQ-TREE
|
||||
renumbers away unused states from the full 16, see this module's own
|
||||
"`--iqtree`... state count" discussion) back to the canonical 16-state
|
||||
alphabet (`STATE_SYMBOL`, `sankoff.rs`) — so a pattern like "compact state
|
||||
0 has zero exchangeability with every other state" (`R(a,b) =
|
||||
exp(-cost(a,b)) = 0` for an entire row/column) couldn't be traced back to
|
||||
which real state that was, or whether it was expected (a state combination
|
||||
that was simply never observed alongside anything else in the calibration
|
||||
data, giving `cost = -ln(0) = +∞`) or a genuine problem.
|
||||
|
||||
**Fix**: `write_iqtree_states_csv` (`iqtree.rs`) writes
|
||||
`<prefix>_iqtree_states.csv` — one row per surviving state,
|
||||
`iqtree_symbol,canonical_symbol,frequency`, frequency at full `f64`
|
||||
precision (not the model file's truncated 6 decimals). Written alongside
|
||||
`_iqtree.model`/`_iqtree.fasta` from the same `CompactAlphabet` both
|
||||
already use, so there is no risk of the three files disagreeing. Covered
|
||||
by `states_csv_maps_compact_symbols_back_to_canonical_ones`.
|
||||
|
||||
The zero-exchangeability pattern itself (state 0 in the user's report,
|
||||
frequency 3.26%, `R=0` with every other state) is not yet explained —
|
||||
plausibly a genuinely unobserved transition in the calibration
|
||||
(`cardinality_transitions`/`composition_transitions` count `0` for every
|
||||
pair involving it), which is a legitimate, if numerically extreme, result
|
||||
of `-ln(0)`, not necessarily a bug — not investigated further.
|
||||
|
||||
## References
|
||||
|
||||
The Mash mutation-rate model this discussion contrasts with:
|
||||
|
||||
Reference in New Issue
Block a user