fix: resolve test race conditions, add logging, and fix CI deadlock
Re-enables the `numa` feature in CI workflows to prevent container/cgroup deadlocks while preserving validation correctness. Fixes concurrent test race conditions by replacing thread-local parameter storage with process-wide atomics and mutex locks. Integrates `tracing-subscriber` for structured logging and adds thread-ID tracking to debug worker lifecycles. Additionally bumps the crate version, updates `.gitignore`, documents experimental evolutionary distance pipelines, and refactors hardcoded test constants.
This commit is contained in:
@@ -473,6 +473,79 @@ matrix derived from the state graph, `c_ts`/`c_tv`/`c_gl`/`c_ctx` as
|
||||
tunable parameters) is directly usable there — no new tooling required
|
||||
before testing it.
|
||||
|
||||
### Experiment: TNT run on real data (2026-08-11)
|
||||
|
||||
**Status:** validated at genus/family/order scale within Bacteria; not
|
||||
informative across domains with this character type. Exploratory only —
|
||||
run entirely outside the repo (`/tmp/tnt_run`, TNT installed locally under
|
||||
`TNT/`), no new Rust code. Kept here as the record of what was learned.
|
||||
|
||||
**Pipeline.** `obikmer distance --snp` emits one IUPAC-coded pseudo-alignment
|
||||
row per genome (`snp_pseudo_alignment`, one column per family with
|
||||
`family_size() >= 2`). A small external Python script decodes IUPAC back to
|
||||
the 16-state bitmask, applies the set-edit-distance formula above, and
|
||||
emits a complete TNT script (`xread` matrix + `smatrix` step-matrix + `hold`/
|
||||
`mult` search). No new Rust code was needed for this pass.
|
||||
|
||||
**Calibration ("quick option").** Rather than modifying
|
||||
`write_raw_snp_distance_csv` to emit raw counts, `c_ctx` was approximated as
|
||||
the unweighted mean of the pairwise `snp/(snp+shared)` ratios already
|
||||
present in `rawsnp.csv`, restricted to the relevant taxon subset. Used
|
||||
values: `mu = gamma = 10`, `c_ctx = 18` (ratio `c_ctx/mu = 1.8`, consistent
|
||||
with the observed pairwise ratios for genomes within Enterobacteriaceae/
|
||||
Eubacteria, which cluster around 1.5-2 and barely move as the taxon set
|
||||
widens — the context signal is stable, not sensitive to which subset is
|
||||
chosen). The more principled route (raw counts, weighted `p_hat`, Ts/Tv
|
||||
split) remains a follow-up, not yet done.
|
||||
|
||||
**Run 1 — Enterobacteriaceae (11 taxa: 4 *E. coli*, 4 *Salmonella enterica*,
|
||||
3 *Klebsiella pneumoniae*).** All three genera recovered as monophyletic.
|
||||
Initial read of the exported (unrooted, TNT/Nexus `[&U]`) tree as showing a
|
||||
genus-arrangement disagreement with known systematics (*Escherichieae*:
|
||||
*Escherichia*+*Salmonella* sister vs. more distant *Klebsielleae*) was
|
||||
**wrong** — diagnosed via `force = (taxa);` monophyly-constraint test
|
||||
(identical score constrained vs. free ⟹ no real disagreement). Root cause of
|
||||
the misreading: with exactly 3 clades and no outgroup, an unrooted tree has
|
||||
only **one possible topology** (a single trifurcation) — there is no
|
||||
internal arrangement to get right or wrong. This run cannot test
|
||||
inter-genus relationships at all; it can only test intra-genus monophyly
|
||||
(which held).
|
||||
|
||||
**Run 2 — Eubacteria (18 taxa: run 1 + *Acidobacterium capsulatum*,
|
||||
*Opitutus terrae*, *Bacillus subtilis*, *Shouchella clausii*, *Wolbachia*
|
||||
endosymbiont, *Proteus mirabilis*, *Yersinia ruckeri*).** The
|
||||
Enterobacteriaceae substructure from run 1 is reproduced identically, now
|
||||
correctly rooted by real outgroups, resolving the tribal arrangement left
|
||||
undetermined in run 1: *Escherichia*+*Salmonella* sister, *Klebsiella* more
|
||||
distant — matching known systematics. Outgroup placement: `(Bacillus,
|
||||
Shouchella)` sister pair (Firmicutes/*Bacillales*) splits from all
|
||||
Proteobacteria — a correct phylum-level split; `Wolbachia`
|
||||
(Alphaproteobacteria) splits from the Gammaproteobacteria block
|
||||
(`Proteus`, `Yersinia`, Enterobacteriaceae) — a correct class-level split.
|
||||
Lower confidence: the fine nested order `(Proteus, (Yersinia,
|
||||
Enterobacteriaceae))` and the relative position of `Acidobacterium` vs.
|
||||
`Opitutus` — plausible, not independently verified against current
|
||||
Enterobacterales family-level literature.
|
||||
|
||||
**Run 3 — full domain set (20 taxa: run 2 + *Candidozyma auris* [yeast] and
|
||||
*Saccharolobus islandicus* [archaeon]).** The bacterial clade from run 2 is
|
||||
reproduced **unchanged and intact** — a real robustness signal, the method
|
||||
does not fragment the ingroup when unrelated deep taxa are added. But the
|
||||
result carries **no information on Bacteria/Archaea/Eukarya relationships**:
|
||||
with exactly one eukaryote, one archaeon and one bacterial clade, the
|
||||
unrooted tree is again forced into the single 3-clade trifurcation from run
|
||||
1's caveat — there is no second representative of either outgroup domain to
|
||||
resolve internal arrangement, so nothing about their relative position can
|
||||
be read from the topology (a "ladder" ordering in the exported tree is
|
||||
serialization, not signal). Independently, `rawsnp.csv` shows *why* this
|
||||
character type cannot reach further: pairwise ratios involving
|
||||
*Candidozyma*/*Saccharolobus* are almost all `NA` (no central-position
|
||||
family shared at all) or saturated at `1.0` (every shared family differs) —
|
||||
central-position families require literal 31 bp context conservation, which
|
||||
simply does not survive domain-level divergence. Cross-domain placement
|
||||
would need conserved-marker characters (rRNA, ribosomal proteins), not this
|
||||
estimator.
|
||||
|
||||
## Heterozygosity, ploidy, and consensus-assembly inputs
|
||||
|
||||
A within-genome multiplicity signal (more than one of the 4 central forms
|
||||
|
||||
Reference in New Issue
Block a user