fix: resolve test race conditions, add logging, and fix CI deadlock
Release / create-release (push) Successful in 2m26s
ci.yml / build (pull_request) Successful in 4m4s
Release / build-linux-x86_64 (push) Successful in 8m19s
Release / build-macos-arm64 (push) Successful in 1m48s

Re-enables the `numa` feature in CI workflows to prevent container/cgroup deadlocks while preserving validation correctness. Fixes concurrent test race conditions by replacing thread-local parameter storage with process-wide atomics and mutex locks. Integrates `tracing-subscriber` for structured logging and adds thread-ID tracking to debug worker lifecycles. Additionally bumps the crate version, updates `.gitignore`, documents experimental evolutionary distance pipelines, and refactors hardcoded test constants.
This commit is contained in:
Eric Coissac
2026-08-11 23:04:06 +02:00
parent 4f6d442688
commit c95c47155e
11 changed files with 169 additions and 33 deletions
+73
View File
@@ -473,6 +473,79 @@ matrix derived from the state graph, `c_ts`/`c_tv`/`c_gl`/`c_ctx` as
tunable parameters) is directly usable there — no new tooling required
before testing it.
### Experiment: TNT run on real data (2026-08-11)
**Status:** validated at genus/family/order scale within Bacteria; not
informative across domains with this character type. Exploratory only —
run entirely outside the repo (`/tmp/tnt_run`, TNT installed locally under
`TNT/`), no new Rust code. Kept here as the record of what was learned.
**Pipeline.** `obikmer distance --snp` emits one IUPAC-coded pseudo-alignment
row per genome (`snp_pseudo_alignment`, one column per family with
`family_size() >= 2`). A small external Python script decodes IUPAC back to
the 16-state bitmask, applies the set-edit-distance formula above, and
emits a complete TNT script (`xread` matrix + `smatrix` step-matrix + `hold`/
`mult` search). No new Rust code was needed for this pass.
**Calibration ("quick option").** Rather than modifying
`write_raw_snp_distance_csv` to emit raw counts, `c_ctx` was approximated as
the unweighted mean of the pairwise `snp/(snp+shared)` ratios already
present in `rawsnp.csv`, restricted to the relevant taxon subset. Used
values: `mu = gamma = 10`, `c_ctx = 18` (ratio `c_ctx/mu = 1.8`, consistent
with the observed pairwise ratios for genomes within Enterobacteriaceae/
Eubacteria, which cluster around 1.5-2 and barely move as the taxon set
widens — the context signal is stable, not sensitive to which subset is
chosen). The more principled route (raw counts, weighted `p_hat`, Ts/Tv
split) remains a follow-up, not yet done.
**Run 1 — Enterobacteriaceae (11 taxa: 4 *E. coli*, 4 *Salmonella enterica*,
3 *Klebsiella pneumoniae*).** All three genera recovered as monophyletic.
Initial read of the exported (unrooted, TNT/Nexus `[&U]`) tree as showing a
genus-arrangement disagreement with known systematics (*Escherichieae*:
*Escherichia*+*Salmonella* sister vs. more distant *Klebsielleae*) was
**wrong** — diagnosed via `force = (taxa);` monophyly-constraint test
(identical score constrained vs. free ⟹ no real disagreement). Root cause of
the misreading: with exactly 3 clades and no outgroup, an unrooted tree has
only **one possible topology** (a single trifurcation) — there is no
internal arrangement to get right or wrong. This run cannot test
inter-genus relationships at all; it can only test intra-genus monophyly
(which held).
**Run 2 — Eubacteria (18 taxa: run 1 + *Acidobacterium capsulatum*,
*Opitutus terrae*, *Bacillus subtilis*, *Shouchella clausii*, *Wolbachia*
endosymbiont, *Proteus mirabilis*, *Yersinia ruckeri*).** The
Enterobacteriaceae substructure from run 1 is reproduced identically, now
correctly rooted by real outgroups, resolving the tribal arrangement left
undetermined in run 1: *Escherichia*+*Salmonella* sister, *Klebsiella* more
distant — matching known systematics. Outgroup placement: `(Bacillus,
Shouchella)` sister pair (Firmicutes/*Bacillales*) splits from all
Proteobacteria — a correct phylum-level split; `Wolbachia`
(Alphaproteobacteria) splits from the Gammaproteobacteria block
(`Proteus`, `Yersinia`, Enterobacteriaceae) — a correct class-level split.
Lower confidence: the fine nested order `(Proteus, (Yersinia,
Enterobacteriaceae))` and the relative position of `Acidobacterium` vs.
`Opitutus` — plausible, not independently verified against current
Enterobacterales family-level literature.
**Run 3 — full domain set (20 taxa: run 2 + *Candidozyma auris* [yeast] and
*Saccharolobus islandicus* [archaeon]).** The bacterial clade from run 2 is
reproduced **unchanged and intact** — a real robustness signal, the method
does not fragment the ingroup when unrelated deep taxa are added. But the
result carries **no information on Bacteria/Archaea/Eukarya relationships**:
with exactly one eukaryote, one archaeon and one bacterial clade, the
unrooted tree is again forced into the single 3-clade trifurcation from run
1's caveat — there is no second representative of either outgroup domain to
resolve internal arrangement, so nothing about their relative position can
be read from the topology (a "ladder" ordering in the exported tree is
serialization, not signal). Independently, `rawsnp.csv` shows *why* this
character type cannot reach further: pairwise ratios involving
*Candidozyma*/*Saccharolobus* are almost all `NA` (no central-position
family shared at all) or saturated at `1.0` (every shared family differs) —
central-position families require literal 31 bp context conservation, which
simply does not survive domain-level divergence. Cross-domain placement
would need conserved-marker characters (rRNA, ribosomal proteins), not this
estimator.
## Heterozygosity, ploidy, and consensus-assembly inputs
A within-genome multiplicity signal (more than one of the 4 central forms