OBIKmers/obikmer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 34 Wiki Activity

Introduce session persistence and artifact caching

Browse Source 
This change implements a robust session management system allowing users to persist distance calculation samples and parameters across invocations. It includes concurrency-safe directory locking, deterministic parameter encoding, artifact integrity checks, and caching mechanisms to skip resampling when valid data is available.
This commit is contained in:
Eric Coissac
2026-09-12 07:46:55 +02:00
parent a8bcf5ffac
commit 7861e886e7
15 changed files with 746 additions and 59 deletions
Download Patch File Download Diff File Expand all files Collapse all files
UserDocMD/usage/phylo.md
+12 -1
View File
@@ -202,7 +202,18 @@ A family is eligible for a genome pair $(i,j)$ only if both genomes carry exactl
`--subsample`, `--free-loss`, `--no-ambiguity`, `--entropy`/`--entropy-sd` are shared by `--pseudo-alignment`, `--sankoff` (and everything it implies: `--tnt`/`--phyg`/`--iqtree`), and a `snp-*` `--distance` value — one draw feeds all of them in a single invocation. `--subsample` is mandatory for `--pseudo-alignment`/`--sankoff`; for a `snp-*` `--distance` value it is optional (omitted means every non-monomorphic family in the index, not an approximation).
Combining `--sankoff` (or `--tnt`/`--phyg`/`--iqtree`) with a `snp-*` `--distance` value in the same command reuses that one draw for both — the distance and the Sankoff calibration/alignment are guaranteed to be computed from the *identical* set of sampled sites, never two independent samples, so the two outputs are directly comparable. This only holds within a single command; running them as two separate `obikmer phylo` invocations draws two independent samples even with the same flags.
Combining `--sankoff` (or `--tnt`/`--phyg`/`--iqtree`) with a `snp-*` `--distance` value in the same command reuses that one draw for both — the distance and the Sankoff calibration/alignment are guaranteed to be computed from the *identical* set of sampled sites, never two independent samples, so the two outputs are directly comparable. This only holds within a single command; running them as two separate `obikmer phylo` invocations draws two independent samples even with the same flags — unless `--session` is used (below), which currently covers a `snp-*` `--distance` value only, not `--sankoff`/`--tnt`/`--phyg`/`--iqtree`.
### `--session`: reusing a sample across separate commands
| Option | Default | Description |
|---|---|---|
| `--session DIR` | none | Persist the `snp-*` `--distance` sample in `DIR` so a later, separate `obikmer phylo` invocation with the exact same selection parameters restores it instead of resampling |
| `--session-force` | off | With `--session DIR`: overwrite its saved parameters and cached sample instead of erroring out when this run's parameters don't match. No effect without `--session` |
`DIR` is created if it doesn't exist. If it already holds a sample built with different `--subsample`/`--free-loss`/`--no-ambiguity`/`--exclude-genome`/`--min-shared-family`/`--entropy`/`--entropy-sd` values than this run, the command exits with an error rather than silently using either the old or the new values — pass `--session-force` to discard the old sample and rebuild under the new parameters, or point `--session` at a different directory to keep both.
Only the `snp-*` `--distance` sample (not `--sankoff`/`--tnt`/`--phyg`/`--iqtree`'s calibration/alignment) is persisted today. Without `--session`, nothing changes: every invocation resamples, as before.
Without `--subsample`, every variable family (family size ≥ 2) is used. With `--subsample N`, roughly `N` families are kept instead, drawn in proportion to how many candidate families each part of the index actually holds, so the sample stays representative of the whole index. If the index has fewer than `N` candidate families, `--subsample` has no effect.