Merge pull request 'Push twnyxtspltux' (#76 ) from push-twnyxtspltux into main

Reviewed-on: #76
Implement session persistence and checkpointing
2026-09-12 05:57:25 +00:00 · 2026-09-12 07:48:51 +02:00 · 2026-09-12 07:46:55 +02:00 · 2026-09-12 07:45:32 +02:00 · 2026-09-12 07:44:51 +02:00 · 2026-09-12 07:44:39 +02:00
1102 changed files with 2502 additions and 979 deletions
@@ -30,25 +30,7 @@ data-stress
 *.bin.idx
 *.prsb
 Betula_exilis--IGA-24-33
-benchmark/genomes
-benchmark/genomes_orig
-benchmark/simulated_data
-benchmark/specimen_index_presence
-benchmark/global_index_count_dense
-benchmark/global_index_presence
-benchmark/global_index_presence_orig
-benchmark/global_index_presence_sav
-benchmark/global_index_presence_dense
-benchmark/all_specific
-benchmark/global_index_count
-benchmark/specimen_index_count
-benchmark/stats
-benchmark/reference_index
-benchmark/reference_dist
-benchmark/obikmer_dist
-benchmark/specific_index_count
-benchmark/specific_index_presence
-benchmark/query_data
+benchmark/run/
 TNT
 phyg
 biblio
@@ -1,87 +0,0 @@
-# Plan d'amélioration technique - obiskio
-
-## 1. Contexte et objectifs
- **Objectif** : Renforcer la robustesse, la maintenabilité et les performances de la crate `obiskio`.
- **Priorités** : 
-  1. Gestion des erreurs
-  2. Optimisation de la mémoire du pool
-  3. Robustesse concurrente
-  4. Couverture de tests
-  5. Documentation
-
---
-
-## 2. Axes d'amélioration détaillés
-
-### 2.1 Gestion des erreurs
- **Problème** : `SKError` ne couvre pas tous les cas (format invalide, taille maximale, CRC)
- **Actions** :
-  - Ajouter variante `ParseError(String)` dans `src/error.rs`
-  - Valider les tailles de SuperKmer avant parsing
-  - Remplacer `expect()` par `unwrap_or_else` avec messages explicites
-  - Documenter chaque variante d’erreur dans le README
-
-### 2.2 Optimisation du pool de fichiers
- **Problème** : `SKFilePool` utilise un `Vec<WriteEntry>` non contraint et n’effectue pas de nettoyage en cas d’erreur
- **Actions** :
-  - Implémenter un `LimitedVec` avec limite stricte à `MAX_POOL_SIZE`
-  - Créer `clear_memory()` qui supprime les entrées orphelines
-  - Ajouter `evict_lru_threshold()` pour éviction proactive
-  - Introduire un `RwLock` pour les opérations de lecture massives
-
-### 2.3 Robustesse concurrente
- **Problème** : Risque de deadlocks dans `SKFileWriter::write_batch()` et `SKFileReader::reopen_and_seek()`
- **Actions** :
-  - Remplacer `Mutex` par `RwLock` pour les accès en lecture
-  - Ajouter un compteur de blocage et logs de timeout
-  - Utiliser `std::thread::park_timeout` pour débloquer
-  - Insérer `debug_assert!` sur les états invariants
-
-### 2.4 Couverture de tests
- **Problème** : Absence de benchmarks, de tests de migration, de résilience de fichiers corrompus
- **Actions** :
-  - Benchmarks I/O sur 10k+ SuperKmer avec `criterion`
-  - Tests de migration de version de fichier `.meta` → `.v2.meta`
-  - Tests de corruption volontaire (truncature, inversion de bits)
-  - Tests de stress sur pool saturation (100 threads)
-
-### 2.5 Documentation & exemples
- **Actions** :
-  - Ajouter des examples dans chaque module (`# Examples`)
-  - Documenter la logique LRU avec diagrammes Mermaid
-  - Créer un guide « How to recover from eviction »
-  - Mettre à jour le `README.md` avec tableau des variantes d’erreur
-
---
-
-## 3. Plan d'exécution (Roadmap)
-
-| Sprint | Durée | Livrables clés |
-|--------|-------|----------------|
-| **S1** | 2 jours | Refactorisation `SKError`, ajout de tests unitaires |
-| **S2** | 3 jours | Implémentation `clear_memory()` + `LimitedVec` |
-| **S3** | 2 jours | Passage à `RwLock`, ajout de compteurs de blocage |
-| **S4** | 2 jours | Benchmarks + tests de migration |
-| **S5** | 1 jour | Documentation finale & mise à jour du README |
-
---
-
-## 4. Dépendances externes
- Mettre à jour `niffler` vers la version 2.0 (performance compression)
- Évaluer `bincode` vs `serde_json` pour les métas (I/O)
- Ajouter dépendance `criterion` (dev‑dependencies)
-
---
-
-## 5. KPI de suivi
- **Couverture de tests** : ≥85 % des chemins critiques
- **Latence moyenne d’écriture** : ↓15 % après optimisation du pool
- **Taux d’erreurs résolues** : 100 % des nouvelles variantes couvertes
- **Temps de build CI** : ≤5 min pour l’ensemble des benchmarks
-
---
-
-## 6. Validation finale
- Revue de code avec `cargo clippy -- -D warnings`
- Analyse de toxicité avec `cargo deny open-source-licenses`
- Vérification de la conformité aux standards de naming du projet
@@ -1,2 +0,0 @@
-/cache
-/project.local.yml
@@ -1,169 +0,0 @@
-# the name by which the project can be referenced within Serena/when chatting with the LLM.
-project_name: "obikmer"
-
-# the encoding used by text files in the project
-# For a list of possible encodings, see https://docs.python.org/3.11/library/codecs.html#standard-encodings
-encoding: "utf-8"
-
-# line ending convention to use when writing source files.
-# Possible values: unset (use global setting), "lf", "crlf", or "native" (platform default)
-# This does not affect Serena's own files (e.g. memories and configuration files), which always use native line endings.
-line_ending:
-
-# The language backend to use for this project.
-# If not set, the global setting from serena_config.yml is used.
-# Valid values: LSP, JetBrains
-# Note: the backend is fixed at startup. If a project with a different backend
-# is activated post-init, an error will be returned.
-language_backend:
-
-# whether to use project's .gitignore files to ignore files
-ignore_all_files_in_gitignore: true
-
-# advanced configuration option allowing to configure language server-specific options.
-# Maps the language key to the options.
-# The settings are considered only if the project is trusted (see global configuration to define trusted projects).
-# See https://oraios.github.io/serena/02-usage/050_configuration.html#language-server-specific-settings
-ls_specific_settings: {}
-
-# list of additional paths to ignore in this project.
-# Same syntax as gitignore, so you can use * and **.
-# Important: quote patterns that start with `*`, otherwise YAML treats them as aliases.
-# Example:
-#   ignored_paths:
-#     - "examples/**"
-#     - ".worktrees/**"
-#     - "**/bin/**"
-#     - "**/obj/**"
-# Note: global ignored_paths from serena_config.yml are also applied additively.
-ignored_paths: []
-
-# whether the project is in read-only mode
-# If set to true, all editing tools will be disabled and attempts to use them will result in an error
-# Added on 2025-04-18
-read_only: false
-
-# list of tool names to exclude.
-# This extends the existing exclusions (e.g. from the global configuration)
-# Find the list of tools here: https://oraios.github.io/serena/01-about/035_tools.html
-excluded_tools: []
-
-# list of tools to include that would otherwise be disabled (particularly optional tools that are disabled by default).
-# This extends the existing inclusions (e.g. from the global configuration).
-# Find the list of tools here: https://oraios.github.io/serena/01-about/035_tools.html
-included_optional_tools: []
-
-# fixed set of tools to use as the base tool set (if non-empty), replacing Serena's default set of tools.
-# This cannot be combined with non-empty excluded_tools or included_optional_tools.
-# Find the list of tools here: https://oraios.github.io/serena/01-about/035_tools.html
-fixed_tools: []
-
-# list of mode names that are to be activated by default, overriding the setting in the global configuration.
-# The full set of modes to be activated is base_modes (from global config) + default_modes + added_modes.
-# If the setting is undefined/empty, the default_modes from the global configuration (serena_config.yml) apply.
-# Otherwise, this overrides the setting from the global configuration (serena_config.yml).
-# Therefore, you can set this to [] if you do not want the default modes defined in the global config to apply
-# for this project.
-# This setting can, in turn, be overridden by CLI parameters (--mode).
-# See https://oraios.github.io/serena/02-usage/050_configuration.html#modes
-default_modes:
-
-# list of mode names to be activated additionally for this project, e.g. ["query-projects"]
-# The full set of modes to be activated is base_modes (from global config) + default_modes + added_modes.
-# See https://oraios.github.io/serena/02-usage/050_configuration.html#modes
-added_modes:
-
-# initial prompt for the project. It will always be given to the LLM upon activating the project
-# (contrary to the memories, which are loaded on demand).
-initial_prompt: ""
-
-# time budget (seconds) per tool call for the retrieval of additional symbol information
-# such as docstrings or parameter information.
-# This overrides the corresponding setting in the global configuration; see the documentation there.
-# If null or missing, use the setting from the global configuration.
-symbol_info_budget:
-
-# list of regex patterns which, when matched, mark a memory entry as read‑only.
-# Extends the list from the global configuration, merging the two lists.
-read_only_memory_patterns: []
-
-# list of regex patterns for memories to completely ignore.
-# Matching memories will not appear in list_memories or activate_project output
-# and cannot be accessed via read_memory or write_memory.
-# To access ignored memory files, use the read_file tool on the raw file path.
-# Extends the list from the global configuration, merging the two lists.
-# Example: ["_archive/.*", "_episodes/.*"]
-ignored_memory_patterns: []
-
-# list of additional workspace folder paths for cross-package reference support.
-# Paths can be absolute or relative to the project root.
-# Each folder is registered as an LSP workspace folder, enabling language servers to discover
-# symbols and references across package boundaries, but these folders are not indexed by Serena,
-# i.e. the respective symbols will not be found using Serena's symbol search tools.
-# Example:
-#   additional_workspace_folders:
-#     - ../sibling-package
-#     - ../shared-lib
-ls_additional_workspace_folders: []
-
-# list of language servers to start when using the LSP backend; choose from:
-#   ada                    al                     angular                ansible                bash
-#   bsl                    clojure                cpp                    cpp_ccls               crystal
-#   csharp                 csharp_omnisharp       cue                    dart                   deno
-#   elixir                 elm                    erlang                 fortran                fsharp
-#   gdscript               gleam                  go                     groovy                 haskell
-#   haxe                   hlsl                   html                   java                   json
-#   julia                  kotlin                 latex                  lean4                  lua
-#   luau                   markdown               matlab                 msl                    nextflow
-#   nix                    ocaml                  pascal                 perl                   php
-#   php_phpactor           php_phpantom           powershell             python                 python_basedpyright
-#   python_jedi            python_pyrefly         python_ty              qml                    r
-#   rego                   ruby                   ruby_solargraph        rust                   scala
-#   scss                   solidity               svelte                 swift                  systemverilog
-#   terraform              toml                   typescript             typescript_vts         vue
-#   wolfram                yaml                   zig
-#   (This list may be outdated; generated with scripts/print_language_list.py;
-#   For the current list, see values of the LanguageServerId enum here:
-#   https://github.com/oraios/serena/blob/main/src/solidlsp/ls_config.py)
-# For some languages, there are several alternative language servers, e.g. csharp_omnisharp, ruby_solargraph.)
-# Note:
-#   - For C, use cpp
-#   - For JavaScript, use typescript
-#   - For Angular projects, use angular (subsumes typescript+html; requires `npm install` in the project root)
-#   - For Svelte projects, use svelte (subsumes typescript/javascript for .svelte projects; requires npm)
-#   - For Deno projects, use deno (serves the same .ts/.js files as typescript; requires the deno CLI on PATH)
-#   - For SCSS / Sass / plain CSS, use scss (some-sass-language-server handles all three)
-#   - For Free Pascal/Lazarus, use pascal
-# Special requirements:
-#   Some language servers require additional setup/installations.
-#   See here for details: https://oraios.github.io/serena/01-about/020_programming-languages.html#language-servers
-# When using multiple language servers, the first language server that supports a given file will be used for that file.
-# The first language server is the default language and the respective language server will be used as a fallback.
-# Note that when using the JetBrains backend, language servers are not used and this list is correspondingly ignored.
-language_servers:
- rust
-
-# list of workspace folder paths (LSP backend only).
-# These folders will be used to build up Serena's symbol index.
-# Paths must be within the project root and should thus be relative to the project root.
-# Furthermore, the paths should not be filtered by ignore settings.
-# Default setting: The entire project root folder (".") is considered.
-# In (large) monorepos, this can be used to index only subfolders of the project root, e.g.
-#   ls_workspace_folders:
-#     - "./subproject1"
-#     - "./subproject2"
-ls_workspace_folders:
- .
-
-# optional shell command to run before the language backend (LSP or JetBrains) is initialised.
-# the command runs in the project root directory and is only executed if the project is trusted
-# (see trusted_project_path_patterns in the global configuration).
-# serena waits for the command to exit: a non-zero exit code is logged as an error but does not
-# abort activation. a per-project timeout (activation_command_timeout, default 180s) is the safety
-# backstop for non-terminating commands; on expiry the process is killed and activation continues.
-# example: activation_command: "npx nx run-many -t build"
-activation_command:
-
-# maximum time in seconds to wait for activation_command to complete before killing it (default 180s).
-# must be a positive number.
-activation_command_timeout: 180.0
@@ -1,59 +0,0 @@
-// Project tasks configuration. See https://zed.dev/docs/tasks for documentation.
-//
-// Example:
-[
-  {
-    "label": "Example task",
-    "command": "for i in {1..5}; do echo \"Hello $i/5\"; sleep 1; done",
-    //"args": [],
-    // Env overrides for the command, will be appended to the terminal's environment from the settings.
-    "env": { "foo": "bar" },
-    // Current working directory to spawn the command into, defaults to current project root.
-    //"cwd": "/path/to/working/directory",
-    // Whether to use a new terminal tab or reuse the existing one to spawn the process, defaults to `false`.
-    "use_new_terminal": false,
-    // Whether to allow multiple instances of the same task to be run, or rather wait for the existing ones to finish, defaults to `false`.
-    "allow_concurrent_runs": false,
-    // What to do with the terminal pane and tab, after the command was started:
-    // * `always` — always show the task's pane, and focus the corresponding tab in it (default)
-    // * `no_focus` — always show the task's pane, add the task's tab in it, but don't focus it
-    // * `never` — do not alter focus, but still add/reuse the task's tab in its pane
-    "reveal": "always",
-    // Where to place the task's terminal item after starting the task:
-    // * `dock` — in the terminal dock, "regular" terminal items' place (default)
-    // * `center` — in the central pane group, "main" editor area
-    "reveal_target": "dock",
-    // What to do with the terminal pane and tab, after the command had finished:
-    // * `never` — Do nothing when the command finishes (default)
-    // * `always` — always hide the terminal tab, hide the pane also if it was the last tab in it
-    // * `on_success` — hide the terminal tab on task success only, otherwise behaves similar to `always`
-    "hide": "never",
-    // Which shell to use when running a task inside the terminal.
-    // May take 3 values:
-    // 1. (default) Use the system's default terminal configuration in /etc/passwd
-    //      "shell": "system"
-    // 2. A program:
-    //      "shell": {
-    //        "program": "sh"
-    //      }
-    // 3. A program with arguments:
-    //     "shell": {
-    //         "with_arguments": {
-    //           "program": "/bin/bash",
-    //           "args": ["--login"]
-    //         }
-    //     }
-    "shell": "system",
-    // Whether to show the task line in the output of the spawned task, defaults to `true`.
-    "show_summary": true,
-    // Whether to show the command line in the output of the spawned task, defaults to `true`.
-    "show_command": true,
-    // Which edited buffers to save before running the task:
-    // * `all` — save all edited buffers
-    // * `current` — save currently active buffer only
-    // * `none` — don't save any buffers
-    "save": "none",
-    // Represents the tags for inline runnable indicators, or spawning multiple tasks at once.
-    // "tags": []
-  },
-]
@@ -2249,9 +2249,85 @@ F84/TN93 and is included below. `snp-` prefix on every CLI value.
 **`+Γ` rate-heterogeneity modifier, applicable to `snp-jc`, `snp-k2p`,
 `snp-k81`, `snp-t92`, `snp-f84`, `snp-hky85`, `snp-tn93`** (not `snp-raw`,
 nothing to correct; not `snp-logdet`, no standard gamma formulation) — same
-formula as the base correction, weighted by a shape parameter `α` supplied
-by the user (`--gamma-shape <alpha>`), not estimated by ML. A modifier on
-existing values, not a separate enum arm per distance.
+formula as the base correction, weighted by a shape parameter `α` either
+supplied by the user (`--gamma-shape <alpha>`) or estimated from the data
+(`--gamma-shape auto`/`estimate`, method-of-moments — not ML; see
+"Automatic α estimation" below). A modifier on existing values, not a
+separate enum arm per distance.
+
+### Automatic α estimation (`--gamma-shape auto`)
+
+**Correction (verified against the primary source, 2026-09-11):** Jin &
+Nei (1990) itself (*"Limitations of the Evolutionary Parsimony Method of
+Phylogenetic Analysis"*, Mol. Biol. Evol. 7(2):82–102 — the paper this
+whole `+Γ` correction is cited from, confirmed algebraically to match this
+codebase's `corrected_log`/`k2p` exactly against their eq. A4, general, and
+A8, the `a = 1` case) contains **no** data-driven α-estimation procedure.
+Their own recommendation (p. 98) is a fixed default: *"we suggest that the
+gamma distance with a = 1 [eq. A8] be used. However, one may choose a
+different gamma distance, estimating a from data. Wilson et al. (1989)
+recently used a distance with a = 1/2 for restriction-site data of
+mitochondrial DNA in hominoids."* — i.e. Jin & Nei explicitly punt
+data-driven estimation to a *different* paper (Wilson et al. 1989), not
+read/verified here. The estimator below is therefore **not** "Jin & Nei's
+method" under any framing — that attribution (present in an earlier
+revision of this section) was wrong, not just under-cited.
+
+**Implemented** (`PartitionDispersion`, `obikphylo/src/siblings/algorithms/pairwise.rs`)
+as an independent method-of-moments estimator, unrelated to any specific
+published procedure: pools substitution counts by **partition** rather
+than by genome pair, during the same `reduce_pairwise` pass that builds
+`PairwiseTally` (no second scan).
+
+For partition `i`: `n_i` = substitutions pooled over every genome pair,
+`L_i` = eligible loci pooled over every genome pair, `R_i = n_i / L_i`.
+Modeling among-site rate heterogeneity the same way as the `+Γ` correction
+itself (a `Gamma(α, α)`-distributed, mean-1, multiplicative rate shared by
+every locus in a partition — the classical Poisson-Gamma/negative-binomial
+mixture, the general identity behind gamma-rate-heterogeneity corrections,
+also behind Uzzell & Corbin's (1971) original observation that substitution
+counts across sites/regions are over-dispersed relative to Poisson):
+
+\[
+\mathbb{E}[R_i] = \mu \qquad \mathrm{Var}[R_i] = \frac{\mu}{L_i} + \frac{\mu^2}{\alpha}
+\]
+
+Weighting each partition's squared deviation by its own `L_i` cancels the
+Poisson term before attributing what's left to `α`:
+
+\[
+\hat\mu = \frac{\sum_i n_i}{\sum_i L_i} \qquad
+V = \frac{\sum_i L_i (R_i-\hat\mu)^2}{\sum_i L_i} \qquad
+\hat\alpha = \frac{\hat\mu^2}{V - \hat\mu/\bar L}
+\]
+
+where `\bar L` is the mean partition size. Returns "no estimate" (falls
+back to the uncorrected formula, warns) when fewer than 2 partitions have
+data, `\hat\mu \le 0`, or `V` doesn't exceed the Poisson floor
+`\hat\mu/\bar L` — no detectable over-dispersion, `α` would be unbounded.
+
+**Caveat, stated explicitly rather than left implicit**: unlike every
+closed-form correction in `snp_distance.rs` (each verified line-by-line
+against `ape`'s `dist_dna.c`, and now also against Jin & Nei 1990 directly
+for the base `+Γ` formula), this estimator is derived from first
+principles (the general Poisson-Gamma/negative-binomial identity) with no
+primary-source procedure behind it at all — not Jin & Nei's (confirmed
+above), and Wilson et al. (1989), the paper they point to instead, hasn't
+been read/verified either. Mathematically self-consistent (re-derived and
+checked, not guessed), but a from-scratch method, not a literature
+implementation. If `--gamma-shape` needs a value with a literature
+pedigree rather than an estimated one, Jin & Nei's own stated default,
+`α = 1` (`--gamma-shape 1`), is the better-supported choice today.
+
+Deliberately **not** gated by `--sankoff-ratio-ceiling` the way
+`base_pair_tally` is (same precedent as `cardinality_tally` — see its own
+doc comment): that filter excludes individual saturated *pairs* from a
+composition estimate computed once at the very end, from the complete
+`PairwiseTally`; the partition axis needed here only exists transiently,
+one partition at a time, while `PairwiseTally` is still being built — long
+before any pair's final SNP ratio (and thus its ratio_ceiling eligibility)
+is known. `--exclude-genome` isn't applied either, matching
+`reduce_pairwise`'s own raw per-pair fold.

 **Implemented now: `snp-raw`, `snp-jc`, `snp-k2p`, `snp-k81`, `snp-f81`,
 `snp-t92`, `snp-tn93`, `snp-tv`, all with `+Γ` except `raw`/`tv`** — see
@@ -2391,6 +2467,397 @@ mechanical substitution but weren't independently checked against an
 `ape`-equivalent reference for those three specifically — flagged here, not
 silently assumed correct.

+### α-estimation literature survey and design discussion (2026-09-11)
+
+Follow-up discussion after implementing `PartitionDispersion::estimate_alpha`
+(previous section), triggered by the user pointing out that the literature
+conflates two distinct problems: *estimating α from an alignment* versus
+*using a given α in a Jin-Nei-style corrected distance*. Jin & Nei (1990)
+itself is squarely in the second camp — α is a user-supplied parameter
+there, not something their paper estimates (confirmed by reading the paper
+directly, see previous section). This section catalogs the actual
+α-estimation literature and records which approaches were considered and
+why each was or wasn't adopted.
+
+**Bibliography** (methodological papers, not exhaustive users of `+Γ`):
+
+| Method | Reference | Needs a tree? | Estimates "true" α or a task-specific one? | Cost |
+|---|---|---|---|---|
+| ML, continuous Γ (foundational) | **Gu, Fu & Li (1995)**, MBE 12:546–557 — read in full | yes, topology fixed a priori | true α (+ θ jointly, invariant+Γ) | very high, ≤5–6 taxa in practice |
+| ML, discrete Γ | Yang (1993/94), MBE 10:1396; JME 39:306 | yes | true α | high |
+| Parsimony-based ML on inferred changes | Yang & Kumar (1996), MBE 13:650 | yes, equal branch lengths assumed | true α, biased when branches unequal | low |
+| Corrected substitutions/site → NB fit | **Gu & Zhang (1997)**, MBE 14:1106 — read in full | yes, + ancestral reconstruction | true α, close to ML accuracy | low (given a tree) |
+| Topology-optimizing grid search | **Guindon & Gascuel (2002)**, MBE 19:534 — read in full | yes, but built *from* the candidate distances themselves (BIONJ) | **not** true α — deliberately biased *upward* for topological accuracy | low–moderate (grid × tree build) |
+| Free (non-parametric) rate distribution | Susko et al. (2003), Syst. Biol. 52:594 | yes | tests the Γ assumption itself | high |
+| Parsimony counts/site → NB fit | **Wakeley (1993)**, JME 37:613 — read in full | yes, NJ tree + Fitch parsimony ancestral reconstruction | true α, biased *upward* by parsimony (conservative) | low (given a tree) |
+| Poisson-Gamma/negative-binomial, foundational | Uzzell & Corbin (1971), Science 172:1089 — **not read**, only seen via citations | unknown — likely also count-per-site/tree-based, unverified | true α (indirect, via NB overdispersion) | unknown |
+| Bayesian posterior over α | BPP/MrBayes/BEAST2/RevBayes | yes | posterior, not point estimate | very high |
+
+**Correction (2026-09-11, after reading Wakeley in full):** an earlier
+revision of this table filed Wakeley (1993) under "no tree needed" —
+wrong. Wakeley's method: build a tree (neighbor-joining, Saitou & Nei
+1987, on the real mtDNA data; coalescent-simulated for the simulation
+study), reconstruct ancestral states at internal nodes by Fitch (1971)
+parsimony, count the *minimum* number of changes per site this implies,
+then fit either a two-rate Poisson mixture or a gamma-distributed-rates
+model (⟹ negative binomial, parameters by the methods of Cohen 1965 and
+Johnson & Kotz 1969) to the resulting per-site count distribution — the
+same general "tree + ancestral reconstruction + per-site counts + NB fit"
+shape as Gu & Zhang (1997), just using raw parsimony counts directly
+rather than Gu & Zhang's multiple-hit-corrected `k̂`. Confirmed by
+simulation (Wakeley's own Fig. 1) that parsimony underestimates both the
+mean *and* (more severely) the variance of per-site change counts, which
+biases `α` **upward** (toward apparent uniformity) — consistent with Gu &
+Zhang's own citation of Wakeley for exactly this bias, and with the
+general "parsimony overestimates α" pattern noted throughout this
+literature. Real hypervariable-region-1 mtDNA data gave `α̂ ≈ 0.44–0.60`
+per data set, `0.47` combined — despite the conservative bias, still
+solidly in the "strong heterogeneity" range, which is the paper's own
+point: the method is biased but usefully conservative, not useless.
+
+**Where `PartitionDispersion::estimate_alpha` actually sits**: nowhere in
+this table's tree-dependent rows. It shares only the *abstract
+mathematical identity* (Poisson-Gamma mixture ⟹ negative binomial
+over-dispersion) with Wakeley/Uzzell-Corbin/Gu&Zhang, applied via a
+genuinely different, tree-free and ancestral-reconstruction-free route:
+pairwise genome-to-genome substitution counts pooled by *partition*
+instead of per-site counts pooled across an alignment's columns after
+ancestral state reconstruction. As far as this survey has established
+(Uzzell & Corbin not read, so not fully ruled out), **no tree and no
+ancestral reconstruction of any kind** appears to be a genuinely
+distinguishing property of `PartitionDispersion`, not something it
+inherits from prior art — consistent with it being an independent
+derivation rather than a literature implementation (see previous
+section's caveat, now on firmer footing).
+
+**Why a tree-free estimate is even valid — the identifiability question.**
+Raised directly by the user after reading Gu, Fu & Li (1995), which states
+explicitly: *"one cannot estimate the rate heterogeneity when the number
+of sequences is <3."* This is a real non-identifiability result, not a
+practical inconvenience: with only 2 sequences, a site's history is
+summarized by one binary observation (differ / don't differ), collapsed
+across all sites into two aggregate numbers (transition and transversion
+proportions) — divergence time and among-site rate variance are
+confounded in that pair of numbers, with no way to separate them. Every
+method surveyed above (Gu,Fu&Li, Yang, Wakeley, Gu&Zhang) needs ≥3
+sequences *sharing one evolutionary history* (a tree) specifically to get
+multiple independent looks at the *same* site's rate across different
+lineages, which is what identifies its variance.
+
+`PartitionDispersion` doesn't violate this, because it isn't estimating
+the same quantity. The literature above estimates rate variation **across
+sites, within one shared tree**. `PartitionDispersion` estimates rate
+variation **across partitions, pooled over every genome pair in the
+index** — each genome pair stands in for one independent draw of a
+partition's relative rate, the same identifying role multiple lineages
+play in the tree-based methods, just substituting "genome pair" for
+"lineage" and "partition" for "site." With `P` genome pairs (potentially
+in the hundreds here) all contributing to every partition's pooled count,
+the ≥3-samples identifiability requirement is met by the pair count, not
+by tree depth — closer in spirit to Jin & Nei's own original motivating
+scenario (comparing several genomic *regions* across one fixed panel of
+taxa) than to single-tree site-rate estimation.
+
+This substitution carries its own assumption, which must be named rather
+than left implicit: that a partition's *relative* rate (fast/slow) is
+reasonably stable **across genome pairs**, not just across sites within
+one pair — the direct analogue of the "no lineage-specific rate variation,
+only site-specific" assumption every tree-based `+Γ` method already makes,
+just moved from lineage→pair and site→partition. If this breaks — e.g. an
+index mixing very closely related and very divergent genome pairs, where a
+partition's saturation behavior differs qualitatively between the two —
+the estimate could be misled in a way a tree-based method would at least
+have the topology to detect and a tree-free, pool-everything method
+cannot. Not yet tested against real data with strongly heterogeneous
+pairwise divergence; worth keeping in mind as the main open validity
+question for `--gamma-shape auto`, not the citation question (now settled)
+this whole discussion started from.
+
+**Gu, Fu & Li (1995), read in full — the foundational ML method, not a
+candidate to implement.** Models rate variation as invariant+Γ (`θ` =
+proportion of invariant sites, `α` = gamma shape among the variable sites),
+and derives the exact joint likelihood over nucleotide *configurations*
+across all `n` sequences on a **fixed, a-priori-known tree topology** —
+for 3 sequences this is closed-form (Jukes-Cantor), for `n` sequences it's
+a sum over `4^n` configurations weighted by coefficients tied to the
+specific topology's branching structure (their eq. 32–34), with branch
+lengths, `α`, and `θ` jointly optimized by Newton-Raphson (Hessian over all
+free parameters). The site-rate integral has the same closed form
+`E = θ + (1-θ)(1+D/α)^{-α}` that underlies `+Γ` distance corrections
+generally (the same mechanism as `corrected_log`, not a coincidence — Jin &
+Nei's own formula is the `θ=0` special case applied to a pairwise `D`
+rather than a whole-tree one). Explicitly stated by the authors as
+infeasible beyond about 5–6 taxa (their own simulations stop at 5); not a
+candidate for obikmer's typical genome counts. Notable contribution worth
+keeping in mind regardless: `ρ = (1+θα)/(1+α)` is shown to be a more robust
+summary of rate heterogeneity than `α` alone whenever `θ` isn't negligible
+— `α` and `θ` are confounded (nearly-invariant sites can come from either a
+small `α`'s left tail or from a nonzero `θ`), so `α` alone can be very
+unstable while `ρ` stays well-behaved. `PartitionDispersion` has no `θ`
+term at all (no invariant-site component), so this confound doesn't arise
+for it the same way, but it's worth remembering if an invariant+Γ variant
+is ever considered.
+
+**Gu & Zhang (1997), read in full — rejected for now.** Their procedure:
+(1) a tree with least-squares branch lengths must already exist; (2)
+ancestral states are reconstructed at every internal node (they use Zhang &
+Nei 1997's likelihood method, but note plain parsimony would work, just
+less accurately); (3) per site, branches are split into "changed"/"unchanged"
+given the ancestral reconstruction, and the *expected* substitution count
+`k̂` (corrected for multiple hits) is obtained by solving their eq. (7)
+(or eq. 12 for the generalized model) numerically — a root-find per site,
+not a closed form; (4) the site-level `k̂` values (real-valued, not
+integers) are fit to a negative binomial (Uzzell & Corbin 1971) by ML to
+get α — again no closed form, numerical optimization. This is a real
+per-site pipeline requiring a tree *before* distances/α can be computed —
+the reverse of obikmer's current dependency order (`snp_distance` computes
+distances, which `--nj`/`--upgma` only turn into a tree *afterward*). Their
+own answer to this chicken-and-egg problem is an iterative
+distance→tree→α→distance loop, which they themselves flag as expensive.
+Adopting this would mean a new ancestral-reconstruction subsystem (even a
+parsimony/Fitch-only version) plus a per-family nonlinear solve plus an NB
+ML fit — a much bigger addition than `PartitionDispersion`, not something
+to build without a concrete need beyond "closer to Gu&Zhang's own
+simulated accuracy than our moment estimator."
+
+**Guindon & Gascuel (2002), read in full — plausible future addition, not implemented.**
+Their **Q criterion**: for a candidate α, build a tree (they use BIONJ) from
+the α-corrected distance matrix; for every internal branch, group taxa into
+the 4 subtrees it separates (A, B, C, D), compute mean inter-subtree
+distances \(\bar\delta_{AB}\), \(\bar\delta_{CD}\), etc., and let `S ≤ M ≤ L`
+be the three pairings' sums (`{AB,CD}`, `{AC,BD}`, `{AD,BC}`); the branch's
+reliability score is `Q_branch = L − M` (zero when the four points are
+perfectly tree-additive). The whole-tree `Q` is the mean over internal
+branches (negative/zero branches excluded). `α* = argmin_α Q(α)` over a
+grid (their run: ~60–100 points, 0.1 to 5000, finer spacing where
+sensitivity is highest). Complexity: `O(n²l)` once for the base
+frequencies, then `O(n²r)` for the `r` candidate distance matrices, then
+`O(n³r)` worst case for building `r` trees and evaluating `Q` on each
+(same order as the tree-building step itself, so "free" relative to it).
+
+Central finding, **the opposite direction from an initial misreading in
+this discussion**: `α_opt` (Guindon-Gascuel's topology-optimal value) is
+**always ≥ the true α**, not the reverse — underestimating rate
+heterogeneity (i.e., picking a larger α than reality) reduces the variance
+of distance estimates and thereby improves NJ/BIONJ topological accuracy,
+especially when the molecular clock roughly holds (where `α_opt → ∞`, i.e.
+no correction at all is topologically best). This is an *empirical
+regularity* observed across their tested conditions (η ∈ {0.5, 2.0}, true
+α ∈ {0.1, 0.7}, 20-taxon trees) — not a theorem — so treating it as a hard
+guarantee would be another unverified-claim mistake of the kind this whole
+discussion has been correcting.
+
+**Proposed integration (not yet implemented)**: since `α_true ≤ α_opt`
+empirically, `PartitionDispersion::estimate_alpha()`'s output could seed
+the *lower bound* of Guindon-Gascuel's grid (search `[α̂, ∞)` instead of
+`[0.1, 5000]`), narrowing the grid substantially and reusing the existing
+estimator rather than needing an independent "true α" source — this also
+sidesteps Gu & Zhang's tree-first dependency problem entirely, since
+Guindon-Gascuel's own grid already builds a tree per candidate. Still
+requires: (a) a **BIONJ** implementation (see below), (b) the Q-criterion
+subtree-grouping logic, (c) a grid-search driver. Not started.
+
+**BIONJ availability in the Rust ecosystem — checked 2026-09-11, none found.**
+Searched crates.io directly (API query for "bionj": 0 results) and read the
+docs.rs pages of the most plausible candidates: `phylotree` (builds/reads/
+manipulates trees but has no distance-matrix reconstruction — no NJ, no
+BIONJ, no UPGMA despite web-search summaries claiming otherwise, corrected
+after checking the actual docs), `speedytree` (NJ only — Canonical/RapidNJ/
+Hybrid variants, explicitly no BIONJ), `nj` (plain NJ only), `phylo` (no
+distance-based reconstruction at all). `obikphylo::neighbor_joining`
+(`tree.rs`) is plain NJ, not BIONJ — BIONJ (Gascuel 1997) differs only in
+how branch lengths are weighted during agglomeration (accounts for
+distance-estimate variance, not just the sum), a modest delta over an
+existing NJ implementation rather than a rewrite, but would need to be
+written in-house if the Guindon-Gascuel integration above is ever pursued
+(their own paper notes other tree-building methods gave similar results in
+their simulations, so plain NJ may be an acceptable substitute if BIONJ
+proper is judged not worth the effort).
+
+### Sharing one sample across algorithms (`--sankoff`/`--tnt`/`--phyg`/`--iqtree` + `snp-*` `--distance`) — implemented 2026-09-11
+
+**Correctness bug, not just a performance one.** `--sankoff`/`--tnt`/`--phyg`/
+`--iqtree` (via `SiblingExt::sankoff_bundle`) and a `snp-*` `--distance`
+(via `SiblingExt::snp_distance`) always consume the *exact same* selection
+parameters when requested together in one `obikmer phylo` invocation
+(`n`/`free_loss`/`no_ambiguity`/excluded-set/`entropy_bias` — the CLI has
+no way to give them different values in one run, see `cmd/phylo/args.rs`).
+Before this fix, each independently called `sample_index`, and
+`sample_layer`'s `rand::rng()` (`subsample.rs:235`) is a **thread-local
+generator that advances across calls, not reseeded each time** — so the
+second call silently drew a *different* random sample of sites than the
+first, even with identical parameters. Verified empirically (throwaway
+8-genome index): running `snp-jc` alone vs. combined with `--sankoff`
+produced visibly different distance matrices from the same index/params.
+This defeated the actual point of combining these flags — running several
+algorithms (Sankoff/TNT/PhyG/IQ-TREE calibration, a `snp-*` distance) on
+one *identical* site selection for direct comparison.
+
+**Fix**: `SankoffBundle` (`sankoff.rs`) now retains its `PairwiseTally`/
+`PartitionDispersion` internally (previously consumed into `raw`/
+`base_pair_tally`/`cardinality_tally` and dropped) and exposes
+`SankoffBundle::snp_distance(kind, gamma_shape)`, which computes the
+matrix from that *same* already-sampled tally — no second `sample_index`
+call. The shared post-sampling logic (the `--gamma-shape` support check,
+`alpha` resolution, final matrix build — previously all inline in the
+`snp_distance()` free function) was factored into
+`pub(crate) fn distance_matrix(tally, dispersion, kind, gamma_shape)` in
+`snp_distance.rs`, called by both the standalone `snp_distance()` (after
+its own fresh sampling) and `SankoffBundle::snp_distance` (reusing the
+bundle's). `cmd/phylo/mod.rs` keeps the `Option<SankoffBundle>` alive past
+the Sankoff-family `if` block and, when a `snp-*` `--distance` is also
+requested, calls `bundle.snp_distance(...)` instead of
+`cache.snp_distance(...)` whenever a bundle was built — since both branches
+are driven by the same `args.*` fields, the parameters trivially always
+match when both fire; no runtime "do the params match" check needed.
+Verified end-to-end: the reused path logs `(reusing the Sankoff bundle's
+sample)` and the `snp_distance` stage timer reads `0ms` (`formula(kind)`
+is `O(n²)` post-processing, no I/O), vs. ~150-230ms for a fresh sample on
+the same tiny test index.
+
+#### Explicit `--session` (implemented 2026-09-12)
+
+The in-process fix above only covered reuse *within one command*.
+`--session DIR` (`cmd/phylo/args.rs`) names a directory (outside the
+index) that persists the `snp-*` `--distance` sample — the
+`PairwiseTally`/`PartitionDispersion` pair `sample_index` would otherwise
+rebuild from scratch — across *separate* `obikmer phylo` invocations.
+
+**Crate placement, as planned**: a new **`obiksession`** crate
+(`src/obiksession/`), domain-agnostic — no `PairwiseTally`/"site" concept
+anywhere in it. `Session::open(dir, params: &[u8], force: bool)` handles
+directory lifecycle and the params-conflict check against an opaque byte
+blob; `Session::store(name, bytes)`/`Session::restore(name) -> Option<Mmap>`
+handle a checksummed artifact cache. `obikphylo`
+(`siblings/algorithms/session_cache.rs`, shared by `snp_distance.rs` and
+`sankoff.rs` — see below) is the consumer: it serializes its own
+`PairwiseTally`/`PartitionDispersion`/`SnpAlignment` to bytes (via `rkyv`)
+and hands them to `obiksession`, which never sees their type.
+
+**Serialization**: `rkyv` 0.8.18, added as planned. One deviation from the
+original zero-copy pitch, scoped down deliberately: artifacts are restored
+via `rkyv::from_bytes` (full owned deserialize) rather than `rkyv::access`
+(zero-copy over the `mmap`). True zero-copy would require every
+`PairwiseTally` query method (`categories`, `base_freq`, `pair`, ...) to
+work generically over `Archived<PairwiseTally>` as well as the owned type
+— a separate, larger change not needed to get the actual win (skipping
+`sample_index`'s expensive re-scan of the sibling annex; deserializing an
+already-in-memory-sized count array is comparatively cheap CPU, not I/O).
+`#[derive(Archive, Serialize, Deserialize)]` was added to `PairStats`,
+`PairwiseTally`, and `PartitionDispersion` (`pairwise.rs`) for this.
+
+**Startup behavior, as designed**: `Session::open` creates `DIR` if
+missing (fresh, params recorded); if `DIR` already holds different saved
+params, `cmd/phylo` reports a hard error and exits rather than silently
+preferring one side, unless `--session-force` is given (which discards the
+directory's cached artifacts and starts over under the new params) —
+implements exactly the "explicit escape hatch, not silent override"
+decision from the design discussion. `obiksession` itself doesn't decode
+either side's bytes to produce a field-by-field diff (true to "opaque
+blob," left to whoever needs it); `cmd/phylo`'s conflict message is
+currently a single generic sentence listing which flags could be the
+cause, not a computed diff — an honest v1 simplification, not a limitation
+of `obiksession` itself.
+
+**Per-artifact caching, as designed**: only `PairwiseTally`/
+`PartitionDispersion` are cached (`snp_distance.rs`'s `restore_tally`/
+`store_tally`) — `ratio_ceiling`/`gamma_shape` remain pure post-processing
+over whatever tally is in hand, cached or fresh, exactly as planned.
+
+**Locking, reconsidered and kept as the whole-session `DirLock`.** First
+written up as a corner cut ("per-entry locking was sketched, not
+implemented"). On actually working through the concrete race, whole-session
+locking turned out to be the *correct* choice, not a simplification:
+`--session-force` resets a session by deleting its whole `artifacts/`
+directory. If the directory lock were only held briefly (during
+`Session::open`, then released so a long-running `Session` doesn't block
+other processes from touching *different* artifacts), a second process
+calling `open(..., force: true)` could acquire the lock, see mismatched
+params, and wipe `artifacts/` **while a first process's already-open
+`Session` is mid-checkpoint** — genuine corruption risk, not just wasted
+work. Holding the lock for the `Session`'s entire lifetime makes that
+sequence impossible: a `force` reset must wait for every currently-open
+`Session` on that directory to be dropped first. The concurrency this
+gives up (two *different* processes reading/writing *different* artifacts
+of the same session at once) isn't exercised by anything today — only one
+code path (`snp_distance`/`sankoff_bundle`, sequentially per invocation)
+uses sessions at all.
+
+**Per-layer chunked/resumable dumps — implemented.** `subsample::sample_index`
+gained a `skip_layers: usize` parameter (skip the first N (partition,
+layer) pairs, in its own deterministic iteration order, entirely — no
+I/O, no `on_layer` call) and its `on_layer` callback gained a `raw_index`
+argument (that pair's 0-based position in the same order), so a caller can
+tell exactly how far a run has gotten. `snp_distance`/`sankoff_bundle`
+checkpoint their tally/dispersion (and, for `sankoff_bundle`, the
+in-progress `SnpAlignment`) to the session every
+`session_cache::CHECKPOINT_INTERVAL_LAYERS` (8) layers or
+`CHECKPOINT_INTERVAL` (30s) of wall time, whichever comes first — bounding
+how much completed scanning a crash can lose, without paying a full
+tally-rewrite on every one of what can be thousands of layers. A `progress`
+artifact records how many layers are reflected in the checkpoint, with a
+`PROGRESS_COMPLETE` (`u64::MAX`) sentinel once every layer has been
+processed (distinct from any real layer count, so "finished" is never
+confused with "stopped after N layers where N happens to equal the total").
+
+Verified with a real kill: built a 10-genome/64-partition index, ran
+`--distance snp-jc --session DIR` under `timeout 0.4`, confirmed a
+checkpoint at `progress=8` (exactly `CHECKPOINT_INTERVAL_LAYERS`) was on
+disk, then re-ran the same command and confirmed it resumed from layer 8
+and completed successfully — 0-effort verification that the mechanism
+itself is real, not just plausible-sounding.
+
+**Important correction from that same test — resume is *not* bit-reproducible,
+and shouldn't be.** The output matrix from the resumed run differed from
+an uninterrupted control run against the same index/params. Root cause:
+`sample_layer`'s random draws come from `rand::rng()`, a per-process,
+OS-seeded generator with no continuity across a process boundary — the
+layers processed before a kill and the layers processed after a resume
+come from two independent random streams. An earlier revision of this doc
+(and of the code comments) claimed resume "reproduces the exact same
+sample an uninterrupted run would have" — that's wrong, and was corrected
+after the user pointed out the actual reasoning error: true randomness has
+no notion of continuity in the first place, so there was never a
+correctness requirement for a resumed run to match an uninterrupted one
+bit-for-bit. The sample produced after a resume is exactly as legitimate a
+random draw as one from an uninterrupted run — just a *different* one,
+same as running the command twice without `--session` already gives two
+different samples today. That variance is explicitly wanted (see below),
+not a defect to fix. Comments in `subsample.rs`/`session_cache.rs` were
+corrected to state this plainly instead of the false reproducibility claim.
+
+**Explicitly declined: deterministic/seeded sampling.** Raised as a
+possible fix for the above (seed `sample_layer`'s RNG from a per-session
+value plus `(partition, layer)`, making a resume reproduce bit-identical
+results, and, if applied everywhere, making *any* two runs with identical
+parameters produce identical samples). The user explicitly declined this
+for now: repeated runs are relied on to measure sampling variance, and a
+`--seed` flag would need its own explicit design if wanted later — this is
+not an accidental gap, it's a stated preference to revisit only if asked.
+
+**`SankoffBundle` integration — implemented.** `sankoff_bundle` accepts
+the same `session: Option<&Session>` and uses the *same* artifact names
+(`session_cache::{TALLY_ARTIFACT, DISPERSION_ARTIFACT, PROGRESS_ARTIFACT}`)
+as `snp_distance`, plus its own `ALIGNMENT_ARTIFACT` (`sankoff.rs`) folded
+in lockstep with the tally at every checkpoint (same `on_layer` call
+produces both, so they can never drift out of sync). Consequence verified
+directly: a tally checkpointed by `--sankoff --session DIR` is restored by
+a *later, separate* `--distance snp-k2p --session DIR` invocation with no
+`--sankoff` at all, and vice versa — cross-consumer reuse, not just
+same-command reuse, exactly the synergy this design aimed for.
+
+**Per-artifact caching, as designed**: only the tally/dispersion (plus,
+for `sankoff_bundle`, the alignment) are cached — `ratio_ceiling`/
+`gamma_shape` remain pure post-processing over whatever tally is in hand,
+cached or fresh, exactly as planned.
+
+**Still not done — genuine follow-up, not a justified tradeoff:**
+**exhaustive-sample caching for `--subsample`-varying exploration.** The
+design discussion's suggestion — cache the exhaustive (`n = None`) sample
+once and subsample *from* it on each run instead of one session per exact
+`n` — wasn't implemented; a session today is still scoped to one exact
+parameter tuple, `--subsample` included.
+
 ### Output format: PHYLIP-relaxed by default for the distance matrix

 **Implemented.** The primary distance-matrix output
@@ -17,7 +17,7 @@ obikmer phylo INDEX [OPTIONS]
 | Option | Default | Description |
 |---|---|---|
 | `--distance` | `jaccard` | See the two tables below for the full list of accepted values |
-| `--gamma-shape ALPHA` | none | Rate-heterogeneity correction, for `snp-*` values that support it (see below). No effect on the other values; rejected if given together with a value that doesn't support it |
+| `--gamma-shape ALPHA\|auto` | none | Rate-heterogeneity correction, for `snp-*` values that support it (see below). Either a fixed $\alpha$ or `auto`/`estimate` to fit it from the data (see "Automatic $\alpha$ estimation" below). No effect on the other values; rejected if given together with a value that doesn't support it |
 | `--presence-threshold` | `1` | Minimum count for a kmer to be considered present, for `jaccard`/`mash` on a count index |
 | `--csv` | off | Write the matrix as plain CSV instead of the default relaxed-PHYLIP format |
 | `--shared-kmers` | off | Also write the shared-kmer count matrix. Only valid with a whole-index metric, not a `snp-*` value |
@@ -121,6 +121,28 @@ $$d = Q$$

 `--gamma-shape ALPHA` applies to every value above except `snp-raw` and `snp-tv`: each $-\ln(x)$ term in the formulas above is replaced by $\alpha\left(x^{-1/\alpha}-1\right)$ (the same weight, same $x$).

+### Automatic $\alpha$ estimation (`--gamma-shape auto`)
+
+`--gamma-shape auto` (or the equivalent `--gamma-shape estimate`) fits $\alpha$ from the index itself instead of requiring a user-supplied value, using a method-of-moments estimator computed once, from the same sampling pass that builds the pairwise substitution tally — no extra scan of the index.
+
+The estimator pools substitution counts by **partition** rather than by genome pair: for partition $i$, let $n_i$ be the total number of substitutions observed across every genome pair, and $L_i$ the total number of eligible loci across every genome pair, in that partition. Define the partition's observed substitution rate:
+
+$$R_i = \frac{n_i}{L_i}$$
+
+Under a single shared substitution rate with no among-site heterogeneity, each $R_i$ would vary only by Poisson sampling noise. Rate heterogeneity is modeled, as elsewhere in this correction, by a $\mathrm{Gamma}(\alpha,\alpha)$-distributed multiplicative rate (mean 1) shared by every locus in a partition — the classical Poisson–Gamma (negative-binomial) mixture. Under that model:
+
+$$\mathbb{E}[R_i] = \mu \qquad \mathrm{Var}[R_i] = \frac{\mu}{L_i} + \frac{\mu^2}{\alpha}$$
+
+where $\mu$ is the pooled substitution rate across every partition. Weighting each partition's squared deviation by its own $L_i$ removes the first (Poisson) term before attributing what's left to genuine rate heterogeneity:
+
+$$\hat\mu = \frac{\sum_i n_i}{\sum_i L_i} \qquad V = \frac{\sum_i L_i\,(R_i-\hat\mu)^2}{\sum_i L_i} \qquad \bar L = \frac{\sum_i L_i}{\text{number of partitions}}$$
+
+$$\hat\alpha = \frac{\hat\mu^2}{V - \hat\mu/\bar L}$$
+
+If the measured variance $V$ doesn't exceed the Poisson floor $\hat\mu/\bar L$ (no detectable over-dispersion across partitions — the data are consistent with a single shared rate), $\alpha$ is left undefined: the correction is silently disabled for that run rather than applying a fabricated value, and a warning is logged. When an estimate is produced, it's logged at the `info` level before the distance matrix is computed.
+
+Note: this is a method-of-moments estimator derived from the standard Poisson–Gamma relationship between substitution counts and gamma-distributed rate variation, applied per-partition — it is not part of Jin & Nei's (1990) original publication, which only defines the `+Γ` distance formula itself and, absent an estimate, recommends the fixed default $\alpha = 1$ (`--gamma-shape 1`) rather than proposing a way to estimate it from data. `alpha < 1` indicates strong among-site rate heterogeneity (many near-invariant loci, a few fast ones); `alpha` growing large makes the correction converge to the uncorrected formula.
+
 ### Output

 Without `-o`, the matrix goes to stdout in relaxed-PHYLIP format (`n` on the first line, then one `label<TAB>value...` row per genome). With `--csv`, the format is instead a header row `genome,<label1>,<label2>,...` followed by one `<label>,<value1>,<value2>,...` row per genome, 6 decimals. Both formats are symmetric with a zero diagonal, except where noted below.
@@ -180,6 +202,23 @@ A family is eligible for a genome pair $(i,j)$ only if both genomes carry exactl

 `--subsample`, `--free-loss`, `--no-ambiguity`, `--entropy`/`--entropy-sd` are shared by `--pseudo-alignment`, `--sankoff` (and everything it implies: `--tnt`/`--phyg`/`--iqtree`), and a `snp-*` `--distance` value — one draw feeds all of them in a single invocation. `--subsample` is mandatory for `--pseudo-alignment`/`--sankoff`; for a `snp-*` `--distance` value it is optional (omitted means every non-monomorphic family in the index, not an approximation).

+Combining `--sankoff` (or `--tnt`/`--phyg`/`--iqtree`) with a `snp-*` `--distance` value in the same command reuses that one draw for both — the distance and the Sankoff calibration/alignment are guaranteed to be computed from the *identical* set of sampled sites, never two independent samples, so the two outputs are directly comparable. This only holds within a single command; running them as two separate `obikmer phylo` invocations draws two independent samples even with the same flags — unless `--session` is used (below).
+
+Every invocation, with or without `--session`, draws its own fresh random sample by default — running the same command twice gives two different (but equally valid) samples, which is useful for measuring sampling variance and is kept that way deliberately. `--session` does not change this: it makes a *specific* sample reusable on request, it does not make sampling itself reproducible from one independent run to the next.
+
+### `--session`: reusing a sample across separate commands
+
+| Option | Default | Description |
+|---|---|---|
+| `--session DIR` | none | Persist the sample (and, for `--sankoff`, its calibration/alignment) in `DIR` so a later, separate `obikmer phylo` invocation with the exact same selection parameters restores it instead of resampling |
+| `--session-force` | off | With `--session DIR`: overwrite its saved parameters and cached sample instead of erroring out when this run's parameters don't match. No effect without `--session` |
+
+`DIR` is created if it doesn't exist. If it already holds a sample built with different `--subsample`/`--free-loss`/`--no-ambiguity`/`--exclude-genome`/`--min-shared-family`/`--entropy`/`--entropy-sd` values than this run, the command exits with an error rather than silently using either the old or the new values — pass `--session-force` to discard the old sample and rebuild under the new parameters, or point `--session` at a different directory to keep both.
+
+A `--sankoff`-family run and a `snp-*` `--distance` run share the same cached sample when pointed at the same `--session DIR` — build it once with either, reuse it from the other, in either order, across separate commands.
+
+If a run using `--session` is interrupted (crash, kill, `Ctrl-C`), the next run against the same `--session DIR` resumes from the last automatic checkpoint (roughly every 8 partitions'-worth of sampling progress) instead of starting over. The resumed run's sample is **not** guaranteed to be identical to what an uninterrupted run would have produced past that checkpoint — each process draws its own independent random sequence, same as any two separate invocations do — but nothing already checkpointed is lost, and no work needs redoing beyond that point.
+
 Without `--subsample`, every variable family (family size ≥ 2) is used. With `--subsample N`, roughly `N` families are kept instead, drawn in proportion to how many candidate families each part of the index actually holds, so the sample stays representative of the whole index. If the index has fewer than `N` candidate families, `--subsample` has no effect.

 ### `--shannon`: measuring how informative a family is
@@ -2,23 +2,27 @@
 BINARY  := ../src/target/release/obikmer
 VENV_PY := ../.venv/bin/python3

-GENOMES := $(wildcard genomes/*.fna.gz)
+# All generated/downloaded artifacts live under RUN/ so the whole tree can be
+# gitignored with a single entry (benchmark/run/) — see benchmark/README.md.
+RUN := run
+
+GENOMES := $(wildcard $(RUN)/genomes/*.fna.gz)

 # SPECIMENS, SPECIES, and the full dependency graph are generated by
 # make_deps.py from the genome FASTA headers — like .d files in C.
 # Make rebuilds deps.mk whenever genomes/ changes and restarts.
 -include deps.mk

-REF_NPZS              := $(SPECIMENS:%=reference_index/%.npz)
-REF_DIST_CSVS         := $(addprefix reference_dist/, \
+REF_NPZS              := $(SPECIMENS:%=$(RUN)/reference_index/%.npz)
+REF_DIST_CSVS         := $(addprefix $(RUN)/reference_dist/, \
 	shared_kmers.csv hamming_dist.csv jaccard_dist.csv \
 	bray_curtis_dist.csv relfreq_bray_curtis_dist.csv \
 	euclidean_dist.csv relfreq_euclidean_dist.csv \
 	hellinger_dist.csv hellinger_euclidean_dist.csv)
-OBIKMER_PRESENCE_DIST := $(addprefix obikmer_dist/presence/, \
+OBIKMER_PRESENCE_DIST := $(addprefix $(RUN)/obikmer_dist/presence/, \
 	jaccard_dist.csv jaccard_shared.csv jaccard_nj.nwk \
 	hamming_dist.csv hamming_nj.nwk)
-OBIKMER_COUNT_DIST    := $(addprefix obikmer_dist/count/, \
+OBIKMER_COUNT_DIST    := $(addprefix $(RUN)/obikmer_dist/count/, \
 	jaccard_dist.csv jaccard_shared.csv jaccard_nj.nwk \
 	bray_curtis_dist.csv bray_curtis_nj.nwk \
 	relfreq_bray_curtis_dist.csv relfreq_bray_curtis_nj.nwk \
@@ -26,28 +30,28 @@ OBIKMER_COUNT_DIST    := $(addprefix obikmer_dist/count/, \
 	relfreq_euclidean_dist.csv relfreq_euclidean_nj.nwk \
 	hellinger_dist.csv hellinger_nj.nwk \
 	hellinger_euclidean_dist.csv hellinger_euclidean_nj.nwk)
-DIST_COMPARISON       := stats/dist_comparison/summary.csv
-PRESENCE_DONE         := $(SPECIMENS:%=specimen_index_presence/%/index.done)
-PRESENCE_STATS        := $(SPECIMENS:%=stats/indexing_presence/%.stats)
-COUNT_DONE            := $(SPECIMENS:%=specimen_index_count/%/index.done)
-COUNT_STATS           := $(SPECIMENS:%=stats/indexing_count/%.stats)
-VERIFY_PRESENCE_STATS := $(SPECIMENS:%=stats/verify_presence/%.stats)
-VERIFY_COUNT_STATS    := $(SPECIMENS:%=stats/verify_count/%.stats)
-SPECIFIC_PRESENCE_DONE  := $(SPECIES:%=specific_index_presence/%/index.done)
-SPECIFIC_PRESENCE_STATS := $(SPECIES:%=stats/specific_kmer_presence/%.stats)
-SPECIFIC_COUNT_DONE     := $(SPECIES:%=specific_index_count/%/index.done)
-SPECIFIC_COUNT_STATS    := $(SPECIES:%=stats/specific_kmer_count/%.stats)
-SIMULATED_READS := $(foreach s,$(SPECIMENS),simulated_data/$(subst --,/,$s)/reads_R1.fastq.gz)
-QUERY_READS      := $(foreach s,$(QUERY_SPECIMENS),query_data/$(subst --,/,$s)/reads_R1.fastq.gz)
-QUERY_PRESENCE_DENSE_DONE   := $(QUERY_SPECIMENS:%=query_presence_dense/%.fasta.gz)
-QUERY_PRESENCE_DENSE_STATS  := $(QUERY_SPECIMENS:%=stats/query_presence_dense/%.stats)
-QUERY_PRESENCE_SPARSE_DONE  := $(QUERY_SPECIMENS:%=query_presence_sparse/%.fasta.gz)
-QUERY_PRESENCE_SPARSE_STATS := $(QUERY_SPECIMENS:%=stats/query_presence_sparse/%.stats)
-QUERY_COUNT_DENSE_DONE      := $(QUERY_SPECIMENS:%=query_count_dense/%.fasta.gz)
-QUERY_COUNT_DENSE_STATS     := $(QUERY_SPECIMENS:%=stats/query_count_dense/%.stats)
-QUERY_COUNT_SPARSE_DONE     := $(QUERY_SPECIMENS:%=query_count_sparse/%.fasta.gz)
-QUERY_COUNT_SPARSE_STATS    := $(QUERY_SPECIMENS:%=stats/query_count_sparse/%.stats)
-VERIFY_QUERY_STATS := $(QUERY_SPECIMENS:%=stats/verify_query/%.stats)
+DIST_COMPARISON       := $(RUN)/stats/dist_comparison/summary.csv
+PRESENCE_DONE         := $(SPECIMENS:%=$(RUN)/specimen_index_presence/%/index.done)
+PRESENCE_STATS        := $(SPECIMENS:%=$(RUN)/stats/indexing_presence/%.stats)
+COUNT_DONE            := $(SPECIMENS:%=$(RUN)/specimen_index_count/%/index.done)
+COUNT_STATS           := $(SPECIMENS:%=$(RUN)/stats/indexing_count/%.stats)
+VERIFY_PRESENCE_STATS := $(SPECIMENS:%=$(RUN)/stats/verify_presence/%.stats)
+VERIFY_COUNT_STATS    := $(SPECIMENS:%=$(RUN)/stats/verify_count/%.stats)
+SPECIFIC_PRESENCE_DONE  := $(SPECIES:%=$(RUN)/specific_index_presence/%/index.done)
+SPECIFIC_PRESENCE_STATS := $(SPECIES:%=$(RUN)/stats/specific_kmer_presence/%.stats)
+SPECIFIC_COUNT_DONE     := $(SPECIES:%=$(RUN)/specific_index_count/%/index.done)
+SPECIFIC_COUNT_STATS    := $(SPECIES:%=$(RUN)/stats/specific_kmer_count/%.stats)
+SIMULATED_READS := $(foreach s,$(SPECIMENS),$(RUN)/simulated_data/$(subst --,/,$s)/reads_R1.fastq.gz)
+QUERY_READS      := $(foreach s,$(QUERY_SPECIMENS),$(RUN)/query_data/$(subst --,/,$s)/reads_R1.fastq.gz)
+QUERY_PRESENCE_DENSE_DONE   := $(QUERY_SPECIMENS:%=$(RUN)/query_presence_dense/%.fasta.gz)
+QUERY_PRESENCE_DENSE_STATS  := $(QUERY_SPECIMENS:%=$(RUN)/stats/query_presence_dense/%.stats)
+QUERY_PRESENCE_SPARSE_DONE  := $(QUERY_SPECIMENS:%=$(RUN)/query_presence_sparse/%.fasta.gz)
+QUERY_PRESENCE_SPARSE_STATS := $(QUERY_SPECIMENS:%=$(RUN)/stats/query_presence_sparse/%.stats)
+QUERY_COUNT_DENSE_DONE      := $(QUERY_SPECIMENS:%=$(RUN)/query_count_dense/%.fasta.gz)
+QUERY_COUNT_DENSE_STATS     := $(QUERY_SPECIMENS:%=$(RUN)/stats/query_count_dense/%.stats)
+QUERY_COUNT_SPARSE_DONE     := $(QUERY_SPECIMENS:%=$(RUN)/query_count_sparse/%.fasta.gz)
+QUERY_COUNT_SPARSE_STATS    := $(QUERY_SPECIMENS:%=$(RUN)/stats/query_count_sparse/%.stats)
+VERIFY_QUERY_STATS := $(QUERY_SPECIMENS:%=$(RUN)/stats/verify_query/%.stats)

 .NOTPARALLEL:

@@ -69,8 +73,8 @@ VERIFY_QUERY_STATS := $(QUERY_SPECIMENS:%=stats/verify_query/%.stats)
        aggregate_query_count_dense aggregate_query_count_sparse \
        verify_query aggregate_verify_query

-verify_merge_presence: stats/verify_merge_presence/current.csv
-verify_merge_count:    stats/verify_merge_count/current.csv
+verify_merge_presence: $(RUN)/stats/verify_merge_presence/current.csv
+verify_merge_count:    $(RUN)/stats/verify_merge_count/current.csv

 all: aggregate_verify_presence aggregate_verify_count \
     verify_merge_presence verify_merge_count \
@@ -104,7 +108,7 @@ simulate_query: $(QUERY_READS)
 # ── reference kmer sets ───────────────────────────────────────────────────────
 # Prerequisites (reads → npz) are in deps.mk.

-reference_index/%.npz:
+$(RUN)/reference_index/%.npz:
 	bash build_reference.sh $*

 reference: $(REF_NPZS)
@@ -112,57 +116,58 @@ reference: $(REF_NPZS)
 # ── reference distance matrices ───────────────────────────────────────────────

 $(REF_DIST_CSVS) &: $(REF_NPZS) build_reference_dist.py
-	$(VENV_PY) build_reference_dist.py
+	$(VENV_PY) build_reference_dist.py \
+		--ref-dir $(RUN)/reference_index --out-dir $(RUN)/reference_dist

 reference_dist: $(REF_DIST_CSVS)

 # ── obikmer phylo (presence index) ──────────────────────────────────────────

-$(OBIKMER_PRESENCE_DIST) &: global_index_presence/index.done $(BINARY)
-	mkdir -p obikmer_dist/presence
+$(OBIKMER_PRESENCE_DIST) &: $(RUN)/global_index_presence/index.done $(BINARY)
+	mkdir -p $(RUN)/obikmer_dist/presence
 	$(BINARY) phylo \
-		--output obikmer_dist/presence/jaccard \
+		--output $(RUN)/obikmer_dist/presence/jaccard \
 		--distance jaccard --csv --shared-kmers --nj \
-		global_index_presence
+		$(RUN)/global_index_presence
 	$(BINARY) phylo \
-		--output obikmer_dist/presence/hamming \
+		--output $(RUN)/obikmer_dist/presence/hamming \
 		--distance hamming --csv --nj \
-		global_index_presence
+		$(RUN)/global_index_presence

 obikmer_dist_presence: $(OBIKMER_PRESENCE_DIST)

 # ── obikmer phylo (count index) ─────────────────────────────────────────────

-$(OBIKMER_COUNT_DIST) &: global_index_count/index.done $(BINARY)
-	mkdir -p obikmer_dist/count
+$(OBIKMER_COUNT_DIST) &: $(RUN)/global_index_count/index.done $(BINARY)
+	mkdir -p $(RUN)/obikmer_dist/count
 	$(BINARY) phylo \
-		--output obikmer_dist/count/jaccard \
+		--output $(RUN)/obikmer_dist/count/jaccard \
 		--distance jaccard --csv --shared-kmers --nj \
-		global_index_count
+		$(RUN)/global_index_count
 	$(BINARY) phylo \
-		--output obikmer_dist/count/bray_curtis \
+		--output $(RUN)/obikmer_dist/count/bray_curtis \
 		--distance bray-curtis --csv --nj \
-		global_index_count
+		$(RUN)/global_index_count
 	$(BINARY) phylo \
-		--output obikmer_dist/count/relfreq_bray_curtis \
+		--output $(RUN)/obikmer_dist/count/relfreq_bray_curtis \
 		--distance relfreq-bray-curtis --csv --nj \
-		global_index_count
+		$(RUN)/global_index_count
 	$(BINARY) phylo \
-		--output obikmer_dist/count/euclidean \
+		--output $(RUN)/obikmer_dist/count/euclidean \
 		--distance euclidean --csv --nj \
-		global_index_count
+		$(RUN)/global_index_count
 	$(BINARY) phylo \
-		--output obikmer_dist/count/relfreq_euclidean \
+		--output $(RUN)/obikmer_dist/count/relfreq_euclidean \
 		--distance relfreq-euclidean --csv --nj \
-		global_index_count
+		$(RUN)/global_index_count
 	$(BINARY) phylo \
-		--output obikmer_dist/count/hellinger \
+		--output $(RUN)/obikmer_dist/count/hellinger \
 		--distance hellinger --csv --nj \
-		global_index_count
+		$(RUN)/global_index_count
 	$(BINARY) phylo \
-		--output obikmer_dist/count/hellinger_euclidean \
+		--output $(RUN)/obikmer_dist/count/hellinger_euclidean \
 		--distance hellinger-euclidean --csv --nj \
-		global_index_count
+		$(RUN)/global_index_count

 obikmer_dist_count: $(OBIKMER_COUNT_DIST)

@@ -171,19 +176,19 @@ obikmer_dist: obikmer_dist_presence obikmer_dist_count
 # ── distance comparison ───────────────────────────────────────────────────────

 $(DIST_COMPARISON): $(REF_DIST_CSVS) $(OBIKMER_PRESENCE_DIST) $(OBIKMER_COUNT_DIST) compare_all_dist.py
-	$(VENV_PY) compare_all_dist.py --out $(DIST_COMPARISON)
+	$(VENV_PY) compare_all_dist.py --run-dir $(RUN) --out $(DIST_COMPARISON)

 dist_comparison: $(DIST_COMPARISON)

 # ── per-specimen indexing ─────────────────────────────────────────────────────
 # Prerequisites (reads → index.done + .stats) are in deps.mk.

-specimen_index_presence/%/index.done \
-stats/indexing_presence/%.stats &: $(BINARY)
+$(RUN)/specimen_index_presence/%/index.done \
+$(RUN)/stats/indexing_presence/%.stats &: $(BINARY)
 	bash index_one_presence.sh $*

-specimen_index_count/%/index.done \
-stats/indexing_count/%.stats &: $(BINARY)
+$(RUN)/specimen_index_count/%/index.done \
+$(RUN)/stats/indexing_count/%.stats &: $(BINARY)
 	bash index_one_count.sh $*

 index_presence: $(PRESENCE_DONE)
@@ -199,22 +204,22 @@ aggregate_index_count: $(COUNT_STATS)

 # ── global merge ──────────────────────────────────────────────────────────────

-global_index_presence/index.done: $(PRESENCE_DONE) $(BINARY)
+$(RUN)/global_index_presence/index.done: $(PRESENCE_DONE) $(BINARY)
 	bash merge_presence.sh

-global_index_count/index.done: $(COUNT_DONE) $(BINARY)
+$(RUN)/global_index_count/index.done: $(COUNT_DONE) $(BINARY)
 	bash merge_count.sh

-merge_presence: global_index_presence/index.done
-merge_count:    global_index_count/index.done
+merge_presence: $(RUN)/global_index_presence/index.done
+merge_count:    $(RUN)/global_index_count/index.done

 # ── per-specimen verification ─────────────────────────────────────────────────
 # Prerequisites (index.done + npz → .stats) are in deps.mk.

-stats/verify_presence/%.stats:
+$(RUN)/stats/verify_presence/%.stats:
 	bash verify_one_presence.sh $*

-stats/verify_count/%.stats:
+$(RUN)/stats/verify_count/%.stats:
 	bash verify_one_count.sh $*

 verify_presence: $(VERIFY_PRESENCE_STATS)
@@ -231,12 +236,12 @@ aggregate_verify_count: $(VERIFY_COUNT_STATS)
 # ── species-specific indexes ──────────────────────────────────────────────────
 # Prerequisites (global index → specific index) are in deps.mk.

-specific_index_presence/%/index.done \
-stats/specific_kmer_presence/%.stats &: $(BINARY)
+$(RUN)/specific_index_presence/%/index.done \
+$(RUN)/stats/specific_kmer_presence/%.stats &: $(BINARY)
 	bash filter_one_presence.sh $*

-specific_index_count/%/index.done \
-stats/specific_kmer_count/%.stats &: $(BINARY)
+$(RUN)/specific_index_count/%/index.done \
+$(RUN)/stats/specific_kmer_count/%.stats &: $(BINARY)
 	bash filter_one_count.sh $*

 filter_presence: $(SPECIFIC_PRESENCE_DONE)
@@ -250,10 +255,10 @@ aggregate_filter_count: $(SPECIFIC_COUNT_STATS)

 # ── merged index verification ─────────────────────────────────────────────────

-stats/verify_merge_presence/current.csv: $(REF_NPZS) global_index_presence/index.done
+$(RUN)/stats/verify_merge_presence/current.csv: $(REF_NPZS) $(RUN)/global_index_presence/index.done
 	bash verify_merge_presence.sh

-stats/verify_merge_count/current.csv: $(REF_NPZS) global_index_count/index.done
+$(RUN)/stats/verify_merge_count/current.csv: $(REF_NPZS) $(RUN)/global_index_count/index.done
 	bash verify_merge_count.sh

 # ── dense variants (query benchmark) ────────────────────────────────────────────
@@ -262,34 +267,34 @@ stats/verify_merge_count/current.csv: $(REF_NPZS) global_index_count/index.done
 # built explicitly here, from a hard-link-based copy (see
 # copy_index_hardlink.sh) rather than a full `cp -r`.

-global_index_presence_dense/index.done: global_index_presence/index.done $(BINARY)
+$(RUN)/global_index_presence_dense/index.done: $(RUN)/global_index_presence/index.done $(BINARY)
 	bash pack_dense.sh presence

 # Rebuilt from the per-specimen count sources directly (via `merge --dense`),
 # not repacked from global_index_count — see pack_dense.sh's own comment.
-global_index_count_dense/index.done: $(COUNT_DONE) $(BINARY)
+$(RUN)/global_index_count_dense/index.done: $(COUNT_DONE) $(BINARY)
 	bash pack_dense.sh count

-pack_dense_presence: global_index_presence_dense/index.done
-pack_dense_count:    global_index_count_dense/index.done
+pack_dense_presence: $(RUN)/global_index_presence_dense/index.done
+pack_dense_count:    $(RUN)/global_index_count_dense/index.done

 # ── query: dense vs sparse, presence and count ──────────────────────────────────
 # Prerequisites (reads + index → output + .stats) are in deps.mk.

-query_presence_dense/%.fasta.gz \
-stats/query_presence_dense/%.stats &: $(BINARY) global_index_presence_dense/index.done
+$(RUN)/query_presence_dense/%.fasta.gz \
+$(RUN)/stats/query_presence_dense/%.stats &: $(BINARY) $(RUN)/global_index_presence_dense/index.done
 	bash query_one.sh presence dense $*

-query_presence_sparse/%.fasta.gz \
-stats/query_presence_sparse/%.stats &: $(BINARY) global_index_presence/index.done
+$(RUN)/query_presence_sparse/%.fasta.gz \
+$(RUN)/stats/query_presence_sparse/%.stats &: $(BINARY) $(RUN)/global_index_presence/index.done
 	bash query_one.sh presence sparse $*

-query_count_dense/%.fasta.gz \
-stats/query_count_dense/%.stats &: $(BINARY) global_index_count_dense/index.done
+$(RUN)/query_count_dense/%.fasta.gz \
+$(RUN)/stats/query_count_dense/%.stats &: $(BINARY) $(RUN)/global_index_count_dense/index.done
 	bash query_one.sh count dense $*

-query_count_sparse/%.fasta.gz \
-stats/query_count_sparse/%.stats &: $(BINARY) global_index_count/index.done
+$(RUN)/query_count_sparse/%.fasta.gz \
+$(RUN)/stats/query_count_sparse/%.stats &: $(BINARY) $(RUN)/global_index_count/index.done
 	bash query_one.sh count sparse $*

 query_presence_dense:  $(QUERY_PRESENCE_DENSE_DONE)
@@ -311,7 +316,7 @@ aggregate_query_count_sparse: $(QUERY_COUNT_SPARSE_STATS)

 # ── query: dense/sparse regression ──────────────────────────────────────────────

-stats/verify_query/%.stats:
+$(RUN)/stats/verify_query/%.stats:
 	bash verify_query_one.sh $*

 verify_query: $(VERIFY_QUERY_STATS)
@@ -8,108 +8,124 @@ gmake simulate     # simulation only
 gmake reference    # reference kmer sets only
 ```

+All generated and downloaded artifacts live under `run/` (see
+[Directory layout](#directory-layout)), so the whole tree is gitignored with
+a single `benchmark/run/` entry.
+
 ## Pipeline overview

 ```mermaid
 flowchart TD
-    GENOMES["genomes/*.fna.gz"]
+    GENOMES["run/genomes/*.fna.gz"]
    BIN["obikmer binary"]

    GENOMES --> simulate
-    simulate --> simdata[("simulated_data/")]
+    simulate --> simdata[("run/simulated_data/")]

    simdata --> reference
-    reference --> refnpz[("reference_index/*.npz")]
+    reference --> refnpz[("run/reference_index/*.npz")]

    subgraph presence ["Presence track"]
        simdata  --> index_presence
        BIN      --> index_presence
-        index_presence --> pres_done[("specimen_index_presence/")]
-        index_presence --> pres_istats[("stats/indexing_presence/")]
+        index_presence --> pres_done[("run/specimen_index_presence/")]
+        index_presence --> pres_istats[("run/stats/indexing_presence/")]
        pres_istats --> aggregate_index_presence

        pres_done --> merge_presence
        BIN       --> merge_presence
-        merge_presence --> gpres[("global_index_presence/")]
+        merge_presence --> gpres[("run/global_index_presence/")]

        refnpz    --> verify_presence
        pres_done --> verify_presence
-        verify_presence --> vpres_stats[("stats/verify_presence/")]
+        verify_presence --> vpres_stats[("run/stats/verify_presence/")]
        vpres_stats --> aggregate_verify_presence

        gpres --> filter_presence
        BIN   --> filter_presence
-        filter_presence --> spec_pres[("specific_index_presence/")]
-        filter_presence --> spec_pres_stats[("stats/specific_kmer_presence/")]
+        filter_presence --> spec_pres[("run/specific_index_presence/")]
+        filter_presence --> spec_pres_stats[("run/stats/specific_kmer_presence/")]
        spec_pres_stats --> aggregate_filter_presence

        refnpz --> verify_merge_presence
        gpres  --> verify_merge_presence
-        verify_merge_presence --> vmp[("stats/verify_merge_presence/")]
+        verify_merge_presence --> vmp[("run/stats/verify_merge_presence/")]
    end

    subgraph count ["Count track"]
        simdata --> index_count
        BIN     --> index_count
-        index_count --> count_done[("specimen_index_count/")]
-        index_count --> count_istats[("stats/indexing_count/")]
+        index_count --> count_done[("run/specimen_index_count/")]
+        index_count --> count_istats[("run/stats/indexing_count/")]
        count_istats --> aggregate_index_count

        count_done --> merge_count
        BIN        --> merge_count
-        merge_count --> gcount[("global_index_count/")]
+        merge_count --> gcount[("run/global_index_count/")]

        refnpz     --> verify_count
        count_done --> verify_count
-        verify_count --> vcount_stats[("stats/verify_count/")]
+        verify_count --> vcount_stats[("run/stats/verify_count/")]
        vcount_stats --> aggregate_verify_count

        gcount --> filter_count
        BIN    --> filter_count
-        filter_count --> spec_count[("specific_index_count/")]
-        filter_count --> spec_count_stats[("stats/specific_kmer_count/")]
+        filter_count --> spec_count[("run/specific_index_count/")]
+        filter_count --> spec_count_stats[("run/stats/specific_kmer_count/")]
        spec_count_stats --> aggregate_filter_count

        refnpz --> verify_merge_count
        gcount --> verify_merge_count
-        verify_merge_count --> vmc[("stats/verify_merge_count/")]
+        verify_merge_count --> vmc[("run/stats/verify_merge_count/")]
    end

    subgraph query ["Query track (2 specimens: E. coli + archaeon)"]
        GENOMES --> simulate_query
-        simulate_query --> qdata[("query_data/")]
+        simulate_query --> qdata[("run/query_data/")]

-        gpres --> pack_sparse
-        BIN   --> pack_sparse
-        pack_sparse --> gsparse[("global_index_presence_sparse/")]
+        gpres --> pack_dense_presence
+        BIN   --> pack_dense_presence
+        pack_dense_presence --> gpresd[("run/global_index_presence_dense/")]

-        qdata  --> query_dense
-        gpres  --> query_dense
-        BIN    --> query_dense
-        query_dense --> qd[("query_dense/")]
-        query_dense --> qd_stats[("stats/query_dense/")]
-        qd_stats --> aggregate_query_dense
+        count_done --> pack_dense_count
+        BIN        --> pack_dense_count
+        pack_dense_count --> gcountd[("run/global_index_count_dense/")]

-        qdata   --> query_sparse
-        gsparse --> query_sparse
-        BIN     --> query_sparse
-        query_sparse --> qs[("query_sparse/")]
-        query_sparse --> qs_stats[("stats/query_sparse/")]
-        qs_stats --> aggregate_query_sparse
+        qdata  --> query_presence_dense
+        gpresd --> query_presence_dense
+        query_presence_dense --> qpd[("run/query_presence_dense/")]
+        qpd --> aggregate_query_presence_dense

-        qd --> verify_query
-        qs --> verify_query
-        verify_query --> vq_stats[("stats/verify_query/")]
+        qdata  --> query_presence_sparse
+        gpres  --> query_presence_sparse
+        query_presence_sparse --> qps[("run/query_presence_sparse/")]
+        qps --> aggregate_query_presence_sparse
+
+        qdata   --> query_count_dense
+        gcountd --> query_count_dense
+        query_count_dense --> qcd[("run/query_count_dense/")]
+        qcd --> aggregate_query_count_dense
+
+        qdata  --> query_count_sparse
+        gcount --> query_count_sparse
+        query_count_sparse --> qcs[("run/query_count_sparse/")]
+        qcs --> aggregate_query_count_sparse
+
+        qpd --> verify_query
+        qps --> verify_query
+        verify_query --> vq_stats[("run/stats/verify_query/")]
        vq_stats --> aggregate_verify_query
    end

-    aggregate_verify_presence  --> all
-    aggregate_verify_count     --> all
-    vmp                        --> all
-    vmc                        --> all
-    aggregate_query_dense      --> all
-    aggregate_query_sparse     --> all
-    aggregate_verify_query     --> all
+    aggregate_verify_presence         --> all
+    aggregate_verify_count            --> all
+    vmp                               --> all
+    vmc                               --> all
+    aggregate_query_presence_dense    --> all
+    aggregate_query_presence_sparse   --> all
+    aggregate_query_count_dense       --> all
+    aggregate_query_count_sparse      --> all
+    aggregate_verify_query            --> all
    all -. "$(MAKE) re-eval" .-> aggregate_filter_presence
    all -. "$(MAKE) re-eval" .-> aggregate_filter_count
 ```
@@ -137,44 +153,59 @@ flowchart TD
 | `verify_merge_presence` | `verify_merge_presence.sh` | Verify global presence index against all reference sets |
 | `verify_merge_count` | `verify_merge_count.sh` | Verify global count index against all reference sets |
 | `simulate_query` | `simulate_query_one.sh` | Simulate a fixed-size (100k pairs) read set per query specimen |
-| `pack_sparse` | `pack_sparse.sh` | Build `global_index_presence_sparse/` from `global_index_presence/` |
-| `query_dense` | `query_one.sh dense` | Query each query specimen's reads against the dense global index |
-| `query_sparse` | `query_one.sh sparse` | Query each query specimen's reads against the sparse global index |
-| `aggregate_query_dense` | `aggregate_stats.sh` | Aggregate dense query wall/RSS stats |
-| `aggregate_query_sparse` | `aggregate_stats.sh` | Aggregate sparse query wall/RSS stats |
-| `verify_query` | `verify_query_one.sh` | Diff dense vs sparse query output per specimen (regression check) |
+| `pack_dense_presence` | `pack_dense.sh presence` | Build `global_index_presence_dense/` from `global_index_presence/` |
+| `pack_dense_count` | `pack_dense.sh count` | Build `global_index_count_dense/` from the per-specimen count sources |
+| `query_presence_dense` | `query_one.sh presence dense` | Query against the dense presence global index |
+| `query_presence_sparse` | `query_one.sh presence sparse` | Query against the sparse (as-merged) presence global index |
+| `query_count_dense` | `query_one.sh count dense` | Query against the dense count global index |
+| `query_count_sparse` | `query_one.sh count sparse` | Query against the sparse (as-merged) count global index |
+| `aggregate_query_presence_dense` | `aggregate_stats.sh` | Aggregate dense presence-query wall/RSS stats |
+| `aggregate_query_presence_sparse` | `aggregate_stats.sh` | Aggregate sparse presence-query wall/RSS stats |
+| `aggregate_query_count_dense` | `aggregate_stats.sh` | Aggregate dense count-query wall/RSS stats |
+| `aggregate_query_count_sparse` | `aggregate_stats.sh` | Aggregate sparse count-query wall/RSS stats |
+| `verify_query` | `verify_query_one.sh` | Diff dense vs sparse presence-query output per specimen (regression check) |
 | `aggregate_verify_query` | `aggregate_stats.sh` | Aggregate dense/sparse query regression stats |

 ## Directory layout

 ```
 benchmark/
-├── genomes/                        # input reference genomes (.fna.gz)
-├── simulated_data/                 # generated by simulate
-│   └── <species>/<specimen>/
-├── query_data/                     # generated by simulate_query (2 specimens, fixed 100k pairs)
-│   └── <species>/<specimen>/
-├── reference_index/                # reference kmer sets (.npz)
-├── specimen_index_presence/        # per-specimen presence indexes
-├── specimen_index_count/           # per-specimen count indexes
-├── global_index_presence/          # merged global presence index (dense-packed)
-├── global_index_presence_sparse/   # global presence index, sparse-packed (query benchmark)
-├── global_index_count/             # merged global count index
-├── specific_index_presence/        # species-specific presence indexes
-├── specific_index_count/           # species-specific count indexes
-├── query_dense/                    # query output against global_index_presence
-├── query_sparse/                   # query output against global_index_presence_sparse
-└── stats/                          # all benchmark statistics
-    ├── indexing_presence/
-    ├── indexing_count/
-    ├── verify_presence/
-    ├── verify_count/
-    ├── specific_kmer_presence/
-    ├── specific_kmer_count/
-    ├── verify_merge_presence/
-    ├── verify_merge_count/
-    ├── pack_sparse/
-    ├── query_dense/
-    ├── query_sparse/
-    └── verify_query/
+└── run/                                 # everything generated/downloaded — gitignored as a whole
+    ├── genomes/                         # input reference genomes (.fna.gz), downloaded by downloads.sh
+    ├── simulated_data/                  # generated by simulate
+    │   └── <species>/<specimen>/
+    ├── query_data/                      # generated by simulate_query (2 specimens, fixed 100k pairs)
+    │   └── <species>/<specimen>/
+    ├── reference_index/                 # reference kmer sets (.npz)
+    ├── reference_dist/                  # reference pairwise distance matrices
+    ├── obikmer_dist/                    # obikmer phylo distance matrices (presence/, count/)
+    ├── specimen_index_presence/         # per-specimen presence indexes
+    ├── specimen_index_count/            # per-specimen count indexes
+    ├── global_index_presence/           # merged global presence index (sparse, as merged)
+    ├── global_index_presence_dense/     # global presence index, dense-repacked (query benchmark)
+    ├── global_index_count/              # merged global count index (sparse, as merged)
+    ├── global_index_count_dense/        # global count index, dense-repacked (query benchmark)
+    ├── specific_index_presence/         # species-specific presence indexes
+    ├── specific_index_count/            # species-specific count indexes
+    ├── query_presence_dense/            # query output against global_index_presence_dense
+    ├── query_presence_sparse/           # query output against global_index_presence
+    ├── query_count_dense/               # query output against global_index_count_dense
+    ├── query_count_sparse/              # query output against global_index_count
+    └── stats/                           # all benchmark statistics
+        ├── indexing_presence/
+        ├── indexing_count/
+        ├── verify_presence/
+        ├── verify_count/
+        ├── specific_kmer_presence/
+        ├── specific_kmer_count/
+        ├── verify_merge_presence/
+        ├── verify_merge_count/
+        ├── pack_dense_presence/
+        ├── pack_dense_count/
+        ├── query_presence_dense/
+        ├── query_presence_sparse/
+        ├── query_count_dense/
+        ├── query_count_sparse/
+        ├── dist_comparison/
+        └── verify_query/
 ```
@@ -11,7 +11,7 @@ set -euo pipefail

 TYPE="$1"
 SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
-STATS_DIR="${SCRIPT_DIR}/stats/${TYPE}"
+STATS_DIR="${SCRIPT_DIR}/run/stats/${TYPE}"

 case "${TYPE}" in
    indexing_presence|indexing_count)
@@ -2,8 +2,9 @@
 set -euo pipefail

 SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
-SIMDATA_DIR="${SCRIPT_DIR}/simulated_data"
-REF_DIR="${SCRIPT_DIR}/reference_index"
+RUN_DIR="${SCRIPT_DIR}/run"
+SIMDATA_DIR="${RUN_DIR}/simulated_data"
+REF_DIR="${RUN_DIR}/reference_index"
 PYTHON="${SCRIPT_DIR}/../.venv/bin/python3"
 BUILD_PY="${SCRIPT_DIR}/build_reference.py"

@@ -176,10 +176,10 @@ def write_csv(path: Path, labels: list[str], mat: np.ndarray, fmt: str) -> None:
 def main() -> None:
    ap = argparse.ArgumentParser(description=__doc__,
                                 formatter_class=argparse.RawDescriptionHelpFormatter)
-    ap.add_argument('--ref-dir', default='reference_index',
-                    help='Directory with per-specimen .npz files (default: reference_index)')
-    ap.add_argument('--out-dir', default='reference_dist',
-                    help='Output directory for CSV files (default: reference_dist)')
+    ap.add_argument('--ref-dir', default='run/reference_index',
+                    help='Directory with per-specimen .npz files (default: run/reference_index)')
+    ap.add_argument('--out-dir', default='run/reference_dist',
+                    help='Output directory for CSV files (default: run/reference_dist)')
    args = ap.parse_args()

    ref_dir = Path(args.ref_dir)
@@ -150,17 +150,20 @@ def main() -> None:
                                 formatter_class=argparse.RawDescriptionHelpFormatter)
    ap.add_argument('--tol', type=float, default=1e-4,
                    help='Max abs diff threshold for PASS/FAIL (default 1e-4)')
-    ap.add_argument('--out', default='stats/dist_comparison/summary.csv',
+    ap.add_argument('--run-dir', default='run',
+                    help='Root directory holding reference_dist/ and obikmer_dist/ (default: run)')
+    ap.add_argument('--out', default='run/stats/dist_comparison/summary.csv',
                    help='Output summary CSV path')
    args = ap.parse_args()

    out_path = Path(args.out)
    out_path.parent.mkdir(parents=True, exist_ok=True)

+    run_dir = Path(args.run_dir)
    print(f'Comparing {len(COMPARISONS)} matrix pairs…', file=sys.stderr)
    rows = []
    for label, ref, obi in COMPARISONS:
-        rows.append(compare(label, Path(ref), Path(obi), tol=args.tol))
+        rows.append(compare(label, run_dir / ref, run_dir / obi, tol=args.tol))

    fields = ['comparison', 'max_abs', 'mean_abs', 'rmse', 'n_pairs', 'status']
    with out_path.open('w', newline='') as fh:
@@ -2,212 +2,212 @@ SPECIMENS := Escherichia_coli--K-12_MG1655 Escherichia_coli--EDL933 Salmonella_e
 SPECIES   := Escherichia_coli Salmonella_enterica Bacillus_subtilis Shouchella_clausii Klebsiella_pneumoniae Opitutus_terrae Saccharolobus_islandicus Acidobacterium_capsulatum Proteus_mirabilis Wolbachia_endosymbiont Yersinia_ruckeri Candidozyma_auris

 # Escherichia_coli--K-12_MG1655
-simulated_data/Escherichia_coli/K-12_MG1655/reads_R1.fastq.gz: genomes/GCF_000005845.2_ASM584v2_genomic.fna.gz
-reference_index/Escherichia_coli--K-12_MG1655.npz: simulated_data/Escherichia_coli/K-12_MG1655/reads_R1.fastq.gz
-specimen_index_presence/Escherichia_coli--K-12_MG1655/index.done stats/indexing_presence/Escherichia_coli--K-12_MG1655.stats: simulated_data/Escherichia_coli/K-12_MG1655/reads_R1.fastq.gz
-specimen_index_count/Escherichia_coli--K-12_MG1655/index.done stats/indexing_count/Escherichia_coli--K-12_MG1655.stats: simulated_data/Escherichia_coli/K-12_MG1655/reads_R1.fastq.gz
-stats/verify_presence/Escherichia_coli--K-12_MG1655.stats: reference_index/Escherichia_coli--K-12_MG1655.npz specimen_index_presence/Escherichia_coli--K-12_MG1655/index.done
-stats/verify_count/Escherichia_coli--K-12_MG1655.stats: reference_index/Escherichia_coli--K-12_MG1655.npz specimen_index_count/Escherichia_coli--K-12_MG1655/index.done
+run/simulated_data/Escherichia_coli/K-12_MG1655/reads_R1.fastq.gz: run/genomes/GCF_000005845.2_ASM584v2_genomic.fna.gz
+run/reference_index/Escherichia_coli--K-12_MG1655.npz: run/simulated_data/Escherichia_coli/K-12_MG1655/reads_R1.fastq.gz
+run/specimen_index_presence/Escherichia_coli--K-12_MG1655/index.done run/stats/indexing_presence/Escherichia_coli--K-12_MG1655.stats: run/simulated_data/Escherichia_coli/K-12_MG1655/reads_R1.fastq.gz
+run/specimen_index_count/Escherichia_coli--K-12_MG1655/index.done run/stats/indexing_count/Escherichia_coli--K-12_MG1655.stats: run/simulated_data/Escherichia_coli/K-12_MG1655/reads_R1.fastq.gz
+run/stats/verify_presence/Escherichia_coli--K-12_MG1655.stats: run/reference_index/Escherichia_coli--K-12_MG1655.npz run/specimen_index_presence/Escherichia_coli--K-12_MG1655/index.done
+run/stats/verify_count/Escherichia_coli--K-12_MG1655.stats: run/reference_index/Escherichia_coli--K-12_MG1655.npz run/specimen_index_count/Escherichia_coli--K-12_MG1655/index.done

 # Escherichia_coli--EDL933
-simulated_data/Escherichia_coli/EDL933/reads_R1.fastq.gz: genomes/GCF_000006665.1_ASM666v1_genomic.fna.gz
-reference_index/Escherichia_coli--EDL933.npz: simulated_data/Escherichia_coli/EDL933/reads_R1.fastq.gz
-specimen_index_presence/Escherichia_coli--EDL933/index.done stats/indexing_presence/Escherichia_coli--EDL933.stats: simulated_data/Escherichia_coli/EDL933/reads_R1.fastq.gz
-specimen_index_count/Escherichia_coli--EDL933/index.done stats/indexing_count/Escherichia_coli--EDL933.stats: simulated_data/Escherichia_coli/EDL933/reads_R1.fastq.gz
-stats/verify_presence/Escherichia_coli--EDL933.stats: reference_index/Escherichia_coli--EDL933.npz specimen_index_presence/Escherichia_coli--EDL933/index.done
-stats/verify_count/Escherichia_coli--EDL933.stats: reference_index/Escherichia_coli--EDL933.npz specimen_index_count/Escherichia_coli--EDL933/index.done
+run/simulated_data/Escherichia_coli/EDL933/reads_R1.fastq.gz: run/genomes/GCF_000006665.1_ASM666v1_genomic.fna.gz
+run/reference_index/Escherichia_coli--EDL933.npz: run/simulated_data/Escherichia_coli/EDL933/reads_R1.fastq.gz
+run/specimen_index_presence/Escherichia_coli--EDL933/index.done run/stats/indexing_presence/Escherichia_coli--EDL933.stats: run/simulated_data/Escherichia_coli/EDL933/reads_R1.fastq.gz
+run/specimen_index_count/Escherichia_coli--EDL933/index.done run/stats/indexing_count/Escherichia_coli--EDL933.stats: run/simulated_data/Escherichia_coli/EDL933/reads_R1.fastq.gz
+run/stats/verify_presence/Escherichia_coli--EDL933.stats: run/reference_index/Escherichia_coli--EDL933.npz run/specimen_index_presence/Escherichia_coli--EDL933/index.done
+run/stats/verify_count/Escherichia_coli--EDL933.stats: run/reference_index/Escherichia_coli--EDL933.npz run/specimen_index_count/Escherichia_coli--EDL933/index.done

 # Salmonella_enterica--LT2
-simulated_data/Salmonella_enterica/LT2/reads_R1.fastq.gz: genomes/GCF_000006945.2_ASM694v2_genomic.fna.gz
-reference_index/Salmonella_enterica--LT2.npz: simulated_data/Salmonella_enterica/LT2/reads_R1.fastq.gz
-specimen_index_presence/Salmonella_enterica--LT2/index.done stats/indexing_presence/Salmonella_enterica--LT2.stats: simulated_data/Salmonella_enterica/LT2/reads_R1.fastq.gz
-specimen_index_count/Salmonella_enterica--LT2/index.done stats/indexing_count/Salmonella_enterica--LT2.stats: simulated_data/Salmonella_enterica/LT2/reads_R1.fastq.gz
-stats/verify_presence/Salmonella_enterica--LT2.stats: reference_index/Salmonella_enterica--LT2.npz specimen_index_presence/Salmonella_enterica--LT2/index.done
-stats/verify_count/Salmonella_enterica--LT2.stats: reference_index/Salmonella_enterica--LT2.npz specimen_index_count/Salmonella_enterica--LT2/index.done
+run/simulated_data/Salmonella_enterica/LT2/reads_R1.fastq.gz: run/genomes/GCF_000006945.2_ASM694v2_genomic.fna.gz
+run/reference_index/Salmonella_enterica--LT2.npz: run/simulated_data/Salmonella_enterica/LT2/reads_R1.fastq.gz
+run/specimen_index_presence/Salmonella_enterica--LT2/index.done run/stats/indexing_presence/Salmonella_enterica--LT2.stats: run/simulated_data/Salmonella_enterica/LT2/reads_R1.fastq.gz
+run/specimen_index_count/Salmonella_enterica--LT2/index.done run/stats/indexing_count/Salmonella_enterica--LT2.stats: run/simulated_data/Salmonella_enterica/LT2/reads_R1.fastq.gz
+run/stats/verify_presence/Salmonella_enterica--LT2.stats: run/reference_index/Salmonella_enterica--LT2.npz run/specimen_index_presence/Salmonella_enterica--LT2/index.done
+run/stats/verify_count/Salmonella_enterica--LT2.stats: run/reference_index/Salmonella_enterica--LT2.npz run/specimen_index_count/Salmonella_enterica--LT2/index.done

 # Escherichia_coli--CFT073
-simulated_data/Escherichia_coli/CFT073/reads_R1.fastq.gz: genomes/GCF_000007445.1_ASM744v1_genomic.fna.gz
-reference_index/Escherichia_coli--CFT073.npz: simulated_data/Escherichia_coli/CFT073/reads_R1.fastq.gz
-specimen_index_presence/Escherichia_coli--CFT073/index.done stats/indexing_presence/Escherichia_coli--CFT073.stats: simulated_data/Escherichia_coli/CFT073/reads_R1.fastq.gz
-specimen_index_count/Escherichia_coli--CFT073/index.done stats/indexing_count/Escherichia_coli--CFT073.stats: simulated_data/Escherichia_coli/CFT073/reads_R1.fastq.gz
-stats/verify_presence/Escherichia_coli--CFT073.stats: reference_index/Escherichia_coli--CFT073.npz specimen_index_presence/Escherichia_coli--CFT073/index.done
-stats/verify_count/Escherichia_coli--CFT073.stats: reference_index/Escherichia_coli--CFT073.npz specimen_index_count/Escherichia_coli--CFT073/index.done
+run/simulated_data/Escherichia_coli/CFT073/reads_R1.fastq.gz: run/genomes/GCF_000007445.1_ASM744v1_genomic.fna.gz
+run/reference_index/Escherichia_coli--CFT073.npz: run/simulated_data/Escherichia_coli/CFT073/reads_R1.fastq.gz
+run/specimen_index_presence/Escherichia_coli--CFT073/index.done run/stats/indexing_presence/Escherichia_coli--CFT073.stats: run/simulated_data/Escherichia_coli/CFT073/reads_R1.fastq.gz
+run/specimen_index_count/Escherichia_coli--CFT073/index.done run/stats/indexing_count/Escherichia_coli--CFT073.stats: run/simulated_data/Escherichia_coli/CFT073/reads_R1.fastq.gz
+run/stats/verify_presence/Escherichia_coli--CFT073.stats: run/reference_index/Escherichia_coli--CFT073.npz run/specimen_index_presence/Escherichia_coli--CFT073/index.done
+run/stats/verify_count/Escherichia_coli--CFT073.stats: run/reference_index/Escherichia_coli--CFT073.npz run/specimen_index_count/Escherichia_coli--CFT073/index.done

 # Bacillus_subtilis--168
-simulated_data/Bacillus_subtilis/168/reads_R1.fastq.gz: genomes/GCF_000009045.1_ASM904v1_genomic.fna.gz
-reference_index/Bacillus_subtilis--168.npz: simulated_data/Bacillus_subtilis/168/reads_R1.fastq.gz
-specimen_index_presence/Bacillus_subtilis--168/index.done stats/indexing_presence/Bacillus_subtilis--168.stats: simulated_data/Bacillus_subtilis/168/reads_R1.fastq.gz
-specimen_index_count/Bacillus_subtilis--168/index.done stats/indexing_count/Bacillus_subtilis--168.stats: simulated_data/Bacillus_subtilis/168/reads_R1.fastq.gz
-stats/verify_presence/Bacillus_subtilis--168.stats: reference_index/Bacillus_subtilis--168.npz specimen_index_presence/Bacillus_subtilis--168/index.done
-stats/verify_count/Bacillus_subtilis--168.stats: reference_index/Bacillus_subtilis--168.npz specimen_index_count/Bacillus_subtilis--168/index.done
+run/simulated_data/Bacillus_subtilis/168/reads_R1.fastq.gz: run/genomes/GCF_000009045.1_ASM904v1_genomic.fna.gz
+run/reference_index/Bacillus_subtilis--168.npz: run/simulated_data/Bacillus_subtilis/168/reads_R1.fastq.gz
+run/specimen_index_presence/Bacillus_subtilis--168/index.done run/stats/indexing_presence/Bacillus_subtilis--168.stats: run/simulated_data/Bacillus_subtilis/168/reads_R1.fastq.gz
+run/specimen_index_count/Bacillus_subtilis--168/index.done run/stats/indexing_count/Bacillus_subtilis--168.stats: run/simulated_data/Bacillus_subtilis/168/reads_R1.fastq.gz
+run/stats/verify_presence/Bacillus_subtilis--168.stats: run/reference_index/Bacillus_subtilis--168.npz run/specimen_index_presence/Bacillus_subtilis--168/index.done
+run/stats/verify_count/Bacillus_subtilis--168.stats: run/reference_index/Bacillus_subtilis--168.npz run/specimen_index_count/Bacillus_subtilis--168/index.done

 # Salmonella_enterica--P125109
-simulated_data/Salmonella_enterica/P125109/reads_R1.fastq.gz: genomes/GCF_000009505.1_ASM950v1_genomic.fna.gz
-reference_index/Salmonella_enterica--P125109.npz: simulated_data/Salmonella_enterica/P125109/reads_R1.fastq.gz
-specimen_index_presence/Salmonella_enterica--P125109/index.done stats/indexing_presence/Salmonella_enterica--P125109.stats: simulated_data/Salmonella_enterica/P125109/reads_R1.fastq.gz
-specimen_index_count/Salmonella_enterica--P125109/index.done stats/indexing_count/Salmonella_enterica--P125109.stats: simulated_data/Salmonella_enterica/P125109/reads_R1.fastq.gz
-stats/verify_presence/Salmonella_enterica--P125109.stats: reference_index/Salmonella_enterica--P125109.npz specimen_index_presence/Salmonella_enterica--P125109/index.done
-stats/verify_count/Salmonella_enterica--P125109.stats: reference_index/Salmonella_enterica--P125109.npz specimen_index_count/Salmonella_enterica--P125109/index.done
+run/simulated_data/Salmonella_enterica/P125109/reads_R1.fastq.gz: run/genomes/GCF_000009505.1_ASM950v1_genomic.fna.gz
+run/reference_index/Salmonella_enterica--P125109.npz: run/simulated_data/Salmonella_enterica/P125109/reads_R1.fastq.gz
+run/specimen_index_presence/Salmonella_enterica--P125109/index.done run/stats/indexing_presence/Salmonella_enterica--P125109.stats: run/simulated_data/Salmonella_enterica/P125109/reads_R1.fastq.gz
+run/specimen_index_count/Salmonella_enterica--P125109/index.done run/stats/indexing_count/Salmonella_enterica--P125109.stats: run/simulated_data/Salmonella_enterica/P125109/reads_R1.fastq.gz
+run/stats/verify_presence/Salmonella_enterica--P125109.stats: run/reference_index/Salmonella_enterica--P125109.npz run/specimen_index_presence/Salmonella_enterica--P125109/index.done
+run/stats/verify_count/Salmonella_enterica--P125109.stats: run/reference_index/Salmonella_enterica--P125109.npz run/specimen_index_count/Salmonella_enterica--P125109/index.done

 # Shouchella_clausii--KSM-K16
-simulated_data/Shouchella_clausii/KSM-K16/reads_R1.fastq.gz: genomes/GCF_000009825.1_ASM982v1_genomic.fna.gz
-reference_index/Shouchella_clausii--KSM-K16.npz: simulated_data/Shouchella_clausii/KSM-K16/reads_R1.fastq.gz
-specimen_index_presence/Shouchella_clausii--KSM-K16/index.done stats/indexing_presence/Shouchella_clausii--KSM-K16.stats: simulated_data/Shouchella_clausii/KSM-K16/reads_R1.fastq.gz
-specimen_index_count/Shouchella_clausii--KSM-K16/index.done stats/indexing_count/Shouchella_clausii--KSM-K16.stats: simulated_data/Shouchella_clausii/KSM-K16/reads_R1.fastq.gz
-stats/verify_presence/Shouchella_clausii--KSM-K16.stats: reference_index/Shouchella_clausii--KSM-K16.npz specimen_index_presence/Shouchella_clausii--KSM-K16/index.done
-stats/verify_count/Shouchella_clausii--KSM-K16.stats: reference_index/Shouchella_clausii--KSM-K16.npz specimen_index_count/Shouchella_clausii--KSM-K16/index.done
+run/simulated_data/Shouchella_clausii/KSM-K16/reads_R1.fastq.gz: run/genomes/GCF_000009825.1_ASM982v1_genomic.fna.gz
+run/reference_index/Shouchella_clausii--KSM-K16.npz: run/simulated_data/Shouchella_clausii/KSM-K16/reads_R1.fastq.gz
+run/specimen_index_presence/Shouchella_clausii--KSM-K16/index.done run/stats/indexing_presence/Shouchella_clausii--KSM-K16.stats: run/simulated_data/Shouchella_clausii/KSM-K16/reads_R1.fastq.gz
+run/specimen_index_count/Shouchella_clausii--KSM-K16/index.done run/stats/indexing_count/Shouchella_clausii--KSM-K16.stats: run/simulated_data/Shouchella_clausii/KSM-K16/reads_R1.fastq.gz
+run/stats/verify_presence/Shouchella_clausii--KSM-K16.stats: run/reference_index/Shouchella_clausii--KSM-K16.npz run/specimen_index_presence/Shouchella_clausii--KSM-K16/index.done
+run/stats/verify_count/Shouchella_clausii--KSM-K16.stats: run/reference_index/Shouchella_clausii--KSM-K16.npz run/specimen_index_count/Shouchella_clausii--KSM-K16/index.done

 # Escherichia_coli--K-12_W3110
-simulated_data/Escherichia_coli/K-12_W3110/reads_R1.fastq.gz: genomes/GCF_000010245.2_ASM1024v1_genomic.fna.gz
-reference_index/Escherichia_coli--K-12_W3110.npz: simulated_data/Escherichia_coli/K-12_W3110/reads_R1.fastq.gz
-specimen_index_presence/Escherichia_coli--K-12_W3110/index.done stats/indexing_presence/Escherichia_coli--K-12_W3110.stats: simulated_data/Escherichia_coli/K-12_W3110/reads_R1.fastq.gz
-specimen_index_count/Escherichia_coli--K-12_W3110/index.done stats/indexing_count/Escherichia_coli--K-12_W3110.stats: simulated_data/Escherichia_coli/K-12_W3110/reads_R1.fastq.gz
-stats/verify_presence/Escherichia_coli--K-12_W3110.stats: reference_index/Escherichia_coli--K-12_W3110.npz specimen_index_presence/Escherichia_coli--K-12_W3110/index.done
-stats/verify_count/Escherichia_coli--K-12_W3110.stats: reference_index/Escherichia_coli--K-12_W3110.npz specimen_index_count/Escherichia_coli--K-12_W3110/index.done
+run/simulated_data/Escherichia_coli/K-12_W3110/reads_R1.fastq.gz: run/genomes/GCF_000010245.2_ASM1024v1_genomic.fna.gz
+run/reference_index/Escherichia_coli--K-12_W3110.npz: run/simulated_data/Escherichia_coli/K-12_W3110/reads_R1.fastq.gz
+run/specimen_index_presence/Escherichia_coli--K-12_W3110/index.done run/stats/indexing_presence/Escherichia_coli--K-12_W3110.stats: run/simulated_data/Escherichia_coli/K-12_W3110/reads_R1.fastq.gz
+run/specimen_index_count/Escherichia_coli--K-12_W3110/index.done run/stats/indexing_count/Escherichia_coli--K-12_W3110.stats: run/simulated_data/Escherichia_coli/K-12_W3110/reads_R1.fastq.gz
+run/stats/verify_presence/Escherichia_coli--K-12_W3110.stats: run/reference_index/Escherichia_coli--K-12_W3110.npz run/specimen_index_presence/Escherichia_coli--K-12_W3110/index.done
+run/stats/verify_count/Escherichia_coli--K-12_W3110.stats: run/reference_index/Escherichia_coli--K-12_W3110.npz run/specimen_index_count/Escherichia_coli--K-12_W3110/index.done

 # Klebsiella_pneumoniae--MGH_78578
-simulated_data/Klebsiella_pneumoniae/MGH_78578/reads_R1.fastq.gz: genomes/GCF_000016305.1_ASM1630v1_genomic.fna.gz
-reference_index/Klebsiella_pneumoniae--MGH_78578.npz: simulated_data/Klebsiella_pneumoniae/MGH_78578/reads_R1.fastq.gz
-specimen_index_presence/Klebsiella_pneumoniae--MGH_78578/index.done stats/indexing_presence/Klebsiella_pneumoniae--MGH_78578.stats: simulated_data/Klebsiella_pneumoniae/MGH_78578/reads_R1.fastq.gz
-specimen_index_count/Klebsiella_pneumoniae--MGH_78578/index.done stats/indexing_count/Klebsiella_pneumoniae--MGH_78578.stats: simulated_data/Klebsiella_pneumoniae/MGH_78578/reads_R1.fastq.gz
-stats/verify_presence/Klebsiella_pneumoniae--MGH_78578.stats: reference_index/Klebsiella_pneumoniae--MGH_78578.npz specimen_index_presence/Klebsiella_pneumoniae--MGH_78578/index.done
-stats/verify_count/Klebsiella_pneumoniae--MGH_78578.stats: reference_index/Klebsiella_pneumoniae--MGH_78578.npz specimen_index_count/Klebsiella_pneumoniae--MGH_78578/index.done
+run/simulated_data/Klebsiella_pneumoniae/MGH_78578/reads_R1.fastq.gz: run/genomes/GCF_000016305.1_ASM1630v1_genomic.fna.gz
+run/reference_index/Klebsiella_pneumoniae--MGH_78578.npz: run/simulated_data/Klebsiella_pneumoniae/MGH_78578/reads_R1.fastq.gz
+run/specimen_index_presence/Klebsiella_pneumoniae--MGH_78578/index.done run/stats/indexing_presence/Klebsiella_pneumoniae--MGH_78578.stats: run/simulated_data/Klebsiella_pneumoniae/MGH_78578/reads_R1.fastq.gz
+run/specimen_index_count/Klebsiella_pneumoniae--MGH_78578/index.done run/stats/indexing_count/Klebsiella_pneumoniae--MGH_78578.stats: run/simulated_data/Klebsiella_pneumoniae/MGH_78578/reads_R1.fastq.gz
+run/stats/verify_presence/Klebsiella_pneumoniae--MGH_78578.stats: run/reference_index/Klebsiella_pneumoniae--MGH_78578.npz run/specimen_index_presence/Klebsiella_pneumoniae--MGH_78578/index.done
+run/stats/verify_count/Klebsiella_pneumoniae--MGH_78578.stats: run/reference_index/Klebsiella_pneumoniae--MGH_78578.npz run/specimen_index_count/Klebsiella_pneumoniae--MGH_78578/index.done

 # Opitutus_terrae--PB90-1
-simulated_data/Opitutus_terrae/PB90-1/reads_R1.fastq.gz: genomes/GCF_000019965.1_ASM1996v1_genomic.fna.gz
-reference_index/Opitutus_terrae--PB90-1.npz: simulated_data/Opitutus_terrae/PB90-1/reads_R1.fastq.gz
-specimen_index_presence/Opitutus_terrae--PB90-1/index.done stats/indexing_presence/Opitutus_terrae--PB90-1.stats: simulated_data/Opitutus_terrae/PB90-1/reads_R1.fastq.gz
-specimen_index_count/Opitutus_terrae--PB90-1/index.done stats/indexing_count/Opitutus_terrae--PB90-1.stats: simulated_data/Opitutus_terrae/PB90-1/reads_R1.fastq.gz
-stats/verify_presence/Opitutus_terrae--PB90-1.stats: reference_index/Opitutus_terrae--PB90-1.npz specimen_index_presence/Opitutus_terrae--PB90-1/index.done
-stats/verify_count/Opitutus_terrae--PB90-1.stats: reference_index/Opitutus_terrae--PB90-1.npz specimen_index_count/Opitutus_terrae--PB90-1/index.done
+run/simulated_data/Opitutus_terrae/PB90-1/reads_R1.fastq.gz: run/genomes/GCF_000019965.1_ASM1996v1_genomic.fna.gz
+run/reference_index/Opitutus_terrae--PB90-1.npz: run/simulated_data/Opitutus_terrae/PB90-1/reads_R1.fastq.gz
+run/specimen_index_presence/Opitutus_terrae--PB90-1/index.done run/stats/indexing_presence/Opitutus_terrae--PB90-1.stats: run/simulated_data/Opitutus_terrae/PB90-1/reads_R1.fastq.gz
+run/specimen_index_count/Opitutus_terrae--PB90-1/index.done run/stats/indexing_count/Opitutus_terrae--PB90-1.stats: run/simulated_data/Opitutus_terrae/PB90-1/reads_R1.fastq.gz
+run/stats/verify_presence/Opitutus_terrae--PB90-1.stats: run/reference_index/Opitutus_terrae--PB90-1.npz run/specimen_index_presence/Opitutus_terrae--PB90-1/index.done
+run/stats/verify_count/Opitutus_terrae--PB90-1.stats: run/reference_index/Opitutus_terrae--PB90-1.npz run/specimen_index_count/Opitutus_terrae--PB90-1/index.done

 # Saccharolobus_islandicus--M.16.4
-simulated_data/Saccharolobus_islandicus/M.16.4/reads_R1.fastq.gz: genomes/GCF_000022445.1_ASM2244v1_genomic.fna.gz
-reference_index/Saccharolobus_islandicus--M.16.4.npz: simulated_data/Saccharolobus_islandicus/M.16.4/reads_R1.fastq.gz
-specimen_index_presence/Saccharolobus_islandicus--M.16.4/index.done stats/indexing_presence/Saccharolobus_islandicus--M.16.4.stats: simulated_data/Saccharolobus_islandicus/M.16.4/reads_R1.fastq.gz
-specimen_index_count/Saccharolobus_islandicus--M.16.4/index.done stats/indexing_count/Saccharolobus_islandicus--M.16.4.stats: simulated_data/Saccharolobus_islandicus/M.16.4/reads_R1.fastq.gz
-stats/verify_presence/Saccharolobus_islandicus--M.16.4.stats: reference_index/Saccharolobus_islandicus--M.16.4.npz specimen_index_presence/Saccharolobus_islandicus--M.16.4/index.done
-stats/verify_count/Saccharolobus_islandicus--M.16.4.stats: reference_index/Saccharolobus_islandicus--M.16.4.npz specimen_index_count/Saccharolobus_islandicus--M.16.4/index.done
+run/simulated_data/Saccharolobus_islandicus/M.16.4/reads_R1.fastq.gz: run/genomes/GCF_000022445.1_ASM2244v1_genomic.fna.gz
+run/reference_index/Saccharolobus_islandicus--M.16.4.npz: run/simulated_data/Saccharolobus_islandicus/M.16.4/reads_R1.fastq.gz
+run/specimen_index_presence/Saccharolobus_islandicus--M.16.4/index.done run/stats/indexing_presence/Saccharolobus_islandicus--M.16.4.stats: run/simulated_data/Saccharolobus_islandicus/M.16.4/reads_R1.fastq.gz
+run/specimen_index_count/Saccharolobus_islandicus--M.16.4/index.done run/stats/indexing_count/Saccharolobus_islandicus--M.16.4.stats: run/simulated_data/Saccharolobus_islandicus/M.16.4/reads_R1.fastq.gz
+run/stats/verify_presence/Saccharolobus_islandicus--M.16.4.stats: run/reference_index/Saccharolobus_islandicus--M.16.4.npz run/specimen_index_presence/Saccharolobus_islandicus--M.16.4/index.done
+run/stats/verify_count/Saccharolobus_islandicus--M.16.4.stats: run/reference_index/Saccharolobus_islandicus--M.16.4.npz run/specimen_index_count/Saccharolobus_islandicus--M.16.4/index.done

 # Acidobacterium_capsulatum--ATCC_51196
-simulated_data/Acidobacterium_capsulatum/ATCC_51196/reads_R1.fastq.gz: genomes/GCF_000022565.1_ASM2256v1_genomic.fna.gz
-reference_index/Acidobacterium_capsulatum--ATCC_51196.npz: simulated_data/Acidobacterium_capsulatum/ATCC_51196/reads_R1.fastq.gz
-specimen_index_presence/Acidobacterium_capsulatum--ATCC_51196/index.done stats/indexing_presence/Acidobacterium_capsulatum--ATCC_51196.stats: simulated_data/Acidobacterium_capsulatum/ATCC_51196/reads_R1.fastq.gz
-specimen_index_count/Acidobacterium_capsulatum--ATCC_51196/index.done stats/indexing_count/Acidobacterium_capsulatum--ATCC_51196.stats: simulated_data/Acidobacterium_capsulatum/ATCC_51196/reads_R1.fastq.gz
-stats/verify_presence/Acidobacterium_capsulatum--ATCC_51196.stats: reference_index/Acidobacterium_capsulatum--ATCC_51196.npz specimen_index_presence/Acidobacterium_capsulatum--ATCC_51196/index.done
-stats/verify_count/Acidobacterium_capsulatum--ATCC_51196.stats: reference_index/Acidobacterium_capsulatum--ATCC_51196.npz specimen_index_count/Acidobacterium_capsulatum--ATCC_51196/index.done
+run/simulated_data/Acidobacterium_capsulatum/ATCC_51196/reads_R1.fastq.gz: run/genomes/GCF_000022565.1_ASM2256v1_genomic.fna.gz
+run/reference_index/Acidobacterium_capsulatum--ATCC_51196.npz: run/simulated_data/Acidobacterium_capsulatum/ATCC_51196/reads_R1.fastq.gz
+run/specimen_index_presence/Acidobacterium_capsulatum--ATCC_51196/index.done run/stats/indexing_presence/Acidobacterium_capsulatum--ATCC_51196.stats: run/simulated_data/Acidobacterium_capsulatum/ATCC_51196/reads_R1.fastq.gz
+run/specimen_index_count/Acidobacterium_capsulatum--ATCC_51196/index.done run/stats/indexing_count/Acidobacterium_capsulatum--ATCC_51196.stats: run/simulated_data/Acidobacterium_capsulatum/ATCC_51196/reads_R1.fastq.gz
+run/stats/verify_presence/Acidobacterium_capsulatum--ATCC_51196.stats: run/reference_index/Acidobacterium_capsulatum--ATCC_51196.npz run/specimen_index_presence/Acidobacterium_capsulatum--ATCC_51196/index.done
+run/stats/verify_count/Acidobacterium_capsulatum--ATCC_51196.stats: run/reference_index/Acidobacterium_capsulatum--ATCC_51196.npz run/specimen_index_count/Acidobacterium_capsulatum--ATCC_51196/index.done

 # Salmonella_enterica--AKU_12601
-simulated_data/Salmonella_enterica/AKU_12601/reads_R1.fastq.gz: genomes/GCF_000026565.1_ASM2656v1_genomic.fna.gz
-reference_index/Salmonella_enterica--AKU_12601.npz: simulated_data/Salmonella_enterica/AKU_12601/reads_R1.fastq.gz
-specimen_index_presence/Salmonella_enterica--AKU_12601/index.done stats/indexing_presence/Salmonella_enterica--AKU_12601.stats: simulated_data/Salmonella_enterica/AKU_12601/reads_R1.fastq.gz
-specimen_index_count/Salmonella_enterica--AKU_12601/index.done stats/indexing_count/Salmonella_enterica--AKU_12601.stats: simulated_data/Salmonella_enterica/AKU_12601/reads_R1.fastq.gz
-stats/verify_presence/Salmonella_enterica--AKU_12601.stats: reference_index/Salmonella_enterica--AKU_12601.npz specimen_index_presence/Salmonella_enterica--AKU_12601/index.done
-stats/verify_count/Salmonella_enterica--AKU_12601.stats: reference_index/Salmonella_enterica--AKU_12601.npz specimen_index_count/Salmonella_enterica--AKU_12601/index.done
+run/simulated_data/Salmonella_enterica/AKU_12601/reads_R1.fastq.gz: run/genomes/GCF_000026565.1_ASM2656v1_genomic.fna.gz
+run/reference_index/Salmonella_enterica--AKU_12601.npz: run/simulated_data/Salmonella_enterica/AKU_12601/reads_R1.fastq.gz
+run/specimen_index_presence/Salmonella_enterica--AKU_12601/index.done run/stats/indexing_presence/Salmonella_enterica--AKU_12601.stats: run/simulated_data/Salmonella_enterica/AKU_12601/reads_R1.fastq.gz
+run/specimen_index_count/Salmonella_enterica--AKU_12601/index.done run/stats/indexing_count/Salmonella_enterica--AKU_12601.stats: run/simulated_data/Salmonella_enterica/AKU_12601/reads_R1.fastq.gz
+run/stats/verify_presence/Salmonella_enterica--AKU_12601.stats: run/reference_index/Salmonella_enterica--AKU_12601.npz run/specimen_index_presence/Salmonella_enterica--AKU_12601/index.done
+run/stats/verify_count/Salmonella_enterica--AKU_12601.stats: run/reference_index/Salmonella_enterica--AKU_12601.npz run/specimen_index_count/Salmonella_enterica--AKU_12601/index.done

 # Proteus_mirabilis--HI4320
-simulated_data/Proteus_mirabilis/HI4320/reads_R1.fastq.gz: genomes/GCF_000069965.1_ASM6996v1_genomic.fna.gz
-reference_index/Proteus_mirabilis--HI4320.npz: simulated_data/Proteus_mirabilis/HI4320/reads_R1.fastq.gz
-specimen_index_presence/Proteus_mirabilis--HI4320/index.done stats/indexing_presence/Proteus_mirabilis--HI4320.stats: simulated_data/Proteus_mirabilis/HI4320/reads_R1.fastq.gz
-specimen_index_count/Proteus_mirabilis--HI4320/index.done stats/indexing_count/Proteus_mirabilis--HI4320.stats: simulated_data/Proteus_mirabilis/HI4320/reads_R1.fastq.gz
-stats/verify_presence/Proteus_mirabilis--HI4320.stats: reference_index/Proteus_mirabilis--HI4320.npz specimen_index_presence/Proteus_mirabilis--HI4320/index.done
-stats/verify_count/Proteus_mirabilis--HI4320.stats: reference_index/Proteus_mirabilis--HI4320.npz specimen_index_count/Proteus_mirabilis--HI4320/index.done
+run/simulated_data/Proteus_mirabilis/HI4320/reads_R1.fastq.gz: run/genomes/GCF_000069965.1_ASM6996v1_genomic.fna.gz
+run/reference_index/Proteus_mirabilis--HI4320.npz: run/simulated_data/Proteus_mirabilis/HI4320/reads_R1.fastq.gz
+run/specimen_index_presence/Proteus_mirabilis--HI4320/index.done run/stats/indexing_presence/Proteus_mirabilis--HI4320.stats: run/simulated_data/Proteus_mirabilis/HI4320/reads_R1.fastq.gz
+run/specimen_index_count/Proteus_mirabilis--HI4320/index.done run/stats/indexing_count/Proteus_mirabilis--HI4320.stats: run/simulated_data/Proteus_mirabilis/HI4320/reads_R1.fastq.gz
+run/stats/verify_presence/Proteus_mirabilis--HI4320.stats: run/reference_index/Proteus_mirabilis--HI4320.npz run/specimen_index_presence/Proteus_mirabilis--HI4320/index.done
+run/stats/verify_count/Proteus_mirabilis--HI4320.stats: run/reference_index/Proteus_mirabilis--HI4320.npz run/specimen_index_count/Proteus_mirabilis--HI4320/index.done

 # Salmonella_enterica--CT18
-simulated_data/Salmonella_enterica/CT18/reads_R1.fastq.gz: genomes/GCF_000195995.1_ASM19599v1_genomic.fna.gz
-reference_index/Salmonella_enterica--CT18.npz: simulated_data/Salmonella_enterica/CT18/reads_R1.fastq.gz
-specimen_index_presence/Salmonella_enterica--CT18/index.done stats/indexing_presence/Salmonella_enterica--CT18.stats: simulated_data/Salmonella_enterica/CT18/reads_R1.fastq.gz
-specimen_index_count/Salmonella_enterica--CT18/index.done stats/indexing_count/Salmonella_enterica--CT18.stats: simulated_data/Salmonella_enterica/CT18/reads_R1.fastq.gz
-stats/verify_presence/Salmonella_enterica--CT18.stats: reference_index/Salmonella_enterica--CT18.npz specimen_index_presence/Salmonella_enterica--CT18/index.done
-stats/verify_count/Salmonella_enterica--CT18.stats: reference_index/Salmonella_enterica--CT18.npz specimen_index_count/Salmonella_enterica--CT18/index.done
+run/simulated_data/Salmonella_enterica/CT18/reads_R1.fastq.gz: run/genomes/GCF_000195995.1_ASM19599v1_genomic.fna.gz
+run/reference_index/Salmonella_enterica--CT18.npz: run/simulated_data/Salmonella_enterica/CT18/reads_R1.fastq.gz
+run/specimen_index_presence/Salmonella_enterica--CT18/index.done run/stats/indexing_presence/Salmonella_enterica--CT18.stats: run/simulated_data/Salmonella_enterica/CT18/reads_R1.fastq.gz
+run/specimen_index_count/Salmonella_enterica--CT18/index.done run/stats/indexing_count/Salmonella_enterica--CT18.stats: run/simulated_data/Salmonella_enterica/CT18/reads_R1.fastq.gz
+run/stats/verify_presence/Salmonella_enterica--CT18.stats: run/reference_index/Salmonella_enterica--CT18.npz run/specimen_index_presence/Salmonella_enterica--CT18/index.done
+run/stats/verify_count/Salmonella_enterica--CT18.stats: run/reference_index/Salmonella_enterica--CT18.npz run/specimen_index_count/Salmonella_enterica--CT18/index.done

 # Klebsiella_pneumoniae--HS11286
-simulated_data/Klebsiella_pneumoniae/HS11286/reads_R1.fastq.gz: genomes/GCF_000240185.1_ASM24018v2_genomic.fna.gz
-reference_index/Klebsiella_pneumoniae--HS11286.npz: simulated_data/Klebsiella_pneumoniae/HS11286/reads_R1.fastq.gz
-specimen_index_presence/Klebsiella_pneumoniae--HS11286/index.done stats/indexing_presence/Klebsiella_pneumoniae--HS11286.stats: simulated_data/Klebsiella_pneumoniae/HS11286/reads_R1.fastq.gz
-specimen_index_count/Klebsiella_pneumoniae--HS11286/index.done stats/indexing_count/Klebsiella_pneumoniae--HS11286.stats: simulated_data/Klebsiella_pneumoniae/HS11286/reads_R1.fastq.gz
-stats/verify_presence/Klebsiella_pneumoniae--HS11286.stats: reference_index/Klebsiella_pneumoniae--HS11286.npz specimen_index_presence/Klebsiella_pneumoniae--HS11286/index.done
-stats/verify_count/Klebsiella_pneumoniae--HS11286.stats: reference_index/Klebsiella_pneumoniae--HS11286.npz specimen_index_count/Klebsiella_pneumoniae--HS11286/index.done
+run/simulated_data/Klebsiella_pneumoniae/HS11286/reads_R1.fastq.gz: run/genomes/GCF_000240185.1_ASM24018v2_genomic.fna.gz
+run/reference_index/Klebsiella_pneumoniae--HS11286.npz: run/simulated_data/Klebsiella_pneumoniae/HS11286/reads_R1.fastq.gz
+run/specimen_index_presence/Klebsiella_pneumoniae--HS11286/index.done run/stats/indexing_presence/Klebsiella_pneumoniae--HS11286.stats: run/simulated_data/Klebsiella_pneumoniae/HS11286/reads_R1.fastq.gz
+run/specimen_index_count/Klebsiella_pneumoniae--HS11286/index.done run/stats/indexing_count/Klebsiella_pneumoniae--HS11286.stats: run/simulated_data/Klebsiella_pneumoniae/HS11286/reads_R1.fastq.gz
+run/stats/verify_presence/Klebsiella_pneumoniae--HS11286.stats: run/reference_index/Klebsiella_pneumoniae--HS11286.npz run/specimen_index_presence/Klebsiella_pneumoniae--HS11286/index.done
+run/stats/verify_count/Klebsiella_pneumoniae--HS11286.stats: run/reference_index/Klebsiella_pneumoniae--HS11286.npz run/specimen_index_count/Klebsiella_pneumoniae--HS11286/index.done

 # Wolbachia_endosymbiont--GCF_000306885.1_ASM30688v1
-simulated_data/Wolbachia_endosymbiont/GCF_000306885.1_ASM30688v1/reads_R1.fastq.gz: genomes/GCF_000306885.1_ASM30688v1_genomic.fna.gz
-reference_index/Wolbachia_endosymbiont--GCF_000306885.1_ASM30688v1.npz: simulated_data/Wolbachia_endosymbiont/GCF_000306885.1_ASM30688v1/reads_R1.fastq.gz
-specimen_index_presence/Wolbachia_endosymbiont--GCF_000306885.1_ASM30688v1/index.done stats/indexing_presence/Wolbachia_endosymbiont--GCF_000306885.1_ASM30688v1.stats: simulated_data/Wolbachia_endosymbiont/GCF_000306885.1_ASM30688v1/reads_R1.fastq.gz
-specimen_index_count/Wolbachia_endosymbiont--GCF_000306885.1_ASM30688v1/index.done stats/indexing_count/Wolbachia_endosymbiont--GCF_000306885.1_ASM30688v1.stats: simulated_data/Wolbachia_endosymbiont/GCF_000306885.1_ASM30688v1/reads_R1.fastq.gz
-stats/verify_presence/Wolbachia_endosymbiont--GCF_000306885.1_ASM30688v1.stats: reference_index/Wolbachia_endosymbiont--GCF_000306885.1_ASM30688v1.npz specimen_index_presence/Wolbachia_endosymbiont--GCF_000306885.1_ASM30688v1/index.done
-stats/verify_count/Wolbachia_endosymbiont--GCF_000306885.1_ASM30688v1.stats: reference_index/Wolbachia_endosymbiont--GCF_000306885.1_ASM30688v1.npz specimen_index_count/Wolbachia_endosymbiont--GCF_000306885.1_ASM30688v1/index.done
+run/simulated_data/Wolbachia_endosymbiont/GCF_000306885.1_ASM30688v1/reads_R1.fastq.gz: run/genomes/GCF_000306885.1_ASM30688v1_genomic.fna.gz
+run/reference_index/Wolbachia_endosymbiont--GCF_000306885.1_ASM30688v1.npz: run/simulated_data/Wolbachia_endosymbiont/GCF_000306885.1_ASM30688v1/reads_R1.fastq.gz
+run/specimen_index_presence/Wolbachia_endosymbiont--GCF_000306885.1_ASM30688v1/index.done run/stats/indexing_presence/Wolbachia_endosymbiont--GCF_000306885.1_ASM30688v1.stats: run/simulated_data/Wolbachia_endosymbiont/GCF_000306885.1_ASM30688v1/reads_R1.fastq.gz
+run/specimen_index_count/Wolbachia_endosymbiont--GCF_000306885.1_ASM30688v1/index.done run/stats/indexing_count/Wolbachia_endosymbiont--GCF_000306885.1_ASM30688v1.stats: run/simulated_data/Wolbachia_endosymbiont/GCF_000306885.1_ASM30688v1/reads_R1.fastq.gz
+run/stats/verify_presence/Wolbachia_endosymbiont--GCF_000306885.1_ASM30688v1.stats: run/reference_index/Wolbachia_endosymbiont--GCF_000306885.1_ASM30688v1.npz run/specimen_index_presence/Wolbachia_endosymbiont--GCF_000306885.1_ASM30688v1/index.done
+run/stats/verify_count/Wolbachia_endosymbiont--GCF_000306885.1_ASM30688v1.stats: run/reference_index/Wolbachia_endosymbiont--GCF_000306885.1_ASM30688v1.npz run/specimen_index_count/Wolbachia_endosymbiont--GCF_000306885.1_ASM30688v1/index.done

 # Klebsiella_pneumoniae--ATCC_13883
-simulated_data/Klebsiella_pneumoniae/ATCC_13883/reads_R1.fastq.gz: genomes/GCF_000742135.1_ASM74213v1_genomic.fna.gz
-reference_index/Klebsiella_pneumoniae--ATCC_13883.npz: simulated_data/Klebsiella_pneumoniae/ATCC_13883/reads_R1.fastq.gz
-specimen_index_presence/Klebsiella_pneumoniae--ATCC_13883/index.done stats/indexing_presence/Klebsiella_pneumoniae--ATCC_13883.stats: simulated_data/Klebsiella_pneumoniae/ATCC_13883/reads_R1.fastq.gz
-specimen_index_count/Klebsiella_pneumoniae--ATCC_13883/index.done stats/indexing_count/Klebsiella_pneumoniae--ATCC_13883.stats: simulated_data/Klebsiella_pneumoniae/ATCC_13883/reads_R1.fastq.gz
-stats/verify_presence/Klebsiella_pneumoniae--ATCC_13883.stats: reference_index/Klebsiella_pneumoniae--ATCC_13883.npz specimen_index_presence/Klebsiella_pneumoniae--ATCC_13883/index.done
-stats/verify_count/Klebsiella_pneumoniae--ATCC_13883.stats: reference_index/Klebsiella_pneumoniae--ATCC_13883.npz specimen_index_count/Klebsiella_pneumoniae--ATCC_13883/index.done
+run/simulated_data/Klebsiella_pneumoniae/ATCC_13883/reads_R1.fastq.gz: run/genomes/GCF_000742135.1_ASM74213v1_genomic.fna.gz
+run/reference_index/Klebsiella_pneumoniae--ATCC_13883.npz: run/simulated_data/Klebsiella_pneumoniae/ATCC_13883/reads_R1.fastq.gz
+run/specimen_index_presence/Klebsiella_pneumoniae--ATCC_13883/index.done run/stats/indexing_presence/Klebsiella_pneumoniae--ATCC_13883.stats: run/simulated_data/Klebsiella_pneumoniae/ATCC_13883/reads_R1.fastq.gz
+run/specimen_index_count/Klebsiella_pneumoniae--ATCC_13883/index.done run/stats/indexing_count/Klebsiella_pneumoniae--ATCC_13883.stats: run/simulated_data/Klebsiella_pneumoniae/ATCC_13883/reads_R1.fastq.gz
+run/stats/verify_presence/Klebsiella_pneumoniae--ATCC_13883.stats: run/reference_index/Klebsiella_pneumoniae--ATCC_13883.npz run/specimen_index_presence/Klebsiella_pneumoniae--ATCC_13883/index.done
+run/stats/verify_count/Klebsiella_pneumoniae--ATCC_13883.stats: run/reference_index/Klebsiella_pneumoniae--ATCC_13883.npz run/specimen_index_count/Klebsiella_pneumoniae--ATCC_13883/index.done

 # Yersinia_ruckeri--YRB
-simulated_data/Yersinia_ruckeri/YRB/reads_R1.fastq.gz: genomes/GCF_000834255.1_ASM83425v1_genomic.fna.gz
-reference_index/Yersinia_ruckeri--YRB.npz: simulated_data/Yersinia_ruckeri/YRB/reads_R1.fastq.gz
-specimen_index_presence/Yersinia_ruckeri--YRB/index.done stats/indexing_presence/Yersinia_ruckeri--YRB.stats: simulated_data/Yersinia_ruckeri/YRB/reads_R1.fastq.gz
-specimen_index_count/Yersinia_ruckeri--YRB/index.done stats/indexing_count/Yersinia_ruckeri--YRB.stats: simulated_data/Yersinia_ruckeri/YRB/reads_R1.fastq.gz
-stats/verify_presence/Yersinia_ruckeri--YRB.stats: reference_index/Yersinia_ruckeri--YRB.npz specimen_index_presence/Yersinia_ruckeri--YRB/index.done
-stats/verify_count/Yersinia_ruckeri--YRB.stats: reference_index/Yersinia_ruckeri--YRB.npz specimen_index_count/Yersinia_ruckeri--YRB/index.done
+run/simulated_data/Yersinia_ruckeri/YRB/reads_R1.fastq.gz: run/genomes/GCF_000834255.1_ASM83425v1_genomic.fna.gz
+run/reference_index/Yersinia_ruckeri--YRB.npz: run/simulated_data/Yersinia_ruckeri/YRB/reads_R1.fastq.gz
+run/specimen_index_presence/Yersinia_ruckeri--YRB/index.done run/stats/indexing_presence/Yersinia_ruckeri--YRB.stats: run/simulated_data/Yersinia_ruckeri/YRB/reads_R1.fastq.gz
+run/specimen_index_count/Yersinia_ruckeri--YRB/index.done run/stats/indexing_count/Yersinia_ruckeri--YRB.stats: run/simulated_data/Yersinia_ruckeri/YRB/reads_R1.fastq.gz
+run/stats/verify_presence/Yersinia_ruckeri--YRB.stats: run/reference_index/Yersinia_ruckeri--YRB.npz run/specimen_index_presence/Yersinia_ruckeri--YRB/index.done
+run/stats/verify_count/Yersinia_ruckeri--YRB.stats: run/reference_index/Yersinia_ruckeri--YRB.npz run/specimen_index_count/Yersinia_ruckeri--YRB/index.done

 # Candidozyma_auris--GCF_003013715.1_ASM301371v2
-simulated_data/Candidozyma_auris/GCF_003013715.1_ASM301371v2/reads_R1.fastq.gz: genomes/GCF_003013715.1_ASM301371v2_genomic.fna.gz
-reference_index/Candidozyma_auris--GCF_003013715.1_ASM301371v2.npz: simulated_data/Candidozyma_auris/GCF_003013715.1_ASM301371v2/reads_R1.fastq.gz
-specimen_index_presence/Candidozyma_auris--GCF_003013715.1_ASM301371v2/index.done stats/indexing_presence/Candidozyma_auris--GCF_003013715.1_ASM301371v2.stats: simulated_data/Candidozyma_auris/GCF_003013715.1_ASM301371v2/reads_R1.fastq.gz
-specimen_index_count/Candidozyma_auris--GCF_003013715.1_ASM301371v2/index.done stats/indexing_count/Candidozyma_auris--GCF_003013715.1_ASM301371v2.stats: simulated_data/Candidozyma_auris/GCF_003013715.1_ASM301371v2/reads_R1.fastq.gz
-stats/verify_presence/Candidozyma_auris--GCF_003013715.1_ASM301371v2.stats: reference_index/Candidozyma_auris--GCF_003013715.1_ASM301371v2.npz specimen_index_presence/Candidozyma_auris--GCF_003013715.1_ASM301371v2/index.done
-stats/verify_count/Candidozyma_auris--GCF_003013715.1_ASM301371v2.stats: reference_index/Candidozyma_auris--GCF_003013715.1_ASM301371v2.npz specimen_index_count/Candidozyma_auris--GCF_003013715.1_ASM301371v2/index.done
+run/simulated_data/Candidozyma_auris/GCF_003013715.1_ASM301371v2/reads_R1.fastq.gz: run/genomes/GCF_003013715.1_ASM301371v2_genomic.fna.gz
+run/reference_index/Candidozyma_auris--GCF_003013715.1_ASM301371v2.npz: run/simulated_data/Candidozyma_auris/GCF_003013715.1_ASM301371v2/reads_R1.fastq.gz
+run/specimen_index_presence/Candidozyma_auris--GCF_003013715.1_ASM301371v2/index.done run/stats/indexing_presence/Candidozyma_auris--GCF_003013715.1_ASM301371v2.stats: run/simulated_data/Candidozyma_auris/GCF_003013715.1_ASM301371v2/reads_R1.fastq.gz
+run/specimen_index_count/Candidozyma_auris--GCF_003013715.1_ASM301371v2/index.done run/stats/indexing_count/Candidozyma_auris--GCF_003013715.1_ASM301371v2.stats: run/simulated_data/Candidozyma_auris/GCF_003013715.1_ASM301371v2/reads_R1.fastq.gz
+run/stats/verify_presence/Candidozyma_auris--GCF_003013715.1_ASM301371v2.stats: run/reference_index/Candidozyma_auris--GCF_003013715.1_ASM301371v2.npz run/specimen_index_presence/Candidozyma_auris--GCF_003013715.1_ASM301371v2/index.done
+run/stats/verify_count/Candidozyma_auris--GCF_003013715.1_ASM301371v2.stats: run/reference_index/Candidozyma_auris--GCF_003013715.1_ASM301371v2.npz run/specimen_index_count/Candidozyma_auris--GCF_003013715.1_ASM301371v2/index.done

 # Escherichia_coli
-specific_index_presence/Escherichia_coli/index.done stats/specific_kmer_presence/Escherichia_coli.stats: global_index_presence/index.done
-specific_index_count/Escherichia_coli/index.done stats/specific_kmer_count/Escherichia_coli.stats: global_index_count/index.done
+run/specific_index_presence/Escherichia_coli/index.done run/stats/specific_kmer_presence/Escherichia_coli.stats: run/global_index_presence/index.done
+run/specific_index_count/Escherichia_coli/index.done run/stats/specific_kmer_count/Escherichia_coli.stats: run/global_index_count/index.done
 # Salmonella_enterica
-specific_index_presence/Salmonella_enterica/index.done stats/specific_kmer_presence/Salmonella_enterica.stats: global_index_presence/index.done
-specific_index_count/Salmonella_enterica/index.done stats/specific_kmer_count/Salmonella_enterica.stats: global_index_count/index.done
+run/specific_index_presence/Salmonella_enterica/index.done run/stats/specific_kmer_presence/Salmonella_enterica.stats: run/global_index_presence/index.done
+run/specific_index_count/Salmonella_enterica/index.done run/stats/specific_kmer_count/Salmonella_enterica.stats: run/global_index_count/index.done
 # Bacillus_subtilis
-specific_index_presence/Bacillus_subtilis/index.done stats/specific_kmer_presence/Bacillus_subtilis.stats: global_index_presence/index.done
-specific_index_count/Bacillus_subtilis/index.done stats/specific_kmer_count/Bacillus_subtilis.stats: global_index_count/index.done
+run/specific_index_presence/Bacillus_subtilis/index.done run/stats/specific_kmer_presence/Bacillus_subtilis.stats: run/global_index_presence/index.done
+run/specific_index_count/Bacillus_subtilis/index.done run/stats/specific_kmer_count/Bacillus_subtilis.stats: run/global_index_count/index.done
 # Shouchella_clausii
-specific_index_presence/Shouchella_clausii/index.done stats/specific_kmer_presence/Shouchella_clausii.stats: global_index_presence/index.done
-specific_index_count/Shouchella_clausii/index.done stats/specific_kmer_count/Shouchella_clausii.stats: global_index_count/index.done
+run/specific_index_presence/Shouchella_clausii/index.done run/stats/specific_kmer_presence/Shouchella_clausii.stats: run/global_index_presence/index.done
+run/specific_index_count/Shouchella_clausii/index.done run/stats/specific_kmer_count/Shouchella_clausii.stats: run/global_index_count/index.done
 # Klebsiella_pneumoniae
-specific_index_presence/Klebsiella_pneumoniae/index.done stats/specific_kmer_presence/Klebsiella_pneumoniae.stats: global_index_presence/index.done
-specific_index_count/Klebsiella_pneumoniae/index.done stats/specific_kmer_count/Klebsiella_pneumoniae.stats: global_index_count/index.done
+run/specific_index_presence/Klebsiella_pneumoniae/index.done run/stats/specific_kmer_presence/Klebsiella_pneumoniae.stats: run/global_index_presence/index.done
+run/specific_index_count/Klebsiella_pneumoniae/index.done run/stats/specific_kmer_count/Klebsiella_pneumoniae.stats: run/global_index_count/index.done
 # Opitutus_terrae
-specific_index_presence/Opitutus_terrae/index.done stats/specific_kmer_presence/Opitutus_terrae.stats: global_index_presence/index.done
-specific_index_count/Opitutus_terrae/index.done stats/specific_kmer_count/Opitutus_terrae.stats: global_index_count/index.done
+run/specific_index_presence/Opitutus_terrae/index.done run/stats/specific_kmer_presence/Opitutus_terrae.stats: run/global_index_presence/index.done
+run/specific_index_count/Opitutus_terrae/index.done run/stats/specific_kmer_count/Opitutus_terrae.stats: run/global_index_count/index.done
 # Saccharolobus_islandicus
-specific_index_presence/Saccharolobus_islandicus/index.done stats/specific_kmer_presence/Saccharolobus_islandicus.stats: global_index_presence/index.done
-specific_index_count/Saccharolobus_islandicus/index.done stats/specific_kmer_count/Saccharolobus_islandicus.stats: global_index_count/index.done
+run/specific_index_presence/Saccharolobus_islandicus/index.done run/stats/specific_kmer_presence/Saccharolobus_islandicus.stats: run/global_index_presence/index.done
+run/specific_index_count/Saccharolobus_islandicus/index.done run/stats/specific_kmer_count/Saccharolobus_islandicus.stats: run/global_index_count/index.done
 # Acidobacterium_capsulatum
-specific_index_presence/Acidobacterium_capsulatum/index.done stats/specific_kmer_presence/Acidobacterium_capsulatum.stats: global_index_presence/index.done
-specific_index_count/Acidobacterium_capsulatum/index.done stats/specific_kmer_count/Acidobacterium_capsulatum.stats: global_index_count/index.done
+run/specific_index_presence/Acidobacterium_capsulatum/index.done run/stats/specific_kmer_presence/Acidobacterium_capsulatum.stats: run/global_index_presence/index.done
+run/specific_index_count/Acidobacterium_capsulatum/index.done run/stats/specific_kmer_count/Acidobacterium_capsulatum.stats: run/global_index_count/index.done
 # Proteus_mirabilis
-specific_index_presence/Proteus_mirabilis/index.done stats/specific_kmer_presence/Proteus_mirabilis.stats: global_index_presence/index.done
-specific_index_count/Proteus_mirabilis/index.done stats/specific_kmer_count/Proteus_mirabilis.stats: global_index_count/index.done
+run/specific_index_presence/Proteus_mirabilis/index.done run/stats/specific_kmer_presence/Proteus_mirabilis.stats: run/global_index_presence/index.done
+run/specific_index_count/Proteus_mirabilis/index.done run/stats/specific_kmer_count/Proteus_mirabilis.stats: run/global_index_count/index.done
 # Wolbachia_endosymbiont
-specific_index_presence/Wolbachia_endosymbiont/index.done stats/specific_kmer_presence/Wolbachia_endosymbiont.stats: global_index_presence/index.done
-specific_index_count/Wolbachia_endosymbiont/index.done stats/specific_kmer_count/Wolbachia_endosymbiont.stats: global_index_count/index.done
+run/specific_index_presence/Wolbachia_endosymbiont/index.done run/stats/specific_kmer_presence/Wolbachia_endosymbiont.stats: run/global_index_presence/index.done
+run/specific_index_count/Wolbachia_endosymbiont/index.done run/stats/specific_kmer_count/Wolbachia_endosymbiont.stats: run/global_index_count/index.done
 # Yersinia_ruckeri
-specific_index_presence/Yersinia_ruckeri/index.done stats/specific_kmer_presence/Yersinia_ruckeri.stats: global_index_presence/index.done
-specific_index_count/Yersinia_ruckeri/index.done stats/specific_kmer_count/Yersinia_ruckeri.stats: global_index_count/index.done
+run/specific_index_presence/Yersinia_ruckeri/index.done run/stats/specific_kmer_presence/Yersinia_ruckeri.stats: run/global_index_presence/index.done
+run/specific_index_count/Yersinia_ruckeri/index.done run/stats/specific_kmer_count/Yersinia_ruckeri.stats: run/global_index_count/index.done
 # Candidozyma_auris
-specific_index_presence/Candidozyma_auris/index.done stats/specific_kmer_presence/Candidozyma_auris.stats: global_index_presence/index.done
-specific_index_count/Candidozyma_auris/index.done stats/specific_kmer_count/Candidozyma_auris.stats: global_index_count/index.done
+run/specific_index_presence/Candidozyma_auris/index.done run/stats/specific_kmer_presence/Candidozyma_auris.stats: run/global_index_presence/index.done
+run/specific_index_count/Candidozyma_auris/index.done run/stats/specific_kmer_count/Candidozyma_auris.stats: run/global_index_count/index.done

 QUERY_SPECIMENS := Escherichia_coli--K-12_MG1655 Saccharolobus_islandicus--M.16.4

 # query: Escherichia_coli--K-12_MG1655
-query_data/Escherichia_coli/K-12_MG1655/reads_R1.fastq.gz: genomes/GCF_000005845.2_ASM584v2_genomic.fna.gz
-query_dense/Escherichia_coli--K-12_MG1655.fasta.gz stats/query_dense/Escherichia_coli--K-12_MG1655.stats: query_data/Escherichia_coli/K-12_MG1655/reads_R1.fastq.gz global_index_presence/index.done
-query_sparse/Escherichia_coli--K-12_MG1655.fasta.gz stats/query_sparse/Escherichia_coli--K-12_MG1655.stats: query_data/Escherichia_coli/K-12_MG1655/reads_R1.fastq.gz global_index_presence_sparse/index.done
-stats/verify_query/Escherichia_coli--K-12_MG1655.stats: query_dense/Escherichia_coli--K-12_MG1655.fasta.gz query_sparse/Escherichia_coli--K-12_MG1655.fasta.gz
+run/query_data/Escherichia_coli/K-12_MG1655/reads_R1.fastq.gz: run/genomes/GCF_000005845.2_ASM584v2_genomic.fna.gz
+run/query_presence_dense/Escherichia_coli--K-12_MG1655.fasta.gz run/stats/query_presence_dense/Escherichia_coli--K-12_MG1655.stats: run/query_data/Escherichia_coli/K-12_MG1655/reads_R1.fastq.gz run/global_index_presence_dense/index.done
+run/query_presence_sparse/Escherichia_coli--K-12_MG1655.fasta.gz run/stats/query_presence_sparse/Escherichia_coli--K-12_MG1655.stats: run/query_data/Escherichia_coli/K-12_MG1655/reads_R1.fastq.gz run/global_index_presence/index.done
+run/stats/verify_query/Escherichia_coli--K-12_MG1655.stats: run/query_presence_dense/Escherichia_coli--K-12_MG1655.fasta.gz run/query_presence_sparse/Escherichia_coli--K-12_MG1655.fasta.gz

 # query: Saccharolobus_islandicus--M.16.4
-query_data/Saccharolobus_islandicus/M.16.4/reads_R1.fastq.gz: genomes/GCF_000022445.1_ASM2244v1_genomic.fna.gz
-query_dense/Saccharolobus_islandicus--M.16.4.fasta.gz stats/query_dense/Saccharolobus_islandicus--M.16.4.stats: query_data/Saccharolobus_islandicus/M.16.4/reads_R1.fastq.gz global_index_presence/index.done
-query_sparse/Saccharolobus_islandicus--M.16.4.fasta.gz stats/query_sparse/Saccharolobus_islandicus--M.16.4.stats: query_data/Saccharolobus_islandicus/M.16.4/reads_R1.fastq.gz global_index_presence_sparse/index.done
-stats/verify_query/Saccharolobus_islandicus--M.16.4.stats: query_dense/Saccharolobus_islandicus--M.16.4.fasta.gz query_sparse/Saccharolobus_islandicus--M.16.4.fasta.gz
+run/query_data/Saccharolobus_islandicus/M.16.4/reads_R1.fastq.gz: run/genomes/GCF_000022445.1_ASM2244v1_genomic.fna.gz
+run/query_presence_dense/Saccharolobus_islandicus--M.16.4.fasta.gz run/stats/query_presence_dense/Saccharolobus_islandicus--M.16.4.stats: run/query_data/Saccharolobus_islandicus/M.16.4/reads_R1.fastq.gz run/global_index_presence_dense/index.done
+run/query_presence_sparse/Saccharolobus_islandicus--M.16.4.fasta.gz run/stats/query_presence_sparse/Saccharolobus_islandicus--M.16.4.stats: run/query_data/Saccharolobus_islandicus/M.16.4/reads_R1.fastq.gz run/global_index_presence/index.done
+run/stats/verify_query/Saccharolobus_islandicus--M.16.4.stats: run/query_presence_dense/Saccharolobus_islandicus--M.16.4.fasta.gz run/query_presence_sparse/Saccharolobus_islandicus--M.16.4.fasta.gz
@@ -29,7 +29,10 @@ assemblies=(
    GCF_000834255.1
 )

-mkdir -p genomes
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+GENOMES_DIR="${SCRIPT_DIR}/run/genomes"
+
+mkdir -p "${GENOMES_DIR}"

 for acc in "${assemblies[@]}"; do
    echo "Downloading ${acc}"
@@ -41,7 +44,7 @@ for acc in "${assemblies[@]}"; do
    unzip -q "${acc}.zip" -d "${acc}"
    find "${acc}" -name "*.fna" |
        while read file; do
-            obiconvert -Z ${file} >genomes/$(basename ${file}).gz
+            obiconvert -Z ${file} >"${GENOMES_DIR}/$(basename ${file}).gz"
        done

    rm -rf "${acc}" "${acc}.zip"
@@ -12,11 +12,12 @@ set -euo pipefail

 SPECIES="$1"
 SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+RUN_DIR="${SCRIPT_DIR}/run"
 BINARY="${SCRIPT_DIR}/../src/target/release/obikmer"

-SOURCE="${SCRIPT_DIR}/global_index_count"
-OUTPUT="${SCRIPT_DIR}/specific_index_count/${SPECIES}"
-STATS_DIR="${SCRIPT_DIR}/stats/specific_kmer_count"
+SOURCE="${RUN_DIR}/global_index_count"
+OUTPUT="${RUN_DIR}/specific_index_count/${SPECIES}"
+STATS_DIR="${RUN_DIR}/stats/specific_kmer_count"
 STATS_FILE="${STATS_DIR}/${SPECIES}.stats"

 mkdir -p "${STATS_DIR}"
@@ -12,11 +12,12 @@ set -euo pipefail

 SPECIES="$1"
 SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+RUN_DIR="${SCRIPT_DIR}/run"
 BINARY="${SCRIPT_DIR}/../src/target/release/obikmer"

-SOURCE="${SCRIPT_DIR}/global_index_presence"
-OUTPUT="${SCRIPT_DIR}/specific_index_presence/${SPECIES}"
-STATS_DIR="${SCRIPT_DIR}/stats/specific_kmer_presence"
+SOURCE="${RUN_DIR}/global_index_presence"
+OUTPUT="${RUN_DIR}/specific_index_presence/${SPECIES}"
+STATS_DIR="${RUN_DIR}/stats/specific_kmer_presence"
 STATS_FILE="${STATS_DIR}/${SPECIES}.stats"

 mkdir -p "${STATS_DIR}"
@@ -8,14 +8,15 @@ set -euo pipefail

 SPECIMEN="$1"
 SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+RUN_DIR="${SCRIPT_DIR}/run"
 BINARY="${SCRIPT_DIR}/../src/target/release/obikmer"

 species="${SPECIMEN%%--*}"
 strain="${SPECIMEN#*--}"

-READS_DIR="${SCRIPT_DIR}/simulated_data/${species}/${strain}"
-INDEX_PATH="${SCRIPT_DIR}/specimen_index_count/${SPECIMEN}"
-STATS_DIR="${SCRIPT_DIR}/stats/indexing_count"
+READS_DIR="${RUN_DIR}/simulated_data/${species}/${strain}"
+INDEX_PATH="${RUN_DIR}/specimen_index_count/${SPECIMEN}"
+STATS_DIR="${RUN_DIR}/stats/indexing_count"
 STATS_FILE="${STATS_DIR}/${SPECIMEN}.stats"

 mkdir -p "${STATS_DIR}"
@@ -8,14 +8,15 @@ set -euo pipefail

 SPECIMEN="$1"
 SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+RUN_DIR="${SCRIPT_DIR}/run"
 BINARY="${SCRIPT_DIR}/../src/target/release/obikmer"

 species="${SPECIMEN%%--*}"
 strain="${SPECIMEN#*--}"

-READS_DIR="${SCRIPT_DIR}/simulated_data/${species}/${strain}"
-INDEX_PATH="${SCRIPT_DIR}/specimen_index_presence/${SPECIMEN}"
-STATS_DIR="${SCRIPT_DIR}/stats/indexing_presence"
+READS_DIR="${RUN_DIR}/simulated_data/${species}/${strain}"
+INDEX_PATH="${RUN_DIR}/specimen_index_presence/${SPECIMEN}"
+STATS_DIR="${RUN_DIR}/stats/indexing_presence"
 STATS_FILE="${STATS_DIR}/${SPECIMEN}.stats"

 mkdir -p "${STATS_DIR}"
@@ -13,6 +13,10 @@ STOP_WORDS    = {'complete', 'chromosome', 'whole', 'sequence', 'genome',
                 'endosymbiont', 'of'}
 STOP_PREFIXES = ('scaffold', 'contig', 'plasmid')

+# All generated paths live under RUN/ (see Makefile) so the whole tree can be
+# gitignored with a single entry.
+RUN = 'run'
+
 # Specimens used as read sources for the query benchmark (see
 # DevDocMD/implementation/benchmark_query_testing.md): one common bacterium,
 # one distant lineage (the only archaeon in SPECIES).
@@ -80,7 +84,7 @@ def main():
        defn    = first_definition(path)
        sp, st  = parse_organism(defn, gcf_id)
        specimen = f'{sp}--{st}'
-        sim_dir  = f'simulated_data/{sp}/{st}'
+        sim_dir  = f'{RUN}/simulated_data/{sp}/{st}'
        entries.append((specimen, sp, sim_dir, path))
        if sp not in species_seen:
            species_seen.append(sp)
@@ -91,13 +95,13 @@ def main():

    for specimen, species, sim_dir, genome in entries:
        reads = f'{sim_dir}/reads_R1.fastq.gz'
-        p_done  = f'specimen_index_presence/{specimen}/index.done'
-        p_stats = f'stats/indexing_presence/{specimen}.stats'
-        c_done  = f'specimen_index_count/{specimen}/index.done'
-        c_stats = f'stats/indexing_count/{specimen}.stats'
-        ref     = f'reference_index/{specimen}.npz'
-        vp      = f'stats/verify_presence/{specimen}.stats'
-        vc      = f'stats/verify_count/{specimen}.stats'
+        p_done  = f'{RUN}/specimen_index_presence/{specimen}/index.done'
+        p_stats = f'{RUN}/stats/indexing_presence/{specimen}.stats'
+        c_done  = f'{RUN}/specimen_index_count/{specimen}/index.done'
+        c_stats = f'{RUN}/stats/indexing_count/{specimen}.stats'
+        ref     = f'{RUN}/reference_index/{specimen}.npz'
+        vp      = f'{RUN}/stats/verify_presence/{specimen}.stats'
+        vc      = f'{RUN}/stats/verify_count/{specimen}.stats'

        print()
        print(f'# {specimen}')
@@ -110,13 +114,13 @@ def main():

    print()
    for sp in species_seen:
-        sp_done  = f'specific_index_presence/{sp}/index.done'
-        sp_stats = f'stats/specific_kmer_presence/{sp}.stats'
-        sc_done  = f'specific_index_count/{sp}/index.done'
-        sc_stats = f'stats/specific_kmer_count/{sp}.stats'
+        sp_done  = f'{RUN}/specific_index_presence/{sp}/index.done'
+        sp_stats = f'{RUN}/stats/specific_kmer_presence/{sp}.stats'
+        sc_done  = f'{RUN}/specific_index_count/{sp}/index.done'
+        sc_stats = f'{RUN}/stats/specific_kmer_count/{sp}.stats'
        print(f'# {sp}')
-        print(f'{sp_done} {sp_stats}: global_index_presence/index.done')
-        print(f'{sc_done} {sc_stats}: global_index_count/index.done')
+        print(f'{sp_done} {sp_stats}: {RUN}/global_index_presence/index.done')
+        print(f'{sc_done} {sc_stats}: {RUN}/global_index_count/index.done')

    print()
    print('QUERY_SPECIMENS :=', ' '.join(QUERY_SPECIMENS))
@@ -126,17 +130,17 @@ def main():
        _, species, sim_dir, genome = by_specimen[specimen]
        query_dir = sim_dir.replace('simulated_data/', 'query_data/', 1)
        reads      = f'{query_dir}/reads_R1.fastq.gz'
-        dense_out  = f'query_dense/{specimen}.fasta.gz'
-        dense_stat = f'stats/query_dense/{specimen}.stats'
-        sparse_out  = f'query_sparse/{specimen}.fasta.gz'
-        sparse_stat = f'stats/query_sparse/{specimen}.stats'
-        vq_stat     = f'stats/verify_query/{specimen}.stats'
+        dense_out  = f'{RUN}/query_presence_dense/{specimen}.fasta.gz'
+        dense_stat = f'{RUN}/stats/query_presence_dense/{specimen}.stats'
+        sparse_out  = f'{RUN}/query_presence_sparse/{specimen}.fasta.gz'
+        sparse_stat = f'{RUN}/stats/query_presence_sparse/{specimen}.stats'
+        vq_stat     = f'{RUN}/stats/verify_query/{specimen}.stats'

        print()
        print(f'# query: {specimen}')
        print(f'{reads}: {genome}')
-        print(f'{dense_out} {dense_stat}: {reads} global_index_presence/index.done')
-        print(f'{sparse_out} {sparse_stat}: {reads} global_index_presence_sparse/index.done')
+        print(f'{dense_out} {dense_stat}: {reads} {RUN}/global_index_presence_dense/index.done')
+        print(f'{sparse_out} {sparse_stat}: {reads} {RUN}/global_index_presence/index.done')
        print(f'{vq_stat}: {dense_out} {sparse_out}')


@@ -2,10 +2,11 @@
 set -euo pipefail

 SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+RUN_DIR="${SCRIPT_DIR}/run"
 BINARY="${SCRIPT_DIR}/../src/target/release/obikmer"
-IDX_DIR="${SCRIPT_DIR}/specimen_index_count"
-OUTPUT="${SCRIPT_DIR}/global_index_count"
-STATS_DIR="${SCRIPT_DIR}/stats/merge_count"
+IDX_DIR="${RUN_DIR}/specimen_index_count"
+OUTPUT="${RUN_DIR}/global_index_count"
+STATS_DIR="${RUN_DIR}/stats/merge_count"

 mkdir -p "${STATS_DIR}"

@@ -2,10 +2,11 @@
 set -euo pipefail

 SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+RUN_DIR="${SCRIPT_DIR}/run"
 BINARY="${SCRIPT_DIR}/../src/target/release/obikmer"
-IDX_DIR="${SCRIPT_DIR}/specimen_index_presence"
-OUTPUT="${SCRIPT_DIR}/global_index_presence"
-STATS_DIR="${SCRIPT_DIR}/stats/merge_presence"
+IDX_DIR="${RUN_DIR}/specimen_index_presence"
+OUTPUT="${RUN_DIR}/global_index_presence"
+STATS_DIR="${RUN_DIR}/stats/merge_presence"

 mkdir -p "${STATS_DIR}"

@@ -30,10 +30,11 @@ set -euo pipefail

 KIND="$1"
 SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+RUN_DIR="${SCRIPT_DIR}/run"
 BINARY="${SCRIPT_DIR}/../src/target/release/obikmer"
-SOURCE="${SCRIPT_DIR}/global_index_${KIND}"
-OUTPUT="${SCRIPT_DIR}/global_index_${KIND}_dense"
-STATS_DIR="${SCRIPT_DIR}/stats/pack_dense_${KIND}"
+SOURCE="${RUN_DIR}/global_index_${KIND}"
+OUTPUT="${RUN_DIR}/global_index_${KIND}_dense"
+STATS_DIR="${RUN_DIR}/stats/pack_dense_${KIND}"
 STATS_FILE="${STATS_DIR}/current.stats"

 mkdir -p "${STATS_DIR}"
@@ -14,6 +14,7 @@ KIND="$1"
 MODE="$2"
 SPECIMEN="$3"
 SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+RUN_DIR="${SCRIPT_DIR}/run"
 BINARY="${SCRIPT_DIR}/../src/target/release/obikmer"

 case "${KIND}" in
@@ -22,17 +23,17 @@ case "${KIND}" in
 esac

 case "${MODE}" in
-    sparse) INDEX="${SCRIPT_DIR}/global_index_${KIND}" ;;
-    dense)  INDEX="${SCRIPT_DIR}/global_index_${KIND}_dense" ;;
+    sparse) INDEX="${RUN_DIR}/global_index_${KIND}" ;;
+    dense)  INDEX="${RUN_DIR}/global_index_${KIND}_dense" ;;
    *) echo "ERROR: unknown mode '${MODE}' (expected dense|sparse)" >&2; exit 1 ;;
 esac

 species="${SPECIMEN%%--*}"
 strain="${SPECIMEN#*--}"

-READS_DIR="${SCRIPT_DIR}/query_data/${species}/${strain}"
-OUT_DIR="${SCRIPT_DIR}/query_${KIND}_${MODE}"
-STATS_DIR="${SCRIPT_DIR}/stats/query_${KIND}_${MODE}"
+READS_DIR="${RUN_DIR}/query_data/${species}/${strain}"
+OUT_DIR="${RUN_DIR}/query_${KIND}_${MODE}"
+STATS_DIR="${RUN_DIR}/stats/query_${KIND}_${MODE}"
 OUT_FILE="${OUT_DIR}/${SPECIMEN}.fasta.gz"
 STATS_FILE="${STATS_DIR}/${SPECIMEN}.stats"

@@ -4,8 +4,9 @@
 set -euo pipefail

 SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+RUN_DIR="${SCRIPT_DIR}/run"

-for genome_file in "${SCRIPT_DIR}"/genomes/*.fna.gz; do
+for genome_file in "${RUN_DIR}"/genomes/*.fna.gz; do
    out_dir=$("${SCRIPT_DIR}/../.venv/bin/python3" "${SCRIPT_DIR}/make_deps.py" \
        --dir-for "${genome_file}")
    bash "${SCRIPT_DIR}/simulate_one.sh" "${genome_file}" "${out_dir}"
@@ -1,21 +0,0 @@
-genome,Candidozyma_auris--GCF_003013715.1_ASM301371v2,Acidobacterium_capsulatum--ATCC_51196,Bacillus_subtilis--168,Escherichia_coli--CFT073,Escherichia_coli--EDL933,Escherichia_coli--K-12_MG1655,Escherichia_coli--K-12_W3110,Klebsiella_pneumoniae--ATCC_13883,Klebsiella_pneumoniae--HS11286,Klebsiella_pneumoniae--MGH_78578,Opitutus_terrae--PB90-1,Proteus_mirabilis--HI4320,Saccharolobus_islandicus--M.16.4,Salmonella_enterica--AKU_12601,Salmonella_enterica--CT18,Salmonella_enterica--LT2,Salmonella_enterica--P125109,Shouchella_clausii--KSM-K16,Wolbachia_endosymbiont--GCF_000306885.1_ASM30688v1,Yersinia_ruckeri--YRB
-Candidozyma_auris--GCF_003013715.1_ASM301371v2,0.000000,1.000000,1.000000,1.000000,1.000000,1.000000,1.000000,1.000000,1.000000,1.000000,1.000000,1.000000,1.000000,1.000000,1.000000,1.000000,1.000000,1.000000,1.000000,1.000000
-Acidobacterium_capsulatum--ATCC_51196,1.000000,0.000000,0.999981,0.999990,0.999989,0.999987,0.999987,0.999990,0.999988,0.999988,0.999994,0.999989,1.000000,0.999988,0.999987,0.999987,0.999988,0.999989,0.999991,0.999987
-Bacillus_subtilis--168,1.000000,0.999981,0.000000,0.999990,0.999989,0.999989,0.999989,0.999989,0.999988,0.999986,0.999995,0.999985,0.999999,0.999988,0.999987,0.999989,0.999988,0.999778,0.999993,0.999987
-Escherichia_coli--CFT073,1.000000,0.999990,0.999990,0.000000,0.825741,0.807495,0.807218,0.991156,0.996855,0.997849,0.999996,0.999633,1.000000,0.993885,0.996736,0.994148,0.993821,0.999991,0.999984,0.999291
-Escherichia_coli--EDL933,1.000000,0.999989,0.999989,0.825741,0.000000,0.735107,0.734775,0.996126,0.998058,0.997908,0.999997,0.999640,1.000000,0.993993,0.997126,0.994390,0.994059,0.999991,0.999986,0.999292
-Escherichia_coli--K-12_MG1655,1.000000,0.999987,0.999989,0.807495,0.735107,0.000000,0.382567,0.996190,0.997747,0.997455,0.999996,0.999604,1.000000,0.993444,0.996645,0.993773,0.993431,0.999989,0.999984,0.999174
-Escherichia_coli--K-12_W3110,1.000000,0.999987,0.999989,0.807218,0.734775,0.382567,0.000000,0.996220,0.997761,0.997467,0.999995,0.999604,1.000000,0.993445,0.996669,0.993769,0.993443,0.999990,0.999985,0.999165
-Klebsiella_pneumoniae--ATCC_13883,1.000000,0.999990,0.999989,0.991156,0.996126,0.996190,0.996220,0.000000,0.845220,0.840545,0.999997,0.999648,1.000000,0.996177,0.998128,0.996268,0.996052,0.999990,0.999987,0.999325
-Klebsiella_pneumoniae--HS11286,1.000000,0.999988,0.999988,0.996855,0.998058,0.997747,0.997761,0.845220,0.000000,0.906475,0.999996,0.999683,1.000000,0.997724,0.995697,0.997776,0.997769,0.999989,0.999979,0.999463
-Klebsiella_pneumoniae--MGH_78578,1.000000,0.999988,0.999986,0.997849,0.997908,0.997455,0.997467,0.840545,0.906475,0.000000,0.999996,0.999704,1.000000,0.997928,0.995054,0.997844,0.997868,0.999990,0.999980,0.999479
-Opitutus_terrae--PB90-1,1.000000,0.999994,0.999995,0.999996,0.999997,0.999996,0.999995,0.999997,0.999996,0.999996,0.000000,0.999997,0.999998,0.999996,0.999996,0.999996,0.999995,0.999997,0.999993,0.999996
-Proteus_mirabilis--HI4320,1.000000,0.999989,0.999985,0.999633,0.999640,0.999604,0.999604,0.999648,0.999683,0.999704,0.999997,0.000000,1.000000,0.999604,0.999699,0.999622,0.999613,0.999987,0.999983,0.999505
-Saccharolobus_islandicus--M.16.4,1.000000,1.000000,0.999999,1.000000,1.000000,1.000000,1.000000,1.000000,1.000000,1.000000,0.999998,1.000000,0.000000,1.000000,1.000000,1.000000,1.000000,1.000000,1.000000,1.000000
-Salmonella_enterica--AKU_12601,1.000000,0.999988,0.999988,0.993885,0.993993,0.993444,0.993445,0.996177,0.997724,0.997928,0.999996,0.999604,1.000000,0.000000,0.869238,0.682277,0.663383,0.999990,0.999985,0.999260
-Salmonella_enterica--CT18,1.000000,0.999987,0.999987,0.996736,0.997126,0.996645,0.996669,0.998128,0.995697,0.995054,0.999996,0.999699,1.000000,0.869238,0.000000,0.890872,0.886148,0.999988,0.999976,0.999524
-Salmonella_enterica--LT2,1.000000,0.999987,0.999989,0.994148,0.994390,0.993773,0.993769,0.996268,0.997776,0.997844,0.999996,0.999622,1.000000,0.682277,0.890872,0.000000,0.622606,0.999989,0.999985,0.999296
-Salmonella_enterica--P125109,1.000000,0.999988,0.999988,0.993821,0.994059,0.993431,0.993443,0.996052,0.997769,0.997868,0.999995,0.999613,1.000000,0.663383,0.886148,0.622606,0.000000,0.999988,0.999983,0.999270
-Shouchella_clausii--KSM-K16,1.000000,0.999989,0.999778,0.999991,0.999991,0.999989,0.999990,0.999990,0.999989,0.999990,0.999997,0.999987,1.000000,0.999990,0.999988,0.999989,0.999988,0.000000,0.999991,0.999988
-Wolbachia_endosymbiont--GCF_000306885.1_ASM30688v1,1.000000,0.999991,0.999993,0.999984,0.999986,0.999984,0.999985,0.999987,0.999979,0.999980,0.999993,0.999983,1.000000,0.999985,0.999976,0.999985,0.999983,0.999991,0.000000,0.999983
-Yersinia_ruckeri--YRB,1.000000,0.999987,0.999987,0.999291,0.999292,0.999174,0.999165,0.999325,0.999463,0.999479,0.999996,0.999505,1.000000,0.999260,0.999524,0.999296,0.999270,0.999988,0.999983,0.000000
@@ -1 +0,0 @@
-(((((((((((Candidozyma_auris--GCF_003013715.1_ASM301371v2:0.5000001881725941,Saccharolobus_islandicus--M.16.4:0.4999993211600824):0.0000023411501775538747,Opitutus_terrae--PB90-1:0.499997075187947):0.0000029791191795691675,(Acidobacterium_capsulatum--ATCC_51196:0.49999227771334689,(Bacillus_subtilis--168:0.49988797935621456,Shouchella_clausii--KSM-K16:0.49988984146059159):0.0001037210285571577):0.0000023959836053522034):0.0000034093646568700288,Wolbachia_endosymbiont--GCF_000306885.1_ASM30688v1:0.4999920159222422):0.000199555100890203,Proteus_mirabilis--HI4320:0.49979129185300427):0.00010103619067070024,Yersinia_ruckeri--YRB:0.4996806650749249):0.0013719139155004,(Klebsiella_pneumoniae--HS11286:0.43798845051648258,(Klebsiella_pneumoniae--ATCC_13883:0.41780293826821265,Klebsiella_pneumoniae--MGH_78578:0.42274184870836559):0.017586732339732737):0.0604124197073832):0.0006482538063555254,(Salmonella_enterica--CT18:0.43952894448143017,(Salmonella_enterica--AKU_12601:0.3357977326267918,(Salmonella_enterica--LT2:0.31203395843666389,Salmonella_enterica--P125109:0.31057217324861216):0.025729515856701136):0.10292985918524672):0.05825411485542886):0.08937928015651564,Escherichia_coli--CFT073:0.40806501650701029):0.0410131211869626,Escherichia_coli--EDL933:0.3681464750911808):0.1755112579711463,Escherichia_coli--K-12_MG1655:0.19129818036662728,Escherichia_coli--K-12_W3110:0.19126872019906239);
@@ -1,21 +0,0 @@
-genome,Candidozyma_auris--GCF_003013715.1_ASM301371v2,Acidobacterium_capsulatum--ATCC_51196,Bacillus_subtilis--168,Escherichia_coli--CFT073,Escherichia_coli--EDL933,Escherichia_coli--K-12_MG1655,Escherichia_coli--K-12_W3110,Klebsiella_pneumoniae--ATCC_13883,Klebsiella_pneumoniae--HS11286,Klebsiella_pneumoniae--MGH_78578,Opitutus_terrae--PB90-1,Proteus_mirabilis--HI4320,Saccharolobus_islandicus--M.16.4,Salmonella_enterica--AKU_12601,Salmonella_enterica--CT18,Salmonella_enterica--LT2,Salmonella_enterica--P125109,Shouchella_clausii--KSM-K16,Wolbachia_endosymbiont--GCF_000306885.1_ASM30688v1,Yersinia_ruckeri--YRB
-Candidozyma_auris--GCF_003013715.1_ASM301371v2,0,0,0,0,0,0,0,0,0,0,0,0,8,0,1,0,0,0,0,3
-Acidobacterium_capsulatum--ATCC_51196,0,0,203,119,128,141,140,116,109,111,78,112,0,136,109,147,134,117,55,129
-Bacillus_subtilis--168,0,203,0,124,132,128,123,133,109,130,66,158,6,131,112,124,135,2393,46,124
-Escherichia_coli--CFT073,0,119,124,0,1966777,1998059,1999094,117743,32029,22312,63,4225,0,74946,31918,73311,76585,113,128,7854
-Escherichia_coli--EDL933,0,128,132,1966777,0,2627885,2628700,52488,20134,22064,48,4202,0,74655,28602,71244,74665,112,108,7963
-Escherichia_coli--K-12_MG1655,0,141,128,1998059,2627885,0,4452541,48302,21382,24602,47,4277,0,75729,30449,73622,76778,119,111,8566
-Escherichia_coli--K-12_W3110,0,140,123,1999094,2628700,4452541,0,47894,21226,24470,68,4278,0,75658,30207,73614,76583,112,108,8660
-Klebsiella_pneumoniae--ATCC_13883,0,116,133,117743,52488,48302,47894,0,1416091,1477759,42,4172,0,48296,18988,48144,50416,120,106,7712
-Klebsiella_pneumoniae--HS11286,0,109,109,32029,20134,21382,21226,1416091,0,644063,42,2738,0,21498,29758,21606,21376,99,102,4417
-Klebsiella_pneumoniae--MGH_78578,0,111,130,22312,22064,24602,24470,1477759,644063,0,42,2614,0,19948,35067,21330,20813,97,102,4374
-Opitutus_terrae--PB90-1,0,78,66,63,48,47,68,42,42,42,0,43,18,57,42,53,66,39,58,43
-Proteus_mirabilis--HI4320,0,112,158,4225,4202,4277,4278,4172,2738,2614,43,0,0,4254,2481,4166,4215,131,103,4704
-Saccharolobus_islandicus--M.16.4,8,0,6,0,0,0,0,0,0,0,18,0,0,0,0,0,0,0,0,0
-Salmonella_enterica--AKU_12601,0,136,131,74946,74655,75729,75658,48296,21498,19948,57,4254,0,0,1047731,2857146,2951421,117,108,7643
-Salmonella_enterica--CT18,1,109,112,31918,28602,30449,30207,18988,29758,35067,42,2481,0,1047731,0,917948,940297,106,106,3716
-Salmonella_enterica--LT2,0,147,124,73311,71244,73622,73614,48144,21606,21330,53,4166,0,2857146,917948,0,3284800,122,108,7460
-Salmonella_enterica--P125109,0,134,135,76585,74665,76778,76583,50416,21376,20813,66,4215,0,2951421,940297,3284800,0,134,124,7645
-Shouchella_clausii--KSM-K16,0,117,2393,113,112,119,112,120,99,97,39,131,0,117,106,122,134,0,58,124
-Wolbachia_endosymbiont--GCF_000306885.1_ASM30688v1,0,55,46,128,108,111,108,106,102,102,58,103,0,108,106,108,124,58,0,96
-Yersinia_ruckeri--YRB,3,129,124,7854,7963,8566,8660,7712,4417,4374,43,4704,0,7643,3716,7460,7645,124,96,0
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
coissac	755cf85334	Merge pull request 'Push twnyxtspltux' (#76 ) from push-twnyxtspltux into main Reviewed-on: #76	2026-09-12 05:57:25 +00:00
Eric Coissac	2693a6af09	Implement session persistence and checkpointing Release / create-release (push) Successful in 2m32s Details Release / build-linux-x86_64 (push) Successful in 8m21s Details Release / build-macos-arm64 (push) Successful in 2m1s Details ci.yml / build (pull_request) Successful in 3m45s Details Introduces session persistence via the `--session` option, allowing sampled sets to be restored. Implements chunked dumps and shared checkpointing for alignment and tally artifacts, ensuring state restoration upon interruption. Refines sampling logic and introduces serialization mechanisms for state management.	2026-09-12 07:48:51 +02:00
Eric Coissac	7861e886e7	Introduce session persistence and artifact caching This change implements a robust session management system allowing users to persist distance calculation samples and parameters across invocations. It includes concurrency-safe directory locking, deterministic parameter encoding, artifact integrity checks, and caching mechanisms to skip resampling when valid data is available.	2026-09-12 07:46:55 +02:00
Eric Coissac	a8bcf5ffac	Refactor distance matrix calculation logic for consistency Ensures that combined distance matrix calculations use the exact same site selection by retaining sampled data within Sankoff bundles. This involves refactoring post-sampling logic into shared functions, implementing `SankoffBundle` to reuse internal tally data, and adding validation to guarantee consistent site selection across all distance calculation paths.	2026-09-12 07:45:32 +02:00
Eric Coissac	be838da256	Refine theoretical justification for alpha estimation. Document limitations of various $\alpha$-estimation methods, clarifying that tree-based methods require a tree for accurate heterogeneity estimation and that `PartitionDispersion` avoids violating identifiability requirements.	2026-09-12 07:44:51 +02:00
Eric Coissac	020b391636	Implement gamma shape correction for SNP distance calculation Introduces support for rate heterogeneity via a Poisson-Gamma mixture model. This includes new command-line options (`--gamma-shape`, `--gamma-shape auto`) to enable automatic estimation of the shape parameter $\alpha$ based on substitution counts across partitions. The correction is applied to the distance metric, with logic to disable the correction if variance checks fail.	2026-09-12 07:44:39 +02:00
coissac	e846d35adb	Merge pull request 'Feat: Introduce resource profiling and monitoring APIs' (#75 ) from push-sktolxuyskpp into main Reviewed-on: #75	2026-09-11 07:51:03 +00:00
Eric Coissac	c0e1968654	Feat: Introduce resource profiling and monitoring APIs Release / create-release (push) Successful in 2m27s Details ci.yml / build (pull_request) Successful in 3m52s Details Release / build-linux-x86_64 (push) Successful in 7m50s Details Release / build-macos-arm64 (push) Successful in 2m0s Details This change introduces a comprehensive profiling system, allowing for periodic resource usage monitoring (CPU, I/O, memory) via a background thread. It also exposes new public APIs for budget management, locking, NUMA partitioning, and progress tracking.	2026-09-11 09:50:09 +02:00
coissac	e280b6d0a3	Merge pull request 'Implement NUMA topology detection and CPU budget capping' (#74 ) from push-tnkxqmwztnmm into main Reviewed-on: #74	2026-09-11 05:23:42 +00:00
Eric Coissac	98dba1802d	Implement NUMA topology detection and CPU budget capping Release / create-release (push) Successful in 2m30s Details ci.yml / build (pull_request) Successful in 3m50s Details Release / build-linux-x86_64 (push) Successful in 8m23s Details Release / build-macos-arm64 (push) Successful in 1m57s Details This change introduces logic to detect NUMA topology, calculate per-node worker limits based on available CPU cores, and cap resource allocation to ensure worker counts respect the physical capacity of each node.	2026-09-11 07:22:35 +02:00
coissac	dd4285b269	Merge pull request 'Introduce CPU budget and dynamic thread limiting' (#73 ) from push-lztkokoxqpuw into main Reviewed-on: #73	2026-09-10 15:10:19 +00:00
Eric Coissac	b66a488277	Introduce CPU budget and dynamic thread limiting Release / create-release (push) Successful in 3m4s Details ci.yml / build (pull_request) Successful in 4m17s Details Release / build-linux-x86_64 (push) Successful in 8m11s Details Release / build-macos-arm64 (push) Successful in 2m2s Details This change implements mechanisms to enforce resource limits based on the available CPU budget, including setting a hard CPU ceiling and calculating effective thread counts for CLI, query, and superkmer operations.	2026-09-10 17:08:32 +02:00
Eric Coissac	b954032faa	chore: remove obsolete configuration and memory files Deletes unused tool configurations, planning plans, and local memory files across multiple directories.	2026-08-31 09:09:40 +02:00
coissac	43d89d8f02	Merge pull request 'disable default features for obisys dependency' (#72 ) from push-wvkttrwzkqoo into main Reviewed-on: #72	2026-08-29 06:18:11 +00:00
Eric Coissac	8bb1d52514	disable default features for obisys dependency Release / create-release (push) Successful in 2m32s Details Release / build-linux-x86_64 (push) Successful in 7m57s Details Release / build-macos-arm64 (push) Successful in 1m53s Details ci.yml / build (pull_request) Successful in 3m42s Details Update obisys dependency configuration across multiple crates to explicitly set default-features = false. This restricts compilation to only explicitly enabled feature flags, affecting build resolution and potentially compile-time paths or binary size. Also updates the obikmer package version to 1.3.1.	2026-08-29 08:13:49 +02:00
coissac	77d4429645	Merge pull request 'chore: bump obikmer to 1.3.0 and disable obisys default features' (#71 ) from push-sllyqzuztwkr into main Reviewed-on: #71	2026-08-29 05:51:37 +00:00
Eric Coissac	4d55258ada	chore: bump obikmer to 1.3.0 and disable obisys default features Release / create-release (push) Successful in 2m32s Details Release / build-linux-x86_64 (push) Successful in 8m22s Details Release / build-macos-arm64 (push) Failing after 1m15s Details ci.yml / build (pull_request) Successful in 3m33s Details Updates package version and explicitly disables default features for the local obisys dependency.	2026-08-29 07:46:56 +02:00
coissac	7e28ca4853	Merge pull request 'Push zunrplorkwkt' (#70 ) from push-zunrplorkwkt into main Reviewed-on: #70	2026-08-28 23:15:37 +00:00
Eric Coissac	52f26809af	refactor(benchmark): consolidate generated artifacts under run/ directory ci.yml / build (pull_request) Successful in 3m49s Details Restructure the benchmark pipeline to direct all simulated data, indices, statistics, and query outputs into a unified `run/` directory. Update Makefile targets, shell scripts, and Python utilities to resolve paths relative to this new base. Adjust documentation and dependency tracking to match the revised layout, and remove outdated temporary artifacts.	2026-08-29 00:19:31 +02:00
				`@@ -1 +0,0 @@`
				(((((((((((Candidozyma_auris--GCF_003013715.1_ASM301371v2:0.5000001881725941,Saccharolobus_islandicus--M.16.4:0.4999993211600824):0.0000023411501775538747,Opitutus_terrae--PB90-1:0.499997075187947):0.0000029791191795691675,(Acidobacterium_capsulatum--ATCC_51196:0.49999227771334689,(Bacillus_subtilis--168:0.49988797935621456,Shouchella_clausii--KSM-K16:0.49988984146059159):0.0001037210285571577):0.0000023959836053522034):0.0000034093646568700288,Wolbachia_endosymbiont--GCF_000306885.1_ASM30688v1:0.4999920159222422):0.000199555100890203,Proteus_mirabilis--HI4320:0.49979129185300427):0.00010103619067070024,Yersinia_ruckeri--YRB:0.4996806650749249):0.0013719139155004,(Klebsiella_pneumoniae--HS11286:0.43798845051648258,(Klebsiella_pneumoniae--ATCC_13883:0.41780293826821265,Klebsiella_pneumoniae--MGH_78578:0.42274184870836559):0.017586732339732737):0.0604124197073832):0.0006482538063555254,(Salmonella_enterica--CT18:0.43952894448143017,(Salmonella_enterica--AKU_12601:0.3357977326267918,(Salmonella_enterica--LT2:0.31203395843666389,Salmonella_enterica--P125109:0.31057217324861216):0.025729515856701136):0.10292985918524672):0.05825411485542886):0.08937928015651564,Escherichia_coli--CFT073:0.40806501650701029):0.0410131211869626,Escherichia_coli--EDL933:0.3681464750911808):0.1755112579711463,Escherichia_coli--K-12_MG1655:0.19129818036662728,Escherichia_coli--K-12_W3110:0.19126872019906239);