Push wpvskkyozppn #78

Merged
coissac merged 15 commits from push-wpvskkyozppn into main 2026-10-15 10:38:59 +00:00
10 changed files with 520 additions and 30 deletions
Showing only changes of commit fe38424682 - Show all commits

No files matched your search

@@ -0,0 +1,179 @@
# MPHF batch queries — site inventory and expected gains
Investigation of 2026-10-14 (`ptr_hash` 2.1.2, `epserde` 0.13). Question: where would batched MPHF queries (software prefetch of the random accesses) save time? Status of each claim is marked **[read]** (verified in source), **[computed]** (derived from stated sizes) or **[estimate]** (reasoning, not measured). Measurements, once made, are in the last section.
## 1. What `ptr_hash` 2.1.2 actually offers [read]
`Mphf = PtrHash<u64, CubicEps, F = CachelineEfVec, Xxh3Int, V = Vec<u8>, SINGLE_PART = false, REMAP = true>`.
- **`V` is the pilot array** (one byte per bucket), **`F` is only the remap storage**. A layer's `mphf.bin` (~2.4 bits/key **[computed]**: 40 MB for ~130 M keys) is therefore almost entirely a byte array indexed by bucket. The batch APIs prefetch `pilots[bucket]` — the right line. The remap (read only when `slot >= n`, a small fraction of keys) is **not** prefetched.
- `index_batch<K>(xs: [Q; K]) -> [usize; K]` — stable, not feature-gated; hash K keys, prefetch K pilots, query. Remainder (`len % K`) is the caller's job.
- `index_stream<B>(keys: IntoIterator) -> impl Iterator` — ring buffer of B keys; handles head and tail itself; accepts any iterator. **Its `next()` is `unimplemented!`**: only fold-based consumption works (`for_each`, `fold`, `sum`). `collect()`, `zip`, `for x in it` panic. Consume with `for_each` into a pre-sized `Vec`.
- `index_batch_exact` needs the nightly `unstable` feature; `index_batch_exact2` is `#[doc(hidden)]`. Neither is usable.
- Prefetch is `prefetch-index 0.2.1`: `_mm_prefetch(T0)` on x86_64, asm on aarch64, **no-op elsewhere**.
- **`SINGLE_PART`**: neither `index_stream` nor `index_batch` branches on it — both always run the multi-part `bucket()`/`slot()`. Only `index()` has the faster single-part path. So with batched queries the `SINGLE_PART = true` advantage disappears, and `false` (our choice) costs nothing while keeping parallel construction (`true` also panics with sharding).
- Old `ptr_hash` 1.1.0 (used by `tools/migrate-mphf`) has the same family with an extra `MINIMAL` const: `index_stream<B, MINIMAL, Q>`, `index_batch<K, MINIMAL, Q>`. No new dependency needed there.
```rust
// consuming index_stream: fold-based only
let mut out = Vec::with_capacity(kmers.len());
mphf.index_stream::<32, _>(kmers.iter().map(|k| k.raw()))
.for_each(|slot| out.push(slot));
```
## 2. Existing "batch" APIs that are not batched [read]
`MphfLayer::hash_batch` and `MphfOnly::index_batch` loop `.index()` one kmer at a time. Consumers: `obikfilter/partition_iter.rs` (`iter_layer_kmers`), `obikmerge/partition_merge.rs` (pass 2: `hash_batch`, `index_batch`), `obikphylo/siblings/algorithms/family_scan.rs` (two sites).
## 3. Cost chain of a membership query (`MphfLayer::find`) [read]
1. `pilots[bucket]` — the only access `ptr_hash` prefetches.
2. Exact/Hybrid-exact: `evidence.decode(slot)` — 4 B/slot, ~520 MB per 130 M-slot layer **[computed]**, random mmap read.
3. `verify_canonical_kmer` → `block_offsets[...]` (O(1) with the default `--block-size 1`; a forward scan of up to `2^block_bits` records otherwise).
4. `extract_kmer_raw` — random read in the `unitigs.bin` mmap.
Approx/Hybrid-fast: pilots then `fingerprint.get(slot)` — one extra random access.
Each stage is independent across kmers, so a staged batch (slots → prefetch evidence → decode, prefetch offsets/unitigs → verify) can overlap all of them. `ptr_hash` only helps with stage 1; stages 2–4 need prefetch support in `obikindex`/`obiskio` (not existing). Prefetch on an unmapped (cold, not paged-in) mmap page is dropped by the CPU, so none of this helps on a cold index.
Layer sizes: `layer_0` MPHF 2–3 MB (cache-resident), `layer_1` ~40 MB (beyond one CCD's L3 on EPYC-class machines — **hypothesis**, prod CPU not verified, and contended by 192 threads). Gains are expected mainly on `layer_1`. For branch-free independent loops (`hash_batch`) out-of-order execution already overlaps several misses, so the explicit prefetch gains less **[estimate]** than for branchy loops (`find` with early `break`).
## 4. Call-site inventory
(a) already batched; (b) batch buildable; (c) hard/pointless.
| Site | What | Class |
|---|---|---|
| `mphf_layer.rs` `hash_batch`, `MphfOnly::index_batch` | loop of `.index()` | (a) |
| `obikfilter/partition_iter.rs:106` | `hash_batch`, 32768 kmers | (a) |
| `obikmerge/partition_merge.rs:489, 521` | `hash_batch`, `index_batch`, 4096 | (a) |
| `obikphylo/.../family_scan.rs:285, 397` | `hash_batch` | (a) |
| `obikquery/query_layer.rs` stage 1 | `find_in_layer` per kmer, layer by layer, `break` on hit | (b) |
| `partition_merge.rs` `find_batch_grouped` | `find_slot` per kmer over dst layers | (b) |
| `partition_merge.rs` (`build_graph` filter) | `all(find_slot.is_none())`, every layer, per kmer; `build_graph` takes `Fn(CanonicalKmer)->bool` | (b), API change |
| `family_scan.rs` slow path; `annex.rs` | `find_slot` / `find_in_partition` per variant | (b) |
| `mphf_layer.rs` `build`, `build_exact_evidence`, `build_approx_evidence` | `mphf.index` then random writes (evidence, fingerprint, `seen`) | (b), chunked |
| `obikindexer/counter/count.rs:47` | `mphf.index` then `counts.get/set` | (b), hard (needs (kmer, count) pairs) |
| `obikindexer/extensions/private_builder.rs:177` | abundance filter | (b), conditional |
| `private_builder.rs:196`, `verify.rs`, `IndexCache::find_in_*` | per-kmer by nature / sampled | (c) |
| `tools/migrate-mphf/plan.rs:41-42` | `old.index` + `new.index` per kmer, random writes | (b) |
Dead or suspect code found along the way (candidates for removal, not removed):
- `MphfLayer::find_strict` — no caller.
- the single-kmer `hash` chain (`MphfLayer::hash` → `TypedLayer::hash` → `KmerLayer::hash` → `IndexCache::hash`) — no caller outside the chain.
- `IndexCache::find` — no caller.
- `obikmerge/src/merge_layer/` — not declared in `lib.rs` (`mod merge; mod partition_merge;` only); still imports the `LayeredMap` removed in 2026-08. To be confirmed by the compiler.
## 5. Interactions
- **Verification chain**: the right shape is one `find_batch(&[CanonicalKmer], &mut Vec<Option<usize>>)` on `MphfLayer`, staged as in §3, so every consumer benefits (rule: enrich the shared API, no local loops).
- **Memory**: callers already hold bounded batches (4096 / 32768). A buffer-reusing variant avoids one `Vec<usize>` per call. `index_stream` itself only allocates a B-element ring on the stack.
- **Parallelism**: no new parallelism. Batches already come out of `obipipeline` stages; the change lives in the transform body. Construction pass 2 is sequential inside a partition: chunk the streamed iterator (1–4 k kmers), never collect the layer. In `query_partition_with` the unique-kmer `HashMap` is already materialised; iterate it chunk-wise.
## 6. Benchmark
`benchmark/bench_mphf.sh LABEL` times, on the 20-genome benchmark data, `index` (2 genomes), `merge` (20 presence indexes), `query` (presence/count × 2 genomes), `filter`, `convert` (approx/exact round trip) and `phylo` (`--sibling-annex`, `snp-jc`), `REPS` times each, and records order-independent output digests that must not change across labels. See the results section below.
**Limit of this data set**: 20 small genomes → per-partition `layer_1` MPHFs of ~100 KB, fully cache-resident on the dev machine (Apple M3 Max, aarch64). This benchmark can detect regressions and the effect on instruction-level overlap; it **cannot** measure the DRAM-latency gain that motivates prefetch at 40 MB/layer. A faithful test needs a prod-scale layer (see the throwaway benchmark below).
### Throwaway prod-scale benchmark (not written)
Open one real partition's `layer_1` (~130 M keys) and `layer_0` (~8 M) with `MphfOnly`/`MphfLayer`; key sets: present (sample of `iter_kmers`, random order and sequential order) and absent (random, checked absent); measure kmers/s for `index`, `index_stream::<B>` (B ∈ 8…128), `index_batch::<K>` (K ∈ 8…64), and the full `find` vs staged `find_batch`; at 1, 8, 32, 192 threads; pinned to one NUMA node, then spread; warm and cold page cache stated; 5 repetitions, min/median; sizes of `mphf.bin`, `evidence.bin`, `unitigs.bin(.idx)` reported; first check `index_stream` slots == `index` slots. The ~8–12 ns/key figure of the PtrHash paper (quoted in `mphf.md`) is a paper claim, not tied to our sizes or machines.
## 7. Not verified
No measurement at prod scale; prod CPU architecture and cache sizes; evidence mode and `--block-size` used in prod; whether `migrate-mphf` already ran; share of total index/merge/query time spent in these loops; `count.rs` f0 per partition at prod scale; whether `merge_layer/` is dead (only the head of `lib.rs` read).
## 8. First improvement and measurements (2026-10-15)
**Implemented** (query stage 1, the top-ranked site; the same change adds the plumbing the other sites will reuse):
- `MphfLayer` now holds a `MphfOnly` instead of its own `MemCase<Mphf>` (one mmap-open path, one place for the stream call).
- `MphfOnly::for_each_slot(kmers, f)` wraps `index_stream::<32>`; `index_batch`/`hash_batch` now really use it (they previously looped `.index()`). Callback-shaped because `index_stream`'s `next()` is `unimplemented!`.
- `MphfLayer::find_batch` / `TypedLayer::find_slot_batch` / `KmerLayer::find_slot_batch`; the mode-dependent half of `find` is factored into `MphfLayer::confirm`, shared by both.
- `query_partition_with` stage 1 is layer-major over chunks of 4096 unique kmers: each layer receives the still-unresolved kmers as one batch (first hit wins, `n_mphf_calls` unchanged).
- Not done: staged prefetch of evidence/offsets/unitigs (needs a prefetch primitive; `prefetch-index` is already in the tree via `ptr_hash`, but adding it to `obikindex` is a dependency decision), and the other (b) sites.
**Measurements** (`benchmark/bench_mphf.sh`, `benchmark/summarize_bench.py`; Apple M3 Max, aarch64; 20 genomes at 15x). Interleaved A/B, 8 runs each, E. coli reads queried against the 20-genome presence index:
| | wall (s) | user CPU (s) |
|---|---|---|
| before | 5.94–6.02 (median 5.96) | 10.95–11.07 |
| after | 5.63–6.00 (median 5.67) | 10.33–10.66 |
About −5 % wall and −5.5 % CPU, ranges non-overlapping **[measured]**. All output digests (query ×4, merge, filter, convert round trip, phylo) are identical before and after.
**Reading the number.** Small by construction: this data set's `layer_1` MPHFs are ~100 KB per partition, cache-resident, and query time is dominated by reading and k-merising the reads. The gain seen here is instruction-level overlap, not DRAM-latency hiding. The latter, the reason for the whole exercise, remains **unmeasured** and needs a prod-scale layer (§6).
**Noise caveat.** With a load average of 16–22 from unrelated processes, single-run walls of `index`, `merge` and `filter` moved ±10–40 % between identical binaries; only interleaved repeated A/B runs were trustworthy. The `index`/`merge`/`filter`/`convert`/`phylo` rows of the first comparison are noise, not effects (`index` pass 2 was not changed at all; a later interleaved `filter` A/B was inconclusive, 13.0 s vs 13.8 s medians with a 10–20 s spread).
**Stale benchmark indexes.** The indexes already under `benchmark/run/` (`specimen_index_*`, `global_index_*`, built 2026-08-28) carry `ptr_hash` 1.x MPHFs and fail with "Major version mismatch" on current binaries (`tools/migrate-mphf` converts them). `bench_mphf.sh` therefore builds its own copies under `run/bench_mphf/data/` on first use.
## 9. Benchmark procedure
Prerequisites: the simulated reads (`benchmark/run/simulated_data/`) and query reads (`benchmark/run/query_data/`) produced by `gmake simulate simulate_query`; macOS BSD `time -l` and `md5 -q` (the script uses both), Python 3.
1. Build the binary to measure and **copy it** under a name that identifies it (cargo overwrites `target/release/obikmer`): `cp src/target/release/obikmer benchmark/run/bench_mphf/obikmer_<label>`.
2. `BINARY=$PWD/run/bench_mphf/obikmer_<label> benchmark/bench_mphf.sh <label>` (from `benchmark/`; `REPS=3` by default, `CPU_MAX=n` to restrict cores). The first call builds the shared data set in `run/bench_mphf/data/` with that binary (≈ 2 min); keep it for every label — the on-disk format must not differ between the binaries compared, otherwise delete `data/` and rebuild.
3. Output: `run/bench_mphf/<label>/results.csv` (wall and peak RSS per step and repetition, plus one row `step:stage` per Reporter stage) and `checksums.txt`.
4. Compare: `python3 benchmark/summarize_bench.py <ref_label> <new_label>` (medians and ratio) and `diff run/bench_mphf/<ref>/checksums.txt run/bench_mphf/<new>/checksums.txt` — **any difference is a bug**, not a performance matter.
5. A ratio is only a candidate effect. Check the spread by running the **same** binary twice under two labels (the script's own noise floor was ±10–40 % per step on a loaded laptop), or confirm with interleaved A/B runs of the single command (alternate the two binaries, ≥ 8 runs, compare ranges of wall and user CPU from `/usr/bin/time -l`).
6. Rule of thumb for this data set: only `query` (≈ 5 s, stable to ±1 %) resolves effects of a few percent; `index`, `merge`, `filter`, `phylo` need interleaved runs or a quiet machine.
Step → code exercised:
| Step | Code on the MPHF path |
|---|---|
| `index_*` (`:index` stage) | `MphfLayer::build` pass 2 |
| `merge_20_presence` | `find_batch_grouped`, `build_graph` filter, `hash_batch`, `index_batch` |
| `query_*` | `query_partition_with` stage 1 |
| `filter_species` | `iter_layer_kmers` → `hash_batch` |
| `convert_to_*` | `build_exact_evidence`, `build_approx_evidence` |
| `phylo_sibling_annex` | `annex.rs` `find_in_partition` per variant |
| `phylo_snp_distance` | `family_scan.rs` `hash_batch`, `find_slot` |
## 10. Second step — construction pass 2 (2026-10-15)
**Implemented**: `layer/slotted.rs`, `SlottedKmers`, an iterator adapter yielding `(slot, kmer, chunk_id, rank)` for the wrapped `(kmer, chunk_id, rank)` stream, computing slots 4096 kmers at a time through `index_stream::<32>` (bounded memory, never the whole layer). The five loops of `MphfLayer::build` (Exact, Approx, Hybrid), `build_exact_evidence` and `build_approx_evidence` now iterate it instead of calling `mphf.index` per kmer. Output digests unchanged.
**Measured** (interleaved A/B against the first-step binary, 8 runs each, wall / user CPU):
| Command | before | after |
|---|---|---|
| `convert --exact-evidence` (20-genome index) | 0.56–0.68 s / 2.22–2.67 s | 0.59–0.74 s / 2.25–2.52 s |
| `index` E. coli reads | 2.45–2.63 s / 13.1–14.3 s | 2.48–2.75 s / 13.2–14.0 s |
**No measurable effect** on this data set (differences within the run-to-run spread; if anything `convert` is ≈ 4 % slower in wall time). Expected here: the MPHF is cache-resident and `index`/`convert` time is dominated by other stages (scatter, dereplication, counting, unitig decoding, the random evidence/`seen` writes themselves). The construction gain, if any, only exists where the pilot array does not fit in cache, and the random *writes* to `evidence.bin`/`seen`/`fingerprint.bin` (not prefetched by `ptr_hash`) probably dominate there too. Whether to keep this change is therefore a decision to take after the prod-scale benchmark (§6), not on this data.
## 11. Why the two changes gain so little (profiling, 2026-10-15)
Measured with macOS `sample` (1 ms stack sampling) on the 20-genome data, one run each — coarse, but the orders of magnitude are unambiguous.
**Query.** In a 2 s window on the `query` stage (16 threads, ≈ 3 100 active samples against ≈ 55 000 samples of idle waiting):
- `query_partition_with` ≈ 260 samples (8 % of active CPU), of which `find_batch` 217, of which **`verify_canonical_kmer` 185** and everything else in `find_batch` (the `ptr_hash` stream and `confirm`) **32**.
- `verify_canonical_kmer` is the single largest active function of the window (399 top-of-stack samples). The MPHF lookup is ≈ 1 % of active CPU: even a free MPHF could not give more.
- Reason: per-partition pilot arrays are ~100 KB (≈ 25 MB over 256 partitions) and stay cached; the evidence/`unitigs.bin`/offset accesses of the verification (§3, stages 2–4) are the ones that miss. This data set already has the problem on the verification side, not on the MPHF side.
- The threads are mostly idle: the stage is bounded by reading/parsing/k-merising the reads (`rolling_stat`, entropy tracker, string formatting) and by sequential parts, not by lookups.
**What produced the −5 %.** Interleaved A/B, 8 runs each, median wall: original 6.4 s; chunked layer-major restructuring with plain per-kmer `index` (stream disabled) 6.0 s; restructuring + `index_stream` 5.85 s. **Most of the gain comes from the restructuring, not from the prefetch** (the prefetch adds ≈ 2 %, at the limit of the noise). Hypothesis, not isolated: the old loop called `IndexCache::get_layer` — an `Arc<KmerLayer>` clone, i.e. an atomic increment/decrement on a refcount shared by all threads — once per kmer per layer; the batched loop does it once per layer per chunk of 4096.
**Index (pass 2).** In a 1 s window late in the `index` stage, `SlottedKmers` appears with ≈ 7 samples out of 304 per thread (≈ 2 %), against ≈ 200 for the de Bruijn graph work (`obidebruinj`) and ≈ 35 for `ptr_hash::build` (pass 1, MPHF *construction*). Pass 2 is a few percent of `index` and `convert` here: nothing to win even if its lookups were free. `convert` is `build_*_evidence` on 2 MB-scale layers, same conclusion.
**Consequences for priorities.**
1. On this data the next lever is the verification (`verify_canonical_kmer`, evidence decode), not the MPHF — consistent with §3, now observed.
2. Hot `Arc` clones in per-kmer paths (`get_layer`, `IndexCache::find_in_layer`, `find_slot` loops in merge and phylo) are worth auditing independently of MPHF batching.
3. The DRAM-latency gain of prefetch is untested until a prod-scale layer is available (§6).
## 12. Calibration: a single skim at fewer partitions (2026-10-15)
`Vaccinium myrtillus` skim (2 × ~440 MB fastq.gz, `theta 0`), indexed with the current binary, to find the partition count that gives prod-sized MPHFs (`--partitions` is a power of two; MPHF size ≈ 2.4 bits/key **[measured]**):
| `-p` | keys / partition | `mphf.bin` / partition | index wall | peak RSS |
|---|---|---|---|---|
| 256 (default) | ≈ 1.4 M **[computed from p=16]** | ≈ 0.4 MB **[computed]** | – | – |
| 16 | 22.1 M | 6.65 MB | 120 s | 12.3 GB |
| 4 | 86–90 M | 25.9–27.0 MB | 171 s | 19.5 GB |
Total ≈ 354 M distinct kmers for one skim. `-p 4` is already in the 30–45 MB range of the prod `layer_1` MPHFs; `-p 2` would give ≈ 53 MB. Indexes kept in `benchmark/run/bench_mphf/skim_p16` and `skim_p4` (2.3 GB each). Note the index stage itself is dominated by graph and MPHF construction, as in §11.
+152
View File
@@ -0,0 +1,152 @@
#!/usr/bin/env bash
# Usage: bench_mphf.sh LABEL
#
# Measures the obikmer commands that sit on MPHF lookups (see
# DevDocMD/implementation/mphf_batch_queries.md for the site inventory), on the
# 20-genome benchmark data already built under run/ (`gmake all` first).
# Nothing under run/ other than run/bench_mphf/ is written.
#
# Env:
# BINARY obikmer binary to measure (default ../src/target/release/obikmer)
# REPS repetitions per step (default 3)
# CPU_MAX passed as --cpu-max when set (default: all cores)
#
# Output: run/bench_mphf/LABEL/results.csv (label,step,rep,wall_s,maxrss_b)
# run/bench_mphf/LABEL/checksums.txt (order-independent output digests,
# to be diffed across LABELs: a speed-up must not change any of them)
set -euo pipefail
LABEL="$1"
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
RUN_DIR="${SCRIPT_DIR}/run"
BINARY="${BINARY:-${SCRIPT_DIR}/../src/target/release/obikmer}"
REPS="${REPS:-3}"
OUT="${RUN_DIR}/bench_mphf/${LABEL}"
CPU=()
[[ -n "${CPU_MAX:-}" ]] && CPU=(--cpu-max "${CPU_MAX}")
rm -rf "${OUT}"; mkdir -p "${OUT}"
RESULTS="${OUT}/results.csv"
SUMS="${OUT}/checksums.txt"
echo "label,step,rep,wall_s,maxrss_b" >"${RESULTS}"
: >"${SUMS}"
# timed STEP REP CMD... — wall time + peak RSS via BSD time(1); stdout of CMD
# goes to $OUT/last.stdout, stderr to $OUT/last.stderr.
timed() {
local step="$1" rep="$2"; shift 2
local tf="${OUT}/time.tmp"
/usr/bin/time -l -o "${tf}" "$@" >"${OUT}/last.stdout" 2>"${OUT}/last.stderr" \
|| { echo "FAILED: $step: $*" >&2; cat "${OUT}/last.stderr" >&2; exit 1; }
local wall rss
wall=$(awk '/ real /{print $1}' "${tf}")
rss=$(awk '/maximum resident set size/{print $1}' "${tf}")
echo "${LABEL},${step},${rep},${wall},${rss}" >>"${RESULTS}"
# per-stage wall times from the Reporter log lines ("done stage=X wall_secs=Y")
sed $'s/\x1b\\[[0-9;]*m//g' "${OUT}/last.stderr" \
| awk -v L="${LABEL}" -v S="${step}" -v R="${rep}" \
'/ done stage=/{ for(i=1;i<=NF;i++){ if($i ~ /^stage=/) st=substr($i,7); if($i ~ /^wall_secs=/) w=substr($i,11) } print L","S":"st","R","w",0" }' \
>>"${RESULTS}"
printf ' %-28s rep %s %8s s\n' "${step}" "${rep}" "${wall}" >&2
}
digest() { LC_ALL=C sort "$1" | md5 -q; }
# ── 0. data: the indexes under run/ may have been built by an older
# MPHF generation (ptr_hash 1.x, unreadable by current binaries), so the
# bench builds its own, once, from the simulated reads. Shared by all LABELs;
# delete run/bench_mphf/data to force a rebuild.
DATA="${RUN_DIR}/bench_mphf/data"
if [[ ! -f "${DATA}/.ready" ]]; then
echo "preparing data with ${BINARY}" >&2
mkdir -p "${DATA}/presence" "${DATA}/count"
for d in "${RUN_DIR}"/simulated_data/*/*/; do
strain="$(basename "$d")"; species="$(basename "$(dirname "$d")")"
sp="${species}--${strain}"
for kind in presence count; do
extra=(); [[ $kind == count ]] && extra=(--with-counts)
"${BINARY}" "${CPU[@]}" index --output "${DATA}/${kind}/${sp}" --force \
--theta 0 "${extra[@]}" --label "${sp}" --meta "species=${species}" \
"$d/reads_R1.fastq.gz" "$d/reads_R2.fastq.gz" 2>/dev/null
done
done
"${BINARY}" "${CPU[@]}" merge --output "${DATA}/global_presence" --force \
--force-presence "${DATA}"/presence/*/ 2>/dev/null
"${BINARY}" "${CPU[@]}" merge --output "${DATA}/global_count" --force \
"${DATA}"/count/*/ 2>/dev/null
touch "${DATA}/.ready"
fi
sources=("${DATA}"/presence/*/)
# ── 1. index: build pass 2 (mphf.index per kmer + evidence writes) ──────────
for sp in Escherichia_coli--K-12_MG1655 Candidozyma_auris--GCF_003013715.1_ASM301371v2; do
species="${sp%%--*}"; strain="${sp#*--}"
reads="${RUN_DIR}/simulated_data/${species}/${strain}"
for rep in $(seq "${REPS}"); do
timed "index_${species}" "${rep}" "${BINARY}" "${CPU[@]}" index \
--output "${OUT}/idx_${species}" --force --theta 0 --label "${sp}" \
"${reads}/reads_R1.fastq.gz" "${reads}/reads_R2.fastq.gz"
done
done
# ── 2. merge: find_batch_grouped, build_graph filter, hash_batch, index_batch ─
for rep in $(seq "${REPS}"); do
timed "merge_20_presence" "${rep}" "${BINARY}" "${CPU[@]}" merge \
--output "${OUT}/merged" --force --force-presence "${sources[@]}"
done
"${BINARY}" "${CPU[@]}" dump "${OUT}/merged" >"${OUT}/merged.dump"
echo "merged_dump $(digest "${OUT}/merged.dump")" >>"${SUMS}"
rm -f "${OUT}/merged.dump"
# ── 3. query: query stage 1 (find per kmer, layer by layer) ─────────────────
for kind in presence count; do
for sp in Escherichia_coli/K-12_MG1655 Saccharolobus_islandicus/M.16.4; do
reads="${RUN_DIR}/query_data/${sp}"
for rep in $(seq "${REPS}"); do
timed "query_${kind}_${sp%%/*}" "${rep}" "${BINARY}" "${CPU[@]}" query \
--count-missing "${DATA}/global_${kind}" \
"${reads}/reads_R1.fastq.gz" "${reads}/reads_R2.fastq.gz"
done
echo "query_${kind}_${sp%%/*} $(digest "${OUT}/last.stdout")" >>"${SUMS}"
done
done
# ── 4. filter: iter_layer_kmers → hash_batch + fill_sub_matrix ──────────────
for rep in $(seq "${REPS}"); do
timed "filter_species" "${rep}" "${BINARY}" "${CPU[@]}" filter \
--output "${OUT}/filtered" --force --ingroup "species=Escherichia_coli" \
--outgroup all --min-frac 0.5 --max-frac 1.0 --max-outgroup-count 0 \
"${DATA}/global_presence"
done
"${BINARY}" "${CPU[@]}" dump "${OUT}/filtered" >"${OUT}/filtered.dump"
echo "filtered_dump $(digest "${OUT}/filtered.dump")" >>"${SUMS}"
rm -f "${OUT}/filtered.dump"
# ── 5. convert: build_exact_evidence / build_approx_evidence ────────────────
# On an APFS clone (cp -c) so run/global_index_presence is untouched.
for rep in $(seq "${REPS}"); do
rm -rf "${OUT}/conv"; cp -cR "${DATA}/global_presence" "${OUT}/conv"
timed "convert_to_approx8" "${rep}" "${BINARY}" "${CPU[@]}" convert \
--approx-evidence 8 "${OUT}/conv"
timed "convert_to_exact" "${rep}" "${BINARY}" "${CPU[@]}" convert \
--exact-evidence "${OUT}/conv"
done
"${BINARY}" "${CPU[@]}" dump "${OUT}/conv" >"${OUT}/conv.dump"
echo "convert_roundtrip_dump $(digest "${OUT}/conv.dump")" >>"${SUMS}"
rm -rf "${OUT}/conv" "${OUT}/conv.dump"
# ── 6. phylo: sibling annex (annex.rs: find_in_partition per variant) and
# its consumer (family_scan.rs: hash_batch / find_slot) ──────────────────
for rep in $(seq "${REPS}"); do
rm -rf "${OUT}/phy"; cp -cR "${DATA}/global_presence" "${OUT}/phy"
timed "phylo_sibling_annex" "${rep}" "${BINARY}" "${CPU[@]}" phylo \
--sibling-annex "${OUT}/phy"
timed "phylo_snp_distance" "${rep}" "${BINARY}" "${CPU[@]}" phylo \
--output "${OUT}/snp" --distance snp-jc --csv "${OUT}/phy"
done
echo "phylo_snp_dist $(digest "${OUT}/snp_dist.csv")" >>"${SUMS}"
rm -rf "${OUT}/phy" "${OUT}/idx_"* "${OUT}/merged" "${OUT}/filtered" \
"${OUT}/last.stdout" "${OUT}/time.tmp"
echo "results: ${RESULTS}" >&2
echo "checksums: ${SUMS}" >&2
+14
View File
@@ -0,0 +1,14 @@
#!/usr/bin/env python3
"""Usage: summarize_bench.py LABEL [LABEL...] — median wall per step, side by side."""
import csv, statistics, sys, os
base = os.path.join(os.path.dirname(os.path.abspath(__file__)), "run", "bench_mphf")
data = {}
for lab in sys.argv[1:]:
for r in csv.DictReader(open(f"{base}/{lab}/results.csv")):
data.setdefault(r["step"], {}).setdefault(lab, []).append(float(r["wall_s"]))
print(f"{'step':32}" + "".join(f"{l:>14}" for l in sys.argv[1:]) + (" ratio" if len(sys.argv) > 2 else ""))
for step, d in data.items():
med = [statistics.median(d[l]) for l in sys.argv[1:]]
line = f"{step:32}" + "".join(f"{m:14.2f}" for m in med)
if len(med) > 1 and med[0]: line += f" {med[-1]/med[0]:.3f}"
print(line)
+1
View File
@@ -48,6 +48,7 @@ nav:
- obipipeline library: implementation/obipipeline.md
- On-disk storage: implementation/storage.md
- MPHF selection: implementation/mphf.md
- "MPHF batch queries (inventory)": implementation/mphf_batch_queries.md
- Unitig evidence encoding: implementation/unitig_evidence.md
- Evidence elimination (discussion): implementation/evidence_elimination.md
- obikindex layer tier: implementation/layer_tier.md
+11
View File
@@ -199,6 +199,17 @@ impl KmerLayer {
}
}
/// Batch form of [`find_slot`](Self::find_slot): `out[i]` is
/// `find_slot(kmers[i])`, MPHF lookups prefetched together. `out` is
/// cleared and reused.
pub fn find_slot_batch(&self, kmers: &[CanonicalKmer], out: &mut Vec<Option<usize>>) {
match self {
KmerLayer::Count { layer, .. } => layer.find_slot_batch(kmers, out),
KmerLayer::Presence { layer, .. } => layer.find_slot_batch(kmers, out),
KmerLayer::Empty { .. } => panic!("Layer::find_slot_batch() called on an Empty layer"),
}
}
/// Raw MPHF batch lookup: kmer → slot, no membership check — for
/// callers that already know every kmer is a member of *this* layer, so
/// the evidence check `find_slot`/`find` would perform is redundant.
+1
View File
@@ -3,6 +3,7 @@ pub mod evidence;
pub mod fingerprint;
pub mod meta;
pub(crate) mod mphf_layer;
pub(crate) mod slotted;
pub mod typed_layer;
pub(crate) mod utils;
+58 -23
View File
@@ -14,6 +14,7 @@ use crate::index::error::{OKIError, OKIResult};
use crate::layer::evidence::{Evidence, EvidenceWriter};
use crate::layer::fingerprint::{FingerprintVec, FingerprintVecWriter};
use crate::layer::meta::IndexMode;
use crate::layer::slotted::SlottedKmers;
pub(crate) const MPHF_FILE: &str = "mphf.bin";
pub(crate) const UNITIGS_FILE: &str = "unitigs.bin";
@@ -162,7 +163,7 @@ pub struct MphfLayer {
/// not a bug).
unitigs: Arc<UnitigFileReader>,
mphf: MemCase<Mphf>,
mphf: MphfOnly,
ev: OpenLayerEvidence,
n: usize,
}
@@ -175,8 +176,7 @@ impl MphfLayer {
/// not stored: `KmerLayer`, one level up, is what owns a layer's
/// directory as a persistent field, not this type.
pub fn open(dir: &Path) -> OKIResult<Self> {
let mphf: MemCase<Mphf> = unsafe { Mphf::mmap(&dir.join(MPHF_FILE), Flags::empty()) }
.map_err(|e| OKIError::InvalidData { context: "layer", detail: e.to_string() })?;
let mphf = MphfOnly::open(dir)?;
let unitigs = Arc::new(UnitigFileReader::open(&dir.join(UNITIGS_FILE))?);
let ev = LayerEvidence::at(dir)?.open()?;
let n = match &ev {
@@ -195,7 +195,14 @@ impl MphfLayer {
/// - Hybrid: fingerprint check (fast path), zero false positives via `find_strict`.
#[inline]
pub fn find(&self, kmer: CanonicalKmer) -> Option<usize> {
let slot = self.mphf.uncase().index(&kmer.raw());
self.confirm(self.mphf.index(kmer), kmer)
}
/// Membership check of `kmer` against the slot the MPHF gave it — the
/// mode-dependent half of [`find`](Self::find), shared with
/// [`find_batch`](Self::find_batch).
#[inline]
fn confirm(&self, slot: usize, kmer: CanonicalKmer) -> Option<usize> {
if slot >= self.n {
return None;
}
@@ -219,13 +226,29 @@ impl MphfLayer {
}
}
/// Batch form of [`find`](Self::find): `out[i]` is `find(kmers[i])`.
///
/// The MPHF lookups of the whole slice go through `ptr_hash`'s prefetching
/// stream (see [`MphfOnly::for_each_slot`]), so their random pilot reads
/// overlap instead of being paid one after the other; the membership
/// check runs on each slot as it comes out. `out` is cleared and reused.
pub fn find_batch(&self, kmers: &[CanonicalKmer], out: &mut Vec<Option<usize>>) {
out.clear();
out.reserve(kmers.len());
let mut i = 0;
self.mphf.for_each_slot(kmers, |slot| {
out.push(self.confirm(slot, kmers[i]));
i += 1;
});
}
/// Always-exact lookup — zero false positives regardless of mode.
///
/// - Exact/Hybrid: O(1) via evidence + `verify_canonical_kmer`.
/// - Approx: O(n) sequential scan of `unitigs.bin` to confirm the kmer
/// that owns the slot, then exact comparison.
pub fn find_strict(&self, kmer: CanonicalKmer) -> Option<usize> {
let slot = self.mphf.uncase().index(&kmer.raw());
let slot = self.mphf.index(kmer);
if slot >= self.n {
return None;
}
@@ -247,7 +270,7 @@ impl MphfLayer {
// the second file-open + mmap + kmer-count scan a fresh
// `open_sequential` call would repeat on every call.
for (stored, _, _) in self.unitigs.iter_indexed_canonical_kmers() {
if self.mphf.uncase().index(&stored.raw()) == slot {
if self.mphf.index(stored) == slot {
return if stored == kmer { Some(slot) } else { None };
}
}
@@ -278,16 +301,16 @@ impl MphfLayer {
/// when the kmer is actually present in the layer; callers that need an
/// existence test should use [`find`](Self::find) instead.
pub fn hash(&self, kmer: CanonicalKmer) -> usize {
self.mphf.uncase().index(&kmer.raw())
self.mphf.index(kmer)
}
/// Batch raw MPHF lookup: kmers → slots.
///
/// Returns a [`Vec`] of slots, one per input kmer, in the same order as the
/// input slice. Like [`index`](Self::index), no membership check is
/// performed.
/// input slice. Like [`hash`](Self::hash), no membership check is
/// performed. Prefetching — see [`MphfOnly::index_batch`].
pub fn hash_batch(&self, kmers: &[CanonicalKmer]) -> Vec<usize> {
kmers.iter().map(|k| self.mphf.uncase().index(&k.raw())).collect()
self.mphf.index_batch(kmers)
}
/// Iterate over all canonical kmers in the layer, in deterministic order.
@@ -445,10 +468,27 @@ impl MphfOnly {
self.0.uncase().index(&kmer.raw())
}
/// Batch form of [`index`](Self::index) — one MPHF lookup pass over the
/// whole slice instead of one call per kmer.
/// Batch form of [`index`](Self::index): one prefetching `ptr_hash`
/// stream over the whole slice, so the random pilot reads of different
/// kmers overlap instead of being paid one after the other.
pub fn index_batch(&self, kmers: &[CanonicalKmer]) -> Vec<usize> {
kmers.iter().map(|&k| self.index(k)).collect()
let mut slots = Vec::with_capacity(kmers.len());
self.for_each_slot(kmers, |slot| slots.push(slot));
slots
}
/// Calls `f(slot)` for each kmer of `kmers`, in order.
///
/// Callback-shaped on purpose: `ptr_hash`'s `index_stream` only
/// implements `fold` (its `next()` is `unimplemented!`), so it cannot be
/// returned as an ordinary iterator without a panic trap for callers.
/// Look-ahead of 32 keys, as recommended by `ptr_hash`.
#[inline]
pub fn for_each_slot(&self, kmers: &[CanonicalKmer], f: impl FnMut(usize)) {
self.0
.uncase()
.index_stream::<32, _>(kmers.iter().map(|k| k.raw()))
.for_each(f);
}
}
@@ -478,8 +518,7 @@ impl MphfLayer {
let mut ev = EvidenceWriter::new(n);
let mut seen = vec![0u8; (n + 7) / 8];
for (kmer, chunk_id, rank) in unitigs.iter_indexed_canonical_kmers() {
let slot = mphf.index(&kmer.raw());
for (slot, _, chunk_id, rank) in SlottedKmers::new(&mphf, unitigs.iter_indexed_canonical_kmers()) {
if slot >= n {
return Err(OKIError::InvalidData { context: "mphf", detail: "slot out of bounds".into() });
}
@@ -521,8 +560,7 @@ impl MphfLayer {
let mut fw = FingerprintVecWriter::new(n, b);
for (kmer, _, _) in unitigs.iter_indexed_canonical_kmers() {
let slot = mphf.index(&kmer.raw());
for (slot, kmer, _, _) in SlottedKmers::new(&mphf, unitigs.iter_indexed_canonical_kmers()) {
if slot >= n {
return Err(OKIError::InvalidData { context: "mphf", detail: "slot out of bounds".into() });
}
@@ -592,8 +630,7 @@ impl MphfLayer {
match mode {
IndexMode::Exact => {
let mut ev = EvidenceWriter::new(n);
for (kmer, chunk_id, rank) in unitigs2.iter_indexed_canonical_kmers() {
let slot = mphf.index(&kmer.raw());
for (slot, kmer, chunk_id, rank) in SlottedKmers::new(&mphf, unitigs2.iter_indexed_canonical_kmers()) {
if slot >= n {
return Err(OKIError::InvalidData { context: "mphf", detail: "slot out of bounds".into() });
}
@@ -612,8 +649,7 @@ impl MphfLayer {
IndexMode::Approx { b, .. } => {
let mut fw = FingerprintVecWriter::new(n, *b);
for (kmer, _, _) in unitigs2.iter_indexed_canonical_kmers() {
let slot = mphf.index(&kmer.raw());
for (slot, kmer, _, _) in SlottedKmers::new(&mphf, unitigs2.iter_indexed_canonical_kmers()) {
if slot >= n {
return Err(OKIError::InvalidData { context: "mphf", detail: "slot out of bounds".into() });
}
@@ -632,8 +668,7 @@ impl MphfLayer {
IndexMode::Hybrid { b, .. } => {
let mut ev = EvidenceWriter::new(n);
let mut fw = FingerprintVecWriter::new(n, *b);
for (kmer, chunk_id, rank) in unitigs2.iter_indexed_canonical_kmers() {
let slot = mphf.index(&kmer.raw());
for (slot, kmer, chunk_id, rank) in SlottedKmers::new(&mphf, unitigs2.iter_indexed_canonical_kmers()) {
if slot >= n {
return Err(OKIError::InvalidData { context: "mphf", detail: "slot out of bounds".into() });
}
+58
View File
@@ -0,0 +1,58 @@
//! A layer's kmers streamed together with their MPHF slot — the shape every
//! build pass needs (`slot` indexes evidence, fingerprint and bitmap writes).
use obikseq::CanonicalKmer;
use crate::layer::mphf_layer::Mphf;
/// Kmers per `ptr_hash` stream: large enough to amortise its ring-buffer
/// start-up, small enough to stay in L1/L2 and never hold the layer.
const CHUNK: usize = 4096;
/// `(slot, kmer, chunk_id, rank)` for every `(kmer, chunk_id, rank)` of the
/// wrapped iterator, in the same order. The MPHF lookups are done a chunk at
/// a time through `ptr_hash`'s prefetching stream, so their random pilot
/// reads overlap; memory stays bounded by one chunk.
pub(crate) struct SlottedKmers<'a, I> {
mphf: &'a Mphf,
kmers: I,
chunk: Vec<(CanonicalKmer, usize, usize)>,
slots: Vec<usize>,
pos: usize,
}
impl<'a, I: Iterator<Item = (CanonicalKmer, usize, usize)>> SlottedKmers<'a, I> {
pub(crate) fn new(mphf: &'a Mphf, kmers: I) -> Self {
Self {
mphf,
kmers,
chunk: Vec::with_capacity(CHUNK),
slots: Vec::with_capacity(CHUNK),
pos: 0,
}
}
}
impl<I: Iterator<Item = (CanonicalKmer, usize, usize)>> Iterator for SlottedKmers<'_, I> {
type Item = (usize, CanonicalKmer, usize, usize);
fn next(&mut self) -> Option<Self::Item> {
if self.pos == self.chunk.len() {
self.chunk.clear();
self.chunk.extend(self.kmers.by_ref().take(CHUNK));
if self.chunk.is_empty() {
return None;
}
self.slots.clear();
let slots = &mut self.slots;
self.mphf
.index_stream::<32, _>(self.chunk.iter().map(|(k, _, _)| k.raw()))
.for_each(|slot| slots.push(slot));
self.pos = 0;
}
let (kmer, chunk_id, rank) = self.chunk[self.pos];
let slot = self.slots[self.pos];
self.pos += 1;
Some((slot, kmer, chunk_id, rank))
}
}
+7
View File
@@ -189,6 +189,13 @@ impl<D: LayerData> TypedLayer<D> {
self.kmerstore.find(kmer)
}
/// Batch form of [`find_slot`](Self::find_slot): `out[i]` is
/// `find_slot(kmers[i])`, with the MPHF lookups prefetched together
/// (see `MphfLayer::find_batch`). `out` is cleared and reused.
pub fn find_slot_batch(&self, kmers: &[CanonicalKmer], out: &mut Vec<Option<usize>>) {
self.kmerstore.find_batch(kmers, out)
}
pub fn n(&self) -> usize {
self.kmerstore.n()
}
+39 -7
View File
@@ -134,16 +134,48 @@ impl QueryPartition for IndexCache {
let mut by_layer: Vec<HashMap<usize, &Vec<KmerDesc>>> =
(0..n_layer).map(|_| HashMap::new()).collect();
for (kmer, descs) in kmers {
stats.n_unique_kmers += 1;
// Layer-major over fixed-size chunks of the (unordered) kmer map:
// each layer sees the chunk's still-unresolved kmers as one batch, so
// its MPHF lookups are prefetched together; a kmer found in an
// earlier layer never reaches the later ones (first hit wins, as
// before). Chunked, not collected — the map is never copied whole.
const CHUNK: usize = 4096;
let mut entries = kmers.iter();
let mut pending: Vec<(CanonicalKmer, &Vec<KmerDesc>)> = Vec::with_capacity(CHUNK);
let mut batch: Vec<CanonicalKmer> = Vec::with_capacity(CHUNK);
let mut found: Vec<Option<usize>> = Vec::with_capacity(CHUNK);
loop {
pending.clear();
pending.extend(entries.by_ref().take(CHUNK).map(|(k, d)| (*k, d)));
if pending.is_empty() {
break;
}
stats.n_unique_kmers += pending.len();
for l in 0..n_layer {
stats.n_mphf_calls += 1;
if let Some(slot) = self.find_in_layer(part_idx, l, *kmer) {
by_layer[l].insert(slot, descs);
on_event(QueryHit::Found(descs));
stats.n_hits += 1;
if pending.is_empty() {
break;
}
let layer = self
.get_layer(part_idx, l)
.expect("layer within n_layer(part_idx)");
batch.clear();
batch.extend(pending.iter().map(|&(k, _)| k));
stats.n_mphf_calls += batch.len();
layer.find_slot_batch(&batch, &mut found);
let mut i = 0;
pending.retain(|&(_, descs)| {
let hit = found[i];
i += 1;
match hit {
Some(slot) => {
by_layer[l].insert(slot, descs);
on_event(QueryHit::Found(descs));
stats.n_hits += 1;
false
}
None => true,
}
});
}
}