diff --git a/DevDocMD/implementation/mphf_batch_queries.md b/DevDocMD/implementation/mphf_batch_queries.md new file mode 100644 index 00000000..48eb7335 --- /dev/null +++ b/DevDocMD/implementation/mphf_batch_queries.md @@ -0,0 +1,179 @@ +# MPHF batch queries — site inventory and expected gains + +Investigation of 2026-10-14 (`ptr_hash` 2.1.2, `epserde` 0.13). Question: where would batched MPHF queries (software prefetch of the random accesses) save time? Status of each claim is marked **[read]** (verified in source), **[computed]** (derived from stated sizes) or **[estimate]** (reasoning, not measured). Measurements, once made, are in the last section. + +## 1. What `ptr_hash` 2.1.2 actually offers [read] + +`Mphf = PtrHash, SINGLE_PART = false, REMAP = true>`. + +- **`V` is the pilot array** (one byte per bucket), **`F` is only the remap storage**. A layer's `mphf.bin` (~2.4 bits/key **[computed]**: 40 MB for ~130 M keys) is therefore almost entirely a byte array indexed by bucket. The batch APIs prefetch `pilots[bucket]` — the right line. The remap (read only when `slot >= n`, a small fraction of keys) is **not** prefetched. +- `index_batch(xs: [Q; K]) -> [usize; K]` — stable, not feature-gated; hash K keys, prefetch K pilots, query. Remainder (`len % K`) is the caller's job. +- `index_stream(keys: IntoIterator) -> impl Iterator` — ring buffer of B keys; handles head and tail itself; accepts any iterator. **Its `next()` is `unimplemented!`**: only fold-based consumption works (`for_each`, `fold`, `sum`). `collect()`, `zip`, `for x in it` panic. Consume with `for_each` into a pre-sized `Vec`. +- `index_batch_exact` needs the nightly `unstable` feature; `index_batch_exact2` is `#[doc(hidden)]`. Neither is usable. +- Prefetch is `prefetch-index 0.2.1`: `_mm_prefetch(T0)` on x86_64, asm on aarch64, **no-op elsewhere**. +- **`SINGLE_PART`**: neither `index_stream` nor `index_batch` branches on it — both always run the multi-part `bucket()`/`slot()`. Only `index()` has the faster single-part path. So with batched queries the `SINGLE_PART = true` advantage disappears, and `false` (our choice) costs nothing while keeping parallel construction (`true` also panics with sharding). +- Old `ptr_hash` 1.1.0 (used by `tools/migrate-mphf`) has the same family with an extra `MINIMAL` const: `index_stream`, `index_batch`. No new dependency needed there. + +```rust +// consuming index_stream: fold-based only +let mut out = Vec::with_capacity(kmers.len()); +mphf.index_stream::<32, _>(kmers.iter().map(|k| k.raw())) + .for_each(|slot| out.push(slot)); +``` + +## 2. Existing "batch" APIs that are not batched [read] + +`MphfLayer::hash_batch` and `MphfOnly::index_batch` loop `.index()` one kmer at a time. Consumers: `obikfilter/partition_iter.rs` (`iter_layer_kmers`), `obikmerge/partition_merge.rs` (pass 2: `hash_batch`, `index_batch`), `obikphylo/siblings/algorithms/family_scan.rs` (two sites). + +## 3. Cost chain of a membership query (`MphfLayer::find`) [read] + +1. `pilots[bucket]` — the only access `ptr_hash` prefetches. +2. Exact/Hybrid-exact: `evidence.decode(slot)` — 4 B/slot, ~520 MB per 130 M-slot layer **[computed]**, random mmap read. +3. `verify_canonical_kmer` → `block_offsets[...]` (O(1) with the default `--block-size 1`; a forward scan of up to `2^block_bits` records otherwise). +4. `extract_kmer_raw` — random read in the `unitigs.bin` mmap. + +Approx/Hybrid-fast: pilots then `fingerprint.get(slot)` — one extra random access. +Each stage is independent across kmers, so a staged batch (slots → prefetch evidence → decode, prefetch offsets/unitigs → verify) can overlap all of them. `ptr_hash` only helps with stage 1; stages 2–4 need prefetch support in `obikindex`/`obiskio` (not existing). Prefetch on an unmapped (cold, not paged-in) mmap page is dropped by the CPU, so none of this helps on a cold index. + +Layer sizes: `layer_0` MPHF 2–3 MB (cache-resident), `layer_1` ~40 MB (beyond one CCD's L3 on EPYC-class machines — **hypothesis**, prod CPU not verified, and contended by 192 threads). Gains are expected mainly on `layer_1`. For branch-free independent loops (`hash_batch`) out-of-order execution already overlaps several misses, so the explicit prefetch gains less **[estimate]** than for branchy loops (`find` with early `break`). + +## 4. Call-site inventory + +(a) already batched; (b) batch buildable; (c) hard/pointless. + +| Site | What | Class | +|---|---|---| +| `mphf_layer.rs` `hash_batch`, `MphfOnly::index_batch` | loop of `.index()` | (a) | +| `obikfilter/partition_iter.rs:106` | `hash_batch`, 32768 kmers | (a) | +| `obikmerge/partition_merge.rs:489, 521` | `hash_batch`, `index_batch`, 4096 | (a) | +| `obikphylo/.../family_scan.rs:285, 397` | `hash_batch` | (a) | +| `obikquery/query_layer.rs` stage 1 | `find_in_layer` per kmer, layer by layer, `break` on hit | (b) | +| `partition_merge.rs` `find_batch_grouped` | `find_slot` per kmer over dst layers | (b) | +| `partition_merge.rs` (`build_graph` filter) | `all(find_slot.is_none())`, every layer, per kmer; `build_graph` takes `Fn(CanonicalKmer)->bool` | (b), API change | +| `family_scan.rs` slow path; `annex.rs` | `find_slot` / `find_in_partition` per variant | (b) | +| `mphf_layer.rs` `build`, `build_exact_evidence`, `build_approx_evidence` | `mphf.index` then random writes (evidence, fingerprint, `seen`) | (b), chunked | +| `obikindexer/counter/count.rs:47` | `mphf.index` then `counts.get/set` | (b), hard (needs (kmer, count) pairs) | +| `obikindexer/extensions/private_builder.rs:177` | abundance filter | (b), conditional | +| `private_builder.rs:196`, `verify.rs`, `IndexCache::find_in_*` | per-kmer by nature / sampled | (c) | +| `tools/migrate-mphf/plan.rs:41-42` | `old.index` + `new.index` per kmer, random writes | (b) | + +Dead or suspect code found along the way (candidates for removal, not removed): + +- `MphfLayer::find_strict` — no caller. +- the single-kmer `hash` chain (`MphfLayer::hash` → `TypedLayer::hash` → `KmerLayer::hash` → `IndexCache::hash`) — no caller outside the chain. +- `IndexCache::find` — no caller. +- `obikmerge/src/merge_layer/` — not declared in `lib.rs` (`mod merge; mod partition_merge;` only); still imports the `LayeredMap` removed in 2026-08. To be confirmed by the compiler. + +## 5. Interactions + +- **Verification chain**: the right shape is one `find_batch(&[CanonicalKmer], &mut Vec>)` on `MphfLayer`, staged as in §3, so every consumer benefits (rule: enrich the shared API, no local loops). +- **Memory**: callers already hold bounded batches (4096 / 32768). A buffer-reusing variant avoids one `Vec` per call. `index_stream` itself only allocates a B-element ring on the stack. +- **Parallelism**: no new parallelism. Batches already come out of `obipipeline` stages; the change lives in the transform body. Construction pass 2 is sequential inside a partition: chunk the streamed iterator (1–4 k kmers), never collect the layer. In `query_partition_with` the unique-kmer `HashMap` is already materialised; iterate it chunk-wise. + +## 6. Benchmark + +`benchmark/bench_mphf.sh LABEL` times, on the 20-genome benchmark data, `index` (2 genomes), `merge` (20 presence indexes), `query` (presence/count × 2 genomes), `filter`, `convert` (approx/exact round trip) and `phylo` (`--sibling-annex`, `snp-jc`), `REPS` times each, and records order-independent output digests that must not change across labels. See the results section below. + +**Limit of this data set**: 20 small genomes → per-partition `layer_1` MPHFs of ~100 KB, fully cache-resident on the dev machine (Apple M3 Max, aarch64). This benchmark can detect regressions and the effect on instruction-level overlap; it **cannot** measure the DRAM-latency gain that motivates prefetch at 40 MB/layer. A faithful test needs a prod-scale layer (see the throwaway benchmark below). + +### Throwaway prod-scale benchmark (not written) + +Open one real partition's `layer_1` (~130 M keys) and `layer_0` (~8 M) with `MphfOnly`/`MphfLayer`; key sets: present (sample of `iter_kmers`, random order and sequential order) and absent (random, checked absent); measure kmers/s for `index`, `index_stream::` (B ∈ 8…128), `index_batch::` (K ∈ 8…64), and the full `find` vs staged `find_batch`; at 1, 8, 32, 192 threads; pinned to one NUMA node, then spread; warm and cold page cache stated; 5 repetitions, min/median; sizes of `mphf.bin`, `evidence.bin`, `unitigs.bin(.idx)` reported; first check `index_stream` slots == `index` slots. The ~8–12 ns/key figure of the PtrHash paper (quoted in `mphf.md`) is a paper claim, not tied to our sizes or machines. + +## 7. Not verified + +No measurement at prod scale; prod CPU architecture and cache sizes; evidence mode and `--block-size` used in prod; whether `migrate-mphf` already ran; share of total index/merge/query time spent in these loops; `count.rs` f0 per partition at prod scale; whether `merge_layer/` is dead (only the head of `lib.rs` read). + +## 8. First improvement and measurements (2026-10-15) + +**Implemented** (query stage 1, the top-ranked site; the same change adds the plumbing the other sites will reuse): + +- `MphfLayer` now holds a `MphfOnly` instead of its own `MemCase` (one mmap-open path, one place for the stream call). +- `MphfOnly::for_each_slot(kmers, f)` wraps `index_stream::<32>`; `index_batch`/`hash_batch` now really use it (they previously looped `.index()`). Callback-shaped because `index_stream`'s `next()` is `unimplemented!`. +- `MphfLayer::find_batch` / `TypedLayer::find_slot_batch` / `KmerLayer::find_slot_batch`; the mode-dependent half of `find` is factored into `MphfLayer::confirm`, shared by both. +- `query_partition_with` stage 1 is layer-major over chunks of 4096 unique kmers: each layer receives the still-unresolved kmers as one batch (first hit wins, `n_mphf_calls` unchanged). +- Not done: staged prefetch of evidence/offsets/unitigs (needs a prefetch primitive; `prefetch-index` is already in the tree via `ptr_hash`, but adding it to `obikindex` is a dependency decision), and the other (b) sites. + +**Measurements** (`benchmark/bench_mphf.sh`, `benchmark/summarize_bench.py`; Apple M3 Max, aarch64; 20 genomes at 15x). Interleaved A/B, 8 runs each, E. coli reads queried against the 20-genome presence index: + +| | wall (s) | user CPU (s) | +|---|---|---| +| before | 5.94–6.02 (median 5.96) | 10.95–11.07 | +| after | 5.63–6.00 (median 5.67) | 10.33–10.66 | + +About −5 % wall and −5.5 % CPU, ranges non-overlapping **[measured]**. All output digests (query ×4, merge, filter, convert round trip, phylo) are identical before and after. + +**Reading the number.** Small by construction: this data set's `layer_1` MPHFs are ~100 KB per partition, cache-resident, and query time is dominated by reading and k-merising the reads. The gain seen here is instruction-level overlap, not DRAM-latency hiding. The latter, the reason for the whole exercise, remains **unmeasured** and needs a prod-scale layer (§6). + +**Noise caveat.** With a load average of 16–22 from unrelated processes, single-run walls of `index`, `merge` and `filter` moved ±10–40 % between identical binaries; only interleaved repeated A/B runs were trustworthy. The `index`/`merge`/`filter`/`convert`/`phylo` rows of the first comparison are noise, not effects (`index` pass 2 was not changed at all; a later interleaved `filter` A/B was inconclusive, 13.0 s vs 13.8 s medians with a 10–20 s spread). + +**Stale benchmark indexes.** The indexes already under `benchmark/run/` (`specimen_index_*`, `global_index_*`, built 2026-08-28) carry `ptr_hash` 1.x MPHFs and fail with "Major version mismatch" on current binaries (`tools/migrate-mphf` converts them). `bench_mphf.sh` therefore builds its own copies under `run/bench_mphf/data/` on first use. + +## 9. Benchmark procedure + +Prerequisites: the simulated reads (`benchmark/run/simulated_data/`) and query reads (`benchmark/run/query_data/`) produced by `gmake simulate simulate_query`; macOS BSD `time -l` and `md5 -q` (the script uses both), Python 3. + +1. Build the binary to measure and **copy it** under a name that identifies it (cargo overwrites `target/release/obikmer`): `cp src/target/release/obikmer benchmark/run/bench_mphf/obikmer_