Implement batch lookup and chunked data management
Introduces support for batched MPHF lookups, chunked data handling via `SlottedKmers`, and a comprehensive benchmarking suite to measure performance across the indexing and querying pipeline. This includes new layer functionality for batch operations, chunking logic in the query layer, and scripts to measure wall time and resource usage for various commands. Documentation related to specific implementation details has also been cleaned up.
This commit is contained in:
1 parent
8b87078478
commit
fe38424682
10 files changed
+518
-28
No files matched your search
@@ -0,0 +1,179 @@
|
||||
# MPHF batch queries — site inventory and expected gains
|
||||
|
||||
Investigation of 2026-10-14 (`ptr_hash` 2.1.2, `epserde` 0.13). Question: where would batched MPHF queries (software prefetch of the random accesses) save time? Status of each claim is marked **[read]** (verified in source), **[computed]** (derived from stated sizes) or **[estimate]** (reasoning, not measured). Measurements, once made, are in the last section.
|
||||
|
||||
## 1. What `ptr_hash` 2.1.2 actually offers [read]
|
||||
|
||||
`Mphf = PtrHash<u64, CubicEps, F = CachelineEfVec, Xxh3Int, V = Vec<u8>, SINGLE_PART = false, REMAP = true>`.
|
||||
|
||||
- **`V` is the pilot array** (one byte per bucket), **`F` is only the remap storage**. A layer's `mphf.bin` (~2.4 bits/key **[computed]**: 40 MB for ~130 M keys) is therefore almost entirely a byte array indexed by bucket. The batch APIs prefetch `pilots[bucket]` — the right line. The remap (read only when `slot >= n`, a small fraction of keys) is **not** prefetched.
|
||||
- `index_batch<K>(xs: [Q; K]) -> [usize; K]` — stable, not feature-gated; hash K keys, prefetch K pilots, query. Remainder (`len % K`) is the caller's job.
|
||||
- `index_stream<B>(keys: IntoIterator) -> impl Iterator` — ring buffer of B keys; handles head and tail itself; accepts any iterator. **Its `next()` is `unimplemented!`**: only fold-based consumption works (`for_each`, `fold`, `sum`). `collect()`, `zip`, `for x in it` panic. Consume with `for_each` into a pre-sized `Vec`.
|
||||
- `index_batch_exact` needs the nightly `unstable` feature; `index_batch_exact2` is `#[doc(hidden)]`. Neither is usable.
|
||||
- Prefetch is `prefetch-index 0.2.1`: `_mm_prefetch(T0)` on x86_64, asm on aarch64, **no-op elsewhere**.
|
||||
- **`SINGLE_PART`**: neither `index_stream` nor `index_batch` branches on it — both always run the multi-part `bucket()`/`slot()`. Only `index()` has the faster single-part path. So with batched queries the `SINGLE_PART = true` advantage disappears, and `false` (our choice) costs nothing while keeping parallel construction (`true` also panics with sharding).
|
||||
- Old `ptr_hash` 1.1.0 (used by `tools/migrate-mphf`) has the same family with an extra `MINIMAL` const: `index_stream<B, MINIMAL, Q>`, `index_batch<K, MINIMAL, Q>`. No new dependency needed there.
|
||||
|
||||
```rust
|
||||
// consuming index_stream: fold-based only
|
||||
let mut out = Vec::with_capacity(kmers.len());
|
||||
mphf.index_stream::<32, _>(kmers.iter().map(|k| k.raw()))
|
||||
.for_each(|slot| out.push(slot));
|
||||
```
|
||||
|
||||
## 2. Existing "batch" APIs that are not batched [read]
|
||||
|
||||
`MphfLayer::hash_batch` and `MphfOnly::index_batch` loop `.index()` one kmer at a time. Consumers: `obikfilter/partition_iter.rs` (`iter_layer_kmers`), `obikmerge/partition_merge.rs` (pass 2: `hash_batch`, `index_batch`), `obikphylo/siblings/algorithms/family_scan.rs` (two sites).
|
||||
|
||||
## 3. Cost chain of a membership query (`MphfLayer::find`) [read]
|
||||
|
||||
1. `pilots[bucket]` — the only access `ptr_hash` prefetches.
|
||||
2. Exact/Hybrid-exact: `evidence.decode(slot)` — 4 B/slot, ~520 MB per 130 M-slot layer **[computed]**, random mmap read.
|
||||
3. `verify_canonical_kmer` → `block_offsets[...]` (O(1) with the default `--block-size 1`; a forward scan of up to `2^block_bits` records otherwise).
|
||||
4. `extract_kmer_raw` — random read in the `unitigs.bin` mmap.
|
||||
|
||||
Approx/Hybrid-fast: pilots then `fingerprint.get(slot)` — one extra random access.
|
||||
Each stage is independent across kmers, so a staged batch (slots → prefetch evidence → decode, prefetch offsets/unitigs → verify) can overlap all of them. `ptr_hash` only helps with stage 1; stages 2–4 need prefetch support in `obikindex`/`obiskio` (not existing). Prefetch on an unmapped (cold, not paged-in) mmap page is dropped by the CPU, so none of this helps on a cold index.
|
||||
|
||||
Layer sizes: `layer_0` MPHF 2–3 MB (cache-resident), `layer_1` ~40 MB (beyond one CCD's L3 on EPYC-class machines — **hypothesis**, prod CPU not verified, and contended by 192 threads). Gains are expected mainly on `layer_1`. For branch-free independent loops (`hash_batch`) out-of-order execution already overlaps several misses, so the explicit prefetch gains less **[estimate]** than for branchy loops (`find` with early `break`).
|
||||
|
||||
## 4. Call-site inventory
|
||||
|
||||
(a) already batched; (b) batch buildable; (c) hard/pointless.
|
||||
|
||||
| Site | What | Class |
|
||||
|---|---|---|
|
||||
| `mphf_layer.rs` `hash_batch`, `MphfOnly::index_batch` | loop of `.index()` | (a) |
|
||||
| `obikfilter/partition_iter.rs:106` | `hash_batch`, 32768 kmers | (a) |
|
||||
| `obikmerge/partition_merge.rs:489, 521` | `hash_batch`, `index_batch`, 4096 | (a) |
|
||||
| `obikphylo/.../family_scan.rs:285, 397` | `hash_batch` | (a) |
|
||||
| `obikquery/query_layer.rs` stage 1 | `find_in_layer` per kmer, layer by layer, `break` on hit | (b) |
|
||||
| `partition_merge.rs` `find_batch_grouped` | `find_slot` per kmer over dst layers | (b) |
|
||||
| `partition_merge.rs` (`build_graph` filter) | `all(find_slot.is_none())`, every layer, per kmer; `build_graph` takes `Fn(CanonicalKmer)->bool` | (b), API change |
|
||||
| `family_scan.rs` slow path; `annex.rs` | `find_slot` / `find_in_partition` per variant | (b) |
|
||||
| `mphf_layer.rs` `build`, `build_exact_evidence`, `build_approx_evidence` | `mphf.index` then random writes (evidence, fingerprint, `seen`) | (b), chunked |
|
||||
| `obikindexer/counter/count.rs:47` | `mphf.index` then `counts.get/set` | (b), hard (needs (kmer, count) pairs) |
|
||||
| `obikindexer/extensions/private_builder.rs:177` | abundance filter | (b), conditional |
|
||||
| `private_builder.rs:196`, `verify.rs`, `IndexCache::find_in_*` | per-kmer by nature / sampled | (c) |
|
||||
| `tools/migrate-mphf/plan.rs:41-42` | `old.index` + `new.index` per kmer, random writes | (b) |
|
||||
|
||||
Dead or suspect code found along the way (candidates for removal, not removed):
|
||||
|
||||
- `MphfLayer::find_strict` — no caller.
|
||||
- the single-kmer `hash` chain (`MphfLayer::hash` → `TypedLayer::hash` → `KmerLayer::hash` → `IndexCache::hash`) — no caller outside the chain.
|
||||
- `IndexCache::find` — no caller.
|
||||
- `obikmerge/src/merge_layer/` — not declared in `lib.rs` (`mod merge; mod partition_merge;` only); still imports the `LayeredMap` removed in 2026-08. To be confirmed by the compiler.
|
||||
|
||||
## 5. Interactions
|
||||
|
||||
- **Verification chain**: the right shape is one `find_batch(&[CanonicalKmer], &mut Vec<Option<usize>>)` on `MphfLayer`, staged as in §3, so every consumer benefits (rule: enrich the shared API, no local loops).
|
||||
- **Memory**: callers already hold bounded batches (4096 / 32768). A buffer-reusing variant avoids one `Vec<usize>` per call. `index_stream` itself only allocates a B-element ring on the stack.
|
||||
- **Parallelism**: no new parallelism. Batches already come out of `obipipeline` stages; the change lives in the transform body. Construction pass 2 is sequential inside a partition: chunk the streamed iterator (1–4 k kmers), never collect the layer. In `query_partition_with` the unique-kmer `HashMap` is already materialised; iterate it chunk-wise.
|
||||
|
||||
## 6. Benchmark
|
||||
|
||||
`benchmark/bench_mphf.sh LABEL` times, on the 20-genome benchmark data, `index` (2 genomes), `merge` (20 presence indexes), `query` (presence/count × 2 genomes), `filter`, `convert` (approx/exact round trip) and `phylo` (`--sibling-annex`, `snp-jc`), `REPS` times each, and records order-independent output digests that must not change across labels. See the results section below.
|
||||
|
||||
**Limit of this data set**: 20 small genomes → per-partition `layer_1` MPHFs of ~100 KB, fully cache-resident on the dev machine (Apple M3 Max, aarch64). This benchmark can detect regressions and the effect on instruction-level overlap; it **cannot** measure the DRAM-latency gain that motivates prefetch at 40 MB/layer. A faithful test needs a prod-scale layer (see the throwaway benchmark below).
|
||||
|
||||
### Throwaway prod-scale benchmark (not written)
|
||||
|
||||
Open one real partition's `layer_1` (~130 M keys) and `layer_0` (~8 M) with `MphfOnly`/`MphfLayer`; key sets: present (sample of `iter_kmers`, random order and sequential order) and absent (random, checked absent); measure kmers/s for `index`, `index_stream::<B>` (B ∈ 8…128), `index_batch::<K>` (K ∈ 8…64), and the full `find` vs staged `find_batch`; at 1, 8, 32, 192 threads; pinned to one NUMA node, then spread; warm and cold page cache stated; 5 repetitions, min/median; sizes of `mphf.bin`, `evidence.bin`, `unitigs.bin(.idx)` reported; first check `index_stream` slots == `index` slots. The ~8–12 ns/key figure of the PtrHash paper (quoted in `mphf.md`) is a paper claim, not tied to our sizes or machines.
|
||||
|
||||
## 7. Not verified
|
||||
|
||||
No measurement at prod scale; prod CPU architecture and cache sizes; evidence mode and `--block-size` used in prod; whether `migrate-mphf` already ran; share of total index/merge/query time spent in these loops; `count.rs` f0 per partition at prod scale; whether `merge_layer/` is dead (only the head of `lib.rs` read).
|
||||
|
||||
## 8. First improvement and measurements (2026-10-15)
|
||||
|
||||
**Implemented** (query stage 1, the top-ranked site; the same change adds the plumbing the other sites will reuse):
|
||||
|
||||
- `MphfLayer` now holds a `MphfOnly` instead of its own `MemCase<Mphf>` (one mmap-open path, one place for the stream call).
|
||||
- `MphfOnly::for_each_slot(kmers, f)` wraps `index_stream::<32>`; `index_batch`/`hash_batch` now really use it (they previously looped `.index()`). Callback-shaped because `index_stream`'s `next()` is `unimplemented!`.
|
||||
- `MphfLayer::find_batch` / `TypedLayer::find_slot_batch` / `KmerLayer::find_slot_batch`; the mode-dependent half of `find` is factored into `MphfLayer::confirm`, shared by both.
|
||||
- `query_partition_with` stage 1 is layer-major over chunks of 4096 unique kmers: each layer receives the still-unresolved kmers as one batch (first hit wins, `n_mphf_calls` unchanged).
|
||||
- Not done: staged prefetch of evidence/offsets/unitigs (needs a prefetch primitive; `prefetch-index` is already in the tree via `ptr_hash`, but adding it to `obikindex` is a dependency decision), and the other (b) sites.
|
||||
|
||||
**Measurements** (`benchmark/bench_mphf.sh`, `benchmark/summarize_bench.py`; Apple M3 Max, aarch64; 20 genomes at 15x). Interleaved A/B, 8 runs each, E. coli reads queried against the 20-genome presence index:
|
||||
|
||||
| | wall (s) | user CPU (s) |
|
||||
|---|---|---|
|
||||
| before | 5.94–6.02 (median 5.96) | 10.95–11.07 |
|
||||
| after | 5.63–6.00 (median 5.67) | 10.33–10.66 |
|
||||
|
||||
About −5 % wall and −5.5 % CPU, ranges non-overlapping **[measured]**. All output digests (query ×4, merge, filter, convert round trip, phylo) are identical before and after.
|
||||
|
||||
**Reading the number.** Small by construction: this data set's `layer_1` MPHFs are ~100 KB per partition, cache-resident, and query time is dominated by reading and k-merising the reads. The gain seen here is instruction-level overlap, not DRAM-latency hiding. The latter, the reason for the whole exercise, remains **unmeasured** and needs a prod-scale layer (§6).
|
||||
|
||||
**Noise caveat.** With a load average of 16–22 from unrelated processes, single-run walls of `index`, `merge` and `filter` moved ±10–40 % between identical binaries; only interleaved repeated A/B runs were trustworthy. The `index`/`merge`/`filter`/`convert`/`phylo` rows of the first comparison are noise, not effects (`index` pass 2 was not changed at all; a later interleaved `filter` A/B was inconclusive, 13.0 s vs 13.8 s medians with a 10–20 s spread).
|
||||
|
||||
**Stale benchmark indexes.** The indexes already under `benchmark/run/` (`specimen_index_*`, `global_index_*`, built 2026-08-28) carry `ptr_hash` 1.x MPHFs and fail with "Major version mismatch" on current binaries (`tools/migrate-mphf` converts them). `bench_mphf.sh` therefore builds its own copies under `run/bench_mphf/data/` on first use.
|
||||
|
||||
## 9. Benchmark procedure
|
||||
|
||||
Prerequisites: the simulated reads (`benchmark/run/simulated_data/`) and query reads (`benchmark/run/query_data/`) produced by `gmake simulate simulate_query`; macOS BSD `time -l` and `md5 -q` (the script uses both), Python 3.
|
||||
|
||||
1. Build the binary to measure and **copy it** under a name that identifies it (cargo overwrites `target/release/obikmer`): `cp src/target/release/obikmer benchmark/run/bench_mphf/obikmer_<label>`.
|
||||
2. `BINARY=$PWD/run/bench_mphf/obikmer_<label> benchmark/bench_mphf.sh <label>` (from `benchmark/`; `REPS=3` by default, `CPU_MAX=n` to restrict cores). The first call builds the shared data set in `run/bench_mphf/data/` with that binary (≈ 2 min); keep it for every label — the on-disk format must not differ between the binaries compared, otherwise delete `data/` and rebuild.
|
||||
3. Output: `run/bench_mphf/<label>/results.csv` (wall and peak RSS per step and repetition, plus one row `step:stage` per Reporter stage) and `checksums.txt`.
|
||||
4. Compare: `python3 benchmark/summarize_bench.py <ref_label> <new_label>` (medians and ratio) and `diff run/bench_mphf/<ref>/checksums.txt run/bench_mphf/<new>/checksums.txt` — **any difference is a bug**, not a performance matter.
|
||||
5. A ratio is only a candidate effect. Check the spread by running the **same** binary twice under two labels (the script's own noise floor was ±10–40 % per step on a loaded laptop), or confirm with interleaved A/B runs of the single command (alternate the two binaries, ≥ 8 runs, compare ranges of wall and user CPU from `/usr/bin/time -l`).
|
||||
6. Rule of thumb for this data set: only `query` (≈ 5 s, stable to ±1 %) resolves effects of a few percent; `index`, `merge`, `filter`, `phylo` need interleaved runs or a quiet machine.
|
||||
|
||||
Step → code exercised:
|
||||
|
||||
| Step | Code on the MPHF path |
|
||||
|---|---|
|
||||
| `index_*` (`:index` stage) | `MphfLayer::build` pass 2 |
|
||||
| `merge_20_presence` | `find_batch_grouped`, `build_graph` filter, `hash_batch`, `index_batch` |
|
||||
| `query_*` | `query_partition_with` stage 1 |
|
||||
| `filter_species` | `iter_layer_kmers` → `hash_batch` |
|
||||
| `convert_to_*` | `build_exact_evidence`, `build_approx_evidence` |
|
||||
| `phylo_sibling_annex` | `annex.rs` `find_in_partition` per variant |
|
||||
| `phylo_snp_distance` | `family_scan.rs` `hash_batch`, `find_slot` |
|
||||
|
||||
## 10. Second step — construction pass 2 (2026-10-15)
|
||||
|
||||
**Implemented**: `layer/slotted.rs`, `SlottedKmers`, an iterator adapter yielding `(slot, kmer, chunk_id, rank)` for the wrapped `(kmer, chunk_id, rank)` stream, computing slots 4096 kmers at a time through `index_stream::<32>` (bounded memory, never the whole layer). The five loops of `MphfLayer::build` (Exact, Approx, Hybrid), `build_exact_evidence` and `build_approx_evidence` now iterate it instead of calling `mphf.index` per kmer. Output digests unchanged.
|
||||
|
||||
**Measured** (interleaved A/B against the first-step binary, 8 runs each, wall / user CPU):
|
||||
|
||||
| Command | before | after |
|
||||
|---|---|---|
|
||||
| `convert --exact-evidence` (20-genome index) | 0.56–0.68 s / 2.22–2.67 s | 0.59–0.74 s / 2.25–2.52 s |
|
||||
| `index` E. coli reads | 2.45–2.63 s / 13.1–14.3 s | 2.48–2.75 s / 13.2–14.0 s |
|
||||
|
||||
**No measurable effect** on this data set (differences within the run-to-run spread; if anything `convert` is ≈ 4 % slower in wall time). Expected here: the MPHF is cache-resident and `index`/`convert` time is dominated by other stages (scatter, dereplication, counting, unitig decoding, the random evidence/`seen` writes themselves). The construction gain, if any, only exists where the pilot array does not fit in cache, and the random *writes* to `evidence.bin`/`seen`/`fingerprint.bin` (not prefetched by `ptr_hash`) probably dominate there too. Whether to keep this change is therefore a decision to take after the prod-scale benchmark (§6), not on this data.
|
||||
|
||||
## 11. Why the two changes gain so little (profiling, 2026-10-15)
|
||||
|
||||
Measured with macOS `sample` (1 ms stack sampling) on the 20-genome data, one run each — coarse, but the orders of magnitude are unambiguous.
|
||||
|
||||
**Query.** In a 2 s window on the `query` stage (16 threads, ≈ 3 100 active samples against ≈ 55 000 samples of idle waiting):
|
||||
|
||||
- `query_partition_with` ≈ 260 samples (8 % of active CPU), of which `find_batch` 217, of which **`verify_canonical_kmer` 185** and everything else in `find_batch` (the `ptr_hash` stream and `confirm`) **32**.
|
||||
- `verify_canonical_kmer` is the single largest active function of the window (399 top-of-stack samples). The MPHF lookup is ≈ 1 % of active CPU: even a free MPHF could not give more.
|
||||
- Reason: per-partition pilot arrays are ~100 KB (≈ 25 MB over 256 partitions) and stay cached; the evidence/`unitigs.bin`/offset accesses of the verification (§3, stages 2–4) are the ones that miss. This data set already has the problem on the verification side, not on the MPHF side.
|
||||
- The threads are mostly idle: the stage is bounded by reading/parsing/k-merising the reads (`rolling_stat`, entropy tracker, string formatting) and by sequential parts, not by lookups.
|
||||
|
||||
**What produced the −5 %.** Interleaved A/B, 8 runs each, median wall: original 6.4 s; chunked layer-major restructuring with plain per-kmer `index` (stream disabled) 6.0 s; restructuring + `index_stream` 5.85 s. **Most of the gain comes from the restructuring, not from the prefetch** (the prefetch adds ≈ 2 %, at the limit of the noise). Hypothesis, not isolated: the old loop called `IndexCache::get_layer` — an `Arc<KmerLayer>` clone, i.e. an atomic increment/decrement on a refcount shared by all threads — once per kmer per layer; the batched loop does it once per layer per chunk of 4096.
|
||||
|
||||
**Index (pass 2).** In a 1 s window late in the `index` stage, `SlottedKmers` appears with ≈ 7 samples out of 304 per thread (≈ 2 %), against ≈ 200 for the de Bruijn graph work (`obidebruinj`) and ≈ 35 for `ptr_hash::build` (pass 1, MPHF *construction*). Pass 2 is a few percent of `index` and `convert` here: nothing to win even if its lookups were free. `convert` is `build_*_evidence` on 2 MB-scale layers, same conclusion.
|
||||
|
||||
**Consequences for priorities.**
|
||||
|
||||
1. On this data the next lever is the verification (`verify_canonical_kmer`, evidence decode), not the MPHF — consistent with §3, now observed.
|
||||
2. Hot `Arc` clones in per-kmer paths (`get_layer`, `IndexCache::find_in_layer`, `find_slot` loops in merge and phylo) are worth auditing independently of MPHF batching.
|
||||
3. The DRAM-latency gain of prefetch is untested until a prod-scale layer is available (§6).
|
||||
|
||||
## 12. Calibration: a single skim at fewer partitions (2026-10-15)
|
||||
|
||||
`Vaccinium myrtillus` skim (2 × ~440 MB fastq.gz, `theta 0`), indexed with the current binary, to find the partition count that gives prod-sized MPHFs (`--partitions` is a power of two; MPHF size ≈ 2.4 bits/key **[measured]**):
|
||||
|
||||
| `-p` | keys / partition | `mphf.bin` / partition | index wall | peak RSS |
|
||||
|---|---|---|---|---|
|
||||
| 256 (default) | ≈ 1.4 M **[computed from p=16]** | ≈ 0.4 MB **[computed]** | – | – |
|
||||
| 16 | 22.1 M | 6.65 MB | 120 s | 12.3 GB |
|
||||
| 4 | 86–90 M | 25.9–27.0 MB | 171 s | 19.5 GB |
|
||||
|
||||
Total ≈ 354 M distinct kmers for one skim. `-p 4` is already in the 30–45 MB range of the prod `layer_1` MPHFs; `-p 2` would give ≈ 53 MB. Indexes kept in `benchmark/run/bench_mphf/skim_p16` and `skim_p4` (2.3 GB each). Note the index stage itself is dominated by graph and MPHF construction, as in §11.
|
||||
Executable
+152
@@ -0,0 +1,152 @@
|
||||
#!/usr/bin/env bash
|
||||
# Usage: bench_mphf.sh LABEL
|
||||
#
|
||||
# Measures the obikmer commands that sit on MPHF lookups (see
|
||||
# DevDocMD/implementation/mphf_batch_queries.md for the site inventory), on the
|
||||
# 20-genome benchmark data already built under run/ (`gmake all` first).
|
||||
# Nothing under run/ other than run/bench_mphf/ is written.
|
||||
#
|
||||
# Env:
|
||||
# BINARY obikmer binary to measure (default ../src/target/release/obikmer)
|
||||
# REPS repetitions per step (default 3)
|
||||
# CPU_MAX passed as --cpu-max when set (default: all cores)
|
||||
#
|
||||
# Output: run/bench_mphf/LABEL/results.csv (label,step,rep,wall_s,maxrss_b)
|
||||
# run/bench_mphf/LABEL/checksums.txt (order-independent output digests,
|
||||
# to be diffed across LABELs: a speed-up must not change any of them)
|
||||
set -euo pipefail
|
||||
|
||||
LABEL="$1"
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
RUN_DIR="${SCRIPT_DIR}/run"
|
||||
BINARY="${BINARY:-${SCRIPT_DIR}/../src/target/release/obikmer}"
|
||||
REPS="${REPS:-3}"
|
||||
OUT="${RUN_DIR}/bench_mphf/${LABEL}"
|
||||
CPU=()
|
||||
[[ -n "${CPU_MAX:-}" ]] && CPU=(--cpu-max "${CPU_MAX}")
|
||||
|
||||
rm -rf "${OUT}"; mkdir -p "${OUT}"
|
||||
RESULTS="${OUT}/results.csv"
|
||||
SUMS="${OUT}/checksums.txt"
|
||||
echo "label,step,rep,wall_s,maxrss_b" >"${RESULTS}"
|
||||
: >"${SUMS}"
|
||||
|
||||
# timed STEP REP CMD... — wall time + peak RSS via BSD time(1); stdout of CMD
|
||||
# goes to $OUT/last.stdout, stderr to $OUT/last.stderr.
|
||||
timed() {
|
||||
local step="$1" rep="$2"; shift 2
|
||||
local tf="${OUT}/time.tmp"
|
||||
/usr/bin/time -l -o "${tf}" "$@" >"${OUT}/last.stdout" 2>"${OUT}/last.stderr" \
|
||||
|| { echo "FAILED: $step: $*" >&2; cat "${OUT}/last.stderr" >&2; exit 1; }
|
||||
local wall rss
|
||||
wall=$(awk '/ real /{print $1}' "${tf}")
|
||||
rss=$(awk '/maximum resident set size/{print $1}' "${tf}")
|
||||
echo "${LABEL},${step},${rep},${wall},${rss}" >>"${RESULTS}"
|
||||
# per-stage wall times from the Reporter log lines ("done stage=X wall_secs=Y")
|
||||
sed $'s/\x1b\\[[0-9;]*m//g' "${OUT}/last.stderr" \
|
||||
| awk -v L="${LABEL}" -v S="${step}" -v R="${rep}" \
|
||||
'/ done stage=/{ for(i=1;i<=NF;i++){ if($i ~ /^stage=/) st=substr($i,7); if($i ~ /^wall_secs=/) w=substr($i,11) } print L","S":"st","R","w",0" }' \
|
||||
>>"${RESULTS}"
|
||||
printf ' %-28s rep %s %8s s\n' "${step}" "${rep}" "${wall}" >&2
|
||||
}
|
||||
|
||||
digest() { LC_ALL=C sort "$1" | md5 -q; }
|
||||
|
||||
# ── 0. data: the indexes under run/ may have been built by an older
|
||||
# MPHF generation (ptr_hash 1.x, unreadable by current binaries), so the
|
||||
# bench builds its own, once, from the simulated reads. Shared by all LABELs;
|
||||
# delete run/bench_mphf/data to force a rebuild.
|
||||
DATA="${RUN_DIR}/bench_mphf/data"
|
||||
if [[ ! -f "${DATA}/.ready" ]]; then
|
||||
echo "preparing data with ${BINARY}" >&2
|
||||
mkdir -p "${DATA}/presence" "${DATA}/count"
|
||||
for d in "${RUN_DIR}"/simulated_data/*/*/; do
|
||||
strain="$(basename "$d")"; species="$(basename "$(dirname "$d")")"
|
||||
sp="${species}--${strain}"
|
||||
for kind in presence count; do
|
||||
extra=(); [[ $kind == count ]] && extra=(--with-counts)
|
||||
"${BINARY}" "${CPU[@]}" index --output "${DATA}/${kind}/${sp}" --force \
|
||||
--theta 0 "${extra[@]}" --label "${sp}" --meta "species=${species}" \
|
||||
"$d/reads_R1.fastq.gz" "$d/reads_R2.fastq.gz" 2>/dev/null
|
||||
done
|
||||
done
|
||||
"${BINARY}" "${CPU[@]}" merge --output "${DATA}/global_presence" --force \
|
||||
--force-presence "${DATA}"/presence/*/ 2>/dev/null
|
||||
"${BINARY}" "${CPU[@]}" merge --output "${DATA}/global_count" --force \
|
||||
"${DATA}"/count/*/ 2>/dev/null
|
||||
touch "${DATA}/.ready"
|
||||
fi
|
||||
sources=("${DATA}"/presence/*/)
|
||||
|
||||
# ── 1. index: build pass 2 (mphf.index per kmer + evidence writes) ──────────
|
||||
for sp in Escherichia_coli--K-12_MG1655 Candidozyma_auris--GCF_003013715.1_ASM301371v2; do
|
||||
species="${sp%%--*}"; strain="${sp#*--}"
|
||||
reads="${RUN_DIR}/simulated_data/${species}/${strain}"
|
||||
for rep in $(seq "${REPS}"); do
|
||||
timed "index_${species}" "${rep}" "${BINARY}" "${CPU[@]}" index \
|
||||
--output "${OUT}/idx_${species}" --force --theta 0 --label "${sp}" \
|
||||
"${reads}/reads_R1.fastq.gz" "${reads}/reads_R2.fastq.gz"
|
||||
done
|
||||
done
|
||||
|
||||
# ── 2. merge: find_batch_grouped, build_graph filter, hash_batch, index_batch ─
|
||||
for rep in $(seq "${REPS}"); do
|
||||
timed "merge_20_presence" "${rep}" "${BINARY}" "${CPU[@]}" merge \
|
||||
--output "${OUT}/merged" --force --force-presence "${sources[@]}"
|
||||
done
|
||||
"${BINARY}" "${CPU[@]}" dump "${OUT}/merged" >"${OUT}/merged.dump"
|
||||
echo "merged_dump $(digest "${OUT}/merged.dump")" >>"${SUMS}"
|
||||
rm -f "${OUT}/merged.dump"
|
||||
|
||||
# ── 3. query: query stage 1 (find per kmer, layer by layer) ─────────────────
|
||||
for kind in presence count; do
|
||||
for sp in Escherichia_coli/K-12_MG1655 Saccharolobus_islandicus/M.16.4; do
|
||||
reads="${RUN_DIR}/query_data/${sp}"
|
||||
for rep in $(seq "${REPS}"); do
|
||||
timed "query_${kind}_${sp%%/*}" "${rep}" "${BINARY}" "${CPU[@]}" query \
|
||||
--count-missing "${DATA}/global_${kind}" \
|
||||
"${reads}/reads_R1.fastq.gz" "${reads}/reads_R2.fastq.gz"
|
||||
done
|
||||
echo "query_${kind}_${sp%%/*} $(digest "${OUT}/last.stdout")" >>"${SUMS}"
|
||||
done
|
||||
done
|
||||
|
||||
# ── 4. filter: iter_layer_kmers → hash_batch + fill_sub_matrix ──────────────
|
||||
for rep in $(seq "${REPS}"); do
|
||||
timed "filter_species" "${rep}" "${BINARY}" "${CPU[@]}" filter \
|
||||
--output "${OUT}/filtered" --force --ingroup "species=Escherichia_coli" \
|
||||
--outgroup all --min-frac 0.5 --max-frac 1.0 --max-outgroup-count 0 \
|
||||
"${DATA}/global_presence"
|
||||
done
|
||||
"${BINARY}" "${CPU[@]}" dump "${OUT}/filtered" >"${OUT}/filtered.dump"
|
||||
echo "filtered_dump $(digest "${OUT}/filtered.dump")" >>"${SUMS}"
|
||||
rm -f "${OUT}/filtered.dump"
|
||||
|
||||
# ── 5. convert: build_exact_evidence / build_approx_evidence ────────────────
|
||||
# On an APFS clone (cp -c) so run/global_index_presence is untouched.
|
||||
for rep in $(seq "${REPS}"); do
|
||||
rm -rf "${OUT}/conv"; cp -cR "${DATA}/global_presence" "${OUT}/conv"
|
||||
timed "convert_to_approx8" "${rep}" "${BINARY}" "${CPU[@]}" convert \
|
||||
--approx-evidence 8 "${OUT}/conv"
|
||||
timed "convert_to_exact" "${rep}" "${BINARY}" "${CPU[@]}" convert \
|
||||
--exact-evidence "${OUT}/conv"
|
||||
done
|
||||
"${BINARY}" "${CPU[@]}" dump "${OUT}/conv" >"${OUT}/conv.dump"
|
||||
echo "convert_roundtrip_dump $(digest "${OUT}/conv.dump")" >>"${SUMS}"
|
||||
rm -rf "${OUT}/conv" "${OUT}/conv.dump"
|
||||
|
||||
# ── 6. phylo: sibling annex (annex.rs: find_in_partition per variant) and
|
||||
# its consumer (family_scan.rs: hash_batch / find_slot) ──────────────────
|
||||
for rep in $(seq "${REPS}"); do
|
||||
rm -rf "${OUT}/phy"; cp -cR "${DATA}/global_presence" "${OUT}/phy"
|
||||
timed "phylo_sibling_annex" "${rep}" "${BINARY}" "${CPU[@]}" phylo \
|
||||
--sibling-annex "${OUT}/phy"
|
||||
timed "phylo_snp_distance" "${rep}" "${BINARY}" "${CPU[@]}" phylo \
|
||||
--output "${OUT}/snp" --distance snp-jc --csv "${OUT}/phy"
|
||||
done
|
||||
echo "phylo_snp_dist $(digest "${OUT}/snp_dist.csv")" >>"${SUMS}"
|
||||
rm -rf "${OUT}/phy" "${OUT}/idx_"* "${OUT}/merged" "${OUT}/filtered" \
|
||||
"${OUT}/last.stdout" "${OUT}/time.tmp"
|
||||
|
||||
echo "results: ${RESULTS}" >&2
|
||||
echo "checksums: ${SUMS}" >&2
|
||||
@@ -0,0 +1,14 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Usage: summarize_bench.py LABEL [LABEL...] — median wall per step, side by side."""
|
||||
import csv, statistics, sys, os
|
||||
base = os.path.join(os.path.dirname(os.path.abspath(__file__)), "run", "bench_mphf")
|
||||
data = {}
|
||||
for lab in sys.argv[1:]:
|
||||
for r in csv.DictReader(open(f"{base}/{lab}/results.csv")):
|
||||
data.setdefault(r["step"], {}).setdefault(lab, []).append(float(r["wall_s"]))
|
||||
print(f"{'step':32}" + "".join(f"{l:>14}" for l in sys.argv[1:]) + (" ratio" if len(sys.argv) > 2 else ""))
|
||||
for step, d in data.items():
|
||||
med = [statistics.median(d[l]) for l in sys.argv[1:]]
|
||||
line = f"{step:32}" + "".join(f"{m:14.2f}" for m in med)
|
||||
if len(med) > 1 and med[0]: line += f" {med[-1]/med[0]:.3f}"
|
||||
print(line)
|
||||
@@ -48,6 +48,7 @@ nav:
|
||||
- obipipeline library: implementation/obipipeline.md
|
||||
- On-disk storage: implementation/storage.md
|
||||
- MPHF selection: implementation/mphf.md
|
||||
- "MPHF batch queries (inventory)": implementation/mphf_batch_queries.md
|
||||
- Unitig evidence encoding: implementation/unitig_evidence.md
|
||||
- Evidence elimination (discussion): implementation/evidence_elimination.md
|
||||
- obikindex layer tier: implementation/layer_tier.md
|
||||
|
||||
@@ -199,6 +199,17 @@ impl KmerLayer {
|
||||
}
|
||||
}
|
||||
|
||||
/// Batch form of [`find_slot`](Self::find_slot): `out[i]` is
|
||||
/// `find_slot(kmers[i])`, MPHF lookups prefetched together. `out` is
|
||||
/// cleared and reused.
|
||||
pub fn find_slot_batch(&self, kmers: &[CanonicalKmer], out: &mut Vec<Option<usize>>) {
|
||||
match self {
|
||||
KmerLayer::Count { layer, .. } => layer.find_slot_batch(kmers, out),
|
||||
KmerLayer::Presence { layer, .. } => layer.find_slot_batch(kmers, out),
|
||||
KmerLayer::Empty { .. } => panic!("Layer::find_slot_batch() called on an Empty layer"),
|
||||
}
|
||||
}
|
||||
|
||||
/// Raw MPHF batch lookup: kmer → slot, no membership check — for
|
||||
/// callers that already know every kmer is a member of *this* layer, so
|
||||
/// the evidence check `find_slot`/`find` would perform is redundant.
|
||||
|
||||
@@ -3,6 +3,7 @@ pub mod evidence;
|
||||
pub mod fingerprint;
|
||||
pub mod meta;
|
||||
pub(crate) mod mphf_layer;
|
||||
pub(crate) mod slotted;
|
||||
pub mod typed_layer;
|
||||
pub(crate) mod utils;
|
||||
|
||||
|
||||
@@ -14,6 +14,7 @@ use crate::index::error::{OKIError, OKIResult};
|
||||
use crate::layer::evidence::{Evidence, EvidenceWriter};
|
||||
use crate::layer::fingerprint::{FingerprintVec, FingerprintVecWriter};
|
||||
use crate::layer::meta::IndexMode;
|
||||
use crate::layer::slotted::SlottedKmers;
|
||||
|
||||
pub(crate) const MPHF_FILE: &str = "mphf.bin";
|
||||
pub(crate) const UNITIGS_FILE: &str = "unitigs.bin";
|
||||
@@ -162,7 +163,7 @@ pub struct MphfLayer {
|
||||
/// not a bug).
|
||||
unitigs: Arc<UnitigFileReader>,
|
||||
|
||||
mphf: MemCase<Mphf>,
|
||||
mphf: MphfOnly,
|
||||
ev: OpenLayerEvidence,
|
||||
n: usize,
|
||||
}
|
||||
@@ -175,8 +176,7 @@ impl MphfLayer {
|
||||
/// not stored: `KmerLayer`, one level up, is what owns a layer's
|
||||
/// directory as a persistent field, not this type.
|
||||
pub fn open(dir: &Path) -> OKIResult<Self> {
|
||||
let mphf: MemCase<Mphf> = unsafe { Mphf::mmap(&dir.join(MPHF_FILE), Flags::empty()) }
|
||||
.map_err(|e| OKIError::InvalidData { context: "layer", detail: e.to_string() })?;
|
||||
let mphf = MphfOnly::open(dir)?;
|
||||
let unitigs = Arc::new(UnitigFileReader::open(&dir.join(UNITIGS_FILE))?);
|
||||
let ev = LayerEvidence::at(dir)?.open()?;
|
||||
let n = match &ev {
|
||||
@@ -195,7 +195,14 @@ impl MphfLayer {
|
||||
/// - Hybrid: fingerprint check (fast path), zero false positives via `find_strict`.
|
||||
#[inline]
|
||||
pub fn find(&self, kmer: CanonicalKmer) -> Option<usize> {
|
||||
let slot = self.mphf.uncase().index(&kmer.raw());
|
||||
self.confirm(self.mphf.index(kmer), kmer)
|
||||
}
|
||||
|
||||
/// Membership check of `kmer` against the slot the MPHF gave it — the
|
||||
/// mode-dependent half of [`find`](Self::find), shared with
|
||||
/// [`find_batch`](Self::find_batch).
|
||||
#[inline]
|
||||
fn confirm(&self, slot: usize, kmer: CanonicalKmer) -> Option<usize> {
|
||||
if slot >= self.n {
|
||||
return None;
|
||||
}
|
||||
@@ -219,13 +226,29 @@ impl MphfLayer {
|
||||
}
|
||||
}
|
||||
|
||||
/// Batch form of [`find`](Self::find): `out[i]` is `find(kmers[i])`.
|
||||
///
|
||||
/// The MPHF lookups of the whole slice go through `ptr_hash`'s prefetching
|
||||
/// stream (see [`MphfOnly::for_each_slot`]), so their random pilot reads
|
||||
/// overlap instead of being paid one after the other; the membership
|
||||
/// check runs on each slot as it comes out. `out` is cleared and reused.
|
||||
pub fn find_batch(&self, kmers: &[CanonicalKmer], out: &mut Vec<Option<usize>>) {
|
||||
out.clear();
|
||||
out.reserve(kmers.len());
|
||||
let mut i = 0;
|
||||
self.mphf.for_each_slot(kmers, |slot| {
|
||||
out.push(self.confirm(slot, kmers[i]));
|
||||
i += 1;
|
||||
});
|
||||
}
|
||||
|
||||
/// Always-exact lookup — zero false positives regardless of mode.
|
||||
///
|
||||
/// - Exact/Hybrid: O(1) via evidence + `verify_canonical_kmer`.
|
||||
/// - Approx: O(n) sequential scan of `unitigs.bin` to confirm the kmer
|
||||
/// that owns the slot, then exact comparison.
|
||||
pub fn find_strict(&self, kmer: CanonicalKmer) -> Option<usize> {
|
||||
let slot = self.mphf.uncase().index(&kmer.raw());
|
||||
let slot = self.mphf.index(kmer);
|
||||
if slot >= self.n {
|
||||
return None;
|
||||
}
|
||||
@@ -247,7 +270,7 @@ impl MphfLayer {
|
||||
// the second file-open + mmap + kmer-count scan a fresh
|
||||
// `open_sequential` call would repeat on every call.
|
||||
for (stored, _, _) in self.unitigs.iter_indexed_canonical_kmers() {
|
||||
if self.mphf.uncase().index(&stored.raw()) == slot {
|
||||
if self.mphf.index(stored) == slot {
|
||||
return if stored == kmer { Some(slot) } else { None };
|
||||
}
|
||||
}
|
||||
@@ -278,16 +301,16 @@ impl MphfLayer {
|
||||
/// when the kmer is actually present in the layer; callers that need an
|
||||
/// existence test should use [`find`](Self::find) instead.
|
||||
pub fn hash(&self, kmer: CanonicalKmer) -> usize {
|
||||
self.mphf.uncase().index(&kmer.raw())
|
||||
self.mphf.index(kmer)
|
||||
}
|
||||
|
||||
/// Batch raw MPHF lookup: kmers → slots.
|
||||
///
|
||||
/// Returns a [`Vec`] of slots, one per input kmer, in the same order as the
|
||||
/// input slice. Like [`index`](Self::index), no membership check is
|
||||
/// performed.
|
||||
/// input slice. Like [`hash`](Self::hash), no membership check is
|
||||
/// performed. Prefetching — see [`MphfOnly::index_batch`].
|
||||
pub fn hash_batch(&self, kmers: &[CanonicalKmer]) -> Vec<usize> {
|
||||
kmers.iter().map(|k| self.mphf.uncase().index(&k.raw())).collect()
|
||||
self.mphf.index_batch(kmers)
|
||||
}
|
||||
|
||||
/// Iterate over all canonical kmers in the layer, in deterministic order.
|
||||
@@ -445,10 +468,27 @@ impl MphfOnly {
|
||||
self.0.uncase().index(&kmer.raw())
|
||||
}
|
||||
|
||||
/// Batch form of [`index`](Self::index) — one MPHF lookup pass over the
|
||||
/// whole slice instead of one call per kmer.
|
||||
/// Batch form of [`index`](Self::index): one prefetching `ptr_hash`
|
||||
/// stream over the whole slice, so the random pilot reads of different
|
||||
/// kmers overlap instead of being paid one after the other.
|
||||
pub fn index_batch(&self, kmers: &[CanonicalKmer]) -> Vec<usize> {
|
||||
kmers.iter().map(|&k| self.index(k)).collect()
|
||||
let mut slots = Vec::with_capacity(kmers.len());
|
||||
self.for_each_slot(kmers, |slot| slots.push(slot));
|
||||
slots
|
||||
}
|
||||
|
||||
/// Calls `f(slot)` for each kmer of `kmers`, in order.
|
||||
///
|
||||
/// Callback-shaped on purpose: `ptr_hash`'s `index_stream` only
|
||||
/// implements `fold` (its `next()` is `unimplemented!`), so it cannot be
|
||||
/// returned as an ordinary iterator without a panic trap for callers.
|
||||
/// Look-ahead of 32 keys, as recommended by `ptr_hash`.
|
||||
#[inline]
|
||||
pub fn for_each_slot(&self, kmers: &[CanonicalKmer], f: impl FnMut(usize)) {
|
||||
self.0
|
||||
.uncase()
|
||||
.index_stream::<32, _>(kmers.iter().map(|k| k.raw()))
|
||||
.for_each(f);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -478,8 +518,7 @@ impl MphfLayer {
|
||||
let mut ev = EvidenceWriter::new(n);
|
||||
let mut seen = vec![0u8; (n + 7) / 8];
|
||||
|
||||
for (kmer, chunk_id, rank) in unitigs.iter_indexed_canonical_kmers() {
|
||||
let slot = mphf.index(&kmer.raw());
|
||||
for (slot, _, chunk_id, rank) in SlottedKmers::new(&mphf, unitigs.iter_indexed_canonical_kmers()) {
|
||||
if slot >= n {
|
||||
return Err(OKIError::InvalidData { context: "mphf", detail: "slot out of bounds".into() });
|
||||
}
|
||||
@@ -521,8 +560,7 @@ impl MphfLayer {
|
||||
|
||||
let mut fw = FingerprintVecWriter::new(n, b);
|
||||
|
||||
for (kmer, _, _) in unitigs.iter_indexed_canonical_kmers() {
|
||||
let slot = mphf.index(&kmer.raw());
|
||||
for (slot, kmer, _, _) in SlottedKmers::new(&mphf, unitigs.iter_indexed_canonical_kmers()) {
|
||||
if slot >= n {
|
||||
return Err(OKIError::InvalidData { context: "mphf", detail: "slot out of bounds".into() });
|
||||
}
|
||||
@@ -592,8 +630,7 @@ impl MphfLayer {
|
||||
match mode {
|
||||
IndexMode::Exact => {
|
||||
let mut ev = EvidenceWriter::new(n);
|
||||
for (kmer, chunk_id, rank) in unitigs2.iter_indexed_canonical_kmers() {
|
||||
let slot = mphf.index(&kmer.raw());
|
||||
for (slot, kmer, chunk_id, rank) in SlottedKmers::new(&mphf, unitigs2.iter_indexed_canonical_kmers()) {
|
||||
if slot >= n {
|
||||
return Err(OKIError::InvalidData { context: "mphf", detail: "slot out of bounds".into() });
|
||||
}
|
||||
@@ -612,8 +649,7 @@ impl MphfLayer {
|
||||
|
||||
IndexMode::Approx { b, .. } => {
|
||||
let mut fw = FingerprintVecWriter::new(n, *b);
|
||||
for (kmer, _, _) in unitigs2.iter_indexed_canonical_kmers() {
|
||||
let slot = mphf.index(&kmer.raw());
|
||||
for (slot, kmer, _, _) in SlottedKmers::new(&mphf, unitigs2.iter_indexed_canonical_kmers()) {
|
||||
if slot >= n {
|
||||
return Err(OKIError::InvalidData { context: "mphf", detail: "slot out of bounds".into() });
|
||||
}
|
||||
@@ -632,8 +668,7 @@ impl MphfLayer {
|
||||
IndexMode::Hybrid { b, .. } => {
|
||||
let mut ev = EvidenceWriter::new(n);
|
||||
let mut fw = FingerprintVecWriter::new(n, *b);
|
||||
for (kmer, chunk_id, rank) in unitigs2.iter_indexed_canonical_kmers() {
|
||||
let slot = mphf.index(&kmer.raw());
|
||||
for (slot, kmer, chunk_id, rank) in SlottedKmers::new(&mphf, unitigs2.iter_indexed_canonical_kmers()) {
|
||||
if slot >= n {
|
||||
return Err(OKIError::InvalidData { context: "mphf", detail: "slot out of bounds".into() });
|
||||
}
|
||||
|
||||
@@ -0,0 +1,58 @@
|
||||
//! A layer's kmers streamed together with their MPHF slot — the shape every
|
||||
//! build pass needs (`slot` indexes evidence, fingerprint and bitmap writes).
|
||||
|
||||
use obikseq::CanonicalKmer;
|
||||
|
||||
use crate::layer::mphf_layer::Mphf;
|
||||
|
||||
/// Kmers per `ptr_hash` stream: large enough to amortise its ring-buffer
|
||||
/// start-up, small enough to stay in L1/L2 and never hold the layer.
|
||||
const CHUNK: usize = 4096;
|
||||
|
||||
/// `(slot, kmer, chunk_id, rank)` for every `(kmer, chunk_id, rank)` of the
|
||||
/// wrapped iterator, in the same order. The MPHF lookups are done a chunk at
|
||||
/// a time through `ptr_hash`'s prefetching stream, so their random pilot
|
||||
/// reads overlap; memory stays bounded by one chunk.
|
||||
pub(crate) struct SlottedKmers<'a, I> {
|
||||
mphf: &'a Mphf,
|
||||
kmers: I,
|
||||
chunk: Vec<(CanonicalKmer, usize, usize)>,
|
||||
slots: Vec<usize>,
|
||||
pos: usize,
|
||||
}
|
||||
|
||||
impl<'a, I: Iterator<Item = (CanonicalKmer, usize, usize)>> SlottedKmers<'a, I> {
|
||||
pub(crate) fn new(mphf: &'a Mphf, kmers: I) -> Self {
|
||||
Self {
|
||||
mphf,
|
||||
kmers,
|
||||
chunk: Vec::with_capacity(CHUNK),
|
||||
slots: Vec::with_capacity(CHUNK),
|
||||
pos: 0,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl<I: Iterator<Item = (CanonicalKmer, usize, usize)>> Iterator for SlottedKmers<'_, I> {
|
||||
type Item = (usize, CanonicalKmer, usize, usize);
|
||||
|
||||
fn next(&mut self) -> Option<Self::Item> {
|
||||
if self.pos == self.chunk.len() {
|
||||
self.chunk.clear();
|
||||
self.chunk.extend(self.kmers.by_ref().take(CHUNK));
|
||||
if self.chunk.is_empty() {
|
||||
return None;
|
||||
}
|
||||
self.slots.clear();
|
||||
let slots = &mut self.slots;
|
||||
self.mphf
|
||||
.index_stream::<32, _>(self.chunk.iter().map(|(k, _, _)| k.raw()))
|
||||
.for_each(|slot| slots.push(slot));
|
||||
self.pos = 0;
|
||||
}
|
||||
let (kmer, chunk_id, rank) = self.chunk[self.pos];
|
||||
let slot = self.slots[self.pos];
|
||||
self.pos += 1;
|
||||
Some((slot, kmer, chunk_id, rank))
|
||||
}
|
||||
}
|
||||
@@ -189,6 +189,13 @@ impl<D: LayerData> TypedLayer<D> {
|
||||
self.kmerstore.find(kmer)
|
||||
}
|
||||
|
||||
/// Batch form of [`find_slot`](Self::find_slot): `out[i]` is
|
||||
/// `find_slot(kmers[i])`, with the MPHF lookups prefetched together
|
||||
/// (see `MphfLayer::find_batch`). `out` is cleared and reused.
|
||||
pub fn find_slot_batch(&self, kmers: &[CanonicalKmer], out: &mut Vec<Option<usize>>) {
|
||||
self.kmerstore.find_batch(kmers, out)
|
||||
}
|
||||
|
||||
pub fn n(&self) -> usize {
|
||||
self.kmerstore.n()
|
||||
}
|
||||
|
||||
@@ -134,16 +134,48 @@ impl QueryPartition for IndexCache {
|
||||
let mut by_layer: Vec<HashMap<usize, &Vec<KmerDesc>>> =
|
||||
(0..n_layer).map(|_| HashMap::new()).collect();
|
||||
|
||||
for (kmer, descs) in kmers {
|
||||
stats.n_unique_kmers += 1;
|
||||
// Layer-major over fixed-size chunks of the (unordered) kmer map:
|
||||
// each layer sees the chunk's still-unresolved kmers as one batch, so
|
||||
// its MPHF lookups are prefetched together; a kmer found in an
|
||||
// earlier layer never reaches the later ones (first hit wins, as
|
||||
// before). Chunked, not collected — the map is never copied whole.
|
||||
const CHUNK: usize = 4096;
|
||||
let mut entries = kmers.iter();
|
||||
let mut pending: Vec<(CanonicalKmer, &Vec<KmerDesc>)> = Vec::with_capacity(CHUNK);
|
||||
let mut batch: Vec<CanonicalKmer> = Vec::with_capacity(CHUNK);
|
||||
let mut found: Vec<Option<usize>> = Vec::with_capacity(CHUNK);
|
||||
loop {
|
||||
pending.clear();
|
||||
pending.extend(entries.by_ref().take(CHUNK).map(|(k, d)| (*k, d)));
|
||||
if pending.is_empty() {
|
||||
break;
|
||||
}
|
||||
stats.n_unique_kmers += pending.len();
|
||||
for l in 0..n_layer {
|
||||
stats.n_mphf_calls += 1;
|
||||
if let Some(slot) = self.find_in_layer(part_idx, l, *kmer) {
|
||||
if pending.is_empty() {
|
||||
break;
|
||||
}
|
||||
let layer = self
|
||||
.get_layer(part_idx, l)
|
||||
.expect("layer within n_layer(part_idx)");
|
||||
batch.clear();
|
||||
batch.extend(pending.iter().map(|&(k, _)| k));
|
||||
stats.n_mphf_calls += batch.len();
|
||||
layer.find_slot_batch(&batch, &mut found);
|
||||
let mut i = 0;
|
||||
pending.retain(|&(_, descs)| {
|
||||
let hit = found[i];
|
||||
i += 1;
|
||||
match hit {
|
||||
Some(slot) => {
|
||||
by_layer[l].insert(slot, descs);
|
||||
on_event(QueryHit::Found(descs));
|
||||
stats.n_hits += 1;
|
||||
break;
|
||||
false
|
||||
}
|
||||
None => true,
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
Reference in new issue
Block a user