|
|
|
@@ -494,7 +494,7 @@
|
|
|
|
|
<span class="md-ellipsis">
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Sibling annex (discussion)
|
|
|
|
|
Sibling annex
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
@@ -632,6 +632,46 @@
|
|
|
|
|
</a>
|
|
|
|
|
</li>
|
|
|
|
|
<li class="md-nav__item">
|
|
|
|
|
<a class="md-nav__link" href="#sparse-matrix-formats">
|
|
|
|
|
<span class="md-ellipsis">
|
|
|
|
|
|
|
|
|
|
Sparse matrix formats
|
|
|
|
|
|
|
|
|
|
</span>
|
|
|
|
|
</a>
|
|
|
|
|
<nav aria-label="Sparse matrix formats" class="md-nav">
|
|
|
|
|
<ul class="md-nav__list">
|
|
|
|
|
<li class="md-nav__item">
|
|
|
|
|
<a class="md-nav__link" href="#persistentsparsebitmatrix">
|
|
|
|
|
<span class="md-ellipsis">
|
|
|
|
|
|
|
|
|
|
PersistentSparseBitMatrix
|
|
|
|
|
|
|
|
|
|
</span>
|
|
|
|
|
</a>
|
|
|
|
|
</li>
|
|
|
|
|
<li class="md-nav__item">
|
|
|
|
|
<a class="md-nav__link" href="#nonzero_iter-the-one-traversal-per-format">
|
|
|
|
|
<span class="md-ellipsis">
|
|
|
|
|
|
|
|
|
|
nonzero_iter — the one traversal per format
|
|
|
|
|
|
|
|
|
|
</span>
|
|
|
|
|
</a>
|
|
|
|
|
</li>
|
|
|
|
|
<li class="md-nav__item">
|
|
|
|
|
<a class="md-nav__link" href="#persistentsparsecompactintmatrix">
|
|
|
|
|
<span class="md-ellipsis">
|
|
|
|
|
|
|
|
|
|
PersistentSparseCompactIntMatrix
|
|
|
|
|
|
|
|
|
|
</span>
|
|
|
|
|
</a>
|
|
|
|
|
</li>
|
|
|
|
|
</ul>
|
|
|
|
|
</nav>
|
|
|
|
|
</li>
|
|
|
|
|
<li class="md-nav__item">
|
|
|
|
|
<a class="md-nav__link" href="#aggregation-traits-matrix-level">
|
|
|
|
|
<span class="md-ellipsis">
|
|
|
|
|
|
|
|
|
@@ -1191,7 +1231,7 @@ Pass 1 — byte max, SIMD-vectorizable, O(n)
|
|
|
|
|
<tr>
|
|
|
|
|
<td><code>Sparse</code></td>
|
|
|
|
|
<td>bit: <code>sparse_meta.json</code> + PFIV/Elias-Fano component files, row-major. Int: same support files (built on <code>PersistentSparseBitMatrix</code> internally) plus <code>singleton_values.pciv</code>/<code>multi_values.pciv</code>/<code>multi_offsets</code> for the per-row, non-deduplicated values</td>
|
|
|
|
|
<td><code>pack --sparse</code>; see <a href="../../architecture/siblings/">siblings.md</a> for the sparse-vs-dense access-pattern trade-off</td>
|
|
|
|
|
<td><code>pack --sparse</code>; see "Sparse matrix formats" below for the sparse-vs-dense access-pattern trade-off</td>
|
|
|
|
|
</tr>
|
|
|
|
|
<tr>
|
|
|
|
|
<td><code>Implicit</code> (bit only)</td>
|
|
|
|
@@ -1201,7 +1241,6 @@ Pass 1 — byte max, SIMD-vectorizable, O(n)
|
|
|
|
|
</tbody>
|
|
|
|
|
</table>
|
|
|
|
|
<p><code>PersistentBitMatrix::open(layer_dir)</code> auto-detects the variant, in order: <code>matrix.pbmx</code> → Packed, <code>presence/meta.json</code> → Columnar, <code>presence/sparse_meta.json</code> → Sparse, <code>layer_meta.json</code> (no presence dir at all) → Implicit. <code>PersistentCompactIntMatrix::open(layer_dir)</code> mirrors the same priority order minus <code>Implicit</code> (there's no implicit count matrix — counts always have at least one on-disk column): <code>matrix.pcmx</code> → Packed, <code>counts/meta.json</code> → Columnar, <code>counts/singleton_values.pciv</code> → Sparse. <code>col_view</code>/<code>col</code>/<code>sub_matrix</code> panic on <code>Sparse</code>/<code>Implicit</code> where the operation has no direct-slice equivalent (Sparse is k-mer-major, not column-major; Implicit has no backing storage) — callers needing per-column data on those variants go through <code>row</code>/<code>fill_row</code>.</p>
|
|
|
|
|
<p>Unlike the bit side, <code>PersistentSparseCompactIntMatrix</code>'s values are <em>not</em> deduplicated across rows — two rows can share the same non-zero column set (same <code>dict_id</code> in the shared support) while carrying different counts — so its <code>CountPartials</code> impl can't reuse the support's dict-multiplicity shortcut the way <code>BitPartials for PersistentSparseBitMatrix</code> does. It still avoids the naive <code>O(n_cols² × n)</code> column-pair scan via a single row-major pass (<code>row_major_pairwise</code> in <code>sparse_intmatrix.rs</code>), reconstructing the squared-difference formulas (<code>euclidean</code>/<code>relfreq_euclidean</code>/<code>hellinger</code>) from per-column marginals via <code>Σ(a-b)² = Σa²+Σb²-2Σab</code> — see <a href="../../architecture/siblings/">siblings.md</a>'s "<code>PersistentCompactIntMatrix::Sparse</code> — implemented" entry for the full derivation.</p>
|
|
|
|
|
<p><code>col_view(c)</code> returns the appropriate view directly:</p>
|
|
|
|
|
<div class="highlight"><pre><span></span><code><span class="c1">// PersistentBitMatrix</span>
|
|
|
|
|
<span class="k">pub</span><span class="w"> </span><span class="k">fn</span><span class="w"> </span><span class="nf">col_view</span><span class="p">(</span><span class="o">&</span><span class="bp">self</span><span class="p">,</span><span class="w"> </span><span class="n">c</span><span class="p">:</span><span class="w"> </span><span class="kt">usize</span><span class="p">)</span><span class="w"> </span><span class="p">-></span><span class="w"> </span><span class="nc">BitSliceView</span><span class="o"><'</span><span class="nb">_</span><span class="o">></span>
|
|
|
|
@@ -1212,6 +1251,183 @@ Pass 1 — byte max, SIMD-vectorizable, O(n)
|
|
|
|
|
<p>No wrapper enums (<code>BitColView</code>, <code>IntColView</code>): the caller receives a <code>Copy</code> view struct immediately usable with any view method or bulk builder method.</p>
|
|
|
|
|
<p><code>pack_compact_int_matrix</code> and <code>pack_bit_matrix</code> convert columnar → packed format.</p>
|
|
|
|
|
<hr/>
|
|
|
|
|
<h2 id="sparse-matrix-formats">Sparse matrix formats</h2>
|
|
|
|
|
<p>Motivated by real k-mer-family data being extremely row-sparse (most rows —
|
|
|
|
|
k-mers — present in only a handful of genome columns out of dozens to
|
|
|
|
|
thousands): a row-major, deduplicated alternative to the dense column-major
|
|
|
|
|
formats above, built once per layer like the rest of the pipeline (never the
|
|
|
|
|
whole multi-billion-row index at once). Implements the exact-duplicate
|
|
|
|
|
special case of Alanko, Bille, Gørtz, Navarro, Puglisi, "Compact Data
|
|
|
|
|
Structures for Collections of Sets" (2025,
|
|
|
|
|
<code>biblio/Alanko et al. - Compact Data Structures for Collections of
|
|
|
|
|
Sets.pdf</code>) — a plain dedup dictionary, not their full subset-containment
|
|
|
|
|
hierarchy.</p>
|
|
|
|
|
<h3 id="persistentsparsebitmatrix"><code>PersistentSparseBitMatrix</code></h3>
|
|
|
|
|
<p>Four mmap-backed on-disk components: an <code>is_multi</code> rank-capable flag per row
|
|
|
|
|
(singleton vs. multi-genome), a fixed-bit-width array for singleton rows
|
|
|
|
|
(genome index directly, <code>ceil(log2(n_cols))</code> bits), a separate fixed-width
|
|
|
|
|
array for multi-genome rows (<code>dict_id</code>, <code>ceil(log2(n_distinct_multi_sets))</code>
|
|
|
|
|
bits — kept apart from the singleton array because <code>n_distinct_multi_sets</code>
|
|
|
|
|
can be large in absolute terms even when multi-genome rows are a small
|
|
|
|
|
<em>fraction</em> of all rows, and a shared array would force every row, singletons
|
|
|
|
|
included, to pay the wider width), and a deduplicated dictionary of distinct
|
|
|
|
|
multi-genome sets (Elias-Fano-encoded byte offsets + a varint-encoded values
|
|
|
|
|
blob). Supporting primitives: <code>PersistentFixedIntVec</code> (arbitrary,
|
|
|
|
|
runtime-parameterised bit width, width <code>0</code> included — needed once an
|
|
|
|
|
<code>EliasFano</code> reopen bug surfaced, see below), <code>PersistentRankSelectBitVec</code>
|
|
|
|
|
(rank1/rank0/select1 on top of the crate's existing <code>count_ones</code>, via
|
|
|
|
|
<code>common_traits::SelectInWord</code>), <code>EliasFano</code> (composes the two). <code>BinaryMatrix</code>
|
|
|
|
|
(<code>n</code>, <code>n_cols</code>, <code>row</code>/<code>fill_row</code>, <code>fill_sub_matrix</code>, <code>count_ones</code>) is the
|
|
|
|
|
trait unifying dense and sparse at the call sites that need both
|
|
|
|
|
interchangeably; column-oriented methods (<code>col</code>, <code>col_view</code>, the
|
|
|
|
|
<code>partial_*_dist_matrix</code> family) stay dense-only — <code>Sparse</code>/<code>Implicit</code> panic
|
|
|
|
|
there (see the <code>PersistentBitMatrix::open</code> note above).</p>
|
|
|
|
|
<p>Measured on real data (one layer, 30.2M rows, 91 genomes):</p>
|
|
|
|
|
<table>
|
|
|
|
|
<thead>
|
|
|
|
|
<tr>
|
|
|
|
|
<th></th>
|
|
|
|
|
<th>dense</th>
|
|
|
|
|
<th>sparse</th>
|
|
|
|
|
<th>ratio</th>
|
|
|
|
|
</tr>
|
|
|
|
|
</thead>
|
|
|
|
|
<tbody>
|
|
|
|
|
<tr>
|
|
|
|
|
<td>on-disk size</td>
|
|
|
|
|
<td>328.1MB</td>
|
|
|
|
|
<td>43.7MB</td>
|
|
|
|
|
<td><strong>7.5x</strong> smaller</td>
|
|
|
|
|
</tr>
|
|
|
|
|
<tr>
|
|
|
|
|
<td>build time / peak RSS</td>
|
|
|
|
|
<td>—</td>
|
|
|
|
|
<td>4.26s / 628MB</td>
|
|
|
|
|
<td>per-layer, in-memory construction</td>
|
|
|
|
|
</tr>
|
|
|
|
|
<tr>
|
|
|
|
|
<td>row access, sequential (2M reads)</td>
|
|
|
|
|
<td>43ns/row</td>
|
|
|
|
|
<td>32ns/row</td>
|
|
|
|
|
<td>sparse faster (smaller structure, better cache fit)</td>
|
|
|
|
|
</tr>
|
|
|
|
|
<tr>
|
|
|
|
|
<td>row access, random (2M reads)</td>
|
|
|
|
|
<td>409ns/row</td>
|
|
|
|
|
<td>85ns/row</td>
|
|
|
|
|
<td>sparse <strong>~4.8x faster</strong> — the <code>--shannon</code>/family-lookup access shape</td>
|
|
|
|
|
</tr>
|
|
|
|
|
<tr>
|
|
|
|
|
<td>column access, one full column (30.2M rows)</td>
|
|
|
|
|
<td>11.5ms</td>
|
|
|
|
|
<td>993ms</td>
|
|
|
|
|
<td>sparse <strong>86x slower</strong> — no native column method, every read decodes a full row to keep one bit</td>
|
|
|
|
|
</tr>
|
|
|
|
|
</tbody>
|
|
|
|
|
</table>
|
|
|
|
|
<p>The row-access wins (compactness was the stated design goal, not speed) come
|
|
|
|
|
from dense's genome-major layout scattering a single row read across a much
|
|
|
|
|
bigger file — costlier than sparse's rank/select/varint decode once the file
|
|
|
|
|
is this much smaller. Column access is the flip side of the same layout
|
|
|
|
|
choice; <code>nonzero_iter</code> (below) is what removes it from the hot path that
|
|
|
|
|
actually matters (<code>query</code>).</p>
|
|
|
|
|
<p>Two reopen-correctness bugs, now fixed at the source: (1) <code>EliasFano::open</code>
|
|
|
|
|
re-derived its low-bits width from the persisted low-vector's own width
|
|
|
|
|
byte; the width-<code>0</code> case was built with a dummy 1-bit placeholder (the
|
|
|
|
|
builder used to reject true width <code>0</code>), silently doubling every reopened
|
|
|
|
|
value. Fixed by making <code>PersistentFixedIntVec</code> genuinely support width <code>0</code>
|
|
|
|
|
(no storage, <code>get</code> always <code>0</code>). (2) An empty row (cardinality 0) used to be
|
|
|
|
|
recorded as a singleton at genome <code>0</code>, indistinguishable on read-back from a
|
|
|
|
|
<em>real</em> singleton at genome <code>0</code>. Fixed by routing cardinality-0 rows through
|
|
|
|
|
the dictionary path (a genuine empty entry).</p>
|
|
|
|
|
<p>Still deferred: full Alanko et al. subset-hierarchy compression (only the
|
|
|
|
|
exact-duplicate case is built), a sparse <code>PersistentCompactIntMatrix</code>
|
|
|
|
|
variant beyond what's described below, BRWT-style column-correlation
|
|
|
|
|
exploitation, and a row-major rewrite of
|
|
|
|
|
<code>partial_jaccard_dist_matrix</code>/<code>partial_hamming_dist_matrix</code>/etc. as a
|
|
|
|
|
per-row co-occurrence accumulation (<code>O(Σ_rows k̄²)</code>) so <code>obikindex</code>'s
|
|
|
|
|
<code>--metric</code> distance-matrix path stops paying the column-access penalty on
|
|
|
|
|
<code>Sparse</code> — needs its own design pass (how it plugs into
|
|
|
|
|
<code>BitPartials</code>/<code>ColumnWeights</code> so both matrix types keep serving <code>--metric</code>),
|
|
|
|
|
not just "port the loop".</p>
|
|
|
|
|
<h3 id="nonzero_iter-the-one-traversal-per-format"><code>nonzero_iter</code> — the one traversal per format</h3>
|
|
|
|
|
<p><code>fn nonzero_iter<'a>(&'a self, slots: &'a [usize]) -> Box<dyn Iterator<Item = (usize, usize, u32)> + 'a></code>
|
|
|
|
|
(item: index into <code>slots</code>, column, value) is the single foundational
|
|
|
|
|
per-format traversal primitive every consumer builds on — <code>fill_sub_matrix</code>
|
|
|
|
|
and a <code>for_each</code>-style callback are both trivial, free consumers of it
|
|
|
|
|
(<code>.for_each(f)</code> is <code>Iterator::for_each</code>; <code>fill_sub_matrix</code> drains the
|
|
|
|
|
iterator and scatters into <code>out[][]</code>), instead of two independently
|
|
|
|
|
hand-maintained traversals that can silently diverge. <code>Box<dyn Iterator></code>,
|
|
|
|
|
not a bare <code>impl Iterator</code>: <code>Columnar</code>/<code>Packed</code>/<code>Sparse</code>/<code>Implicit</code> are
|
|
|
|
|
genuinely different concrete types and the method isn't on a
|
|
|
|
|
dyn-compatible trait (<code>BinaryMatrix</code> is used as <code>dyn BinaryMatrix</code>
|
|
|
|
|
elsewhere, and RPITIT methods aren't dyn-compatible) — one <code>Box</code> per
|
|
|
|
|
<code>nonzero_iter</code> call, not per cell.</p>
|
|
|
|
|
<p>Per-format implementation: <code>PersistentSparseBitMatrix</code> decodes natively,
|
|
|
|
|
buffered one row at a time (<code>std::iter::from_fn</code> over the existing
|
|
|
|
|
<code>for_each_genome_in_row</code>, no <code>n_cols</code>-wide allocation ever). <code>Columnar</code>/
|
|
|
|
|
<code>Packed</code> loop columns, collecting each column's hits via
|
|
|
|
|
<code>BitSliceView::nonzero_among_sorted</code>/<code>IntSliceView::nonzero_among_sorted</code>
|
|
|
|
|
(a <code>filter</code>/<code>filter_map</code> over the vector-level <code>get()</code>, no new state
|
|
|
|
|
machine — <code>std</code>'s own <code>Filter</code> iterator already is one) into one <code>Vec</code> via
|
|
|
|
|
<code>.extend()</code> (not <code>flat_map</code>: a <code>flat_map</code> closure can't lazily return
|
|
|
|
|
something borrowing its own captured sort permutation across separate calls
|
|
|
|
|
without either boxing per-column or fighting the borrow checker — eager
|
|
|
|
|
collection sidesteps it at zero extra cost, since <code>fill_sub_matrix</code> already
|
|
|
|
|
fully materialises anyway). <code>Implicit</code> is trivial (<code>slots.iter().map(|&i| (i, 0, 1))</code>).</p>
|
|
|
|
|
<p>This closed a real dispatch bug by construction, not by patching it:
|
|
|
|
|
<code>PersistentBitMatrix::fill_sub_matrix</code>/<code>sub_matrix</code> used to reimplement a
|
|
|
|
|
naive per-<code>(column, slot)</code> loop instead of delegating to
|
|
|
|
|
<code>PersistentSparseBitMatrix</code>'s own efficient native method for <code>Self::Sparse</code>
|
|
|
|
|
— two hand-written traversals for the same format, free to drift apart, and
|
|
|
|
|
they had (<code>obikphylo</code>'s own cache layer had independently built a parallel
|
|
|
|
|
enum wrapper that <em>did</em> dispatch correctly, evidence the bug was worked
|
|
|
|
|
around rather than fixed at the source). With <code>fill_sub_matrix</code>/<code>sub_matrix</code>
|
|
|
|
|
rewritten to drain <code>nonzero_iter</code>, there is only one traversal per format
|
|
|
|
|
left to get right. <code>KmerPartition::query_partition_with</code>
|
|
|
|
|
(<code>obikindex/src/partition/query_layer.rs</code>) — previously column-major
|
|
|
|
|
(<code>for g { for slot { col_value } }</code>, which for <code>Sparse</code> rebuilt a full row
|
|
|
|
|
per single cell touched, O(hits × n_cols) instead of O(hits) — the original
|
|
|
|
|
motivation for this primitive) — now issues one <code>layer.nonzero_iter(&slot_list)</code>
|
|
|
|
|
call per layer, format-agnostic. Dense/sparse query performance parity
|
|
|
|
|
confirmed on real data (previously sparse 30-50% slower than dense,
|
|
|
|
|
reproducibly; now within ~1-3% either way — noise level).</p>
|
|
|
|
|
<h3 id="persistentsparsecompactintmatrix"><code>PersistentSparseCompactIntMatrix</code></h3>
|
|
|
|
|
<p>The count-matrix analogue, built on <code>PersistentSparseBitMatrix</code> as its
|
|
|
|
|
"which columns are non-zero" support, values <strong>not</strong> deduplicated (unlike
|
|
|
|
|
the bit side, two rows can share the same non-zero column set — same
|
|
|
|
|
<code>dict_id</code> — while carrying different counts, so the dict-multiplicity
|
|
|
|
|
shortcut <code>BitPartials for PersistentSparseBitMatrix</code> uses doesn't carry
|
|
|
|
|
over). <code>CountPartials</code> here: a single row-major pass (<code>row_major_pairwise</code>
|
|
|
|
|
in <code>sparse_intmatrix.rs</code>, decodes each row once via
|
|
|
|
|
<code>for_each_cell_in_row</code>, nests over that row's own co-present columns) —
|
|
|
|
|
<code>O(Σ k̄²)</code> over populated rows, avoiding the naive <code>O(n_cols² × n)</code>
|
|
|
|
|
column-pair rescan. <code>min(a,b)</code>-based kernels (<code>bray</code>, <code>relfreq_bray</code>) need
|
|
|
|
|
no correction — they vanish when either side is absent. <code>a·b</code> and <code>√(a·b)</code>
|
|
|
|
|
kernels (<code>euclidean</code>/<code>relfreq_euclidean</code>/<code>hellinger</code>) do need one, since
|
|
|
|
|
<code>(a-0)² = a² ≠ 0</code> unlike the <code>min</code>-based formulas — reconstructed from
|
|
|
|
|
per-column marginals via <code>Σ(a-b)² = Σa²+Σb²-2Σab</code>.
|
|
|
|
|
<code>threshold_jaccard(1)</code> shortcuts to <code>BitPartials::partial_jaccard</code>
|
|
|
|
|
(threshold 1 is exactly presence); <code>threshold_jaccard(0)</code> is closed-form
|
|
|
|
|
(every <code>u32</code> is <code>≥ 0</code>).</p>
|
|
|
|
|
<p>Wiring the <code>threshold==1</code> shortcut surfaced two pre-existing formula bugs in
|
|
|
|
|
<code>BitPartials for PersistentSparseBitMatrix</code> (present since that type's own
|
|
|
|
|
introduction, never caught because no test had compared <code>Sparse</code>'s raw
|
|
|
|
|
<code>partial_*</code> output against dense on real data — only the diagonal-blind
|
|
|
|
|
<code>jaccard_dist_matrix</code>/<code>hamming_dist_matrix</code> finalisations were tested),
|
|
|
|
|
fixed at the source:</p>
|
|
|
|
|
<ol>
|
|
|
|
|
<li><code>partial_jaccard</code>'s diagonal used to be <code>(0, 2×col_weights[i])</code> instead
|
|
|
|
|
of a genuine self-comparison <code>(col_weights[i], col_weights[i])</code> —
|
|
|
|
|
<code>col_weights_and_pair_counts</code>'s <code>inter</code> never pairs a column with itself
|
|
|
|
|
by construction, so the diagonal needs its own case.</li>
|
|
|
|
|
<li><code>partial_hamming</code>'s off-diagonal formula itself was wrong: it computed
|
|
|
|
|
<code>total - union</code> (rows where <em>neither</em> column is present) instead of the
|
|
|
|
|
actual Hamming distance <code>col_weights[i] + col_weights[j] - 2×inter[i,j]</code>
|
|
|
|
|
(symmetric-difference size) — the two only coincide when
|
|
|
|
|
<code>col_weights[i] + col_weights[j] == total</code>, which small/synthetic test
|
|
|
|
|
data can easily satisfy by accident.</li>
|
|
|
|
|
</ol>
|
|
|
|
|
<h2 id="aggregation-traits-matrix-level">Aggregation traits (matrix level)</h2>
|
|
|
|
|
<h3 id="columnweights">ColumnWeights</h3>
|
|
|
|
|
<div class="highlight"><pre><span></span><code><span class="k">trait</span><span class="w"> </span><span class="n">ColumnWeights</span><span class="p">:</span><span class="w"> </span><span class="nb">Send</span><span class="w"> </span><span class="o">+</span><span class="w"> </span><span class="nb">Sync</span><span class="w"> </span><span class="p">{</span>
|
|
|
|
|