Implement sparse matrix optimizations and update architecture documentation

This commit introduces significant functional improvements related to sparse matrix handling, including new storage layers, performance fixes for sparse operations, and bug corrections in matrix logic. Additionally, architectural documentation is cleaned up by removing obsolete entries and modifying structural text related to the Sibling Annex discussion.
This commit is contained in:
Eric Coissac committed 2026-10-13 08:52:37 +02:00
1 parent fdfc1eb3dd
commit 95bbc0b2ae
64 files changed
+1374 -2330

No files matched your search

@@ -1265,7 +1265,7 @@
<span class="md-ellipsis">
Sibling annex (discussion)
Sibling annex
@@ -1485,8 +1485,9 @@ this pipeline.</p>
by <code>verify_presence</code>/<code>verify_merge_presence</code>, which only check index
<em>content</em> against the <code>.npz</code> truth, never the query API.</li>
<li><code>pack --sparse</code> produces a presence-matrix format documented (see
<a href="../../architecture/siblings/">siblings.md</a>) as faster for single-row
access (query) and slower for column-oriented access (phylo <code>--metric</code>).
<a href="../obicompactvec/">obicompactvec.md</a>, "Sparse matrix formats") as faster
for single-row access (query) and slower for column-oriented access
(phylo <code>--metric</code>).
<code>global_index_presence/</code> built by <code>merge_presence.sh</code> is always packed
dense (packing is a stage inside <code>merge</code>, not a separate <code>pack</code>
invocation) — there was no dense/sparse regression check.</li>