OBIKmers/obikmer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 34 Wiki Activity

feat: enable index resumption and enforce directory creation

Browse Source 
The command now supports reopening existing indexes instead of failing when the output file exists. Control flow branches between opening an existing index and constructing a new one, moving configuration setup exclusively to the creation path. Directory existence is enforced upfront with proper I/O error propagation. The --force flag retains its original semantics by removing the target directory before proceeding with a fresh build.
This commit is contained in:
Eric Coissac
2026-08-21 05:06:38 +02:00
parent abc51c2add
commit 5c1584967f
161 changed files with 5274 additions and 845 deletions
Download Patch File Download Diff File Expand all files Collapse all files
DevDoc/implementation/benchmark_query_testing/index.html
+31 -3
View File
@@ -12,7 +12,7 @@
<link rel="prev" href="../obitaxonomy/">
<link rel="next" href="../../architecture/sequences/invariant/">
<link rel="next" href="../partition_layer_cache/">
@@ -733,14 +733,14 @@
<li class="md-nav__item">
<a href="../obilayeredmap/" class="md-nav__link">
<a href="../layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -1113,6 +1113,34 @@
<li class="md-nav__item">
<a href="../partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
DevDoc/implementation/chunkreader.refs/index.html
+30 -2
View File
@@ -727,14 +727,14 @@
<li class="md-nav__item">
<a href="../obilayeredmap/" class="md-nav__link">
<a href="../layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -972,6 +972,34 @@
<li class="md-nav__item">
<a href="../partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
DevDoc/implementation/chunkreader/index.html
+14 -2
View File
@@ -405,11 +405,11 @@
</a>
</li>
<li class="md-nav__item">
<a class="md-nav__link" href="../obilayeredmap/">
<a class="md-nav__link" href="../layer_tier/">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -509,6 +509,18 @@
</span>
</a>
</li>
<li class="md-nav__item">
<a class="md-nav__link" href="../partition_layer_cache/">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
DevDoc/implementation/evidence_elimination.refs/index.html
+33 -5
View File
@@ -727,14 +727,14 @@
<li class="md-nav__item">
<a href="../obilayeredmap/" class="md-nav__link">
<a href="../layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -972,6 +972,34 @@
<li class="md-nav__item">
<a href="../partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
@@ -1242,9 +1270,9 @@
<h1 id="coverage-implementationevidence_eliminationmd">Coverage: implementation/evidence_elimination.md</h1>
<h2 id="code-couvert">Code couvert</h2>
<ul>
<li><code>obilayeredmap/src/fingerprint.rs</code> — FingerprintVec, FingerprintVecWriter, stockage b bits/slot, matches()</li>
<li><code>obilayeredmap/src/mphf_layer.rs</code> — build_approx_evidence(dir, b, z), find_approx()</li>
<li><code>obilayeredmap/src/meta.rs</code> — EvidenceKind::Approx { b, z }, LayerMeta</li>
<li><code>obikindex/src/layer/fingerprint.rs</code> — FingerprintVec, FingerprintVecWriter, stockage b bits/slot, matches()</li>
<li><code>obikindex/src/layer/mphf_layer.rs</code> — build_approx_evidence(dir, b, z), find_approx()</li>
<li><code>obikindex/src/layer/meta.rs</code> — EvidenceKind::Approx { b, z }, LayerMeta</li>
<li><code>obikindex/src/reindex.rs</code> — KmerIndex::reindex(), conversion exact↔approx en place</li>
<li><code>obikmer/src/cmd/reindex.rs</code> — CLI reindex, options --approx, -z, --evidence-bits, --fp, --block-size</li>
<li><code>obikmer/src/cmd/index.rs</code> — resolve_approx_params(), options --approx, -z, --evidence-bits, --fp</li>
DevDoc/implementation/evidence_elimination/index.html
+31 -3
View File
@@ -12,7 +12,7 @@
<link rel="prev" href="../unitig_evidence/">
<link rel="next" href="../obilayeredmap/">
<link rel="next" href="../layer_tier/">
@@ -868,14 +868,14 @@
<li class="md-nav__item">
<a href="../obilayeredmap/" class="md-nav__link">
<a href="../layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -1113,6 +1113,34 @@
<li class="md-nav__item">
<a href="../partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
DevDoc/implementation/filtering/index.html
+32 -4
View File
@@ -733,14 +733,14 @@
<li class="md-nav__item">
<a href="../obilayeredmap/" class="md-nav__link">
<a href="../layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -1236,6 +1236,34 @@
<li class="md-nav__item">
<a href="../partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
@@ -2158,7 +2186,7 @@ obikmer<span class="w"> </span>phylo<span class="w"> </span>myindex<span class="
<h2 id="implementation">Implementation</h2>
<ul>
<li>
<p><strong><code>obikpartitionner::filter::GroupQuorumFilter</code></strong> — implements <code>KmerFilter</code>
<p><strong><code>obikindex::partition::filter::GroupQuorumFilter</code></strong> — implements <code>KmerFilter</code>
using pre-computed ingroup and outgroup index vectors. The heavy logic
(predicate parsing, three-value evaluation, genome classification) happens
once before any iteration; each k-mer row evaluation is a simple index
@@ -2171,7 +2199,7 @@ obikmer<span class="w"> </span>phylo<span class="w"> </span>myindex<span class="
list.</p>
</li>
<li>
<p><strong><code>obikpartitionner::KmerPartition::iter_partition_kmers</code></strong> — accepts
<p><strong><code>obikindex::partition::KmerPartition::iter_partition_kmers</code></strong> — accepts
<code>filters: &amp;[Box&lt;dyn KmerFilter&gt;]</code> and applies them per-kmer before invoking
the callback. <code>filter</code>, <code>dump</code>, and <code>unitig</code> all go through this single
entry point.</p>
DevDoc/implementation/kmer.refs/index.html
+30 -2
View File
@@ -727,14 +727,14 @@
<li class="md-nav__item">
<a href="../obilayeredmap/" class="md-nav__link">
<a href="../layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -972,6 +972,34 @@
<li class="md-nav__item">
<a href="../partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
DevDoc/implementation/kmer/index.html
+30 -2
View File
@@ -868,14 +868,14 @@
<li class="md-nav__item">
<a href="../obilayeredmap/" class="md-nav__link">
<a href="../layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -1113,6 +1113,34 @@
<li class="md-nav__item">
<a href="../partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
DevDoc/implementation/obilayeredmap.refs/index.html → DevDoc/implementation/layer_tier.refs/index.html
+40 -12
View File
@@ -19,7 +19,7 @@
<title>Obilayeredmap.refs - obikmer</title>
<title>Layer tier.refs - obikmer</title>
@@ -60,7 +60,7 @@
<div data-md-component="skip">
<a href="#coverage-implementationobilayeredmapmd" class="md-skip">
<a href="#coverage-implementationlayer_tiermd" class="md-skip">
Skip to content
</a>
@@ -96,7 +96,7 @@
<div class="md-header__topic" data-md-component="header-topic">
<span class="md-ellipsis">
Obilayeredmap.refs
Layer tier.refs
</span>
</div>
@@ -727,14 +727,14 @@
<li class="md-nav__item">
<a href="../obilayeredmap/" class="md-nav__link">
<a href="../layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -972,6 +972,34 @@
<li class="md-nav__item">
<a href="../partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
@@ -1239,15 +1267,15 @@
<!-- coverage sidecar — ne pas ajouter au nav mkdocs -->
<h1 id="coverage-implementationobilayeredmapmd">Coverage: implementation/obilayeredmap.md</h1>
<h1 id="coverage-implementationlayer_tiermd">Coverage: implementation/layer_tier.md</h1>
<h2 id="code-couvert">Code couvert</h2>
<ul>
<li><code>obilayeredmap/src/mphf_layer.rs</code> — MphfLayer, LayerEvidence enum (Exact/Approx), find(), find_exact(), find_approx()</li>
<li><code>obilayeredmap/src/layer.rs</code> — Layer<D>, trait LayerData, modes () / PersistentCompactIntMatrix / PersistentBitMatrix, build(), build_evidence(), append_genome_column()</li>
<li><code>obilayeredmap/src/map.rs</code> — LayeredMap<D>, push_layer(), query()</li>
<li><code>obilayeredmap/src/evidence.rs</code> — Evidence, EvidenceWriter, encodage chunk_id:rank</li>
<li><code>obilayeredmap/src/fingerprint.rs</code> — FingerprintVec, FingerprintVecWriter, matches()</li>
<li><code>obilayeredmap/src/meta.rs</code> — LayerMeta, EvidenceKind (Exact / Approx { b, z })</li>
<li><code>obikindex/src/layer/mphf_layer.rs</code> — MphfLayer, LayerEvidence enum (Exact/Approx), find(), find_exact(), find_approx()</li>
<li><code>obikindex/src/layer/typed_layer.rs</code> — Layer<D>, trait LayerData, modes () / PersistentCompactIntMatrix / PersistentBitMatrix, build(), build_evidence(), append_genome_column()</li>
<li><code>obikindex/src/layer/map.rs</code> — LayeredMap<D>, push_layer(), query()</li>
<li><code>obikindex/src/layer/evidence.rs</code> — Evidence, EvidenceWriter, encodage chunk_id:rank</li>
<li><code>obikindex/src/layer/fingerprint.rs</code> — FingerprintVec, FingerprintVecWriter, matches()</li>
<li><code>obikindex/src/layer/meta.rs</code> — LayerMeta, EvidenceKind (Exact / Approx { b, z })</li>
</ul>
<h2 id="notes">Notes</h2>
<p>FORT RISQUE DE DÉRIVE. C'est le fichier le plus affecté par les changements récents :
DevDoc/implementation/obilayeredmap/index.html → DevDoc/implementation/layer_tier/index.html
+35 -7
View File
@@ -23,7 +23,7 @@
<title>obilayeredmap crate - obikmer</title>
<title>obikindex layer tier - obikmer</title>
@@ -64,7 +64,7 @@
<div data-md-component="skip">
<a href="#obilayeredmap-layered-kmer-index-crate" class="md-skip">
<a href="#obikindexlayer-the-layer-tier" class="md-skip">
Skip to content
</a>
@@ -100,7 +100,7 @@
<div class="md-header__topic" data-md-component="header-topic">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
</span>
</div>
@@ -749,7 +749,7 @@
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -767,7 +767,7 @@
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -1352,6 +1352,34 @@
<li class="md-nav__item">
<a href="../partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
@@ -1923,9 +1951,9 @@
<h1 id="obilayeredmap-layered-kmer-index-crate">obilayeredmap — layered kmer index crate</h1>
<h1 id="obikindexlayer-the-layer-tier">obikindex::layer — the Layer tier</h1>
<h2 id="purpose">Purpose</h2>
<p><code>obilayeredmap</code> implements a persistent, incrementally extensible kmer index. Each layer covers a disjoint kmer set and wraps a <code>ptr_hash</code> MPHF with associated per-slot data. Adding a new dataset never rebuilds existing layers.</p>
<p><code>obikindex::layer</code> (the <code>layer/</code> submodule of the <code>obikindex</code> crate — a standalone <code>obilayeredmap</code> crate until 2026-08-21, folded back in alongside <code>obikpartition</code> as part of a broader <code>Index { Partition { Layer } }</code> submodule regrouping) implements a persistent, incrementally extensible kmer index. Each layer covers a disjoint kmer set and wraps a <code>ptr_hash</code> MPHF with associated per-slot data. Adding a new dataset never rebuilds existing layers.</p>
<hr />
<h2 id="three-usage-modes">Three usage modes</h2>
<p>The MPHF + evidence infrastructure is the same for all modes. The <strong>payload</strong> varies.</p>
DevDoc/implementation/merge.refs/index.html
+33 -5
View File
@@ -727,14 +727,14 @@
<li class="md-nav__item">
<a href="../obilayeredmap/" class="md-nav__link">
<a href="../layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -972,6 +972,34 @@
<li class="md-nav__item">
<a href="../partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
@@ -1243,9 +1271,9 @@
<h2 id="code-couvert">Code couvert</h2>
<ul>
<li><code>obikindex/src/merge.rs</code> — <code>KmerIndex::merge()</code>, validation de compatibilité d'évidence, <code>validate_evidence_compat()</code></li>
<li><code>obikpartitionner/src/merge_layer.rs</code> — <code>merge_partition()</code>, construction de la nouvelle layer, paramètre <code>block_bits</code></li>
<li><code>obikpartitionner/src/rebuild_layer.rs</code> — <code>rebuild_partition()</code>, paramètre <code>block_bits</code></li>
<li><code>obilayeredmap/src/layer.rs</code> — <code>Layer::append_genome_column()</code> (PersistentCompactIntMatrix et PersistentBitMatrix)</li>
<li><code>obikindex/src/partition/merge_layer.rs</code> — <code>merge_partition()</code>, construction de la nouvelle layer, paramètre <code>block_bits</code></li>
<li><code>obikindex/src/partition/rebuild_layer.rs</code> — <code>rebuild_partition()</code>, paramètre <code>block_bits</code></li>
<li><code>obikindex/src/layer/typed_layer.rs</code> — <code>Layer::append_genome_column()</code> (PersistentCompactIntMatrix et PersistentBitMatrix)</li>
<li><code>obicompactvec/src/intmatrix.rs</code> — <code>append_column</code> pour PersistentCompactIntMatrix</li>
<li><code>obicompactvec/src/bitmatrix.rs</code> — <code>append_column</code> pour PersistentBitMatrix</li>
</ul>
DevDoc/implementation/merge/index.html
+30 -2
View File
@@ -733,14 +733,14 @@
<li class="md-nav__item">
<a href="../obilayeredmap/" class="md-nav__link">
<a href="../layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -1185,6 +1185,34 @@
<li class="md-nav__item">
<a href="../partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
DevDoc/implementation/merge_parallelism/index.html
+30 -2
View File
@@ -733,14 +733,14 @@
<li class="md-nav__item">
<a href="../obilayeredmap/" class="md-nav__link">
<a href="../layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -1180,6 +1180,34 @@
<li class="md-nav__item">
<a href="../partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
DevDoc/implementation/mphf.refs/index.html
+32 -4
View File
@@ -727,14 +727,14 @@
<li class="md-nav__item">
<a href="../obilayeredmap/" class="md-nav__link">
<a href="../layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -972,6 +972,34 @@
<li class="md-nav__item">
<a href="../partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
@@ -1242,8 +1270,8 @@
<h1 id="coverage-implementationmphfmd">Coverage: implementation/mphf.md</h1>
<h2 id="code-couvert">Code couvert</h2>
<ul>
<li><code>obilayeredmap/src/mphf_layer.rs</code> — type Mphf (PtrHash + CubicEps + CachelineEfVec + Xx64), construction en 2 passes, <code>build()</code>, <code>build_exact_evidence()</code>, <code>build_approx_evidence()</code>, <code>build_evidence()</code></li>
<li><code>obikpartitionner/src/index_layer.rs</code> — <code>build_index_layer()</code> avec passage de <code>block_bits</code></li>
<li><code>obikindex/src/layer/mphf_layer.rs</code> — type Mphf (PtrHash + CubicEps + CachelineEfVec + Xx64), construction en 2 passes, <code>build()</code>, <code>build_exact_evidence()</code>, <code>build_approx_evidence()</code>, <code>build_evidence()</code></li>
<li><code>obikindex/src/partition/index_layer.rs</code> — <code>build_index_layer()</code> avec passage de <code>block_bits</code></li>
</ul>
<h2 id="notes">Notes</h2>
<p>FORT RISQUE DE DÉRIVE. Changements récents :
DevDoc/implementation/mphf/index.html
+33 -5
View File
@@ -946,14 +946,14 @@
<li class="md-nav__item">
<a href="../obilayeredmap/" class="md-nav__link">
<a href="../layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -1191,6 +1191,34 @@
<li class="md-nav__item">
<a href="../partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
@@ -1605,7 +1633,7 @@
<h2 id="why-two-phases-are-needed">Why two phases are needed</h2>
<p>Kmer indexing per partition proceeds in two phases. The separation is necessary because the exact number of surviving unique kmers is not known until after counting and filtering low-abundance kmers.</p>
<h3 id="phase-1-provisional-mphf-kmer-spectrum">Phase 1 — provisional MPHF + kmer spectrum</h3>
<p>Implemented in <code>obikpartitionner::KmerPartition::count_kmer()</code> → <code>count_partition()</code>.</p>
<p>Implemented in <code>obikindex::partition::KmerPartition::count_kmer()</code> → <code>count_partition()</code>.</p>
<ol>
<li><strong>External sort</strong>: read the dereplicated superkmer file; extract the raw <code>u64</code> canonical kmer value for every kmer of every superkmer. Sort in RAM-bounded chunks (adaptive budget: 40% of available RAM ÷ n_threads, minimum 1 M kmers per chunk), then k-way merge with inline dedup. Result: <code>sorted_unique.bin</code> — a flat array of f0 distinct sorted <code>u64</code> values. Exact kmer count f0 is known at this point.</li>
<li><strong>Build provisional MPHF</strong> (ptr_hash, same configuration as phase 2) over <code>sorted_unique.bin</code> using <code>new_from_par_iter</code>. Delete <code>sorted_unique.bin</code> immediately after. Persist to <code>mphf1.bin</code>.</li>
@@ -1695,7 +1723,7 @@
<hr />
<h2 id="multilayer-index-architecture">Multilayer index architecture</h2>
<h3 id="layer-structure">Layer structure</h3>
<p>Each layer is a self-contained unit. See <a href="../obilayeredmap/">obilayeredmap</a> for the full on-disk layout. The MPHF-relevant files are:</p>
<p>Each layer is a self-contained unit. See <a href="../layer_tier/">obikindex::layer</a> for the full on-disk layout. The MPHF-relevant files are:</p>
<div class="highlight"><pre><span></span><code>layer_i/
unitigs.bin — packed 2-bit nucleotide sequences (kmer evidence source)
unitigs.bin.idx — random-access block index (block_bits controls granularity)
@@ -1730,7 +1758,7 @@ MphfLayer::build_approx_evidence(dir, b, z)
Uses open_sequential(); never writes .idx
</code></pre></div>
<p>There is no <code>build_evidence</code> dispatch wrapper. Callers choose the appropriate post-hoc build directly.</p>
<p>In <code>obikpartitionner</code>, <code>build_index_layer</code> receives <code>block_bits: u8</code> from <code>IndexConfig::block_bits</code> and forwards it directly to <code>Layer::build</code> and <code>Layer::build_approx_evidence</code>.</p>
<p>In <code>obikindex::partition</code>, <code>build_index_layer</code> receives <code>block_bits: u8</code> from <code>IndexConfig::block_bits</code> and forwards it directly to <code>Layer::build</code> and <code>Layer::build_approx_evidence</code>.</p>
<h3 id="membership-verification">Membership verification</h3>
<p>ptr_hash maps any input to a valid slot — it does not natively detect absent keys. Membership is verified using the evidence entry:</p>
<ul>
DevDoc/implementation/obicompactvec/index.html
+14 -2
View File
@@ -320,11 +320,11 @@
</a>
</li>
<li class="md-nav__item">
<a class="md-nav__link" href="../obilayeredmap/">
<a class="md-nav__link" href="../layer_tier/">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -424,6 +424,18 @@
</span>
</a>
</li>
<li class="md-nav__item">
<a class="md-nav__link" href="../partition_layer_cache/">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
DevDoc/implementation/obipipeline.refs/index.html
+31 -3
View File
@@ -727,14 +727,14 @@
<li class="md-nav__item">
<a href="../obilayeredmap/" class="md-nav__link">
<a href="../layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -972,6 +972,34 @@
<li class="md-nav__item">
<a href="../partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
@@ -1247,7 +1275,7 @@
</ul>
<h2 id="notes">Notes</h2>
<p>Document stable (librairie générique, peu de risque de dérive).
Vérifier si <code>obipipeline</code> est toujours utilisé dans la phase scatter de <code>obikpartitionner</code>
Vérifier si <code>obipipeline</code> est toujours utilisé dans la phase scatter de <code>obikindex::partition</code>
ou s'il a été remplacé par Rayon dans certains chemins.</p>
DevDoc/implementation/obipipeline/index.html
+30 -2
View File
@@ -896,14 +896,14 @@
<li class="md-nav__item">
<a href="../obilayeredmap/" class="md-nav__link">
<a href="../layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -1141,6 +1141,34 @@
<li class="md-nav__item">
<a href="../partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
DevDoc/implementation/obitaxonomy/index.html
+30 -2
View File
@@ -733,14 +733,14 @@
<li class="md-nav__item">
<a href="../obilayeredmap/" class="md-nav__link">
<a href="../layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -1153,6 +1153,34 @@
<li class="md-nav__item">
<a href="../partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
DevDoc/implementation/partition_layer_cache/index.html
+2843
View File
File diff suppressed because it is too large Load Diff
DevDoc/implementation/persistent_bit_vec.refs/index.html
+30 -2
View File
@@ -727,14 +727,14 @@
<li class="md-nav__item">
<a href="../obilayeredmap/" class="md-nav__link">
<a href="../layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -972,6 +972,34 @@
<li class="md-nav__item">
<a href="../partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
DevDoc/implementation/persistent_bit_vec/index.html
+30 -2
View File
@@ -733,14 +733,14 @@
<li class="md-nav__item">
<a href="../obilayeredmap/" class="md-nav__link">
<a href="../layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -1253,6 +1253,34 @@
<li class="md-nav__item">
<a href="../partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
DevDoc/implementation/persistent_compact_int_vec.refs/index.html
+30 -2
View File
@@ -727,14 +727,14 @@
<li class="md-nav__item">
<a href="../obilayeredmap/" class="md-nav__link">
<a href="../layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -972,6 +972,34 @@
<li class="md-nav__item">
<a href="../partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
DevDoc/implementation/persistent_compact_int_vec/index.html
+31 -3
View File
@@ -9,7 +9,7 @@
<link rel="prev" href="../obilayeredmap/">
<link rel="prev" href="../layer_tier/">
<link rel="next" href="../persistent_bit_vec/">
@@ -733,14 +733,14 @@
<li class="md-nav__item">
<a href="../obilayeredmap/" class="md-nav__link">
<a href="../layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -1236,6 +1236,34 @@
<li class="md-nav__item">
<a href="../partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
DevDoc/implementation/pipeline.refs/index.html
+34 -6
View File
@@ -727,14 +727,14 @@
<li class="md-nav__item">
<a href="../obilayeredmap/" class="md-nav__link">
<a href="../layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -972,6 +972,34 @@
<li class="md-nav__item">
<a href="../partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
@@ -1242,12 +1270,12 @@
<h1 id="coverage-implementationpipelinemd">Coverage: implementation/pipeline.md</h1>
<h2 id="code-couvert">Code couvert</h2>
<ul>
<li><code>obikpartitionner/src/partition.rs</code> — estimation des paramètres (phase 0)</li>
<li><code>obikindex/src/partition/partition.rs</code> — estimation des paramètres (phase 0)</li>
<li><code>obiskbuilder/src/iter.rs</code> — scatter : filtre entropie, extraction superkmers, routage partition (phase 1)</li>
<li><code>obikpartitionner/src/filter.rs</code> — déduplication bucket-sort (phase 2)</li>
<li><code>obikpartitionner/src/kmer_sort.rs</code> — tri externe + agrégation de comptages (phase 3)</li>
<li><code>obikindex/src/partition/filter.rs</code> — déduplication bucket-sort (phase 2)</li>
<li><code>obikindex/src/partition/kmer_sort.rs</code> — tri externe + agrégation de comptages (phase 3)</li>
<li><code>obidebruinj/src/debruijn.rs</code> — graphe De Bruijn, extraction des unitigs (phase 5)</li>
<li><code>obikpartitionner/src/index_layer.rs</code> — construction MPHF + évidence (phase 6), paramètre <code>block_bits</code></li>
<li><code>obikindex/src/partition/index_layer.rs</code> — construction MPHF + évidence (phase 6), paramètre <code>block_bits</code></li>
<li><code>obikindex/src/index.rs</code> — <code>build_layers()</code>, <code>dereplicate_and_count()</code></li>
</ul>
<h2 id="notes">Notes</h2>
DevDoc/implementation/pipeline/index.html
+31 -3
View File
@@ -857,14 +857,14 @@
<li class="md-nav__item">
<a href="../obilayeredmap/" class="md-nav__link">
<a href="../layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -1102,6 +1102,34 @@
<li class="md-nav__item">
<a href="../partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
@@ -1574,7 +1602,7 @@ branching / dead-end → unitig start or end
counts/ ← PersistentCompactIntMatrix (with_counts only)
</code></pre></div>
<p><strong>Cleanup:</strong> unless <code>--keep-intermediate</code> is set, <code>remove_build_artifacts</code> deletes <code>dereplicated.skmer.zst</code>, <code>mphf1.bin</code>, and <code>counts1.bin</code> after all partitions are indexed.</p>
<p>See <a href="../obilayeredmap/">obilayeredmap</a> and <a href="../mphf/">MPHF selection</a> for data structure details.</p>
<p>See <a href="../layer_tier/">obikindex::layer</a> and <a href="../mphf/">MPHF selection</a> for data structure details.</p>
<p><strong>Query path (exact evidence):</strong></p>
<div class="highlight"><pre><span></span><code>query kmer q
→ canonical_minimizer(q) → hash → PART → part_XXXXX/
DevDoc/implementation/select/index.html
+30 -2
View File
@@ -733,14 +733,14 @@
<li class="md-nav__item">
<a href="../obilayeredmap/" class="md-nav__link">
<a href="../layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -1224,6 +1224,34 @@
<li class="md-nav__item">
<a href="../partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
DevDoc/implementation/storage.refs/index.html
+31 -3
View File
@@ -727,14 +727,14 @@
<li class="md-nav__item">
<a href="../obilayeredmap/" class="md-nav__link">
<a href="../layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -972,6 +972,34 @@
<li class="md-nav__item">
<a href="../partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
@@ -1244,7 +1272,7 @@
<ul>
<li><code>obikindex/src/meta.rs</code> — IndexMeta, IndexConfig (version, config, genomes)</li>
<li><code>obikindex/src/index.rs</code> — layout sur disque : partitions/, index.meta</li>
<li><code>obilayeredmap/src/meta.rs</code> — LayerMeta (evidence kind), PartitionMeta (n_layers)</li>
<li><code>obikindex/src/layer/meta.rs</code> — LayerMeta (evidence kind), PartitionMeta (n_layers)</li>
<li><code>obiskio/src/unitig_index.rs</code> — fichiers unitigs.bin + unitigs.bin.idx</li>
</ul>
<h2 id="notes">Notes</h2>
DevDoc/implementation/storage/index.html
+30 -2
View File
@@ -929,14 +929,14 @@
<li class="md-nav__item">
<a href="../obilayeredmap/" class="md-nav__link">
<a href="../layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -1174,6 +1174,34 @@
<li class="md-nav__item">
<a href="../partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
DevDoc/implementation/superkmer.refs/index.html
+30 -2
View File
@@ -727,14 +727,14 @@
<li class="md-nav__item">
<a href="../obilayeredmap/" class="md-nav__link">
<a href="../layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -972,6 +972,34 @@
<li class="md-nav__item">
<a href="../partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
DevDoc/implementation/superkmer/index.html
+30 -2
View File
@@ -835,14 +835,14 @@
<li class="md-nav__item">
<a href="../obilayeredmap/" class="md-nav__link">
<a href="../layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -1080,6 +1080,34 @@
<li class="md-nav__item">
<a href="../partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
DevDoc/implementation/unitig_evidence.refs/index.html
+31 -3
View File
@@ -727,14 +727,14 @@
<li class="md-nav__item">
<a href="../obilayeredmap/" class="md-nav__link">
<a href="../layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -972,6 +972,34 @@
<li class="md-nav__item">
<a href="../partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
@@ -1243,7 +1271,7 @@
<h2 id="code-couvert">Code couvert</h2>
<ul>
<li><code>obiskio/src/unitig_index.rs</code> — format unitigs.bin + unitigs.bin.idx, UnitigFileWriter, UnitigFileReader, build_unitig_idx(), DEFAULT_BLOCK_BITS=0, chemin chaud block_bits=0 dans chunk_start()</li>
<li><code>obilayeredmap/src/evidence.rs</code> — encodage Evidence (chunk_id 25 bits | rank 7 bits), EvidenceWriter</li>
<li><code>obikindex/src/layer/evidence.rs</code> — encodage Evidence (chunk_id 25 bits | rank 7 bits), EvidenceWriter</li>
<li><code>obidebruinj/src/debruijn.rs</code> — extraction unitigs, chunking à MAX_KMERS_PER_CHUNK</li>
</ul>
<h2 id="notes">Notes</h2>
DevDoc/implementation/unitig_evidence/index.html
+30 -2
View File
@@ -957,14 +957,14 @@
<li class="md-nav__item">
<a href="../obilayeredmap/" class="md-nav__link">
<a href="../layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -1202,6 +1202,34 @@
<li class="md-nav__item">
<a href="../partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>