OBIKmers/obikmer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 34 Wiki Activity

feat: enable index resumption and enforce directory creation

Browse Source 
The command now supports reopening existing indexes instead of failing when the output file exists. Control flow branches between opening an existing index and constructing a new one, moving configuration setup exclusively to the creation path. Directory existence is enforced upfront with proper I/O error propagation. The --force flag retains its original semantics by removing the target directory before proceeding with a fresh build.
This commit is contained in:
Eric Coissac
2026-08-21 05:06:38 +02:00
parent abc51c2add
commit 5c1584967f
161 changed files with 5274 additions and 845 deletions
Download Patch File Download Diff File Expand all files Collapse all files
DevDoc/theory/encoding.refs/index.html
+30 -2
View File
@@ -727,14 +727,14 @@
<li class="md-nav__item">
<a href="../../implementation/obilayeredmap/" class="md-nav__link">
<a href="../../implementation/layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -972,6 +972,34 @@
<li class="md-nav__item">
<a href="../../implementation/partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
DevDoc/theory/encoding/index.html
+30 -2
View File
@@ -802,14 +802,14 @@
<li class="md-nav__item">
<a href="../../implementation/obilayeredmap/" class="md-nav__link">
<a href="../../implementation/layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -1047,6 +1047,34 @@
<li class="md-nav__item">
<a href="../../implementation/partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
DevDoc/theory/entropy.refs/index.html
+30 -2
View File
@@ -727,14 +727,14 @@
<li class="md-nav__item">
<a href="../../implementation/obilayeredmap/" class="md-nav__link">
<a href="../../implementation/layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -972,6 +972,34 @@
<li class="md-nav__item">
<a href="../../implementation/partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
DevDoc/theory/entropy/index.html
+30 -2
View File
@@ -868,14 +868,14 @@
<li class="md-nav__item">
<a href="../../implementation/obilayeredmap/" class="md-nav__link">
<a href="../../implementation/layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -1113,6 +1113,34 @@
<li class="md-nav__item">
<a href="../../implementation/partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
DevDoc/theory/evolutionary_distances/index.html
+32 -4
View File
@@ -1244,14 +1244,14 @@
<li class="md-nav__item">
<a href="../../implementation/obilayeredmap/" class="md-nav__link">
<a href="../../implementation/layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -1489,6 +1489,34 @@
<li class="md-nav__item">
<a href="../../implementation/partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
@@ -3582,7 +3610,7 @@ diagonal entry directly, at no extra lookup cost.</p>
<p>Mirror <code>distance.rs</code>: open the presence or count store per partition. But
instead of a per-partition <code>partial_*</code>, run the sequential source sweep:</p>
<div class="highlight"><pre><span></span><code>for p in 0..n_partitions: # OUTER — sequential
open source partition p&#39;s layers (QueryLayer-style, obikpartitionner)
open source partition p&#39;s layers (QueryLayer-style, obikindex::partition)
enumerate distinct canonical k-mers of p (one per MPHF slot) with their
presence/count vectors # column-major, as query stage 2
par_iter over these source k-mers: # INNER — rayon, thread-local tally
@@ -3605,7 +3633,7 @@ instead of a per-partition <code>partial_*</code>, run the sequential source swe
merge thread-local tallies into global SnpTally
</code></pre></div>
<p>The inner lookup is precisely <code>QueryLayer::find_slot</code> +
<code>col_value(g, slot)</code> (<code>obikpartitionner/src/query_layer.rs</code>) — reuse or factor
<code>col_value(g, slot)</code> (<code>obikindex/src/partition/query_layer.rs</code>) — reuse or factor
out that path rather than reimplementing MPHF access. Enumerating "all distinct
k-mers of a partition with their vectors" is the <code>dump</code>/<code>query</code> stage-2
column-major scan already implemented in <code>dump_layer.rs</code> /
DevDoc/theory/indexing.refs/index.html
+32 -4
View File
@@ -727,14 +727,14 @@
<li class="md-nav__item">
<a href="../../implementation/obilayeredmap/" class="md-nav__link">
<a href="../../implementation/layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -972,6 +972,34 @@
<li class="md-nav__item">
<a href="../../implementation/partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
@@ -1242,8 +1270,8 @@
<h1 id="coverage-theoryindexingmd">Coverage: theory/indexing.md</h1>
<h2 id="code-couvert">Code couvert</h2>
<ul>
<li><code>obikpartitionner/src/partition.rs</code> — routage par hash de minimiseur, choix des paramètres</li>
<li><code>obikpartitionner/src/lib.rs</code> — structure KmerPartition, nombre de partitions</li>
<li><code>obikindex/src/partition/partition.rs</code> — routage par hash de minimiseur, choix des paramètres</li>
<li><code>obikindex/src/partition/mod.rs</code> — structure KmerPartition, nombre de partitions</li>
</ul>
<h2 id="notes">Notes</h2>
<p>Vérifier que la doc mentionne bien que le nombre de partitions est une puissance de 2
DevDoc/theory/indexing/index.html
+30 -2
View File
@@ -802,14 +802,14 @@
<li class="md-nav__item">
<a href="../../implementation/obilayeredmap/" class="md-nav__link">
<a href="../../implementation/layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -1047,6 +1047,34 @@
<li class="md-nav__item">
<a href="../../implementation/partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
DevDoc/theory/minimizer.refs/index.html
+30 -2
View File
@@ -727,14 +727,14 @@
<li class="md-nav__item">
<a href="../../implementation/obilayeredmap/" class="md-nav__link">
<a href="../../implementation/layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -972,6 +972,34 @@
<li class="md-nav__item">
<a href="../../implementation/partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>
DevDoc/theory/minimizer/index.html
+30 -2
View File
@@ -846,14 +846,14 @@
<li class="md-nav__item">
<a href="../../implementation/obilayeredmap/" class="md-nav__link">
<a href="../../implementation/layer_tier/" class="md-nav__link">
<span class="md-ellipsis">
obilayeredmap crate
obikindex layer tier
@@ -1091,6 +1091,34 @@
<li class="md-nav__item">
<a href="../../implementation/partition_layer_cache/" class="md-nav__link">
<span class="md-ellipsis">
Partition and layer caching (discussion)
</span>
</a>
</li>
</ul>
</nav>