refactor(benchmark): consolidate generated artifacts under run/ directory

Restructure the benchmark pipeline to direct all simulated data, indices, statistics, and query outputs into a unified `run/` directory. Update Makefile targets, shell scripts, and Python utilities to resolve paths relative to this new base. Adjust documentation and dependency tracking to match the revised layout, and remove outdated temporary artifacts.

.kilo/plans/1778319532254-eager-panda.md

@@ -0,0 +1,87 @@
+# Plan d'amélioration technique - obiskio
+
+## 1. Contexte et objectifs
+- **Objectif** : Renforcer la robustesse, la maintenabilité et les performances de la crate `obiskio`.
+- **Priorités** : 
+  1. Gestion des erreurs
+  2. Optimisation de la mémoire du pool
+  3. Robustesse concurrente
+  4. Couverture de tests
+  5. Documentation
+
+---
+
+## 2. Axes d'amélioration détaillés
+
+### 2.1 Gestion des erreurs
+- **Problème** : `SKError` ne couvre pas tous les cas (format invalide, taille maximale, CRC)
+- **Actions** :
+  - Ajouter variante `ParseError(String)` dans `src/error.rs`
+  - Valider les tailles de SuperKmer avant parsing
+  - Remplacer `expect()` par `unwrap_or_else` avec messages explicites
+  - Documenter chaque variante d’erreur dans le README
+
+### 2.2 Optimisation du pool de fichiers
+- **Problème** : `SKFilePool` utilise un `Vec<WriteEntry>` non contraint et n’effectue pas de nettoyage en cas d’erreur
+- **Actions** :
+  - Implémenter un `LimitedVec` avec limite stricte à `MAX_POOL_SIZE`
+  - Créer `clear_memory()` qui supprime les entrées orphelines
+  - Ajouter `evict_lru_threshold()` pour éviction proactive
+  - Introduire un `RwLock` pour les opérations de lecture massives
+
+### 2.3 Robustesse concurrente
+- **Problème** : Risque de deadlocks dans `SKFileWriter::write_batch()` et `SKFileReader::reopen_and_seek()`
+- **Actions** :
+  - Remplacer `Mutex` par `RwLock` pour les accès en lecture
+  - Ajouter un compteur de blocage et logs de timeout
+  - Utiliser `std::thread::park_timeout` pour débloquer
+  - Insérer `debug_assert!` sur les états invariants
+
+### 2.4 Couverture de tests
+- **Problème** : Absence de benchmarks, de tests de migration, de résilience de fichiers corrompus
+- **Actions** :
+  - Benchmarks I/O sur 10k+ SuperKmer avec `criterion`
+  - Tests de migration de version de fichier `.meta` → `.v2.meta`
+  - Tests de corruption volontaire (truncature, inversion de bits)
+  - Tests de stress sur pool saturation (100 threads)
+
+### 2.5 Documentation & exemples
+- **Actions** :
+  - Ajouter des examples dans chaque module (`# Examples`)
+  - Documenter la logique LRU avec diagrammes Mermaid
+  - Créer un guide « How to recover from eviction »
+  - Mettre à jour le `README.md` avec tableau des variantes d’erreur
+
+---
+
+## 3. Plan d'exécution (Roadmap)
+
+| Sprint | Durée | Livrables clés |
+|--------|-------|----------------|
+| **S1** | 2 jours | Refactorisation `SKError`, ajout de tests unitaires |
+| **S2** | 3 jours | Implémentation `clear_memory()` + `LimitedVec` |
+| **S3** | 2 jours | Passage à `RwLock`, ajout de compteurs de blocage |
+| **S4** | 2 jours | Benchmarks + tests de migration |
+| **S5** | 1 jour | Documentation finale & mise à jour du README |
+
+---
+
+## 4. Dépendances externes
+- Mettre à jour `niffler` vers la version 2.0 (performance compression)
+- Évaluer `bincode` vs `serde_json` pour les métas (I/O)
+- Ajouter dépendance `criterion` (dev‑dependencies)
+
+---
+
+## 5. KPI de suivi
+- **Couverture de tests** : ≥85 % des chemins critiques
+- **Latence moyenne d’écriture** : ↓15 % après optimisation du pool
+- **Taux d’erreurs résolues** : 100 % des nouvelles variantes couvertes
+- **Temps de build CI** : ≤5 min pour l’ensemble des benchmarks
+
+---
+
+## 6. Validation finale
+- Revue de code avec `cargo clippy -- -D warnings`
+- Analyse de toxicité avec `cargo deny open-source-licenses`
+- Vérification de la conformité aux standards de naming du projet

.serena/.gitignore

@@ -0,0 +1,2 @@
+/cache
+/project.local.yml

.serena/project.yml

@@ -0,0 +1,169 @@
+# the name by which the project can be referenced within Serena/when chatting with the LLM.
+project_name: "obikmer"
+
+# the encoding used by text files in the project
+# For a list of possible encodings, see https://docs.python.org/3.11/library/codecs.html#standard-encodings
+encoding: "utf-8"
+
+# line ending convention to use when writing source files.
+# Possible values: unset (use global setting), "lf", "crlf", or "native" (platform default)
+# This does not affect Serena's own files (e.g. memories and configuration files), which always use native line endings.
+line_ending:
+
+# The language backend to use for this project.
+# If not set, the global setting from serena_config.yml is used.
+# Valid values: LSP, JetBrains
+# Note: the backend is fixed at startup. If a project with a different backend
+# is activated post-init, an error will be returned.
+language_backend:
+
+# whether to use project's .gitignore files to ignore files
+ignore_all_files_in_gitignore: true
+
+# advanced configuration option allowing to configure language server-specific options.
+# Maps the language key to the options.
+# The settings are considered only if the project is trusted (see global configuration to define trusted projects).
+# See https://oraios.github.io/serena/02-usage/050_configuration.html#language-server-specific-settings
+ls_specific_settings: {}
+
+# list of additional paths to ignore in this project.
+# Same syntax as gitignore, so you can use * and **.
+# Important: quote patterns that start with `*`, otherwise YAML treats them as aliases.
+# Example:
+#   ignored_paths:
+#     - "examples/**"
+#     - ".worktrees/**"
+#     - "**/bin/**"
+#     - "**/obj/**"
+# Note: global ignored_paths from serena_config.yml are also applied additively.
+ignored_paths: []
+
+# whether the project is in read-only mode
+# If set to true, all editing tools will be disabled and attempts to use them will result in an error
+# Added on 2025-04-18
+read_only: false
+
+# list of tool names to exclude.
+# This extends the existing exclusions (e.g. from the global configuration)
+# Find the list of tools here: https://oraios.github.io/serena/01-about/035_tools.html
+excluded_tools: []
+
+# list of tools to include that would otherwise be disabled (particularly optional tools that are disabled by default).
+# This extends the existing inclusions (e.g. from the global configuration).
+# Find the list of tools here: https://oraios.github.io/serena/01-about/035_tools.html
+included_optional_tools: []
+
+# fixed set of tools to use as the base tool set (if non-empty), replacing Serena's default set of tools.
+# This cannot be combined with non-empty excluded_tools or included_optional_tools.
+# Find the list of tools here: https://oraios.github.io/serena/01-about/035_tools.html
+fixed_tools: []
+
+# list of mode names that are to be activated by default, overriding the setting in the global configuration.
+# The full set of modes to be activated is base_modes (from global config) + default_modes + added_modes.
+# If the setting is undefined/empty, the default_modes from the global configuration (serena_config.yml) apply.
+# Otherwise, this overrides the setting from the global configuration (serena_config.yml).
+# Therefore, you can set this to [] if you do not want the default modes defined in the global config to apply
+# for this project.
+# This setting can, in turn, be overridden by CLI parameters (--mode).
+# See https://oraios.github.io/serena/02-usage/050_configuration.html#modes
+default_modes:
+
+# list of mode names to be activated additionally for this project, e.g. ["query-projects"]
+# The full set of modes to be activated is base_modes (from global config) + default_modes + added_modes.
+# See https://oraios.github.io/serena/02-usage/050_configuration.html#modes
+added_modes:
+
+# initial prompt for the project. It will always be given to the LLM upon activating the project
+# (contrary to the memories, which are loaded on demand).
+initial_prompt: ""
+
+# time budget (seconds) per tool call for the retrieval of additional symbol information
+# such as docstrings or parameter information.
+# This overrides the corresponding setting in the global configuration; see the documentation there.
+# If null or missing, use the setting from the global configuration.
+symbol_info_budget:
+
+# list of regex patterns which, when matched, mark a memory entry as read‑only.
+# Extends the list from the global configuration, merging the two lists.
+read_only_memory_patterns: []
+
+# list of regex patterns for memories to completely ignore.
+# Matching memories will not appear in list_memories or activate_project output
+# and cannot be accessed via read_memory or write_memory.
+# To access ignored memory files, use the read_file tool on the raw file path.
+# Extends the list from the global configuration, merging the two lists.
+# Example: ["_archive/.*", "_episodes/.*"]
+ignored_memory_patterns: []
+
+# list of additional workspace folder paths for cross-package reference support.
+# Paths can be absolute or relative to the project root.
+# Each folder is registered as an LSP workspace folder, enabling language servers to discover
+# symbols and references across package boundaries, but these folders are not indexed by Serena,
+# i.e. the respective symbols will not be found using Serena's symbol search tools.
+# Example:
+#   additional_workspace_folders:
+#     - ../sibling-package
+#     - ../shared-lib
+ls_additional_workspace_folders: []
+
+# list of language servers to start when using the LSP backend; choose from:
+#   ada                    al                     angular                ansible                bash
+#   bsl                    clojure                cpp                    cpp_ccls               crystal
+#   csharp                 csharp_omnisharp       cue                    dart                   deno
+#   elixir                 elm                    erlang                 fortran                fsharp
+#   gdscript               gleam                  go                     groovy                 haskell
+#   haxe                   hlsl                   html                   java                   json
+#   julia                  kotlin                 latex                  lean4                  lua
+#   luau                   markdown               matlab                 msl                    nextflow
+#   nix                    ocaml                  pascal                 perl                   php
+#   php_phpactor           php_phpantom           powershell             python                 python_basedpyright
+#   python_jedi            python_pyrefly         python_ty              qml                    r
+#   rego                   ruby                   ruby_solargraph        rust                   scala
+#   scss                   solidity               svelte                 swift                  systemverilog
+#   terraform              toml                   typescript             typescript_vts         vue
+#   wolfram                yaml                   zig
+#   (This list may be outdated; generated with scripts/print_language_list.py;
+#   For the current list, see values of the LanguageServerId enum here:
+#   https://github.com/oraios/serena/blob/main/src/solidlsp/ls_config.py)
+# For some languages, there are several alternative language servers, e.g. csharp_omnisharp, ruby_solargraph.)
+# Note:
+#   - For C, use cpp
+#   - For JavaScript, use typescript
+#   - For Angular projects, use angular (subsumes typescript+html; requires `npm install` in the project root)
+#   - For Svelte projects, use svelte (subsumes typescript/javascript for .svelte projects; requires npm)
+#   - For Deno projects, use deno (serves the same .ts/.js files as typescript; requires the deno CLI on PATH)
+#   - For SCSS / Sass / plain CSS, use scss (some-sass-language-server handles all three)
+#   - For Free Pascal/Lazarus, use pascal
+# Special requirements:
+#   Some language servers require additional setup/installations.
+#   See here for details: https://oraios.github.io/serena/01-about/020_programming-languages.html#language-servers
+# When using multiple language servers, the first language server that supports a given file will be used for that file.
+# The first language server is the default language and the respective language server will be used as a fallback.
+# Note that when using the JetBrains backend, language servers are not used and this list is correspondingly ignored.
+language_servers:
+- rust
+
+# list of workspace folder paths (LSP backend only).
+# These folders will be used to build up Serena's symbol index.
+# Paths must be within the project root and should thus be relative to the project root.
+# Furthermore, the paths should not be filtered by ignore settings.
+# Default setting: The entire project root folder (".") is considered.
+# In (large) monorepos, this can be used to index only subfolders of the project root, e.g.
+#   ls_workspace_folders:
+#     - "./subproject1"
+#     - "./subproject2"
+ls_workspace_folders:
+- .
+
+# optional shell command to run before the language backend (LSP or JetBrains) is initialised.
+# the command runs in the project root directory and is only executed if the project is trusted
+# (see trusted_project_path_patterns in the global configuration).
+# serena waits for the command to exit: a non-zero exit code is logged as an error but does not
+# abort activation. a per-project timeout (activation_command_timeout, default 180s) is the safety
+# backstop for non-terminating commands; on expiry the process is killed and activation continues.
+# example: activation_command: "npx nx run-many -t build"
+activation_command:
+
+# maximum time in seconds to wait for activation_command to complete before killing it (default 180s).
+# must be a positive number.
+activation_command_timeout: 180.0

.zed/tasks.json

@@ -0,0 +1,59 @@
+// Project tasks configuration. See https://zed.dev/docs/tasks for documentation.
+//
+// Example:
+[
+  {
+    "label": "Example task",
+    "command": "for i in {1..5}; do echo \"Hello $i/5\"; sleep 1; done",
+    //"args": [],
+    // Env overrides for the command, will be appended to the terminal's environment from the settings.
+    "env": { "foo": "bar" },
+    // Current working directory to spawn the command into, defaults to current project root.
+    //"cwd": "/path/to/working/directory",
+    // Whether to use a new terminal tab or reuse the existing one to spawn the process, defaults to `false`.
+    "use_new_terminal": false,
+    // Whether to allow multiple instances of the same task to be run, or rather wait for the existing ones to finish, defaults to `false`.
+    "allow_concurrent_runs": false,
+    // What to do with the terminal pane and tab, after the command was started:
+    // * `always` — always show the task's pane, and focus the corresponding tab in it (default)
+    // * `no_focus` — always show the task's pane, add the task's tab in it, but don't focus it
+    // * `never` — do not alter focus, but still add/reuse the task's tab in its pane
+    "reveal": "always",
+    // Where to place the task's terminal item after starting the task:
+    // * `dock` — in the terminal dock, "regular" terminal items' place (default)
+    // * `center` — in the central pane group, "main" editor area
+    "reveal_target": "dock",
+    // What to do with the terminal pane and tab, after the command had finished:
+    // * `never` — Do nothing when the command finishes (default)
+    // * `always` — always hide the terminal tab, hide the pane also if it was the last tab in it
+    // * `on_success` — hide the terminal tab on task success only, otherwise behaves similar to `always`
+    "hide": "never",
+    // Which shell to use when running a task inside the terminal.
+    // May take 3 values:
+    // 1. (default) Use the system's default terminal configuration in /etc/passwd
+    //      "shell": "system"
+    // 2. A program:
+    //      "shell": {
+    //        "program": "sh"
+    //      }
+    // 3. A program with arguments:
+    //     "shell": {
+    //         "with_arguments": {
+    //           "program": "/bin/bash",
+    //           "args": ["--login"]
+    //         }
+    //     }
+    "shell": "system",
+    // Whether to show the task line in the output of the spawned task, defaults to `true`.
+    "show_summary": true,
+    // Whether to show the command line in the output of the spawned task, defaults to `true`.
+    "show_command": true,
+    // Which edited buffers to save before running the task:
+    // * `all` — save all edited buffers
+    // * `current` — save currently active buffer only
+    // * `none` — don't save any buffers
+    "save": "none",
+    // Represents the tags for inline runnable indicators, or spawning multiple tasks at once.
+    // "tags": []
+  },
+]

memory/MEMORY.md

@@ -0,0 +1,5 @@
+# Memory Index
+
+- [Project domain](project_domain.md) — obikmer est pour la génomique (génomes individuels), pas la métagénomique
+- [No architectural decisions without authorization](feedback_architectural_decisions.md) — toute décision architecturale (mémoire, algo, structure) requiert l'accord explicite de l'utilisateur avant toute action
+- [Phases intra-partition parallèles](feedback_phases_parallelism.md) — graph build, compute_degrees, unitig traversal, MPHF utilisent Rayon — ne jamais les appeler "séquentielles"

memory/feedback_architectural_decisions.md

@@ -0,0 +1,17 @@
+---
+name: No architectural decisions without explicit authorization
+description: Never make architectural or design decisions without explicit user approval — code decisions are the user's alone
+type: feedback
+---
+
+Never make architectural decisions unilaterally. This includes:
+- Memory layout or footprint changes
+- Algorithm or data structure choices (HashSet vs streaming, etc.)
+- Dependency additions or substitutions
+- Structural refactors that go beyond the exact task requested
+
+If a bug or inefficiency is observed, **report it and propose alternatives** — do not fix it without explicit authorization.
+
+**Why:** The user optimizes for minimal memory footprint at all times. Introducing a HashSet in `count_kmer()` (replacing the intended streaming GOFunction construction from the sidecar estimate) caused a serious memory regression that went unreported. This is inadmissible on a project where memory efficiency is a core constraint.
+
+**How to apply:** When editing code and noticing an architectural issue (even a clear improvement), stop, describe the problem and options, and wait for explicit go-ahead before touching anything.

memory/feedback_phases_parallelism.md

@@ -0,0 +1,12 @@
+---
+name: feedback-phases-parallelism
+description: Les phases intra-partition (graph build, compute_degrees, unitig traversal, MPHF) utilisent toutes Rayon — elles ne sont PAS séquentielles
+metadata:
+  type: feedback
+---
+
+Ne jamais qualifier les phases intra-partition de "séquentielles". Chaque phase (graph build, compute_degrees, unitig traversal, MPHF build) utilise Rayon en interne et s'exécute en parallèle sur plusieurs cœurs.
+
+**Why:** L'utilisateur a corrigé ce point plusieurs fois. Le décrire comme "séquentiel" est une erreur factuelle qui fausse l'analyse de performance.
+
+**How to apply:** Quand on analyse l'efficacité CPU ou les 25% manquants, chercher la cause dans le déséquilibre de charge entre partitions, la contention Rayon entre workers, ou la latence inter-partitions — pas dans une prétendue sérialisation des phases.

src/Cargo.lock

@@ -1480,7 +1480,7 @@ dependencies = [
 
 [[package]]
 name = "obikmer"
-version = "1.3.1"
+version = "1.2.3"
 dependencies = [
  "clap",
  "csv",

src/obikdump/Cargo.toml

@@ -11,5 +11,5 @@ obikseq       = { path = "../obikseq" }
 obidebruinj   = { path = "../obidebruinj" }
 obifastwrite  = { path = "../obifastwrite" }
 obicompactvec = { path = "../obicompactvec" }
-obisys        = { path = "../obisys", default-features = false }
+obisys        = { path = "../obisys" }
 rayon         = "1"

src/obikfilter/Cargo.toml

@@ -14,5 +14,5 @@ obikidxcache  = { path = "../obikidxcache" }
 obikindexer   = { path = "../obikindexer" }
 obidebruinj   = { path = "../obidebruinj" }
 obikalgorithm = { path = "../obikalgorithm" }
-obisys        = { path = "../obisys", default-features = false }
+obisys        = { path = "../obisys" }
 tracing       = "0.1.44"

src/obikindex/Cargo.toml

@@ -7,7 +7,7 @@ edition = "2024"
 obikseq          = { path = "../obikseq" }
 obitaxonomy      = { path = "../obitaxonomy" }
 obiskio          = { path = "../obiskio" }
-obisys           = { path = "../obisys", default-features = false }
+obisys           = { path = "../obisys" }
 obicompactvec    = { path = "../obicompactvec" }
 obidebruinj      = { path = "../obidebruinj" }
 obipipeline      = { path = "../obipipeline" }

src/obikindexer/Cargo.toml

@@ -9,7 +9,7 @@ obikalgorithm    = { path = "../obikalgorithm" }
 obikseq          = { path = "../obikseq" }
 obidebruinj      = { path = "../obidebruinj" }
 obiskio          = { path = "../obiskio" }
-obisys           = { path = "../obisys", default-features = false }
+obisys           = { path = "../obisys" }
 obicompactvec    = { path = "../obicompactvec" }
 obipipeline      = { path = "../obipipeline" }
 obiread          = { path = "../obiread" }

src/obikmer/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "obikmer"
-version = "1.3.1"
+version = "1.2.3"
 edition = "2024"
 
 [[bin]]
@@ -11,7 +11,7 @@ path = "src/main.rs"
 obikseq       = { path = "../obikseq" }
 obiread       = { path = "../obiread" }
 obipipeline   = { path = "../obipipeline" }
-obisys        = { path = "../obisys", default-features = false }
+obisys        = { path = "../obisys" }
 obikindex     = { path = "../obikindex", default-features = false }
 obikindexer   = { path = "../obikindexer" }
 obikalgorithm = { path = "../obikalgorithm" }

src/obikmerge/Cargo.toml

@@ -11,7 +11,7 @@ obicompactvec = { path = "../obicompactvec" }
 obiskio       = { path = "../obiskio" }
 obikseq       = { path = "../obikseq" }
 obipipeline   = { path = "../obipipeline" }
-obisys        = { path = "../obisys", default-features = false }
+obisys        = { path = "../obisys" }
 rayon         = "1"
 tracing       = "0.1.44"
 

src/obikphylo/Cargo.toml

@@ -7,7 +7,7 @@ edition = "2024"
 obikindex        = { path = "../obikindex", default-features = false }
 obikseq          = { path = "../obikseq" }
 obiskio          = { path = "../obiskio" }
-obisys           = { path = "../obisys", default-features = false }
+obisys           = { path = "../obisys" }
 obicompactvec    = { path = "../obicompactvec" }
 obikidxcache     = { path = "../obikidxcache" }
 obiskbuilder     = { path = "../obiskbuilder" }

src/obikrebuild/Cargo.toml

@@ -10,5 +10,5 @@ obikfilter    = { path = "../obikfilter" }
 obikidxcache  = { path = "../obikidxcache" }
 obikseq       = { path = "../obikseq" }
 obidebruinj   = { path = "../obidebruinj" }
-obisys        = { path = "../obisys", default-features = false }
+obisys        = { path = "../obisys" }
 tracing       = "0.1.44"

src/obikselect/Cargo.toml

@@ -8,5 +8,5 @@ obikindex     = { path = "../obikindex" }
 obikfilter    = { path = "../obikfilter" }
 obikalgorithm = { path = "../obikalgorithm" }
 obicompactvec = { path = "../obicompactvec" }
-obisys        = { path = "../obisys", default-features = false }
+obisys        = { path = "../obisys" }
 tracing       = "0.1.44"