OBIKmers/obikmer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 8 Wiki Activity

refactor: replace rayon with NUMA-aware PartitionRunner

Browse Source 
Replaces `rayon` parallel iteration across index, rebuild, reindex, and select modules with a custom `PartitionRunner`. This introduces NUMA-aware task distribution with CPU pinning and round-robin scheduling, eliminating `Arc`, `Mutex`, and atomic synchronization primitives in favor of a flat, pre-spawned worker architecture. Error handling is simplified via `.map_err()` and the `?` operator, while progress bar updates are decoupled into dedicated callbacks.
This commit is contained in:
Eric Coissac
2026-06-15 15:45:04 +02:00
parent bc92dc4592
commit b6fcbc545f
8 changed files with 198 additions and 319 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docmd/installation.md
+2 -2
View File
@@ -60,13 +60,13 @@ HPC home directories are typically on a network filesystem (Lustre, NFS) optimis
**Always redirect the build directory to a local scratch disk:**
```bash
CARGO_TARGET_DIR=/scratch/local/$USER/cargo-target cargo build --release
CARGO_TARGET_DIR=/scratch/$USER/cargo-target cargo build --release
```
Adapt the path to the local scratch available on your cluster (`/var/tmp`, `/tmp`, `/scratch/local`, etc.). Once built, copy the binary to a permanent location:
```bash
cp /scratch/local/$USER/cargo-target/release/obikmer ~/bin/
cp /scratch/$USER/cargo-target/release/obikmer ~/bin/
```
## NUMA support