feat: introduce NUMA-aware PartitionRunner for adaptive parallelism

Replace NUMA-naive Rayon loops and ad-hoc adaptive pools with a unified `PartitionRunner` that manages a NUMA-aware worker pool. The implementation uses pinned Rayon thread pools per node and activates dormant threads based on real-time CPU efficiency metrics. This standardizes partition-level parallelism, optimizes memory locality, and eliminates cross-socket traffic. Includes architecture documentation and updates mkdocs navigation.
This commit is contained in:
Eric Coissac
2026-06-15 11:24:37 +02:00
parent 313d73838a
commit 7a87e911b6
3 changed files with 429 additions and 1 deletions
+1
View File
@@ -57,6 +57,7 @@ nav:
- Sequences: architecture/sequences/invariant.md
- Kmer index: architecture/index_architecture.md
- NUMA-aware worker pools: architecture/numa_worker_pools.md
- NUMA-aware partition runner: architecture/numa_partition_runner.md
watch:
- docmd