OBIKmers/obikmer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 8 Wiki Activity

feat: add memory-aware parallel merge scheduling and CLI flags

Browse Source 
Introduces a memory-aware scheduling strategy for parallel partition merging that replaces unbounded concurrency with a First-Fit Decreasing approach gated by a thread-safe `MemoryBudget` semaphore. An adaptive expansion factor, seeded by a sequential pilot run, dynamically caps concurrent workers to prevent hashbrown OOMs. Adds a `--budget-fraction` CLI flag to configure RAM allocation, enhances the CLI to accept multiple indexes, and introduces comprehensive partition diagnostics including memory utilization tracking, concurrency metrics, and statistical summaries with ASCII histograms. Updates documentation and navigation accordingly.
This commit is contained in:
Eric Coissac
2026-06-12 11:26:24 +02:00
parent f44fe042bc
commit b5e027f23b
7 changed files with 742 additions and 81 deletions
Download Patch File Download Diff File Expand all files Collapse all files
mkdocs.yml
+1
View File
@@ -49,6 +49,7 @@ nav:
- PersistentCompactIntVec: implementation/persistent_compact_int_vec.md
- PersistentBitVec: implementation/persistent_bit_vec.md
- Merge command: implementation/merge.md
- Merge parallelism & memory: implementation/merge_parallelism.md
- Kmer filtering: implementation/filtering.md
- Select command: implementation/select.md
- Architecture: