Push rwqsmuvystym #24

Merged
coissac merged 3 commits from push-rwqsmuvystym into main 2026-06-12 19:33:21 +00:00

3 Commits

Author SHA1 Message Date
Eric Coissac b2c8373586 refactor: parallelize merge layer with WorkerPool pipeline
Replaces the synchronous sequential loop with a multi-threaded pipeline using `WorkerPool` and custom stage macros. Shared mutable state is wrapped in `Arc<Mutex<>>` for thread-safe updates, while pipeline errors are centralized via `Arc<Mutex<Option<String>>>` to propagate failures before thread join.
2026-06-12 21:32:53 +02:00
Eric Coissac ba49af6f9e refactor: parallelize merge and partition logic with obipipeline
Introduce the `obipipeline` dependency and refactor merge and partition logic to leverage parallel execution. Update `merge_partitions` to use rayon with dynamic memory budgeting and concurrency control via a pilot run. Refactor Pass 1 to concurrently read unitigs, filter kmers through a shared `LayeredMap`, and populate the graph safely. Simplify diagnostics to report total kmer counts and replace manual flags with graph length validation.
2026-06-12 21:32:04 +02:00
Eric Coissac 2bc189e962 feat: dynamically compute seed expansion based on RSS
Introduce a `peak_rss_bytes()` utility for accurate per-phase RAM measurement. Replace the genome-length heuristic with a dynamic seed expansion ratio based on actual RSS delta. Explicitly drop the `GraphDeBruijn` instance before MPHF construction to prevent resource contention and ensure proper memory management.
2026-06-12 16:39:38 +02:00