2026-04-16 22:38:20 +02:00
<!doctype html>
< html lang = "en" class = "no-js" >
< head >
< meta charset = "utf-8" >
< meta name = "viewport" content = "width=device-width,initial-scale=1" >
2026-08-15 20:56:29 +02:00
< link rel = "prev" href = "../siblings/" >
2026-04-16 22:38:20 +02:00
2026-08-15 20:56:29 +02:00
< link rel = "next" href = "../numa_partition_runner/" >
2026-04-16 22:38:20 +02:00
< link rel = "icon" href = "../../assets/images/favicon.png" >
< meta name = "generator" content = "mkdocs-1.6.1, mkdocs-material-9.7.6" >
2026-08-15 20:56:29 +02:00
< title > NUMA-aware worker pools - obikmer</ title >
2026-04-16 22:38:20 +02:00
< link rel = "stylesheet" href = "../../assets/stylesheets/main.484c7ddc.min.css" >
< link rel = "preconnect" href = "https://fonts.gstatic.com" crossorigin >
< link rel = "stylesheet" href = "https://fonts.googleapis.com/css?family=Roboto:300,300i,400,400i,700,700i%7CRoboto+Mono:400,400i,700,700i&display=fallback" >
< style >: root { --md-text-font : "Roboto" ; --md-code-font : "Roboto Mono" }</ style >
< script > __md_scope = new URL ( "../.." , location ), __md_hash = e =>[... e ]. reduce ((( e , _ )=>( e << 5 ) - e + _ . charCodeAt ( 0 )), 0 ), __md_get = ( e , _ = localStorage , t = __md_scope )=> JSON . parse ( _ . getItem ( t . pathname + "." + e )), __md_set = ( e , _ , t = localStorage , a = __md_scope )=>{ try { t . setItem ( a . pathname + "." + e , JSON . stringify ( _ ))} catch ( e ){}}</ script >
</ head >
< body dir = "ltr" >
< input class = "md-toggle" data-md-toggle = "drawer" type = "checkbox" id = "__drawer" autocomplete = "off" >
< input class = "md-toggle" data-md-toggle = "search" type = "checkbox" id = "__search" autocomplete = "off" >
< label class = "md-overlay" for = "__drawer" ></ label >
< div data-md-component = "skip" >
2026-08-15 20:56:29 +02:00
< a href = "#numa-aware-worker-pools-for-merge" class = "md-skip" >
2026-04-16 22:38:20 +02:00
Skip to content
</ a >
</ div >
< div data-md-component = "announce" >
</ div >
< header class = "md-header md-header--shadow" data-md-component = "header" >
< nav class = "md-header__inner md-grid" aria-label = "Header" >
< a href = "../.." title = "obikmer" class = "md-header__button md-logo" aria-label = "obikmer" data-md-component = "logo" >
< svg xmlns = "http://www.w3.org/2000/svg" viewBox = "0 0 24 24" >< path d = "M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54" /></ svg >
</ a >
< label class = "md-header__button md-icon" for = "__drawer" >
< svg xmlns = "http://www.w3.org/2000/svg" viewBox = "0 0 24 24" >< path d = "M3 6h18v2H3zm0 5h18v2H3zm0 5h18v2H3z" /></ svg >
</ label >
< div class = "md-header__title" data-md-component = "header-title" >
< div class = "md-header__ellipsis" >
< div class = "md-header__topic" >
< span class = "md-ellipsis" >
obikmer
</ span >
</ div >
< div class = "md-header__topic" data-md-component = "header-topic" >
< span class = "md-ellipsis" >
2026-08-15 20:56:29 +02:00
NUMA-aware worker pools
2026-04-16 22:38:20 +02:00
</ span >
</ div >
</ div >
</ div >
< script > var palette = __md_get ( "__palette" ); if ( palette && palette . color ){ if ( "(prefers-color-scheme)" === palette . color . media ){ var media = matchMedia ( "(prefers-color-scheme: light)" ), input = document . querySelector ( media . matches ? "[data-md-color-media='(prefers-color-scheme: light)']" : "[data-md-color-media='(prefers-color-scheme: dark)']" ); palette . color . media = input . getAttribute ( "data-md-color-media" ), palette . color . scheme = input . getAttribute ( "data-md-color-scheme" ), palette . color . primary = input . getAttribute ( "data-md-color-primary" ), palette . color . accent = input . getAttribute ( "data-md-color-accent" )} for ( var [ key , value ] of Object . entries ( palette . color )) document . body . setAttribute ( "data-md-color-" + key , value )}</ script >
</ nav >
</ header >
< div class = "md-container" data-md-component = "container" >
< main class = "md-main" data-md-component = "main" >
< div class = "md-main__inner md-grid" >
< div class = "md-sidebar md-sidebar--primary" data-md-component = "sidebar" data-md-type = "navigation" >
< div class = "md-sidebar__scrollwrap" >
< div class = "md-sidebar__inner" >
< nav class = "md-nav md-nav--primary" aria-label = "Navigation" data-md-level = "0" >
< label class = "md-nav__title" for = "__drawer" >
< a href = "../.." title = "obikmer" class = "md-nav__button md-logo" aria-label = "obikmer" data-md-component = "logo" >
< svg xmlns = "http://www.w3.org/2000/svg" viewBox = "0 0 24 24" >< path d = "M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54" /></ svg >
</ a >
obikmer
</ label >
< ul class = "md-nav__list" data-md-scrollfix >
< li class = "md-nav__item" >
< a href = "../.." class = "md-nav__link" >
< span class = "md-ellipsis" >
Home
</ span >
</ a >
</ li >
2026-08-15 20:56:29 +02:00
< li class = "md-nav__item" >
< a href = "../../installation/" class = "md-nav__link" >
< span class = "md-ellipsis" >
2026-04-16 22:38:20 +02:00
2026-08-15 20:56:29 +02:00
Installation
</ span >
</ a >
</ li >
2026-04-16 22:38:20 +02:00
2026-08-15 20:56:29 +02:00
< li class = "md-nav__item md-nav__item--nested" >
2026-04-16 22:38:20 +02:00
2026-08-15 20:56:29 +02:00
< input class = "md-nav__toggle md-toggle " type = "checkbox" id = "__nav_3" >
2026-04-16 22:38:20 +02:00
2026-08-15 20:56:29 +02:00
< label class = "md-nav__link" for = "__nav_3" id = "__nav_3_label" tabindex = "0" >
2026-04-16 22:38:20 +02:00
< span class = "md-ellipsis" >
Theory
</ span >
< span class = "md-nav__icon md-icon" ></ span >
</ label >
2026-08-15 20:56:29 +02:00
< nav class = "md-nav" data-md-level = "1" aria-labelledby = "__nav_3_label" aria-expanded = "false" >
< label class = "md-nav__title" for = "__nav_3" >
2026-04-16 22:38:20 +02:00
< span class = "md-nav__icon md-icon" ></ span >
Theory
</ label >
< ul class = "md-nav__list" data-md-scrollfix >
< li class = "md-nav__item" >
2026-04-29 22:52:42 +02:00
< a href = "../../kmers/" class = "md-nav__link" >
2026-04-16 22:38:20 +02:00
< span class = "md-ellipsis" >
Kmers and super-kmers
</ span >
</ a >
</ li >
< li class = "md-nav__item" >
2026-08-15 20:56:29 +02:00
< a href = "../../theory/encoding/" class = "md-nav__link" >
2026-04-16 22:38:20 +02:00
< span class = "md-ellipsis" >
DNA encoding
</ span >
</ a >
</ li >
2026-08-15 20:56:29 +02:00
< li class = "md-nav__item" >
< a href = "../../theory/entropy/" class = "md-nav__link" >
2026-04-16 22:38:20 +02:00
< span class = "md-ellipsis" >
Entropy filter
</ span >
</ a >
</ li >
2026-04-29 22:52:42 +02:00
< li class = "md-nav__item" >
2026-08-15 20:56:29 +02:00
< a href = "../../theory/minimizer/" class = "md-nav__link" >
2026-04-29 22:52:42 +02:00
< span class = "md-ellipsis" >
Minimizer selection
</ span >
</ a >
</ li >
2026-04-16 22:38:20 +02:00
< li class = "md-nav__item" >
2026-08-15 20:56:29 +02:00
< a href = "../../theory/indexing/" class = "md-nav__link" >
2026-04-16 22:38:20 +02:00
< span class = "md-ellipsis" >
Partitioning architecture
</ span >
</ a >
</ li >
2026-08-15 20:56:29 +02:00
< li class = "md-nav__item" >
< a href = "../../theory/evolutionary_distances/" class = "md-nav__link" >
< span class = "md-ellipsis" >
Central-position SNP distance (discussion)
</ span >
</ a >
</ li >
2026-04-16 22:38:20 +02:00
</ ul >
</ nav >
</ li >
< li class = "md-nav__item md-nav__item--nested" >
2026-08-15 20:56:29 +02:00
< input class = "md-nav__toggle md-toggle " type = "checkbox" id = "__nav_4" >
2026-04-16 22:38:20 +02:00
2026-08-15 20:56:29 +02:00
< label class = "md-nav__link" for = "__nav_4" id = "__nav_4_label" tabindex = "0" >
2026-04-16 22:38:20 +02:00
< span class = "md-ellipsis" >
Implementation
</ span >
< span class = "md-nav__icon md-icon" ></ span >
</ label >
2026-08-15 20:56:29 +02:00
< nav class = "md-nav" data-md-level = "1" aria-labelledby = "__nav_4_label" aria-expanded = "false" >
< label class = "md-nav__title" for = "__nav_4" >
2026-04-16 22:38:20 +02:00
< span class = "md-nav__icon md-icon" ></ span >
Implementation
</ label >
< ul class = "md-nav__list" data-md-scrollfix >
< li class = "md-nav__item" >
< a href = "../../implementation/superkmer/" class = "md-nav__link" >
< span class = "md-ellipsis" >
SuperKmer
</ span >
</ a >
</ li >
< li class = "md-nav__item" >
< a href = "../../implementation/kmer/" class = "md-nav__link" >
< span class = "md-ellipsis" >
Kmer
</ span >
</ a >
</ li >
< li class = "md-nav__item" >
< a href = "../../implementation/chunkreader/" class = "md-nav__link" >
< span class = "md-ellipsis" >
Chunk reader
</ span >
</ a >
</ li >
< li class = "md-nav__item" >
< a href = "../../implementation/pipeline/" class = "md-nav__link" >
< span class = "md-ellipsis" >
Construction pipeline
</ span >
</ a >
</ li >
2026-04-29 22:52:42 +02:00
< li class = "md-nav__item" >
< a href = "../../implementation/obipipeline/" class = "md-nav__link" >
< span class = "md-ellipsis" >
obipipeline library
</ span >
</ a >
</ li >
2026-04-16 22:38:20 +02:00
< li class = "md-nav__item" >
< a href = "../../implementation/storage/" class = "md-nav__link" >
< span class = "md-ellipsis" >
On-disk storage
</ span >
</ a >
</ li >
< li class = "md-nav__item" >
< a href = "../../implementation/mphf/" class = "md-nav__link" >
< span class = "md-ellipsis" >
MPHF selection
</ span >
</ a >
</ li >
2026-04-29 22:52:42 +02:00
< li class = "md-nav__item" >
< a href = "../../implementation/unitig_evidence/" class = "md-nav__link" >
< span class = "md-ellipsis" >
Unitig evidence encoding
</ span >
</ a >
</ li >
2026-05-15 21:07:23 +08:00
2026-06-04 21:27:01 +02:00
< li class = "md-nav__item" >
< a href = "../../implementation/evidence_elimination/" class = "md-nav__link" >
< span class = "md-ellipsis" >
Evidence elimination (discussion)
</ span >
</ a >
</ li >
2026-05-15 21:07:23 +08:00
< li class = "md-nav__item" >
< a href = "../../implementation/obilayeredmap/" class = "md-nav__link" >
< span class = "md-ellipsis" >
obilayeredmap crate
</ span >
</ a >
</ li >
< li class = "md-nav__item" >
< a href = "../../implementation/persistent_compact_int_vec/" class = "md-nav__link" >
< span class = "md-ellipsis" >
PersistentCompactIntVec
</ span >
</ a >
</ li >
< li class = "md-nav__item" >
< a href = "../../implementation/persistent_bit_vec/" class = "md-nav__link" >
< span class = "md-ellipsis" >
PersistentBitVec
</ span >
</ a >
</ li >
2026-06-04 21:27:01 +02:00
< li class = "md-nav__item" >
< a href = "../../implementation/merge/" class = "md-nav__link" >
< span class = "md-ellipsis" >
Merge command
</ span >
</ a >
</ li >
< li class = "md-nav__item" >
2026-08-15 20:56:29 +02:00
< a href = "../../implementation/merge_parallelism/" class = "md-nav__link" >
2026-06-04 21:27:01 +02:00
< span class = "md-ellipsis" >
2026-08-15 20:56:29 +02:00
Merge parallelism & memory
</ span >
</ a >
</ li >
< li class = "md-nav__item" >
< a href = "../../implementation/filtering/" class = "md-nav__link" >
< span class = "md-ellipsis" >
Kmer filtering
</ span >
</ a >
</ li >
< li class = "md-nav__item" >
< a href = "../../implementation/select/" class = "md-nav__link" >
< span class = "md-ellipsis" >
Select command
</ span >
</ a >
</ li >
< li class = "md-nav__item" >
< a href = "../../implementation/obitaxonomy/" class = "md-nav__link" >
< span class = "md-ellipsis" >
obitaxonomy crate
2026-06-04 21:27:01 +02:00
</ span >
</ a >
</ li >
2026-04-16 22:38:20 +02:00
</ ul >
</ nav >
</ li >
2026-08-15 20:56:29 +02:00
2026-04-16 22:38:20 +02:00
2026-08-15 20:56:29 +02:00
< li class = "md-nav__item md-nav__item--active md-nav__item--nested" >
2026-04-16 22:38:20 +02:00
2026-08-15 20:56:29 +02:00
< input class = "md-nav__toggle md-toggle " type = "checkbox" id = "__nav_5" checked >
2026-04-16 22:38:20 +02:00
2026-08-15 20:56:29 +02:00
< label class = "md-nav__link" for = "__nav_5" id = "__nav_5_label" tabindex = "0" >
2026-04-16 22:38:20 +02:00
< span class = "md-ellipsis" >
Architecture
</ span >
< span class = "md-nav__icon md-icon" ></ span >
</ label >
2026-08-15 20:56:29 +02:00
< nav class = "md-nav" data-md-level = "1" aria-labelledby = "__nav_5_label" aria-expanded = "true" >
< label class = "md-nav__title" for = "__nav_5" >
2026-04-16 22:38:20 +02:00
< span class = "md-nav__icon md-icon" ></ span >
Architecture
</ label >
< ul class = "md-nav__list" data-md-scrollfix >
< li class = "md-nav__item" >
2026-08-15 20:56:29 +02:00
< a href = "../sequences/invariant/" class = "md-nav__link" >
2026-04-16 22:38:20 +02:00
< span class = "md-ellipsis" >
Sequences
</ span >
</ a >
</ li >
2026-05-15 21:07:23 +08:00
< li class = "md-nav__item" >
2026-08-15 20:56:29 +02:00
< a href = "../index_architecture/" class = "md-nav__link" >
2026-05-15 21:07:23 +08:00
< span class = "md-ellipsis" >
Kmer index
</ span >
</ a >
</ li >
2026-08-15 20:56:29 +02:00
< li class = "md-nav__item" >
< a href = "../siblings/" class = "md-nav__link" >
< span class = "md-ellipsis" >
Sibling annex (discussion)
</ span >
</ a >
</ li >
< li class = "md-nav__item md-nav__item--active" >
< input class = "md-nav__toggle md-toggle" type = "checkbox" id = "__toc" >
< label class = "md-nav__link md-nav__link--active" for = "__toc" >
< span class = "md-ellipsis" >
NUMA-aware worker pools
</ span >
< span class = "md-nav__icon md-icon" ></ span >
</ label >
< a href = "./" class = "md-nav__link md-nav__link--active" >
< span class = "md-ellipsis" >
NUMA-aware worker pools
</ span >
</ a >
< nav class = "md-nav md-nav--secondary" aria-label = "Table of contents" >
< label class = "md-nav__title" for = "__toc" >
< span class = "md-nav__icon md-icon" ></ span >
Table of contents
</ label >
< ul class = "md-nav__list" data-md-component = "toc" data-md-scrollfix >
< li class = "md-nav__item" >
< a href = "#problem" class = "md-nav__link" >
< span class = "md-ellipsis" >
Problem
</ span >
</ a >
</ li >
< li class = "md-nav__item" >
< a href = "#strategy" class = "md-nav__link" >
< span class = "md-ellipsis" >
Strategy
</ span >
</ a >
</ li >
< li class = "md-nav__item" >
< a href = "#rayon-thread-pool-isolation" class = "md-nav__link" >
< span class = "md-ellipsis" >
Rayon thread pool isolation
</ span >
</ a >
</ li >
< li class = "md-nav__item" >
< a href = "#thread-pinning" class = "md-nav__link" >
< span class = "md-ellipsis" >
Thread pinning
</ span >
</ a >
</ li >
< li class = "md-nav__item" >
< a href = "#memory-locality" class = "md-nav__link" >
< span class = "md-ellipsis" >
Memory locality
</ span >
</ a >
</ li >
< li class = "md-nav__item" >
< a href = "#adaptive-spawn-criterion" class = "md-nav__link" >
< span class = "md-ellipsis" >
Adaptive spawn criterion
</ span >
</ a >
</ li >
< li class = "md-nav__item" >
< a href = "#required-changes" class = "md-nav__link" >
< span class = "md-ellipsis" >
Required changes
</ span >
</ a >
</ li >
< li class = "md-nav__item" >
< a href = "#platform-guard" class = "md-nav__link" >
< span class = "md-ellipsis" >
Platform guard
</ span >
</ a >
</ li >
< li class = "md-nav__item" >
< a href = "#open-questions" class = "md-nav__link" >
< span class = "md-ellipsis" >
Open questions
</ span >
</ a >
</ li >
</ ul >
</ nav >
</ li >
< li class = "md-nav__item" >
< a href = "../numa_partition_runner/" class = "md-nav__link" >
< span class = "md-ellipsis" >
NUMA-aware partition runner
</ span >
</ a >
</ li >
2026-04-16 22:38:20 +02:00
</ ul >
</ nav >
</ li >
</ ul >
</ nav >
</ div >
</ div >
</ div >
< div class = "md-sidebar md-sidebar--secondary" data-md-component = "sidebar" data-md-type = "toc" >
< div class = "md-sidebar__scrollwrap" >
< div class = "md-sidebar__inner" >
< nav class = "md-nav md-nav--secondary" aria-label = "Table of contents" >
< label class = "md-nav__title" for = "__toc" >
< span class = "md-nav__icon md-icon" ></ span >
Table of contents
</ label >
< ul class = "md-nav__list" data-md-component = "toc" data-md-scrollfix >
< li class = "md-nav__item" >
2026-08-15 20:56:29 +02:00
< a href = "#problem" class = "md-nav__link" >
2026-04-16 22:38:20 +02:00
< span class = "md-ellipsis" >
2026-08-15 20:56:29 +02:00
Problem
2026-04-16 22:38:20 +02:00
</ span >
</ a >
</ li >
< li class = "md-nav__item" >
2026-08-15 20:56:29 +02:00
< a href = "#strategy" class = "md-nav__link" >
2026-04-16 22:38:20 +02:00
< span class = "md-ellipsis" >
2026-08-15 20:56:29 +02:00
Strategy
2026-04-16 22:38:20 +02:00
</ span >
</ a >
</ li >
< li class = "md-nav__item" >
2026-08-15 20:56:29 +02:00
< a href = "#rayon-thread-pool-isolation" class = "md-nav__link" >
2026-04-16 22:38:20 +02:00
< span class = "md-ellipsis" >
2026-08-15 20:56:29 +02:00
Rayon thread pool isolation
2026-04-16 22:38:20 +02:00
</ span >
</ a >
</ li >
< li class = "md-nav__item" >
2026-08-15 20:56:29 +02:00
< a href = "#thread-pinning" class = "md-nav__link" >
2026-04-16 22:38:20 +02:00
< span class = "md-ellipsis" >
2026-08-15 20:56:29 +02:00
Thread pinning
2026-04-16 22:38:20 +02:00
</ span >
</ a >
</ li >
< li class = "md-nav__item" >
2026-08-15 20:56:29 +02:00
< a href = "#memory-locality" class = "md-nav__link" >
2026-04-16 22:38:20 +02:00
< span class = "md-ellipsis" >
2026-08-15 20:56:29 +02:00
Memory locality
2026-04-16 22:38:20 +02:00
</ span >
</ a >
</ li >
< li class = "md-nav__item" >
2026-08-15 20:56:29 +02:00
< a href = "#adaptive-spawn-criterion" class = "md-nav__link" >
2026-04-16 22:38:20 +02:00
< span class = "md-ellipsis" >
2026-08-15 20:56:29 +02:00
Adaptive spawn criterion
2026-04-16 22:38:20 +02:00
</ span >
</ a >
</ li >
< li class = "md-nav__item" >
2026-08-15 20:56:29 +02:00
< a href = "#required-changes" class = "md-nav__link" >
2026-04-16 22:38:20 +02:00
< span class = "md-ellipsis" >
2026-08-15 20:56:29 +02:00
Required changes
</ span >
</ a >
</ li >
< li class = "md-nav__item" >
< a href = "#platform-guard" class = "md-nav__link" >
< span class = "md-ellipsis" >
Platform guard
</ span >
</ a >
</ li >
< li class = "md-nav__item" >
< a href = "#open-questions" class = "md-nav__link" >
< span class = "md-ellipsis" >
Open questions
2026-04-16 22:38:20 +02:00
</ span >
</ a >
</ li >
</ ul >
</ nav >
</ div >
</ div >
</ div >
< div class = "md-content" data-md-component = "content" >
< article class = "md-content__inner md-typeset" >
2026-08-15 20:56:29 +02:00
< h1 id = "numa-aware-worker-pools-for-merge" > NUMA-aware worker pools for merge</ h1 >
< h2 id = "problem" > Problem</ h2 >
< p > The merge command's bottleneck is < code > compute_degrees</ code > in < code > obidebruinj</ code > : a random pointer-chase over 20– – × </ p >
< p > Modern HPC nodes are multi-socket NUMA machines (observed: 2 sockets × × </ p >
2026-04-16 22:38:20 +02:00
< ul >
2026-08-15 20:56:29 +02:00
< li > Full 192-core run: ~15 min/partition (× </ li >
< li >< code > taskset</ code > restricted to 4 NUMA nodes (96 cores): ~90 s/partition</ li >
< li > OAR job on 1 NUMA node (24 cores): ~80 s/partition, same throughput as 96 cores</ li >
</ ul >
< p >< strong > Conclusion</ strong > : the bottleneck is memory bandwidth per NUMA node, not core count. 24 cores on one NUMA node achieve the same throughput as 96 cores across four.</ p >
< h2 id = "strategy" > Strategy</ h2 >
< p > Run N worker groups in parallel, one per NUMA node, each with its own Rayon thread pool whose threads are pinned to the NUMA node's CPUs. Linux's first-touch policy then places graph allocations on local DRAM automatically — no explicit NUMA allocator needed.</ p >
< p > Expected throughput: N × × – > 60 min with the current single-pool approach.</ p >
< h2 id = "rayon-thread-pool-isolation" > Rayon thread pool isolation</ h2 >
< p > Rayon provides < code > ThreadPool::install(|| { ... })</ code > : any Rayon call (< code > par_iter</ code > , < code > current_num_threads</ code > , etc.) inside the closure uses < em > that</ em > pool exclusively. Wrapping < code > merge_partition</ code > in < code > pool.install()</ code > redirects all downstream Rayon calls — including those in < code > debruijn.rs</ code > and < code > partition.rs</ code > — without touching those crates.</ p >
< div class = "highlight" >< pre >< span ></ span >< code >< span class = "c1" > // worker thread, assigned to NUMA pool `pool`</ span >
< span class = "n" > pool</ span >< span class = "p" > .</ span >< span class = "n" > install</ span >< span class = "p" > (</ span >< span class = "o" > ||</ span >< span class = "w" > </ span >< span class = "p" > {</ span >
< span class = "w" > </ span >< span class = "n" > dst_partition</ span >< span class = "p" > .</ span >< span class = "n" > merge_partition</ span >< span class = "p" > (</ span >< span class = "n" > i</ span >< span class = "p" > ,</ span >< span class = "w" > </ span >< span class = "n" > srcs</ span >< span class = "p" > ,</ span >< span class = "w" > </ span >< span class = "n" > mode</ span >< span class = "p" > ,</ span >< span class = "w" > </ span >< span class = "n" > n_dst_genomes</ span >< span class = "p" > ,</ span >< span class = "w" > </ span >< span class = "n" > block_bits</ span >< span class = "p" > ,</ span >< span class = "w" > </ span >< span class = "n" > evidence</ span >< span class = "p" > )</ span >
< span class = "p" > })</ span >
</ code ></ pre ></ div >
< p >< code > rayon::current_num_threads()</ code > inside < code > merge_partition</ code > will return the pool size (e.g. 24), not the global thread count — which is the right value for buffer sizing.</ p >
< h2 id = "thread-pinning" > Thread pinning</ h2 >
< p >< code > ThreadPoolBuilder::spawn_handler</ code > provides a hook executed for each thread at creation. Inside, < code > libc::sched_setaffinity</ code > pins the thread to a CPU set:</ p >
< div class = "highlight" >< pre >< span ></ span >< code >< span class = "kd" > let</ span >< span class = "w" > </ span >< span class = "n" > cpus</ span >< span class = "p" > :</ span >< span class = "w" > </ span >< span class = "nb" > Vec</ span >< span class = "o" > < </ span >< span class = "kt" > usize</ span >< span class = "o" > > </ span >< span class = "w" > </ span >< span class = "o" > =</ span >< span class = "w" > </ span >< span class = "n" > numa_node_cpus</ span >< span class = "p" > (</ span >< span class = "n" > node</ span >< span class = "p" > );</ span >< span class = "w" > </ span >< span class = "c1" > // from /sys/devices/system/node/nodeN/cpulist</ span >
< span class = "n" > rayon</ span >< span class = "p" > ::</ span >< span class = "n" > ThreadPoolBuilder</ span >< span class = "p" > ::</ span >< span class = "n" > new</ span >< span class = "p" > ()</ span >
< span class = "w" > </ span >< span class = "p" > .</ span >< span class = "n" > num_threads</ span >< span class = "p" > (</ span >< span class = "n" > cpus</ span >< span class = "p" > .</ span >< span class = "n" > len</ span >< span class = "p" > ())</ span >
< span class = "w" > </ span >< span class = "p" > .</ span >< span class = "n" > spawn_handler</ span >< span class = "p" > (</ span >< span class = "k" > move</ span >< span class = "w" > </ span >< span class = "o" > |</ span >< span class = "n" > thread</ span >< span class = "o" > |</ span >< span class = "w" > </ span >< span class = "p" > {</ span >
< span class = "w" > </ span >< span class = "kd" > let</ span >< span class = "w" > </ span >< span class = "k" > mut</ span >< span class = "w" > </ span >< span class = "n" > b</ span >< span class = "w" > </ span >< span class = "o" > =</ span >< span class = "w" > </ span >< span class = "n" > std</ span >< span class = "p" > ::</ span >< span class = "n" > thread</ span >< span class = "p" > ::</ span >< span class = "n" > Builder</ span >< span class = "p" > ::</ span >< span class = "n" > new</ span >< span class = "p" > ();</ span >
< span class = "w" > </ span >< span class = "n" > std</ span >< span class = "p" > ::</ span >< span class = "n" > thread</ span >< span class = "p" > ::</ span >< span class = "n" > Builder</ span >< span class = "p" > ::</ span >< span class = "n" > new</ span >< span class = "p" > ().</ span >< span class = "n" > spawn</ span >< span class = "p" > (</ span >< span class = "k" > move</ span >< span class = "w" > </ span >< span class = "o" > ||</ span >< span class = "w" > </ span >< span class = "p" > {</ span >
< span class = "w" > </ span >< span class = "n" > pin_to_cpus</ span >< span class = "p" > (</ span >< span class = "o" > & </ span >< span class = "n" > cpus</ span >< span class = "p" > );</ span >< span class = "w" > </ span >< span class = "c1" > // sched_setaffinity via libc</ span >
< span class = "w" > </ span >< span class = "n" > thread</ span >< span class = "p" > .</ span >< span class = "n" > run</ span >< span class = "p" > ()</ span >
< span class = "w" > </ span >< span class = "p" > })</ span >< span class = "o" > ?</ span >< span class = "p" > ;</ span >
< span class = "w" > </ span >< span class = "nb" > Ok</ span >< span class = "p" > (())</ span >
< span class = "w" > </ span >< span class = "p" > })</ span >
< span class = "w" > </ span >< span class = "p" > .</ span >< span class = "n" > build</ span >< span class = "p" > ()</ span >< span class = "o" > ?</ span >
</ code ></ pre ></ div >
< p > NUMA topology is read from < code > /sys/devices/system/node/node*/cpulist</ code > — no < code > libnuma</ code > dependency required. If the < code > numa</ code > crate is linked, < code > numa_available()</ code > / < code > numa_run_on_node()</ code > are an alternative.</ p >
< h2 id = "memory-locality" > Memory locality</ h2 >
< p > Linux allocates pages on the NUMA node of the thread that first writes them (first-touch policy). Once Rayon threads are pinned to node N, all graph data built by those threads lands on node N's DRAM. No changes to the allocator, no explicit < code > numa_alloc_onnode</ code > calls.</ p >
< h2 id = "adaptive-spawn-criterion" > Adaptive spawn criterion</ h2 >
< p > The current criterion uses < code > std::thread::available_parallelism()</ code > (returns total cores = 192) and < code > max_workers = n_cores / 2</ code > . With NUMA pools:</ p >
< ul >
< li >< code > n_cores</ code > per pool = cores per NUMA node (e.g. 24)</ li >
< li >< code > max_workers</ code > per pool = pool size / 2 (e.g. 12)</ li >
< li > CPU efficiency is measured per pool, not globally</ li >
</ ul >
< p > Each NUMA group runs its own independent adaptive pool. Workers are distributed across NUMA groups round-robin or by workload (partition assignment can be pre-split by NUMA group index).</ p >
< h2 id = "required-changes" > Required changes</ h2 >
< table >
< thead >
< tr >
< th > File</ th >
< th > Change</ th >
</ tr >
</ thead >
< tbody >
< tr >
< td >< code > obikindex/src/merge.rs</ code ></ td >
< td > Detect NUMA topology; build N < code > ThreadPool</ code > s with pinned threads; assign each pre-spawned worker to a pool; wrap < code > merge_partition</ code > in < code > pool.install()</ code ></ td >
</ tr >
< tr >
< td >< code > obikindex/src/merge.rs</ code ></ td >
< td > Replace < code > available_parallelism()</ code > with per-NUMA core count for spawn criterion</ td >
</ tr >
< tr >
< td >< code > obikpartitionner/src/merge_layer.rs</ code ></ td >
< td > No change — < code > merge_partition</ code > already works inside any Rayon context</ td >
</ tr >
< tr >
< td >< code > obidebruinj/src/debruijn.rs</ code ></ td >
< td > No change — < code > par_iter</ code > and < code > current_num_threads</ code > are pool-context-aware</ td >
</ tr >
< tr >
< td >< code > obikpartitionner/src/partition.rs</ code ></ td >
< td > No change — same reason</ td >
</ tr >
</ tbody >
</ table >
< h2 id = "platform-guard" > Platform guard</ h2 >
< p > NUMA pinning is Linux-only. The fallback is the current single global pool:</ p >
< div class = "highlight" >< pre >< span ></ span >< code >< span class = "cp" > #[cfg(target_os = </ span >< span class = "s" > " linux" </ span >< span class = "cp" > )]</ span >
< span class = "k" > fn</ span >< span class = "w" > </ span >< span class = "nf" > build_numa_pools</ span >< span class = "p" > ()</ span >< span class = "w" > </ span >< span class = "p" > -> </ span >< span class = "w" > </ span >< span class = "nb" > Option</ span >< span class = "o" > < </ span >< span class = "nb" > Vec</ span >< span class = "o" > < </ span >< span class = "n" > rayon</ span >< span class = "p" > ::</ span >< span class = "n" > ThreadPool</ span >< span class = "o" > >> </ span >< span class = "w" > </ span >< span class = "p" > {</ span >< span class = "w" > </ span >< span class = "o" > ..</ span >< span class = "p" > .</ span >< span class = "w" > </ span >< span class = "p" > }</ span >
< span class = "cp" > #[cfg(not(target_os = </ span >< span class = "s" > " linux" </ span >< span class = "cp" > ))]</ span >
< span class = "k" > fn</ span >< span class = "w" > </ span >< span class = "nf" > build_numa_pools</ span >< span class = "p" > ()</ span >< span class = "w" > </ span >< span class = "p" > -> </ span >< span class = "w" > </ span >< span class = "nb" > Option</ span >< span class = "o" > < </ span >< span class = "nb" > Vec</ span >< span class = "o" > < </ span >< span class = "n" > rayon</ span >< span class = "p" > ::</ span >< span class = "n" > ThreadPool</ span >< span class = "o" > >> </ span >< span class = "w" > </ span >< span class = "p" > {</ span >< span class = "w" > </ span >< span class = "nb" > None</ span >< span class = "w" > </ span >< span class = "p" > }</ span >
</ code ></ pre ></ div >
< p > When < code > build_numa_pools()</ code > returns < code > None</ code > (macOS, UMA, or single-socket), < code > merge.rs</ code > uses the existing code path unchanged.</ p >
< h2 id = "open-questions" > Open questions</ h2 >
< ul >
< li >< strong > Partition assignment</ strong > : split partitions by NUMA group up-front (static) or use a shared queue with per-group workers stealing from a common pool? Static split is simpler; stealing is better for load balance when partitions vary widely in size.</ li >
< li >< strong > Intra-NUMA adaptive criterion</ strong > : with 24 cores and ~3– < code > n_cores = 24</ code > .</ li >
< li >< strong > I/O</ strong > : partition data (unitig files) is on a shared filesystem. With 8 concurrent NUMA groups, I/O concurrency increases 8× </ li >
2026-04-16 22:38:20 +02:00
</ ul >
</ article >
</ div >
< script > var target = document . getElementById ( location . hash . slice ( 1 )); target && target . name && ( target . checked = target . name . startsWith ( "__tabbed_" ))</ script >
</ div >
</ main >
< footer class = "md-footer" >
< div class = "md-footer-meta md-typeset" >
< div class = "md-footer-meta__inner md-grid" >
< div class = "md-copyright" >
Made with
< a href = "https://squidfunk.github.io/mkdocs-material/" target = "_blank" rel = "noopener" >
Material for MkDocs
</ a >
</ div >
</ div >
</ div >
</ footer >
</ div >
< div class = "md-dialog" data-md-component = "dialog" >
< div class = "md-dialog__inner md-typeset" ></ div >
</ div >
< script id = "__config" type = "application/json" >{ "annotate" : null , "base" : "../.." , "features" : [], "search" : "../../assets/javascripts/workers/search.2c215733.min.js" , "tags" : null , "translations" : { "clipboard.copied" : "Copied to clipboard" , "clipboard.copy" : "Copy to clipboard" , "search.result.more.one" : "1 more on this page" , "search.result.more.other" : "# more on this page" , "search.result.none" : "No matching documents" , "search.result.one" : "1 matching document" , "search.result.other" : "# matching documents" , "search.result.placeholder" : "Type to start searching" , "search.result.term.missing" : "Missing" , "select.version" : "Select version" }, "version" : null }</ script >
< script src = "../../assets/javascripts/bundle.79ae519e.min.js" ></ script >
< script src = "https://unpkg.com/mathjax@3/es5/tex-mml-chtml.js" ></ script >
</ body >
</ html >
