docs: document partition runner deadlock fix and select CLI updates

Update documentation to reflect architectural fixes in the NUMA partition runner and implementation changes in the merge and select modules. The partition runner now wraps task execution in `catch_unwind`, emitting panic events that count toward completion thresholds to prevent controller deadlocks, while re-raising the first payload on return. CLI documentation for select has been updated to mandate output directories, introduce default hard-linking with automatic fallback, and document presence aggregation optimizations. Known issues such as self-merge crashes and sparse-packed index gaps are now documented as failing fast due to these improvements.
This commit is contained in:
Eric Coissac
2026-08-28 23:25:46 +02:00
parent 6b0c0867cc
commit 579cfd8752
6 changed files with 243 additions and 93 deletions
+20 -1
View File
@@ -304,10 +304,29 @@ step, capped by that node's remaining dormant workers and by the run's total
budget (`n_total`) — balance across nodes is now guaranteed by construction,
not incidental to channel implementation details.
## Panic propagation (fixed, 2026-08-28)
A worker whose `f(i)` panicked used to never send its `WorkerEvent::Completed`
— the controller's `while completed < n_total` loop then waited forever for
an event that partition could no longer produce, since other live
workers/timer threads kept `event_rx` open. Discovered via `obikselect`
panicking on an unimplemented `Sparse` matrix case (see `implementation/select.md`):
the process hung indefinitely instead of erroring out.
Fixed: each worker wraps `f(i)` in `std::panic::catch_unwind` and sends a new
`WorkerEvent::Panicked(i, payload)` instead of silently dropping the
partition. The controller counts it toward `completed` (unblocking the loop)
and keeps the first payload seen; once `run` returns, `panic::resume_unwind`
re-raises it on the caller's thread — the original message/backtrace still
surfaces, from the right place, instead of a silent deadlock. Takes priority
over a plain `Err` (a panic means a bug, not a normal typed failure).
## Open questions
- **Error handling**: `run` currently returns the first error; remaining errors
are dropped. A `Vec<E>` return would give complete diagnostics.
are dropped. A `Vec<E>` return would give complete diagnostics. (Panics are
now caught and re-raised individually — see above — this only concerns
plain `Err` results.)
- **`INITIAL_DIVISOR` / `GROWTH_DIVISOR` tuning**: currently `4` and `8`
(start at 1/4 of a node's cores, grow by 1/8 per step), chosen to fix an