docs: document partition runner deadlock fix and select CLI updates
Update documentation to reflect architectural fixes in the NUMA partition runner and implementation changes in the merge and select modules. The partition runner now wraps task execution in `catch_unwind`, emitting panic events that count toward completion thresholds to prevent controller deadlocks, while re-raising the first payload on return. CLI documentation for select has been updated to mandate output directories, introduce default hard-linking with automatic fallback, and document presence aggregation optimizations. Known issues such as self-merge crashes and sparse-packed index gaps are now documented as failing fast due to these improvements.
This commit is contained in:
@@ -304,10 +304,29 @@ step, capped by that node's remaining dormant workers and by the run's total
|
||||
budget (`n_total`) — balance across nodes is now guaranteed by construction,
|
||||
not incidental to channel implementation details.
|
||||
|
||||
## Panic propagation (fixed, 2026-08-28)
|
||||
|
||||
A worker whose `f(i)` panicked used to never send its `WorkerEvent::Completed`
|
||||
— the controller's `while completed < n_total` loop then waited forever for
|
||||
an event that partition could no longer produce, since other live
|
||||
workers/timer threads kept `event_rx` open. Discovered via `obikselect`
|
||||
panicking on an unimplemented `Sparse` matrix case (see `implementation/select.md`):
|
||||
the process hung indefinitely instead of erroring out.
|
||||
|
||||
Fixed: each worker wraps `f(i)` in `std::panic::catch_unwind` and sends a new
|
||||
`WorkerEvent::Panicked(i, payload)` instead of silently dropping the
|
||||
partition. The controller counts it toward `completed` (unblocking the loop)
|
||||
and keeps the first payload seen; once `run` returns, `panic::resume_unwind`
|
||||
re-raises it on the caller's thread — the original message/backtrace still
|
||||
surfaces, from the right place, instead of a silent deadlock. Takes priority
|
||||
over a plain `Err` (a panic means a bug, not a normal typed failure).
|
||||
|
||||
## Open questions
|
||||
|
||||
- **Error handling**: `run` currently returns the first error; remaining errors
|
||||
are dropped. A `Vec<E>` return would give complete diagnostics.
|
||||
are dropped. A `Vec<E>` return would give complete diagnostics. (Panics are
|
||||
now caught and re-raised individually — see above — this only concerns
|
||||
plain `Err` results.)
|
||||
|
||||
- **`INITIAL_DIVISOR` / `GROWTH_DIVISOR` tuning**: currently `4` and `8`
|
||||
(start at 1/4 of a node's cores, grow by 1/8 per step), chosen to fix an
|
||||
|
||||
Reference in New Issue
Block a user