Add proportional subsampling and Shannon entropy calculation

Introduces `--subsample N` and `--shannon` CLI flags to cap retained variable families via proportional reservoir sampling and compute per-family Shannon entropy. Updates the family scanning API to support explicit selection filtering with early-exit optimization, resolving an indexing drift issue in monomorphic layers. Streams entropy metrics for 15-state and 4-nucleotide spaces directly to CSV while maintaining parallel processing across sibling layers.
2026-08-16 21:31:06 +02:00
parent f5e4bbfc6b
commit 45b19503a1
17 changed files with 841 additions and 70 deletions
@@ -15,6 +15,7 @@ obiskbuilder     = { path = "../obiskbuilder" }
 obipipeline      = { path = "../obipipeline" }
 memmap2          = "0.9"
 ndarray          = "0.16"
+rand             = "0.8"
 rayon            = "1"
 tracing          = "0.1.44"