Add proportional subsampling and Shannon entropy calculation

Introduces `--subsample N` and `--shannon` CLI flags to cap retained variable families via proportional reservoir sampling and compute per-family Shannon entropy. Updates the family scanning API to support explicit selection filtering with early-exit optimization, resolving an indexing drift issue in monomorphic layers. Streams entropy metrics for 15-state and 4-nucleotide spaces directly to CSV while maintaining parallel processing across sibling layers.

This commit is contained in:

Eric Coissac

2026-08-16 21:31:06 +02:00

parent f5e4bbfc6b

commit 45b19503a1

17 changed files with 841 additions and 70 deletions

src/Cargo.lock

Generated

View File

@@ -1858,6 +1858,7 @@ dependencies = [
  "obiskbuilder",
  "obiskio",
  "obisys",
  "rand 0.8.6",
  "rayon",
  "tempfile",
  "tracing",