OBIKmers/obikmer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 24 Wiki Activity

feat: support negative count filters as group size offsets

Browse Source 
Updates CLI parsing to accept negative integers for count filters, interpreting them as offsets from the group size (e.g., `-1` means all but one). A resolution closure enforces a floor of 1 to prevent unconstrained filtering on small groups. Additionally, refines evolutionary distance documentation to condition comparisons on local homology, replacing union-based Jaccard with a self-contained `SnpTally`. This unified approach streamlines SNP and shared count computation, incorporates paralogy and heterozygosity handling, and enables direct derivation of corrected distance matrices without external dependencies.
This commit is contained in:
Eric Coissac
2026-08-10 12:38:35 +02:00
parent 45df9919e5
commit 8bc6d533e5
3 changed files with 464 additions and 48 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docmd/implementation/filtering.md
+45 -4
View File
@@ -92,18 +92,48 @@ For each genome:
| Flag | Applies to | Meaning |
|------|-----------|---------|
| `--min-count N` | ingroup | k-mer present in at least N ingroup genomes |
| `--max-count N` | ingroup | k-mer present in at most N ingroup genomes |
| `--min-count N` | ingroup | k-mer present in at least N ingroup genomes (N may be negative, see below) |
| `--max-count N` | ingroup | k-mer present in at most N ingroup genomes (N may be negative, see below) |
| `--min-frac F` | ingroup | k-mer present in at least fraction F of ingroup genomes |
| `--max-frac F` | ingroup | k-mer present in at most fraction F of ingroup genomes |
| `--min-outgroup-count N` | outgroup | k-mer present in at least N outgroup genomes |
| `--max-outgroup-count N` | outgroup | k-mer present in at most N outgroup genomes |
| `--min-outgroup-count N` | outgroup | k-mer present in at least N outgroup genomes (N may be negative, see below) |
| `--max-outgroup-count N` | outgroup | k-mer present in at most N outgroup genomes (N may be negative, see below) |
| `--min-outgroup-frac F` | outgroup | k-mer present in at least fraction F of outgroup genomes |
| `--max-outgroup-frac F` | outgroup | k-mer present in at most fraction F of outgroup genomes |
| `--min-total-count N` | all genomes | sum of per-genome counts ≥ N (`filter` only) |
| `--max-total-count N` | all genomes | sum of per-genome counts ≤ N (`filter` only) |
| `--presence-threshold N` | all | per-genome count > N to be considered "present" (default 0) |
### Negative counts — offset from group size
The four integer count flags (`--min-count`, `--max-count`, `--min-outgroup-count`,
`--max-outgroup-count`) accept **negative** values, interpreted as an offset counted
down from the group size `n`, resolved at run time once `n` is known:
| Value | Effective threshold |
|-------|---------------------|
| `N ≥ 0` | literal absolute count `N` |
| `-x` (x > 0) | `max(1, n − x)` — "all but x" |
`-1` literally means *all but one*, `-2` *all but two*, and so on. This expresses
a quorum relative to the group size that a plain fraction cannot state exactly
(e.g. "present in every genome except at most one" is `n−1`, which is `0.9` for
`n = 10` but `0.857…` for `n = 7`).
The threshold is **floored at 1**, never 0: the negative form always keeps
constraining the group. Without the floor, `--min-count -1` on a singleton
ingroup (`n = 1`) would resolve to `0` ("at least 0") and silently drop the
constraint; the floor makes it `1` ("present in that one genome") instead.
To express a count of `0` (e.g. "absent from the ingroup"), use the literal `0`,
not a negative — `0` and `-0` are indistinguishable, so the offset form starts at
`-1`.
> **Edge case** — on an *empty* group (`n = 0`, e.g. a predicate matching no
> genome), a negative count still resolves to `1`, an impossible constraint that
> rejects every k-mer. This is consistent with an empty group letting nothing
> through, but differs from the "no constraint" behaviour of the fraction flags.
**Conditional defaults** — the defaults for `--min-frac` and `--max-outgroup-count` depend on two conditions:
whether the corresponding group was declared, **and** whether any quorum flag for that group was explicitly set.
@@ -215,6 +245,17 @@ obikmer filter src --output dst \
--max-outgroup-count 0
```
Noise-tolerant core — keep k-mers present in *all but one* ingroup genome
(`-1` = `n−1`) and absent from *all but one* of the outgroup:
```sh
obikmer filter src --output dst \
--ingroup "genus=Betula" \
--outgroup "*" \
--min-count -1 \
--max-outgroup-count -1
```
To dump only k-mers specific to *Betula nana*:
```sh