OBIKmers/obikmer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 27 Wiki Activity

docs: add obikmer user guide and MkDocs build configuration

Browse Source 
Introduces a comprehensive documentation set covering theoretical foundations, CLI usage, installation, and system architecture. Adds MkDocs configuration and Makefile targets to generate, serve with live reload, and clean the documentation site. Includes citation styles and bibliography files for academic references.
This commit is contained in:
Eric Coissac
2026-08-13 17:19:01 +02:00
parent 0c86ea0385
commit 6acafa7f2c
28 changed files with 1497 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
UserDocMD/usage/annotate.md
+25
View File
@@ -0,0 +1,25 @@
# annotate
Add or update genome metadata of an index from a CSV file, or dump the current metadata as CSV.
```bash
obikmer annotate INDEX --csv FILE [OPTIONS]
obikmer annotate INDEX --dump
```
## Arguments
| Argument | Description |
|---|---|
| `INDEX` | Index directory to annotate (modified in place) |
## Options
| Option | Default | Description |
|---|---|---|
| `--csv` | — | CSV file of metadata to apply (must contain an id column); required unless `--dump` is used |
| `--sep` | `,` | CSV field separator |
| `--id-col` | `id` | Name of the column containing genome labels |
| `--na-value` | `NA` | Value meaning "remove this field" (deletes the existing key if present) |
| `--no-overwrite` | off | Do not overwrite existing metadata keys |
| `--dump` | off | Print all genome metadata as CSV to stdout instead of applying a file |
UserDocMD/usage/distance.md
+97
View File
@@ -0,0 +1,97 @@
# distance
Compute pairwise distance matrices between the genomes stored in an index, optionally build trees (NJ/UPGMA), and optionally derive a central-position SNP model with exports for external phylogenetic tools (TNT, PhyG, IQ-TREE).
```bash
obikmer distance INDEX [OPTIONS]
```
## Arguments
| Argument | Description |
|---|---|
| `INDEX` | Index directory |
## Distance matrix
| Option | Default | Description |
|---|---|---|
| `--metric` | `jaccard` | One of `jaccard`, `mash`, `hamming`, `bray-curtis`, `relfreq-bray-curtis`, `euclidean`, `relfreq-euclidean`, `hellinger`, `hellinger-euclidean` |
| `--presence-threshold` | `1` | Minimum count for a kmer to be considered present, for Jaccard/Mash on a count index |
| `--shared-kmers` | off | Also write the shared-kmer count matrix |
| `--nj` | off | Compute and write a Neighbor-Joining tree (Newick) |
| `--upgma` | off | Compute and write a UPGMA tree (Newick) |
| `-o, --output` | none (stdout) | Output file prefix; without it, the distance matrix is printed to stdout as CSV |
`hamming` requires a presence/absence index. All other metrics work on either index type; on a presence index, `jaccard`/`mash`/`hamming` are the only ones available.
### Metric definitions
- **jaccard**: $D = 1 - \dfrac{|A \cap B|}{|A \cup B|}$ over the sets of kmers present in each genome.
- **mash**: derived from the Jaccard distance via $D = -\dfrac{1}{k} \ln\!\left(\dfrac{2J}{1+J}\right)$ where $J = 1 - D_{\text{jaccard}}$ and $k$ is the index's kmer size; clamped to 1.0 when $J \le 0$.
- **hamming**: number of kmer positions where presence differs between the two genomes (presence index only, not normalized): $D = \sum_i \mathbb{1}[a_i \ne b_i]$.
- **bray-curtis**: $D = 1 - \dfrac{2 \sum_i \min(c_i^A, c_i^B)}{\sum_i c_i^A + \sum_i c_i^B}$ on raw per-kmer counts.
- **relfreq-bray-curtis**: the same formula computed on per-genome relative frequencies $p_i = c_i / \sum_j c_j$ instead of raw counts.
- **euclidean**: $D = \sqrt{\sum_i (c_i^A - c_i^B)^2}$ on raw counts.
- **relfreq-euclidean**: the same formula on relative frequencies.
- **hellinger**: $D = \dfrac{1}{\sqrt{2}} \sqrt{\sum_i \left(\sqrt{p_i^A} - \sqrt{p_i^B}\right)^2}$ on relative frequencies, bounded in $[0, 1]$.
- **hellinger-euclidean**: the unnormalized variant, $D = \sqrt{2} \times D_{\text{hellinger}}$.
## Central-position SNP model
This is a separate operation from the distance-matrix computation above: if any option below is used, no `--metric` matrix is computed in the same invocation.
A **family** is the set of up to 4 kmers that share identical flanking sequence and differ only at the exact central base. Because $k$ is odd, the central position is well defined and maps to itself under reverse complementation. All computations below first require building the **sibling annex**, an index-wide record of which of the 4 possible central bases are observed at each family, across every genome.
| Option | Default | Description |
|---|---|---|
| `--sibling-annex` | off | Build the sibling presence-mask annex (prerequisite for every option below) |
| `--exclude-genome LABEL` | none | Exclude a genome (repeatable) from every SNP/Sankoff/export computation below |
| `--sibling-stats` | off | Write the family-size (sibling count) distribution, per genome and globally |
| `--raw-snp-distance` | off | Write the single-copy central-SNP p-distance matrix |
| `--raw-snp-counts` | off | Write per-pair diagnostic counts (n_snp, n_shared, n_eligible) instead of a matrix |
| `--snp` | off | Write a SNP-only pseudo-alignment in FASTA, IUPAC-coded |
### Locus eligibility
A family is eligible for a genome pair $(i, j)$ only if genome $i$ carries exactly one of the family's observed forms (single-copy, unambiguous) and genome $j$ also carries exactly one. A genome carrying more than one form at a locus makes that locus ineligible for any pair involving it.
`--raw-snp-distance` tallies, over every eligible locus of every genome pair, $n_{\text{snp}}$ (the two genomes' single forms differ) versus $n_{\text{shared}}$ (they agree — this includes invariant families). The output ratio is $\hat{p} = \dfrac{n_{\text{snp}}}{n_{\text{snp}} + n_{\text{shared}}}$.
`--snp` restricts itself to *variable* families (family size $\ge 2$) and writes one FASTA record per genome, one column per family, IUPAC-coded from each genome's presence mask at that family (a single form → the plain base; several forms → the matching IUPAC ambiguity code; no form → `-`).
`--exclude-genome` removes a genome from these computations, re-checking column variability among the remaining genomes so that a column made monomorphic by the exclusion is dropped rather than kept artificially. It does not affect the `--metric` distance-matrix path.
## Sankoff calibration and phylogenetic exports
| Option | Default | Description |
|---|---|---|
| `--sankoff` | off | Calibrate a 16-state parsimony cost matrix and matching pseudo-alignment |
| `--sankoff-ratio-ceiling` | `0.5` | Exclude genome pairs whose raw SNP ratio exceeds this value from the calibration |
| `--tnt` | off | Also write a TNT script (implies `--sankoff`) |
| `--phyg` | off | Also write PhyG input files (implies `--sankoff`) |
| `--iqtree` | off | Also write an IQ-TREE custom model and alignment (implies `--sankoff`) |
| `--sankoff-cost-scale` | `100` | Integer scaling factor applied to costs before rounding (required by TNT/PhyG's integer-only cost commands) |
### The 16-state model
Each family is treated as a character with 16 possible states: one per subset of the 4 possible central bases actually observed (including the empty subset). Calibration combines two tallies, both restricted to genome pairs at or below `--sankoff-ratio-ceiling`:
- a $5 \times 5$ transition matrix over family cardinality (0–4 observed forms) between paired genomes, and
- a $4 \times 4$ base-substitution transition matrix from unambiguous single-copy loci,
which are combined into a row-normalized $16 \times 16$ transition probability matrix $P$, converted to a symmetric cost matrix via $\text{cost}(a,b) = -\ln P(a,b)$.
`--sankoff` alone writes the cost matrix, the calibration parameters, and a pseudo-alignment recoded so the empty state uses the symbol `0` (never a gap character, to avoid ambiguity with external tools' own gap semantics). It does not run any external tool.
### Exports
All three exports reuse the `--sankoff` calibrated matrix and pseudo-alignment, recoded for the target tool:
- **`--tnt`**: a self-contained TNT script (alignment recoded to TNT's fixed 16-symbol alphabet, integer-scaled cost matrix re-closed to a metric, a default search block).
- **`--phyg`**: a custom cost-matrix file plus a PhyG script reusing the `--sankoff` alignment directly.
- **`--iqtree`**: a custom substitution-model file (exchangeability matrix recovered as $R(a,b) = e^{-\text{cost}(a,b)}$, plus empirical state frequencies) and a matching alignment, for maximum-likelihood inference with real branch lengths (unlike the parsimony step-counts from TNT/PhyG). Only states actually occurring in the alignment are kept and compactly renumbered.
## Output files
With `-o/--output PREFIX`, the relevant subset of the following files is written: `<prefix>_dist.csv`, `<prefix>_shared.csv`, `<prefix>_nj.nwk`, `<prefix>_upgma.nwk`, `<prefix>_siblings.csv`, `<prefix>_rawsnp.csv`, `<prefix>_rawsnp_counts.csv`, `<prefix>_snp.fasta`, `<prefix>_sankoff_matrix.csv`, `<prefix>_sankoff_params.yaml`, `<prefix>_sankoff.fasta`, `<prefix>_sankoff.tnt`, `<prefix>_sankoff.tcm`, `<prefix>_sankoff.pg`, `<prefix>_iqtree.model`, `<prefix>_iqtree.fasta`. Without `-o`, only the plain `--metric` distance matrix is produced, on stdout.
UserDocMD/usage/dump.md
+25
View File
@@ -0,0 +1,25 @@
# dump
Dump all kmers of an index as CSV, one row per kmer, with per-genome counts or presence.
```bash
obikmer dump INDEX [OPTIONS]
```
## Arguments
| Argument | Description |
|---|---|
| `INDEX` | Index directory to dump |
## Options
| Option | Default | Description |
|---|---|---|
| `--force-presence` | off | Output presence/absence (0/1) even if the index stores counts |
| `--debug` | off | Prefix each row with the partition and layer columns |
| `--head N` | none | Limit output to the first N kmers |
`dump` also accepts the shared [predicate options](filter.md#predicate-options) (`--ingroup`, `--outgroup`, `--min-count`, etc.) to restrict which kmers are dumped.
Output is CSV on stdout.
UserDocMD/usage/estimate.md
+18
View File
@@ -0,0 +1,18 @@
# estimate
Estimate approximate-index parameters (z, evidence bits, false-positive rate) before building an index with `--approx`, without touching any files.
```bash
obikmer estimate [OPTIONS]
```
## Options
| Option | Default | Description |
|---|---|---|
| `-k, --kmer-size` | `31` | Kmer size used at query time (matches `index`'s `--kmer-size`) |
| `-z, --findere-z` | none | Findere z parameter |
| `--evidence-bits` | none | Fingerprint bits per slot (b) |
| `--fp` | none | Target false-positive rate per z-window |
Any two of `-z`, `--evidence-bits`, `--fp` may be given; the third is derived using the same model as `index --approx` and `reindex --approx` ($FP = 1 / 2^{b \cdot z}$). The report printed to stdout includes: query $k$, effective indexed $k$ ($k-z+1$), $z$, evidence bits, per-kmer false-positive rate, and per-z-window false-positive rate.
UserDocMD/usage/filter.md
+47
View File
@@ -0,0 +1,47 @@
# filter
Apply row-level selection to an index: retain only kmers matching ingroup/outgroup predicates over genome membership, plus optional total-count and complexity thresholds. The output is a new, single-layer index.
```bash
obikmer filter SOURCE -o OUTPUT [OPTIONS]
```
## Arguments
| Argument | Description |
|---|---|
| `SOURCE` | Source index directory |
## Options
| Option | Default | Description |
|---|---|---|
| `-o, --output` | — (required) | Output index directory |
| `-f, --force` | off | Overwrite an existing output directory |
| `--presence` | off | Output presence/absence instead of counts |
| `--min-total-count` | none | Minimum total count across all genomes (count index only) |
| `--max-total-count` | none | Maximum total count across all genomes |
| `--min-complexity` | none | Minimum normalized entropy (same score as `--theta` at index build time), recomputed from the stored unitig sequences |
| `--complexity-level-max` | `6` | Maximum sub-word size for the complexity score (used only with `--min-complexity`) |
## Predicate options
| Option | Default | Description |
|---|---|---|
| `--ingroup` | none | Ingroup predicate (repeatable; each occurrence is ANDed) |
| `--outgroup` | none | Outgroup predicate (repeatable; each occurrence is ORed) |
| `--min-count` | 0, or group size + N if negative | Minimum number of ingroup genomes carrying the kmer |
| `--max-count` | ingroup group size | Maximum number of ingroup genomes carrying the kmer |
| `--min-frac` | `1.0` if `--ingroup` given without an explicit quorum, else `0.0` | Minimum fraction of ingroup genomes |
| `--max-frac` | `1.0` | Maximum fraction of ingroup genomes |
| `--min-outgroup-count` | `0` | Minimum number of outgroup genomes carrying the kmer |
| `--max-outgroup-count` | `0` if `--outgroup` given without an explicit quorum, else outgroup group size | Maximum number of outgroup genomes |
| `--min-outgroup-frac` | `0.0` | Minimum fraction of outgroup genomes |
| `--max-outgroup-frac` | `1.0` | Maximum fraction of outgroup genomes |
| `--presence-threshold` | `0` | Minimum count for a genome to be considered a carrier of a kmer |
See [Genome predicates and taxonomy paths](predicates.md) for the predicate syntax used by `--ingroup`/`--outgroup`.
A negative `--min-count`/`--max-count` is interpreted as an offset from the group size — e.g. `--min-count=-1` means "all but one".
Declaring `--ingroup` with no explicit ingroup quorum flag implicitly sets `--min-frac 1.0` (present in every ingroup genome). Declaring `--outgroup` with no explicit outgroup quorum flag implicitly sets `--max-outgroup-count 0` (absent from every outgroup genome). Any explicit quorum flag for a group disables that group's implicit default.
UserDocMD/usage/index_command.md
+50
View File
@@ -0,0 +1,50 @@
# index
Build a genome index from one or more sequence files. Construction proceeds in phases (scatter → dereplicate → count → layered MPHF), described in [On-disk storage](../formats/index_layout.md).
```bash
obikmer index -o OUTPUT [OPTIONS] [INPUTS...]
```
## Arguments
| Argument | Description |
|---|---|
| `INPUTS...` | Input sequence files or directories (FASTA/FASTQ/GenBank, gzip optional). If omitted, reads from stdin. |
## Options
| Option | Default | Description |
|---|---|---|
| `-o, --output` | — (required) | Output index directory |
| `--force` | off | Overwrite an existing output directory |
| `--label` | input file name without extension | Genome label stored in the index |
| `--meta KEY=VALUE` | none | Attach a categorical metadata field to the genome (repeatable) |
| `-k, --kmer-size` | `31` | Kmer size (odd, in [11, 31]) |
| `-m, --minimizer-size` | `11` | Minimizer size (odd, in $[3, k-1]$) |
| `--theta` | `0.7` | Entropy threshold for the low-complexity filter |
| `--level-max` | `6` | Maximum sub-word size for the entropy score |
| `-p, --partitions` | `256` | Number of partitions (rounded up to a power of 2) |
| `-T, --threads` | detected core count | Number of worker threads |
| `--max-open-files` | `threads / 4` (min 1) | Maximum number of input files open simultaneously |
| `--min-abundance` | `1` | Minimum abundance (inclusive) for a kmer to be retained |
| `--max-abundance` | none | Maximum abundance (inclusive) |
| `--with-counts` | off | Store per-kmer counts; otherwise only presence/absence is stored |
| `--keep-intermediate` | off | Keep intermediate build files instead of deleting them after construction |
| `--approx` | off | Use approximate evidence (Findere fingerprint) instead of exact evidence |
| `-z, --findere-z` | see below | Findere z parameter: number of consecutive kmers that must all match (approximate evidence only) |
| `--evidence-bits` | see below | Fingerprint bits per slot (b), approximate evidence only |
| `--fp` | see below | Target false-positive rate per z-window, approximate evidence only |
| `--block-size` | `1` | Block size, in unitigs, for the exact on-disk index (rounded up to a power of 2) |
## Exact vs. approximate evidence
By default, an index stores **exact** evidence: a kmer is either present or absent (or has an exact count with `--with-counts`), with no false positives.
With `--approx`, evidence is stored as a compact **fingerprint** instead, trading a small, tunable false-positive rate for reduced memory/disk usage. The false-positive model is:
$$FP = \frac{1}{2^{b \cdot z}}$$
where $b$ is `--evidence-bits` and $z$ is `--findere-z`. Any two of `-z`, `--evidence-bits`, `--fp` can be given and the third is derived; if none are given, defaults are $b=8$, $z=1$ ($FP \approx 1/256$). See [`estimate`](estimate.md) to explore this trade-off before building an index, and [`reindex`](reindex.md) to convert an existing index between the two representations.
`z` must be strictly less than k: the effective indexed kmer length under approximate evidence is k−z+1.
UserDocMD/usage/merge.md
+29
View File
@@ -0,0 +1,29 @@
# merge
Merge multiple built indexes into a single index.
```bash
obikmer merge -o OUTPUT SOURCE... [OPTIONS]
```
## Arguments
| Argument | Description |
|---|---|
| `SOURCE...` | Index directories to merge (at least one required) |
## Options
| Option | Default | Description |
|---|---|---|
| `-o, --output` | — (required) | Output index directory |
| `--force` | off | Overwrite an existing output directory |
| `--force-presence` | off | Store the merged index as presence/absence even if all sources have counts |
| `--rename-duplicates` | off | Disambiguate duplicate genome labels (`.1`, `.2`, …) instead of failing |
| `--budget-fraction` | `0.5` | Fraction of available RAM reserved as the memory budget for parallel partition merging |
## Behaviour
The output mode is chosen automatically: if every source index stores counts, the merged index stores counts too; otherwise it is presence/absence. `--force-presence` forces presence/absence regardless of the sources.
By default, merging two indexes that share a genome label fails with an error; `--rename-duplicates` instead appends a numeric suffix to keep both copies.
UserDocMD/usage/pack.md
+15
View File
@@ -0,0 +1,15 @@
# pack
Pack an index's per-column matrix files into a single-file format to reduce query-time I/O (fewer file opens per query).
```bash
obikmer pack INDEX
```
## Arguments
| Argument | Description |
|---|---|
| `INDEX` | Index directory to pack (modified in place) |
The index directory is locked for exclusive access while packing.
UserDocMD/usage/predicates.md
+46
View File
@@ -0,0 +1,46 @@
# Genome predicates and taxonomy paths
Several commands ([`filter`](filter.md), [`select`](select.md), [`dump`](dump.md), [`unitig`](unitig.md)) select or group genomes using the same predicate language over genome metadata (see [`annotate`](annotate.md) for attaching metadata to a genome).
## Predicate syntax
| Form | Meaning |
|---|---|
| `*` or `all` | Matches every genome (case-insensitive) |
| `key=v1\|v2` | Genome's `key` metadata equals one of the listed values |
| `key!=v` | Genome's `key` metadata does not equal `v` |
| `key~path` | Genome's `key` metadata (a taxonomy path) matches `path` (ancestry match) |
| `key!~path` | Genome's `key` metadata does not match `path` |
A genome whose metadata does not contain `key` at all cannot be classified by that predicate and is excluded from the relevant group's quorum count.
Multiple `--ingroup` predicates are combined with AND; multiple `--outgroup` predicates are combined with OR. When both an ingroup and an outgroup predicate would match the same genome, ingroup classification wins.
## Taxonomy paths
A metadata value is treated as a taxonomy path when it starts with the literal prefix `taxonomy:/`; any other value is treated as a plain string and only supports `=`/`!=`.
```
taxonomy:/segment1@rank1/segment2@rank2/...
```
Each segment is a name, optionally annotated with a rank (e.g. `@family`, `@genus`, `@species`); ranks are optional and can be mixed within a path. The `@` character is reserved inside taxonomy paths and cannot appear in segment names or rank labels.
### Path matching (`~` / `!~`)
Matching compares segment names only (ranks are informational, not part of the match), with anchoring controlled by leading/trailing `/`:
| Pattern | Matches |
|---|---|
| `A/B` | anywhere in the path |
| `/A/B` | at the start of the path (prefix) |
| `A/B$` | at the end of the path (suffix) |
| `/A/B$` | the entire path (exact) |
A rank-qualified query, `key@rank=value`, matches only when the path's segment at that specific rank equals `value`.
### Example
```bash
obikmer filter source -o output --ingroup "taxon~/Betulaceae/Betula"
```
UserDocMD/usage/query.md
+38
View File
@@ -0,0 +1,38 @@
# query
Query an index with sequences and annotate each query with the kmer matches found.
```bash
obikmer query INDEX INPUTS... [OPTIONS]
```
## Arguments
| Argument | Description |
|---|---|
| `INDEX` | Index directory to query against |
| `INPUTS...` | Input sequence files (FASTA/FASTQ, gzip optional); at least one required |
## Options
| Option | Default | Description |
|---|---|---|
| `--detail` | off | Report per-position, per-genome coverage vectors in the output |
| `--count-missing` | off | Also count query kmers absent from the index |
| `--force-presence` | off | Report presence (0/1) per genome instead of raw counts |
| `--presence-threshold` | `1` | Minimum accumulated count to declare a genome present (implies `--force-presence`) |
| `-z, --findere-z` | derived from the index metadata | Override the Findere z parameter |
| `-T, --threads` | detected core count | Number of worker threads |
| `--chunk-size` | auto-sized (available RAM ÷ threads, clamped to 4–256 MiB) | I/O chunk size, in MiB |
| `--max-open-files` | `threads / 4` (min 1) | Maximum number of input files open simultaneously |
## Output
FASTA on stdout, one record per query, annotated in the OBITools-style header format `>id {"key":value,...}`:
- `kmer_count`: total number of kmers matched
- `kmer_missing`: number of query kmers absent from the index (only with `--count-missing`)
- `kmer_strict_matches`: per-genome match counts
- `coverage`: per-position, per-genome coverage vectors (only with `--detail`)
`--mismatch` is accepted by the CLI but not currently functional; using it produces a warning and is ignored.
UserDocMD/usage/reindex.md
+25
View File
@@ -0,0 +1,25 @@
# reindex
Convert an existing index's evidence representation in place, between exact and approximate.
```bash
obikmer reindex INDEX [OPTIONS]
```
## Arguments
| Argument | Description |
|---|---|
| `INDEX` | Index directory to convert (modified in place) |
## Options
| Option | Default | Description |
|---|---|---|
| `--approx` | off | Convert to approximate evidence (default direction is approximate → exact); requires `-z`/`--evidence-bits`/`--fp` |
| `-z, --findere-z` | none | Findere z parameter (≥ 1) |
| `--evidence-bits` | none | Fingerprint bits per slot (b) |
| `--fp` | none | Target false-positive rate per z-window |
| `--block-size` | `1` | Block size for the exact on-disk index (ignored when converting to approximate) |
See [`index`](index_command.md#exact-vs-approximate-evidence) for the exact/approximate trade-off and the underlying false-positive model, and [`estimate`](estimate.md) to explore parameters beforehand. The index directory is locked for exclusive access during conversion.
UserDocMD/usage/select.md
+35
View File
@@ -0,0 +1,35 @@
# select
Project and/or aggregate the genome columns of an index into a new (or in-place) index. Where [`filter`](filter.md) selects rows (kmers), `select` operates on columns (genomes): grouping several genomes into one aggregated column, reordering columns, or dropping some.
```bash
obikmer select SOURCE (--output OUTPUT | --in-place) [OPTIONS]
```
## Arguments
| Argument | Description |
|---|---|
| `SOURCE` | Source index directory |
## Options
| Option | Default | Description |
|---|---|---|
| `--output` | — | Output index directory (mutually exclusive with `--in-place`) |
| `--in-place` | off | Rewrite the source index in place (mutually exclusive with `--output`) |
| `-f, --force` | off | Overwrite an existing output directory |
| `--group NAME:PRED` | none | Define a named group of genomes by predicate (repeatable; mutually exclusive with `--aggregate-by`) |
| `--group-op NAME:OP` | none | Aggregation operator for a named group |
| `--aggregate-by KEY` | none | Automatically create one group per distinct value of a metadata key (mutually exclusive with `--group`) |
| `--aggregate-op OP` | none | Aggregation operator applied to every auto-generated group |
| `--select COL,...` | all columns | Output columns, in order (group names or genome labels) |
| `--presence-threshold` | `0` | Minimum count for a genome to be considered a carrier (logical operators only) |
## Aggregation operators
`any`, `all`, `none` (logical, evaluated against `--presence-threshold`), `sum`, `min`, `max` (numeric, count index only). If a group's operator is left unspecified, it defaults to `any` when the source is a presence/absence index and `sum` when it stores counts.
A `select` never changes the underlying kmer set — only the per-genome data (counts or presence) is rewritten, so an unaggregated pass-through column (a plain genome label in `--select`) is a cheap copy.
At least one of `--output`/`--in-place` is required, and at least one output column must be defined; every name listed in `--select` must resolve to either a defined group or an existing genome label. See [Genome predicates and taxonomy paths](predicates.md) for the predicate syntax used by `--group`.
UserDocMD/usage/superkmer.md
+27
View File
@@ -0,0 +1,27 @@
# superkmer
Extract super-kmers from one or more sequence files and write them to stdout, without building a full index. Useful for inspecting or piping the super-kmer decomposition of a dataset.
```bash
obikmer superkmer [OPTIONS] [INPUTS...]
```
## Arguments
| Argument | Description |
|---|---|
| `INPUTS...` | Input sequence files or directories (FASTA/FASTQ/GenBank, gzip optional). If omitted, reads from stdin. |
## Options
| Option | Default | Description |
|---|---|---|
| `-k, --kmer-size` | `31` | Kmer size (must be odd, in [11, 31]) |
| `-m, --minimizer-size` | `11` | Minimizer size (must be odd, in $[3, k-1]$) |
| `--theta` | `0.7` | Entropy threshold; kmers with a normalized entropy at or below this value are excluded |
| `--level-max` | `6` | Maximum sub-word size used for the entropy score |
| `-p, --partitions` | `256` | Number of partitions (rounded up to the next power of 2) |
| `-T, --threads` | detected core count | Number of worker threads |
| `--max-open-files` | `threads / 4` (min 1) | Maximum number of input files open simultaneously |
Output is written to stdout in the internal scatter format used by `index`; it is primarily intended to be piped into other tools or inspected for debugging.
UserDocMD/usage/unitig.md
+19
View File
@@ -0,0 +1,19 @@
# unitig
Dump the unitigs of an index as FASTA. A unitig is a maximal non-branching path through the de Bruijn graph implied by the index's kmers; the concatenation of every unitig reconstructs every stored kmer exactly once.
```bash
obikmer unitig INDEX [OPTIONS]
```
## Arguments
| Argument | Description |
|---|---|
| `INDEX` | Index directory |
## Options
`unitig` accepts the shared [predicate options](filter.md#predicate-options) (`--ingroup`, `--outgroup`, `--min-count`, etc.) to restrict which kmers are included before the unitigs are enumerated.
Output is FASTA on stdout.
UserDocMD/usage/utils.md
+26
View File
@@ -0,0 +1,26 @@
# utils
Miscellaneous index maintenance and inspection utilities.
```bash
obikmer utils INDEXES... [OPTIONS]
```
## Arguments
| Argument | Description |
|---|---|
| `INDEXES...` | One or more index directories |
## Options
| Option | Scope | Description |
|---|---|---|
| `--new-label NEW=OLD` | single index only | Rename a genome label |
| `--upgrade-index` | single index only | Add any missing layer metadata files to an older index |
| `--bits-per-kmer` | single index only | Print bits-per-kmer statistics |
| `--stats` | single index only | Print per-genome kmer counts as CSV |
| `--partition-stats` | one or more indexes | Print a partition-size distribution report |
| `--csv FILE` | with `--partition-stats` | Also write raw per-(partition, source) data to FILE as CSV |
At least one operation option must be given. All options except `--partition-stats` require exactly one index directory.