Files

30 lines
1.8 KiB
Markdown
Raw Permalink Normal View History

# pack
Pack an index's per-column matrix files into a single-file format to reduce query-time I/O (fewer file opens per query).
```bash
2026-08-15 20:56:29 +02:00
obikmer pack INDEX [--sparse]
```
## Arguments
| Argument | Description |
|---|---|
| `INDEX` | Index directory to pack (modified in place) |
2026-08-15 20:56:29 +02:00
## Options
| Option | Default | Description |
|---|---|---|
| `--sparse` | off | Pack presence/absence matrices into a sparse, deduplicated format instead of the dense one |
The index directory is locked for exclusive access while packing.
2026-08-15 20:56:29 +02:00
## `--sparse`
Presence/absence data (which genomes carry each kmer) is often mostly empty — most kmers are present in only a handful of genomes out of the whole collection. The default (dense) packed format stores one bit per genome for every kmer regardless of how many genomes actually carry it; `--sparse` instead stores each kmer's genome list directly, and deduplicates identical lists shared by many kmers (common in real data, since kmers from the same conserved region tend to be carried by the same genomes).
On real genome collections this has measured at roughly 7x smaller on disk than the dense format, and single-kmer lookups (the shape `phylo`'s sibling-annex/entropy/Sankoff computations use) are typically faster too, since the smaller files mean less data to read from disk. The trade-off: reading a whole genome column at once (used by `--metric` distance-matrix computations) is much slower on the sparse format than on the dense one, since there is no native column layout to read sequentially — prefer the dense format (the default, no `--sparse`) for indexes you mainly query with `phylo`'s plain `--metric` distance matrices.
Count matrices (`--metric` on a count index) are not affected by `--sparse` — only presence/absence matrices are.