obitools4/autodoc/docmd/pkg_obitools_obikmersim.md

# `obikmersim`: K-mer–Based Sequence Similarity Analysis Package

`obikmersim` is a high-performance Python package for **k-mer–driven sequence comparison and alignment**, tailored for biological read analysis (e.g., amplicons, metagenomes). It enables rapid matching of query sequences against reference databases using efficient k-mer indexing, followed by localized alignment with quality-aware consensus refinement. Designed for scalability and flexibility, it supports sparse k-mer representations, orientation detection (forward/reverse-complement), and configurable filtering thresholds.

---

## Public API Overview

### 1. **K-mer Indexing & Matching Workers**
#### `MakeCountMatchWorker(reference_sequences, k=21, min_count=2, sparse=False)`
- **Purpose**: Build a `KmerMap` from reference sequences and match queries via shared k-mers.
- **Functionality**:
  - Indexes all *k*-mers (with optional sparsity mask) from reference sequences.
  - For each query, retrieves candidate references sharing ≥ `min_count` k-mers.
  - Returns annotated results: query ID, matched references, match count, *k*, and sparsity flag.
- **Use Case**: Fast pre-screening for taxonomic assignment or read clustering.

#### `MakeKmerAlignWorker(count_match_worker, delta=50, penalty_scale=1.0, gap_factor=-2)`
- **Purpose**: Perform *k*-mer–seeded local alignment with quality-aware consensus.
- **Functionality**:
  - Uses shared k-mers from `count_match_worker` to seed alignment candidates.
  - Runs local pairwise alignments (via internal aligner) and builds quality-weighted consensus (`ReadAlign`, `BuildQualityConsensus`).
  - Computes:
    - `% identity`
    - Residual similarity (k-mer–aware alignment score)
    - Alignment length & orientation (`+`/`−`)
  - Filters output by `min_identity=80%`, optional min alignment length.
- **Use Case**: Precise read assignment, error correction via consensus.

---

### 2. **CLI Configuration Options**
#### `KmerSimCountOptionSet`
- Defines CLI arguments for k-mer counting/matching:
  - `--kmer-size` (int, default=21)
  - `--sparse` (bool): Enable sparse k-mer masking
  - `--reference <file>`: Reference FASTA/FASTQ path(s)
  - `--min-count` (int, default=2): Minimum shared k-mer count
  - `--self`: Perform self-comparison (query = reference)

#### `KmerSimMatchOptionSet`
- Extends counting options with alignment scoring parameters:
  - `--delta` (int, default=50): Max k-mer separation for seeding
  - `--penalty-scale` (float, default=1.0): Mismatch/gap scaling factor
  - `--gap-factor` (float, default=−2): Gap penalty coefficient
  - `--fast-absolute`: Use fast absolute scoring (no dynamic programming)

#### Composite Sets
- `CountOptionSet` / `MatchOptionSet`: Combine k-mer options with generic I/O conversion settings (e.g., via `obiconvert`).

---

### 3. **CLI Helpers & Accessors**
#### `CLIKmerSize(args)`
- Returns parsed k-mer size from CLI args.

#### `CLIReference(args, format="fasta")`
- Loads reference sequences into memory (supports batched/parallel reading).

#### `CLISelf(args)`
- Returns boolean flag for self-comparison mode.

---

### 4. **Core CLI Wrappers**
#### `CLILookForSharedKmers(args)`
- Orchestrates k-mer counting/matching pipeline:
  - Builds `count_match_worker`
  - Iterates over query sequences (from stdin or file)
  - Outputs match annotations in structured format.

#### `CLIAlignSequences(args)`
- Runs full alignment pipeline:
  - Uses `count_match_worker` to seed candidates
  - Invokes `kmer_align_worker`
  - Outputs aligned pairs with identity, orientation, and quality metrics.

---

## Key Technical Features
- **Sparse K-mers**: Mask positions (e.g., Ns or degenerate bases) via bitmasks.
- **Orientation Handling**: Auto-detect reverse-complement matches during seeding/alignment.
- **Fast Heuristic Scoring**: Preliminary alignment score estimation before full path resolution (reduces compute).
- **Quality-Aware Consensus**: Integrates base quality scores during alignment refinement.
- **Configurable Filtering**: Thresholds on identity, length, and k-mer support.

---

## Typical Workflows
| Workflow | Tools Used |
|---------|------------|
| Taxonomic screening of amplicons | `CLILookForSharedKmers` + sparse mode |
| Read error correction via reference consensus | `CLIAlignSequences` with quality-aware alignment |
| *In silico* PCR specificity check | `CLISelf()` + min-count filtering |
| Large-scale metagenomic read assignment | Batched parallel execution with `CLIReference` |

---

## Output Format
Results are returned as structured records (e.g., dictionaries or dataclasses) with fields:
- `query_id`, `reference_ids`
- `match_count`, `kmer_size`, `sparse_mode`
- For alignments:  
  `%identity`, `alignment_length`, `orientation` (`+1`/`−1`)  
  `residual_similarity`, `consensus_quality`

All public functions are documented with type hints and include unit tests.