# Sibling annex — architecture (discussion)

Status: architecture decided (2026-08-14). Implementation not yet mandated.

## Two index spaces, uncorrelated

Every kmer stored in a `Layer` lives in two independent index spaces:

- **Iteration order**: its position when enumerating `unitigs.bin` (the
  superkmer file), deterministic but arbitrary with respect to slot.
- **MPHF slot**: `MphfLayer::index(kmer)`, the number the MPHF assigns.

The two are not correlated by any formula. Converting from one to the other
requires either recomputing the MPHF (kmer → slot) or scanning the iteration
stream (kmer → order). There is no `slot → kmer` operation: the MPHF is a
one-way function, not an invertible bijection with a stored inverse. Any
method that reconstructs a kmer from a bare slot number is wrong by
construction, regardless of the mechanism used (MPHF re-hash, or evidence
decode + direct unitig read). See `MphfLayer::kmer_at`
(`obilayeredmap/src/mphf_layer.rs`) — flagged for removal, currently called
from `obikindex/siblings/build.rs:122` and `family_scan.rs:173`.

## Two pipelines, never mixed

| | origin of the kmer | membership known? | correct mapping |
|---|---|---|---|
| **query pipeline** | external (caller-supplied) | no | `query`/`find`/`find_strict` — MPHF + evidence check |
| **iteration pipeline** | enumerated from this layer's own `unitigs.bin` | yes, by construction | `index`/`index_batch` — MPHF only, no evidence |

Evidence exists solely to answer "is this external kmer a member of the
layer" for the query pipeline. Using it (or the MPHF) to go the other way —
recover a kmer from a slot, or re-verify a kmer that was just produced by
iterating the layer — is a conceptual error: evidence can be probabilistic
(`Approx` mode), so any slot→kmer attempt is unsound in general, and
pointless even in `Exact`/`Hybrid` mode since the kmer was already known.

## Sibling annex: an iteration-pipeline artifact only

The sibling annex (`FamilyMask`/`SiblingAnnex`, `.psib`,
`obicompactvec/src/siblingannex.rs`) records, per kmer, whether it is a
family minorant and which family members are present in the index. Its only
consumers (`obikindex/siblings/stats.rs`, `family_scan.rs`) enumerate it
exhaustively (`0..annex.len()`); no query-pipeline code path touches it.

**Decision**: the annex must be persisted in iteration order, not slot
order. This lets readers zip-iterate `Layer::iter_kmers()` and the annex
file directly — one linear, cache-friendly pass, no MPHF/slot indirection,
no `kmer_at`. It also enables specialized iterators building on this zip:
minorants-only iteration, batch-of-kmers → batch-of-family-members, etc.

Today the annex is built and stored in **slot** order
(`build_layer_sibling_annex`, `siblings/build.rs`): `slot_kmer` is populated
via `(0..n_slots).map(|slot| mphf.kmer_at(slot))`, and the origin `slot` is
threaded through the whole cross-partition reconciliation pipeline (variant
generation, `query_partition_with`, final `mask[slot].fetch_or(...)`). This
must change to iterating `iter_kmers()`/`enumerate_kmers()` and threading
the **iteration index** instead of the slot end to end — eliminating
`kmer_at` from the build path entirely, not just the read path. No
slot-indexed intermediate is needed even during construction; the
iteration-order id is sufficient throughout.

The cross-partition side of the same pipeline is unaffected: checking
whether a generated family-variant kmer exists in another partition is a
genuine query-pipeline operation (the variant's membership in the *target*
partition is unknown) and must keep going through
`KmerPartition::query_partition_with` (MPHF + evidence), never a raw
`index()`.

## Pending work

- Remove `MphfLayer::kmer_at`.
- `siblings/build.rs`: build `slot_kmer`-equivalent via iteration, not
  `kmer_at`; thread iteration index instead of slot through the
  variant/reconciliation pipeline; persist the annex in iteration order.
- `siblings/family_scan.rs`, `stats.rs`: read the annex via zipped
  iteration (`iter_kmers().zip(annex_iter)`) instead of `0..annex.len()` +
  `kmer_at`.