OBIKmers/obikmer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 34 Wiki Activity

refine k-mer index architecture documentation and remove obsolete spec

Browse Source 
Introduces raw mapping and iteration APIs that bypass membership checks, clarifies variant-specific storage layouts and auto-detection logic, and documents optimized batch access patterns with caller-provided buffers. Removes the outdated obicompactvector_reflexion.md specification to consolidate architectural details into current implementation docs.
This commit is contained in:
Eric Coissac
2026-08-20 13:20:29 +02:00
parent 3b65319529
commit 0da725ffe9
4 changed files with 62 additions and 107 deletions
Download Patch File Download Diff File Expand all files Collapse all files
DevDocMD/implementation/obicompactvec.md
+9 -5
View File
@@ -292,14 +292,18 @@ Pass 1 — byte max, SIMD-vectorizable, O(n)
## Matrix types
Four matrix types, two encodings × two formats:
Both matrix types are enums behind a transparent API — the caller never matches on the variant. `PersistentCompactIntMatrix` has two variants (`Columnar`, `Packed`). `PersistentBitMatrix` has four:
| | Columnar format | Packed format |
| Variant | Storage | When |
|---|---|---|
| **Bit** | `PersistentBitMatrix` (Columnar variant) | `PersistentBitMatrix` (Packed variant) |
| **Int** | `PersistentCompactIntMatrix` (Columnar variant) | `PersistentCompactIntMatrix` (Packed variant) |
| `Columnar` | one `.pbiv`/`.pciv` file per column + `meta.json` | build-time default (`*Builder::new`) |
| `Packed` | single `matrix.pbmx` mmap file | query-optimised, produced by `pack_bit_matrix`/`pack_compact_int_matrix` |
| `Sparse` (bit only) | `sparse_meta.json` + PFIV/Elias-Fano component files, row-major | `pack --sparse`; see [siblings.md](../architecture/siblings.md) for the sparse-vs-dense access-pattern trade-off |
| `Implicit` (bit only) | no file at all | mono-genome presence layers — `n_cols` is always reported as `1`, every value is `true` |
Both matrix types are enums (`Columnar` / `Packed` / `Implicit` for bit) behind a transparent API. `col_view(c)` returns the appropriate view directly:
`PersistentBitMatrix::open(layer_dir)` auto-detects the variant, in order: `matrix.pbmx` → Packed, `presence/meta.json` → Columnar, `presence/sparse_meta.json` → Sparse, `layer_meta.json` (no presence dir at all) → Implicit. `col_view`/`col`/`sub_matrix` panic on `Sparse`/`Implicit` where the operation has no direct-slice equivalent (Sparse is k-mer-major, not column-major; Implicit has no backing storage) — callers needing per-column data on those variants go through `row`/`fill_row`.
`col_view(c)` returns the appropriate view directly:
```rust
// PersistentBitMatrix