Eric Coissac
b5e027f23b
Introduces a memory-aware scheduling strategy for parallel partition merging that replaces unbounded concurrency with a First-Fit Decreasing approach gated by a thread-safe `MemoryBudget` semaphore. An adaptive expansion factor, seeded by a sequential pilot run, dynamically caps concurrent workers to prevent hashbrown OOMs. Adds a `--budget-fraction` CLI flag to configure RAM allocation, enhances the CLI to accept multiple indexes, and introduces comprehensive partition diagnostics including memory utilization tracking, concurrency metrics, and statistical summaries with ASCII histograms. Updates documentation and navigation accordingly.
62 lines
1.9 KiB
YAML
62 lines
1.9 KiB
YAML
site_name: obikmer
|
|
docs_dir: docmd
|
|
site_dir: doc
|
|
|
|
theme:
|
|
name: material
|
|
|
|
plugins:
|
|
- mermaid2:
|
|
- bibtex:
|
|
bib_file: docmd/references.bib
|
|
csl_file: docmd/ecology-letters.csl
|
|
enable_inline_citations: false
|
|
|
|
markdown_extensions:
|
|
- admonition
|
|
- footnotes
|
|
- tables
|
|
- pymdownx.superfences:
|
|
custom_fences:
|
|
- name: mermaid
|
|
class: mermaid
|
|
format: !!python/name:pymdownx.superfences.fence_code_format
|
|
- pymdownx.arithmatex:
|
|
generic: true
|
|
|
|
extra_javascript:
|
|
- https://unpkg.com/mathjax@3/es5/tex-mml-chtml.js
|
|
|
|
nav:
|
|
- Home: index.md
|
|
- Theory:
|
|
- Kmers and super-kmers: kmers.md
|
|
- DNA encoding: theory/encoding.md
|
|
- Entropy filter: theory/entropy.md
|
|
- Minimizer selection: theory/minimizer.md
|
|
- Partitioning architecture: theory/indexing.md
|
|
- Implementation:
|
|
- SuperKmer: implementation/superkmer.md
|
|
- Kmer: implementation/kmer.md
|
|
- Chunk reader: implementation/chunkreader.md
|
|
- Construction pipeline: implementation/pipeline.md
|
|
- obipipeline library: implementation/obipipeline.md
|
|
- On-disk storage: implementation/storage.md
|
|
- MPHF selection: implementation/mphf.md
|
|
- Unitig evidence encoding: implementation/unitig_evidence.md
|
|
- Evidence elimination (discussion): implementation/evidence_elimination.md
|
|
- obilayeredmap crate: implementation/obilayeredmap.md
|
|
- PersistentCompactIntVec: implementation/persistent_compact_int_vec.md
|
|
- PersistentBitVec: implementation/persistent_bit_vec.md
|
|
- Merge command: implementation/merge.md
|
|
- Merge parallelism & memory: implementation/merge_parallelism.md
|
|
- Kmer filtering: implementation/filtering.md
|
|
- Select command: implementation/select.md
|
|
- Architecture:
|
|
- Sequences: architecture/sequences/invariant.md
|
|
- Kmer index: architecture/index_architecture.md
|
|
|
|
watch:
|
|
- docmd
|
|
- mkdocs.yml
|