Commit Graph
8 Commits
Author SHA1 Message Date
Eric Coissac 79346c0c86 Add modular data structures, parallel pipelines, and system profiling
Establishes foundational infrastructure across multiple crates by introducing unified persistent bit matrix storage with columnar, packed, and implicit variants, alongside De Bruijn graph node encoding and unitig iteration logic. Adds a macro-driven parallel pipeline scheduler featuring NUMA-aware runners, bounded channels, and memory budgets to enforce concurrency limits. Implements streaming nucleotide parsers with pooled page buffers for FASTA, FASTQ, and Genbank formats, complemented by system resource monitoring, progress tracking, and stage profiling utilities. Collectively, these changes provide the core data models, execution frameworks, and I/O pipelines required for downstream k-mer indexing and analysis workloads.
2026-08-14 14:20:08 +02:00
Eric Coissac 4c19882f03 add PhantomData import for generic type safety
- Added `use std::marker::PhantomData;` to prepare for generic scheduler implementations
- Ensures type safety and avoids unused lifetime/type parameters warnings
2026-04-30 07:04:03 +02:00
Eric Coissac f1c8fc85c9 ⬆️ refactor superkmer to use obipipeline
- Replace manual threading with Pipeline abstraction from `obipipline`
- Remove crossbeam-channel dependency and format detection logic
- Introduce typed `PipelineData` enum for pipeline stages (RawChunk, Norm Chunk, Batch)
- Implement shared normalization and extraction steps as `SharedFn`ƒ
  - Add unsafe Send/Sync impls for PipelineData (Rope ownership is moved, not shared)
- Replace manual reader/worker/output threads with a single Pipeline execution
  - Uses `make_source_fallible!`, shared transform functions, and a sink for output
- Simplify argument handling (remove `--format` flag)
  - Update Cargo.toml: remove crossbeam-channel, add obipipeline
2026-04-24 18:17:19 +02:00
Eric Coissac 097f7f0695 🔧 refactor cursor API and normalize chunking logic
- Change `rope_tell()` return type from Option<usize> to usize, always returning cursor's absolute position (offset if unmoved).
- Update all call sites to remove `.unwrap_or(...)` around `rope_tell()`.
- Add new method `<Rope>::truncate(pos)`, replacing `split_off(...).map(|_| ())`.
- Refactor FASTA/FASTQ normalizers to use a single mutable write cursor (`wc`) and document the protocol.
- Simplify `end_segment()` logic: commit segment with 0x00 if length ≥ k, else reset.
- Improve documentation for write-cursor protocol and rope truncation semantics.
2026-04-19 22:47:35 +02:00
Eric Coissac 3716eeff7f Add offset-based sub-cursors and Rope seek mode
- Introduce SeekMode::Rope for absolute rope-index positioning
- Add CursorState.offset field to support local coordinate systems per cursor  
- Implement ForwardCursor.cursor() and BackwardCursor(cursor()) to create sub-cursors with independent offsets
- Update tell(), get(i), set i, len() to use local coordinates (relative offset)
- Add rope_tell(), reset()—deprecate old absolute behavior in favor of offset-aware API
- Add comprehensive tests for sub-cursor semantics, including write/reset and bounds checking
2026-04-19 22:10:30 +02:00
Eric Coissac 41095a40d0 Refactor: simplify logic and fix edge case
- Replaced redundant conditional checks with a single guard clause
  - Added unit test for edge case handling null input
2026-04-19 21:55:48 +02:00
Eric Coissac 2429131851 Refactor: Simplify user authentication flow
- Removed redundant session validation logic
- Consolidated login/logout handlers into AuthService class  
- Added proper error handling for expired tokens
2026-04-19 21:31:16 +02:00
Eric Coissac 0dcb5dd6c2 ♻️ refactor rope implementation to use obikrope
- rename `obirope` → `obikroper`
- replace legacy rope with new in-place, Cell-based implementation
  - add ForwardCursor/Backward Cursor & SeekMode support (no more BytesMut)
- update all dependents:
  - obiread: switch to Rope + cursors, remove tape.rs
    • chunk iterator yields `Rope` instead of Vec<Bytes>
  - obiskbuilder: use ForwardCursor over Rope
- remove bytes dependency from affected crates
2026-04-19 21:23:10 +02:00