+ obiskio: add binary I/O with LRU pool and compression

- Add new obiskio crate for high-performance SuperKmer serialization/deserialization
- Implement binary codec with 2-bit packed sequence encoding and raw header format (32 bits)
- Add transparent compression support via niffler: Zstd, Gzip/Bgzf/Lz4
- Implement SKFilePool with LRU-based fd management, max-concurrent-fd limiting (75% of ulimit)
- Add SKFileWriter with batched writes, configurable flush threshold (8 KiB default), and two-phase locking
- Add SKFileReader with sequential access, LRU recovery via reopen_and_seek()
+ New obikpartitionner crate: basic header/seq handling for binary super-kmer format
- Bump niffler from 2.7 to v3, add dependencies: allocator-api2, bitflags(>=1), errno/fastrand/rustix/tempfile/lru/hashbrown/bzip2/thiserror
- Update workspace members to include obikpartitionner andobiskio

This commit is contained in:

Eric Coissac

2026-04-24 21:07:58 +02:00

parent d4e4289aff

commit c09d17401d

13 changed files with 1324 additions and 5 deletions

									
										src/obikpartitionner/Cargo.toml
									
		+6
		
												View File
												
				@@ -0,0 +1,6 @@

				[package]

				name = "obikpartitionner"

				version = "0.1.0"

				edition = "2024"

				[dependencies]