2 Commits

Author	SHA1	Message	Date
Eric Coissac	a016ad5b8a	Refactor kmer index to disk-based partitioning with minimizer ... Refactor kmer index package to use disk-based partitioning with minimizer - Replace roaring64 bitmaps with disk-based kmer index - Implement partitioned kmer sets with delta-varint encoding - Add support for frequency filtering during construction - Introduce new builder pattern for index construction - Add streaming operations for set operations (union, intersect, etc.) - Add support for super-kmer encoding during construction - Update command line tool to use new index format - Remove dependency on roaring bitmap library This change introduces a new architecture for kmer indexing that is more memory efficient and scalable for large datasets.	2026-02-09 17:52:37 +01:00
Eric Coissac	500144051a	Add jj Makefile targets and k-mer encoding utilities ... Add new Makefile targets for jj operations (jjnew, jjpush, jjfetch) to streamline commit workflow. Introduce k-mer encoding utilities in pkg/obikmer: - EncodeKmers: converts DNA sequences to encoded k-mers - ReverseComplement: computes reverse complement of k-mers - NormalizeKmer: returns canonical form of k-mers - EncodeNormalizedKmers: encodes sequences with normalized k-mers Add comprehensive tests for k-mer encoding functions including edge cases, buffer reuse, and performance benchmarks. Document k-mer index design for large genomes, covering: - Use cases and objectives - Volume estimations - Distance metrics (Jaccard, Sørensen-Dice, Bray-Curtis) - Indexing options (Bloom filters, sorted sets, MPHF) - Optimization techniques (k-2-mer indexing) - MinHash for distance acceleration - Recommended architecture for presence/absence and counting queries	2026-02-04 14:27:10 +01:00