Refactor kmer index to disk-based partitioning with minimizer

Refactor kmer index package to use disk-based partitioning with minimizer

- Replace roaring64 bitmaps with disk-based kmer index
- Implement partitioned kmer sets with delta-varint encoding
- Add support for frequency filtering during construction
- Introduce new builder pattern for index construction
- Add streaming operations for set operations (union, intersect, etc.)
- Add support for super-kmer encoding during construction
- Update command line tool to use new index format
- Remove dependency on roaring bitmap library

This change introduces a new architecture for kmer indexing that is more memory efficient and scalable for large datasets.
This commit is contained in:
Eric Coissac
2026-02-09 17:50:33 +01:00
parent 09d437d10f
commit a016ad5b8a
8 changed files with 987 additions and 0 deletions

View File

@@ -82,6 +82,13 @@ func (ks *KmerSet) AddSequences(sequences *obiseq.BioSequenceSlice) {
}
}
// AddSequenceSlice adds all k-mers from a slice of sequences
func (ks *KmerSet) AddSequenceSlice(sequences *obiseq.BioSequenceSlice) {
for _, seq := range *sequences {
ks.AddSequence(seq)
}
}
// Contains checks if a k-mer is in the set
func (ks *KmerSet) Contains(kmer uint64) bool {
return ks.bitmap.Contains(kmer)