obitools/obitools4
2
0
Fork 0
mirror of https://github.com/metabarcoding/obitools4.git synced 2026-04-30 03:50:39 +00:00
Code Issues Packages Projects Releases Wiki Activity
758 Commits 13 Branches 35 Tags
f32b29db4f0236ffee8eba5a0b4e72d6bbbd9360
Commit Graph

46 Commits

Author SHA1 Message Date
Eric Coissac a92393dd51 ⬆️ update go.mod dependencies and improve error messages 
- Bump github.com/buger/jsonparser from v1.1.1 to v1.2
- Add error details in log.Fatalf calls for better debugging
 2026-04-13 14:08:13 +02:00
Eric Coissac c23368e929 update dependencies and Go toolchain to 1.25.0 
Update go.mod and go.work to Go 1.25.0, bump several direct dependencies (e.g., go-getoptions, gval, regexp2, go-json, progressbar, logrus, testify), update indirect dependencies accordingly, and remove obsolete toolchain directive.
 2026-03-13 16:09:34 +01:00
Eric Coissac f78543ee75 Refactor k-mer index building to use disk-based KmerSetGroupBuilder 
Refactor k-mer index building to use the new disk-based KmerSetGroupBuilder instead of the old KmerSet and FrequencyFilter approaches. This change introduces a more efficient and scalable approach to building k-mer indices by using partitioned disk storage with streaming operations.

- Replace BuildKmerIndex and BuildFrequencyFilterIndex with KmerSetGroupBuilder
- Add support for frequency filtering via WithMinFrequency option
- Remove deprecated k-mer set persistence methods
- Update CLI to use new builder approach
- Add new disk-based k-mer operations (union, intersect, difference, quorum)
- Introduce KDI (K-mer Delta Index) file format for efficient storage
- Add K-way merge operations for combining sorted k-mer streams
- Update documentation and examples to reflect new API

This refactoring provides better memory usage, faster operations on large datasets, and more flexible k-mer set operations.
 2026-02-10 06:49:31 +01:00
Eric Coissac b26b76cbf8 Add TOML persistence support for KmerSet and KmerSetGroup 
This commit adds support for saving and loading KmerSet and KmerSetGroup structures using TOML, YAML, and JSON formats for metadata. It includes:

- Added github.com/pelletier/go-toml/v2 dependency
- Implemented Save and Load methods for KmerSet and KmerSetGroup
- Added metadata persistence with support for multiple formats (TOML, YAML, JSON)
- Added helper functions for format detection and metadata handling
- Updated version commit hash
 2026-02-05 14:57:22 +01:00
Eric Coissac 28162ac36f Ajout du filtre de fréquence avec v niveaux Roaring Bitmaps 
Implémentation complète du filtre de fréquence utilisant v niveaux de Roaring Bitmaps pour éliminer efficacement les erreurs de séquençage.

- Ajout de la logique de filtrage par fréquence avec v niveaux
- Intégration des bibliothèques RoaringBitmap et bitset
- Ajout d'exemples d'utilisation et de documentation
- Implémentation de l'itérateur de k-mers pour une utilisation mémoire efficace
- Optimisation pour les distributions skewed typiques du séquençage

Ce changement permet de filtrer les k-mers par fréquence minimale avec une utilisation mémoire optimale et une seule passe sur les données.
 2026-02-04 21:21:10 +01:00
Eric Coissac c2f38e737b Update of the packages 2025-04-10 15:16:36 +02:00
Eric Coissac 8671285d02 add the --min-sample-count option to obiclean. 2025-02-24 08:48:31 +01:00
Eric Coissac 773e54965d Patch a bug on compressed output 2025-02-05 14:18:24 +01:00
Eric Coissac b9bee5f426 Changes to be committed: 
modified:   go.mod
	modified:   go.sum
	modified:   pkg/obilua/obilib.go
	modified:   pkg/obilua/obiseq.go
	modified:   pkg/obilua/obiseqslice.go
	new file:   pkg/obilua/obitaxon.go
	new file:   pkg/obilua/obitaxonomy.go
	modified:   pkg/obioptions/version.go
 2025-02-02 16:52:52 +01:00
Eric Coissac 795df34d1a Changes to be committed: 
modified:   cmd/obitools/obitag/main.go
	modified:   cmd/obitools/obitag2/main.go
	modified:   go.mod
	modified:   go.sum
	modified:   pkg/obiformats/ncbitaxdump/read.go
	modified:   pkg/obioptions/version.go
	modified:   pkg/obiseq/attributes.go
	modified:   pkg/obiseq/taxonomy_lca.go
	modified:   pkg/obiseq/taxonomy_methods.go
	modified:   pkg/obiseq/taxonomy_predicate.go
	modified:   pkg/obitax/inner.go
	modified:   pkg/obitax/lca.go
	new file:   pkg/obitax/taxid.go
	modified:   pkg/obitax/taxon.go
	modified:   pkg/obitax/taxonomy.go
	modified:   pkg/obitax/taxonslice.go
	modified:   pkg/obitools/obicleandb/obicleandb.go
	modified:   pkg/obitools/obigrep/options.go
	modified:   pkg/obitools/obilandmark/obilandmark.go
	modified:   pkg/obitools/obilandmark/options.go
	modified:   pkg/obitools/obirefidx/famlilyindexing.go
	modified:   pkg/obitools/obirefidx/geomindexing.go
	modified:   pkg/obitools/obirefidx/obirefidx.go
	modified:   pkg/obitools/obirefidx/options.go
	modified:   pkg/obitools/obitag/obigeomtag.go
	modified:   pkg/obitools/obitag/obitag.go
	modified:   pkg/obitools/obitag/options.go
	modified:   pkg/obiutils/strings.go
 2024-12-19 13:36:59 +01:00
Eric Coissac 3d06978808 a functional new version of obifind 2024-11-24 19:33:24 +01:00
Eric Coissac 9e8a7fd9be Patch a bug in fastq reader 2024-10-20 16:07:43 +02:00
Eric Coissac bdb96dda94 Adds the obimicrosat command 2024-08-05 15:31:20 +02:00
Eric Coissac c7ed47e110 first version of obidemerge, obijoin and a new filter for obicleandb but to be finnished 
Former-commit-id: 8a1ed26e5548c30db75644c294d478ec4d753f19
 2024-07-10 15:21:42 +02:00
Eric Coissac c1f03cb1f6 Switch to faster json library go-json and sonic 
Former-commit-id: ab9b4723f1dcf79fe5c073fff4d86f4f6969edfd
 2024-06-23 00:36:08 +02:00
Eric Coissac 411124d1b3 Add automatic rules to manage version number 
Former-commit-id: f4fcc1927f4169025c1d8cc88c5f3abcdc76037c
 2024-06-01 17:26:16 +02:00
Eric Coissac d249902073 A lot of changes and debug... A complete first 
version of the recipy in the Cookbook about
avaluating primers


Former-commit-id: 6c816f5546b3f472f6930d112abe3da04d2d5a55
 2024-05-26 12:56:29 +02:00
Eric Coissac 55ce36f329 Update of obipcr and homogenization of logging 
Former-commit-id: 46abf47c19ace5248042c02cf1f81d9f6c12eb10
 2024-05-16 15:18:30 +02:00
Eric Coissac e25c3b7365 Bug in the json parsing 
Former-commit-id: e652b04b9fa7e11124eb5930c7bec4613c808758
 2024-03-19 13:50:13 +01:00
coissac b40015deb7 first preliminary version of obiscript. 
Former-commit-id: 0d2c0fc5e33e0873ba5c04aca4cf7dd69aa83c90
 2024-03-06 12:56:44 -03:00
coissac 38c49e9f38 Switch GOPATH to the local directory in Makefile 
Former-commit-id: ae00f1136021e6fcd5047aadb4762653cf003d5d
 2024-02-27 07:22:57 +01:00
coissac 95caebec8f update modules 
Former-commit-id: 04651744d21f0db3406a0aa7bb9cdf9729140349
 2024-02-16 18:12:59 +01:00
coissac dedf125f6e Patch a bug in Subsequence and adds few unit tests. 
Former-commit-id: caddc9ad6523e4ef02899bfe83cc8681ef674383
 2023-11-08 10:16:34 +02:00
coissac 61c30f9b6a Patch rev complement and first implementation of --auto in obicsv 
Former-commit-id: f3020e81283b1073c4d1c2d2ff0887e3998e6764
 2023-11-07 09:37:07 +02:00
coissac 62b57f4ede A go implementation of the fasta reader 
Former-commit-id: 603592c4761fb0722e9e0501d78de1bd3ba238fa
 2023-09-01 09:30:12 +02:00
coissac 45ea4dacf7 Every small changes to include into Release 4.0.4 
Former-commit-id: b2a48c5b5c471f072785e02d4bb0f149a09cdf6f
 2023-08-27 17:22:51 +02:00
coissac be47ec909c add the --skip-empty option 
Former-commit-id: ec9cb0ecaf90a61bf9289cf4c089b5cc2fcb65a5
 2023-07-17 14:24:02 +02:00
coissac f6628a1ba2 Update install script 
Former-commit-id: 0dc6b22490fe6228e0099bd96fb1c53ee2754f03
 2023-05-23 10:09:42 +02:00
coissac 0746c95f4f Update the package dependencies 
Former-commit-id: e3fecfd65ec03f3a354338f16c52c130ea25c7f9
 2023-03-28 21:30:25 +07:00
coissac 98eac2360c adds a rule to update packages and push the corresponding updates module files 
Former-commit-id: 1ffa805e74571f9d970dcb185f187dd586c1a190
 2023-03-21 22:02:18 +07:00
coissac b3922c3896 Produce less weird crash on non existing files 
Former-commit-id: 74bb27bd53c685be530632994bd2ba24c1f362e1
 2023-03-07 17:34:25 +07:00
coissac d4b185b716 Adds some new dependencies 
Former-commit-id: 2f31ea6f852651e1ffca1d9ce78b17bddd26f2bb
 2023-03-07 11:12:39 +07:00
coissac 6c5fc8f65b Save change in various files 
Former-commit-id: 428f8ee77c584b79cc2ef45eef2902c3e0754c77
 2023-02-23 23:45:41 +01:00
coissac 08521c74e2 Adds Aho-Corasick matching to obiannotate 2023-02-09 15:59:11 +01:00
coissac 18be4becec Add some new packages 2022-11-14 14:36:04 +01:00
coissac aae3398701 Replace the CopyMap function by the MustFillMap and adds a InterfaceToIntMap function 2022-10-12 22:56:38 +02:00
coissac cd202ba72e Adds of new modules 2022-08-23 15:09:51 +02:00
coissac 5dd835d3e7 A first functional version of obiclean 2022-08-20 18:01:07 +02:00
coissac 0a36447e2a Adds the log module 2022-02-24 12:15:36 +01:00
coissac f18cc034bb add an option alias -O for the old obi format 2022-02-24 07:26:55 +01:00
coissac eb32620bb3 Adds a first version of a new obidistribute command 2022-02-14 00:01:01 +01:00
coissac 1544bafde1 Updates modules 2022-02-09 22:03:38 +01:00
coissac d5a1a191f8 update go.sum 2022-02-07 11:54:47 +01:00
coissac e1b7e1761c Change the memory management for thee BuildAlignment function 2022-01-15 19:10:16 +01:00
coissac b9b9c0f179 Patch module name from oa2 to obitools 2022-01-13 23:43:01 +01:00
coissac f53bf1b804 First commit 2022-01-13 23:27:39 +01:00