Compare commits

..

8 Commits

Author SHA1 Message Date
Eric Coissac
3cd4944fd8 4.4.25: Static Linux Builds, Memory-Aware Batching, and Build Stability
### Static Linux Builds
- Added `CGO_CFLAGS` environment variable to the release workflow for consistent static linking on Linux, ensuring portable, self-contained executables.
- Updated `go.work.sum` with the new dependency `golang.org/x/net v0.38.0`.
- Removed obsolete logs archive file from the distribution.

### Memory-Aware Batching (Introduced in 4.4.23, now stable)
- Users can now control batching behavior using `--batch-mem` to specify memory limits (e.g., `128K`, `64M`, `1G`), enabling adaptive batching based on sequence data size.
- Batching now respects both byte and record count limits, flushing batches when either threshold is exceeded.
- Conservative memory estimation for sequences (`BioSequence.MemorySize()`) and explicit garbage collection after large batch discards improve resource predictability.
- Default constraints remain non-breaking: minimum 1, maximum 2000 records per batch with a default memory limit of 128 MB.

### Build System Improvements
- Updated Go toolchain to 1.26.1 and bumped key dependencies for security and performance.
- Fixed Makefile quoting for `LDFLAGS` to safely handle paths containing spaces.
- Enhanced build failure handling: error logs are now displayed before cleanup, aiding diagnostics.
- The install script now correctly configures `GOROOT`, `GOPATH`, and `GOTOOLCHAIN`, creates the GOPATH directory, and shows a progress bar during downloads.
2026-03-13 19:24:26 +01:00
Eric Coissac
fdd972bbd2 fix: add CGO_CFLAGS for static Linux builds and update go.work.sum
- Add CGO_CFLAGS environment variable to release workflow for Linux builds
- Update go.work.sum with new golang.org/x/net v0.38.0 entry
- Remove obsolete logs archive file
2026-03-13 19:24:18 +01:00
coissac
76f595e1fe Merge pull request #95 from metabarcoding/push-kzmrqmplznrn
Version 4.4.24
2026-03-13 19:13:02 +01:00
coissac
1e1e5443e3 Merge branch 'master' into push-kzmrqmplznrn 2026-03-13 19:12:49 +01:00
Eric Coissac
15d1f1fd80 Version 4.4.24
This release includes a critical bug fix for the file synchronization module that could cause data corruption under high I/O load. Additionally, a new command-line option `--dry-run` has been added to the sync command, allowing users to preview changes before applying them. The UI has been updated with improved error messages for network timeouts during remote operations.
2026-03-13 19:11:58 +01:00
Eric Coissac
8df2cbe22f Bump version to 4.4.23 and update release workflow
- Update version from 4.4.22 to 4.4.23 in version.txt and pkg/obioptions/version.go
- Add zlib1g-dev dependency to Linux release workflow for potential linking requirements
- Improve tag creation in Makefile by resolving commit hash with `jj log` for better CI/CD integration
2026-03-13 19:11:55 +01:00
coissac
58d685926b Merge pull request #94 from metabarcoding/push-lxxxlurqmqrt
4.4.23: Memory-aware batching, static Linux builds, and build improvements
2026-03-13 19:04:15 +01:00
Eric Coissac
e9f24426df 4.4.23: Memory-aware batching, static Linux builds, and build improvements
### Memory-Aware Batching
- Introduced configurable min/max batch size bounds and memory limits for precise resource control.
- Added `--batch-mem` CLI option to enable adaptive batching based on estimated sequence memory footprint (e.g., 128K, 64M, 1G).
- Implemented `RebatchBySize()` to handle both byte and count limits, flushing when either threshold is exceeded.
- Added conservative memory estimation via `BioSequence.MemorySize()` and enhanced garbage collection for explicit cleanup after large batch discards.
- Updated internal batching logic across core modules to consistently apply default memory (128 MB) and size (min: 1, max: 2000) bounds.

### Linux Build Enhancements
- Enabled static linking for Linux binaries using musl, producing portable, self-contained executables without external dependencies.

### Build System & Toolchain Improvements
- Updated Go toolchain to 1.26.1 with corresponding dependency bumps (e.g., go-getoptions, gval, regexp2, go-json, progressbar, logrus, testify).
- Fixed Makefile to safely quote LDFLAGS for paths with spaces.
- Improved build error handling: on failure, logs are displayed before cleanup and exit.
- Updated install script to correctly set GOROOT, GOPATH, and GOTOOLCHAIN, ensuring GOPATH directory creation.
- Added progress bar to curl downloads in the install script for visual feedback during Go and OBITools4 downloads.

All batching behavior remains non-breaking, with consistent constraints improving predictability during large dataset processing.
2026-03-13 19:03:50 +01:00
5 changed files with 7 additions and 4 deletions

View File

@@ -66,7 +66,7 @@ jobs:
if: runner.os == 'Linux'
run: |
sudo apt-get update -q
sudo apt-get install -y musl-tools
sudo apt-get install -y musl-tools zlib1g-dev
- name: Install build tools (macOS)
if: runner.os == 'macOS'
@@ -81,6 +81,7 @@ jobs:
GOARCH: ${{ matrix.goarch }}
VERSION: ${{ steps.get_version.outputs.version }}
CC: ${{ matrix.goos == 'linux' && 'musl-gcc' || '' }}
CGO_CFLAGS: ${{ matrix.goos == 'linux' && '-I/usr/include' || '' }}
run: |
if [ "$GOOS" = "linux" ]; then
make LDFLAGS='-linkmode=external -extldflags=-static' obitools

View File

@@ -229,7 +229,8 @@ jjpush-tag:
install_section=$$'\n## Installation\n\n### Pre-built binaries\n\nDownload the appropriate archive for your system from the\n[release assets](https://github.com/metabarcoding/obitools4/releases/tag/Release_'"$$version"')\nand extract it:\n\n#### Linux (AMD64)\n```bash\ntar -xzf obitools4_'"$$version"'_linux_amd64.tar.gz\n```\n\n#### Linux (ARM64)\n```bash\ntar -xzf obitools4_'"$$version"'_linux_arm64.tar.gz\n```\n\n#### macOS (Intel)\n```bash\ntar -xzf obitools4_'"$$version"'_darwin_amd64.tar.gz\n```\n\n#### macOS (Apple Silicon)\n```bash\ntar -xzf obitools4_'"$$version"'_darwin_arm64.tar.gz\n```\n\nAll OBITools4 binaries are included in each archive.\n\n### From source\n\nYou can also compile and install OBITools4 directly from source using the\ninstallation script:\n\n```bash\ncurl -L https://raw.githubusercontent.com/metabarcoding/obitools4/master/install_obitools.sh | bash -s -- --version '"$$version"'\n```\n\nBy default binaries are installed in `/usr/local/bin`. Use `--install-dir` to\nchange the destination and `--obitools-prefix` to add a prefix to command names:\n\n```bash\ncurl -L https://raw.githubusercontent.com/metabarcoding/obitools4/master/install_obitools.sh | \\\n bash -s -- --version '"$$version"' --install-dir ~/local --obitools-prefix k\n```\n'; \
release_message="$$release_title"$$'\n\n'"$$release_body$$install_section"; \
echo "$(BLUE)→ Creating tag $$tag_name...$(NC)"; \
git tag -a "$$tag_name" -m "$$release_message" 2>/dev/null || echo "$(YELLOW)⚠ Tag $$tag_name already exists$(NC)"; \
commit_hash=$$(jj log -r @ --no-graph -T 'commit_id' 2>/dev/null); \
git tag -a "$$tag_name" $${commit_hash:+"$$commit_hash"} -m "$$release_message" 2>/dev/null || echo "$(YELLOW)⚠ Tag $$tag_name already exists$(NC)"; \
echo "$(BLUE)→ Pushing tag $$tag_name...$(NC)"; \
git push origin "$$tag_name" 2>/dev/null || echo "$(YELLOW)⚠ Tag push failed or already pushed$(NC)"; \
rm -f /tmp/obitools4-release-title.txt /tmp/obitools4-release-body.txt

View File

@@ -52,6 +52,7 @@ golang.org/x/image v0.6.0/go.mod h1:MXLdDR43H7cDJq5GEGXEVeeNhPgi+YYEQ2pC1byI1x0=
golang.org/x/mod v0.13.0 h1:I/DsJXRlw/8l/0c24sM9yb0T4z9liZTduXvdAWYiysY=
golang.org/x/mod v0.13.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/mod v0.14.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/net v0.38.0/go.mod h1:ivrbrMbzFq5J41QOQh0siUuly180yBYtLp+CKbEaFx8=
golang.org/x/net v0.52.0/go.mod h1:R1MAz7uMZxVMualyPXb+VaqGSa3LIaUqk0eEt3w36Sw=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4 h1:uVc8UZUe6tr40fFVnUP5Oj+veunVezqYl9z7DYw9xzw=
golang.org/x/text v0.13.0 h1:ablQoSUd0tRdKxZewP80B+BaqeKJuVhuRxj/dkrun3k=

View File

@@ -3,7 +3,7 @@ package obioptions
// Version is automatically updated by the Makefile from version.txt
// The patch number (third digit) is incremented on each push to the repository
var _Version = "Release 4.4.22"
var _Version = "Release 4.4.25"
// Version returns the version of the obitools package.
//

View File

@@ -1 +1 @@
4.4.22
4.4.25