mirror of
https://github.com/metabarcoding/obitools4.git
synced 2026-04-30 12:00:39 +00:00
Eric Coissac
8c7017a99d
- Update obioptions.Version from "Release 4.4.29" to "/v/ Release v5" - Update version.txt from 4.29 → .30 (automated by Makefile)
1.3 KiB
1.3 KiB
CSV Export Functionality in obicsv Package
The obicsv package provides utilities for efficiently writing structured data (e.g., sequence annotations) to CSV format, supporting parallel processing and streaming.
-
FormatCVSBatch(): Converts a batch of CSV records (CSVRecordBatch) into an in-memory buffer, using the provided header and a placeholder for missing values (navalue). It prepends the header only once (for batch order 0). -
WriteCSV(): Writes a CSV-formatted stream from anICSVRecorditerator to anyio.WriteCloser. It supports:- Compression (via
obiutils.CompressStream) - Parallel workers for batch processing (
ParallelWorkers()) - Chunked writing via
obiformats.WriteFileChunk
- Compression (via
-
WriteCSVToStdout()/WriteCSVToFile(): Convenience wrappers:- Outputs to stdout (
os.Stdout) - Writes to a file (with
O_WRONLY, optional append/truncate)
- Outputs to stdout (
-
Key design features:
- Non-blocking, concurrent processing using goroutines
- Graceful shutdown via
WaitAndClose()and channel signaling - Robust handling of missing/invalid values (falls back to
navalue)
-
Dependencies: Leverages internal packages for iteration (
obiitercsv), data formats (obiformats), and utilities (obiutils,logruslogging).