Files
obitools4/autodoc/docmd/pkg/obiformats/file_chunk_write.md
T

27 lines
1.7 KiB
Markdown
Raw Normal View History

2026-04-07 08:36:50 +02:00
# `WriteFileChunk` Function — Semantic Description
The `WriteFileChunk` function in the `obiformats` package implements a **thread-safe, ordered chunk writer** for streaming data to an `io.WriteCloser`. It accepts a destination writer and a flag indicating whether the writer should be closed upon completion.
- **Input**:
- `writer`: An `io.WriteCloser` (e.g., file, buffer) to which data chunks are written.
- `toBeClosed`: Boolean flag specifying if the writer should be closed after all chunks are processed.
- **Core Behavior**:
- Launches a goroutine that consumes `FileChunk` items from an unbuffered channel (`chunk_channel`).
- Ensures **strict sequential ordering** of chunks by their `Order` field (intended for reassembly after parallel or out-of-order processing).
- If a chunk arrives in order (`chunk.Order == nextToPrint`), it is immediately written.
- Out-of-order chunks are buffered in a map (`toBePrinted`) until their predecessor arrives.
- **Buffer Management**:
- After writing an in-order chunk, the function checks for newly consecutive buffered chunks and writes them greedily (e.g., if order 2 arrives, it triggers writing of buffered orders 3,4,... as available).
- **Error Handling**:
- Logs fatal errors on write failures or writer closure issues using `log.Fatalf`.
- **Cleanup & Lifecycle**:
- Closes the underlying writer if requested and unregisters a pipe registration (via `obiutils`) to signal end-of-stream.
- Returns the input channel, enabling external producers to stream `FileChunk` structs.
- **Use Case**:
Designed for robust, ordered reconstruction of large binary/data streams (e.g., sequencing reads) in OBITools4 pipelines, especially where parallel chunking and reassembly occur.