Update documentation to explain that merged indices can serve as sources if at least one source is unmerged, and that the resulting column order follows the base index first. Clarify that explicitly named files are always read regardless of extension, with format detection based on content.
- refactor: prevent I/O errors when re-merging packed outputs by restricting `choose_base` to valid destinations via new `can_be_destination` check
- fix: handle zero-layer partitions in `build_index_layer` and `remove_build_artifacts` to skip unnecessary operations
- feat: allow `PathIter` to treat unrecognized extensions as explicit files, ensuring they are not silently dropped during directory expansion
- test: add comprehensive tests for `PathIter` including extension recognition, explicit file handling, and stdin sentinel behavior
- test: update smoke tests to verify empty partitions, `.fna.gz` indexing, and refusal of merging only packed indexes
- chore: remove obsolete cargo build environment variables from `run.sh` script
Add a prompt template for optimizing the `pack` step and a suite of Docker-based
debug tools including build, run, and monitoring scripts. Enhance `pack_matrices`
with LFD partition sizing and detailed timing reports, and update `stop` with
additional system statistics.
- Add `prompts/pack_optimization.md` to guide optimization of the `pack` step,
specifying context, performance measurements, and deliverable structures.
- Add `scripts/docker_debug/` infrastructure: Dockerfile for musl targets,
`build.sh` for image creation, `run.sh` for parallel indexing and merging,
`monitor.sh` for PID tracking, `gen_fasta.py` for synthetic genomes, and
`gen_fasta_real.py` for real genome generation with `read_fna`.
- Add `scripts/monitor_pack.sh` as a 1 Hz sampler for I/O and memory metrics,
writing samples to CSV and summary reports.
- Add `scripts/watch_merge.sh` to monitor process resource usage, enforcing
memory limits via SIGTERM/SIGKILL guards based on VmRSS and VmSize.
- Modify `src/obikindex/src/index/kmer_index.rs`: add `pending_matrix_bytes` and
`report_pack_timings`; update `pack_matrices` to use LFD strategy for partition
sizing and clarify documentation regarding `stat()` proxy bytes.
- Modify `src/obisys/src/stage.rs`: update `stop` definition to include new
variables (`user_secs`, `sys_secs`, faults, context switches, I/O blocks) and
logic checking for swap usage.