docs: clarify merge re-merge behavior and input file handling

Update documentation to explain that merged indices can serve as sources if at least one source is unmerged, and that the resulting column order follows the base index first. Clarify that explicitly named files are always read regardless of extension, with format detection based on content.

- refactor: prevent I/O errors when re-merging packed outputs by restricting `choose_base` to valid destinations via new `can_be_destination` check
- fix: handle zero-layer partitions in `build_index_layer` and `remove_build_artifacts` to skip unnecessary operations
- feat: allow `PathIter` to treat unrecognized extensions as explicit files, ensuring they are not silently dropped during directory expansion
- test: add comprehensive tests for `PathIter` including extension recognition, explicit file handling, and stdin sentinel behavior
- test: update smoke tests to verify empty partitions, `.fna.gz` indexing, and refusal of merging only packed indexes
- chore: remove obsolete cargo build environment variables from `run.sh` script
This commit is contained in:
Eric Coissac committed 2026-11-19 07:10:58 +01:00
1 parent 07c0a54e04
commit 7f829ea557
11 files changed
+262 -39

No files matched your search

+6 -2
View File
@@ -36,6 +36,10 @@ The merge works partition by partition, and processes several partitions in para
Each genome column being filled is held in memory and written to disk in one go once it is complete. A full disk or an exhausted quota is therefore reported as an ordinary error rather than crashing the program. The largest memory cost comes from the k-mers that are new to the merged index, which are assembled in memory, partition by partition, before being written out.
## Current limitation
## Merging an index that was already merged
An index produced by `merge` cannot itself be used as a source of a later merge yet: the command stops with a "No such file or directory" error. To add genomes to a collection, merge the original single-genome indexes again, together with the new ones.
An index produced by `merge` can be used as one of the sources of a later merge, which is how a collection is extended without starting again from the original genomes. There is one rule: among the sources, at least one must not have been through a merge. A merged index is stored in a compact packed format (see [pack](pack.md)) that is read very efficiently but cannot be extended in place. The merge therefore takes a freshly built index as its starting point and brings the packed index in as a source. A typical case is a merged index of many genomes plus one or a few newly built single-genome indexes, and this works.
If every source has already been merged or packed, the command stops before doing any work and says so. In that case, merge the original single-genome indexes together instead.
The genome columns of the result follow the order of the starting index first, then the other sources, so they may not come out in the order the sources were listed. The genome labels always identify each column unambiguously.