mirror of
https://github.com/metabarcoding/obitools4.git
synced 2026-03-25 13:30:52 +00:00
Add obisuperkmer command implementation and tests
This commit adds the implementation of the obisuperkmer command, including: - The main command in cmd/obitools/obisuperkmer/ - The package implementation in pkg/obitools/obisuperkmer/ - Automated tests in obitests/obitools/obisuperkmer/ - Documentation for the implementation and tests The obisuperkmer command extracts super k-mers from DNA sequences, following the standard OBITools architecture. It includes proper CLI option handling, validation of parameters, and integration with the OBITools pipeline system. Tests cover basic functionality, parameter validation, output format, metadata preservation, and file I/O operations.
This commit is contained in:
Eric Coissac
148
obitests/obitools/obisuperkmer/README.md
Normal file
148
obitests/obitools/obisuperkmer/README.md
Normal file
@@ -0,0 +1,148 @@
|
||||
# Tests pour obisuperkmer
|
||||
|
||||
## Description
|
||||
|
||||
Ce répertoire contient les tests automatisés pour la commande `obisuperkmer`.
|
||||
|
||||
## Fichiers
|
||||
|
||||
- `test.sh` : Script de test principal (exécutable)
|
||||
- `test_sequences.fasta` : Jeu de données de test minimal (3 séquences courtes)
|
||||
- `README.md` : Ce fichier
|
||||
|
||||
## Jeu de données de test
|
||||
|
||||
Le fichier `test_sequences.fasta` contient 3 séquences de 32 nucléotides chacune :
|
||||
|
||||
1. **seq1** : Répétition du motif ACGT (séquence régulière)
|
||||
2. **seq2** : Alternance de blocs homopolymères (AAAA, CCCC, GGGG, TTTT)
|
||||
3. **seq3** : Répétition du motif ATCG (différent de seq1)
|
||||
|
||||
Ces séquences sont volontairement courtes pour :
|
||||
- Minimiser la taille du dépôt Git
|
||||
- Accélérer l'exécution des tests en CI/CD
|
||||
- Tester différents cas d'extraction de super k-mers
|
||||
|
||||
## Tests effectués
|
||||
|
||||
Le script `test.sh` effectue 12 tests :
|
||||
|
||||
### Test 1 : Affichage de l'aide
|
||||
Vérifie que `obisuperkmer -h` s'exécute correctement.
|
||||
|
||||
### Test 2 : Extraction basique avec paramètres par défaut
|
||||
Exécute `obisuperkmer` avec k=21, m=11 (valeurs par défaut).
|
||||
|
||||
### Test 3 : Vérification du fichier de sortie non vide
|
||||
S'assure que la commande produit une sortie.
|
||||
|
||||
### Test 4 : Comptage des super k-mers extraits
|
||||
Vérifie qu'au moins un super k-mer a été extrait.
|
||||
|
||||
### Test 5 : Présence des métadonnées requises
|
||||
Vérifie que chaque super k-mer contient :
|
||||
- `minimizer_value`
|
||||
- `minimizer_seq`
|
||||
- `parent_id`
|
||||
|
||||
### Test 6 : Extraction avec paramètres personnalisés
|
||||
Teste avec k=15 et m=7.
|
||||
|
||||
### Test 7 : Vérification des paramètres dans les métadonnées
|
||||
S'assure que les valeurs k=15 et m=7 sont présentes dans la sortie.
|
||||
|
||||
### Test 8 : Format de sortie FASTA explicite
|
||||
Teste l'option `--fasta-output`.
|
||||
|
||||
### Test 9 : Vérification des IDs des super k-mers
|
||||
S'assure que tous les IDs contiennent "superkmer".
|
||||
|
||||
### Test 10 : Préservation des IDs parents
|
||||
Vérifie que seq1, seq2 et seq3 apparaissent dans la sortie.
|
||||
|
||||
### Test 11 : Option -o pour fichier de sortie
|
||||
Teste la redirection vers un fichier avec `-o`.
|
||||
|
||||
### Test 12 : Vérification de la création du fichier avec -o
|
||||
S'assure que le fichier de sortie a été créé.
|
||||
|
||||
### Test 13 : Cohérence des longueurs
|
||||
Vérifie que la somme des longueurs des super k-mers est inférieure ou égale à la longueur totale des séquences d'entrée.
|
||||
|
||||
## Exécution des tests
|
||||
|
||||
### Localement
|
||||
|
||||
```bash
|
||||
cd /chemin/vers/obitools4/obitests/obitools/obisuperkmer
|
||||
./test.sh
|
||||
```
|
||||
|
||||
### En CI/CD
|
||||
|
||||
Les tests sont automatiquement exécutés lors de chaque commit via le système CI/CD configuré pour le projet.
|
||||
|
||||
### Prérequis
|
||||
|
||||
- La commande `obisuperkmer` doit être compilée et disponible dans `../../build/`
|
||||
- Les dépendances système : bash, grep, etc.
|
||||
|
||||
## Structure du script de test
|
||||
|
||||
Le script suit le pattern standard utilisé par tous les tests OBITools :
|
||||
|
||||
1. **En-tête** : Définition du nom du test et de la commande
|
||||
2. **Variables** : Configuration des chemins et compteurs
|
||||
3. **Fonction cleanup()** : Affiche les résultats et nettoie le répertoire temporaire
|
||||
4. **Fonction log()** : Affiche les messages horodatés
|
||||
5. **Tests** : Série de tests avec incrémentation des compteurs
|
||||
6. **Appel cleanup()** : Nettoyage et sortie avec code de retour approprié
|
||||
|
||||
## Format de sortie
|
||||
|
||||
Chaque test affiche :
|
||||
```
|
||||
[obisuperkmer @ date] message
|
||||
```
|
||||
|
||||
En fin d'exécution :
|
||||
```
|
||||
========================================
|
||||
## Results of the obisuperkmer tests:
|
||||
|
||||
- 12 tests run
|
||||
- 12 successfully completed
|
||||
- 0 failed tests
|
||||
|
||||
Cleaning up the temporary directory...
|
||||
|
||||
========================================
|
||||
```
|
||||
|
||||
## Codes de retour
|
||||
|
||||
- **0** : Tous les tests ont réussi
|
||||
- **1** : Au moins un test a échoué
|
||||
|
||||
## Ajout de nouveaux tests
|
||||
|
||||
Pour ajouter un nouveau test, suivre le pattern :
|
||||
|
||||
```bash
|
||||
((ntest++))
|
||||
if commande_test arguments
|
||||
then
|
||||
log "Description: OK"
|
||||
((success++))
|
||||
else
|
||||
log "Description: failed"
|
||||
((failed++))
|
||||
fi
|
||||
```
|
||||
|
||||
## Notes
|
||||
|
||||
- Les fichiers temporaires sont créés dans `$TMPDIR` (créé par mktemp)
|
||||
- Les fichiers de données sont dans `$TEST_DIR`
|
||||
- La commande testée doit être dans `$OBITOOLS_DIR` (../../build/)
|
||||
- Le répertoire temporaire est automatiquement nettoyé à la fin
|
||||
256
obitests/obitools/obisuperkmer/test.sh
Executable file
256
obitests/obitools/obisuperkmer/test.sh
Executable file
@@ -0,0 +1,256 @@
|
||||
#!/bin/bash
|
||||
|
||||
#
|
||||
# Here give the name of the test serie
|
||||
#
|
||||
|
||||
TEST_NAME=obisuperkmer
|
||||
CMD=obisuperkmer
|
||||
|
||||
######
|
||||
#
|
||||
# Some variable and function definitions: please don't change them
|
||||
#
|
||||
######
|
||||
TEST_DIR="$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")"
|
||||
OBITOOLS_DIR="${TEST_DIR/obitest*/}build"
|
||||
export PATH="${OBITOOLS_DIR}:${PATH}"
|
||||
|
||||
MCMD="$(echo "${CMD:0:4}" | tr '[:lower:]' '[:upper:]')$(echo "${CMD:4}" | tr '[:upper:]' '[:lower:]')"
|
||||
|
||||
TMPDIR="$(mktemp -d)"
|
||||
ntest=0
|
||||
success=0
|
||||
failed=0
|
||||
|
||||
cleanup() {
|
||||
echo "========================================" 1>&2
|
||||
echo "## Results of the $TEST_NAME tests:" 1>&2
|
||||
|
||||
echo 1>&2
|
||||
echo "- $ntest tests run" 1>&2
|
||||
echo "- $success successfully completed" 1>&2
|
||||
echo "- $failed failed tests" 1>&2
|
||||
echo 1>&2
|
||||
echo "Cleaning up the temporary directory..." 1>&2
|
||||
echo 1>&2
|
||||
echo "========================================" 1>&2
|
||||
|
||||
rm -rf "$TMPDIR" # Suppress the temporary directory
|
||||
|
||||
if [ $failed -gt 0 ]; then
|
||||
log "$TEST_NAME tests failed"
|
||||
log
|
||||
log
|
||||
exit 1
|
||||
fi
|
||||
|
||||
log
|
||||
log
|
||||
|
||||
exit 0
|
||||
}
|
||||
|
||||
log() {
|
||||
echo -e "[$TEST_NAME @ $(date)] $*" 1>&2
|
||||
}
|
||||
|
||||
log "Testing $TEST_NAME..."
|
||||
log "Test directory is $TEST_DIR"
|
||||
log "obitools directory is $OBITOOLS_DIR"
|
||||
log "Temporary directory is $TMPDIR"
|
||||
log "files: $(find $TEST_DIR | awk -F'/' '{print $NF}' | tail -n +2)"
|
||||
|
||||
######################################################################
|
||||
####
|
||||
#### Below are the tests
|
||||
####
|
||||
#### Before each test :
|
||||
#### - increment the variable ntest
|
||||
####
|
||||
#### Run the command as the condition of an if / then /else
|
||||
#### - The command must return 0 on success
|
||||
#### - The command must return an exit code different from 0 on failure
|
||||
#### - The datafiles are stored in the same directory than the test script
|
||||
#### - The test script directory is stored in the TEST_DIR variable
|
||||
#### - If result files have to be produced they must be stored
|
||||
#### in the temporary directory (TMPDIR variable)
|
||||
####
|
||||
#### then clause is executed on success of the command
|
||||
#### - Write a success message using the log function
|
||||
#### - increment the variable success
|
||||
####
|
||||
#### else clause is executed on failure of the command
|
||||
#### - Write a failure message using the log function
|
||||
#### - increment the variable failed
|
||||
####
|
||||
######################################################################
|
||||
|
||||
|
||||
|
||||
((ntest++))
|
||||
if $CMD -h > "${TMPDIR}/help.txt" 2>&1
|
||||
then
|
||||
log "$MCMD: printing help OK"
|
||||
((success++))
|
||||
else
|
||||
log "$MCMD: printing help failed"
|
||||
((failed++))
|
||||
fi
|
||||
|
||||
# Test 1: Basic super k-mer extraction with default parameters
|
||||
((ntest++))
|
||||
if obisuperkmer "${TEST_DIR}/test_sequences.fasta" \
|
||||
> "${TMPDIR}/output_default.fasta" 2>&1
|
||||
then
|
||||
log "$MCMD: basic extraction with default parameters OK"
|
||||
((success++))
|
||||
else
|
||||
log "$MCMD: basic extraction with default parameters failed"
|
||||
((failed++))
|
||||
fi
|
||||
|
||||
# Test 2: Verify output is not empty
|
||||
((ntest++))
|
||||
if [ -s "${TMPDIR}/output_default.fasta" ]
|
||||
then
|
||||
log "$MCMD: output file is not empty OK"
|
||||
((success++))
|
||||
else
|
||||
log "$MCMD: output file is empty - failed"
|
||||
((failed++))
|
||||
fi
|
||||
|
||||
# Test 3: Count number of super k-mers extracted (should be > 0)
|
||||
((ntest++))
|
||||
num_sequences=$(grep -c "^>" "${TMPDIR}/output_default.fasta")
|
||||
if [ "$num_sequences" -gt 0 ]
|
||||
then
|
||||
log "$MCMD: extracted $num_sequences super k-mers OK"
|
||||
((success++))
|
||||
else
|
||||
log "$MCMD: no super k-mers extracted - failed"
|
||||
((failed++))
|
||||
fi
|
||||
|
||||
# Test 4: Verify super k-mers have required metadata attributes
|
||||
((ntest++))
|
||||
if grep -q "minimizer_value" "${TMPDIR}/output_default.fasta" && \
|
||||
grep -q "minimizer_seq" "${TMPDIR}/output_default.fasta" && \
|
||||
grep -q "parent_id" "${TMPDIR}/output_default.fasta"
|
||||
then
|
||||
log "$MCMD: super k-mers contain required metadata OK"
|
||||
((success++))
|
||||
else
|
||||
log "$MCMD: super k-mers missing metadata - failed"
|
||||
((failed++))
|
||||
fi
|
||||
|
||||
# Test 5: Extract super k-mers with custom k and m parameters
|
||||
((ntest++))
|
||||
if obisuperkmer -k 15 -m 7 "${TEST_DIR}/test_sequences.fasta" \
|
||||
> "${TMPDIR}/output_k15_m7.fasta" 2>&1
|
||||
then
|
||||
log "$MCMD: extraction with custom k=15, m=7 OK"
|
||||
((success++))
|
||||
else
|
||||
log "$MCMD: extraction with custom k=15, m=7 failed"
|
||||
((failed++))
|
||||
fi
|
||||
|
||||
# Test 6: Verify custom parameters in output metadata
|
||||
((ntest++))
|
||||
if grep -q '"k":15' "${TMPDIR}/output_k15_m7.fasta" && \
|
||||
grep -q '"m":7' "${TMPDIR}/output_k15_m7.fasta"
|
||||
then
|
||||
log "$MCMD: custom parameters correctly set in metadata OK"
|
||||
((success++))
|
||||
else
|
||||
log "$MCMD: custom parameters not in metadata - failed"
|
||||
((failed++))
|
||||
fi
|
||||
|
||||
# Test 7: Test with different output format (FASTA output explicitly)
|
||||
((ntest++))
|
||||
if obisuperkmer --fasta-output -k 21 -m 11 \
|
||||
"${TEST_DIR}/test_sequences.fasta" \
|
||||
> "${TMPDIR}/output_fasta.fasta" 2>&1
|
||||
then
|
||||
log "$MCMD: FASTA output format OK"
|
||||
((success++))
|
||||
else
|
||||
log "$MCMD: FASTA output format failed"
|
||||
((failed++))
|
||||
fi
|
||||
|
||||
# Test 8: Verify all super k-mers have superkmer in their ID
|
||||
((ntest++))
|
||||
if grep "^>" "${TMPDIR}/output_default.fasta" | grep -q "superkmer"
|
||||
then
|
||||
log "$MCMD: super k-mer IDs contain 'superkmer' OK"
|
||||
((success++))
|
||||
else
|
||||
log "$MCMD: super k-mer IDs missing 'superkmer' - failed"
|
||||
((failed++))
|
||||
fi
|
||||
|
||||
# Test 9: Verify parent sequence IDs are preserved
|
||||
((ntest++))
|
||||
if grep -q "seq1" "${TMPDIR}/output_default.fasta" && \
|
||||
grep -q "seq2" "${TMPDIR}/output_default.fasta" && \
|
||||
grep -q "seq3" "${TMPDIR}/output_default.fasta"
|
||||
then
|
||||
log "$MCMD: parent sequence IDs preserved OK"
|
||||
((success++))
|
||||
else
|
||||
log "$MCMD: parent sequence IDs not preserved - failed"
|
||||
((failed++))
|
||||
fi
|
||||
|
||||
# Test 10: Test with output file option
|
||||
((ntest++))
|
||||
if obisuperkmer -o "${TMPDIR}/output_file.fasta" \
|
||||
"${TEST_DIR}/test_sequences.fasta" 2>&1
|
||||
then
|
||||
log "$MCMD: output to file with -o option OK"
|
||||
((success++))
|
||||
else
|
||||
log "$MCMD: output to file with -o option failed"
|
||||
((failed++))
|
||||
fi
|
||||
|
||||
# Test 11: Verify output file was created with -o option
|
||||
((ntest++))
|
||||
if [ -s "${TMPDIR}/output_file.fasta" ]
|
||||
then
|
||||
log "$MCMD: output file created with -o option OK"
|
||||
((success++))
|
||||
else
|
||||
log "$MCMD: output file not created with -o option - failed"
|
||||
((failed++))
|
||||
fi
|
||||
|
||||
# Test 12: Verify super k-mers are shorter than or equal to parent sequences
|
||||
((ntest++))
|
||||
# Count nucleotides in input sequences (excluding headers)
|
||||
input_bases=$(grep -v "^>" "${TEST_DIR}/test_sequences.fasta" | tr -d '\n' | wc -c)
|
||||
# Count nucleotides in output sequences (excluding headers)
|
||||
output_bases=$(grep -v "^>" "${TMPDIR}/output_default.fasta" | tr -d '\n' | wc -c)
|
||||
|
||||
if [ "$output_bases" -le "$input_bases" ]
|
||||
then
|
||||
log "$MCMD: super k-mer total length <= input length OK"
|
||||
((success++))
|
||||
else
|
||||
log "$MCMD: super k-mer total length > input length - failed"
|
||||
((failed++))
|
||||
fi
|
||||
|
||||
#########################################
|
||||
#
|
||||
# At the end of the tests
|
||||
# the cleanup function is called
|
||||
#
|
||||
#########################################
|
||||
|
||||
cleanup
|
||||
6
obitests/obitools/obisuperkmer/test_sequences.fasta
Normal file
6
obitests/obitools/obisuperkmer/test_sequences.fasta
Normal file
@@ -0,0 +1,6 @@
|
||||
>seq1
|
||||
ACGTACGTACGTACGTACGTACGTACGTACGT
|
||||
>seq2
|
||||
AAAACCCCGGGGTTTTAAAACCCCGGGGTTTT
|
||||
>seq3
|
||||
ATCGATCGATCGATCGATCGATCGATCGATCG
|
||||
Reference in New Issue
Block a user