Implement advanced primer search filtering and reference handling

This update introduces new configuration options for pair selection, including multi-match filtering, circularity settings, and reference site validation. It also enhances the search functionality to correctly identify and track repeated sequences and reference locations, improving the accuracy of primer pair identification.

OBIPRIMERS_NOTES.md

@@ -471,5 +471,64 @@ taxonomie après `AssignTaxa`, régler le GC de la commande.
 Avertissements `Taxid ... has to be updated` : viennent du mécanisme existant d'obitools (taxids
 fusionnés dans le FASTA) ; `--update-taxid` les corrige à la lecture.
 
-Reste à faire : mode B, `-U`, `-R`, tests shell `obitests/obitools/obiprimers` (données dans le dépôt),
+Reste à faire : mode B, tests shell `obitests/obitools/obiprimers` (données dans le dépôt),
 documentation, réduction de la mémoire, comparaison systématique `--rank` autre que species.
+(`-U` et `-R` : faits, section 15.)
+
+## 15. Options `-U` (no multi-match) et `-R` (référence) (2026-11-03)
+
+Code : `pkg/obiprimer/multimatch.go` (`DropRepeatedWords`, `DropMultiMatch`), `reference.go`
+(`referenceSites`, `span`), `Corpus.IndexOfExample`, champs `SearchParams.NoMultiMatch/Reference`,
+`PairStats.RefLocation/RefAmplicon`, options CLI `-U/--no-multi-match` et `-R/--reference`, colonnes CSV
+`reference_location` et `reference_amplicon` (présentes seulement avec `-R`, `NA` pour une paire qui
+n'amplifie pas la référence). Tests : `search_test.go` (synthétiques), `search_real_test.go` (opt-in).
+
+### -U
+
+Sémantique lue dans le C (`aproxpattern.c` et `ahocorasick.c`, qui donnent la même règle) : un mot est
+écarté s'il a >= 2 occurrences approchées, **deux brins confondus**, dans **une** séquence quelconque du
+corpus (exemple ou contre-exemple). Le drapeau « multi-mot » du comptage strict (`Repeated`, mot présent
+plusieurs fois dans un exemple) n'en est qu'un cas particulier ; ecoPrimers l'applique d'abord pour
+alléger la recherche, nous aussi (`DropRepeatedWords` puis `DropMultiMatch`).
+
+**Contrôle contre ecoPrimers `-U`** (Cyprinidae, 98,7 s, 1,18 Go) : **1 366 paires, identiques sur tous les
+champs** (primers, good, séquences, taxons, Bc, Bs, Yule, longueurs, GC, Tm). Notre journal :
+34 mots stricts écartés comme répétés, 469 écartés pour multi-match, 703 amorces conservées.
+`testdata/ecoprimers_Cyprinidae_U.pairs.tsv` garde les valeurs de référence (test opt-in
+`TestNoMultiMatchAgainstEcoPrimers`).
+
+Détail sans conséquence sur les résultats : avec `-e 0`, ecoPrimers (chemin Aho-Corasick) ne filtre pas les
+amorces par le quorum de sensibilité et définit « good » par les seuls contre-exemples ; nous appliquons le
+même filtre qu'avec `-e > 0`. Une amorce sous le quorum ne peut pas figurer dans une paire retenue
+(une paire amplifie au plus autant d'exemples que ses amorces en touchent), donc les tables finales
+sont les mêmes ; seuls les décomptes du journal peuvent différer. Non vérifié par une exécution `-e 0`.
+
+### -R
+
+Contrairement à `-U`, **le `-R` d'ecoPrimers est faux** : `refsequence` reçoit le numéro de la séquence
+(`pairs.c`) puis sert d'indice dans la liste des amplicons de la paire (`ecoprimer.c`, `printapair`).
+Mesuré sur Cyprinidae : avec la référence `NC_022183` ecoPrimers coïncide avec nous sur les coordonnées
+des 3 448 paires (coïncidence : l'indice tombe juste) ; avec `NC_083213`, **seuls 382 de ses 3 405
+emplacements sont de vrais sites d'amorces dans la séquence, contre 3 405 sur 3 405 pour obiprimers**
+(vérifié par un script indépendant et par `TestReferenceSitesAreRealSites`). Nous implémentons donc
+l'intention, pas le C :
+
+- `reference_location` : emplacement GenBank 1-based des deux sites, de gauche à droite,
+  `join(a..b,c..d)`, entouré de `complement()` quand `primer1` (tel qu'écrit dans le CSV) se fixe sur le brin
+  moins. Fenêtre à cheval sur la jonction circulaire : coupée en deux (`a..n,1..m`).
+- `reference_amplicon` : région entre les sites, amorces exclues, en minuscules, lue depuis `primer1`.
+- **Orientation** : ecoPrimers oriente l'amplicon par rapport à l'ordre canonique des mots, nous par rapport
+  à `primer1` du CSV (plus lisible). Sur NC_022183 : coordonnées identiques pour les 3 448 paires,
+  amplicon identique pour 2 572 et complémenté-inversé pour 836, tout le reste égal.
+- Plusieurs amplicons de la paire sur la référence : on garde le plus à gauche (le C garde le dernier,
+  par écrasement, mais ses valeurs sont de toute façon indéfinies).
+- Référence introuvable ou contre-exemple : **erreur** (le C ignore silencieusement).
+- Vérifications : 3 408 paires sur NC_022183 et 3 405 sur NC_083213, sites à <= 3 erreurs, amplicon égal à
+  la séquence (ou son complément inverse), orientation cohérente (2 184 paires `primer1` sur le brin +,
+  1 224 sur le brin -).
+
+### Autres constats
+
+- `obiutils` : `TestSetString` est **instable** (échoue ~4 fois sur 6) : `Set.String()` parcourt une map Go
+  et le test attend `[1 2 3]`. Sans rapport avec obiprimers, non corrigé.
+- Un défaut de mon test précédent (`TestCLISearchParams` laissait `_salt = -1`) a été corrigé.

cmd/obitools/obiprimers/main.go

@@ -35,7 +35,7 @@ func main() {
 		log.Fatalf("%v", err)
 	}
 
-	obicsv.CLICSVWriter(obiprimers.PairsToCSVIterator(pairs), true)
+	obicsv.CLICSVWriter(obiprimers.PairsToCSVIterator(pairs, obiprimers.CLIReference() != ""), true)
 
 	obiutils.WaitForLastPipe()
 }

pkg/obiprimer/corpus.go

@@ -106,3 +106,18 @@ func LoadCorpus(seqs obiiter.IBioSequence, sel Selection) (*Corpus, error) {
 	}
 	return c, nil
 }
+
+// IndexOfExample returns the index of the example sequence with the given
+// identifier. It is an error if there is none, or if the sequence is a
+// counterexample.
+func (c *Corpus) IndexOfExample(id string) (int, error) {
+	for i, s := range c.Seqs {
+		if s.Id() == id {
+			if c.Roles[i] != Example {
+				return -1, fmt.Errorf("obiprimer: reference sequence %q is a counterexample, it must be an example", id)
+			}
+			return i, nil
+		}
+	}
+	return -1, fmt.Errorf("obiprimer: reference sequence %q is not among the selected example sequences", id)
+}

pkg/obiprimer/multimatch.go

@@ -0,0 +1,41 @@
+package obiprimer
+
+import "git.metabarcoding.org/obitools/obitools4/obitools4/pkg/obikmer"
+
+// DropRepeatedWords removes the strict words found more than once (on either
+// strand) in at least one example sequence. It is the first half of the
+// ecoPrimers -U option (no multi-match).
+func DropRepeatedWords(words []obikmer.SeqKmerCount) (kept []obikmer.SeqKmerCount, dropped int) {
+	kept = make([]obikmer.SeqKmerCount, 0, len(words))
+	for _, w := range words {
+		if w.Repeated {
+			dropped++
+			continue
+		}
+		kept = append(kept, w)
+	}
+	return kept, dropped
+}
+
+// DropMultiMatch removes the words having two or more approximate occurrences,
+// both strands counted, in at least one sequence of the corpus, whatever its
+// role. It is the second half of the ecoPrimers -U option: the primers that
+// can anneal at several places of a sequence are discarded.
+func DropMultiMatch(wm []WordMatches) (kept []WordMatches, dropped int) {
+	kept = make([]WordMatches, 0, len(wm))
+	for _, w := range wm {
+		multi := false
+		for i := 1; i < len(w.Matches); i++ {
+			if w.Matches[i].Seq == w.Matches[i-1].Seq {
+				multi = true
+				break
+			}
+		}
+		if multi {
+			dropped++
+			continue
+		}
+		kept = append(kept, w)
+	}
+	return kept, dropped
+}

pkg/obiprimer/pairs.go

@@ -22,6 +22,8 @@ type PairParams struct {
 	FalsePositive  float64 // -x: and less than this fraction of the counterexamples (0.1)
 	MinSpecificity float64 // -T: pairs with a lower Bs are dropped (0.6)
 	Circular       bool    // -c
+	Reference      int     // -R: corpus index of the reference sequence (used when HasReference)
+	HasReference   bool
 }
 
 func (p PairParams) validate() error {

pkg/obiprimer/pairs_eval.go

@@ -31,6 +31,14 @@ type PairStats struct {
 	Tm1, Tm2       float64 // melting temperature of the primers against their perfect target
 	MinTm1, MinTm2 float64 // lowest melting temperature over the example amplicons
 	GC1, GC2       int
+
+	// With a reference sequence (-R): where the pair amplifies it. RefLocation
+	// is the GenBank location of the two primer sites (1-based, left to right),
+	// wrapped in complement() when Primer1 binds the other strand; RefAmplicon
+	// is the amplified region, primers excluded, read from Primer1 to Primer2.
+	// Both are empty when the pair does not amplify the reference.
+	RefLocation string
+	RefAmplicon string
 }
 
 // Score is the ranking criterion of ecoPrimers.
@@ -52,6 +60,12 @@ type ampRecord struct {
 	tm1, tm2 float32 // NaN when unavailable
 }
 
+// refHit is an amplicon of a retained pair on the reference sequence.
+type refHit struct {
+	up, down seqMatch
+	swapped  bool
+}
+
 // ampHash hashes the amplicon between the two primers read from the side of
 // the first primer of the pair (reverse complemented when the upstream match is
 // the second primer), so that identical amplicons give the same value
@@ -132,7 +146,7 @@ func FindPairs(c *Corpus, taxa *Taxa, wm []WordMatches, p PairParams, model *obi
 	}
 
 	// Phase 2: the amplicons of the retained pairs only.
-	records := ps.collectAmplicons(c, p, index, model)
+	records, refs := ps.collectAmplicons(c, p, index, model)
 	byPair := make([][]ampRecord, len(kept))
 	for _, r := range records {
 		byPair[r.pair] = append(byPair[r.pair], r)
@@ -142,6 +156,12 @@ func FindPairs(c *Corpus, taxa *Taxa, wm []WordMatches, p PairParams, model *obi
 	for i, k := range kept {
 		st, ok := ps.evaluate(c, taxa, p, k, acc[k], byPair[i], model)
 		if ok {
+			if h, found := refs[int32(i)]; found {
+				// evaluate orders the primers by string; the amplicon is read from the first one
+				s1 := orientedWord(ps.primers[k.p1].Word, p.PrimerLength, k.a1)
+				s2 := orientedWord(ps.primers[k.p2].Word, p.PrimerLength, k.a2)
+				st.RefLocation, st.RefAmplicon = referenceSites(c.Seqs[p.Reference].Sequence(), h, p, s2 < s1)
+			}
 			out = append(out, st)
 		}
 	}
@@ -162,7 +182,7 @@ func boolCmp(a, b bool) int {
 }
 
 // collectAmplicons gathers, in parallel, one record per amplicon of a retained pair.
-func (ps *primerSet) collectAmplicons(c *Corpus, p PairParams, index map[pairKey]int32, model *obithermo.Model) []ampRecord {
+func (ps *primerSet) collectAmplicons(c *Corpus, p PairParams, index map[pairKey]int32, model *obithermo.Model) ([]ampRecord, map[int32]refHit) {
 	lp := p.PrimerLength
 	workers := obidefault.ParallelWorkers()
 	parts := make([][]ampRecord, workers)
@@ -175,6 +195,10 @@ func (ps *primerSet) collectAmplicons(c *Corpus, p PairParams, index map[pairKey
 	}
 	nan := float32(math.NaN())
 
+	// The reference sequence is handled by a single worker, which is the only
+	// writer of refs; the first (leftmost) amplicon of each pair is kept.
+	refs := map[int32]refHit{}
+
 	jobs := make(chan int)
 	var wg sync.WaitGroup
 	for w := range workers {
@@ -191,6 +215,11 @@ func (ps *primerSet) collectAmplicons(c *Corpus, p PairParams, index map[pairKey
 					if !ok {
 						return
 					}
+					if p.HasReference && s == p.Reference {
+						if old, seen := refs[pi]; !seen || o.up.pos < old.up.pos {
+							refs[pi] = refHit{o.up, o.down, o.swapped}
+						}
+					}
 					rec := ampRecord{pair: pi, seq: int32(s), tm1: nan, tm2: nan}
 					if isExample {
 						rec.hash = ampHash(raw, int(o.up.pos)+lp, int(o.down.pos), o.swapped)
@@ -233,7 +262,7 @@ func (ps *primerSet) collectAmplicons(c *Corpus, p PairParams, index map[pairKey
 	for _, part := range parts {
 		all = append(all, part...)
 	}
-	return all
+	return all, refs
 }
 
 func b2i(b bool) int {

pkg/obiprimer/reference.go

@@ -0,0 +1,44 @@
+package obiprimer
+
+import (
+	"fmt"
+	"strings"
+
+	"git.metabarcoding.org/obitools/obitools4/obitools4/pkg/obiseq"
+)
+
+// span writes the 1-based GenBank range of the lp bases starting at the 0-based
+// position pos. On a circular sequence a range running over the junction is
+// split in two.
+func span(pos, lp, n int, circular bool) string {
+	end := pos + lp
+	if end > n && circular {
+		return fmt.Sprintf("%d..%d,1..%d", pos+1, n, end-n)
+	}
+	return fmt.Sprintf("%d..%d", pos+1, end)
+}
+
+// referenceSites describes an amplicon of a pair on the reference sequence raw.
+//
+// The location lists the two primer sites left to right as join(a..b,c..d),
+// inside complement() when the first primer of the pair (Primer1 once the pair
+// is in string order) binds the reverse strand. The amplicon is the region
+// between the sites, read 5'->3' from the first primer, in lower case.
+//
+// swappedByOrder says whether putting the pair in string order exchanged the
+// two primers of the key.
+func referenceSites(raw []byte, h refHit, p PairParams, swappedByOrder bool) (location, amplicon string) {
+	lp, n := p.PrimerLength, len(raw)
+
+	// The key slot 1 is the upstream primer unless the match order was swapped;
+	// the exchange due to the string order flips it again.
+	firstIsUpstream := h.swapped == swappedByOrder
+
+	location = "join(" + span(int(h.up.pos), lp, n, p.Circular) + "," + span(int(h.down.pos), lp, n, p.Circular) + ")"
+	region := []byte(strings.ToLower(string(raw[int(h.up.pos)+lp : int(h.down.pos)])))
+	if firstIsUpstream {
+		return location, string(region)
+	}
+	rc := obiseq.NewBioSequence("", region, "").ReverseComplement(true)
+	return "complement(" + location + ")", string(rc.Sequence())
+}

pkg/obiprimer/search.go

@@ -19,6 +19,8 @@ type SearchParams struct {
 	FalsePositive  float64 // -x: fraction of the counterexamples it may match (0.1)
 	MinSpecificity float64 // -T: pairs with a lower Bs are dropped (0.6)
 	Circular       bool    // -c: sequences are circular
+	NoMultiMatch   bool    // -U: discard the primers matching several times in a sequence
+	Reference      string  // -R: identifier of an example sequence on which the amplicons are located ("" = none)
 
 	Partitions int // partitions of the k-mer counter (default 256)
 	TmpDir     string
@@ -41,6 +43,8 @@ type SearchInfo struct {
 	StrictWords int // words found without error in at least StrictQuorum of the examples
 	MinStrict   int // that quorum, in sequences
 	Matches     int // approximate occurrences of those words
+	Repeated    int // strict words dropped because repeated in an example (-U)
+	MultiMatch  int // words dropped because matching several times in a sequence (-U)
 	Pair        PairInfo
 }
 
@@ -63,6 +67,16 @@ func Search(c *Corpus, taxa *Taxa, p SearchParams) ([]PairStats, *SearchInfo, er
 		return nil, nil, err
 	}
 	info := &SearchInfo{Counts: sw.Counts, StrictWords: len(sw.Words), MinStrict: sw.MinSequences}
+	if p.NoMultiMatch {
+		sw.Words, info.Repeated = DropRepeatedWords(sw.Words)
+	}
+
+	refIndex := -1
+	if p.Reference != "" {
+		if refIndex, err = c.IndexOfExample(p.Reference); err != nil {
+			return nil, info, err
+		}
+	}
 
 	words := make([]uint64, len(sw.Words))
 	for i, w := range sw.Words {
@@ -72,6 +86,9 @@ func Search(c *Corpus, taxa *Taxa, p SearchParams) ([]PairStats, *SearchInfo, er
 	if err != nil {
 		return nil, info, err
 	}
+	if p.NoMultiMatch {
+		wm, info.MultiMatch = DropMultiMatch(wm)
+	}
 	for _, w := range wm {
 		info.Matches += len(w.Matches)
 	}
@@ -85,6 +102,8 @@ func Search(c *Corpus, taxa *Taxa, p SearchParams) ([]PairStats, *SearchInfo, er
 		FalsePositive:  p.FalsePositive,
 		MinSpecificity: p.MinSpecificity,
 		Circular:       p.Circular,
+		Reference:      refIndex,
+		HasReference:   refIndex >= 0,
 	}, p.Model)
 	info.Pair = pinfo
 	return pairs, info, err

pkg/obiprimer/search_real_test.go

@@ -0,0 +1,175 @@
+package obiprimer
+
+import (
+	"bufio"
+	"fmt"
+	"os"
+	"path/filepath"
+	"regexp"
+	"strings"
+	"testing"
+
+	"git.metabarcoding.org/obitools/obitools4/obitools4/pkg/obiformats"
+	"git.metabarcoding.org/obitools/obitools4/obitools4/pkg/obitax"
+)
+
+// loadTutorial reads the Cyprinidae examples of the tutorial data (the opt-in
+// tests need OBIPRIMERS_TUTORIAL, see TestStrictWordsAgainstEcoPrimers).
+func loadTutorial(t *testing.T) (*Corpus, *Taxa, *obitax.Taxonomy) {
+	t.Helper()
+	dir := os.Getenv("OBIPRIMERS_TUTORIAL")
+	if dir == "" {
+		t.Skip("OBIPRIMERS_TUTORIAL not set")
+	}
+	taxonomy, err := obiformats.LoadTaxonomy(filepath.Join(dir, "ncbitaxo_20250211.tgz"), false, false)
+	if err != nil {
+		t.Fatal(err)
+	}
+	restrict, err := BelongingTo(taxonomy, []string{"7953"})
+	if err != nil {
+		t.Fatal(err)
+	}
+	seqs, err := obiformats.ReadSequencesFromFile(filepath.Join(dir, "mito.vert.fasta.gz"))
+	if err != nil {
+		t.Fatal(err)
+	}
+	corpus, err := LoadCorpus(seqs, Selection{Restrict: restrict})
+	if err != nil {
+		t.Fatal(err)
+	}
+	taxa, err := corpus.AssignTaxa(taxonomy, "species")
+	if err != nil {
+		t.Fatal(err)
+	}
+	return corpus, taxa, taxonomy
+}
+
+func tutorialParams(t *testing.T) SearchParams {
+	p := DefaultSearchParams()
+	p.MinLength, p.MaxLength, p.ThreePrime, p.Circular = 30, 150, 2, true
+	p.TmpDir = t.TempDir()
+	return p
+}
+
+// ecoPrimers -U on the Cyprinidae tutorial case (-e 3 -3 2 -l 30 -L 150 -c -r 7953 -U)
+// keeps 1366 pairs. testdata/ecoprimers_Cyprinidae_U.pairs.tsv holds, for each,
+// the primers, the amplified example and counterexample sequences, the amplified
+// example and counterexample taxa, and the identified taxa.
+func TestNoMultiMatchAgainstEcoPrimers(t *testing.T) {
+	corpus, taxa, _ := loadTutorial(t)
+	p := tutorialParams(t)
+	p.NoMultiMatch = true
+	pairs, info, err := Search(corpus, taxa, p)
+	if err != nil {
+		t.Fatal(err)
+	}
+	t.Logf("%+v", *info)
+
+	f, err := os.Open(filepath.Join("testdata", "ecoprimers_Cyprinidae_U.pairs.tsv"))
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer f.Close()
+	want := map[string][5]int{}
+	sc := bufio.NewScanner(f)
+	for sc.Scan() {
+		var p1, p2 string
+		var v [5]int
+		if _, err := fmt.Sscanf(sc.Text(), "%s\t%s\t%d\t%d\t%d\t%d\t%d", &p1, &p2, &v[0], &v[1], &v[2], &v[3], &v[4]); err != nil {
+			t.Fatal(err)
+		}
+		want[p1+"/"+p2] = v
+	}
+
+	if len(pairs) != len(want) {
+		t.Errorf("%d pairs, ecoPrimers -U reports %d", len(pairs), len(want))
+	}
+	bad := 0
+	for _, g := range pairs {
+		v, ok := want[g.Primer1+"/"+g.Primer2]
+		if !ok {
+			t.Errorf("pair %s/%s is not in the ecoPrimers -U result", g.Primer1, g.Primer2)
+			continue
+		}
+		got := [5]int{g.Examples, g.CounterExamples, g.ExampleTaxa, g.CounterTaxa, g.Identified}
+		if got != v {
+			if bad++; bad <= 3 {
+				t.Errorf("pair %s/%s: got %v, ecoPrimers %v", g.Primer1, g.Primer2, got, v)
+			}
+		}
+	}
+}
+
+var locRE = regexp.MustCompile(`(\d+)\.\.(\d+)`)
+
+// ecoPrimers -R is only right for the references that happen to come first: it
+// indexes the amplicons of a pair by sequence number. On NC_083213 only 382 of
+// its 3405 locations are real primer sites, against all 3405 for obiprimers.
+// Here the locations are checked against the sequence itself, so no
+// ecoPrimers output is needed: each location must give two real primer sites
+// (at most 3 substitutions) and the amplicon must be the region between them,
+// read from the first primer.
+func TestReferenceSitesAreRealSites(t *testing.T) {
+	corpus, taxa, _ := loadTutorial(t)
+
+	for _, ref := range []string{"NC_022183", "NC_083213"} {
+		t.Run(ref, func(t *testing.T) {
+			p := tutorialParams(t)
+			p.Reference = ref
+			pairs, _, err := Search(corpus, taxa, p)
+			if err != nil {
+				t.Fatal(err)
+			}
+			idx, err := corpus.IndexOfExample(ref)
+			if err != nil {
+				t.Fatal(err)
+			}
+			seq := strings.ToLower(string(corpus.Seqs[idx].Sequence()))
+
+			hamming := func(a, b string) int {
+				d := 0
+				for i := range a {
+					if a[i] != b[i] {
+						d++
+					}
+				}
+				return d
+			}
+			located, bad := 0, 0
+			for _, pr := range pairs {
+				if pr.RefLocation == "" {
+					continue
+				}
+				located++
+				m := locRE.FindAllStringSubmatch(pr.RefLocation, -1)
+				if len(m) != 2 {
+					t.Fatalf("location %q", pr.RefLocation)
+				}
+				var a1, b1, a2, b2 int
+				fmt.Sscanf(m[0][0], "%d..%d", &a1, &b1)
+				fmt.Sscanf(m[1][0], "%d..%d", &a2, &b2)
+				left, right := seq[a1-1:b1], seq[a2-1:b2]
+				between := seq[b1 : a2-1]
+
+				p1, p2 := strings.ToLower(pr.Primer1), strings.ToLower(pr.Primer2)
+				var ok bool
+				if strings.HasPrefix(pr.RefLocation, "complement(") {
+					ok = hamming(strings.ToLower(revcomp(strings.ToUpper(right))), p1) <= 3 && hamming(left, p2) <= 3 &&
+						pr.RefAmplicon == strings.ToLower(revcomp(strings.ToUpper(between)))
+				} else {
+					ok = hamming(left, p1) <= 3 && hamming(strings.ToLower(revcomp(strings.ToUpper(right))), p2) <= 3 &&
+						pr.RefAmplicon == between
+				}
+				if !ok {
+					if bad++; bad <= 3 {
+						t.Errorf("pair %s/%s: %s is not a real location", pr.Primer1, pr.Primer2, pr.RefLocation)
+					}
+				}
+			}
+			t.Logf("%d of %d pairs amplify %s; %d wrong locations", located, len(pairs), ref, bad)
+			if located == 0 {
+				t.Error("no pair amplifies the reference")
+			}
+		})
+	}
+}

pkg/obiprimer/search_test.go

@@ -0,0 +1,199 @@
+package obiprimer
+
+import (
+	"fmt"
+	"math/rand"
+	"strings"
+	"testing"
+
+	"git.metabarcoding.org/obitools/obitools4/obitools4/pkg/obikmer"
+	"git.metabarcoding.org/obitools/obitools4/obitools4/pkg/obiseq"
+)
+
+// synthetic builds nseq example sequences sharing the block F + insert + rc(R),
+// each with its own flanks and its own substitution inside the insert (so that
+// every sequence has a distinct amplicon, hence Bs = 1, with one taxon each).
+// extraF, if not empty, is appended to the flank of sequence extraIn: that
+// sequence then holds the forward site twice.
+type synthetic struct {
+	corpus  *Corpus
+	taxa    *Taxa
+	f, r, i string // forward primer, reverse primer (5'->3'), insert of sequence 0
+	pos     int    // 0-based start of the block in every sequence
+}
+
+func newSynthetic(nseq int, repeatForwardIn int, seed int64) synthetic {
+	rng := rand.New(rand.NewSource(seed))
+	f := randDNA(rng, 18)
+	r := randDNA(rng, 18)
+	ins := randDNA(rng, 40)
+	const pos = 60
+
+	c := &Corpus{}
+	taxa := &Taxa{Rank: "species", ExampleTaxa: nseq}
+	for k := 0; k < nseq; k++ {
+		in := []byte(ins)
+		in[3+k] = "ACGT"[(strings.IndexByte("ACGT", in[3+k])+1)%4] // one private substitution
+		s := randDNA(rng, pos) + f + string(in) + revcomp(r) + randDNA(rng, 60)
+		if k == repeatForwardIn {
+			s += randDNA(rng, 20) + f // the forward site a second time
+		}
+		c.Seqs = append(c.Seqs, obiseq.NewBioSequence(fmt.Sprintf("s%d", k), []byte(strings.ToLower(s)), ""))
+		c.Roles = append(c.Roles, Example)
+		taxa.Of = append(taxa.Of, int32(k))
+	}
+	return synthetic{c, taxa, f, r, ins, pos}
+}
+
+func (s synthetic) params() SearchParams {
+	p := DefaultSearchParams()
+	p.MaxErrors = 1
+	p.MinLength, p.MaxLength = 20, 100
+	p.Partitions = 8
+	p.TmpDir = ""
+	return p
+}
+
+// find returns the pair made of the primers f and r, whatever their order.
+func find(pairs []PairStats, f, r string) *PairStats {
+	for i := range pairs {
+		if (pairs[i].Primer1 == f && pairs[i].Primer2 == r) || (pairs[i].Primer1 == r && pairs[i].Primer2 == f) {
+			return &pairs[i]
+		}
+	}
+	return nil
+}
+
+func TestSearchFindsThePlantedPair(t *testing.T) {
+	s := newSynthetic(8, -1, 11)
+	pairs, info, err := Search(s.corpus, s.taxa, s.params())
+	if err != nil {
+		t.Fatal(err)
+	}
+	p := find(pairs, s.f, s.r)
+	if p == nil {
+		t.Fatalf("the planted pair was not found among %d pairs (%+v)", len(pairs), info)
+	}
+	if p.Examples != 8 || p.MinLen != 40 || p.MaxLen != 40 || p.Bs != 1 || p.Bc != 1 {
+		t.Errorf("unexpected indices %+v", *p)
+	}
+	if p.RefLocation != "" {
+		t.Error("no reference was asked: no location expected")
+	}
+}
+
+func TestReferenceLocation(t *testing.T) {
+	forwardFirst, reverseFirst := 0, 0
+	for seed := int64(1); seed <= 24; seed++ {
+		s := newSynthetic(8, -1, seed)
+		p := s.params()
+		p.Reference = "s3"
+		pairs, _, err := Search(s.corpus, s.taxa, p)
+		if err != nil {
+			t.Fatal(err)
+		}
+		pair := find(pairs, s.f, s.r)
+		if pair == nil {
+			t.Fatalf("seed %d: planted pair not found", seed)
+		}
+
+		// 1-based sites: forward primer at 61..78, then 40 bases of insert, then the reverse site
+		wantLoc := "join(61..78,119..136)"
+		insert := strings.ToLower(string(s.corpus.Seqs[3].Sequence()[s.pos+18 : s.pos+58]))
+		if pair.Primer1 == s.f { // the amplicon is read from the first primer
+			forwardFirst++
+			if pair.RefLocation != wantLoc || pair.RefAmplicon != insert {
+				t.Errorf("seed %d, forward first: got %s %s", seed, pair.RefLocation, pair.RefAmplicon)
+			}
+		} else {
+			reverseFirst++
+			rc := strings.ToLower(revcomp(strings.ToUpper(insert)))
+			if pair.RefLocation != "complement("+wantLoc+")" || pair.RefAmplicon != rc {
+				t.Errorf("seed %d, reverse first: got %s %s", seed, pair.RefLocation, pair.RefAmplicon)
+			}
+		}
+		if len(pair.RefAmplicon) != pair.MinLen {
+			t.Errorf("seed %d: amplicon length %d, pair length %d", seed, len(pair.RefAmplicon), pair.MinLen)
+		}
+	}
+	if forwardFirst == 0 || reverseFirst == 0 {
+		t.Errorf("both orientations must be exercised: %d forward first, %d reverse first", forwardFirst, reverseFirst)
+	}
+}
+
+func TestReferenceErrors(t *testing.T) {
+	s := newSynthetic(8, -1, 11)
+	p := s.params()
+	p.Reference = "nope"
+	if _, _, err := Search(s.corpus, s.taxa, p); err == nil {
+		t.Error("an unknown reference must be refused")
+	}
+	s.corpus.Roles[2] = CounterExample
+	p.Reference = "s2"
+	if _, _, err := Search(s.corpus, s.taxa, p); err == nil {
+		t.Error("a counterexample cannot be the reference")
+	}
+}
+
+func TestReferenceOnACircularJunction(t *testing.T) {
+	// span writes a range running over the end of a circular sequence in two parts
+	if got := span(95, 10, 100, true); got != "96..100,1..5" {
+		t.Errorf("circular span %q", got)
+	}
+	if got := span(95, 10, 100, false); got != "96..105" {
+		t.Errorf("linear span %q", got)
+	}
+	if got := span(10, 18, 100, true); got != "11..28" {
+		t.Errorf("inner span %q", got)
+	}
+}
+
+func TestNoMultiMatchDropsRepeatedPrimers(t *testing.T) {
+	// the forward primer is found twice in sequence 2
+	s := newSynthetic(8, 2, 11)
+
+	with, infoWith, err := Search(s.corpus, s.taxa, s.params())
+	if err != nil {
+		t.Fatal(err)
+	}
+	if find(with, s.f, s.r) == nil {
+		t.Fatalf("without -U the pair must be found (%+v)", infoWith)
+	}
+
+	p := s.params()
+	p.NoMultiMatch = true
+	without, info, err := Search(s.corpus, s.taxa, p)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if find(without, s.f, s.r) != nil {
+		t.Error("with -U the repeated forward primer must be discarded, and its pair with it")
+	}
+	if info.Repeated == 0 {
+		t.Errorf("the repeated word was not counted: %+v", info)
+	}
+	if len(without) >= len(with) {
+		t.Errorf("-U must not add pairs: %d with, %d without", len(with), len(without))
+	}
+}
+
+func TestDropMultiMatch(t *testing.T) {
+	wm := []WordMatches{
+		{Word: 1, Matches: []Match{{0, 5, 0, false}, {1, 9, 0, true}}},   // one per sequence: kept
+		{Word: 2, Matches: []Match{{0, 5, 0, false}, {0, 50, 1, false}}}, // two in sequence 0
+		{Word: 3, Matches: []Match{{2, 5, 0, false}, {2, 5, 0, true}}},   // both strands of one sequence
+		{Word: 4}, // no match at all: kept
+	}
+	kept, dropped := DropMultiMatch(wm)
+	if dropped != 2 || len(kept) != 2 || kept[0].Word != 1 || kept[1].Word != 4 {
+		t.Errorf("kept %v, dropped %d", kept, dropped)
+	}
+}
+
+func TestDropRepeatedWords(t *testing.T) {
+	words := []obikmer.SeqKmerCount{{Kmer: 1}, {Kmer: 2, Repeated: true}, {Kmer: 3}}
+	kept, dropped := DropRepeatedWords(words)
+	if dropped != 1 || len(kept) != 2 || kept[0].Kmer != 1 || kept[1].Kmer != 3 {
+		t.Errorf("kept %v, dropped %d", kept, dropped)
+	}
+}

pkg/obitools/obiprimers/obiprimers.go

@@ -53,6 +53,10 @@ func CLIPrimerPairs(sequences obiiter.IBioSequence) ([]obiprimer.PairStats, erro
 	pairs, info, err := obiprimer.Search(corpus, taxa, params)
 	if info != nil {
 		log.Infof("%d strict words (found without error in at least %d examples)", info.StrictWords, info.MinStrict)
+		if params.NoMultiMatch {
+			log.Infof("no multi-match: %d strict words dropped as repeated, %d words dropped for matching several times in a sequence",
+				info.Repeated, info.MultiMatch)
+		}
 		log.Infof("%d approximate occurrences; %d conserved primers, %d good",
 			info.Matches, info.Pair.Primers, info.Pair.GoodPrimers)
 		log.Infof("%d candidate pairs, %d amplifying enough examples, %d retained",

pkg/obitools/obiprimers/obiprimers_test.go

@@ -20,7 +20,7 @@ func samplePairs() []obiprimer.PairStats {
 }
 
 func TestPairsToCSVIterator(t *testing.T) {
-	it := PairsToCSVIterator(samplePairs())
+	it := PairsToCSVIterator(samplePairs(), false)
 
 	header := it.Header()
 	if len(header) != len(csvColumns) || header[0] != "index" || header[1] != "primer1" {
@@ -60,7 +60,7 @@ func TestPairsToCSVIterator(t *testing.T) {
 }
 
 func TestEmptyResultStillHasAHeader(t *testing.T) {
-	it := PairsToCSVIterator(nil)
+	it := PairsToCSVIterator(nil, false)
 	if len(it.Header()) != len(csvColumns) {
 		t.Fatalf("header %v", it.Header())
 	}
@@ -88,6 +88,9 @@ func TestEveryColumnIsProduced(t *testing.T) {
 }
 
 func TestCLISearchParams(t *testing.T) {
+	savedMethod, savedSalt := _saltMethod, _salt
+	defer func() { _saltMethod, _salt = savedMethod, savedSalt }()
+
 	p, err := CLISearchParams()
 	if err != nil {
 		t.Fatal(err)
@@ -113,3 +116,47 @@ func TestCLISearchParams(t *testing.T) {
 		t.Error("a negative salt concentration must be refused")
 	}
 }
+
+func TestReferenceColumns(t *testing.T) {
+	pairs := samplePairs()
+	pairs[0].RefLocation = "join(61..78,119..136)"
+	pairs[0].RefAmplicon = "acgt"
+
+	it := PairsToCSVIterator(pairs, true)
+	header := it.Header()
+	n := len(csvColumns)
+	if len(header) != n+2 || header[n] != "reference_location" || header[n+1] != "reference_amplicon" {
+		t.Fatalf("header %v", header)
+	}
+
+	var records []map[string]interface{}
+	for it.Next() {
+		batch := it.Get()
+		for _, r := range batch.Slice() {
+			records = append(records, r)
+		}
+	}
+	if records[0]["reference_location"] != "join(61..78,119..136)" || records[0]["reference_amplicon"] != "acgt" {
+		t.Errorf("reference columns of the first pair: %v", records[0])
+	}
+	if _, present := records[1]["reference_location"]; present {
+		t.Error("a pair that does not amplify the reference must have no location (NA)")
+	}
+
+	// without a reference the columns are absent
+	if got := len(PairsToCSVIterator(pairs, false).Header()); got != n {
+		t.Errorf("%d columns without reference, want %d", got, n)
+	}
+}
+
+func TestSearchParamsCarryTheNewOptions(t *testing.T) {
+	_noMultiMatch, _reference = true, "NC_000001"
+	defer func() { _noMultiMatch, _reference = false, "" }()
+	p, err := CLISearchParams()
+	if err != nil {
+		t.Fatal(err)
+	}
+	if !p.NoMultiMatch || p.Reference != "NC_000001" || CLIReference() != "NC_000001" {
+		t.Errorf("-U/-R not carried: %+v", p)
+	}
+}

pkg/obitools/obiprimers/options.go

@@ -28,6 +28,8 @@ var _minLength = 0
 var _maxLength = 1000
 var _threePrime = 0
 var _circular = false
+var _noMultiMatch = false
+var _reference = ""
 
 var _strictQuorum = 0.7
 var _sensitivity = 0.9
@@ -72,6 +74,17 @@ func OptionSet(options *getoptions.GetOpt) {
 		options.Alias("c"),
 		options.Description("Considers the sequences as circular (whole organelle genomes)."))
 
+	options.BoolVar(&_noMultiMatch, "no-multi-match", _noMultiMatch,
+		options.Alias("U"),
+		options.Description("Discards the primers that match, with the allowed mismatches, more than once "+
+			"in any sequence (examples and counterexamples, both strands)."))
+
+	options.StringVar(&_reference, "reference", _reference,
+		options.Alias("R"),
+		options.ArgName("SEQUENCE_ID"),
+		options.Description("Identifier of an example sequence taken as reference: for each pair amplifying it, "+
+			"adds the location of the primer sites and the amplified sequence."))
+
 	options.IntVar(&_primerLength, "primer-length", _primerLength,
 		options.ArgName("LENGTH"),
 		options.Description("Size of the primers, from 2 to 31."))
@@ -146,6 +159,9 @@ func CLIIgnoreTaxa() []string { return _ignore }
 // CLIExceptionTaxa returns the taxids of the sub-clades turned into counterexamples.
 func CLIExceptionTaxa() []string { return _except }
 
+// CLIReference returns the identifier of the reference sequence, or "".
+func CLIReference() string { return _reference }
+
 // CLIRank returns the taxonomic rank of the coverage and specificity indices.
 func CLIRank() string { return _rank }
 
@@ -176,6 +192,8 @@ func CLISearchParams() (obiprimer.SearchParams, error) {
 		FalsePositive:  _falsePositive,
 		MinSpecificity: _specificity,
 		Circular:       _circular,
+		NoMultiMatch:   _noMultiMatch,
+		Reference:      _reference,
 		Partitions:     _partitions,
 		Model:          model,
 	}, nil

pkg/obitools/obiprimers/output.go

@@ -24,6 +24,9 @@ var csvColumns = []string{
 	"min_length", "max_length", "mean_length", // 19-21
 }
 
+// referenceColumns are added when a reference sequence is given.
+var referenceColumns = []string{"reference_location", "reference_amplicon"}
+
 // round returns x rounded to the given number of decimals; the CSV writer
 // prints it with its shortest representation.
 func round(x float64, decimals int) float64 {
@@ -43,6 +46,10 @@ func pairRecord(index int, p obiprimer.PairStats) obiitercsv.CSVRecord {
 		"bc": round(float64(p.Bc), 4), "identified_taxa": p.Identified, "bs": round(float64(p.Bs), 4),
 		"min_length": p.MinLen, "max_length": p.MaxLen, "mean_length": round(p.MeanLen, 2),
 	}
+	if p.RefLocation != "" {
+		r["reference_location"] = p.RefLocation
+		r["reference_amplicon"] = p.RefAmplicon
+	}
 	for key, v := range map[string]float64{"tm1": p.Tm1, "tm1_min": p.MinTm1, "tm2": p.Tm2, "tm2_min": p.MinTm2} {
 		if !math.IsNaN(v) {
 			r[key] = round(v, 1)
@@ -51,13 +58,20 @@ func pairRecord(index int, p obiprimer.PairStats) obiitercsv.CSVRecord {
 	return r
 }
 
-// PairsToCSVIterator streams the pairs, in order, as CSV records.
+// PairsToCSVIterator streams the pairs, in order, as CSV records. With
-func PairsToCSVIterator(pairs []obiprimer.PairStats) *obiitercsv.ICSVRecord {
+// withReference the two reference columns are written (NA for the pairs that
+// do not amplify the reference sequence).
+func PairsToCSVIterator(pairs []obiprimer.PairStats, withReference bool) *obiitercsv.ICSVRecord {
 	iterator := obiitercsv.NewICSVRecord()
 	iterator.Add(1)
 	for _, column := range csvColumns {
 		iterator.AppendField(column)
 	}
+	if withReference {
+		for _, column := range referenceColumns {
+			iterator.AppendField(column)
+		}
+	}
 
 	go func() {
 		defer iterator.Done()