obitools/obitools4
2
0
Fork 0
mirror of https://github.com/metabarcoding/obitools4.git synced 2025-06-29 16:20:46 +00:00
Code Issues Packages Projects Releases Wiki Activity

Latest version of the doc in HTML

Browse Source
This commit is contained in:
Eric Coissac
2023-01-31 23:04:28 +01:00
parent fa6853fca1
commit 871f22b83d
15 changed files with 1775 additions and 240 deletions
Download Patch File Download Diff File Expand all files Collapse all files

113
doc/_book/annexes.html
View File

@ -7,7 +7,7 @@
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">
<title>OBITools V4 - 4&nbsp; Annexes</title>
<title>OBITools V4 - 5&nbsp; Annexes</title>
<style>
code{white-space: pre-wrap;}
span.smallcaps{font-variant: small-caps;}
@ -70,7 +70,7 @@ ul.task-list li input[type="checkbox"] {
<header id="quarto-header" class="headroom fixed-top">
<nav class="quarto-secondary-nav" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar" aria-controls="quarto-sidebar" aria-expanded="false" aria-label="Toggle sidebar navigation" onclick="if (window.quartoToggleHeadroom) { window.quartoToggleHeadroom(); }">
<div class="container-fluid d-flex justify-content-between">
<h1 class="quarto-secondary-nav-title"><span class="chapter-number">4</span>&nbsp; <span class="chapter-title">Annexes</span></h1>
<h1 class="quarto-secondary-nav-title"><span class="chapter-number">5</span>&nbsp; <span class="chapter-title">Annexes</span></h1>
<button type="button" class="quarto-btn-toggle btn" aria-label="Show secondary navigation">
<i class="bi bi-chevron-right"></i>
</button>
@ -105,17 +105,22 @@ ul.task-list li input[type="checkbox"] {
</li>
<li class="sidebar-item">
<div class="sidebar-item-container">
<a href="./commands.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">2</span>&nbsp; <span class="chapter-title">The <em>OBITools V4</em> commands</span></a>
<a href="./tutorial.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">2</span>&nbsp; <span class="chapter-title">OBITools V4 Tutorial</span></a>
</div>
</li>
<li class="sidebar-item">
<div class="sidebar-item-container">
<a href="./library.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">3</span>&nbsp; <span class="chapter-title">The GO <em>OBITools</em> library</span></a>
<a href="./commands.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">3</span>&nbsp; <span class="chapter-title">The <em>OBITools V4</em> commands</span></a>
</div>
</li>
<li class="sidebar-item">
<div class="sidebar-item-container">
<a href="./annexes.html" class="sidebar-item-text sidebar-link active"><span class="chapter-number">4</span>&nbsp; <span class="chapter-title">Annexes</span></a>
<a href="./library.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">4</span>&nbsp; <span class="chapter-title">The GO <em>OBITools</em> library</span></a>
</div>
</li>
<li class="sidebar-item">
<div class="sidebar-item-container">
<a href="./annexes.html" class="sidebar-item-text sidebar-link active"><span class="chapter-number">5</span>&nbsp; <span class="chapter-title">Annexes</span></a>
</div>
</li>
<li class="sidebar-item">
@ -132,7 +137,7 @@ ul.task-list li input[type="checkbox"] {
<h2 id="toc-title">Table of contents</h2>
<ul>
<li><a href="#sequence-attributes" id="toc-sequence-attributes" class="nav-link active" data-scroll-target="#sequence-attributes"><span class="toc-section-number">4.0.1</span> Sequence attributes</a></li>
<li><a href="#sequence-attributes" id="toc-sequence-attributes" class="nav-link active" data-scroll-target="#sequence-attributes"><span class="toc-section-number">5.0.1</span> Sequence attributes</a></li>
</ul>
</nav>
</div>
@ -141,7 +146,7 @@ ul.task-list li input[type="checkbox"] {
<header id="title-block-header" class="quarto-title-block default">
<div class="quarto-title">
<h1 class="title d-none d-lg-block"><span class="chapter-number">4</span>&nbsp; <span class="chapter-title">Annexes</span></h1>
<h1 class="title d-none d-lg-block"><span class="chapter-number">5</span>&nbsp; <span class="chapter-title">Annexes</span></h1>
</div>
@ -156,79 +161,79 @@ ul.task-list li input[type="checkbox"] {
</header>
<section id="sequence-attributes" class="level3" data-number="4.0.1">
<h3 data-number="4.0.1" class="anchored" data-anchor-id="sequence-attributes"><span class="header-section-number">4.0.1</span> Sequence attributes</h3>
<section id="reserved-sequence-attributes" class="level4" data-number="4.0.1.1">
<h4 data-number="4.0.1.1" class="anchored" data-anchor-id="reserved-sequence-attributes"><span class="header-section-number">4.0.1.1</span> Reserved sequence attributes</h4>
<section id="ali_dir" class="level5" data-number="4.0.1.1.1">
<h5 data-number="4.0.1.1.1" class="anchored" data-anchor-id="ali_dir"><span class="header-section-number">4.0.1.1.1</span> <code>ali_dir</code></h5>
<section id="type-string" class="level6" data-number="4.0.1.1.1.1">
<h6 data-number="4.0.1.1.1.1" class="anchored" data-anchor-id="type-string"><span class="header-section-number">4.0.1.1.1.1</span> Type : <code>string</code></h6>
<section id="sequence-attributes" class="level3" data-number="5.0.1">
<h3 data-number="5.0.1" class="anchored" data-anchor-id="sequence-attributes"><span class="header-section-number">5.0.1</span> Sequence attributes</h3>
<section id="reserved-sequence-attributes" class="level4" data-number="5.0.1.1">
<h4 data-number="5.0.1.1" class="anchored" data-anchor-id="reserved-sequence-attributes"><span class="header-section-number">5.0.1.1</span> Reserved sequence attributes</h4>
<section id="ali_dir" class="level5" data-number="5.0.1.1.1">
<h5 data-number="5.0.1.1.1" class="anchored" data-anchor-id="ali_dir"><span class="header-section-number">5.0.1.1.1</span> <code>ali_dir</code></h5>
<section id="type-string" class="level6" data-number="5.0.1.1.1.1">
<h6 data-number="5.0.1.1.1.1" class="anchored" data-anchor-id="type-string"><span class="header-section-number">5.0.1.1.1.1</span> Type : <code>string</code></h6>
<p>The attribute can contain 2 string values <code>"left"</code> or <code>"right".</code></p>
</section>
<section id="set-by-the-obipairing-tool" class="level6" data-number="4.0.1.1.1.2">
<h6 data-number="4.0.1.1.1.2" class="anchored" data-anchor-id="set-by-the-obipairing-tool"><span class="header-section-number">4.0.1.1.1.2</span> Set by the <em>obipairing</em> tool</h6>
<section id="set-by-the-obipairing-tool" class="level6" data-number="5.0.1.1.1.2">
<h6 data-number="5.0.1.1.1.2" class="anchored" data-anchor-id="set-by-the-obipairing-tool"><span class="header-section-number">5.0.1.1.1.2</span> Set by the <em>obipairing</em> tool</h6>
<p>The alignment generated by <em>obipairing</em> is a 3’-end gap free algorithm. Two cases can occur when aligning the forward and reverse reads. If the barcode is long enough, both the reads overlap only on their 3’ ends. In such case, the alignment direction <code>ali_dir</code> is set to <em>left</em>. If the barcode is shorter than the read length, the paired reads overlap by their 5’ ends, and the complete barcode is sequenced by both the reads. In that later case, <code>ali_dir</code> is set to <em>right</em>.</p>
</section>
</section>
<section id="ali_length" class="level5" data-number="4.0.1.1.2">
<h5 data-number="4.0.1.1.2" class="anchored" data-anchor-id="ali_length"><span class="header-section-number">4.0.1.1.2</span> <code>ali_length</code></h5>
<section id="set-by-the-obipairing-tool-1" class="level6" data-number="4.0.1.1.2.1">
<h6 data-number="4.0.1.1.2.1" class="anchored" data-anchor-id="set-by-the-obipairing-tool-1"><span class="header-section-number">4.0.1.1.2.1</span> Set by the <em>obipairing</em> tool</h6>
<section id="ali_length" class="level5" data-number="5.0.1.1.2">
<h5 data-number="5.0.1.1.2" class="anchored" data-anchor-id="ali_length"><span class="header-section-number">5.0.1.1.2</span> <code>ali_length</code></h5>
<section id="set-by-the-obipairing-tool-1" class="level6" data-number="5.0.1.1.2.1">
<h6 data-number="5.0.1.1.2.1" class="anchored" data-anchor-id="set-by-the-obipairing-tool-1"><span class="header-section-number">5.0.1.1.2.1</span> Set by the <em>obipairing</em> tool</h6>
<p>Length of the aligned parts when merging forward and reverse reads</p>
</section>
</section>
<section id="count-the-number-of-sequence-occurrences" class="level5" data-number="4.0.1.1.3">
<h5 data-number="4.0.1.1.3" class="anchored" data-anchor-id="count-the-number-of-sequence-occurrences"><span class="header-section-number">4.0.1.1.3</span> <code>count</code> : the number of sequence occurrences</h5>
<section id="set-by-the-obiuniq-tool" class="level6" data-number="4.0.1.1.3.1">
<h6 data-number="4.0.1.1.3.1" class="anchored" data-anchor-id="set-by-the-obiuniq-tool"><span class="header-section-number">4.0.1.1.3.1</span> Set by the <em>obiuniq</em> tool</h6>
<section id="count-the-number-of-sequence-occurrences" class="level5" data-number="5.0.1.1.3">
<h5 data-number="5.0.1.1.3" class="anchored" data-anchor-id="count-the-number-of-sequence-occurrences"><span class="header-section-number">5.0.1.1.3</span> <code>count</code> : the number of sequence occurrences</h5>
<section id="set-by-the-obiuniq-tool" class="level6" data-number="5.0.1.1.3.1">
<h6 data-number="5.0.1.1.3.1" class="anchored" data-anchor-id="set-by-the-obiuniq-tool"><span class="header-section-number">5.0.1.1.3.1</span> Set by the <em>obiuniq</em> tool</h6>
<p>The <code>count</code> attribute indicates how-many strictly identical sequences have been merged in a single record. It contains an integer value. If it is absent this means that the sequence record represents a single occurrence of the sequence.</p>
</section>
<section id="getter-method-count" class="level6" data-number="4.0.1.1.3.2">
<h6 data-number="4.0.1.1.3.2" class="anchored" data-anchor-id="getter-method-count"><span class="header-section-number">4.0.1.1.3.2</span> Getter : method <code>Count()</code></h6>
<section id="getter-method-count" class="level6" data-number="5.0.1.1.3.2">
<h6 data-number="5.0.1.1.3.2" class="anchored" data-anchor-id="getter-method-count"><span class="header-section-number">5.0.1.1.3.2</span> Getter : method <code>Count()</code></h6>
<p>The <code>Count()</code> method allows to access to the count attribute as an integer value. If the <code>count</code> attribute is not defined for the given sequence, the value <em>1</em> is returned</p>
</section>
</section>
<section id="merged_" class="level5" data-number="4.0.1.1.4">
<h5 data-number="4.0.1.1.4" class="anchored" data-anchor-id="merged_"><span class="header-section-number">4.0.1.1.4</span> <code>merged_*</code></h5>
<section id="type-mapstringint" class="level6" data-number="4.0.1.1.4.1">
<h6 data-number="4.0.1.1.4.1" class="anchored" data-anchor-id="type-mapstringint"><span class="header-section-number">4.0.1.1.4.1</span> Type : <code>map[string]int</code></h6>
<section id="merged_" class="level5" data-number="5.0.1.1.4">
<h5 data-number="5.0.1.1.4" class="anchored" data-anchor-id="merged_"><span class="header-section-number">5.0.1.1.4</span> <code>merged_*</code></h5>
<section id="type-mapstringint" class="level6" data-number="5.0.1.1.4.1">
<h6 data-number="5.0.1.1.4.1" class="anchored" data-anchor-id="type-mapstringint"><span class="header-section-number">5.0.1.1.4.1</span> Type : <code>map[string]int</code></h6>
</section>
<section id="set-by-the-obiuniq-tool-1" class="level6" data-number="4.0.1.1.4.2">
<h6 data-number="4.0.1.1.4.2" class="anchored" data-anchor-id="set-by-the-obiuniq-tool-1"><span class="header-section-number">4.0.1.1.4.2</span> Set by the <em>obiuniq</em> tool</h6>
<section id="set-by-the-obiuniq-tool-1" class="level6" data-number="5.0.1.1.4.2">
<h6 data-number="5.0.1.1.4.2" class="anchored" data-anchor-id="set-by-the-obiuniq-tool-1"><span class="header-section-number">5.0.1.1.4.2</span> Set by the <em>obiuniq</em> tool</h6>
<p>The <code>-m</code> option of the <em>obiuniq</em> tools allows for keeping track of the distribution of the values stored in given attribute of interest. Often this option is used to summarise distribution of a sequence variant accross samples when <em>obiuniq</em> is run after running <em>obimultiplex</em>. The actual name of the attribute depends on the name of the monitored attribute. If <code>-m</code> option is used with the attribute <em>sample</em>, then this attribute names <em>merged_sample</em>.</p>
</section>
</section>
<section id="mode" class="level5" data-number="4.0.1.1.5">
<h5 data-number="4.0.1.1.5" class="anchored" data-anchor-id="mode"><span class="header-section-number">4.0.1.1.5</span> <code>mode</code></h5>
<section id="set-by-the-obipairing-tool-2" class="level6" data-number="4.0.1.1.5.1">
<h6 data-number="4.0.1.1.5.1" class="anchored" data-anchor-id="set-by-the-obipairing-tool-2"><span class="header-section-number">4.0.1.1.5.1</span> Set by the <em>obipairing</em> tool</h6>
<section id="mode" class="level5" data-number="5.0.1.1.5">
<h5 data-number="5.0.1.1.5" class="anchored" data-anchor-id="mode"><span class="header-section-number">5.0.1.1.5</span> <code>mode</code></h5>
<section id="set-by-the-obipairing-tool-2" class="level6" data-number="5.0.1.1.5.1">
<h6 data-number="5.0.1.1.5.1" class="anchored" data-anchor-id="set-by-the-obipairing-tool-2"><span class="header-section-number">5.0.1.1.5.1</span> Set by the <em>obipairing</em> tool</h6>
<p><strong><code>obitag_ref_index</code></strong></p>
</section>
<section id="set-by-the-obirefidx-tool." class="level6" data-number="4.0.1.1.5.2">
<h6 data-number="4.0.1.1.5.2" class="anchored" data-anchor-id="set-by-the-obirefidx-tool."><span class="header-section-number">4.0.1.1.5.2</span> Set by the <em>obirefidx</em> tool.</h6>
<section id="set-by-the-obirefidx-tool." class="level6" data-number="5.0.1.1.5.2">
<h6 data-number="5.0.1.1.5.2" class="anchored" data-anchor-id="set-by-the-obirefidx-tool."><span class="header-section-number">5.0.1.1.5.2</span> Set by the <em>obirefidx</em> tool.</h6>
<p>It resumes to which taxonomic annotation a match to that sequence must lead according to the number of differences existing between the query sequence and the reference sequence having that tag.</p>
</section>
<section id="getter-method-count-1" class="level6" data-number="4.0.1.1.5.3">
<h6 data-number="4.0.1.1.5.3" class="anchored" data-anchor-id="getter-method-count-1"><span class="header-section-number">4.0.1.1.5.3</span> Getter : method <code>Count()</code></h6>
<section id="getter-method-count-1" class="level6" data-number="5.0.1.1.5.3">
<h6 data-number="5.0.1.1.5.3" class="anchored" data-anchor-id="getter-method-count-1"><span class="header-section-number">5.0.1.1.5.3</span> Getter : method <code>Count()</code></h6>
</section>
</section>
<section id="pairing_mismatches" class="level5" data-number="4.0.1.1.6">
<h5 data-number="4.0.1.1.6" class="anchored" data-anchor-id="pairing_mismatches"><span class="header-section-number">4.0.1.1.6</span> <code>pairing_mismatches</code></h5>
<section id="set-by-the-obipairing-tool-3" class="level6" data-number="4.0.1.1.6.1">
<h6 data-number="4.0.1.1.6.1" class="anchored" data-anchor-id="set-by-the-obipairing-tool-3"><span class="header-section-number">4.0.1.1.6.1</span> Set by the <em>obipairing</em> tool</h6>
<section id="pairing_mismatches" class="level5" data-number="5.0.1.1.6">
<h5 data-number="5.0.1.1.6" class="anchored" data-anchor-id="pairing_mismatches"><span class="header-section-number">5.0.1.1.6</span> <code>pairing_mismatches</code></h5>
<section id="set-by-the-obipairing-tool-3" class="level6" data-number="5.0.1.1.6.1">
<h6 data-number="5.0.1.1.6.1" class="anchored" data-anchor-id="set-by-the-obipairing-tool-3"><span class="header-section-number">5.0.1.1.6.1</span> Set by the <em>obipairing</em> tool</h6>
</section>
</section>
<section id="score" class="level5" data-number="4.0.1.1.7">
<h5 data-number="4.0.1.1.7" class="anchored" data-anchor-id="score"><span class="header-section-number">4.0.1.1.7</span> <code>score</code></h5>
<section id="set-by-the-obipairing-tool-4" class="level6" data-number="4.0.1.1.7.1">
<h6 data-number="4.0.1.1.7.1" class="anchored" data-anchor-id="set-by-the-obipairing-tool-4"><span class="header-section-number">4.0.1.1.7.1</span> Set by the <em>obipairing</em> tool</h6>
<section id="score" class="level5" data-number="5.0.1.1.7">
<h5 data-number="5.0.1.1.7" class="anchored" data-anchor-id="score"><span class="header-section-number">5.0.1.1.7</span> <code>score</code></h5>
<section id="set-by-the-obipairing-tool-4" class="level6" data-number="5.0.1.1.7.1">
<h6 data-number="5.0.1.1.7.1" class="anchored" data-anchor-id="set-by-the-obipairing-tool-4"><span class="header-section-number">5.0.1.1.7.1</span> Set by the <em>obipairing</em> tool</h6>
</section>
</section>
<section id="score_norm" class="level5" data-number="4.0.1.1.8">
<h5 data-number="4.0.1.1.8" class="anchored" data-anchor-id="score_norm"><span class="header-section-number">4.0.1.1.8</span> <code>score_norm</code></h5>
<section id="set-by-the-obipairing-tool-5" class="level6" data-number="4.0.1.1.8.1">
<h6 data-number="4.0.1.1.8.1" class="anchored" data-anchor-id="set-by-the-obipairing-tool-5"><span class="header-section-number">4.0.1.1.8.1</span> Set by the <em>obipairing</em> tool</h6>
<section id="score_norm" class="level5" data-number="5.0.1.1.8">
<h5 data-number="5.0.1.1.8" class="anchored" data-anchor-id="score_norm"><span class="header-section-number">5.0.1.1.8</span> <code>score_norm</code></h5>
<section id="set-by-the-obipairing-tool-5" class="level6" data-number="5.0.1.1.8.1">
<h6 data-number="5.0.1.1.8.1" class="anchored" data-anchor-id="set-by-the-obipairing-tool-5"><span class="header-section-number">5.0.1.1.8.1</span> Set by the <em>obipairing</em> tool</h6>
</section>
@ -373,7 +378,7 @@ window.document.addEventListener("DOMContentLoaded", function (event) {
<nav class="page-navigation">
<div class="nav-page nav-page-previous">
<a href="./library.html" class="pagination-link">
<i class="bi bi-arrow-left-short"></i> <span class="nav-page-text"><span class="chapter-number">3</span>&nbsp; <span class="chapter-title">The GO <em>OBITools</em> library</span></span>
<i class="bi bi-arrow-left-short"></i> <span class="nav-page-text"><span class="chapter-number">4</span>&nbsp; <span class="chapter-title">The GO <em>OBITools</em> library</span></span>
</a>
</div>
<div class="nav-page nav-page-next">

205
doc/_book/commands.html
View File

@ -7,7 +7,7 @@
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">
<title>OBITools V4 - 2&nbsp; The OBITools V4 commands</title>
<title>OBITools V4 - 3&nbsp; The OBITools V4 commands</title>
<style>
code{white-space: pre-wrap;}
span.smallcaps{font-variant: small-caps;}
@ -50,7 +50,7 @@ div.csl-indent {
<script src="site_libs/quarto-search/quarto-search.js"></script>
<meta name="quarto:offset" content="./">
<link href="./library.html" rel="next">
<link href="./intro.html" rel="prev">
<link href="./tutorial.html" rel="prev">
<script src="site_libs/quarto-html/quarto.js"></script>
<script src="site_libs/quarto-html/popper.min.js"></script>
<script src="site_libs/quarto-html/tippy.umd.min.js"></script>
@ -90,7 +90,7 @@ div.csl-indent {
<header id="quarto-header" class="headroom fixed-top">
<nav class="quarto-secondary-nav" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar" aria-controls="quarto-sidebar" aria-expanded="false" aria-label="Toggle sidebar navigation" onclick="if (window.quartoToggleHeadroom) { window.quartoToggleHeadroom(); }">
<div class="container-fluid d-flex justify-content-between">
<h1 class="quarto-secondary-nav-title"><span class="chapter-number">2</span>&nbsp; <span class="chapter-title">The <em>OBITools V4</em> commands</span></h1>
<h1 class="quarto-secondary-nav-title"><span class="chapter-number">3</span>&nbsp; <span class="chapter-title">The <em>OBITools V4</em> commands</span></h1>
<button type="button" class="quarto-btn-toggle btn" aria-label="Show secondary navigation">
<i class="bi bi-chevron-right"></i>
</button>
@ -125,17 +125,22 @@ div.csl-indent {
</li>
<li class="sidebar-item">
<div class="sidebar-item-container">
<a href="./commands.html" class="sidebar-item-text sidebar-link active"><span class="chapter-number">2</span>&nbsp; <span class="chapter-title">The <em>OBITools V4</em> commands</span></a>
<a href="./tutorial.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">2</span>&nbsp; <span class="chapter-title">OBITools V4 Tutorial</span></a>
</div>
</li>
<li class="sidebar-item">
<div class="sidebar-item-container">
<a href="./library.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">3</span>&nbsp; <span class="chapter-title">The GO <em>OBITools</em> library</span></a>
<a href="./commands.html" class="sidebar-item-text sidebar-link active"><span class="chapter-number">3</span>&nbsp; <span class="chapter-title">The <em>OBITools V4</em> commands</span></a>
</div>
</li>
<li class="sidebar-item">
<div class="sidebar-item-container">
<a href="./annexes.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">4</span>&nbsp; <span class="chapter-title">Annexes</span></a>
<a href="./library.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">4</span>&nbsp; <span class="chapter-title">The GO <em>OBITools</em> library</span></a>
</div>
</li>
<li class="sidebar-item">
<div class="sidebar-item-container">
<a href="./annexes.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">5</span>&nbsp; <span class="chapter-title">Annexes</span></a>
</div>
</li>
<li class="sidebar-item">
@ -152,29 +157,29 @@ div.csl-indent {
<h2 id="toc-title">Table of contents</h2>
<ul>
<li><a href="#specifying-the-input-files-to-obitools-commands" id="toc-specifying-the-input-files-to-obitools-commands" class="nav-link active" data-scroll-target="#specifying-the-input-files-to-obitools-commands"><span class="toc-section-number">2.1</span> Specifying the input files to <em>OBITools</em> commands</a></li>
<li><a href="#options-common-to-most-of-the-obitools-commands" id="toc-options-common-to-most-of-the-obitools-commands" class="nav-link" data-scroll-target="#options-common-to-most-of-the-obitools-commands"><span class="toc-section-number">2.2</span> Options common to most of the <em>OBITools</em> commands</a>
<li><a href="#specifying-the-input-files-to-obitools-commands" id="toc-specifying-the-input-files-to-obitools-commands" class="nav-link active" data-scroll-target="#specifying-the-input-files-to-obitools-commands"><span class="toc-section-number">3.1</span> Specifying the input files to <em>OBITools</em> commands</a></li>
<li><a href="#options-common-to-most-of-the-obitools-commands" id="toc-options-common-to-most-of-the-obitools-commands" class="nav-link" data-scroll-target="#options-common-to-most-of-the-obitools-commands"><span class="toc-section-number">3.2</span> Options common to most of the <em>OBITools</em> commands</a>
<ul class="collapse">
<li><a href="#specifying-input-format" id="toc-specifying-input-format" class="nav-link" data-scroll-target="#specifying-input-format"><span class="toc-section-number">2.2.1</span> Specifying input format</a></li>
<li><a href="#specifying-output-format" id="toc-specifying-output-format" class="nav-link" data-scroll-target="#specifying-output-format"><span class="toc-section-number">2.2.2</span> Specifying output format</a></li>
<li><a href="#format-of-the-annotations-in-fasta-and-fastq-files" id="toc-format-of-the-annotations-in-fasta-and-fastq-files" class="nav-link" data-scroll-target="#format-of-the-annotations-in-fasta-and-fastq-files"><span class="toc-section-number">2.2.3</span> Format of the annotations in Fasta and Fastq files</a></li>
<li><a href="#specifying-input-format" id="toc-specifying-input-format" class="nav-link" data-scroll-target="#specifying-input-format"><span class="toc-section-number">3.2.1</span> Specifying input format</a></li>
<li><a href="#specifying-output-format" id="toc-specifying-output-format" class="nav-link" data-scroll-target="#specifying-output-format"><span class="toc-section-number">3.2.2</span> Specifying output format</a></li>
<li><a href="#format-of-the-annotations-in-fasta-and-fastq-files" id="toc-format-of-the-annotations-in-fasta-and-fastq-files" class="nav-link" data-scroll-target="#format-of-the-annotations-in-fasta-and-fastq-files"><span class="toc-section-number">3.2.3</span> Format of the annotations in Fasta and Fastq files</a></li>
</ul></li>
<li><a href="#obitools-expression-language" id="toc-obitools-expression-language" class="nav-link" data-scroll-target="#obitools-expression-language"><span class="toc-section-number">2.3</span> OBITools expression language</a>
<li><a href="#obitools-expression-language" id="toc-obitools-expression-language" class="nav-link" data-scroll-target="#obitools-expression-language"><span class="toc-section-number">3.3</span> OBITools expression language</a>
<ul class="collapse">
<li><a href="#variables-usable-in-the-expression" id="toc-variables-usable-in-the-expression" class="nav-link" data-scroll-target="#variables-usable-in-the-expression"><span class="toc-section-number">2.3.1</span> Variables usable in the expression</a></li>
<li><a href="#function-defined-in-the-language" id="toc-function-defined-in-the-language" class="nav-link" data-scroll-target="#function-defined-in-the-language"><span class="toc-section-number">2.3.2</span> Function defined in the language</a></li>
<li><a href="#accessing-to-the-sequence-annotations" id="toc-accessing-to-the-sequence-annotations" class="nav-link" data-scroll-target="#accessing-to-the-sequence-annotations"><span class="toc-section-number">2.3.3</span> Accessing to the sequence annotations</a></li>
<li><a href="#variables-usable-in-the-expression" id="toc-variables-usable-in-the-expression" class="nav-link" data-scroll-target="#variables-usable-in-the-expression"><span class="toc-section-number">3.3.1</span> Variables usable in the expression</a></li>
<li><a href="#function-defined-in-the-language" id="toc-function-defined-in-the-language" class="nav-link" data-scroll-target="#function-defined-in-the-language"><span class="toc-section-number">3.3.2</span> Function defined in the language</a></li>
<li><a href="#accessing-to-the-sequence-annotations" id="toc-accessing-to-the-sequence-annotations" class="nav-link" data-scroll-target="#accessing-to-the-sequence-annotations"><span class="toc-section-number">3.3.3</span> Accessing to the sequence annotations</a></li>
</ul></li>
<li><a href="#metabarcode-design-and-quality-assessment" id="toc-metabarcode-design-and-quality-assessment" class="nav-link" data-scroll-target="#metabarcode-design-and-quality-assessment"><span class="toc-section-number">2.4</span> Metabarcode design and quality assessment</a></li>
<li><a href="#file-format-conversions" id="toc-file-format-conversions" class="nav-link" data-scroll-target="#file-format-conversions"><span class="toc-section-number">2.5</span> File format conversions</a></li>
<li><a href="#sequence-annotations" id="toc-sequence-annotations" class="nav-link" data-scroll-target="#sequence-annotations"><span class="toc-section-number">2.6</span> Sequence annotations</a></li>
<li><a href="#computations-on-sequences" id="toc-computations-on-sequences" class="nav-link" data-scroll-target="#computations-on-sequences"><span class="toc-section-number">2.7</span> Computations on sequences</a>
<li><a href="#metabarcode-design-and-quality-assessment" id="toc-metabarcode-design-and-quality-assessment" class="nav-link" data-scroll-target="#metabarcode-design-and-quality-assessment"><span class="toc-section-number">3.4</span> Metabarcode design and quality assessment</a></li>
<li><a href="#file-format-conversions" id="toc-file-format-conversions" class="nav-link" data-scroll-target="#file-format-conversions"><span class="toc-section-number">3.5</span> File format conversions</a></li>
<li><a href="#sequence-annotations" id="toc-sequence-annotations" class="nav-link" data-scroll-target="#sequence-annotations"><span class="toc-section-number">3.6</span> Sequence annotations</a></li>
<li><a href="#computations-on-sequences" id="toc-computations-on-sequences" class="nav-link" data-scroll-target="#computations-on-sequences"><span class="toc-section-number">3.7</span> Computations on sequences</a>
<ul class="collapse">
<li><a href="#obipairing" id="toc-obipairing" class="nav-link" data-scroll-target="#obipairing"><span class="toc-section-number">2.7.1</span> <code>obipairing</code></a></li>
<li><a href="#obipairing" id="toc-obipairing" class="nav-link" data-scroll-target="#obipairing"><span class="toc-section-number">3.7.1</span> <code>obipairing</code></a></li>
</ul></li>
<li><a href="#sequence-sampling-and-filtering" id="toc-sequence-sampling-and-filtering" class="nav-link" data-scroll-target="#sequence-sampling-and-filtering"><span class="toc-section-number">2.8</span> Sequence sampling and filtering</a>
<li><a href="#sequence-sampling-and-filtering" id="toc-sequence-sampling-and-filtering" class="nav-link" data-scroll-target="#sequence-sampling-and-filtering"><span class="toc-section-number">3.8</span> Sequence sampling and filtering</a>
<ul class="collapse">
<li><a href="#utilities" id="toc-utilities" class="nav-link" data-scroll-target="#utilities"><span class="toc-section-number">2.8.1</span> Utilities</a></li>
<li><a href="#utilities" id="toc-utilities" class="nav-link" data-scroll-target="#utilities"><span class="toc-section-number">3.8.1</span> Utilities</a></li>
</ul></li>
</ul>
</nav>
@ -184,7 +189,7 @@ div.csl-indent {
<header id="title-block-header" class="quarto-title-block default">
<div class="quarto-title">
<h1 class="title d-none d-lg-block"><span class="chapter-number">2</span>&nbsp; <span class="chapter-title">The <em>OBITools V4</em> commands</span></h1>
<h1 class="title d-none d-lg-block"><span class="chapter-number">3</span>&nbsp; <span class="chapter-title">The <em>OBITools V4</em> commands</span></h1>
</div>
@ -199,13 +204,13 @@ div.csl-indent {
</header>
<section id="specifying-the-input-files-to-obitools-commands" class="level2" data-number="2.1">
<h2 data-number="2.1" class="anchored" data-anchor-id="specifying-the-input-files-to-obitools-commands"><span class="header-section-number">2.1</span> Specifying the input files to <em>OBITools</em> commands</h2>
<section id="specifying-the-input-files-to-obitools-commands" class="level2" data-number="3.1">
<h2 data-number="3.1" class="anchored" data-anchor-id="specifying-the-input-files-to-obitools-commands"><span class="header-section-number">3.1</span> Specifying the input files to <em>OBITools</em> commands</h2>
</section>
<section id="options-common-to-most-of-the-obitools-commands" class="level2" data-number="2.2">
<h2 data-number="2.2" class="anchored" data-anchor-id="options-common-to-most-of-the-obitools-commands"><span class="header-section-number">2.2</span> Options common to most of the <em>OBITools</em> commands</h2>
<section id="specifying-input-format" class="level3" data-number="2.2.1">
<h3 data-number="2.2.1" class="anchored" data-anchor-id="specifying-input-format"><span class="header-section-number">2.2.1</span> Specifying input format</h3>
<section id="options-common-to-most-of-the-obitools-commands" class="level2" data-number="3.2">
<h2 data-number="3.2" class="anchored" data-anchor-id="options-common-to-most-of-the-obitools-commands"><span class="header-section-number">3.2</span> Options common to most of the <em>OBITools</em> commands</h2>
<section id="specifying-input-format" class="level3" data-number="3.2.1">
<h3 data-number="3.2.1" class="anchored" data-anchor-id="specifying-input-format"><span class="header-section-number">3.2.1</span> Specifying input format</h3>
<p>Five sequence formats are accepted for input files. <a href="#fasta-classical" title="Fasta format description">Fasta</a> and <a href="#fastq-classical" title="Fastq format description">Fastq</a> are the main ones, EMBL and Genbank allow the use of flat files produced by these two international databases. The last one, ecoPCR, is maintained for compatibility with previous <em>OBITools</em> and allows to read <em>ecoPCR</em> outputs as sequence files.</p>
<ul>
<li><code>--ecopcr</code> : Read data following the <em>ecoPCR</em> output format.</li>
@ -217,8 +222,8 @@ div.csl-indent {
<li><code>--solexa</code> Decodes quality string according to the Solexa specification. (default: false)</li>
</ul>
</section>
<section id="specifying-output-format" class="level3" data-number="2.2.2">
<h3 data-number="2.2.2" class="anchored" data-anchor-id="specifying-output-format"><span class="header-section-number">2.2.2</span> Specifying output format</h3>
<section id="specifying-output-format" class="level3" data-number="3.2.2">
<h3 data-number="3.2.2" class="anchored" data-anchor-id="specifying-output-format"><span class="header-section-number">3.2.2</span> Specifying output format</h3>
<p>Only two output sequence formats are supported by OBITools, Fasta and Fastq. Fastq is used when output sequences are associated with quality information. Otherwise, Fasta is the default format. However, it is possible to force the output format by using one of the following two options. Forcing the use of Fasta results in the loss of quality information. Conversely, when the Fastq format is forced with sequences that have no quality data, dummy qualities set to 40 for each nucleotide are added.</p>
<ul>
<li><code>--fasta-output</code> Read data following the ecoPCR output format.</li>
@ -229,8 +234,8 @@ div.csl-indent {
<li><code>--no-order</code> When several input files are provided, indicates that there is no order among them. (default: false)</li>
</ul>
</section>
<section id="format-of-the-annotations-in-fasta-and-fastq-files" class="level3" data-number="2.2.3">
<h3 data-number="2.2.3" class="anchored" data-anchor-id="format-of-the-annotations-in-fasta-and-fastq-files"><span class="header-section-number">2.2.3</span> Format of the annotations in Fasta and Fastq files</h3>
<section id="format-of-the-annotations-in-fasta-and-fastq-files" class="level3" data-number="3.2.3">
<h3 data-number="3.2.3" class="anchored" data-anchor-id="format-of-the-annotations-in-fasta-and-fastq-files"><span class="header-section-number">3.2.3</span> Format of the annotations in Fasta and Fastq files</h3>
<p>OBITools extend the <a href="#fasta-classical" title="Fasta format description">Fasta</a> and <a href="#fastq-classical" title="Fastq format description">Fastq</a> formats by introducing a format for the title lines of these formats allowing to annotate every sequence. While the previous version of OBITools used an <em>ad-hoc</em> format for these annotation, this new version introduce the usage of the standard JSON format to store them.</p>
<p>On input, OBITools automatically recognize the format of the annotations, but two options allows to force the parsing following one of them. You should normally not need to use these options.</p>
<ul>
@ -242,8 +247,8 @@ div.csl-indent {
<li><p><code>--output-OBI-header|-O</code> output FASTA/FASTQ title line annotations follow OBI format. (default: false)</p></li>
<li><p><code>--output-json-header</code> output FASTA/FASTQ title line annotations follow json format. (default: false)</p></li>
</ul>
<section id="system-related-options" class="level4" data-number="2.2.3.1">
<h4 data-number="2.2.3.1" class="anchored" data-anchor-id="system-related-options"><span class="header-section-number">2.2.3.1</span> System related options</h4>
<section id="system-related-options" class="level4" data-number="3.2.3.1">
<h4 data-number="3.2.3.1" class="anchored" data-anchor-id="system-related-options"><span class="header-section-number">3.2.3.1</span> System related options</h4>
<ul>
<li><code>--debug</code> (default: false)</li>
<li><code>--help\|-h\|-?</code> (default: false)</li>
@ -253,78 +258,78 @@ div.csl-indent {
</section>
</section>
</section>
<section id="obitools-expression-language" class="level2" data-number="2.3">
<h2 data-number="2.3" class="anchored" data-anchor-id="obitools-expression-language"><span class="header-section-number">2.3</span> OBITools expression language</h2>
<section id="obitools-expression-language" class="level2" data-number="3.3">
<h2 data-number="3.3" class="anchored" data-anchor-id="obitools-expression-language"><span class="header-section-number">3.3</span> OBITools expression language</h2>
<p>Several OBITools (<em>e.g.</em> obigrep, obiannotate) allow the user to specify some simple expressions to compute values or define predicates. This expressions are parsed and evaluated using the <a href="https://pkg.go.dev/github.com/PaesslerAG/gval" title="Gval (Go eVALuate) for evaluating arbitrary expressions Go-like expressions.">gval</a> go package, which allows for evaluating go-Like expression.</p>
<section id="variables-usable-in-the-expression" class="level3" data-number="2.3.1">
<h3 data-number="2.3.1" class="anchored" data-anchor-id="variables-usable-in-the-expression"><span class="header-section-number">2.3.1</span> Variables usable in the expression</h3>
<section id="sequence" class="level4" data-number="2.3.1.1">
<h4 data-number="2.3.1.1" class="anchored" data-anchor-id="sequence"><span class="header-section-number">2.3.1.1</span> sequence</h4>
<section id="variables-usable-in-the-expression" class="level3" data-number="3.3.1">
<h3 data-number="3.3.1" class="anchored" data-anchor-id="variables-usable-in-the-expression"><span class="header-section-number">3.3.1</span> Variables usable in the expression</h3>
<section id="sequence" class="level4" data-number="3.3.1.1">
<h4 data-number="3.3.1.1" class="anchored" data-anchor-id="sequence"><span class="header-section-number">3.3.1.1</span> sequence</h4>
<p>sequence is the sequence object on which the expression is evaluated</p>
</section>
<section id="annotation" class="level4" data-number="2.3.1.2">
<h4 data-number="2.3.1.2" class="anchored" data-anchor-id="annotation"><span class="header-section-number">2.3.1.2</span> annotation</h4>
<section id="annotation" class="level4" data-number="3.3.1.2">
<h4 data-number="3.3.1.2" class="anchored" data-anchor-id="annotation"><span class="header-section-number">3.3.1.2</span> annotation</h4>
</section>
</section>
<section id="function-defined-in-the-language" class="level3" data-number="2.3.2">
<h3 data-number="2.3.2" class="anchored" data-anchor-id="function-defined-in-the-language"><span class="header-section-number">2.3.2</span> Function defined in the language</h3>
<section id="len" class="level4" data-number="2.3.2.1">
<h4 data-number="2.3.2.1" class="anchored" data-anchor-id="len"><span class="header-section-number">2.3.2.1</span> len</h4>
<section id="function-defined-in-the-language" class="level3" data-number="3.3.2">
<h3 data-number="3.3.2" class="anchored" data-anchor-id="function-defined-in-the-language"><span class="header-section-number">3.3.2</span> Function defined in the language</h3>
<section id="len" class="level4" data-number="3.3.2.1">
<h4 data-number="3.3.2.1" class="anchored" data-anchor-id="len"><span class="header-section-number">3.3.2.1</span> len</h4>
</section>
<section id="ismap" class="level4" data-number="2.3.2.2">
<h4 data-number="2.3.2.2" class="anchored" data-anchor-id="ismap"><span class="header-section-number">2.3.2.2</span> ismap</h4>
<section id="ismap" class="level4" data-number="3.3.2.2">
<h4 data-number="3.3.2.2" class="anchored" data-anchor-id="ismap"><span class="header-section-number">3.3.2.2</span> ismap</h4>
</section>
<section id="hasattribute" class="level4" data-number="2.3.2.3">
<h4 data-number="2.3.2.3" class="anchored" data-anchor-id="hasattribute"><span class="header-section-number">2.3.2.3</span> hasattribute</h4>
<section id="hasattribute" class="level4" data-number="3.3.2.3">
<h4 data-number="3.3.2.3" class="anchored" data-anchor-id="hasattribute"><span class="header-section-number">3.3.2.3</span> hasattribute</h4>
</section>
<section id="min" class="level4" data-number="2.3.2.4">
<h4 data-number="2.3.2.4" class="anchored" data-anchor-id="min"><span class="header-section-number">2.3.2.4</span> min</h4>
<section id="min" class="level4" data-number="3.3.2.4">
<h4 data-number="3.3.2.4" class="anchored" data-anchor-id="min"><span class="header-section-number">3.3.2.4</span> min</h4>
</section>
<section id="max" class="level4" data-number="2.3.2.5">
<h4 data-number="2.3.2.5" class="anchored" data-anchor-id="max"><span class="header-section-number">2.3.2.5</span> max</h4>
<section id="max" class="level4" data-number="3.3.2.5">
<h4 data-number="3.3.2.5" class="anchored" data-anchor-id="max"><span class="header-section-number">3.3.2.5</span> max</h4>
</section>
</section>
<section id="accessing-to-the-sequence-annotations" class="level3" data-number="2.3.3">
<h3 data-number="2.3.3" class="anchored" data-anchor-id="accessing-to-the-sequence-annotations"><span class="header-section-number">2.3.3</span> Accessing to the sequence annotations</h3>
<section id="accessing-to-the-sequence-annotations" class="level3" data-number="3.3.3">
<h3 data-number="3.3.3" class="anchored" data-anchor-id="accessing-to-the-sequence-annotations"><span class="header-section-number">3.3.3</span> Accessing to the sequence annotations</h3>
</section>
</section>
<section id="metabarcode-design-and-quality-assessment" class="level2" data-number="2.4">
<h2 data-number="2.4" class="anchored" data-anchor-id="metabarcode-design-and-quality-assessment"><span class="header-section-number">2.4</span> Metabarcode design and quality assessment</h2>
<section id="obipcr" class="level4" data-number="2.4.0.1">
<h4 data-number="2.4.0.1" class="anchored" data-anchor-id="obipcr"><span class="header-section-number">2.4.0.1</span> <code>obipcr</code></h4>
<section id="metabarcode-design-and-quality-assessment" class="level2" data-number="3.4">
<h2 data-number="3.4" class="anchored" data-anchor-id="metabarcode-design-and-quality-assessment"><span class="header-section-number">3.4</span> Metabarcode design and quality assessment</h2>
<section id="obipcr" class="level4" data-number="3.4.0.1">
<h4 data-number="3.4.0.1" class="anchored" data-anchor-id="obipcr"><span class="header-section-number">3.4.0.1</span> <code>obipcr</code></h4>
<blockquote class="blockquote">
<p>Replace the <code>ecoPCR</code> original <em>OBITools</em></p>
</blockquote>
</section>
</section>
<section id="file-format-conversions" class="level2" data-number="2.5">
<h2 data-number="2.5" class="anchored" data-anchor-id="file-format-conversions"><span class="header-section-number">2.5</span> File format conversions</h2>
<section id="obiconvert" class="level4" data-number="2.5.0.1">
<h4 data-number="2.5.0.1" class="anchored" data-anchor-id="obiconvert"><span class="header-section-number">2.5.0.1</span> <code>obiconvert</code></h4>
<section id="file-format-conversions" class="level2" data-number="3.5">
<h2 data-number="3.5" class="anchored" data-anchor-id="file-format-conversions"><span class="header-section-number">3.5</span> File format conversions</h2>
<section id="obiconvert" class="level4" data-number="3.5.0.1">
<h4 data-number="3.5.0.1" class="anchored" data-anchor-id="obiconvert"><span class="header-section-number">3.5.0.1</span> <code>obiconvert</code></h4>
</section>
</section>
<section id="sequence-annotations" class="level2" data-number="2.6">
<h2 data-number="2.6" class="anchored" data-anchor-id="sequence-annotations"><span class="header-section-number">2.6</span> Sequence annotations</h2>
<section id="obitag" class="level4" data-number="2.6.0.1">
<h4 data-number="2.6.0.1" class="anchored" data-anchor-id="obitag"><span class="header-section-number">2.6.0.1</span> <code>obitag</code></h4>
<section id="sequence-annotations" class="level2" data-number="3.6">
<h2 data-number="3.6" class="anchored" data-anchor-id="sequence-annotations"><span class="header-section-number">3.6</span> Sequence annotations</h2>
<section id="obitag" class="level4" data-number="3.6.0.1">
<h4 data-number="3.6.0.1" class="anchored" data-anchor-id="obitag"><span class="header-section-number">3.6.0.1</span> <code>obitag</code></h4>
</section>
</section>
<section id="computations-on-sequences" class="level2" data-number="2.7">
<h2 data-number="2.7" class="anchored" data-anchor-id="computations-on-sequences"><span class="header-section-number">2.7</span> Computations on sequences</h2>
<section id="obipairing" class="level3" data-number="2.7.1">
<h3 data-number="2.7.1" class="anchored" data-anchor-id="obipairing"><span class="header-section-number">2.7.1</span> <code>obipairing</code></h3>
<section id="computations-on-sequences" class="level2" data-number="3.7">
<h2 data-number="3.7" class="anchored" data-anchor-id="computations-on-sequences"><span class="header-section-number">3.7</span> Computations on sequences</h2>
<section id="obipairing" class="level3" data-number="3.7.1">
<h3 data-number="3.7.1" class="anchored" data-anchor-id="obipairing"><span class="header-section-number">3.7.1</span> <code>obipairing</code></h3>
<blockquote class="blockquote">
<p>Replace the <code>illuminapairedends</code> original <em>OBITools</em></p>
</blockquote>
<section id="alignment-procedure" class="level4" data-number="2.7.1.1">
<h4 data-number="2.7.1.1" class="anchored" data-anchor-id="alignment-procedure"><span class="header-section-number">2.7.1.1</span> Alignment procedure</h4>
<section id="alignment-procedure" class="level4" data-number="3.7.1.1">
<h4 data-number="3.7.1.1" class="anchored" data-anchor-id="alignment-procedure"><span class="header-section-number">3.7.1.1</span> Alignment procedure</h4>
<p><code>obipairing</code> is introducing a new alignment algorithm compared to the <code>illuminapairedend</code> command of the <code>OBITools V2</code>. Nethertheless this new algorithm has been design to produce the same results than the previous, except in very few cases.</p>
<p>The new algorithm is a two-step procedure. First, a FASTN-type algorithm <span class="citation" data-cites="Lipman1985-hw">(<a href="references.html#ref-Lipman1985-hw" role="doc-biblioref">Lipman and Pearson 1985</a>)</span> identifies the best offset between the two matched readings. This identifies the region of overlap.</p>
<p>In the second step, the matching regions of the two reads are extracted along with a flanking sequence of <span class="math inline">\(\Delta\)</span> base pairs. The two subsequences are then aligned using a “one side free end-gap” dynamic programming algorithm. This latter step is only called if at least one mismatch is detected by the FASTP step.</p>
<p>Unless the similarity between the two reads at their overlap region is very low, the addition of the flanking regions in the second step of the alignment ensures the same alignment as if the dynamic programming alignment was performed on the full reads.</p>
</section>
<section id="the-scoring-system" class="level4" data-number="2.7.1.2">
<h4 data-number="2.7.1.2" class="anchored" data-anchor-id="the-scoring-system"><span class="header-section-number">2.7.1.2</span> The scoring system</h4>
<section id="the-scoring-system" class="level4" data-number="3.7.1.2">
<h4 data-number="3.7.1.2" class="anchored" data-anchor-id="the-scoring-system"><span class="header-section-number">3.7.1.2</span> The scoring system</h4>
<p>In the dynamic programming step, the match and mismatch scores take into account the quality scores of the two aligned nucleotides. By taking these into account, the probability of a true match can be calculated for each aligned base pair.</p>
<p>If we consider a nucleotide read with a quality score <span class="math inline">\(Q\)</span>, the probability of misreading this base (<span class="math inline">\(P_E\)</span>) is : <span class="math display">\[
P_E = 10^{-\frac{Q}{10}}
@ -389,38 +394,38 @@ P(MATCH | X_1 \neq X_2) = (1-P_{E1})\frac{P_{E2}}{3} + (1-P_{E2})\frac{P_{E1}}
</div>
</div>
</section>
<section id="obimultiplex" class="level4" data-number="2.7.1.3">
<h4 data-number="2.7.1.3" class="anchored" data-anchor-id="obimultiplex"><span class="header-section-number">2.7.1.3</span> <code>obimultiplex</code></h4>
<section id="obimultiplex" class="level4" data-number="3.7.1.3">
<h4 data-number="3.7.1.3" class="anchored" data-anchor-id="obimultiplex"><span class="header-section-number">3.7.1.3</span> <code>obimultiplex</code></h4>
<blockquote class="blockquote">
<p>Replace the <code>ngsfilter</code> original <em>OBITools</em></p>
</blockquote>
</section>
<section id="obicomplement" class="level4" data-number="2.7.1.4">
<h4 data-number="2.7.1.4" class="anchored" data-anchor-id="obicomplement"><span class="header-section-number">2.7.1.4</span> <code>obicomplement</code></h4>
<section id="obicomplement" class="level4" data-number="3.7.1.4">
<h4 data-number="3.7.1.4" class="anchored" data-anchor-id="obicomplement"><span class="header-section-number">3.7.1.4</span> <code>obicomplement</code></h4>
</section>
<section id="obiclean" class="level4" data-number="2.7.1.5">
<h4 data-number="2.7.1.5" class="anchored" data-anchor-id="obiclean"><span class="header-section-number">2.7.1.5</span> <code>obiclean</code></h4>
<section id="obiclean" class="level4" data-number="3.7.1.5">
<h4 data-number="3.7.1.5" class="anchored" data-anchor-id="obiclean"><span class="header-section-number">3.7.1.5</span> <code>obiclean</code></h4>
</section>
<section id="obiuniq" class="level4" data-number="2.7.1.6">
<h4 data-number="2.7.1.6" class="anchored" data-anchor-id="obiuniq"><span class="header-section-number">2.7.1.6</span> <code>obiuniq</code></h4>
<section id="obiuniq" class="level4" data-number="3.7.1.6">
<h4 data-number="3.7.1.6" class="anchored" data-anchor-id="obiuniq"><span class="header-section-number">3.7.1.6</span> <code>obiuniq</code></h4>
</section>
</section>
</section>
<section id="sequence-sampling-and-filtering" class="level2" data-number="2.8">
<h2 data-number="2.8" class="anchored" data-anchor-id="sequence-sampling-and-filtering"><span class="header-section-number">2.8</span> Sequence sampling and filtering</h2>
<section id="obigrep" class="level4" data-number="2.8.0.1">
<h4 data-number="2.8.0.1" class="anchored" data-anchor-id="obigrep"><span class="header-section-number">2.8.0.1</span> <code>obigrep</code></h4>
<section id="sequence-sampling-and-filtering" class="level2" data-number="3.8">
<h2 data-number="3.8" class="anchored" data-anchor-id="sequence-sampling-and-filtering"><span class="header-section-number">3.8</span> Sequence sampling and filtering</h2>
<section id="obigrep" class="level4" data-number="3.8.0.1">
<h4 data-number="3.8.0.1" class="anchored" data-anchor-id="obigrep"><span class="header-section-number">3.8.0.1</span> <code>obigrep</code></h4>
</section>
<section id="utilities" class="level3" data-number="2.8.1">
<h3 data-number="2.8.1" class="anchored" data-anchor-id="utilities"><span class="header-section-number">2.8.1</span> Utilities</h3>
<section id="obicount" class="level4" data-number="2.8.1.1">
<h4 data-number="2.8.1.1" class="anchored" data-anchor-id="obicount"><span class="header-section-number">2.8.1.1</span> <code>obicount</code></h4>
<section id="utilities" class="level3" data-number="3.8.1">
<h3 data-number="3.8.1" class="anchored" data-anchor-id="utilities"><span class="header-section-number">3.8.1</span> Utilities</h3>
<section id="obicount" class="level4" data-number="3.8.1.1">
<h4 data-number="3.8.1.1" class="anchored" data-anchor-id="obicount"><span class="header-section-number">3.8.1.1</span> <code>obicount</code></h4>
</section>
<section id="obidistribute" class="level4" data-number="2.8.1.2">
<h4 data-number="2.8.1.2" class="anchored" data-anchor-id="obidistribute"><span class="header-section-number">2.8.1.2</span> <code>obidistribute</code></h4>
<section id="obidistribute" class="level4" data-number="3.8.1.2">
<h4 data-number="3.8.1.2" class="anchored" data-anchor-id="obidistribute"><span class="header-section-number">3.8.1.2</span> <code>obidistribute</code></h4>
</section>
<section id="obifind" class="level4" data-number="2.8.1.3">
<h4 data-number="2.8.1.3" class="anchored" data-anchor-id="obifind"><span class="header-section-number">2.8.1.3</span> <code>obifind</code></h4>
<section id="obifind" class="level4" data-number="3.8.1.3">
<h4 data-number="3.8.1.3" class="anchored" data-anchor-id="obifind"><span class="header-section-number">3.8.1.3</span> <code>obifind</code></h4>
<blockquote class="blockquote">
<p>Replace the <code>ecofind</code> original <em>OBITools.</em></p>
</blockquote>
@ -571,13 +576,13 @@ window.document.addEventListener("DOMContentLoaded", function (event) {
</script>
<nav class="page-navigation">
<div class="nav-page nav-page-previous">
<a href="./intro.html" class="pagination-link">
<i class="bi bi-arrow-left-short"></i> <span class="nav-page-text"><span class="chapter-number">1</span>&nbsp; <span class="chapter-title">The OBITools</span></span>
<a href="./tutorial.html" class="pagination-link">
<i class="bi bi-arrow-left-short"></i> <span class="nav-page-text"><span class="chapter-number">2</span>&nbsp; <span class="chapter-title">OBITools V4 Tutorial</span></span>
</a>
</div>
<div class="nav-page nav-page-next">
<a href="./library.html" class="pagination-link">
<span class="nav-page-text"><span class="chapter-number">3</span>&nbsp; <span class="chapter-title">The GO <em>OBITools</em> library</span></span> <i class="bi bi-arrow-right-short"></i>
<span class="nav-page-text"><span class="chapter-number">4</span>&nbsp; <span class="chapter-title">The GO <em>OBITools</em> library</span></span> <i class="bi bi-arrow-right-short"></i>
</a>
</div>
</nav>

11
doc/_book/index.html
View File

@ -125,17 +125,22 @@ div.csl-indent {
</li>
<li class="sidebar-item">
<div class="sidebar-item-container">
<a href="./commands.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">2</span>&nbsp; <span class="chapter-title">The <em>OBITools V4</em> commands</span></a>
<a href="./tutorial.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">2</span>&nbsp; <span class="chapter-title">OBITools V4 Tutorial</span></a>
</div>
</li>
<li class="sidebar-item">
<div class="sidebar-item-container">
<a href="./library.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">3</span>&nbsp; <span class="chapter-title">The GO <em>OBITools</em> library</span></a>
<a href="./commands.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">3</span>&nbsp; <span class="chapter-title">The <em>OBITools V4</em> commands</span></a>
</div>
</li>
<li class="sidebar-item">
<div class="sidebar-item-container">
<a href="./annexes.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">4</span>&nbsp; <span class="chapter-title">Annexes</span></a>
<a href="./library.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">4</span>&nbsp; <span class="chapter-title">The GO <em>OBITools</em> library</span></a>
</div>
</li>
<li class="sidebar-item">
<div class="sidebar-item-container">
<a href="./annexes.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">5</span>&nbsp; <span class="chapter-title">Annexes</span></a>
</div>
</li>
<li class="sidebar-item">

17
doc/_book/intro.html
View File

@ -49,7 +49,7 @@ div.csl-indent {
<script src="site_libs/quarto-search/fuse.min.js"></script>
<script src="site_libs/quarto-search/quarto-search.js"></script>
<meta name="quarto:offset" content="./">
<link href="./commands.html" rel="next">
<link href="./tutorial.html" rel="next">
<link href="./index.html" rel="prev">
<script src="site_libs/quarto-html/quarto.js"></script>
<script src="site_libs/quarto-html/popper.min.js"></script>
@ -125,17 +125,22 @@ div.csl-indent {
</li>
<li class="sidebar-item">
<div class="sidebar-item-container">
<a href="./commands.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">2</span>&nbsp; <span class="chapter-title">The <em>OBITools V4</em> commands</span></a>
<a href="./tutorial.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">2</span>&nbsp; <span class="chapter-title">OBITools V4 Tutorial</span></a>
</div>
</li>
<li class="sidebar-item">
<div class="sidebar-item-container">
<a href="./library.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">3</span>&nbsp; <span class="chapter-title">The GO <em>OBITools</em> library</span></a>
<a href="./commands.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">3</span>&nbsp; <span class="chapter-title">The <em>OBITools V4</em> commands</span></a>
</div>
</li>
<li class="sidebar-item">
<div class="sidebar-item-container">
<a href="./annexes.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">4</span>&nbsp; <span class="chapter-title">Annexes</span></a>
<a href="./library.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">4</span>&nbsp; <span class="chapter-title">The GO <em>OBITools</em> library</span></a>
</div>
</li>
<li class="sidebar-item">
<div class="sidebar-item-container">
<a href="./annexes.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">5</span>&nbsp; <span class="chapter-title">Annexes</span></a>
</div>
</li>
<li class="sidebar-item">
@ -524,8 +529,8 @@ window.document.addEventListener("DOMContentLoaded", function (event) {
</a>
</div>
<div class="nav-page nav-page-next">
<a href="./commands.html" class="pagination-link">
<span class="nav-page-text"><span class="chapter-number">2</span>&nbsp; <span class="chapter-title">The <em>OBITools V4</em> commands</span></span> <i class="bi bi-arrow-right-short"></i>
<a href="./tutorial.html" class="pagination-link">
<span class="nav-page-text"><span class="chapter-number">2</span>&nbsp; <span class="chapter-title">OBITools V4 Tutorial</span></span> <i class="bi bi-arrow-right-short"></i>
</a>
</div>
</nav>

83
doc/_book/library.html
View File

@ -7,7 +7,7 @@
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">
<title>OBITools V4 - 3&nbsp; The GO OBITools library</title>
<title>OBITools V4 - 4&nbsp; The GO OBITools library</title>
<style>
code{white-space: pre-wrap;}
span.smallcaps{font-variant: small-caps;}
@ -133,7 +133,7 @@ code span.wa { color: #60a0b0; font-weight: bold; font-style: italic; } /* Warni
<header id="quarto-header" class="headroom fixed-top">
<nav class="quarto-secondary-nav" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar" aria-controls="quarto-sidebar" aria-expanded="false" aria-label="Toggle sidebar navigation" onclick="if (window.quartoToggleHeadroom) { window.quartoToggleHeadroom(); }">
<div class="container-fluid d-flex justify-content-between">
<h1 class="quarto-secondary-nav-title"><span class="chapter-number">3</span>&nbsp; <span class="chapter-title">The GO <em>OBITools</em> library</span></h1>
<h1 class="quarto-secondary-nav-title"><span class="chapter-number">4</span>&nbsp; <span class="chapter-title">The GO <em>OBITools</em> library</span></h1>
<button type="button" class="quarto-btn-toggle btn" aria-label="Show secondary navigation">
<i class="bi bi-chevron-right"></i>
</button>
@ -168,17 +168,22 @@ code span.wa { color: #60a0b0; font-weight: bold; font-style: italic; } /* Warni
</li>
<li class="sidebar-item">
<div class="sidebar-item-container">
<a href="./commands.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">2</span>&nbsp; <span class="chapter-title">The <em>OBITools V4</em> commands</span></a>
<a href="./tutorial.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">2</span>&nbsp; <span class="chapter-title">OBITools V4 Tutorial</span></a>
</div>
</li>
<li class="sidebar-item">
<div class="sidebar-item-container">
<a href="./library.html" class="sidebar-item-text sidebar-link active"><span class="chapter-number">3</span>&nbsp; <span class="chapter-title">The GO <em>OBITools</em> library</span></a>
<a href="./commands.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">3</span>&nbsp; <span class="chapter-title">The <em>OBITools V4</em> commands</span></a>
</div>
</li>
<li class="sidebar-item">
<div class="sidebar-item-container">
<a href="./annexes.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">4</span>&nbsp; <span class="chapter-title">Annexes</span></a>
<a href="./library.html" class="sidebar-item-text sidebar-link active"><span class="chapter-number">4</span>&nbsp; <span class="chapter-title">The GO <em>OBITools</em> library</span></a>
</div>
</li>
<li class="sidebar-item">
<div class="sidebar-item-container">
<a href="./annexes.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">5</span>&nbsp; <span class="chapter-title">Annexes</span></a>
</div>
</li>
<li class="sidebar-item">
@ -195,18 +200,18 @@ code span.wa { color: #60a0b0; font-weight: bold; font-style: italic; } /* Warni
<h2 id="toc-title">Table of contents</h2>
<ul>
<li><a href="#biosequence" id="toc-biosequence" class="nav-link active" data-scroll-target="#biosequence"><span class="toc-section-number">3.1</span> BioSequence</a>
<li><a href="#biosequence" id="toc-biosequence" class="nav-link active" data-scroll-target="#biosequence"><span class="toc-section-number">4.1</span> BioSequence</a>
<ul class="collapse">
<li><a href="#creating-new-instances" id="toc-creating-new-instances" class="nav-link" data-scroll-target="#creating-new-instances"><span class="toc-section-number">3.1.1</span> Creating new instances</a></li>
<li><a href="#end-of-life-of-a-biosequence-instance" id="toc-end-of-life-of-a-biosequence-instance" class="nav-link" data-scroll-target="#end-of-life-of-a-biosequence-instance"><span class="toc-section-number">3.1.2</span> End of life of a <code>BioSequence</code> instance</a></li>
<li><a href="#accessing-to-the-elements-of-a-sequence" id="toc-accessing-to-the-elements-of-a-sequence" class="nav-link" data-scroll-target="#accessing-to-the-elements-of-a-sequence"><span class="toc-section-number">3.1.3</span> Accessing to the elements of a sequence</a></li>
<li><a href="#the-annotations-of-a-sequence" id="toc-the-annotations-of-a-sequence" class="nav-link" data-scroll-target="#the-annotations-of-a-sequence"><span class="toc-section-number">3.1.4</span> The annotations of a sequence</a></li>
<li><a href="#creating-new-instances" id="toc-creating-new-instances" class="nav-link" data-scroll-target="#creating-new-instances"><span class="toc-section-number">4.1.1</span> Creating new instances</a></li>
<li><a href="#end-of-life-of-a-biosequence-instance" id="toc-end-of-life-of-a-biosequence-instance" class="nav-link" data-scroll-target="#end-of-life-of-a-biosequence-instance"><span class="toc-section-number">4.1.2</span> End of life of a <code>BioSequence</code> instance</a></li>
<li><a href="#accessing-to-the-elements-of-a-sequence" id="toc-accessing-to-the-elements-of-a-sequence" class="nav-link" data-scroll-target="#accessing-to-the-elements-of-a-sequence"><span class="toc-section-number">4.1.3</span> Accessing to the elements of a sequence</a></li>
<li><a href="#the-annotations-of-a-sequence" id="toc-the-annotations-of-a-sequence" class="nav-link" data-scroll-target="#the-annotations-of-a-sequence"><span class="toc-section-number">4.1.4</span> The annotations of a sequence</a></li>
</ul></li>
<li><a href="#the-sequence-iterator" id="toc-the-sequence-iterator" class="nav-link" data-scroll-target="#the-sequence-iterator"><span class="toc-section-number">3.2</span> The sequence iterator</a>
<li><a href="#the-sequence-iterator" id="toc-the-sequence-iterator" class="nav-link" data-scroll-target="#the-sequence-iterator"><span class="toc-section-number">4.2</span> The sequence iterator</a>
<ul class="collapse">
<li><a href="#basic-usage-of-a-sequence-iterator" id="toc-basic-usage-of-a-sequence-iterator" class="nav-link" data-scroll-target="#basic-usage-of-a-sequence-iterator"><span class="toc-section-number">3.2.1</span> Basic usage of a sequence iterator</a></li>
<li><a href="#the-pipable-functions" id="toc-the-pipable-functions" class="nav-link" data-scroll-target="#the-pipable-functions"><span class="toc-section-number">3.2.2</span> The <code>Pipable</code> functions</a></li>
<li><a href="#the-teeable-functions" id="toc-the-teeable-functions" class="nav-link" data-scroll-target="#the-teeable-functions"><span class="toc-section-number">3.2.3</span> The <code>Teeable</code> functions</a></li>
<li><a href="#basic-usage-of-a-sequence-iterator" id="toc-basic-usage-of-a-sequence-iterator" class="nav-link" data-scroll-target="#basic-usage-of-a-sequence-iterator"><span class="toc-section-number">4.2.1</span> Basic usage of a sequence iterator</a></li>
<li><a href="#the-pipable-functions" id="toc-the-pipable-functions" class="nav-link" data-scroll-target="#the-pipable-functions"><span class="toc-section-number">4.2.2</span> The <code>Pipable</code> functions</a></li>
<li><a href="#the-teeable-functions" id="toc-the-teeable-functions" class="nav-link" data-scroll-target="#the-teeable-functions"><span class="toc-section-number">4.2.3</span> The <code>Teeable</code> functions</a></li>
</ul></li>
</ul>
</nav>
@ -216,7 +221,7 @@ code span.wa { color: #60a0b0; font-weight: bold; font-style: italic; } /* Warni
<header id="title-block-header" class="quarto-title-block default">
<div class="quarto-title">
<h1 class="title d-none d-lg-block"><span class="chapter-number">3</span>&nbsp; <span class="chapter-title">The GO <em>OBITools</em> library</span></h1>
<h1 class="title d-none d-lg-block"><span class="chapter-number">4</span>&nbsp; <span class="chapter-title">The GO <em>OBITools</em> library</span></h1>
</div>
@ -231,15 +236,15 @@ code span.wa { color: #60a0b0; font-weight: bold; font-style: italic; } /* Warni
</header>
<section id="biosequence" class="level2" data-number="3.1">
<h2 data-number="3.1" class="anchored" data-anchor-id="biosequence"><span class="header-section-number">3.1</span> BioSequence</h2>
<section id="biosequence" class="level2" data-number="4.1">
<h2 data-number="4.1" class="anchored" data-anchor-id="biosequence"><span class="header-section-number">4.1</span> BioSequence</h2>
<p>The <code>BioSequence</code> class is used to represent biological sequences. It allows for storing : - the sequence itself as a <code>[]byte</code> - the sequencing quality score as a <code>[]byte</code> if needed - an identifier as a <code>string</code> - a definition as a <code>string</code> - a set of <em>(key, value)</em> pairs in a <code>map[sting]interface{}</code></p>
<p>BioSequence is defined in the obiseq module and is included using the code</p>
<div class="sourceCode" id="cb1"><pre class="sourceCode go code-with-copy"><code class="sourceCode go"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="kw">import</span> <span class="op">(</span></span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a> <span class="st">"git.metabarcoding.org/lecasofts/go/obitools/pkg/obiseq"</span></span>
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a><span class="op">)</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
<section id="creating-new-instances" class="level3" data-number="3.1.1">
<h3 data-number="3.1.1" class="anchored" data-anchor-id="creating-new-instances"><span class="header-section-number">3.1.1</span> Creating new instances</h3>
<section id="creating-new-instances" class="level3" data-number="4.1.1">
<h3 data-number="4.1.1" class="anchored" data-anchor-id="creating-new-instances"><span class="header-section-number">4.1.1</span> Creating new instances</h3>
<p>To create new instance, use</p>
<ul>
<li><code>MakeBioSequence(id string, sequence []byte, definition string) obiseq.BioSequence</code></li>
@ -266,12 +271,12 @@ code span.wa { color: #60a0b0; font-weight: bold; font-style: italic; } /* Warni
<pre><code>&gt;id definition containing potentially several words
sequence</code></pre>
</section>
<section id="end-of-life-of-a-biosequence-instance" class="level3" data-number="3.1.2">
<h3 data-number="3.1.2" class="anchored" data-anchor-id="end-of-life-of-a-biosequence-instance"><span class="header-section-number">3.1.2</span> End of life of a <code>BioSequence</code> instance</h3>
<section id="end-of-life-of-a-biosequence-instance" class="level3" data-number="4.1.2">
<h3 data-number="4.1.2" class="anchored" data-anchor-id="end-of-life-of-a-biosequence-instance"><span class="header-section-number">4.1.2</span> End of life of a <code>BioSequence</code> instance</h3>
<p>When an instance of <code>BioSequence</code> is no longer in use, it is normally taken over by the GO garbage collector. If you know that an instance will never be used again, you can, if you wish, call the <code>Recycle</code> method on it to store the allocated memory elements in a <code>pool</code> to limit the allocation effort when many sequences are being handled. Once the recycle method has been called on an instance, you must ensure that no other method is called on it.</p>
</section>
<section id="accessing-to-the-elements-of-a-sequence" class="level3" data-number="3.1.3">
<h3 data-number="3.1.3" class="anchored" data-anchor-id="accessing-to-the-elements-of-a-sequence"><span class="header-section-number">3.1.3</span> Accessing to the elements of a sequence</h3>
<section id="accessing-to-the-elements-of-a-sequence" class="level3" data-number="4.1.3">
<h3 data-number="4.1.3" class="anchored" data-anchor-id="accessing-to-the-elements-of-a-sequence"><span class="header-section-number">4.1.3</span> Accessing to the elements of a sequence</h3>
<p>The different elements of an <code>obiseq.BioSequence</code> must be accessed using a set of methods. For the three main elements provided during the creation of a new instance methodes are :</p>
<ul>
<li><code>Id() string</code></li>
@ -300,8 +305,8 @@ sequence</code></pre>
<span id="cb4-14"><a href="#cb4-14" aria-hidden="true" tabindex="-1"></a> myseq<span class="op">.</span>SetId<span class="op">(</span><span class="st">"SPE01_0001"</span><span class="op">)</span></span>
<span id="cb4-15"><a href="#cb4-15" aria-hidden="true" tabindex="-1"></a> fmt<span class="op">.</span>Println<span class="op">(</span>myseq<span class="op">.</span>Id<span class="op">())</span></span>
<span id="cb4-16"><a href="#cb4-16" aria-hidden="true" tabindex="-1"></a><span class="op">}</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
<section id="different-ways-for-accessing-an-editing-the-sequence" class="level4" data-number="3.1.3.1">
<h4 data-number="3.1.3.1" class="anchored" data-anchor-id="different-ways-for-accessing-an-editing-the-sequence"><span class="header-section-number">3.1.3.1</span> Different ways for accessing an editing the sequence</h4>
<section id="different-ways-for-accessing-an-editing-the-sequence" class="level4" data-number="4.1.3.1">
<h4 data-number="4.1.3.1" class="anchored" data-anchor-id="different-ways-for-accessing-an-editing-the-sequence"><span class="header-section-number">4.1.3.1</span> Different ways for accessing an editing the sequence</h4>
<p>If <code>Sequence()</code>and <code>SetSequence(sequence []byte)</code> methods are the basic ones, several other methods exist.</p>
<ul>
<li><code>String() string</code> return the sequence directly converted to a <code>string</code> instance.</li>
@ -326,8 +331,8 @@ sequence</code></pre>
<span id="cb5-11"><a href="#cb5-11" aria-hidden="true" tabindex="-1"></a> fmt<span class="op">.</span>Println<span class="op">(</span>myseq<span class="op">.</span>String<span class="op">())</span></span>
<span id="cb5-12"><a href="#cb5-12" aria-hidden="true" tabindex="-1"></a><span class="op">}</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
</section>
<section id="sequence-quality-scores" class="level4" data-number="3.1.3.2">
<h4 data-number="3.1.3.2" class="anchored" data-anchor-id="sequence-quality-scores"><span class="header-section-number">3.1.3.2</span> Sequence quality scores</h4>
<section id="sequence-quality-scores" class="level4" data-number="4.1.3.2">
<h4 data-number="4.1.3.2" class="anchored" data-anchor-id="sequence-quality-scores"><span class="header-section-number">4.1.3.2</span> Sequence quality scores</h4>
<p>Sequence quality scores cannot be initialized at the time of instance creation. You must use dedicated methods to add quality scores to a sequence.</p>
<p>To be coherent the length of both the DNA sequence and que quality score sequence must be equal. But assessment of this constraint is realized. It is of the programmer responsability to check that invariant.</p>
<p>While accessing to the quality scores relies on the method <code>Quality() []byte</code>, setting the quality need to call one of the following method. They run similarly to their sequence dedicated conterpart.</p>
@ -339,8 +344,8 @@ sequence</code></pre>
<p>In a way analogous to the <code>Clear</code> method, <code>ClearQualities()</code> empties the sequence of quality scores.</p>
</section>
</section>
<section id="the-annotations-of-a-sequence" class="level3" data-number="3.1.4">
<h3 data-number="3.1.4" class="anchored" data-anchor-id="the-annotations-of-a-sequence"><span class="header-section-number">3.1.4</span> The annotations of a sequence</h3>
<section id="the-annotations-of-a-sequence" class="level3" data-number="4.1.4">
<h3 data-number="4.1.4" class="anchored" data-anchor-id="the-annotations-of-a-sequence"><span class="header-section-number">4.1.4</span> The annotations of a sequence</h3>
<p>A sequence can be annotated with attributes. Each attribute is associated with a value. An attribute is identified by its name. The name of an attribute consists of a character string containing no spaces or blank characters. Values can be of several types.</p>
<ul>
<li>Scalar types:
@ -363,11 +368,11 @@ sequence</code></pre>
</ul>
</section>
</section>
<section id="the-sequence-iterator" class="level2" data-number="3.2">
<h2 data-number="3.2" class="anchored" data-anchor-id="the-sequence-iterator"><span class="header-section-number">3.2</span> The sequence iterator</h2>
<section id="the-sequence-iterator" class="level2" data-number="4.2">
<h2 data-number="4.2" class="anchored" data-anchor-id="the-sequence-iterator"><span class="header-section-number">4.2</span> The sequence iterator</h2>
<p>The pakage <em>obiter</em> provides an iterator mecanism for manipulating sequences. The main class provided by this package is <code>obiiter.IBioSequence</code>. An <code>IBioSequence</code> iterator provides batch of sequences.</p>
<section id="basic-usage-of-a-sequence-iterator" class="level3" data-number="3.2.1">
<h3 data-number="3.2.1" class="anchored" data-anchor-id="basic-usage-of-a-sequence-iterator"><span class="header-section-number">3.2.1</span> Basic usage of a sequence iterator</h3>
<section id="basic-usage-of-a-sequence-iterator" class="level3" data-number="4.2.1">
<h3 data-number="4.2.1" class="anchored" data-anchor-id="basic-usage-of-a-sequence-iterator"><span class="header-section-number">4.2.1</span> Basic usage of a sequence iterator</h3>
<p>Many functions, among them functions reading sequences from a text file, return a <code>IBioSequence</code> iterator. The iterator class provides two main methods:</p>
<ul>
<li><code>Next() bool</code></li>
@ -390,12 +395,12 @@ sequence</code></pre>
<span id="cb6-14"><a href="#cb6-14" aria-hidden="true" tabindex="-1"></a><span class="op">}</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
<p>An <code>obiseq.BioSequenceBatch</code> instance is a set of sequences stored in an <code>obiseq.BioSequenceSlice</code> and a sequence number. The number of sequences in a batch is not defined. A batch can even contain zero sequences, if for example all sequences initially included in the batch have been filtered out at some stage of their processing.</p>
</section>
<section id="the-pipable-functions" class="level3" data-number="3.2.2">
<h3 data-number="3.2.2" class="anchored" data-anchor-id="the-pipable-functions"><span class="header-section-number">3.2.2</span> The <code>Pipable</code> functions</h3>
<section id="the-pipable-functions" class="level3" data-number="4.2.2">
<h3 data-number="4.2.2" class="anchored" data-anchor-id="the-pipable-functions"><span class="header-section-number">4.2.2</span> The <code>Pipable</code> functions</h3>
<p>A function consuming a <code>obiiter.IBioSequence</code> and returning a <code>obiiter.IBioSequence</code> is of class <code>obiiter.Pipable</code>.</p>
</section>
<section id="the-teeable-functions" class="level3" data-number="3.2.3">
<h3 data-number="3.2.3" class="anchored" data-anchor-id="the-teeable-functions"><span class="header-section-number">3.2.3</span> The <code>Teeable</code> functions</h3>
<section id="the-teeable-functions" class="level3" data-number="4.2.3">
<h3 data-number="4.2.3" class="anchored" data-anchor-id="the-teeable-functions"><span class="header-section-number">4.2.3</span> The <code>Teeable</code> functions</h3>
<p>A function consuming a <code>obiiter.IBioSequence</code> and returning two <code>obiiter.IBioSequence</code> instance is of class <code>obiiter.Teeable</code>.</p>
@ -539,12 +544,12 @@ window.document.addEventListener("DOMContentLoaded", function (event) {
<nav class="page-navigation">
<div class="nav-page nav-page-previous">
<a href="./commands.html" class="pagination-link">
<i class="bi bi-arrow-left-short"></i> <span class="nav-page-text"><span class="chapter-number">2</span>&nbsp; <span class="chapter-title">The <em>OBITools V4</em> commands</span></span>
<i class="bi bi-arrow-left-short"></i> <span class="nav-page-text"><span class="chapter-number">3</span>&nbsp; <span class="chapter-title">The <em>OBITools V4</em> commands</span></span>
</a>
</div>
<div class="nav-page nav-page-next">
<a href="./annexes.html" class="pagination-link">
<span class="nav-page-text"><span class="chapter-number">4</span>&nbsp; <span class="chapter-title">Annexes</span></span> <i class="bi bi-arrow-right-short"></i>
<span class="nav-page-text"><span class="chapter-number">5</span>&nbsp; <span class="chapter-title">Annexes</span></span> <i class="bi bi-arrow-right-short"></i>
</a>
</div>
</nav>

32
doc/_book/references.html
View File

@ -123,17 +123,22 @@ div.csl-indent {
</li>
<li class="sidebar-item">
<div class="sidebar-item-container">
<a href="./commands.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">2</span>&nbsp; <span class="chapter-title">The <em>OBITools V4</em> commands</span></a>
<a href="./tutorial.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">2</span>&nbsp; <span class="chapter-title">OBITools V4 Tutorial</span></a>
</div>
</li>
<li class="sidebar-item">
<div class="sidebar-item-container">
<a href="./library.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">3</span>&nbsp; <span class="chapter-title">The GO <em>OBITools</em> library</span></a>
<a href="./commands.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">3</span>&nbsp; <span class="chapter-title">The <em>OBITools V4</em> commands</span></a>
</div>
</li>
<li class="sidebar-item">
<div class="sidebar-item-container">
<a href="./annexes.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">4</span>&nbsp; <span class="chapter-title">Annexes</span></a>
<a href="./library.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">4</span>&nbsp; <span class="chapter-title">The GO <em>OBITools</em> library</span></a>
</div>
</li>
<li class="sidebar-item">
<div class="sidebar-item-container">
<a href="./annexes.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">5</span>&nbsp; <span class="chapter-title">Annexes</span></a>
</div>
</li>
<li class="sidebar-item">
@ -186,6 +191,25 @@ Lipman, D J, and W R Pearson. 1985. <span>“<span class="nocase">Rapid
and sensitive protein similarity searches</span>.”</span>
<em>Science</em> 227 (4693): 1435–41. <a href="http://www.ncbi.nlm.nih.gov/pubmed/2983426">http://www.ncbi.nlm.nih.gov/pubmed/2983426</a>.
</div>
<div id="ref-Riaz2011-gn" class="csl-entry" role="doc-biblioentry">
Riaz, Tiayyba, Wasim Shehzad, Alain Viari, François Pompanon, Pierre
Taberlet, and Eric Coissac. 2011. <span>“<span class="nocase">ecoPrimers: inference of new DNA barcode markers from
whole genome sequence analysis</span>.”</span> <em>Nucleic Acids
Research</em> 39 (21): e145. <a href="https://doi.org/10.1093/nar/gkr732">https://doi.org/10.1093/nar/gkr732</a>.
</div>
<div id="ref-Seguritan2001-tg" class="csl-entry" role="doc-biblioentry">
Seguritan, V, and F Rohwer. 2001. <span>“<span class="nocase">FastGroup:
a program to dereplicate libraries of 16S rDNA sequences</span>.”</span>
<em>BMC Bioinformatics</em> 2 (October): 9. <a href="https://doi.org/10.1186/1471-2105-2-9">https://doi.org/10.1186/1471-2105-2-9</a>.
</div>
<div id="ref-Shehzad2012-pn" class="csl-entry" role="doc-biblioentry">
Shehzad, Wasim, Tiayyba Riaz, Muhammad A Nawaz, Christian Miquel, Carole
Poillot, Safdar A Shah, Francois Pompanon, Eric Coissac, and Pierre
Taberlet. 2012. <span>“<span class="nocase">Carnivore diet analysis
based on next-generation sequencing: Application to the leopard cat
(Prionailurus bengalensis) in Pakistan</span>.”</span> <em>Molecular
Ecology</em> 21 (8): 1951–65. <a href="https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1365-294X.2011.05424.x">https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1365-294X.2011.05424.x</a>.
</div>
</div>
@ -327,7 +351,7 @@ window.document.addEventListener("DOMContentLoaded", function (event) {
<nav class="page-navigation">
<div class="nav-page nav-page-previous">
<a href="./annexes.html" class="pagination-link">
<i class="bi bi-arrow-left-short"></i> <span class="nav-page-text"><span class="chapter-number">4</span>&nbsp; <span class="chapter-title">Annexes</span></span>
<i class="bi bi-arrow-left-short"></i> <span class="nav-page-text"><span class="chapter-number">5</span>&nbsp; <span class="chapter-title">Annexes</span></span>
</a>
</div>
<div class="nav-page nav-page-next">

83
doc/_book/search.json
View File

File diff suppressed because one or more lines are too long

4
doc/_book/site_libs/bootstrap/bootstrap.min.css vendored
View File

File diff suppressed because one or more lines are too long

795
doc/_book/tutorial.html Normal file
View File

@ -0,0 +1,795 @@
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"><head>
<meta charset="utf-8">
<meta name="generator" content="quarto-1.2.256">
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">
<title>OBITools V4 - 2&nbsp; OBITools V4 Tutorial</title>
<style>
code{white-space: pre-wrap;}
span.smallcaps{font-variant: small-caps;}
div.columns{display: flex; gap: min(4vw, 1.5em);}
div.column{flex: auto; overflow-x: auto;}
div.hanging-indent{margin-left: 1.5em; text-indent: -1.5em;}
ul.task-list{list-style: none;}
ul.task-list li input[type="checkbox"] {
width: 0.8em;
margin: 0 0.8em 0.2em -1.6em;
vertical-align: middle;
}
pre > code.sourceCode { white-space: pre; position: relative; }
pre > code.sourceCode > span { display: inline-block; line-height: 1.25; }
pre > code.sourceCode > span:empty { height: 1.2em; }
.sourceCode { overflow: visible; }
code.sourceCode > span { color: inherit; text-decoration: inherit; }
div.sourceCode { margin: 1em 0; }
pre.sourceCode { margin: 0; }
@media screen {
div.sourceCode { overflow: auto; }
}
@media print {
pre > code.sourceCode { white-space: pre-wrap; }
pre > code.sourceCode > span { text-indent: -5em; padding-left: 5em; }
}
pre.numberSource code
{ counter-reset: source-line 0; }
pre.numberSource code > span
{ position: relative; left: -4em; counter-increment: source-line; }
pre.numberSource code > span > a:first-child::before
{ content: counter(source-line);
position: relative; left: -1em; text-align: right; vertical-align: baseline;
border: none; display: inline-block;
-webkit-touch-callout: none; -webkit-user-select: none;
-khtml-user-select: none; -moz-user-select: none;
-ms-user-select: none; user-select: none;
padding: 0 4px; width: 4em;
color: #aaaaaa;
}
pre.numberSource { margin-left: 3em; border-left: 1px solid #aaaaaa; padding-left: 4px; }
div.sourceCode
{ }
@media screen {
pre > code.sourceCode > span > a:first-child::before { text-decoration: underline; }
}
code span.al { color: #ff0000; font-weight: bold; } /* Alert */
code span.an { color: #60a0b0; font-weight: bold; font-style: italic; } /* Annotation */
code span.at { color: #7d9029; } /* Attribute */
code span.bn { color: #40a070; } /* BaseN */
code span.bu { color: #008000; } /* BuiltIn */
code span.cf { color: #007020; font-weight: bold; } /* ControlFlow */
code span.ch { color: #4070a0; } /* Char */
code span.cn { color: #880000; } /* Constant */
code span.co { color: #60a0b0; font-style: italic; } /* Comment */
code span.cv { color: #60a0b0; font-weight: bold; font-style: italic; } /* CommentVar */
code span.do { color: #ba2121; font-style: italic; } /* Documentation */
code span.dt { color: #902000; } /* DataType */
code span.dv { color: #40a070; } /* DecVal */
code span.er { color: #ff0000; font-weight: bold; } /* Error */
code span.ex { } /* Extension */
code span.fl { color: #40a070; } /* Float */
code span.fu { color: #06287e; } /* Function */
code span.im { color: #008000; font-weight: bold; } /* Import */
code span.in { color: #60a0b0; font-weight: bold; font-style: italic; } /* Information */
code span.kw { color: #007020; font-weight: bold; } /* Keyword */
code span.op { color: #666666; } /* Operator */
code span.ot { color: #007020; } /* Other */
code span.pp { color: #bc7a00; } /* Preprocessor */
code span.sc { color: #4070a0; } /* SpecialChar */
code span.ss { color: #bb6688; } /* SpecialString */
code span.st { color: #4070a0; } /* String */
code span.va { color: #19177c; } /* Variable */
code span.vs { color: #4070a0; } /* VerbatimString */
code span.wa { color: #60a0b0; font-weight: bold; font-style: italic; } /* Warning */
div.csl-bib-body { }
div.csl-entry {
clear: both;
}
.hanging div.csl-entry {
margin-left:2em;
text-indent:-2em;
}
div.csl-left-margin {
min-width:2em;
float:left;
}
div.csl-right-inline {
margin-left:2em;
padding-left:1em;
}
div.csl-indent {
margin-left: 2em;
}
</style>
<script src="site_libs/quarto-nav/quarto-nav.js"></script>
<script src="site_libs/quarto-nav/headroom.min.js"></script>
<script src="site_libs/clipboard/clipboard.min.js"></script>
<script src="site_libs/quarto-search/autocomplete.umd.js"></script>
<script src="site_libs/quarto-search/fuse.min.js"></script>
<script src="site_libs/quarto-search/quarto-search.js"></script>
<meta name="quarto:offset" content="./">
<link href="./commands.html" rel="next">
<link href="./intro.html" rel="prev">
<script src="site_libs/quarto-html/quarto.js"></script>
<script src="site_libs/quarto-html/popper.min.js"></script>
<script src="site_libs/quarto-html/tippy.umd.min.js"></script>
<script src="site_libs/quarto-html/anchor.min.js"></script>
<link href="site_libs/quarto-html/tippy.css" rel="stylesheet">
<link href="site_libs/quarto-html/quarto-syntax-highlighting.css" rel="stylesheet" id="quarto-text-highlighting-styles">
<script src="site_libs/bootstrap/bootstrap.min.js"></script>
<link href="site_libs/bootstrap/bootstrap-icons.css" rel="stylesheet">
<link href="site_libs/bootstrap/bootstrap.min.css" rel="stylesheet" id="quarto-bootstrap" data-mode="light">
<script id="quarto-search-options" type="application/json">{
"location": "sidebar",
"copy-button": false,
"collapse-after": 3,
"panel-placement": "start",
"type": "textbox",
"limit": 20,
"language": {
"search-no-results-text": "No results",
"search-matching-documents-text": "matching documents",
"search-copy-link-title": "Copy link to search",
"search-hide-matches-text": "Hide additional matches",
"search-more-match-text": "more match in this document",
"search-more-matches-text": "more matches in this document",
"search-clear-button-title": "Clear",
"search-detached-cancel-button-title": "Cancel",
"search-submit-button-title": "Submit"
}
}</script>
<script src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-chtml-full.js" type="text/javascript"></script>
</head>
<body class="nav-sidebar floating">
<div id="quarto-search-results"></div>
<header id="quarto-header" class="headroom fixed-top">
<nav class="quarto-secondary-nav" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar" aria-controls="quarto-sidebar" aria-expanded="false" aria-label="Toggle sidebar navigation" onclick="if (window.quartoToggleHeadroom) { window.quartoToggleHeadroom(); }">
<div class="container-fluid d-flex justify-content-between">
<h1 class="quarto-secondary-nav-title"><span class="chapter-number">2</span>&nbsp; <span class="chapter-title">OBITools V4 Tutorial</span></h1>
<button type="button" class="quarto-btn-toggle btn" aria-label="Show secondary navigation">
<i class="bi bi-chevron-right"></i>
</button>
</div>
</nav>
</header>
<!-- content -->
<div id="quarto-content" class="quarto-container page-columns page-rows-contents page-layout-article">
<!-- sidebar -->
<nav id="quarto-sidebar" class="sidebar collapse sidebar-navigation floating overflow-auto">
<div class="pt-lg-2 mt-2 text-left sidebar-header">
<div class="sidebar-title mb-0 py-0">
<a href="./">OBITools V4</a>
</div>
</div>
<div class="mt-2 flex-shrink-0 align-items-center">
<div class="sidebar-search">
<div id="quarto-search" class="" title="Search"></div>
</div>
</div>
<div class="sidebar-menu-container">
<ul class="list-unstyled mt-1">
<li class="sidebar-item">
<div class="sidebar-item-container">
<a href="./index.html" class="sidebar-item-text sidebar-link">Preface</a>
</div>
</li>
<li class="sidebar-item">
<div class="sidebar-item-container">
<a href="./intro.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">1</span>&nbsp; <span class="chapter-title">The OBITools</span></a>
</div>
</li>
<li class="sidebar-item">
<div class="sidebar-item-container">
<a href="./tutorial.html" class="sidebar-item-text sidebar-link active"><span class="chapter-number">2</span>&nbsp; <span class="chapter-title">OBITools V4 Tutorial</span></a>
</div>
</li>
<li class="sidebar-item">
<div class="sidebar-item-container">
<a href="./commands.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">3</span>&nbsp; <span class="chapter-title">The <em>OBITools V4</em> commands</span></a>
</div>
</li>
<li class="sidebar-item">
<div class="sidebar-item-container">
<a href="./library.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">4</span>&nbsp; <span class="chapter-title">The GO <em>OBITools</em> library</span></a>
</div>
</li>
<li class="sidebar-item">
<div class="sidebar-item-container">
<a href="./annexes.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">5</span>&nbsp; <span class="chapter-title">Annexes</span></a>
</div>
</li>
<li class="sidebar-item">
<div class="sidebar-item-container">
<a href="./references.html" class="sidebar-item-text sidebar-link">References</a>
</div>
</li>
</ul>
</div>
</nav>
<!-- margin-sidebar -->
<div id="quarto-margin-sidebar" class="sidebar margin-sidebar">
<nav id="TOC" role="doc-toc" class="toc-active">
<h2 id="toc-title">Table of contents</h2>
<ul>
<li><a href="#wolves-diet-based-on-dna-metabarcoding" id="toc-wolves-diet-based-on-dna-metabarcoding" class="nav-link active" data-scroll-target="#wolves-diet-based-on-dna-metabarcoding"><span class="toc-section-number">2.1</span> Wolves’ diet based on DNA metabarcoding</a></li>
<li><a href="#step-by-step-analysis" id="toc-step-by-step-analysis" class="nav-link" data-scroll-target="#step-by-step-analysis"><span class="toc-section-number">2.2</span> Step by step analysis</a>
<ul class="collapse">
<li><a href="#recover-full-sequence-reads-from-forward-and-reverse-partial-reads" id="toc-recover-full-sequence-reads-from-forward-and-reverse-partial-reads" class="nav-link" data-scroll-target="#recover-full-sequence-reads-from-forward-and-reverse-partial-reads"><span class="toc-section-number">2.2.1</span> Recover full sequence reads from forward and reverse partial reads</a></li>
<li><a href="#remove-unaligned-sequence-records" id="toc-remove-unaligned-sequence-records" class="nav-link" data-scroll-target="#remove-unaligned-sequence-records"><span class="toc-section-number">2.2.2</span> Remove unaligned sequence records</a></li>
<li><a href="#assign-each-sequence-record-to-the-corresponding-samplemarker-combination" id="toc-assign-each-sequence-record-to-the-corresponding-samplemarker-combination" class="nav-link" data-scroll-target="#assign-each-sequence-record-to-the-corresponding-samplemarker-combination"><span class="toc-section-number">2.2.3</span> Assign each sequence record to the corresponding sample/marker combination</a></li>
<li><a href="#dereplicate-reads-into-uniq-sequences" id="toc-dereplicate-reads-into-uniq-sequences" class="nav-link" data-scroll-target="#dereplicate-reads-into-uniq-sequences"><span class="toc-section-number">2.2.4</span> Dereplicate reads into uniq sequences</a></li>
<li><a href="#denoise-the-sequence-dataset" id="toc-denoise-the-sequence-dataset" class="nav-link" data-scroll-target="#denoise-the-sequence-dataset"><span class="toc-section-number">2.2.5</span> Denoise the sequence dataset</a></li>
<li><a href="#taxonomic-assignment-of-sequences" id="toc-taxonomic-assignment-of-sequences" class="nav-link" data-scroll-target="#taxonomic-assignment-of-sequences"><span class="toc-section-number">2.2.6</span> Taxonomic assignment of sequences</a></li>
<li><a href="#assign-each-sequence-to-a-taxon" id="toc-assign-each-sequence-to-a-taxon" class="nav-link" data-scroll-target="#assign-each-sequence-to-a-taxon"><span class="toc-section-number">2.2.7</span> Assign each sequence to a taxon</a></li>
<li><a href="#generate-the-final-result-table" id="toc-generate-the-final-result-table" class="nav-link" data-scroll-target="#generate-the-final-result-table"><span class="toc-section-number">2.2.8</span> Generate the final result table</a></li>
</ul></li>
</ul>
</nav>
</div>
<!-- main -->
<main class="content" id="quarto-document-content">
<header id="title-block-header" class="quarto-title-block default">
<div class="quarto-title">
<h1 class="title d-none d-lg-block"><span class="chapter-number">2</span>&nbsp; <span class="chapter-title">OBITools V4 Tutorial</span></h1>
</div>
<div class="quarto-title-meta">
</div>
</header>
<p>Here is a short tutorial on how to analyze DNA metabarcoding data produced on Illumina sequencers using:</p>
<ul>
<li>the OBITools</li>
<li>some basic Unix commands</li>
</ul>
<section id="wolves-diet-based-on-dna-metabarcoding" class="level2" data-number="2.1">
<h2 data-number="2.1" class="anchored" data-anchor-id="wolves-diet-based-on-dna-metabarcoding"><span class="header-section-number">2.1</span> Wolves’ diet based on DNA metabarcoding</h2>
<p>The data used in this tutorial correspond to the analysis of four wolf scats, using the protocol published in <span class="citation" data-cites="Shehzad2012-pn">Shehzad et al. (<a href="references.html#ref-Shehzad2012-pn" role="doc-biblioref">2012</a>)</span> for assessing carnivore diet. After extracting DNA from the faeces, the DNA amplifications were carried out using the primers <code>TTAGATACCCCACTATGC</code> and <code>TAGAACAGGCTCCTCTAG</code> amplifiying the <em>12S-V5</em> region <span class="citation" data-cites="Riaz2011-gn">(<a href="references.html#ref-Riaz2011-gn" role="doc-biblioref">Riaz et al. 2011</a>)</span>, together with a wolf blocking oligonucleotide.</p>
<p>The complete data set can be downloaded here: <a href="wolf_diet.tgz">the tutorial dataset</a></p>
<p>Once the data file is downloaded, using a UNIX terminal unarchive the data from the <code>tgz</code> file.</p>
<div class="cell">
<div class="sourceCode cell-code" id="cb1"><pre class="sourceCode bash code-with-copy"><code class="sourceCode bash"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="fu">tar</span> zxvf wolf_diet.tgz</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
</div>
<p>That command create a new directory named <code>wolf_data</code> containing every required data files:</p>
<ul>
<li><p><code>fastq &lt;fastq&gt;</code> files resulting of aGA IIx (Illumina) paired-end (2 x 108 bp) sequencing assay of DNA extracted and amplified from four wolf faeces:</p>
<ul>
<li><code>wolf_F.fastq</code></li>
<li><code>wolf_R.fastq</code></li>
</ul></li>
<li><p>the file describing the primers and tags used for all samples sequenced:</p>
<ul>
<li><code>wolf_diet_ngsfilter.txt</code> The tags correspond to short and specific sequences added on the 5' end of each primer to distinguish the different samples</li>
</ul></li>
<li><p>the file containing the reference database in a fasta format:</p>
<ul>
<li><code>db_v05_r117.fasta</code> This reference database has been extracted from the release 117 of EMBL using <code>obipcr</code></li>
</ul></li>
</ul>
<div class="cell">
</div>
<p>To not mix raw data and processed data a new directory called <code>results</code> is created.</p>
<div class="cell">
<div class="sourceCode cell-code" id="cb2"><pre class="sourceCode bash code-with-copy"><code class="sourceCode bash"><span id="cb2-1"><a href="#cb2-1" aria-hidden="true" tabindex="-1"></a><span class="fu">mkdir</span> results</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
</div>
</section>
<section id="step-by-step-analysis" class="level2" data-number="2.2">
<h2 data-number="2.2" class="anchored" data-anchor-id="step-by-step-analysis"><span class="header-section-number">2.2</span> Step by step analysis</h2>
<section id="recover-full-sequence-reads-from-forward-and-reverse-partial-reads" class="level3" data-number="2.2.1">
<h3 data-number="2.2.1" class="anchored" data-anchor-id="recover-full-sequence-reads-from-forward-and-reverse-partial-reads"><span class="header-section-number">2.2.1</span> Recover full sequence reads from forward and reverse partial reads</h3>
<p>When using the result of a paired-end sequencing assay with supposedly overlapping forward and reverse reads, the first step is to recover the assembled sequence.</p>
<p>The forward and reverse reads of the same fragment are <em>at the same line position</em> in the two fastq files obtained after sequencing. Based on these two files, the assembly of the forward and reverse reads is done with the <code>obipairing</code> utility that aligns the two reads and returns the reconstructed sequence.</p>
<p>In our case, the command is:</p>
<div class="cell">
<div class="sourceCode cell-code" id="cb3"><pre class="sourceCode bash code-with-copy"><code class="sourceCode bash"><span id="cb3-1"><a href="#cb3-1" aria-hidden="true" tabindex="-1"></a><span class="ex">obipairing</span> <span class="at">--min-identity</span><span class="op">=</span>0.8 <span class="dt">\</span></span>
<span id="cb3-2"><a href="#cb3-2" aria-hidden="true" tabindex="-1"></a> <span class="at">--min-overlap</span><span class="op">=</span>10 <span class="dt">\</span></span>
<span id="cb3-3"><a href="#cb3-3" aria-hidden="true" tabindex="-1"></a> <span class="at">-F</span> wolf_data/wolf_F.fastq <span class="dt">\</span></span>
<span id="cb3-4"><a href="#cb3-4" aria-hidden="true" tabindex="-1"></a> <span class="at">-R</span> wolf_data/wolf_R.fastq <span class="dt">\</span></span>
<span id="cb3-5"><a href="#cb3-5" aria-hidden="true" tabindex="-1"></a> <span class="op">&gt;</span> results/wolf.fastq </span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
</div>
<p>The <code>--min-identity</code> and <code>--min-overlap</code> options allow discarding sequences with low alignment quality. If after the aligment, the overlaping parts of the reads is shorter than 10 base pairs or the similarity over this aligned region is below 80% of identity, in the output file, the forward and reverse reads are not aligned but concatenated, and the value of the <code>mode</code> attribute in the sequence header is set to <code>joined</code> instead of <code>alignment</code>.</p>
</section>
<section id="remove-unaligned-sequence-records" class="level3" data-number="2.2.2">
<h3 data-number="2.2.2" class="anchored" data-anchor-id="remove-unaligned-sequence-records"><span class="header-section-number">2.2.2</span> Remove unaligned sequence records</h3>
<p>Unaligned sequences (:py<code class="interpreted-text" role="mod">mode=joined</code>) cannot be used. The following command allows removing them from the dataset:</p>
<div class="cell">
<div class="sourceCode cell-code" id="cb4"><pre class="sourceCode bash code-with-copy"><code class="sourceCode bash"><span id="cb4-1"><a href="#cb4-1" aria-hidden="true" tabindex="-1"></a><span class="ex">obigrep</span> <span class="at">-p</span> <span class="st">'annotations.mode != "join"'</span> <span class="dt">\</span></span>
<span id="cb4-2"><a href="#cb4-2" aria-hidden="true" tabindex="-1"></a> results/wolf.fastq <span class="op">&gt;</span> results/wolf.ali.fastq</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
</div>
<p>The <code>-p</code> requires a go like expression. <code>annotations.mode != "join"</code> means that if the value of the <code>mode</code> annotation of a sequence is different from <code>join</code>, the corresponding sequence record will be kept.</p>
<p>The first sequence record of <code>wolf.ali.fastq</code> can be obtained using the following command line:</p>
<div class="cell">
<div class="sourceCode cell-code" id="cb5"><pre class="sourceCode bash code-with-copy"><code class="sourceCode bash"><span id="cb5-1"><a href="#cb5-1" aria-hidden="true" tabindex="-1"></a><span class="fu">head</span> <span class="at">-n</span> 4 results/wolf.ali.fastq</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
</div>
<p>The folling piece of code appears on thew window of tour terminal.</p>
<pre><code>@HELIUM_000100422_612GNAAXX:7:108:5640:3823#0/1 {"ali_dir":"left","ali_length":62,"mode":"alignment","pairing_mismatches":{"(T:26)-&gt;(G:13)":62,"(T:34)-&gt;(G:18)":48},"score":484,"score_norm":0.968,"seq_a_single":46,"seq_ab_match":60,"seq_b_single":46}
ccgcctcctttagataccccactatgcttagccctaaacacaagtaattaatataacaaaattgttcgccagagtactaccggcaatagcttaaaactcaaaggacttggcggtgctttatacccttctagaggagcctgttctaaggaggcgg
+
CCCCCCCBCCCCCCCCCCCCCCCCCCCCCCBCCCCCBCCCCCCC&lt;CcCccbe[`F`accXV&lt;TA\RYU\\ee_e[XZ[XEEEEEEEEEE?EEEEEEEEEEDEEEEEEECCCCCCCCCCCCCCCCCCCCCCCACCCCCACCCCCCCCCCCCCCCC</code></pre>
</section>
<section id="assign-each-sequence-record-to-the-corresponding-samplemarker-combination" class="level3" data-number="2.2.3">
<h3 data-number="2.2.3" class="anchored" data-anchor-id="assign-each-sequence-record-to-the-corresponding-samplemarker-combination"><span class="header-section-number">2.2.3</span> Assign each sequence record to the corresponding sample/marker combination</h3>
<p>Each sequence record is assigned to its corresponding sample and marker using the data provided in a text file (here <code>wolf_diet_ngsfilter.txt</code>). This text file contains one line per sample, with the name of the experiment (several experiments can be included in the same file), the name of the tags (for example: <code>aattaac</code> if the same tag has been used on each extremity of the PCR products, or <code>aattaac:gaagtag</code> if the tags were different), the sequence of the forward primer, the sequence of the reverse primer, the letter <code>T</code> or <code>F</code> for sample identification using the forward primer and tag only or using both primers and both tags, respectively (see <code>obimultiplex</code> for details).</p>
<div class="cell">
<div class="sourceCode cell-code" id="cb7"><pre class="sourceCode bash code-with-copy"><code class="sourceCode bash"><span id="cb7-1"><a href="#cb7-1" aria-hidden="true" tabindex="-1"></a><span class="ex">obimultiplex</span> <span class="at">-t</span> wolf_data/wolf_diet_ngsfilter.txt <span class="dt">\</span></span>
<span id="cb7-2"><a href="#cb7-2" aria-hidden="true" tabindex="-1"></a> <span class="at">-u</span> results/unidentified.fastq <span class="dt">\</span></span>
<span id="cb7-3"><a href="#cb7-3" aria-hidden="true" tabindex="-1"></a> results/wolf.ali.fastq <span class="dt">\</span></span>
<span id="cb7-4"><a href="#cb7-4" aria-hidden="true" tabindex="-1"></a> <span class="op">&gt;</span> results/wolf.ali.assigned.fastq</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
</div>
<p>This command creates two files:</p>
<ul>
<li><code>unidentified.fastq</code> containing all the sequence records that were not assigned to a sample/marker combination</li>
<li><code>wolf.ali.assigned.fastq</code> containing all the sequence records that were properly assigned to a sample/marker combination</li>
</ul>
<p>Note that each sequence record of the <code>wolf.ali.assigned.fastq</code> file contains only the barcode sequence as the sequences of primers and tags are removed by the <code>obimultiplex</code> program. Information concerning the experiment, sample, primers and tags is added as attributes in the sequence header.</p>
<p>For instance, the first sequence record of <code>wolf.ali.assigned.fastq</code> is:</p>
<pre><code>@HELIUM_000100422_612GNAAXX:7:108:5640:3823#0/1_sub[28..127] {"ali_dir":"left","ali_length":62,"direction":"direct","experiment":"wolf_diet","forward_match":"ttagataccccactatgc","forward_mismatches":0,"forward_primer":"ttagataccccactatgc","forward_tag":"gcctcct","mode":"alignment","pairing_mismatches":{"(T:26)-&gt;(G:13)":35,"(T:34)-&gt;(G:18)":21},"reverse_match":"tagaacaggctcctctag","reverse_mismatches":0,"reverse_primer":"tagaacaggctcctctag","reverse_tag":"gcctcct","sample":"29a_F260619","score":484,"score_norm":0.968,"seq_a_single":46,"seq_ab_match":60,"seq_b_single":46}
ttagccctaaacacaagtaattaatataacaaaattgttcgccagagtactaccggcaatagcttaaaactcaaaggacttggcggtgctttataccctt
+
CCCBCCCCCBCCCCCCC&lt;CcCccbe[`F`accXV&lt;TA\RYU\\ee_e[XZ[XEEEEEEEEEE?EEEEEEEEEEDEEEEEEECCCCCCCCCCCCCCCCCCC</code></pre>
</section>
<section id="dereplicate-reads-into-uniq-sequences" class="level3" data-number="2.2.4">
<h3 data-number="2.2.4" class="anchored" data-anchor-id="dereplicate-reads-into-uniq-sequences"><span class="header-section-number">2.2.4</span> Dereplicate reads into uniq sequences</h3>
<p>The same DNA molecule can be sequenced several times. In order to reduce both file size and computations time, and to get easier interpretable results, it is convenient to work with unique <em>sequences</em> instead of <em>reads</em>. To <em>dereplicate</em> such <em>reads</em> into unique <em>sequences</em>, we use the <code>obiuniq</code> command.</p>
<table class="table">
<colgroup>
<col style="width: 86%">
</colgroup>
<tbody>
<tr class="odd">
<td>Definition: Dereplicate reads into unique sequences</td>
</tr>
<tr class="even">
<td><ol type="1">
<li>compare all the reads in a data set to each other</li>
<li>group strictly identical reads together</li>
<li>output the sequence for each group and its count in the original dataset (in this way, all duplicated reads are removed)</li>
</ol>
<p>Definition adapted from <span class="citation" data-cites="Seguritan2001-tg">Seguritan and Rohwer (<a href="references.html#ref-Seguritan2001-tg" role="doc-biblioref">2001</a>)</span></p></td>
</tr>
</tbody>
</table>
<p>For dereplication, we use the <code>obiuniq</code> command with the <code>-m sample</code>. The <code>-m sample</code> option is used to keep the information of the samples of origin for each uniquesequence.</p>
<div class="cell">
<div class="sourceCode cell-code" id="cb9"><pre class="sourceCode bash code-with-copy"><code class="sourceCode bash"><span id="cb9-1"><a href="#cb9-1" aria-hidden="true" tabindex="-1"></a><span class="ex">obiuniq</span> <span class="at">-m</span> sample <span class="dt">\</span></span>
<span id="cb9-2"><a href="#cb9-2" aria-hidden="true" tabindex="-1"></a> results/wolf.ali.assigned.fastq <span class="dt">\</span></span>
<span id="cb9-3"><a href="#cb9-3" aria-hidden="true" tabindex="-1"></a> <span class="op">&gt;</span> results/wolf.ali.assigned.uniq.fasta</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
</div>
<p>Note that <code>obiuniq</code> returns a fasta file.</p>
<p>The first sequence record of <code>wolf.ali.assigned.uniq.fasta</code> is:</p>
<pre><code>&gt;HELIUM_000100422_612GNAAXX:7:93:6991:1942#0/1_sub[28..126] {"ali_dir":"left","ali_length":63,"count":1,"direction":"reverse","experiment":"wolf_diet","forward_match":"ttagataccccactatgc","forward_mismatches":0,"forward_primer":"ttagataccccactatgc","forward_tag":"gaatatc","merged_sample":{"26a_F040644":1},"mode":"alignment","pairing_mismatches":{"(A:10)-&gt;(G:34)":76,"(C:06)-&gt;(A:34)":58},"reverse_match":"tagaacaggctcctctag","reverse_mismatches":0,"reverse_primer":"tagaacaggctcctctag","reverse_tag":"gaatatc","score":730,"score_norm":0.968,"seq_a_single":45,"seq_ab_match":61,"seq_b_single":45}
ttagccctaaacataaacattcaataaacaagaatgttcgccagagaactactagcaaca
gcctgaaactcaaaggacttggcggtgctttatatccct</code></pre>
<p>The run of <code>obiuniq</code> has added two key=values entries in the header of the fasta sequence:</p>
<ul>
<li><code class="interpreted-text" role="mod">"merged_sample":{"29a_F260619":1}</code>: this sequence have been found once in a single sample called <strong>29a_F260619</strong></li>
<li><code>"count":1</code> : the total count for this sequence is <span class="math inline">\(1\)</span></li>
</ul>
<p>To keep only these two attributes, we can use the <code>obiannotate</code> command:</p>
<div class="cell">
<div class="sourceCode cell-code" id="cb11"><pre class="sourceCode bash code-with-copy"><code class="sourceCode bash"><span id="cb11-1"><a href="#cb11-1" aria-hidden="true" tabindex="-1"></a><span class="ex">obiannotate</span> <span class="at">-k</span> count <span class="at">-k</span> merged_sample <span class="dt">\</span></span>
<span id="cb11-2"><a href="#cb11-2" aria-hidden="true" tabindex="-1"></a> results/wolf.ali.assigned.uniq.fasta <span class="dt">\</span></span>
<span id="cb11-3"><a href="#cb11-3" aria-hidden="true" tabindex="-1"></a> <span class="op">&gt;</span> results/wolf.ali.assigned.simple.fasta</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
</div>
<p>The first five sequence records of <code>wolf.ali.assigned.simple.fasta</code> become:</p>
<pre><code>&gt;HELIUM_000100422_612GNAAXX:7:26:18930:11105#0/1_sub[28..127] {"count":1,"merged_sample":{"29a_F260619":1}}
ttagccctaaacacaagtaattaatataacaaaatwattcgcyagagtactacmggcaat
agctyaaarctcamagrwcttggcggtgctttataccctt
&gt;HELIUM_000100422_612GNAAXX:7:58:5711:11399#0/1_sub[28..127] {"count":1,"merged_sample":{"29a_F260619":1}}
ttagccctaaacacaagtaattaatataacaaaattattcgccagagtwctaccgssaat
agcttaaaactcaaaggactgggcggtgctttataccctt
&gt;HELIUM_000100422_612GNAAXX:7:100:15836:9304#0/1_sub[28..127] {"count":1,"merged_sample":{"29a_F260619":1}}
ttagccctaaacatagataattacacaaacaaaattgttcaccagagtactagcggcaac
agcttaaaactcaaaggacttggcggtgctttataccctt
&gt;HELIUM_000100422_612GNAAXX:7:55:13242:9085#0/1_sub[28..126] {"count":4,"merged_sample":{"26a_F040644":4}}
ttagccctaaacataaacattcaataaacaagagtgttcgccagagtactactagcaaca
gcctgaaactcaaaggacttggcggtgctttacatccct
&gt;HELIUM_000100422_612GNAAXX:7:86:8429:13723#0/1_sub[28..127] {"count":7,"merged_sample":{"15a_F730814":5,"29a_F260619":2}}
ttagccctaaacacaagtaattaatataacaaaattattcgccagagtactaccggcaat
agcttaaaactcaaaggactcggcggtgctttataccctt</code></pre>
</section>
<section id="denoise-the-sequence-dataset" class="level3" data-number="2.2.5">
<h3 data-number="2.2.5" class="anchored" data-anchor-id="denoise-the-sequence-dataset"><span class="header-section-number">2.2.5</span> Denoise the sequence dataset</h3>
<p>To have a set of sequences assigned to their corresponding samples does not mean that all sequences are <em>biologically</em> meaningful i.e.&nbsp;some of these sequences can contains PCR and/or sequencing errors, or chimeras.</p>
<section id="tag-the-sequences-for-pcr-errors-sequence-variants" class="level4 unnumbered">
<h4 class="unnumbered anchored" data-anchor-id="tag-the-sequences-for-pcr-errors-sequence-variants">Tag the sequences for PCR errors (sequence variants)</h4>
<p>The <code>obiclean</code> program tags sequence variants as potential error generated during PCR amplification. We ask it to keep the <span class="title-ref">head</span> sequences (<code>-H</code> option) that are sequences which are not variants of another sequence with a count greater than 5% of their own count (<code>-r 0.05</code> option).</p>
<div class="cell">
<div class="sourceCode cell-code" id="cb13"><pre class="sourceCode bash code-with-copy"><code class="sourceCode bash"><span id="cb13-1"><a href="#cb13-1" aria-hidden="true" tabindex="-1"></a><span class="ex">obiclean</span> <span class="at">-s</span> sample <span class="at">-r</span> 0.05 <span class="at">-H</span> <span class="dt">\</span></span>
<span id="cb13-2"><a href="#cb13-2" aria-hidden="true" tabindex="-1"></a> results/wolf.ali.assigned.simple.fasta <span class="dt">\</span></span>
<span id="cb13-3"><a href="#cb13-3" aria-hidden="true" tabindex="-1"></a> <span class="op">&gt;</span> results/wolf.ali.assigned.simple.clean.fasta </span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
</div>
<p>One of the sequence records of <code>wolf.ali.assigned.simple.clean.fasta</code> is:</p>
<pre><code>&gt;HELIUM_000100422_612GNAAXX:7:66:4039:8016#0/1_sub[28..127] {"count":17,"merged_sample":{"13a_F730603":17},"obiclean_head":true,"obiclean_headcount":1,"obiclean_internalcount":0,"obi
clean_samplecount":1,"obiclean_singletoncount":0,"obiclean_status":{"13a_F730603":"h"},"obiclean_weight":{"13a_F730603":25}}
ctagccttaaacacaaatagttatgcaaacaaaactattcgccagagtactaccggcaac
agcccaaaactcaaaggacttggcggtgcttcacaccctt</code></pre>
<p>To remove such sequences as much as possible, we first discard rare sequences and then rsequence variants that likely correspond to artifacts.</p>
</section>
<section id="get-some-statistics-about-sequence-counts" class="level4 unnumbered">
<h4 class="unnumbered anchored" data-anchor-id="get-some-statistics-about-sequence-counts">Get some statistics about sequence counts</h4>
<div class="cell">
<div class="sourceCode cell-code" id="cb15"><pre class="sourceCode bash code-with-copy"><code class="sourceCode bash"><span id="cb15-1"><a href="#cb15-1" aria-hidden="true" tabindex="-1"></a><span class="ex">obicount</span> results/wolf.ali.assigned.simple.clean.fasta</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
<div class="cell-output cell-output-stdout">
<pre><code>time="2023-01-31T22:44:30+01:00" level=info msg="Appending results/wolf.ali.assigned.simple.clean.fasta file\n"
2749 36409 273387</code></pre>
</div>
</div>
<p>The dataset contains <span class="math inline">\(4313\)</span> sequences variant corresponding to 42452 sequence reads. Most of the variants occur only a single time in the complete dataset and are usualy named <em>singletons</em></p>
<div class="cell">
<div class="sourceCode cell-code" id="cb17"><pre class="sourceCode bash code-with-copy"><code class="sourceCode bash"><span id="cb17-1"><a href="#cb17-1" aria-hidden="true" tabindex="-1"></a><span class="ex">obigrep</span> <span class="at">-p</span> <span class="st">'sequence.Count() == 1'</span> results/wolf.ali.assigned.simple.clean.fasta <span class="dt">\</span></span>
<span id="cb17-2"><a href="#cb17-2" aria-hidden="true" tabindex="-1"></a> <span class="kw">|</span> <span class="ex">obicount</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
<div class="cell-output cell-output-stdout">
<pre><code>time="2023-01-31T22:44:30+01:00" level=info msg="Appending results/wolf.ali.assigned.simple.clean.fasta file\n"
time="2023-01-31T22:44:30+01:00" level=info msg="Reading sequences from stdin in guessed\n"
time="2023-01-31T22:44:30+01:00" level=info msg="On output use JSON headers"
2309 2309 229920</code></pre>
</div>
</div>
<p>In that dataset sigletons corresponds to <span class="math inline">\(3511\)</span> variants.</p>
<p>Using <em>R</em> and the <code>ROBIFastread</code> package able to read headers of the fasta files produced by <em>OBITools</em>, we can get more complete statistics on the distribution of occurrencies.</p>
<div class="cell">
<div class="sourceCode cell-code" id="cb19"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb19-1"><a href="#cb19-1" aria-hidden="true" tabindex="-1"></a><span class="fu">library</span>(ROBIFastread)</span>
<span id="cb19-2"><a href="#cb19-2" aria-hidden="true" tabindex="-1"></a><span class="fu">library</span>(ggplot2)</span>
<span id="cb19-3"><a href="#cb19-3" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb19-4"><a href="#cb19-4" aria-hidden="true" tabindex="-1"></a>seqs <span class="ot">&lt;-</span> <span class="fu">read_obifasta</span>(<span class="st">"results/wolf.ali.assigned.simple.clean.fasta"</span>,<span class="at">keys=</span><span class="st">"count"</span>)</span>
<span id="cb19-5"><a href="#cb19-5" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb19-6"><a href="#cb19-6" aria-hidden="true" tabindex="-1"></a><span class="fu">ggplot</span>(<span class="at">data =</span> seqs, <span class="at">mapping=</span><span class="fu">aes</span>(<span class="at">x =</span> count)) <span class="sc">+</span></span>
<span id="cb19-7"><a href="#cb19-7" aria-hidden="true" tabindex="-1"></a> <span class="fu">geom_histogram</span>(<span class="at">bins=</span><span class="dv">100</span>) <span class="sc">+</span></span>
<span id="cb19-8"><a href="#cb19-8" aria-hidden="true" tabindex="-1"></a> <span class="fu">scale_y_sqrt</span>() <span class="sc">+</span></span>
<span id="cb19-9"><a href="#cb19-9" aria-hidden="true" tabindex="-1"></a> <span class="fu">scale_x_sqrt</span>() <span class="sc">+</span></span>
<span id="cb19-10"><a href="#cb19-10" aria-hidden="true" tabindex="-1"></a> <span class="fu">geom_vline</span>(<span class="at">xintercept =</span> <span class="dv">10</span>, <span class="at">col=</span><span class="st">"red"</span>, <span class="at">lty=</span><span class="dv">2</span>) <span class="sc">+</span></span>
<span id="cb19-11"><a href="#cb19-11" aria-hidden="true" tabindex="-1"></a> <span class="fu">xlab</span>(<span class="st">"number of occurrencies of a variant"</span>) </span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
<div class="cell-output-display">
<p><img src="tutorial_files/figure-html/unnamed-chunk-9-1.png" class="img-fluid" width="672"></p>
</div>
</div>
<p>In a similar way it is also possible to plot the distribution of the sequence length.</p>
<div class="cell">
<div class="sourceCode cell-code" id="cb20"><pre class="sourceCode r code-with-copy"><code class="sourceCode r"><span id="cb20-1"><a href="#cb20-1" aria-hidden="true" tabindex="-1"></a><span class="fu">ggplot</span>(<span class="at">data =</span> seqs, <span class="at">mapping=</span><span class="fu">aes</span>(<span class="at">x =</span> <span class="fu">nchar</span>(sequence))) <span class="sc">+</span></span>
<span id="cb20-2"><a href="#cb20-2" aria-hidden="true" tabindex="-1"></a> <span class="fu">geom_histogram</span>() <span class="sc">+</span></span>
<span id="cb20-3"><a href="#cb20-3" aria-hidden="true" tabindex="-1"></a> <span class="fu">scale_y_log10</span>() <span class="sc">+</span></span>
<span id="cb20-4"><a href="#cb20-4" aria-hidden="true" tabindex="-1"></a> <span class="fu">geom_vline</span>(<span class="at">xintercept =</span> <span class="dv">80</span>, <span class="at">col=</span><span class="st">"red"</span>, <span class="at">lty=</span><span class="dv">2</span>) <span class="sc">+</span></span>
<span id="cb20-5"><a href="#cb20-5" aria-hidden="true" tabindex="-1"></a> <span class="fu">xlab</span>(<span class="st">"sequence lengths in base pair"</span>)</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
<div class="cell-output-display">
<p><img src="tutorial_files/figure-html/unnamed-chunk-10-1.png" class="img-fluid" width="672"></p>
</div>
</div>
</section>
<section id="keep-only-the-sequences-having-a-count-greater-or-equal-to-10-and-a-length-shorter-than-80-bp" class="level4 unnumbered">
<h4 class="unnumbered anchored" data-anchor-id="keep-only-the-sequences-having-a-count-greater-or-equal-to-10-and-a-length-shorter-than-80-bp">Keep only the sequences having a count greater or equal to 10 and a length shorter than 80 bp</h4>
<p>Based on the previous observation, we set the cut-off for keeping sequences for further analysis to a count of 10. To do this, we use the <code class="interpreted-text" role="doc">obigrep &lt;scripts/obigrep&gt;</code> command. The <code>-p 'count&gt;=10'</code> option means that the <code>python</code> expression :py<code class="interpreted-text" role="mod">count&gt;=10</code> must be evaluated to :py<code class="interpreted-text" role="mod">True</code> for each sequence to be kept. Based on previous knowledge we also remove sequences with a length shorter than 80 bp (option -l) as we know that the amplified 12S-V5 barcode for vertebrates must have a length around 100bp.</p>
<div class="cell">
<div class="sourceCode cell-code" id="cb21"><pre class="sourceCode bash code-with-copy"><code class="sourceCode bash"><span id="cb21-1"><a href="#cb21-1" aria-hidden="true" tabindex="-1"></a><span class="ex">obigrep</span> <span class="at">-l</span> 80 <span class="at">-p</span> <span class="st">'sequence.Count() &gt;= 10'</span> results/wolf.ali.assigned.simple.clean.fasta <span class="dt">\</span></span>
<span id="cb21-2"><a href="#cb21-2" aria-hidden="true" tabindex="-1"></a> <span class="op">&gt;</span> results/wolf.ali.assigned.simple.clean.c10.l80.fasta</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
</div>
<p>The first sequence record of <code>results/wolf.ali.assigned.simple.clean.c10.l80.fasta</code> is:</p>
<pre><code>&gt;HELIUM_000100422_612GNAAXX:7:22:2603:18023#0/1_sub[28..127] {"count":12182,"merged_sample":{"15a_F730814":7559,"29a_F260619":4623},"obiclean_head":true,"obiclean_headcount":2,"obiclean_internalcount":0,"obiclean_samplecount":2,"obiclean_singletoncount":0,"obiclean_status":{"15a_F730814":"h","29a_F260619":"h"},"obiclean_weight":{"15a_F730814":9165,"29a_F260619":6275}}
ttagccctaaacacaagtaattaatataacaaaattattcgccagagtactaccggcaat
agcttaaaactcaaaggacttggcggtgctttataccctt</code></pre>
<p>At that time in the data cleanning we have conserved :</p>
<div class="cell">
<div class="sourceCode cell-code" id="cb23"><pre class="sourceCode bash code-with-copy"><code class="sourceCode bash"><span id="cb23-1"><a href="#cb23-1" aria-hidden="true" tabindex="-1"></a><span class="ex">obicount</span> results/wolf.ali.assigned.simple.clean.c10.l80.fasta</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
<div class="cell-output cell-output-stdout">
<pre><code>time="2023-01-31T22:44:32+01:00" level=info msg="Appending results/wolf.ali.assigned.simple.clean.c10.l80.fasta file\n"
26 31337 2585</code></pre>
</div>
</div>
</section>
</section>
<section id="taxonomic-assignment-of-sequences" class="level3" data-number="2.2.6">
<h3 data-number="2.2.6" class="anchored" data-anchor-id="taxonomic-assignment-of-sequences"><span class="header-section-number">2.2.6</span> Taxonomic assignment of sequences</h3>
<p>Once denoising has been done, the next step in diet analysis is to assign the barcodes to the corresponding species in order to get the complete list of species associated to each sample.</p>
<p>Taxonomic assignment of sequences requires a reference database compiling all possible species to be identified in the sample. Assignment is then done based on sequence comparison between sample sequences and reference sequences.</p>
<section id="download-the-taxonomy" class="level4 unnumbered">
<h4 class="unnumbered anchored" data-anchor-id="download-the-taxonomy">Download the taxonomy</h4>
<div class="cell">
<div class="sourceCode cell-code" id="cb25"><pre class="sourceCode bash code-with-copy"><code class="sourceCode bash"><span id="cb25-1"><a href="#cb25-1" aria-hidden="true" tabindex="-1"></a><span class="fu">mkdir</span> TAXO</span>
<span id="cb25-2"><a href="#cb25-2" aria-hidden="true" tabindex="-1"></a><span class="bu">cd</span> TAXO</span>
<span id="cb25-3"><a href="#cb25-3" aria-hidden="true" tabindex="-1"></a><span class="ex">curl</span> http://ftp.ncbi.nih.gov/pub/taxonomy/taxdump.tar.gz <span class="dt">\</span></span>
<span id="cb25-4"><a href="#cb25-4" aria-hidden="true" tabindex="-1"></a> <span class="kw">|</span> <span class="fu">tar</span> <span class="at">-zxvf</span> <span class="at">-</span></span>
<span id="cb25-5"><a href="#cb25-5" aria-hidden="true" tabindex="-1"></a><span class="bu">cd</span> ..</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
</div>
</section>
<section id="build-a-reference-database" class="level4 unnumbered">
<h4 class="unnumbered anchored" data-anchor-id="build-a-reference-database">Build a reference database</h4>
<p>One way to build the reference database is to use the <code class="interpreted-text" role="doc">ecoPCR &lt;scripts/ecoPCR&gt;</code> program to simulate a PCR and to extract all sequences from the EMBL that may be amplified <span class="title-ref">in silico</span> by the two primers (<span class="title-ref">TTAGATACCCCACTATGC</span> and <span class="title-ref">TAGAACAGGCTCCTCTAG</span>) used for PCR amplification.</p>
<p>The full list of steps for building this reference database would then be:</p>
<ol type="1">
<li>Download the whole set of EMBL sequences (available from: <a href="ftp://ftp.ebi.ac.uk/pub/databases/embl/release/" class="uri">ftp://ftp.ebi.ac.uk/pub/databases/embl/release/</a>)</li>
<li>Download the NCBI taxonomy (available from: <a href="ftp://ftp.ncbi.nih.gov/pub/taxonomy/taxdump.tar.gz" class="uri">ftp://ftp.ncbi.nih.gov/pub/taxonomy/taxdump.tar.gz</a>)</li>
<li>Format them into the ecoPCR format (see <code class="interpreted-text" role="doc">obiconvert &lt;scripts/obiconvert&gt;</code> for how you can produce ecoPCR compatible files)</li>
<li>Use ecoPCR to simulate amplification and build a reference database based on putatively amplified barcodes together with their recorded taxonomic information</li>
</ol>
<p>As step 1 and step 3 can be really time-consuming (about one day), we alredy provide the reference database produced by the following commands so that you can skip its construction. Note that as the EMBL database and taxonomic data can evolve daily, if you run the following commands you may end up with quite different results.</p>
<p>Any utility allowing file downloading from a ftp site can be used. In the following commands, we use the commonly used <code>wget</code> <em>Unix</em> command.</p>
<section id="download-the-sequences" class="level5 unnumbered">
<h5 class="unnumbered anchored" data-anchor-id="download-the-sequences">Download the sequences</h5>
<div class="sourceCode" id="cb26"><pre class="sourceCode bash code-with-copy"><code class="sourceCode bash"><span id="cb26-1"><a href="#cb26-1" aria-hidden="true" tabindex="-1"></a><span class="op">&gt;</span> mkdir <span class="ex">EMBL</span></span>
<span id="cb26-2"><a href="#cb26-2" aria-hidden="true" tabindex="-1"></a><span class="op">&gt;</span> cd <span class="ex">EMBL</span></span>
<span id="cb26-3"><a href="#cb26-3" aria-hidden="true" tabindex="-1"></a><span class="op">&gt;</span> wget <span class="ex">-nH</span> <span class="at">--cut-dirs</span><span class="op">=</span>4 <span class="at">-Arel_std_\*.dat.gz</span> <span class="at">-m</span> ftp://ftp.ebi.ac.uk/pub/databases/embl/release/</span>
<span id="cb26-4"><a href="#cb26-4" aria-hidden="true" tabindex="-1"></a><span class="op">&gt;</span> cd <span class="ex">..</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
</section>
<section id="download-the-taxonomy-1" class="level5 unnumbered">
<h5 class="unnumbered anchored" data-anchor-id="download-the-taxonomy-1">Download the taxonomy</h5>
<div class="sourceCode" id="cb27"><pre class="sourceCode bash code-with-copy"><code class="sourceCode bash"><span id="cb27-1"><a href="#cb27-1" aria-hidden="true" tabindex="-1"></a><span class="op">&gt;</span> mkdir <span class="ex">TAXO</span></span>
<span id="cb27-2"><a href="#cb27-2" aria-hidden="true" tabindex="-1"></a><span class="op">&gt;</span> cd <span class="ex">TAXO</span></span>
<span id="cb27-3"><a href="#cb27-3" aria-hidden="true" tabindex="-1"></a><span class="op">&gt;</span> wget <span class="ex">ftp://ftp.ncbi.nih.gov/pub/taxonomy/taxdump.tar.gz</span></span>
<span id="cb27-4"><a href="#cb27-4" aria-hidden="true" tabindex="-1"></a><span class="op">&gt;</span> tar <span class="ex">-zxvf</span> taxdump.tar.gz</span>
<span id="cb27-5"><a href="#cb27-5" aria-hidden="true" tabindex="-1"></a><span class="op">&gt;</span> cd <span class="ex">..</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
</section>
<section id="use-obipcr-to-simulate-an-in-silico-pcr" class="level5 unnumbered">
<h5 class="unnumbered anchored" data-anchor-id="use-obipcr-to-simulate-an-in-silico-pcr">Use obipcr to simulate an in silico` PCR</h5>
<div class="sourceCode" id="cb28"><pre class="sourceCode bash code-with-copy"><code class="sourceCode bash"><span id="cb28-1"><a href="#cb28-1" aria-hidden="true" tabindex="-1"></a><span class="op">&gt;</span> obipcr <span class="ex">-d</span> ./ECODB/embl_last <span class="at">-e</span> 3 <span class="at">-l</span> 50 <span class="at">-L</span> 150 <span class="dt">\ </span></span>
<span id="cb28-2"><a href="#cb28-2" aria-hidden="true" tabindex="-1"></a> <span class="ex">TTAGATACCCCACTATGC</span> TAGAACAGGCTCCTCTAG <span class="op">&gt;</span> v05.ecopcr</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
<p>Note that the primers must be in the same order both in <code>wolf_diet_ngsfilter.txt</code> and in the <code>obipcr</code> command.</p>
</section>
<section id="clean-the-database" class="level5 unnumbered">
<h5 class="unnumbered anchored" data-anchor-id="clean-the-database">Clean the database</h5>
<ol type="1">
<li>filter sequences so that they have a good taxonomic description at the species, genus, and family levels (<code>obigrep</code> command command below).</li>
<li>remove redundant sequences (<code>obiuniq</code> command below).</li>
<li>ensure that the dereplicated sequences have a taxid at the family level (<code>obigrep</code> command below).</li>
<li>ensure that sequences each have a unique identification (<code>obiannotate</code> command below)</li>
</ol>
<div class="sourceCode" id="cb29"><pre class="sourceCode bash code-with-copy"><code class="sourceCode bash"><span id="cb29-1"><a href="#cb29-1" aria-hidden="true" tabindex="-1"></a><span class="op">&gt;</span> obigrep <span class="ex">-d</span> embl_last <span class="at">--require-rank</span><span class="op">=</span>species <span class="dt">\</span></span>
<span id="cb29-2"><a href="#cb29-2" aria-hidden="true" tabindex="-1"></a> <span class="at">--require-rank</span><span class="op">=</span>genus <span class="at">--require-rank</span><span class="op">=</span>family v05.ecopcr <span class="op">&gt;</span> v05_clean.fasta</span>
<span id="cb29-3"><a href="#cb29-3" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb29-4"><a href="#cb29-4" aria-hidden="true" tabindex="-1"></a><span class="op">&gt;</span> obiuniq <span class="ex">-d</span> embl_last <span class="dt">\ </span></span>
<span id="cb29-5"><a href="#cb29-5" aria-hidden="true" tabindex="-1"></a> <span class="ex">v05_clean.fasta</span> <span class="op">&gt;</span> v05_clean_uniq.fasta</span>
<span id="cb29-6"><a href="#cb29-6" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb29-7"><a href="#cb29-7" aria-hidden="true" tabindex="-1"></a><span class="op">&gt;</span> obigrep <span class="ex">-d</span> embl_last <span class="at">--require-rank</span><span class="op">=</span>family <span class="dt">\ </span></span>
<span id="cb29-8"><a href="#cb29-8" aria-hidden="true" tabindex="-1"></a> <span class="ex">v05_clean_uniq.fasta</span> <span class="op">&gt;</span> v05_clean_uniq_clean.fasta</span>
<span id="cb29-9"><a href="#cb29-9" aria-hidden="true" tabindex="-1"></a></span>
<span id="cb29-10"><a href="#cb29-10" aria-hidden="true" tabindex="-1"></a><span class="op">&gt;</span> obiannotate <span class="ex">--uniq-id</span> v05_clean_uniq_clean.fasta <span class="op">&gt;</span> db_v05.fasta</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
<p>obirefidx -t TAXO wolf_data/db_v05_r117.fasta &gt; results/db_v05_r117.indexed.fasta</p>
<div class="warning">
<div class="title">
<p>Warning</p>
</div>
<p>From now on, for the sake of clarity, the following commands will use the filenames of the files provided with the tutorial. If you decided to run the last steps and use the files you have produced, you'll have to use <code>db_v05.fasta</code> instead of <code>db_v05_r117.fasta</code> and <code>embl_last</code> instead of <code>embl_r117</code></p>
</div>
</section>
</section>
</section>
<section id="assign-each-sequence-to-a-taxon" class="level3" data-number="2.2.7">
<h3 data-number="2.2.7" class="anchored" data-anchor-id="assign-each-sequence-to-a-taxon"><span class="header-section-number">2.2.7</span> Assign each sequence to a taxon</h3>
<p>Once the reference database is built, taxonomic assignment can be carried out using the <code>obitag</code> command.</p>
<div class="cell">
<div class="sourceCode cell-code" id="cb30"><pre class="sourceCode bash code-with-copy"><code class="sourceCode bash"><span id="cb30-1"><a href="#cb30-1" aria-hidden="true" tabindex="-1"></a><span class="ex">obitag</span> <span class="at">-t</span> TAXO <span class="at">-R</span> wolf_data/db_v05_r117.indexed.fasta <span class="dt">\</span></span>
<span id="cb30-2"><a href="#cb30-2" aria-hidden="true" tabindex="-1"></a> results/wolf.ali.assigned.simple.clean.c10.l80.fasta <span class="dt">\</span></span>
<span id="cb30-3"><a href="#cb30-3" aria-hidden="true" tabindex="-1"></a> <span class="op">&gt;</span> results/wolf.ali.assigned.simple.clean.c10.l80.taxo.fasta</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
</div>
<p>The <code>obitag</code> adds several attributes in the sequence record header, among them:</p>
<ul>
<li>obitag_bestmatch=ACCESSION where ACCESSION is the id of hte sequence in the reference database that best aligns to the query sequence;</li>
<li>obitag_bestid=FLOAT where FLOAT*100 is the percentage of identity between the best match sequence and the query sequence;</li>
<li>taxid=TAXID where TAXID is the final assignation of the sequence by <code>obitag</code></li>
<li>scientific_name=NAME where NAME is the scientific name of the assigned taxid.</li>
</ul>
<p>The first sequence record of <code>wolf.ali.assigned.simple.clean.c10.l80.taxo.fasta</code> is:</p>
<div class="sourceCode" id="cb31"><pre class="sourceCode bash code-with-copy"><code class="sourceCode bash"><span id="cb31-1"><a href="#cb31-1" aria-hidden="true" tabindex="-1"></a><span class="op">&gt;</span>HELIUM_000100422_612GNAAXX:7:81:18704:12346#0/1_sub[28..126] <span class="dt">{</span><span class="st">"count"</span><span class="dt">:88</span><span class="op">,</span><span class="st">"merged_sample"</span><span class="dt">:{</span><span class="st">"26a_F040644"</span><span class="dt">:88}</span><span class="op">,</span><span class="st">"obiclean_head"</span><span class="dt">:true</span><span class="op">,</span><span class="st">"obiclean_headcount"</span><span class="dt">:1</span><span class="op">,</span><span class="st">"obiclean_internalcount"</span><span class="dt">:0</span><span class="op">,</span><span class="st">"obiclean_samplecount"</span><span class="dt">:1</span><span class="op">,</span><span class="st">"obiclean_singletoncount"</span><span class="dt">:0</span><span class="op">,</span><span class="st">"obiclean_status"</span><span class="dt">:{</span><span class="st">"26a_F040644"</span><span class="dt">:</span><span class="st">"h"</span><span class="dt">}</span><span class="op">,</span><span class="st">"obiclean_weight"</span><span class="dt">:{</span><span class="st">"26a_F040644"</span><span class="dt">:208}</span><span class="op">,</span><span class="st">"obitag_bestid"</span><span class="dt">:0.9207920792079208</span><span class="op">,</span><span class="st">"obitag_bestmatch"</span><span class="dt">:</span><span class="st">"AY769263"</span><span class="op">,</span><span class="st">"obitag_difference"</span><span class="dt">:8</span><span class="op">,</span><span class="st">"obitag_match_count"</span><span class="dt">:1</span><span class="op">,</span><span class="st">"obitag_rank"</span><span class="dt">:</span><span class="st">"clade"</span><span class="op">,</span><span class="st">"scientific_name"</span><span class="dt">:</span><span class="st">"Boreoeutheria"</span><span class="op">,</span><span class="st">"taxid"</span><span class="dt">:1437010}</span></span>
<span id="cb31-2"><a href="#cb31-2" aria-hidden="true" tabindex="-1"></a><span class="ex">ttagccctaaacataaacattcaataaacaagaatgttcgccagaggactactagcaata</span></span>
<span id="cb31-3"><a href="#cb31-3" aria-hidden="true" tabindex="-1"></a><span class="ex">gcttaaaactcaaaggacttggcggtgctttatatccct</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
</section>
<section id="generate-the-final-result-table" class="level3" data-number="2.2.8">
<h3 data-number="2.2.8" class="anchored" data-anchor-id="generate-the-final-result-table"><span class="header-section-number">2.2.8</span> Generate the final result table</h3>
<p>Some unuseful attributes can be removed at this stage.</p>
<ul>
<li>obiclean_head</li>
<li>obiclean_headcount</li>
<li>obiclean_internalcount</li>
<li>obiclean_samplecount</li>
<li>obiclean_singletoncount</li>
</ul>
<div class="cell">
<div class="sourceCode cell-code" id="cb32"><pre class="sourceCode bash code-with-copy"><code class="sourceCode bash"><span id="cb32-1"><a href="#cb32-1" aria-hidden="true" tabindex="-1"></a><span class="ex">obiannotate</span> <span class="at">--delete-tag</span><span class="op">=</span>obiclean_head <span class="dt">\</span></span>
<span id="cb32-2"><a href="#cb32-2" aria-hidden="true" tabindex="-1"></a> <span class="at">--delete-tag</span><span class="op">=</span>obiclean_headcount <span class="dt">\</span></span>
<span id="cb32-3"><a href="#cb32-3" aria-hidden="true" tabindex="-1"></a> <span class="at">--delete-tag</span><span class="op">=</span>obiclean_internalcount <span class="dt">\</span></span>
<span id="cb32-4"><a href="#cb32-4" aria-hidden="true" tabindex="-1"></a> <span class="at">--delete-tag</span><span class="op">=</span>obiclean_samplecount <span class="dt">\</span></span>
<span id="cb32-5"><a href="#cb32-5" aria-hidden="true" tabindex="-1"></a> <span class="at">--delete-tag</span><span class="op">=</span>obiclean_singletoncount <span class="dt">\</span></span>
<span id="cb32-6"><a href="#cb32-6" aria-hidden="true" tabindex="-1"></a> results/wolf.ali.assigned.simple.clean.c10.l80.taxo.fasta <span class="dt">\</span></span>
<span id="cb32-7"><a href="#cb32-7" aria-hidden="true" tabindex="-1"></a> <span class="op">&gt;</span> results/wolf.ali.assigned.simple.clean.c10.l80.taxo.ann.fasta</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
</div>
<p>The first sequence record of <code>wolf.ali.assigned.simple.c10.l80.clean.taxo.ann.fasta</code> is then:</p>
<pre><code>&gt;HELIUM_000100422_612GNAAXX:7:84:16335:5083#0/1_sub[28..126] {"count":96,"merged_sample":{"26a_F040644":11,"29a_F260619":85},"obiclean_status":{"26a_F040644":"s","29a_F260619":"h"},"obiclean_weight":{"26a_F040644":14,"29a_F260619":110},"obitag_bestid":0.9595959595959596,"obitag_bestmatch":"AC187326","obitag_difference":4,"obitag_match_count":1,"obitag_rank":"subspecies","scientific_name":"Canis lupus familiaris","taxid":9615}
ttagccctaaacataagctattccataacaaaataattcgccagagaactactagcaaca
gattaaacctcaaaggacttggcagtgctttatacccct</code></pre>
<dl>
<dt>This file contains 26 sequences. You can deduce the diet of each sample:</dt>
<dd>
<ul>
<li>13a_F730603: Cervus elaphus</li>
<li>15a_F730814: Capreolus capreolus</li>
<li>26a_F040644: Marmota sp. (according to the location, it is Marmota marmota)</li>
<li>29a_F260619: Capreolus capreolus</li>
</ul>
</dd>
</dl>
<p>Note that we also obtained a few wolf sequences although a wolf-blocking oligonucleotide was used.</p>
<div id="refs" class="references csl-bib-body hanging-indent" role="doc-bibliography" style="display: none">
<div id="ref-Riaz2011-gn" class="csl-entry" role="doc-biblioentry">
Riaz, Tiayyba, Wasim Shehzad, Alain Viari, François Pompanon, Pierre Taberlet, and Eric Coissac. 2011. <span>“<span class="nocase">ecoPrimers: inference of new DNA barcode markers from whole genome sequence analysis</span>.”</span> <em>Nucleic Acids Research</em> 39 (21): e145. <a href="https://doi.org/10.1093/nar/gkr732">https://doi.org/10.1093/nar/gkr732</a>.
</div>
<div id="ref-Seguritan2001-tg" class="csl-entry" role="doc-biblioentry">
Seguritan, V, and F Rohwer. 2001. <span>“<span class="nocase">FastGroup: a program to dereplicate libraries of 16S rDNA sequences</span>.”</span> <em>BMC Bioinformatics</em> 2 (October): 9. <a href="https://doi.org/10.1186/1471-2105-2-9">https://doi.org/10.1186/1471-2105-2-9</a>.
</div>
<div id="ref-Shehzad2012-pn" class="csl-entry" role="doc-biblioentry">
Shehzad, Wasim, Tiayyba Riaz, Muhammad A Nawaz, Christian Miquel, Carole Poillot, Safdar A Shah, Francois Pompanon, Eric Coissac, and Pierre Taberlet. 2012. <span>“<span class="nocase">Carnivore diet analysis based on next-generation sequencing: Application to the leopard cat (Prionailurus bengalensis) in Pakistan</span>.”</span> <em>Molecular Ecology</em> 21 (8): 1951–65. <a href="https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1365-294X.2011.05424.x">https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1365-294X.2011.05424.x</a>.
</div>
</div>
</section>
</section>
</main> <!-- /main -->
<script id="quarto-html-after-body" type="application/javascript">
window.document.addEventListener("DOMContentLoaded", function (event) {
const toggleBodyColorMode = (bsSheetEl) => {
const mode = bsSheetEl.getAttribute("data-mode");
const bodyEl = window.document.querySelector("body");
if (mode === "dark") {
bodyEl.classList.add("quarto-dark");
bodyEl.classList.remove("quarto-light");
} else {
bodyEl.classList.add("quarto-light");
bodyEl.classList.remove("quarto-dark");
}
}
const toggleBodyColorPrimary = () => {
const bsSheetEl = window.document.querySelector("link#quarto-bootstrap");
if (bsSheetEl) {
toggleBodyColorMode(bsSheetEl);
}
}
toggleBodyColorPrimary();
const icon = "";
const anchorJS = new window.AnchorJS();
anchorJS.options = {
placement: 'right',
icon: icon
};
anchorJS.add('.anchored');
const clipboard = new window.ClipboardJS('.code-copy-button', {
target: function(trigger) {
return trigger.previousElementSibling;
}
});
clipboard.on('success', function(e) {
// button target
const button = e.trigger;
// don't keep focus
button.blur();
// flash "checked"
button.classList.add('code-copy-button-checked');
var currentTitle = button.getAttribute("title");
button.setAttribute("title", "Copied!");
let tooltip;
if (window.bootstrap) {
button.setAttribute("data-bs-toggle", "tooltip");
button.setAttribute("data-bs-placement", "left");
button.setAttribute("data-bs-title", "Copied!");
tooltip = new bootstrap.Tooltip(button,
{ trigger: "manual",
customClass: "code-copy-button-tooltip",
offset: [0, -8]});
tooltip.show();
}
setTimeout(function() {
if (tooltip) {
tooltip.hide();
button.removeAttribute("data-bs-title");
button.removeAttribute("data-bs-toggle");
button.removeAttribute("data-bs-placement");
}
button.setAttribute("title", currentTitle);
button.classList.remove('code-copy-button-checked');
}, 1000);
// clear code selection
e.clearSelection();
});
function tippyHover(el, contentFn) {
const config = {
allowHTML: true,
content: contentFn,
maxWidth: 500,
delay: 100,
arrow: false,
appendTo: function(el) {
return el.parentElement;
},
interactive: true,
interactiveBorder: 10,
theme: 'quarto',
placement: 'bottom-start'
};
window.tippy(el, config);
}
const noterefs = window.document.querySelectorAll('a[role="doc-noteref"]');
for (var i=0; i<noterefs.length; i++) {
const ref = noterefs[i];
tippyHover(ref, function() {
// use id or data attribute instead here
let href = ref.getAttribute('data-footnote-href') || ref.getAttribute('href');
try { href = new URL(href).hash; } catch {}
const id = href.replace(/^#\/?/, "");
const note = window.document.getElementById(id);
return note.innerHTML;
});
}
const findCites = (el) => {
const parentEl = el.parentElement;
if (parentEl) {
const cites = parentEl.dataset.cites;
if (cites) {
return {
el,
cites: cites.split(' ')
};
} else {
return findCites(el.parentElement)
}
} else {
return undefined;
}
};
var bibliorefs = window.document.querySelectorAll('a[role="doc-biblioref"]');
for (var i=0; i<bibliorefs.length; i++) {
const ref = bibliorefs[i];
const citeInfo = findCites(ref);
if (citeInfo) {
tippyHover(citeInfo.el, function() {
var popup = window.document.createElement('div');
citeInfo.cites.forEach(function(cite) {
var citeDiv = window.document.createElement('div');
citeDiv.classList.add('hanging-indent');
citeDiv.classList.add('csl-entry');
var biblioDiv = window.document.getElementById('ref-' + cite);
if (biblioDiv) {
citeDiv.innerHTML = biblioDiv.innerHTML;
}
popup.appendChild(citeDiv);
});
return popup.innerHTML;
});
}
}
});
</script>
<nav class="page-navigation">
<div class="nav-page nav-page-previous">
<a href="./intro.html" class="pagination-link">
<i class="bi bi-arrow-left-short"></i> <span class="nav-page-text"><span class="chapter-number">1</span>&nbsp; <span class="chapter-title">The OBITools</span></span>
</a>
</div>
<div class="nav-page nav-page-next">
<a href="./commands.html" class="pagination-link">
<span class="nav-page-text"><span class="chapter-number">3</span>&nbsp; <span class="chapter-title">The <em>OBITools V4</em> commands</span></span> <i class="bi bi-arrow-right-short"></i>
</a>
</div>
</nav>
</div> <!-- /content -->
</body></html>

BIN
doc/_book/tutorial_files/figure-html/unnamed-chunk-10-1.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 46 KiB

BIN
doc/_book/tutorial_files/figure-html/unnamed-chunk-9-1.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 45 KiB

BIN
doc/_book/wolf_diet.tgz Normal file
View File

Binary file not shown.

9
doc/_quarto.yml
View File

@ -1,6 +1,8 @@
project:
type: book
engine: jupyter
book:
title: "OBITools V4"
author: "Eric Coissac"
@ -8,6 +10,7 @@ book:
chapters:
- index.qmd
- intro.qmd
- tutorial.qmd
- commands.qmd
- library.qmd
- annexes.qmd
@ -15,11 +18,15 @@ book:
bibliography: book.bib
execute:
freeze: auto
format:
html:
theme: cosmo
theme: zephyr
pdf:
documentclass: scrreprt

100
doc/book.bib
View File

@ -78,3 +78,103 @@
volume = 227,
year = 1985,
bdsk-url-1 = {http://www.ncbi.nlm.nih.gov/pubmed/2983426}}
@ARTICLE{Shehzad2012-pn,
title = "{Carnivore diet analysis based on next-generation sequencing:
Application to the leopard cat (Prionailurus bengalensis) in
Pakistan}",
author = "Shehzad, Wasim and Riaz, Tiayyba and Nawaz, Muhammad A and
Miquel, Christian and Poillot, Carole and Shah, Safdar A and
Pompanon, Francois and Coissac, Eric and Taberlet, Pierre",
journal = "Molecular ecology",
publisher = "Wiley Online Library",
volume = 21,
number = 8,
pages = "1951--1965",
year = 2012,
url = "https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1365-294X.2011.05424.x",
issn = "0962-1083"
}
@ARTICLE{Riaz2011-gn,
title = "{ecoPrimers: inference of new DNA barcode markers from whole
genome sequence analysis}",
author = "Riaz, Tiayyba and Shehzad, Wasim and Viari, Alain and Pompanon,
Fran{\c c}ois and Taberlet, Pierre and Coissac, Eric",
abstract = "Using non-conventional markers, DNA metabarcoding allows
biodiversity assessment from complex substrates. In this article,
we present ecoPrimers, a software for identifying new barcode
markers and their associated PCR primers. ecoPrimers scans whole
genomes to find such markers without a priori knowledge.
ecoPrimers optimizes two quality indices measuring taxonomical
range and discrimination to select the most efficient markers
from a set of reference sequences, according to specific
experimental constraints such as marker length or specifically
targeted taxa. The key step of the algorithm is the
identification of conserved regions among reference sequences for
anchoring primers. We propose an efficient algorithm based on
data mining, that allows the analysis of huge sets of sequences.
We evaluate the efficiency of ecoPrimers by running it on three
different sequence sets: mitochondrial, chloroplast and bacterial
genomes. Identified barcode markers correspond either to barcode
regions already in use for plants or animals, or to new potential
barcodes. Results from empirical experiments carried out on a
promising new barcode for analyzing vertebrate diversity fully
agree with expectations based on bioinformatics analysis. These
tests demonstrate the efficiency of ecoPrimers for inferring new
barcodes fitting with diverse experimental contexts. ecoPrimers
is available as an open source project at:
http://www.grenoble.prabi.fr/trac/ecoPrimers.",
journal = "Nucleic acids research",
volume = 39,
number = 21,
pages = "e145",
month = nov,
year = 2011,
url = "http://dx.doi.org/10.1093/nar/gkr732",
language = "en",
issn = "0305-1048, 1362-4962",
pmid = "21930509",
doi = "10.1093/nar/gkr732",
pmc = "PMC3241669"
}
@ARTICLE{Seguritan2001-tg,
title = "{FastGroup: a program to dereplicate libraries of 16S rDNA
sequences}",
author = "Seguritan, V and Rohwer, F",
abstract = "BACKGROUND: Ribosomal 16S DNA sequences are an essential tool for
identifying and classifying microbes. High-throughput DNA
sequencing now makes it economically possible to produce very
large datasets of 16S rDNA sequences in short time periods,
necessitating new computer tools for analyses. Here we describe
FastGroup, a Java program designed to dereplicate libraries of
16S rDNA sequences. By dereplication we mean to: 1) compare all
the sequences in a data set to each other, 2) group similar
sequences together, and 3) output a representative sequence from
each group. In this way, duplicate sequences are removed from a
library. RESULTS: FastGroup was tested using a library of
single-pass, bacterial 16S rDNA sequences cloned from
coral-associated bacteria. We found that the optimal strategy for
dereplicating these sequences was to: 1) trim ambiguous bases
from the 5' end of the sequences and all sequence 3' of the
conserved Bact517 site, 2) match the sequences from the 3' end,
and 3) group sequences > or =97\% identical to each other.
CONCLUSIONS: The FastGroup program simplifies the dereplication
of 16S rDNA sequence libraries and prepares the raw sequences for
subsequent analyses.",
journal = "BMC bioinformatics",
volume = 2,
pages = "9",
month = oct,
year = 2001,
url = "http://dx.doi.org/10.1186/1471-2105-2-9",
language = "en",
issn = "1471-2105",
pmid = "11707150",
doi = "10.1186/1471-2105-2-9",
pmc = "PMC59723"
}

563
doc/tutorial.qmd Normal file
View File

@ -0,0 +1,563 @@
# OBITools V4 Tutorial
Here is a short tutorial on how to analyze DNA metabarcoding data produced on Illumina sequencers using:
- the OBITools
- some basic Unix commands
## Wolves’ diet based on DNA metabarcoding
The data used in this tutorial correspond to the analysis of four wolf scats, using the protocol published in @Shehzad2012-pn for assessing carnivore diet. After extracting DNA from the faeces, the DNA amplifications were carried out using the primers `TTAGATACCCCACTATGC` and `TAGAACAGGCTCCTCTAG` amplifiying the *12S-V5* region [@Riaz2011-gn], together with a wolf blocking oligonucleotide.
The complete data set can be downloaded here: [the tutorial dataset](wolf_diet.tgz)
Once the data file is downloaded, using a UNIX terminal unarchive the data from the `tgz` file.
```{bash untar_data}
#| output: false
tar zxvf wolf_diet.tgz
```
That command create a new directory named `wolf_data` containing every required data files:
- `fastq <fastq>` files resulting of aGA IIx (Illumina) paired-end (2 x 108 bp)
sequencing assay of DNA extracted and amplified from four wolf faeces:
- `wolf_F.fastq`
- `wolf_R.fastq`
- the file describing the primers and tags used for all samples
sequenced:
- `wolf_diet_ngsfilter.txt` The tags correspond to short and
specific sequences added on the 5\' end of each primer to
distinguish the different samples
- the file containing the reference database in a fasta format:
- `db_v05_r117.fasta` This reference database has been extracted
from the release 117 of EMBL using `obipcr`
```{bash true_mk_directory}
#| output: false
#| echo: false
#| error: true
#|
if [[ ! -d results ]] ; then
mkdir results
fi
```
To not mix raw data and processed data a new directory called `results` is created.
```{bash mk_directory}
#| output: false
#| eval: false
mkdir results
```
## Step by step analysis
### Recover full sequence reads from forward and reverse partial reads
When using the result of a paired-end sequencing assay with supposedly
overlapping forward and reverse reads, the first step is to recover the
assembled sequence.
The forward and reverse reads of the same fragment are *at the same line
position* in the two fastq files obtained after sequencing. Based on
these two files, the assembly of the forward and reverse reads is done
with the `obipairing` utility that aligns the two reads and returns the
reconstructed sequence.
In our case, the command is:
```{bash pairing}
#| output: false
obipairing --min-identity=0.8 \
--min-overlap=10 \
-F wolf_data/wolf_F.fastq \
-R wolf_data/wolf_R.fastq \
> results/wolf.fastq
```
The `--min-identity` and `--min-overlap` options allow
discarding sequences with low alignment quality. If after the aligment,
the overlaping parts of the reads is shorter than 10 base pairs or the
similarity over this aligned region is below 80% of identity, in the output file,
the forward and reverse reads are not aligned but concatenated, and the value of
the `mode` attribute in the sequence header is set to `joined` instead of `alignment`.
### Remove unaligned sequence records
Unaligned sequences (:py`mode=joined`{.interpreted-text role="mod"})
cannot be used. The following command allows removing them from the
dataset:
```{bash}
#| output: false
obigrep -p 'annotations.mode != "join"' \
results/wolf.fastq > results/wolf.ali.fastq
```
The `-p` requires a go like expression. `annotations.mode != "join"` means that
if the value of the `mode` annotation of a sequence is
different from `join`, the corresponding sequence record will be kept.
The first sequence record of `wolf.ali.fastq` can be obtained using the
following command line:
```{bash}
#| eval: false
#| output: false
head -n 4 results/wolf.ali.fastq
```
The folling piece of code appears on thew window of tour terminal.
```
@HELIUM_000100422_612GNAAXX:7:108:5640:3823#0/1 {"ali_dir":"left","ali_length":62,"mode":"alignment","pairing_mismatches":{"(T:26)->(G:13)":62,"(T:34)->(G:18)":48},"score":484,"score_norm":0.968,"seq_a_single":46,"seq_ab_match":60,"seq_b_single":46}
ccgcctcctttagataccccactatgcttagccctaaacacaagtaattaatataacaaaattgttcgccagagtactaccggcaatagcttaaaactcaaaggacttggcggtgctttatacccttctagaggagcctgttctaaggaggcgg
+
CCCCCCCBCCCCCCCCCCCCCCCCCCCCCCBCCCCCBCCCCCCC<CcCccbe[`F`accXV<TA\RYU\\ee_e[XZ[XEEEEEEEEEE?EEEEEEEEEEDEEEEEEECCCCCCCCCCCCCCCCCCCCCCCACCCCCACCCCCCCCCCCCCCCC
```
### Assign each sequence record to the corresponding sample/marker combination
Each sequence record is assigned to its corresponding sample and marker
using the data provided in a text file (here `wolf_diet_ngsfilter.txt`).
This text file contains one line per sample, with the name of the
experiment (several experiments can be included in the same file), the
name of the tags (for example: `aattaac` if the same tag has been used
on each extremity of the PCR products, or `aattaac:gaagtag` if the tags
were different), the sequence of the forward primer, the sequence of the
reverse primer, the letter `T` or `F` for sample identification using
the forward primer and tag only or using both primers and both tags,
respectively (see `obimultiplex` for details).
```{bash}
#| output: false
obimultiplex -t wolf_data/wolf_diet_ngsfilter.txt \
-u results/unidentified.fastq \
results/wolf.ali.fastq \
> results/wolf.ali.assigned.fastq
```
This command creates two files:
- `unidentified.fastq` containing all the sequence records that were
not assigned to a sample/marker combination
- `wolf.ali.assigned.fastq` containing all the sequence records that
were properly assigned to a sample/marker combination
Note that each sequence record of the `wolf.ali.assigned.fastq` file
contains only the barcode sequence as the sequences of primers and tags
are removed by the `obimultiplex ` program. Information concerning the
experiment, sample, primers and tags is added as attributes in the
sequence header.
For instance, the first sequence record of `wolf.ali.assigned.fastq` is:
```
@HELIUM_000100422_612GNAAXX:7:108:5640:3823#0/1_sub[28..127] {"ali_dir":"left","ali_length":62,"direction":"direct","experiment":"wolf_diet","forward_match":"ttagataccccactatgc","forward_mismatches":0,"forward_primer":"ttagataccccactatgc","forward_tag":"gcctcct","mode":"alignment","pairing_mismatches":{"(T:26)->(G:13)":35,"(T:34)->(G:18)":21},"reverse_match":"tagaacaggctcctctag","reverse_mismatches":0,"reverse_primer":"tagaacaggctcctctag","reverse_tag":"gcctcct","sample":"29a_F260619","score":484,"score_norm":0.968,"seq_a_single":46,"seq_ab_match":60,"seq_b_single":46}
ttagccctaaacacaagtaattaatataacaaaattgttcgccagagtactaccggcaatagcttaaaactcaaaggacttggcggtgctttataccctt
+
CCCBCCCCCBCCCCCCC<CcCccbe[`F`accXV<TA\RYU\\ee_e[XZ[XEEEEEEEEEE?EEEEEEEEEEDEEEEEEECCCCCCCCCCCCCCCCCCC
```
### Dereplicate reads into uniq sequences
The same DNA molecule can be sequenced several times. In order to reduce
both file size and computations time, and to get easier interpretable
results, it is convenient to work with unique *sequences* instead of
*reads*. To *dereplicate* such *reads* into unique *sequences*, we use
the `obiuniq` command.
+-------------------------------------------------------------+
| Definition: Dereplicate reads into unique sequences |
+-------------------------------------------------------------+
| 1. compare all the reads in a data set to each other |
| 2. group strictly identical reads together |
| 3. output the sequence for each group and its count in the |
| original dataset (in this way, all duplicated reads are |
| removed) |
| |
| Definition adapted from @Seguritan2001-tg |
+-------------------------------------------------------------+
For dereplication, we use the `obiuniq ` command with the `-m sample`. The `-m sample` option is used
to keep the information of the samples of origin for each uniquesequence.
```{bash}
#| output: false
obiuniq -m sample \
results/wolf.ali.assigned.fastq \
> results/wolf.ali.assigned.uniq.fasta
```
Note that `obiuniq` returns a fasta file.
The first sequence record of `wolf.ali.assigned.uniq.fasta` is:
```
>HELIUM_000100422_612GNAAXX:7:93:6991:1942#0/1_sub[28..126] {"ali_dir":"left","ali_length":63,"count":1,"direction":"reverse","experiment":"wolf_diet","forward_match":"ttagataccccactatgc","forward_mismatches":0,"forward_primer":"ttagataccccactatgc","forward_tag":"gaatatc","merged_sample":{"26a_F040644":1},"mode":"alignment","pairing_mismatches":{"(A:10)->(G:34)":76,"(C:06)->(A:34)":58},"reverse_match":"tagaacaggctcctctag","reverse_mismatches":0,"reverse_primer":"tagaacaggctcctctag","reverse_tag":"gaatatc","score":730,"score_norm":0.968,"seq_a_single":45,"seq_ab_match":61,"seq_b_single":45}
ttagccctaaacataaacattcaataaacaagaatgttcgccagagaactactagcaaca
gcctgaaactcaaaggacttggcggtgctttatatccct
```
The run of `obiuniq` has
added two key=values entries in the header of the fasta sequence:
- `"merged_sample":{"29a_F260619":1}`{.interpreted-text
role="mod"}: this sequence have been found once in a single sample
called **29a_F260619**
- `"count":1` : the total count for this sequence is $1$
To keep only these two attributes, we can use the `obiannotate` command:
```{bash}
#| output: false
obiannotate -k count -k merged_sample \
results/wolf.ali.assigned.uniq.fasta \
> results/wolf.ali.assigned.simple.fasta
```
The first five sequence records of `wolf.ali.assigned.simple.fasta`
become:
```
>HELIUM_000100422_612GNAAXX:7:26:18930:11105#0/1_sub[28..127] {"count":1,"merged_sample":{"29a_F260619":1}}
ttagccctaaacacaagtaattaatataacaaaatwattcgcyagagtactacmggcaat
agctyaaarctcamagrwcttggcggtgctttataccctt
>HELIUM_000100422_612GNAAXX:7:58:5711:11399#0/1_sub[28..127] {"count":1,"merged_sample":{"29a_F260619":1}}
ttagccctaaacacaagtaattaatataacaaaattattcgccagagtwctaccgssaat
agcttaaaactcaaaggactgggcggtgctttataccctt
>HELIUM_000100422_612GNAAXX:7:100:15836:9304#0/1_sub[28..127] {"count":1,"merged_sample":{"29a_F260619":1}}
ttagccctaaacatagataattacacaaacaaaattgttcaccagagtactagcggcaac
agcttaaaactcaaaggacttggcggtgctttataccctt
>HELIUM_000100422_612GNAAXX:7:55:13242:9085#0/1_sub[28..126] {"count":4,"merged_sample":{"26a_F040644":4}}
ttagccctaaacataaacattcaataaacaagagtgttcgccagagtactactagcaaca
gcctgaaactcaaaggacttggcggtgctttacatccct
>HELIUM_000100422_612GNAAXX:7:86:8429:13723#0/1_sub[28..127] {"count":7,"merged_sample":{"15a_F730814":5,"29a_F260619":2}}
ttagccctaaacacaagtaattaatataacaaaattattcgccagagtactaccggcaat
agcttaaaactcaaaggactcggcggtgctttataccctt
```
### Denoise the sequence dataset
To have a set of sequences assigned to their corresponding samples does
not mean that all sequences are *biologically* meaningful i.e. some of
these sequences can contains PCR and/or sequencing errors, or chimeras.
#### Tag the sequences for PCR errors (sequence variants) {.unnumbered}
The `obiclean` program tags sequence variants as potential error generated during
PCR amplification. We ask it to keep the [head]{.title-ref} sequences (`-H` option)
that are sequences which are not variants of another sequence with a count greater than 5% of their own count
(`-r 0.05` option).
```{bash}
#| output: false
obiclean -s sample -r 0.05 -H \
results/wolf.ali.assigned.simple.fasta \
> results/wolf.ali.assigned.simple.clean.fasta
```
One of the sequence records of
`wolf.ali.assigned.simple.clean.fasta` is:
```
>HELIUM_000100422_612GNAAXX:7:66:4039:8016#0/1_sub[28..127] {"count":17,"merged_sample":{"13a_F730603":17},"obiclean_head":true,"obiclean_headcount":1,"obiclean_internalcount":0,"obi
clean_samplecount":1,"obiclean_singletoncount":0,"obiclean_status":{"13a_F730603":"h"},"obiclean_weight":{"13a_F730603":25}}
ctagccttaaacacaaatagttatgcaaacaaaactattcgccagagtactaccggcaac
agcccaaaactcaaaggacttggcggtgcttcacaccctt
```
To remove such sequences as much as possible, we first discard rare
sequences and then rsequence variants that likely correspond to
artifacts.
#### Get some statistics about sequence counts {.unnumbered}
```{bash}
obicount results/wolf.ali.assigned.simple.clean.fasta
```
The dataset contains $4313$ sequences variant corresponding to 42452 sequence reads.
Most of the variants occur only a single time in the complete dataset and are usualy
named *singletons*
```{bash}
obigrep -p 'sequence.Count() == 1' results/wolf.ali.assigned.simple.clean.fasta \
| obicount
```
In that dataset sigletons corresponds to $3511$ variants.
Using *R* and the `ROBIFastread` package able to read headers of the fasta files produced by *OBITools*,
we can get more complete statistics on the distribution of occurrencies.
```{r}
#| warning: false
library(ROBIFastread)
library(ggplot2)
seqs <- read_obifasta("results/wolf.ali.assigned.simple.clean.fasta",keys="count")
ggplot(data = seqs, mapping=aes(x = count)) +
geom_histogram(bins=100) +
scale_y_sqrt() +
scale_x_sqrt() +
geom_vline(xintercept = 10, col="red", lty=2) +
xlab("number of occurrencies of a variant")
```
In a similar way it is also possible to plot the distribution of the sequence length.
```{r}
#| warning: false
ggplot(data = seqs, mapping=aes(x = nchar(sequence))) +
geom_histogram() +
scale_y_log10() +
geom_vline(xintercept = 80, col="red", lty=2) +
xlab("sequence lengths in base pair")
```
#### Keep only the sequences having a count greater or equal to 10 and a length shorter than 80 bp {.unnumbered}
Based on the previous observation, we set the cut-off for keeping
sequences for further analysis to a count of 10. To do this, we use the
`obigrep <scripts/obigrep>`{.interpreted-text role="doc"} command. The
`-p 'count>=10'` option means that the `python` expression
:py`count>=10`{.interpreted-text role="mod"} must be evaluated to
:py`True`{.interpreted-text role="mod"} for each sequence to be kept.
Based on previous knowledge we also remove sequences with a length
shorter than 80 bp (option -l) as we know that the amplified 12S-V5
barcode for vertebrates must have a length around 100bp.
```{bash}
#| output: false
obigrep -l 80 -p 'sequence.Count() >= 10' results/wolf.ali.assigned.simple.clean.fasta \
> results/wolf.ali.assigned.simple.clean.c10.l80.fasta
```
The first sequence record of `results/wolf.ali.assigned.simple.clean.c10.l80.fasta` is:
```
>HELIUM_000100422_612GNAAXX:7:22:2603:18023#0/1_sub[28..127] {"count":12182,"merged_sample":{"15a_F730814":7559,"29a_F260619":4623},"obiclean_head":true,"obiclean_headcount":2,"obiclean_internalcount":0,"obiclean_samplecount":2,"obiclean_singletoncount":0,"obiclean_status":{"15a_F730814":"h","29a_F260619":"h"},"obiclean_weight":{"15a_F730814":9165,"29a_F260619":6275}}
ttagccctaaacacaagtaattaatataacaaaattattcgccagagtactaccggcaat
agcttaaaactcaaaggacttggcggtgctttataccctt
```
At that time in the data cleanning we have conserved :
```{bash}
obicount results/wolf.ali.assigned.simple.clean.c10.l80.fasta
```
### Taxonomic assignment of sequences
Once denoising has been done, the next step in diet analysis is to
assign the barcodes to the corresponding species in order to get the
complete list of species associated to each sample.
Taxonomic assignment of sequences requires a reference database
compiling all possible species to be identified in the sample.
Assignment is then done based on sequence comparison between sample
sequences and reference sequences.
#### Download the taxonomy {.unnumbered}
```{bash}
#| output: false
mkdir TAXO
cd TAXO
curl http://ftp.ncbi.nih.gov/pub/taxonomy/taxdump.tar.gz \
| tar -zxvf -
cd ..
```
#### Build a reference database {.unnumbered}
One way to build the reference database is to use the
`ecoPCR <scripts/ecoPCR>`{.interpreted-text role="doc"} program to
simulate a PCR and to extract all sequences from the EMBL that may be
amplified [in silico]{.title-ref} by the two primers
([TTAGATACCCCACTATGC]{.title-ref} and [TAGAACAGGCTCCTCTAG]{.title-ref})
used for PCR amplification.
The full list of steps for building this reference database would then
be:
1. Download the whole set of EMBL sequences (available from:
<ftp://ftp.ebi.ac.uk/pub/databases/embl/release/>)
2. Download the NCBI taxonomy (available from:
<ftp://ftp.ncbi.nih.gov/pub/taxonomy/taxdump.tar.gz>)
3. Format them into the ecoPCR format (see
`obiconvert <scripts/obiconvert>`{.interpreted-text role="doc"} for
how you can produce ecoPCR compatible files)
4. Use ecoPCR to simulate amplification and build a reference database
based on putatively amplified barcodes together with their recorded
taxonomic information
As step 1 and step 3 can be really time-consuming (about one day), we
alredy provide the reference database produced by the following commands
so that you can skip its construction. Note that as the EMBL database
and taxonomic data can evolve daily, if you run the following commands
you may end up with quite different results.
Any utility allowing file downloading from a ftp site can be used. In
the following commands, we use the commonly used `wget` *Unix* command.
##### Download the sequences {.unnumbered}
``` bash
> mkdir EMBL
> cd EMBL
> wget -nH --cut-dirs=4 -Arel_std_\*.dat.gz -m ftp://ftp.ebi.ac.uk/pub/databases/embl/release/
> cd ..
```
##### Download the taxonomy {.unnumbered}
``` bash
> mkdir TAXO
> cd TAXO
> wget ftp://ftp.ncbi.nih.gov/pub/taxonomy/taxdump.tar.gz
> tar -zxvf taxdump.tar.gz
> cd ..
```
##### Use obipcr to simulate an in silico\` PCR {.unnumbered}
``` bash
> obipcr -d ./ECODB/embl_last -e 3 -l 50 -L 150 \
TTAGATACCCCACTATGC TAGAACAGGCTCCTCTAG > v05.ecopcr
```
Note that the primers must be in the same order both in
`wolf_diet_ngsfilter.txt` and in the `obipcr` command.
##### Clean the database {.unnumbered}
1. filter sequences so that they have a good taxonomic description at
the species, genus, and family levels
(`obigrep` command command below).
2. remove redundant sequences (`obiuniq` command below).
3. ensure that the dereplicated sequences have a taxid at the family
level (`obigrep` command below).
4. ensure that sequences each have a unique identification
(`obiannotate` command below)
``` bash
> obigrep -d embl_last --require-rank=species \
--require-rank=genus --require-rank=family v05.ecopcr > v05_clean.fasta
> obiuniq -d embl_last \
v05_clean.fasta > v05_clean_uniq.fasta
> obigrep -d embl_last --require-rank=family \
v05_clean_uniq.fasta > v05_clean_uniq_clean.fasta
> obiannotate --uniq-id v05_clean_uniq_clean.fasta > db_v05.fasta
```
obirefidx -t TAXO wolf_data/db_v05_r117.fasta > results/db_v05_r117.indexed.fasta
::: warning
::: title
Warning
:::
From now on, for the sake of clarity, the following commands will use
the filenames of the files provided with the tutorial. If you decided to
run the last steps and use the files you have produced, you\'ll have to
use `db_v05.fasta` instead of `db_v05_r117.fasta` and `embl_last`
instead of `embl_r117`
:::
### Assign each sequence to a taxon
Once the reference database is built, taxonomic assignment can be
carried out using the `obitag` command.
```{bash}
#| output: false
obitag -t TAXO -R wolf_data/db_v05_r117.indexed.fasta \
results/wolf.ali.assigned.simple.clean.c10.l80.fasta \
> results/wolf.ali.assigned.simple.clean.c10.l80.taxo.fasta
```
The `obitag` adds several attributes in the sequence record header, among
them:
- obitag_bestmatch=ACCESSION where ACCESSION is the id of hte sequence in
the reference database that best aligns to the query sequence;
- obitag_bestid=FLOAT where FLOAT\*100 is the percentage of identity
between the best match sequence and the query sequence;
- taxid=TAXID where TAXID is the final assignation of the sequence by
`obitag`
- scientific_name=NAME where NAME is the scientific name of the
assigned taxid.
The first sequence record of `wolf.ali.assigned.simple.clean.c10.l80.taxo.fasta` is:
``` bash
>HELIUM_000100422_612GNAAXX:7:81:18704:12346#0/1_sub[28..126] {"count":88,"merged_sample":{"26a_F040644":88},"obiclean_head":true,"obiclean_headcount":1,"obiclean_internalcount":0,"obiclean_samplecount":1,"obiclean_singletoncount":0,"obiclean_status":{"26a_F040644":"h"},"obiclean_weight":{"26a_F040644":208},"obitag_bestid":0.9207920792079208,"obitag_bestmatch":"AY769263","obitag_difference":8,"obitag_match_count":1,"obitag_rank":"clade","scientific_name":"Boreoeutheria","taxid":1437010}
ttagccctaaacataaacattcaataaacaagaatgttcgccagaggactactagcaata
gcttaaaactcaaaggacttggcggtgctttatatccct
```
### Generate the final result table
Some unuseful attributes can be removed at this stage.
- obiclean_head
- obiclean_headcount
- obiclean_internalcount
- obiclean_samplecount
- obiclean_singletoncount
```{bash}
#| output: false
obiannotate --delete-tag=obiclean_head \
--delete-tag=obiclean_headcount \
--delete-tag=obiclean_internalcount \
--delete-tag=obiclean_samplecount \
--delete-tag=obiclean_singletoncount \
results/wolf.ali.assigned.simple.clean.c10.l80.taxo.fasta \
> results/wolf.ali.assigned.simple.clean.c10.l80.taxo.ann.fasta
```
The first sequence record of
`wolf.ali.assigned.simple.c10.l80.clean.taxo.ann.fasta` is then:
```
>HELIUM_000100422_612GNAAXX:7:84:16335:5083#0/1_sub[28..126] {"count":96,"merged_sample":{"26a_F040644":11,"29a_F260619":85},"obiclean_status":{"26a_F040644":"s","29a_F260619":"h"},"obiclean_weight":{"26a_F040644":14,"29a_F260619":110},"obitag_bestid":0.9595959595959596,"obitag_bestmatch":"AC187326","obitag_difference":4,"obitag_match_count":1,"obitag_rank":"subspecies","scientific_name":"Canis lupus familiaris","taxid":9615}
ttagccctaaacataagctattccataacaaaataattcgccagagaactactagcaaca
gattaaacctcaaaggacttggcagtgctttatacccct
```
This file contains 26 sequences. You can deduce the diet of each sample:
: - 13a_F730603: Cervus elaphus
- 15a_F730814: Capreolus capreolus
- 26a_F040644: Marmota sp. (according to the location, it is
Marmota marmota)
- 29a_F260619: Capreolus capreolus
Note that we also obtained a few wolf sequences although a wolf-blocking
oligonucleotide was used.