Pseudo obihead for tests

creating a column not using indexers

LICENSE

@@ -0,0 +1,518 @@
+
+  CeCILL FREE SOFTWARE LICENSE AGREEMENT
+
+Version 2.1 dated 2013-06-21
+
+
+    Notice
+
+This Agreement is a Free Software license agreement that is the result
+of discussions between its authors in order to ensure compliance with
+the two main principles guiding its drafting:
+
+  * firstly, compliance with the principles governing the distribution
+    of Free Software: access to source code, broad rights granted to users,
+  * secondly, the election of a governing law, French law, with which it
+    is conformant, both as regards the law of torts and intellectual
+    property law, and the protection that it offers to both authors and
+    holders of the economic rights over software.
+
+The authors of the CeCILL (for Ce[a] C[nrs] I[nria] L[ogiciel] L[ibre]) 
+license are: 
+
+Commissariat à l'énergie atomique et aux énergies alternatives - CEA, a
+public scientific, technical and industrial research establishment,
+having its principal place of business at 25 rue Leblanc, immeuble Le
+Ponant D, 75015 Paris, France.
+
+Centre National de la Recherche Scientifique - CNRS, a public scientific
+and technological establishment, having its principal place of business
+at 3 rue Michel-Ange, 75794 Paris cedex 16, France.
+
+Institut National de Recherche en Informatique et en Automatique -
+Inria, a public scientific and technological establishment, having its
+principal place of business at Domaine de Voluceau, Rocquencourt, BP
+105, 78153 Le Chesnay cedex, France.
+
+
+    Preamble
+
+The purpose of this Free Software license agreement is to grant users
+the right to modify and redistribute the software governed by this
+license within the framework of an open source distribution model.
+
+The exercising of this right is conditional upon certain obligations for
+users so as to preserve this status for all subsequent redistributions.
+
+In consideration of access to the source code and the rights to copy,
+modify and redistribute granted by the license, users are provided only
+with a limited warranty and the software's author, the holder of the
+economic rights, and the successive licensors only have limited liability.
+
+In this respect, the risks associated with loading, using, modifying
+and/or developing or reproducing the software by the user are brought to
+the user's attention, given its Free Software status, which may make it
+complicated to use, with the result that its use is reserved for
+developers and experienced professionals having in-depth computer
+knowledge. Users are therefore encouraged to load and test the
+suitability of the software as regards their requirements in conditions
+enabling the security of their systems and/or data to be ensured and,
+more generally, to use and operate it in the same conditions of
+security. This Agreement may be freely reproduced and published,
+provided it is not altered, and that no provisions are either added or
+removed herefrom.
+
+This Agreement may apply to any or all software for which the holder of
+the economic rights decides to submit the use thereof to its provisions.
+
+Frequently asked questions can be found on the official website of the
+CeCILL licenses family (http://www.cecill.info/index.en.html) for any 
+necessary clarification.
+
+
+    Article 1 - DEFINITIONS
+
+For the purpose of this Agreement, when the following expressions
+commence with a capital letter, they shall have the following meaning:
+
+Agreement: means this license agreement, and its possible subsequent
+versions and annexes.
+
+Software: means the software in its Object Code and/or Source Code form
+and, where applicable, its documentation, "as is" when the Licensee
+accepts the Agreement.
+
+Initial Software: means the Software in its Source Code and possibly its
+Object Code form and, where applicable, its documentation, "as is" when
+it is first distributed under the terms and conditions of the Agreement.
+
+Modified Software: means the Software modified by at least one
+Contribution.
+
+Source Code: means all the Software's instructions and program lines to
+which access is required so as to modify the Software.
+
+Object Code: means the binary files originating from the compilation of
+the Source Code.
+
+Holder: means the holder(s) of the economic rights over the Initial
+Software.
+
+Licensee: means the Software user(s) having accepted the Agreement.
+
+Contributor: means a Licensee having made at least one Contribution.
+
+Licensor: means the Holder, or any other individual or legal entity, who
+distributes the Software under the Agreement.
+
+Contribution: means any or all modifications, corrections, translations,
+adaptations and/or new functions integrated into the Software by any or
+all Contributors, as well as any or all Internal Modules.
+
+Module: means a set of sources files including their documentation that
+enables supplementary functions or services in addition to those offered
+by the Software.
+
+External Module: means any or all Modules, not derived from the
+Software, so that this Module and the Software run in separate address
+spaces, with one calling the other when they are run.
+
+Internal Module: means any or all Module, connected to the Software so
+that they both execute in the same address space.
+
+GNU GPL: means the GNU General Public License version 2 or any
+subsequent version, as published by the Free Software Foundation Inc.
+
+GNU Affero GPL: means the GNU Affero General Public License version 3 or
+any subsequent version, as published by the Free Software Foundation Inc.
+
+EUPL: means the European Union Public License version 1.1 or any
+subsequent version, as published by the European Commission.
+
+Parties: mean both the Licensee and the Licensor.
+
+These expressions may be used both in singular and plural form.
+
+
+    Article 2 - PURPOSE
+
+The purpose of the Agreement is the grant by the Licensor to the
+Licensee of a non-exclusive, transferable and worldwide license for the
+Software as set forth in Article 5 <#scope> hereinafter for the whole
+term of the protection granted by the rights over said Software.
+
+
+    Article 3 - ACCEPTANCE
+
+3.1 The Licensee shall be deemed as having accepted the terms and
+conditions of this Agreement upon the occurrence of the first of the
+following events:
+
+  * (i) loading the Software by any or all means, notably, by
+    downloading from a remote server, or by loading from a physical medium;
+  * (ii) the first time the Licensee exercises any of the rights granted
+    hereunder.
+
+3.2 One copy of the Agreement, containing a notice relating to the
+characteristics of the Software, to the limited warranty, and to the
+fact that its use is restricted to experienced users has been provided
+to the Licensee prior to its acceptance as set forth in Article 3.1
+<#accepting> hereinabove, and the Licensee hereby acknowledges that it
+has read and understood it.
+
+
+    Article 4 - EFFECTIVE DATE AND TERM
+
+
+      4.1 EFFECTIVE DATE
+
+The Agreement shall become effective on the date when it is accepted by
+the Licensee as set forth in Article 3.1 <#accepting>.
+
+
+      4.2 TERM
+
+The Agreement shall remain in force for the entire legal term of
+protection of the economic rights over the Software.
+
+
+    Article 5 - SCOPE OF RIGHTS GRANTED
+
+The Licensor hereby grants to the Licensee, who accepts, the following
+rights over the Software for any or all use, and for the term of the
+Agreement, on the basis of the terms and conditions set forth hereinafter.
+
+Besides, if the Licensor owns or comes to own one or more patents
+protecting all or part of the functions of the Software or of its
+components, the Licensor undertakes not to enforce the rights granted by
+these patents against successive Licensees using, exploiting or
+modifying the Software. If these patents are transferred, the Licensor
+undertakes to have the transferees subscribe to the obligations set
+forth in this paragraph.
+
+
+      5.1 RIGHT OF USE
+
+The Licensee is authorized to use the Software, without any limitation
+as to its fields of application, with it being hereinafter specified
+that this comprises:
+
+ 1. permanent or temporary reproduction of all or part of the Software
+    by any or all means and in any or all form.
+
+ 2. loading, displaying, running, or storing the Software on any or all
+    medium.
+
+ 3. entitlement to observe, study or test its operation so as to
+    determine the ideas and principles behind any or all constituent
+    elements of said Software. This shall apply when the Licensee
+    carries out any or all loading, displaying, running, transmission or
+    storage operation as regards the Software, that it is entitled to
+    carry out hereunder.
+
+
+      5.2 ENTITLEMENT TO MAKE CONTRIBUTIONS
+
+The right to make Contributions includes the right to translate, adapt,
+arrange, or make any or all modifications to the Software, and the right
+to reproduce the resulting software.
+
+The Licensee is authorized to make any or all Contributions to the
+Software provided that it includes an explicit notice that it is the
+author of said Contribution and indicates the date of the creation thereof.
+
+
+      5.3 RIGHT OF DISTRIBUTION
+
+In particular, the right of distribution includes the right to publish,
+transmit and communicate the Software to the general public on any or
+all medium, and by any or all means, and the right to market, either in
+consideration of a fee, or free of charge, one or more copies of the
+Software by any means.
+
+The Licensee is further authorized to distribute copies of the modified
+or unmodified Software to third parties according to the terms and
+conditions set forth hereinafter.
+
+
+        5.3.1 DISTRIBUTION OF SOFTWARE WITHOUT MODIFICATION
+
+The Licensee is authorized to distribute true copies of the Software in
+Source Code or Object Code form, provided that said distribution
+complies with all the provisions of the Agreement and is accompanied by:
+
+ 1. a copy of the Agreement,
+
+ 2. a notice relating to the limitation of both the Licensor's warranty
+    and liability as set forth in Articles 8 and 9,
+
+and that, in the event that only the Object Code of the Software is
+redistributed, the Licensee allows effective access to the full Source
+Code of the Software for a period of at least three years from the
+distribution of the Software, it being understood that the additional
+acquisition cost of the Source Code shall not exceed the cost of the
+data transfer.
+
+
+        5.3.2 DISTRIBUTION OF MODIFIED SOFTWARE
+
+When the Licensee makes a Contribution to the Software, the terms and
+conditions for the distribution of the resulting Modified Software
+become subject to all the provisions of this Agreement.
+
+The Licensee is authorized to distribute the Modified Software, in
+source code or object code form, provided that said distribution
+complies with all the provisions of the Agreement and is accompanied by:
+
+ 1. a copy of the Agreement,
+
+ 2. a notice relating to the limitation of both the Licensor's warranty
+    and liability as set forth in Articles 8 and 9,
+
+and, in the event that only the object code of the Modified Software is
+redistributed,
+
+ 3. a note stating the conditions of effective access to the full source
+    code of the Modified Software for a period of at least three years
+    from the distribution of the Modified Software, it being understood
+    that the additional acquisition cost of the source code shall not
+    exceed the cost of the data transfer.
+
+
+        5.3.3 DISTRIBUTION OF EXTERNAL MODULES
+
+When the Licensee has developed an External Module, the terms and
+conditions of this Agreement do not apply to said External Module, that
+may be distributed under a separate license agreement.
+
+
+        5.3.4 COMPATIBILITY WITH OTHER LICENSES
+
+The Licensee can include a code that is subject to the provisions of one
+of the versions of the GNU GPL, GNU Affero GPL and/or EUPL in the
+Modified or unmodified Software, and distribute that entire code under
+the terms of the same version of the GNU GPL, GNU Affero GPL and/or EUPL.
+
+The Licensee can include the Modified or unmodified Software in a code
+that is subject to the provisions of one of the versions of the GNU GPL,
+GNU Affero GPL and/or EUPL and distribute that entire code under the
+terms of the same version of the GNU GPL, GNU Affero GPL and/or EUPL.
+
+
+    Article 6 - INTELLECTUAL PROPERTY
+
+
+      6.1 OVER THE INITIAL SOFTWARE
+
+The Holder owns the economic rights over the Initial Software. Any or
+all use of the Initial Software is subject to compliance with the terms
+and conditions under which the Holder has elected to distribute its work
+and no one shall be entitled to modify the terms and conditions for the
+distribution of said Initial Software.
+
+The Holder undertakes that the Initial Software will remain ruled at
+least by this Agreement, for the duration set forth in Article 4.2 <#term>.
+
+
+      6.2 OVER THE CONTRIBUTIONS
+
+The Licensee who develops a Contribution is the owner of the
+intellectual property rights over this Contribution as defined by
+applicable law.
+
+
+      6.3 OVER THE EXTERNAL MODULES
+
+The Licensee who develops an External Module is the owner of the
+intellectual property rights over this External Module as defined by
+applicable law and is free to choose the type of agreement that shall
+govern its distribution.
+
+
+      6.4 JOINT PROVISIONS
+
+The Licensee expressly undertakes:
+
+ 1. not to remove, or modify, in any manner, the intellectual property
+    notices attached to the Software;
+
+ 2. to reproduce said notices, in an identical manner, in the copies of
+    the Software modified or not.
+
+The Licensee undertakes not to directly or indirectly infringe the
+intellectual property rights on the Software of the Holder and/or
+Contributors, and to take, where applicable, vis-à-vis its staff, any
+and all measures required to ensure respect of said intellectual
+property rights of the Holder and/or Contributors.
+
+
+    Article 7 - RELATED SERVICES
+
+7.1 Under no circumstances shall the Agreement oblige the Licensor to
+provide technical assistance or maintenance services for the Software.
+
+However, the Licensor is entitled to offer this type of services. The
+terms and conditions of such technical assistance, and/or such
+maintenance, shall be set forth in a separate instrument. Only the
+Licensor offering said maintenance and/or technical assistance services
+shall incur liability therefor.
+
+7.2 Similarly, any Licensor is entitled to offer to its licensees, under
+its sole responsibility, a warranty, that shall only be binding upon
+itself, for the redistribution of the Software and/or the Modified
+Software, under terms and conditions that it is free to decide. Said
+warranty, and the financial terms and conditions of its application,
+shall be subject of a separate instrument executed between the Licensor
+and the Licensee.
+
+
+    Article 8 - LIABILITY
+
+8.1 Subject to the provisions of Article 8.2, the Licensee shall be
+entitled to claim compensation for any direct loss it may have suffered
+from the Software as a result of a fault on the part of the relevant
+Licensor, subject to providing evidence thereof.
+
+8.2 The Licensor's liability is limited to the commitments made under
+this Agreement and shall not be incurred as a result of in particular:
+(i) loss due the Licensee's total or partial failure to fulfill its
+obligations, (ii) direct or consequential loss that is suffered by the
+Licensee due to the use or performance of the Software, and (iii) more
+generally, any consequential loss. In particular the Parties expressly
+agree that any or all pecuniary or business loss (i.e. loss of data,
+loss of profits, operating loss, loss of customers or orders,
+opportunity cost, any disturbance to business activities) or any or all
+legal proceedings instituted against the Licensee by a third party,
+shall constitute consequential loss and shall not provide entitlement to
+any or all compensation from the Licensor.
+
+
+    Article 9 - WARRANTY
+
+9.1 The Licensee acknowledges that the scientific and technical
+state-of-the-art when the Software was distributed did not enable all
+possible uses to be tested and verified, nor for the presence of
+possible defects to be detected. In this respect, the Licensee's
+attention has been drawn to the risks associated with loading, using,
+modifying and/or developing and reproducing the Software which are
+reserved for experienced users.
+
+The Licensee shall be responsible for verifying, by any or all means,
+the suitability of the product for its requirements, its good working
+order, and for ensuring that it shall not cause damage to either persons
+or properties.
+
+9.2 The Licensor hereby represents, in good faith, that it is entitled
+to grant all the rights over the Software (including in particular the
+rights set forth in Article 5 <#scope>).
+
+9.3 The Licensee acknowledges that the Software is supplied "as is" by
+the Licensor without any other express or tacit warranty, other than
+that provided for in Article 9.2 <#good-faith> and, in particular,
+without any warranty as to its commercial value, its secured, safe,
+innovative or relevant nature.
+
+Specifically, the Licensor does not warrant that the Software is free
+from any error, that it will operate without interruption, that it will
+be compatible with the Licensee's own equipment and software
+configuration, nor that it will meet the Licensee's requirements.
+
+9.4 The Licensor does not either expressly or tacitly warrant that the
+Software does not infringe any third party intellectual property right
+relating to a patent, software or any other property right. Therefore,
+the Licensor disclaims any and all liability towards the Licensee
+arising out of any or all proceedings for infringement that may be
+instituted in respect of the use, modification and redistribution of the
+Software. Nevertheless, should such proceedings be instituted against
+the Licensee, the Licensor shall provide it with technical and legal
+expertise for its defense. Such technical and legal expertise shall be
+decided on a case-by-case basis between the relevant Licensor and the
+Licensee pursuant to a memorandum of understanding. The Licensor
+disclaims any and all liability as regards the Licensee's use of the
+name of the Software. No warranty is given as regards the existence of
+prior rights over the name of the Software or as regards the existence
+of a trademark.
+
+
+    Article 10 - TERMINATION
+
+10.1 In the event of a breach by the Licensee of its obligations
+hereunder, the Licensor may automatically terminate this Agreement
+thirty (30) days after notice has been sent to the Licensee and has
+remained ineffective.
+
+10.2 A Licensee whose Agreement is terminated shall no longer be
+authorized to use, modify or distribute the Software. However, any
+licenses that it may have granted prior to termination of the Agreement
+shall remain valid subject to their having been granted in compliance
+with the terms and conditions hereof.
+
+
+    Article 11 - MISCELLANEOUS
+
+
+      11.1 EXCUSABLE EVENTS
+
+Neither Party shall be liable for any or all delay, or failure to
+perform the Agreement, that may be attributable to an event of force
+majeure, an act of God or an outside cause, such as defective
+functioning or interruptions of the electricity or telecommunications
+networks, network paralysis following a virus attack, intervention by
+government authorities, natural disasters, water damage, earthquakes,
+fire, explosions, strikes and labor unrest, war, etc.
+
+11.2 Any failure by either Party, on one or more occasions, to invoke
+one or more of the provisions hereof, shall under no circumstances be
+interpreted as being a waiver by the interested Party of its right to
+invoke said provision(s) subsequently.
+
+11.3 The Agreement cancels and replaces any or all previous agreements,
+whether written or oral, between the Parties and having the same
+purpose, and constitutes the entirety of the agreement between said
+Parties concerning said purpose. No supplement or modification to the
+terms and conditions hereof shall be effective as between the Parties
+unless it is made in writing and signed by their duly authorized
+representatives.
+
+11.4 In the event that one or more of the provisions hereof were to
+conflict with a current or future applicable act or legislative text,
+said act or legislative text shall prevail, and the Parties shall make
+the necessary amendments so as to comply with said act or legislative
+text. All other provisions shall remain effective. Similarly, invalidity
+of a provision of the Agreement, for any reason whatsoever, shall not
+cause the Agreement as a whole to be invalid.
+
+
+      11.5 LANGUAGE
+
+The Agreement is drafted in both French and English and both versions
+are deemed authentic.
+
+
+    Article 12 - NEW VERSIONS OF THE AGREEMENT
+
+12.1 Any person is authorized to duplicate and distribute copies of this
+Agreement.
+
+12.2 So as to ensure coherence, the wording of this Agreement is
+protected and may only be modified by the authors of the License, who
+reserve the right to periodically publish updates or new versions of the
+Agreement, each with a separate number. These subsequent versions may
+address new issues encountered by Free Software.
+
+12.3 Any Software distributed under a given version of the Agreement may
+only be subsequently distributed under the same version of the Agreement
+or a subsequent version, subject to the provisions of Article 5.3.4
+<#compatibility>.
+
+
+    Article 13 - GOVERNING LAW AND JURISDICTION
+
+13.1 The Agreement is governed by French law. The Parties agree to
+endeavor to seek an amicable solution to any disagreements or disputes
+that may arise during the performance of the Agreement.
+
+13.2 Failing an amicable solution within two (2) months as from their
+occurrence, and unless emergency proceedings are necessary, the
+disagreements or disputes shall be referred to the Paris Courts having
+jurisdiction, by the more diligent Party.

distutils.ext/obidistutils/command/build_ext.py

@@ -7,107 +7,109 @@ Created on 13 fevr. 2014
 from distutils import log
 import os
 
-from Cython.Distutils import build_ext  as ori_build_ext  # @UnresolvedImport
-
-from Cython.Compiler import Options as cython_options  # @UnresolvedImport
 
 from distutils.errors import DistutilsSetupError
 
-class build_ext(ori_build_ext):
-
+try:
+    from Cython.Distutils import build_ext  as ori_build_ext  # @UnresolvedImport
+    from Cython.Compiler import Options as cython_options  # @UnresolvedImport
+    class build_ext(ori_build_ext):
     
-    def modifyDocScripts(self):
-        build_dir_file=open("doc/sphinx/build_dir.txt","w")
-        print(self.build_lib,file=build_dir_file)
-        build_dir_file.close()
         
-    def initialize_options(self):
-        ori_build_ext.initialize_options(self)  # @UndefinedVariable
-        self.littlebigman = None
-        self.built_files = None
-
-    
-    def finalize_options(self):
-        ori_build_ext.finalize_options(self)  # @UndefinedVariable
-
-        self.set_undefined_options('littlebigman',
-                                   ('littlebigman',  'littlebigman'))
-        
-        self.set_undefined_options('build_files',
-                                   ('files',  'built_files'))
-
-        self.cython_c_in_temp = 1
-                   
-        if self.littlebigman =='-DLITTLE_END':
-            if self.define is None:
-                self.define=[('LITTLE_END',None)]
-            else:
-                self.define.append('LITTLE_END',None)
-        
-    def substitute_sources(self,exe_name,sources):
-        """
-        Substitutes source file name starting by an @ by the actual
-        name of the built file (see --> build_files)
-        """
-        sources = list(sources)
-        for i in range(len(sources)):
-            message = "%s :-> %s" % (exe_name,sources[i])
-            if sources[i][0]=='@':
-                try:
-                    filename = self.built_files[sources[i][1:]]
-                except KeyError:
-                    tmpfilename = os.path.join(self.build_temp,sources[i][1:])
-                    if os.path.isfile       (tmpfilename):
-                        filename = tmpfilename
-                    else:
-                        raise DistutilsSetupError(
-                             'The %s filename declared in the source '
-                             'files of the program %s have not been '
-                             'built by the installation process' % (sources[i],
-                                                                     exe_name))
-                sources[i]=filename
-                log.info("%s changed to %s",message,filename)
-            else:
-                log.info("%s ok",message)
-
-        return sources
-
-    def build_extensions(self):
-        # First, sanity-check the 'extensions' list
-        
-        for ext in self.extensions:
-            ext.sources = self.substitute_sources(ext.name,ext.sources)
+        def modifyDocScripts(self):
+            build_dir_file=open("doc/sphinx/build_dir.txt","w")
+            print(self.build_lib,file=build_dir_file)
+            build_dir_file.close()
             
-        self.check_extensions_list(self.extensions)
-
-        for ext in self.extensions:
-            log.info("%s :-> %s",ext.name,ext.sources)
-            ext.sources = self.cython_sources(ext.sources, ext)
-            self.build_extension(ext)
-
-        
-    def run(self):
-        self.modifyDocScripts()
-
-        for cmd_name in self.get_sub_commands():
-            self.run_command(cmd_name)
-
-        cython_options.annotate = True
-        ori_build_ext.run(self)  # @UndefinedVariable
-        
-
-    def has_files(self):
-        return self.distribution.has_files()
-
-    def has_executables(self):
-        return self.distribution.has_executables()
+        def initialize_options(self):
+            ori_build_ext.initialize_options(self)  # @UndefinedVariable
+            self.littlebigman = None
+            self.built_files = None
     
-    sub_commands = [('build_files',has_files),
-                    ('build_cexe', has_executables)
-                    ] + \
-                   ori_build_ext.sub_commands
-
         
+        def finalize_options(self):
+            ori_build_ext.finalize_options(self)  # @UndefinedVariable
+    
+            self.set_undefined_options('littlebigman',
+                                       ('littlebigman',  'littlebigman'))
+            
+            self.set_undefined_options('build_files',
+                                       ('files',  'built_files'))
+    
+            self.cython_c_in_temp = 1
+                       
+            if self.littlebigman =='-DLITTLE_END':
+                if self.define is None:
+                    self.define=[('LITTLE_END',None)]
+                else:
+                    self.define.append('LITTLE_END',None)
+            
+        def substitute_sources(self,exe_name,sources):
+            """
+            Substitutes source file name starting by an @ by the actual
+            name of the built file (see --> build_files)
+            """
+            sources = list(sources)
+            for i in range(len(sources)):
+                message = "%s :-> %s" % (exe_name,sources[i])
+                if sources[i][0]=='@':
+                    try:
+                        filename = self.built_files[sources[i][1:]]
+                    except KeyError:
+                        tmpfilename = os.path.join(self.build_temp,sources[i][1:])
+                        if os.path.isfile       (tmpfilename):
+                            filename = tmpfilename
+                        else:
+                            raise DistutilsSetupError(
+                                 'The %s filename declared in the source '
+                                 'files of the program %s have not been '
+                                 'built by the installation process' % (sources[i],
+                                                                         exe_name))
+                    sources[i]=filename
+                    log.info("%s changed to %s",message,filename)
+                else:
+                    log.info("%s ok",message)
+    
+            return sources
+    
+        def build_extensions(self):
+            # First, sanity-check the 'extensions' list
+            
+            for ext in self.extensions:
+                ext.sources = self.substitute_sources(ext.name,ext.sources)
+                
+            self.check_extensions_list(self.extensions)
+    
+            for ext in self.extensions:
+                log.info("%s :-> %s",ext.name,ext.sources)
+                ext.sources = self.cython_sources(ext.sources, ext)
+                self.build_extension(ext)
+    
+            
+        def run(self):
+            self.modifyDocScripts()
+    
+            for cmd_name in self.get_sub_commands():
+                self.run_command(cmd_name)
+    
+            cython_options.annotate = True
+            ori_build_ext.run(self)  # @UndefinedVariable
+            
+    
+        def has_files(self):
+            return self.distribution.has_files()
+    
+        def has_executables(self):
+            return self.distribution.has_executables()
+        
+        sub_commands = [('build_files',has_files),
+                        ('build_cexe', has_executables)
+                        ] + \
+                       ori_build_ext.sub_commands
+
+except ImportError:
+    from distutils.command import build_ext  # @UnusedImport
+           
 
 
         

distutils.ext/obidistutils/serenity/bootstrappip.py

@@ -0,0 +1,36 @@
+'''
+Created on 22 janv. 2016
+
+@author: coissac
+'''
+
+import sys
+from urllib import request
+import os.path
+
+from obidistutils.serenity.util import get_serenity_dir
+from obidistutils.serenity.rerun import rerun_with_anothe_python
+from obidistutils.serenity.checkpython import is_a_virtualenv_python
+
+getpipurl="https://bootstrap.pypa.io/get-pip.py"
+
+def bootstrap():
+
+    getpipfile=os.path.join(get_serenity_dir(),"get-pip.py")
+    
+    with request.urlopen(getpipurl) as getpip:
+        with open(getpipfile,"wb") as out:
+            for l in getpip:
+                out.write(l)
+                
+    python = sys.executable
+    
+    if is_a_virtualenv_python():
+        command= "%s %s" % (python,getpipfile)        
+    else:
+        command= "%s %s --user" % (python,getpipfile)
+    
+    os.system(command)
+    
+    rerun_with_anothe_python(python)
+    

distutils.ext/obidistutils/serenity/checkpackage.py

@@ -7,8 +7,13 @@ Created on 2 oct. 2014
 import re
 import os
 
-import pip                                              # @UnresolvedImport
-from pip.utils import get_installed_distributions       # @UnresolvedImport
+try:
+    import pip                                              # @UnresolvedImport
+    from pip.utils import get_installed_distributions       # @UnresolvedImport
+except ImportError:
+    from .bootstrappip import bootstrap
+    bootstrap()
+
 from distutils.version import StrictVersion             # @UnusedImport
 from distutils.errors import DistutilsError
 from distutils import log

distutils.ext/obidistutils/serenity/virtual.py

@@ -59,7 +59,7 @@ def serenity_virtualenv(envname,package,version,minversion='3.4',maxversion=None
                     clear=True, 
                     symlinks=False, 
                     with_pip=True)
-        
+                
         # check the newly created virtualenv
         return serenity_virtualenv(envname,package,version)
     

python/obi.py

@@ -16,30 +16,22 @@ It defines classes_and_methods
 @deffield    updated: Updated
 '''
 
-import sys
-import pkgutil
-import argparse
-import logging
-import json
 
-default_config = {
+default_config = { 'software'       : "The OBITools",
+                   'log'            : False,
+                   'loglevel'       : 'INFO',
+                   'progress'       : True,
+                   'defaultdms'     : None
+                  }
 
-    'obi' :        { 'log'            : True,
-                     'loglevel'       : 'INFO',
-                     'version'        : False,
-                     'progress'       : True
-                   }
-                 
-}
+root_config_name='obi'
 
-
-
-from obitools3 import command
+from obitools3.apps.config import getConfiguration     # @UnresolvedImport
 from obitools3.version import version
 
-__all__ = []
+__all__     = []
 __version__ = version
-__date__ = '2014-09-28'
+__date__    = '2014-09-28'
 __updated__ = '2014-09-28'
 
 DEBUG = 1
@@ -47,182 +39,12 @@ TESTRUN = 0
 PROFILE = 0
 
 
-
-def loadCommand(name,loader):
-    '''
-    Load a command module from its name and an ImpLoader
-    
-    This function is for internal use
-    
-    @param name:   name of the module
-    @type name: str 
-    @param loader: the module loader
-    @type loader: ImpLoader
-    
-    @return the loaded module
-    @rtype: module 
-    '''
-    
-    module = loader.find_module(name).load_module(name)
-    return module
-
-def getCommandsList():
-    '''
-    Returns the list of sub-commands available to the main `obi` command
-    
-    @return: a dict instance with key corresponding to each command and
-             value corresponding to the module
-             
-    @rtype: dict
-    '''
-    cmds = dict((x[1],loadCommand(x[1],x[0])) 
-                for x in pkgutil.iter_modules(command.__path__) 
-                if not x[2])
-    return cmds
-
-def getLogger(config):
-    '''
-    Returns the logger as defined by the command line option
-    or by the config file
-    :param config:
-    '''
- 
-    output = config['obi']['outputfilename'] 
-    level  = config['obi']['loglevel'] 
-    logfile= config['obi']['log'] 
-    
-    rootlogger   = logging.getLogger()
-    logFormatter = logging.Formatter("%(asctime)s [%(levelname)-5.5s]  %(message)s")
-
-    stderrHandler = logging.StreamHandler(sys.stderr)
-    stderrHandler.setFormatter(logFormatter)
-
-    rootlogger.addHandler(stderrHandler)
-    
-    if logfile:
-        fileHandler = logging.FileHandler("%s.log" % output)
-        fileHandler.setFormatter(logFormatter)
-        rootlogger.addHandler(fileHandler)
-    
-    try:
-        loglevel = getattr(logging, level) 
-    except:
-        loglevel = logging.INFO
-        
-    rootlogger.setLevel(loglevel)
-    
-    config['obi']['logger']=rootlogger
-    
-    return rootlogger
-
-
-class ObiParser(argparse.ArgumentParser): 
-    def error(self, message):
-        sys.stderr.write('error: %s\n' % message)
-        self.print_help()
-        sys.exit(2)
-
-def buildArgumentParser():
-    parser = ObiParser()
-    
-    parser.add_argument('--version',   dest='obi:version', 
-                                       action='store_true', 
-                                       default=False, 
-                        help='Print the version of the OBITools')
-
-    parser.add_argument('--no-log',    dest='obi:log', 
-                                       action='store_false', 
-                                       default=None, 
-                        help='Do not create a logfile for the data analyze')
-
-    parser.add_argument('--no-progress', dest='obi:progress', 
-                                       action='store_false', 
-                                       default=None, 
-                        help='Do not print the progress bar during analyzes')
-
-    subparsers = parser.add_subparsers(title='subcommands',
-                                       description='valid subcommands',
-                                       help='additional help')
-    
-    commands = getCommandsList()
-    
-    for c in commands:
-        module = commands[c]
-        
-        if hasattr(module, "run"):
-            if hasattr(module, "__title__"):
-                sub = subparsers.add_parser(c,help=module.__title__)
-            else:
-                sub = subparsers.add_parser(c)
-    
-            if hasattr(module, "addOptions"):
-                module.addOptions(sub)
-            
-            sub.set_defaults(**{'obi:module' : module})
-                        
-    return parser
-
-def buildDefaultConfiguration():
-    global default_config
-    
-    commands = getCommandsList()
-    
-    for c in commands:
-        module = commands[c]
-    
-        assert hasattr(module, "run")
-        
-        if hasattr(module, 'default_config'):
-            default_config[c]=module.default_config
-        else:
-            default_config[c]={}
-                        
-    return default_config
-        
-
-def getConfiguration():
-    global default_config
-
-    if '__done__' in default_config:
-        return default_config
-    
-    parser = buildArgumentParser()
-    options = vars(parser.parse_args())
-    
-    config =  buildDefaultConfiguration()
-    
-    
-    for k in options:
-        section,key = k.split(':')
-        s = config[section]
-        if options[k] is not None:
-            s[key]=options[k]
-            
-    if config['obi']['version']:
-        print("The OBITools - Version %s" % __version__)
-        sys.exit(0)
-
-    if not 'module' in config['obi']:
-        print('\nError: No obi command specified',file=sys.stderr)
-        parser.print_help()
-        sys.exit(2)
-        
-            
-    if config['obi']['outputfilename'] is None:
-        config['obi']['outputfilename']=config['obi']['indexfilename']
-        
-    getLogger(config)
-    
-    config['__done__']=True
-            
-    return config
-    
-
 if __name__ =="__main__":
     
-    config = getConfiguration()    
+    config = getConfiguration(root_config_name,
+                              default_config)    
                 
-    config['obi']['module'].run(config)
+    config[root_config_name]['module'].run(config)
     
     
     

python/obitools3/apps/arguments.pxd

@@ -0,0 +1,3 @@
+#cython: language_level=3
+
+cpdef buildArgumentParser(str configname, str softname)

python/obitools3/apps/arguments.pyx

@@ -0,0 +1,61 @@
+#cython: language_level=3
+
+'''
+Created on 27 mars 2016
+
+@author: coissac
+'''
+
+import argparse
+import sys
+
+
+from .command import getCommandsList
+
+class ObiParser(argparse.ArgumentParser): 
+    def error(self, message):
+        sys.stderr.write('error: %s\n' % message)
+        self.print_help()
+        sys.exit(2)
+
+cpdef buildArgumentParser(str configname, 
+                          str softname):
+    parser = ObiParser()
+    
+    parser.add_argument('--version',   dest='%s:version' % configname, 
+                                       action='store_true', 
+                                       default=False, 
+                        help='Print the version of %s' % softname)
+
+    parser.add_argument('--log',       dest='%s:log' % configname, 
+                                       action='store',
+                                       type=str,
+                                       default=None, 
+                        help='Create a logfile')
+
+    parser.add_argument('--no-progress', dest='%s:progress' % configname, 
+                                       action='store_false', 
+                                       default=None, 
+                        help='Do not print the progress bar during analyzes')
+
+    subparsers = parser.add_subparsers(title='subcommands',
+                                       description='valid subcommands',
+                                       help='additional help')
+    
+    commands = getCommandsList()
+    
+    for c in commands:
+        module = commands[c]
+        
+        if hasattr(module, "run"):
+            if hasattr(module, "__title__"):
+                sub = subparsers.add_parser(c,help=module.__title__)
+            else:
+                sub = subparsers.add_parser(c)
+    
+            if hasattr(module, "addOptions"):
+                module.addOptions(sub)
+            
+            sub.set_defaults(**{'%s:module'  % configname : module})
+                        
+    return parser

python/obitools3/apps/command.pxd

@@ -0,0 +1,3 @@
+#cython: language_level=3
+
+cdef object loadCommand(str name,loader)

python/obitools3/apps/command.pyx

@@ -0,0 +1,44 @@
+#cython: language_level=3
+
+'''
+Created on 27 mars 2016
+
+@author: coissac
+'''
+
+import pkgutil
+
+from obitools3 import commands
+
+cdef object loadCommand(str name,loader):
+    '''
+    Load a command module from its name and an ImpLoader
+    
+    This function is for internal use
+    
+    @param name:   name of the module
+    @type name: str 
+    @param loader: the module loader
+    @type loader: ImpLoader
+    
+    @return the loaded module
+    @rtype: module 
+    '''
+    
+    module = loader.find_module(name).load_module(name)
+    return module
+
+def getCommandsList():
+    '''
+    Returns the list of sub-commands available to the main `obi` command
+    
+    @return: a dict instance with key corresponding to each command and
+             value corresponding to the module
+             
+    @rtype: dict
+    '''
+    
+    cdef dict cmds = dict((x[1],loadCommand(x[1],x[0])) 
+                           for x in pkgutil.iter_modules(commands.__path__) 
+                           if not x[2])
+    return cmds

python/obitools3/apps/config.pxd

@@ -0,0 +1,10 @@
+#cython: language_level=3
+
+cpdef str setRootConfigName(str rootname)
+cpdef str getRootConfigName()
+
+cdef dict buildDefaultConfiguration(str root_config_name,
+                                    dict  config)
+
+cpdef dict getConfiguration(str root_config_name=?,
+                                    dict  config=?)

python/obitools3/apps/config.pyx

@@ -0,0 +1,103 @@
+#cython: language_level=3
+
+'''
+Created on 27 mars 2016
+
+@author: coissac
+'''
+
+import sys
+
+from .command   import  getCommandsList
+from .logging   cimport getLogger
+from .arguments cimport buildArgumentParser
+
+from ..version import version
+from _curses import version
+
+cdef dict __default_config__ = {}
+
+
+cpdef str setRootConfigName(str rootname):
+    global __default_config__
+    if '__root_config__' in __default_config__:
+        if __default_config__["__root_config__"] in __default_config__:
+            __default_config__[rootname]=__default_config__[__default_config__["__root_config__"]]
+            del __default_config__[__default_config__["__root_config__"]]
+    __default_config__['__root_config__']=rootname
+    return rootname
+
+cpdef str getRootConfigName():
+    global __default_config__
+    return __default_config__.get('__root_config__',None)
+
+cdef dict buildDefaultConfiguration(str root_config_name,
+                                    dict  config):
+    global __default_config__
+    
+    __default_config__.clear()
+    setRootConfigName(root_config_name)    
+
+    __default_config__[root_config_name]=config
+    
+    config['version']=version
+    
+    commands = getCommandsList()
+    
+    for c in commands:
+        module = commands[c]
+    
+        assert hasattr(module, "run")
+        
+        if hasattr(module, 'default_config'):
+            __default_config__[c]=module.default_config
+        else:
+            __default_config__[c]={}
+                        
+    return __default_config__
+        
+
+cpdef dict getConfiguration(str root_config_name="__default__",
+                            dict  config={}):
+    global __default_config__
+
+    if '__done__' in __default_config__:
+        return __default_config__
+    
+    if root_config_name=="__default__":
+        raise RuntimeError("No root_config_name specified")
+    
+    if not config:
+        raise RuntimeError("Base configuration is empty")
+    
+    
+    
+    config =  buildDefaultConfiguration(root_config_name,
+                                        config)
+    
+    parser = buildArgumentParser(root_config_name,
+                                 config[root_config_name]['software'])
+    
+    options = vars(parser.parse_args())
+
+    if options['%s:version' % root_config_name]:
+        print("%s - Version %s" % (config[root_config_name]['software'],
+                                   config[root_config_name]['version']))
+        sys.exit(0)
+    
+    for k in options:
+        section,key = k.split(':')
+        s = config[section]
+        if options[k] is not None:
+            s[key]=options[k]
+            
+    if not 'module' in config[root_config_name]:
+        print('\nError: No command specified',file=sys.stderr)
+        parser.print_help()
+        sys.exit(2)
+                
+    getLogger(config)
+    
+    config['__done__']=True
+            
+    return config

python/obitools3/apps/logging.pxd

@@ -0,0 +1,3 @@
+#cython: language_level=3
+
+cpdef getLogger(dict config)

python/obitools3/apps/logging.pyx

@@ -0,0 +1,46 @@
+#cython: language_level=3
+
+'''
+Created on 27 mars 2016
+
+@author: coissac
+'''
+
+import logging
+import sys
+
+cpdef getLogger(dict config):
+    '''
+    Returns the logger as defined by the command line option
+    or by the config file
+    :param config:
+    '''
+    
+    root = config["__root_config__"]
+ 
+    level  = config[root]['loglevel'] 
+    logfile= config[root]['log'] 
+    
+    rootlogger   = logging.getLogger()
+    logFormatter = logging.Formatter("%(asctime)s [%(levelname)-5.5s]  %(message)s")
+
+    stderrHandler = logging.StreamHandler(sys.stderr)
+    stderrHandler.setFormatter(logFormatter)
+
+    rootlogger.addHandler(stderrHandler)
+    
+    if logfile:
+        fileHandler = logging.FileHandler(logfile)
+        fileHandler.setFormatter(logFormatter)
+        rootlogger.addHandler(fileHandler)
+    
+    try:
+        loglevel = getattr(logging, level) 
+    except:
+        loglevel = logging.INFO
+        
+    rootlogger.setLevel(loglevel)
+    
+    config[root]['logger']=rootlogger
+    
+    return rootlogger

python/obitools3/apps/progress.pxd

@@ -0,0 +1,65 @@
+#cython: language_level=3
+
+from ..utils cimport str2bytes
+
+cdef extern from "stdio.h":
+    struct FILE
+    int fprintf(FILE *stream, char *format, ...)
+    FILE* stderr
+    ctypedef unsigned int off_t "unsigned long long"
+    
+cdef extern from "unistd.h":
+    int fsync(int fd);
+
+cdef extern from "time.h":
+    struct tm :
+        int tm_yday 
+        int tm_hour
+        int tm_min
+        int tm_sec
+
+    enum: CLOCKS_PER_SEC
+    
+    ctypedef int time_t
+    ctypedef int clock_t
+    ctypedef int suseconds_t
+    
+    struct timeval:
+        time_t      tv_sec     #  seconds */
+        suseconds_t tv_usec    #  microseconds */
+
+
+    struct timezone :
+        int tz_minuteswest;    # minutes west of Greenwich
+        int tz_dsttime;        # type of DST correction 
+
+        
+    int gettimeofday(timeval *tv, timezone *tz)
+
+
+    tm *gmtime_r(time_t *clock, tm *result)
+    time_t time(time_t *tloc)
+    clock_t clock()
+
+cdef class ProgressBar:
+    cdef off_t   maxi
+    cdef clock_t starttime
+    cdef clock_t lasttime
+    cdef clock_t tickcount
+    cdef int freq
+    cdef int cycle
+    cdef int arrow
+    cdef int lastlog
+    cdef bint ontty
+    cdef int fd
+    
+    cdef bytes head
+    cdef char *chead
+    
+    cdef object logger
+
+    cdef char  *wheel
+    cdef char  *spaces
+    cdef char*  diese 
+    
+    cdef clock_t clock(self)

python/obitools3/apps/progress.pyx

@@ -0,0 +1,138 @@
+#cython: language_level=3
+
+'''
+Created on 27 mars 2016
+
+@author: coissac
+'''
+
+import sys
+from ..utils cimport bytes2str
+
+cdef class ProgressBar:
+    cdef clock_t clock(self):
+        cdef clock_t t
+        cdef timeval tp
+        cdef clock_t s
+        
+        <void> gettimeofday(&tp,NULL)
+        s = <clock_t> (<double> tp.tv_usec * 1.e-6 * <double> CLOCKS_PER_SEC)
+        t = tp.tv_sec * CLOCKS_PER_SEC + s 
+        
+        return t
+
+    def __init__(self,
+                 off_t maxi,
+                 dict  config,
+                 str head="",
+                 double seconde=0.1):
+        self.starttime = self.clock()
+        self.lasttime  = self.starttime
+        self.tickcount = <clock_t> (seconde * CLOCKS_PER_SEC)
+        self.freq      = 1
+        self.cycle     = 0
+        self.arrow     = 0
+        self.lastlog   = 0
+        
+        self.ontty = sys.stderr.isatty()
+        
+        if (maxi<=0):
+            maxi=1
+            
+        self.maxi = maxi
+        self.head = str2bytes(head)
+        self.chead= self.head 
+        
+        
+        self.logger=config[config["__root_config__"]]["logger"]
+        self.wheel =  '|/-\\'
+        self.spaces='          ' \
+                    '          ' \
+                    '          ' \
+                    '          ' \
+                    '          '
+        self.diese ='##########' \
+                    '##########' \
+                    '##########' \
+                    '##########' \
+                    '##########'  
+                           
+    def __call__(self,object pos):
+        cdef off_t    ipos
+        cdef clock_t  elapsed
+        cdef clock_t  newtime
+        cdef clock_t  delta
+        cdef clock_t  more 
+        cdef double   percent 
+        cdef tm remain
+        cdef int days,hour,minu,sec
+        cdef off_t fraction
+        cdef int twentyth
+        
+        self.cycle+=1
+    
+        if self.cycle % self.freq == 0:
+            self.cycle=1
+            newtime  = self.clock()
+            delta         = newtime - self.lasttime
+            self.lasttime = newtime
+            elapsed       = newtime - self.starttime
+#            print(" ",delta,elapsed,elapsed/CLOCKS_PER_SEC,self.tickcount)
+            
+            if   delta < self.tickcount / 5 :
+                self.freq*=2
+            elif delta > self.tickcount * 5 and self.freq>1:
+                self.freq/=2
+                
+            
+            if callable(pos):
+                ipos=pos()
+            else:
+                ipos=pos
+                
+            if ipos==0:
+                ipos=1                
+
+            percent = <double>ipos/<double>self.maxi
+            more = <time_t>((<double>elapsed / percent * (1. - percent))/CLOCKS_PER_SEC)
+            <void>gmtime_r(&more, &remain)
+            days  = remain.tm_yday 
+            hour  = remain.tm_hour
+            minu  = remain.tm_min
+            sec   = remain.tm_sec
+                
+            if self.ontty:
+                fraction=<int>(percent * 50.)
+                self.arrow=(self.arrow+1) % 4
+                self.diese[fraction]=0
+                self.spaces[50 - fraction]=0
+            
+                if days:
+                    <void>fprintf(stderr,b'\r%s %5.1f %% |%s%c%s] remain : %d days %02d:%02d:%02d',
+                                    self.chead,
+                                    percent*100,
+                                    self.diese,self.wheel[self.arrow],self.spaces,
+                                    days,hour,minu,sec)
+                else:
+                    <void>fprintf(stderr,b'\r%s %5.1f %% |%s%c%s] remain : %02d:%02d:%02d',
+                                    self.chead,
+                                    percent*100.,
+                                    self.diese,self.wheel[self.arrow],self.spaces,
+                                    hour,minu,sec)
+                self.diese[fraction]=b'#'
+                self.spaces[50 - fraction]=b' '
+
+            twentyth = int(percent * 20)
+            if twentyth != self.lastlog:
+                
+                if self.ontty:
+                    <void>fprintf(stderr,b'\n')
+                    
+                self.logger.info('%s %5.1f %% remain : %02d:%02d:%02d' % (
+                                        bytes2str(self.head),
+                                        percent*100.,
+                                        hour,minu,sec))
+                self.lastlog=twentyth
+        else:
+            self.cycle+=1
+

python/obitools3/commands/count.py

@@ -4,6 +4,9 @@ Created on 8 mars 2016
 @author: coissac
 '''
 
+from obitools3.apps.progress import ProgressBar  # @UnresolvedImport
+import time
+
 __title__="Counts sequences in a sequence set"
 
 
@@ -16,14 +19,14 @@ def addOptions(parser):
                                               default=None,
                         help='input data set' )
 
-    group=parser.add_option_group('Obicount specific options')
-    group.add_option('-s','--sequence',
+    group=parser.add_argument_group('Obicount specific options')
+    group.add_argument('-s','--sequence',
                              action="store_true", dest="count:sequence",
                              default=False,
                              help="Prints only the number of sequence records."
                              )
  
-    group.add_option('-a','--all',
+    group.add_argument('-a','--all',
                              action="store_true", dest="count:all",
                              default=False,
                              help="Prints only the total count of sequence records (if a sequence has no `count` attribute, its default count is 1) (default: False)."
@@ -33,4 +36,9 @@ def addOptions(parser):
     
 def run(config):
     # The code of my command
-    pass
+    pb = ProgressBar(1000,config,seconde=1)
+    
+    for i in range(1,1001):
+        pb(i)
+        time.sleep(0.01)
+        

python/obitools3/commands/import.pyx

@@ -0,0 +1,136 @@
+from obitools3.apps.progress cimport ProgressBar  # @UnresolvedImport
+from obitools3.files.universalopener cimport uopen
+from obitools3.parsers.fasta import fastaIterator
+from obitools3.parsers.fastq import fastqIterator
+from obitools3.obidms._obidms import OBIDMS
+
+
+import time
+
+__title__="Counts sequences in a sequence set"
+
+
+default_config = {   'destview'     : None,
+                     'skip'         : 0,
+                     'only'         : None,
+                     'skiperror'    : False,
+                     'seqinformat'  : None,
+                     'moltype'      : 'nuc',
+                     'filename'     : None
+                 }
+
+def addOptions(parser):
+    parser.add_argument(dest='import:filename',     
+                        metavar='<FILENAME>', 
+                        nargs='?', 
+                        default=None,
+                        help='sequence file name to be imported' )
+
+    group=parser.add_argument_group('obi import specific options')
+
+    group.add_argument('--default-dms','-d',
+                     action="store", dest="obi:defaultdms",
+                     metavar='<DMS NAME>',
+                     default=None,
+                     type=str,
+                     help="Name of the default DMS for reading and writing data")
+    
+    
+    group.add_argument('--destination-view','-v',
+                     action="store", dest="import:destview",
+                     metavar='<VIEW NAME>',
+                     default=None,
+                     type=str,
+                     required=True,
+                     help="Name of the default DMS for reading and writing data")
+
+    group=parser.add_argument_group('obi import specific options')
+    
+    group.add_argument('--skip',
+                     action="store", dest="import:skip",
+                     metavar='<N>',
+                     default=None,
+                     type=int,
+                     help="skip the N first sequences")
+
+    group.add_argument('--only',
+                     action="store", dest="import:only",
+                     metavar='<N>',
+                     default=None,
+                     type=int,
+                     help="treat only N sequences")
+
+    group.add_argument('--skip-on-error',
+                     action="store_true", dest="import:skiperror",
+                     default=None,
+                     help="Skip sequence entries with parse error")
+    
+    group.add_argument('--fasta',
+                     action="store_const", dest="import:seqinformat",
+                     default=None,
+                     const='fasta',
+                     help="Input file is in fasta nucleic format (including obitools fasta extentions)")
+
+    group.add_argument('--fastq',
+                     action="store_const", dest="import:seqinformat",
+                     default=None,
+                     const='fastq',
+                     help="Input file is in sanger fastq nucleic format (standard fastq)")
+
+    group.add_argument('--nuc',
+                     action="store_const", dest="import:moltype",
+                     default=None,
+                     const='nuc',
+                     help="Input file contains nucleic sequences")
+    
+    group.add_argument('--prot',
+                     action="store_const", dest="import:moltype",
+                     default=None,
+                     const='pep',
+                     help="Input file contains protein sequences")
+
+    
+    
+def run(config):
+    pb = ProgressBar(35000000,config,seconde=5)
+      
+    inputs = uopen(config['import']['filename'])
+    
+    if config['import']['seqinformat']=='fasta':
+        iseq = fastaIterator(inputs)
+        view_type="NUC_SEQS_VIEW"
+    elif config['import']['seqinformat']=='fastq':
+        iseq = fastqIterator(inputs)
+        view_type="NUC_SEQS_VIEW"
+    else:
+        raise RuntimeError('No file format specified')
+    
+    # Temporary way to handle NA values
+    #NA_list = ["nan"]
+    
+    # Create DMS
+    d = OBIDMS(config['obi']['defaultdms'])
+    
+    # Create view
+    view = d.new_view(config['import']['destview'], view_type=view_type)
+    
+    i = 0
+    for seq in iseq:
+        pb(i)
+        view[i].set_id(seq['id'])
+        view[i].set_definition(seq['definition'])
+        view[i].set_sequence(seq['sequence'])
+        for tag in seq['tags'] :
+            #print(tag, seq['tags'][tag])
+            #if seq['tags'][tag] not in NA_list :
+            view[i][tag] = seq['tags'][tag]
+        i+=1
+
+   #print(view)
+    print(view.__repr__())
+
+    view.save_and_close()
+    d.close()
+    
+    print("Done.")
+        

python/obitools3/files/linebuffer.pxd

@@ -0,0 +1,6 @@
+#cython: language_level=3
+
+
+cdef class LineBuffer:
+    cdef object fileobj
+    cdef int    size

python/obitools3/files/linebuffer.pyx

@@ -0,0 +1,23 @@
+#cython: language_level=3
+
+'''
+Created on 30 mars 2016
+
+@author: coissac
+'''
+
+cdef class LineBuffer:
+
+    def __init__(self,object fileobj,int size=100000000):
+        self.fileobj=fileobj 
+        self.size=size
+    
+    def __iter__(self):
+        cdef list buff = self.fileobj.readlines(self.size)
+        cdef str  l
+        
+        while buff:
+            for l in buff:
+                yield l
+            buff = self.fileobj.readlines(self.size)
+    

python/obitools3/files/uncompress.pxd

@@ -0,0 +1,17 @@
+#cython: language_level=3
+
+cdef class MagicKeyFile:
+    cdef object stream
+    cdef str    stream_mode
+    cdef object binary
+    cdef bytes  key 
+    cdef int    keylength 
+    cdef int    pos
+    
+    cpdef bytes read(self,int size=?)
+    cpdef int tell(self)
+
+    
+cdef class CompressedFile:
+    cdef object accessor
+    

python/obitools3/files/uncompress.pyx

@@ -0,0 +1,114 @@
+#cython: language_level=3
+
+'''
+Created on 28 mars 2016
+
+@author: coissac
+'''
+
+import zipfile
+import bz2
+import gzip
+
+import io
+
+cdef class MagicKeyFile:
+    def __init__(self,stream,length=2):
+        
+        binary=stream
+        self.stream = stream
+        self.stream_mode = None 
+        if hasattr(stream, "mode"):
+            self.stream_mode = stream.mode
+            if (not 'b' in stream.mode and 
+                hasattr(stream, "buffer") and 
+                'b' in stream.buffer.mode):
+                binary=stream.buffer
+
+        if (self.stream_mode is None and 
+            not (hasattr(stream, 'headers') and 
+                 hasattr(stream.headers, "keys") and
+                 'Content-type' in stream.headers)):
+            raise TypeError("stream does not present the good interface")
+        
+        self.binary=binary
+        self.key=binary.read(length)
+        self.keylength=length
+        self.pos=0
+        
+    cpdef bytes read(self,int size=-1):
+        cdef bytes r
+        
+        if self.pos < self.keylength:
+            if size > (self.keylength - self.pos):
+                size = size - self.keylength + self.pos
+                r = self.key[self.pos:] + self.binary.read(size)
+                self.pos=self.keylength + 1
+            elif size >=0 :
+                r = self.key[self.pos:(self.pos+size)]
+                self.pos+=size
+            else:
+                r = self.key[self.pos:] + self.binary.read(size)
+                self.pos=self.keylength + 1
+        else:
+            r = self.binary.read(size)
+            
+        return r
+    
+    cpdef int tell(self):
+        cdef int p
+        
+        if self.pos < self.keylength:
+            p = self.pos 
+        else:
+            p = self.tell()
+        
+        return p
+    
+    def __getattr__(self,name):
+        return getattr(self.binary, name)
+        
+
+    
+cdef class CompressedFile:
+    
+    
+    def __init__(self,stream):
+        cdef int            keylength
+        cdef MagicKeyFile   magic
+        cdef str            compressor
+        cdef bytes          k 
+        cdef object         c
+        
+        cdef dict compress = { 'zip' : (b'\x50\x4b\x03\x04',zipfile.ZipFile),
+                               'bz2' : (b'\x42\x5a\x68',bz2.BZ2File),
+                               'gz'  : (b'\x1f\x8b\x08',gzip.open)
+                   }
+        
+        keylength = max([len(x[0]) for x in compress.values()])
+        magic=MagicKeyFile(stream,keylength)
+        
+        self.accessor = None
+        
+        for compressor in compress:
+            k,c = compress[compressor]
+            if magic.key.startswith(k):
+                self.accessor = c(magic)
+        
+        if self.accessor is None:
+            self.accessor = magic
+        
+        if ((hasattr(stream, 'headers') and 
+             hasattr(stream.headers, "keys") and
+             'Content-type' in stream.headers and
+             stream.headers['Content-type'].startswith('text/')) or
+            'b' not in magic.stream_mode):
+            self.accessor = io.TextIOWrapper(self.accessor)
+        
+    
+    def __getattr__(self,name):
+        return getattr(self.accessor, name)
+    
+    def __iter__(self):
+        for x in self.accessor:
+            yield x 

python/obitools3/files/universalopener.pxd

@@ -0,0 +1,5 @@
+#cython: language_level=3
+
+from .uncompress cimport CompressedFile
+
+cpdef CompressedFile uopen(str name, mode=?)

python/obitools3/files/universalopener.pyx

@@ -0,0 +1,23 @@
+#cython: language_level=3
+
+'''
+Created on 25 mars 2016
+
+@author: coissac
+'''
+
+from urllib.request import urlopen
+
+
+cpdef CompressedFile uopen(str name, mode='r'):
+    cdef CompressedFile c
+    
+    try:
+        f = urlopen(name)
+    except:
+        f = open(name,mode) 
+        
+    c = CompressedFile(f)
+    
+    return c
+    

python/obitools3/obidms/_obidms.cfiles

@@ -8,7 +8,7 @@
 ../../../src/dna_seq_indexer.c
 ../../../src/encode.h
 ../../../src/encode.c
-../../../src/MurmurHash2.h
+../../../src/murmurhash2.h
 ../../../src/murmurhash2.c
 ../../../src/obiavl.h
 ../../../src/obiavl.c

python/obitools3/obidms/_obidms.pxd

@@ -69,7 +69,7 @@ cdef class OBIView_NUC_SEQS(OBIView):
 
     cdef OBIDMS_column ids
     cdef OBIDMS_column sequences
-    cdef OBIDMS_column descriptions
+    cdef OBIDMS_column definitions
     
     cpdef delete_column(self, str column_name)
     

python/obitools3/obidms/_obidms.pyx

@@ -6,9 +6,10 @@ from .capi.obidms cimport obi_dms, \
                           obi_close_dms
                           
 from .capi.obidmscolumn cimport obi_close_column, \
-                                obi_column_format_date, \
                                 OBIDMS_column_p, \
                                 OBIDMS_column_header_p
+
+from .capi.obiutils cimport obi_format_date
                                 
 from .capi.obitypes cimport const_char_p, \
                             OBIType_t, \
@@ -57,8 +58,8 @@ from .capi.obiview cimport Obiview_p, \
                            obi_new_view_cloned_from_name, \
                            obi_new_view_nuc_seqs_cloned_from_name, \
                            obi_open_view,  \
-                           obi_read_views, \
-                           obi_unmap_read_views, \
+                           obi_read_view_infos, \
+                           obi_close_view_infos, \
                            obi_view_delete_column, \
                            obi_view_add_column, \
                            obi_view_get_column, \
@@ -69,7 +70,7 @@ from .capi.obiview cimport Obiview_p, \
                            VIEW_TYPE_NUC_SEQS, \
                            NUC_SEQUENCE_COLUMN, \
                            ID_COLUMN, \
-                           DESCRIPTION_COLUMN
+                           DEFINITION_COLUMN
                            
 from libc.stdlib cimport malloc
 from cpython.pycapsule cimport PyCapsule_New, PyCapsule_GetPointer
@@ -131,7 +132,7 @@ cdef class OBIDMS_column :
         return (self.pointer)[0].header.lines_used
         
     cpdef str get_creation_date(self):
-        return bytes2str(obi_column_format_date((self.pointer)[0].header.creation_date))
+        return bytes2str(obi_format_date((self.pointer)[0].header.creation_date))
     
     cpdef str get_comments(self):
         return bytes2str((self.pointer)[0].header.comments)
@@ -529,7 +530,7 @@ cdef class OBIView_NUC_SEQS(OBIView):
         
         self.ids = self.columns[bytes2str(ID_COLUMN)]
         self.sequences = self.columns[bytes2str(NUC_SEQUENCE_COLUMN)]
-        self.descriptions = self.columns[bytes2str(DESCRIPTION_COLUMN)]
+        self.definitions = self.columns[bytes2str(DEFINITION_COLUMN)]
 
 
     cpdef delete_column(self, str column_name) :
@@ -542,7 +543,7 @@ cdef class OBIView_NUC_SEQS(OBIView):
         cdef OBIDMS_column_header_p header
         cdef str column_n
  
-        if ((column_name == bytes2str(ID_COLUMN)) or (column_name == bytes2str(NUC_SEQUENCE_COLUMN)) or (column_name == bytes2str(DESCRIPTION_COLUMN))) :
+        if ((column_name == bytes2str(ID_COLUMN)) or (column_name == bytes2str(NUC_SEQUENCE_COLUMN)) or (column_name == bytes2str(DEFINITION_COLUMN))) :
             raise Exception("Can't delete an obligatory column from a NUC_SEQS view")
         
         view = self.pointer
@@ -703,7 +704,7 @@ cdef class OBIDMS :
         cdef bytes name_b
         
         views = {}
-        all_views_p = obi_read_views(self.pointer)
+        all_views_p = obi_read_view_infos(self.pointer)
         if all_views_p == NULL :
             raise Exception("No views to read")
         nb_views = <int> (all_views_p.header).view_count
@@ -717,7 +718,7 @@ cdef class OBIDMS :
             views[view_name]["line_count"] = <int> view_p.line_count
             views[view_name]["view_number"] = <int> view_p.view_number
             views[view_name]["created_from"] = bytes2str(view_p.created_from)
-            views[view_name]["creation_date"] = bytes2str(obi_column_format_date(view_p.creation_date))     # TODO move this function in utils or somethings
+            views[view_name]["creation_date"] = bytes2str(obi_format_date(view_p.creation_date))     # TODO move this function in utils or somethings
             if (view_p.all_lines) :
                 views[view_name]["line_selection"] = None
             else :
@@ -731,7 +732,7 @@ cdef class OBIDMS :
                 views[view_name]["column_references"][column_name] = {}
                 views[view_name]["column_references"][column_name]["version"] = column_refs[j].version
         
-        obi_unmap_read_views(all_views_p);
+        obi_close_view_infos(all_views_p);
         
         return views
     

python/obitools3/obidms/_obidmscolumn_bool.cfiles

@@ -1,5 +1,3 @@
-../../../src/obidmscolumn_bool.c
-../../../src/obidmscolumn_bool.h
 ../../../src/bloom.h
 ../../../src/bloom.c
 ../../../src/char_str_indexer.h
@@ -10,7 +8,7 @@
 ../../../src/dna_seq_indexer.c
 ../../../src/encode.h
 ../../../src/encode.c
-../../../src/MurmurHash2.h
+../../../src/murmurhash2.h
 ../../../src/murmurhash2.c
 ../../../src/obiavl.h
 ../../../src/obiavl.c
@@ -39,3 +37,15 @@
 ../../../src/obiview.c
 ../../../src/utils.h
 ../../../src/utils.c
+../../../src/obidmscolumn_bool.c
+../../../src/obidmscolumn_bool.h
+../../../src/obidmscolumn_char.c
+../../../src/obidmscolumn_char.h
+../../../src/obidmscolumn_float.c
+../../../src/obidmscolumn_float.h
+../../../src/obidmscolumn_int.c
+../../../src/obidmscolumn_int.h
+../../../src/obidmscolumn_seq.c
+../../../src/obidmscolumn_seq.h
+../../../src/obidmscolumn_str.c
+../../../src/obidmscolumn_str.h

python/obitools3/obidms/_obidmscolumn_char.cfiles

@@ -1,5 +1,3 @@
-../../../src/obidmscolumn_char.c
-../../../src/obidmscolumn_char.h
 ../../../src/bloom.h
 ../../../src/bloom.c
 ../../../src/char_str_indexer.h
@@ -10,7 +8,7 @@
 ../../../src/dna_seq_indexer.c
 ../../../src/encode.h
 ../../../src/encode.c
-../../../src/MurmurHash2.h
+../../../src/murmurhash2.h
 ../../../src/murmurhash2.c
 ../../../src/obiavl.h
 ../../../src/obiavl.c
@@ -39,3 +37,15 @@
 ../../../src/obiview.c
 ../../../src/utils.h
 ../../../src/utils.c
+../../../src/obidmscolumn_bool.c
+../../../src/obidmscolumn_bool.h
+../../../src/obidmscolumn_char.c
+../../../src/obidmscolumn_char.h
+../../../src/obidmscolumn_float.c
+../../../src/obidmscolumn_float.h
+../../../src/obidmscolumn_int.c
+../../../src/obidmscolumn_int.h
+../../../src/obidmscolumn_seq.c
+../../../src/obidmscolumn_seq.h
+../../../src/obidmscolumn_str.c
+../../../src/obidmscolumn_str.h

python/obitools3/obidms/_obidmscolumn_float.cfiles

@@ -1,16 +1,14 @@
-../../../src/obidmscolumn_float.c
-../../../src/obidmscolumn_float.h
-../../../src/char_str_indexer.h
-../../../src/char_str_indexer.c
 ../../../src/bloom.h
 ../../../src/bloom.c
+../../../src/char_str_indexer.h
+../../../src/char_str_indexer.c
 ../../../src/crc64.h
 ../../../src/crc64.c
 ../../../src/dna_seq_indexer.h
 ../../../src/dna_seq_indexer.c
 ../../../src/encode.h
 ../../../src/encode.c
-../../../src/MurmurHash2.h
+../../../src/murmurhash2.h
 ../../../src/murmurhash2.c
 ../../../src/obiavl.h
 ../../../src/obiavl.c
@@ -39,3 +37,15 @@
 ../../../src/obiview.c
 ../../../src/utils.h
 ../../../src/utils.c
+../../../src/obidmscolumn_bool.c
+../../../src/obidmscolumn_bool.h
+../../../src/obidmscolumn_char.c
+../../../src/obidmscolumn_char.h
+../../../src/obidmscolumn_float.c
+../../../src/obidmscolumn_float.h
+../../../src/obidmscolumn_int.c
+../../../src/obidmscolumn_int.h
+../../../src/obidmscolumn_seq.c
+../../../src/obidmscolumn_seq.h
+../../../src/obidmscolumn_str.c
+../../../src/obidmscolumn_str.h

python/obitools3/obidms/_obidmscolumn_int.cfiles

@@ -1,16 +1,14 @@
-../../../src/obidmscolumn_int.c
-../../../src/obidmscolumn_int.h
-../../../src/char_str_indexer.h
-../../../src/char_str_indexer.c
 ../../../src/bloom.h
 ../../../src/bloom.c
+../../../src/char_str_indexer.h
+../../../src/char_str_indexer.c
 ../../../src/crc64.h
 ../../../src/crc64.c
 ../../../src/dna_seq_indexer.h
 ../../../src/dna_seq_indexer.c
 ../../../src/encode.h
 ../../../src/encode.c
-../../../src/MurmurHash2.h
+../../../src/murmurhash2.h
 ../../../src/murmurhash2.c
 ../../../src/obiavl.h
 ../../../src/obiavl.c
@@ -39,3 +37,15 @@
 ../../../src/obiview.c
 ../../../src/utils.h
 ../../../src/utils.c
+../../../src/obidmscolumn_bool.c
+../../../src/obidmscolumn_bool.h
+../../../src/obidmscolumn_char.c
+../../../src/obidmscolumn_char.h
+../../../src/obidmscolumn_float.c
+../../../src/obidmscolumn_float.h
+../../../src/obidmscolumn_int.c
+../../../src/obidmscolumn_int.h
+../../../src/obidmscolumn_seq.c
+../../../src/obidmscolumn_seq.h
+../../../src/obidmscolumn_str.c
+../../../src/obidmscolumn_str.h

python/obitools3/obidms/_obidmscolumn_seq.cfiles

@@ -1,5 +1,3 @@
-../../../src/obidmscolumn_seq.c
-../../../src/obidmscolumn_seq.h
 ../../../src/bloom.h
 ../../../src/bloom.c
 ../../../src/char_str_indexer.h
@@ -10,7 +8,7 @@
 ../../../src/dna_seq_indexer.c
 ../../../src/encode.h
 ../../../src/encode.c
-../../../src/MurmurHash2.h
+../../../src/murmurhash2.h
 ../../../src/murmurhash2.c
 ../../../src/obiavl.h
 ../../../src/obiavl.c
@@ -39,3 +37,15 @@
 ../../../src/obiview.c
 ../../../src/utils.h
 ../../../src/utils.c
+../../../src/obidmscolumn_bool.c
+../../../src/obidmscolumn_bool.h
+../../../src/obidmscolumn_char.c
+../../../src/obidmscolumn_char.h
+../../../src/obidmscolumn_float.c
+../../../src/obidmscolumn_float.h
+../../../src/obidmscolumn_int.c
+../../../src/obidmscolumn_int.h
+../../../src/obidmscolumn_seq.c
+../../../src/obidmscolumn_seq.h
+../../../src/obidmscolumn_str.c
+../../../src/obidmscolumn_str.h

python/obitools3/obidms/_obidmscolumn_str.cfiles

@@ -1,5 +1,3 @@
-../../../src/obidmscolumn_str.c
-../../../src/obidmscolumn_str.h
 ../../../src/bloom.h
 ../../../src/bloom.c
 ../../../src/char_str_indexer.h
@@ -10,7 +8,7 @@
 ../../../src/dna_seq_indexer.c
 ../../../src/encode.h
 ../../../src/encode.c
-../../../src/MurmurHash2.h
+../../../src/murmurhash2.h
 ../../../src/murmurhash2.c
 ../../../src/obiavl.h
 ../../../src/obiavl.c
@@ -39,3 +37,15 @@
 ../../../src/obiview.c
 ../../../src/utils.h
 ../../../src/utils.c
+../../../src/obidmscolumn_bool.c
+../../../src/obidmscolumn_bool.h
+../../../src/obidmscolumn_char.c
+../../../src/obidmscolumn_char.h
+../../../src/obidmscolumn_float.c
+../../../src/obidmscolumn_float.h
+../../../src/obidmscolumn_int.c
+../../../src/obidmscolumn_int.h
+../../../src/obidmscolumn_seq.c
+../../../src/obidmscolumn_seq.h
+../../../src/obidmscolumn_str.c
+../../../src/obidmscolumn_str.h

python/obitools3/obidms/_obiseq.cfiles

@@ -8,7 +8,7 @@
 ../../../src/dna_seq_indexer.c
 ../../../src/encode.h
 ../../../src/encode.c
-../../../src/MurmurHash2.h
+../../../src/murmurhash2.h
 ../../../src/murmurhash2.c
 ../../../src/obiavl.h
 ../../../src/obiavl.c
@@ -37,3 +37,15 @@
 ../../../src/obiview.c
 ../../../src/utils.h
 ../../../src/utils.c
+../../../src/obidmscolumn_bool.c
+../../../src/obidmscolumn_bool.h
+../../../src/obidmscolumn_char.c
+../../../src/obidmscolumn_char.h
+../../../src/obidmscolumn_float.c
+../../../src/obidmscolumn_float.h
+../../../src/obidmscolumn_int.c
+../../../src/obidmscolumn_int.h
+../../../src/obidmscolumn_seq.c
+../../../src/obidmscolumn_seq.h
+../../../src/obidmscolumn_str.c
+../../../src/obidmscolumn_str.h

python/obitools3/obidms/_obiseq.pxd

@@ -5,13 +5,13 @@ from ._obidms cimport OBIView_line
 
 cdef class OBI_Seq(dict) :
     cdef str id
-    cdef str description
+    cdef str definition
     cdef str sequence
 
     cpdef set_id(self, str id)
     cpdef get_id(self)
-    cpdef set_description(self, str description)
-    cpdef get_description(self)
+    cpdef set_definition(self, str definition)
+    cpdef get_definition(self)
     cpdef get_sequence(self)
 
 
@@ -23,8 +23,8 @@ cdef class OBI_Nuc_Seq(OBI_Seq) :
 cdef class OBI_Nuc_Seq_Stored(OBIView_line) :
     cpdef set_id(self, str id)
     cpdef get_id(self)
-    cpdef set_description(self, str description)
-    cpdef get_description(self)
+    cpdef set_definition(self, str definition)
+    cpdef get_definition(self)
     cpdef set_sequence(self, str sequence)
     cpdef get_sequence(self)
    # cpdef str reverse_complement(self)

python/obitools3/obidms/_obiseq.pyx

@@ -4,15 +4,15 @@ from obitools3.utils cimport bytes2str, str2bytes
 
 from .capi.obiview cimport NUC_SEQUENCE_COLUMN, \
                            ID_COLUMN, \
-                           DESCRIPTION_COLUMN
+                           DEFINITION_COLUMN
 
 cdef class OBI_Seq(dict) :
 
-    def __init__(self, str id, str seq, str description=None) :
+    def __init__(self, str id, str seq, str definition=None) :
         self.set_id(id)
         self.set_sequence(seq)
-        if description is not None :
-            self.set_description(description)
+        if definition is not None :
+            self.set_definition(definition)
         
     cpdef set_id(self, str id) :
         self.id = id
@@ -21,12 +21,12 @@ cdef class OBI_Seq(dict) :
     cpdef get_id(self) :
         return self.id
             
-    cpdef set_description(self, str description) :
-        self.description = description
-        self[bytes2str(DESCRIPTION_COLUMN)] = description
+    cpdef set_definition(self, str definition) :
+        self.definition = definition
+        self[bytes2str(DEFINITION_COLUMN)] = definition
         
-    cpdef get_description(self) :
-        return self.description     # TODO no
+    cpdef get_definition(self) :
+        return self.definition
         
     cpdef get_sequence(self) :
         return self.sequence
@@ -53,11 +53,11 @@ cdef class OBI_Nuc_Seq_Stored(OBIView_line) :
     cpdef get_id(self) :
         return self[bytes2str(ID_COLUMN)]
             
-    cpdef set_description(self, str description) :
-        self[bytes2str(DESCRIPTION_COLUMN)] = description
+    cpdef set_definition(self, str definition) :
+        self[bytes2str(DEFINITION_COLUMN)] = definition
 
-    cpdef get_description(self) :
-        return self[bytes2str(DESCRIPTION_COLUMN)]
+    cpdef get_definition(self) :
+        return self[bytes2str(DEFINITION_COLUMN)]
 
     cpdef set_sequence(self, str sequence) :
         self[bytes2str(NUC_SEQUENCE_COLUMN)] = sequence

python/obitools3/obidms/_obitaxo.cfiles

@@ -8,7 +8,7 @@
 ../../../src/dna_seq_indexer.c
 ../../../src/encode.h
 ../../../src/encode.c
-../../../src/MurmurHash2.h
+../../../src/murmurhash2.h
 ../../../src/murmurhash2.c
 ../../../src/obiavl.h
 ../../../src/obiavl.c
@@ -37,3 +37,15 @@
 ../../../src/obiview.c
 ../../../src/utils.h
 ../../../src/utils.c
+../../../src/obidmscolumn_bool.c
+../../../src/obidmscolumn_bool.h
+../../../src/obidmscolumn_char.c
+../../../src/obidmscolumn_char.h
+../../../src/obidmscolumn_float.c
+../../../src/obidmscolumn_float.h
+../../../src/obidmscolumn_int.c
+../../../src/obidmscolumn_int.h
+../../../src/obidmscolumn_seq.c
+../../../src/obidmscolumn_seq.h
+../../../src/obidmscolumn_str.c
+../../../src/obidmscolumn_str.h

python/obitools3/obidms/capi/obidmscolumn.pxd

@@ -70,8 +70,6 @@ cdef extern from "obidmscolumn.h" nogil:
                                                            obiversion_t version_number)
     
     int obi_close_header(OBIDMS_column_header_p header)
-        
-    char* obi_column_format_date(time_t date)
 
     int obi_select(OBIDMS_column_p line_selection_column, index_t line_to_grep)
 

python/obitools3/obidms/capi/obitypes.pxd

@@ -8,15 +8,11 @@ from posix.types cimport time_t
 cdef extern from *:
     ctypedef char* const_char_p "const char*"
 
+
 cdef extern from "encode.h" nogil:
     bint only_ATGC(const_char_p seq)
 
 
-cdef extern from "obidmscolumn.h" nogil:
-
-    ctypedef int32_t obiversion_t
-
-
 cdef extern from "obitypes.h" nogil:
 
     enum OBIType:
@@ -37,10 +33,12 @@ cdef extern from "obitypes.h" nogil:
     
     ctypedef OBIBool obibool_t
     ctypedef int32_t obiint_t
-    ctypedef double obifloat_t
-    ctypedef char obichar_t  
+    ctypedef double  obifloat_t
+    ctypedef char    obichar_t  
     ctypedef int64_t index_t
 
+    ctypedef int32_t obiversion_t
+
     extern obiint_t      OBIInt_NA
     extern index_t       OBIIdx_NA
     extern obifloat_t    OBIFloat_NA

python/obitools3/obidms/capi/obiutils.pxd

@@ -0,0 +1,12 @@
+#cython: language_level=3
+
+
+from posix.types cimport time_t
+
+from ..capi.obitypes cimport const_char_p
+
+
+cdef extern from "utils.h" nogil:
+
+    const_char_p obi_format_date(time_t date)
+

python/obitools3/obidms/capi/obiview.pxd

@@ -18,7 +18,23 @@ cdef extern from "obiview.h" nogil:
     extern const_char_p VIEW_TYPE_NUC_SEQS
     extern const_char_p NUC_SEQUENCE_COLUMN
     extern const_char_p ID_COLUMN
-    extern const_char_p DESCRIPTION_COLUMN
+    extern const_char_p DEFINITION_COLUMN
+
+    struct Obiview_t :
+        OBIDMS_p            dms
+        const_char_p        name
+        const_char_p        created_from
+        const_char_p        view_type
+        bint                read_only
+        OBIDMS_column_p     line_selection
+        OBIDMS_column_p     new_line_selection
+        index_t             line_count
+        int                 column_count
+        OBIDMS_column_p     columns
+        const_char_p        comments
+
+    ctypedef Obiview_t* Obiview_p
+
 
     struct Column_reference_t :
         const_char_p  column_name
@@ -27,32 +43,17 @@ cdef extern from "obiview.h" nogil:
     ctypedef Column_reference_t* Column_reference_p
 
 
-    struct Obiview_t :
-        OBIDMS_p            dms
-        const_char_p        name
-        OBIDMS_column_p     line_selection
-        OBIDMS_column_p     new_line_selection
-        OBIDMS_column_p     columns
-        bint                read_only
-        Column_reference_t  line_selection_reference
-        index_t             line_count
-        int                 column_count
-        const_char_p        comments
-
-    ctypedef Obiview_t* Obiview_p
-
-
     struct Obiview_infos_t :
         int                 view_number
-        int                 column_count
-        index_t             line_count
+        time_t              creation_date
         const_char_p        name
         const_char_p        created_from
-        time_t              creation_date
+        const_char_p        view_type
         bint                all_lines
         Column_reference_t  line_selection
+        index_t             line_count
+        int                 column_count
         Column_reference_p  column_references
-        const_char_p        view_type
         const_char_p        comments
 
     ctypedef Obiview_infos_t* Obiview_infos_p
@@ -83,10 +84,6 @@ cdef extern from "obiview.h" nogil:
               
     Obiview_p obi_open_view(OBIDMS_p dms, const_char_p view_name)
     
-    Obiviews_infos_all_p obi_read_views(OBIDMS_p dms)
-
-    int obi_unmap_read_views(Obiviews_infos_all_p views)
-
     int obi_view_add_column(Obiview_p view,
                             const_char_p column_name,
                             obiversion_t version_number,
@@ -103,8 +100,6 @@ cdef extern from "obiview.h" nogil:
     int obi_select_line(Obiview_p view, index_t line_nb)
 
     int obi_select_lines(Obiview_p view, index_t* line_nbs)
-
-    OBIDMS_column_p obi_view_clone_column(Obiview_p view, const_char_p column_name)
         
     OBIDMS_column_p obi_view_get_column(Obiview_p view, const_char_p column_name)
 
@@ -116,6 +111,10 @@ cdef extern from "obiview.h" nogil:
     
     int obi_save_and_close_view(Obiview_p view)
     
+    Obiviews_infos_all_p obi_read_view_infos(OBIDMS_p dms)
+
+    int obi_close_view_infos(Obiviews_infos_all_p views)
+
     int obi_column_set_obiint_with_elt_name_in_view(Obiview_p view,
                                                     OBIDMS_column_p column, 
                                                     index_t line_nb, 

python/obitools3/obigrep.py

@@ -0,0 +1,51 @@
+import sys
+import argparse
+
+from obitools3.obidms._obidms import OBIDMS
+
+if __name__ == '__main__': 
+    
+    parser = argparse.ArgumentParser(description='Pseudo obigrep.')
+    
+    parser.add_argument('-V', '--view', dest='view', type=str,
+                       help='Name of the view that should be considered')    
+
+    parser.add_argument('-N', '--new_view', dest='new_view', type=str,
+                       help='Name of the new view that should be created')    
+    
+#     parser.add_argument('-k', '--key', dest='key', type=str,
+#                        help='Name of the key that should be considered')
+# 
+#     parser.add_argument('-c', '--comp', dest='comparison', type=int,
+#                        help='Comparison to be made: -1:< ; 0:== ; 1:>')
+# 
+#     parser.add_argument('-v', '--value', dest='value', type=object,
+#                        help='Value to be compared')
+
+    args = parser.parse_args()
+    
+    d = OBIDMS('tdms')
+    
+    #condition = 1
+    line_selec = []
+    
+    v = d.open_view(args.view)
+    
+    i = 0
+    for l in v :
+        if l['score'] > 350 : 
+            line_selec.append(i)
+        i+=1
+        
+    new_v = d.new_view(args.new_view, view_to_clone=v, line_selection=line_selec, view_type="NUC_SEQS_VIEW", comments="obigrep "+args.view+" to "+args.new_view) #args.key+" "+str(args.comparison)+" "+str(args.value)+" "+)
+
+    print("\n")
+    print(new_v.__repr__())
+
+    v.save_and_close()
+    new_v.save_and_close()
+    
+    d.close()
+
+    print("\nDone.")
+    

python/obitools3/obihead.py

@@ -0,0 +1,43 @@
+import sys
+import argparse
+
+from obitools3.obidms._obidms import OBIDMS
+
+if __name__ == '__main__': 
+    
+    parser = argparse.ArgumentParser(description='Pseudo obihead.')
+    
+    parser.add_argument('-V', '--view', dest='view', type=str,
+                       help='Name of the view that should be considered')    
+
+    parser.add_argument('-N', '--new_view', dest='new_view', type=str,
+                       help='Name of the new view that should be created')    
+    
+    parser.add_argument('-n', '--nb', dest='nb_lines', type=int,
+                       help='Number of lines that should be taken')    
+    
+
+    args = parser.parse_args()
+    
+    d = OBIDMS('tdms')
+    
+    #condition = 1
+    line_selec = []
+    
+    v = d.open_view(args.view)
+    
+    for i in range(0, args.nb_lines) :
+        line_selec.append(i)
+        
+    new_v = d.new_view(args.new_view, view_to_clone=v, line_selection=line_selec, view_type="NUC_SEQS_VIEW", comments="obihead "+str(args.nb_lines)+", "+args.view+" to "+args.new_view) #args.key+" "+str(args.comparison)+" "+str(args.value)+" "+)
+
+    print("\n")
+    print(new_v.__repr__())
+
+    v.save_and_close()
+    new_v.save_and_close()
+    
+    d.close()
+
+    print("\nDone.")
+    

python/obitools3/parsers/fasta.pxd

@@ -0,0 +1,8 @@
+#cython: language_level=3
+
+from .header cimport parseHeader
+from ..files.universalopener cimport uopen
+from ..files.linebuffer cimport LineBuffer
+
+
+    

python/obitools3/parsers/fasta.pyx

@@ -0,0 +1,46 @@
+#cython: language_level=3
+
+'''
+Created on 30 mars 2016
+
+@author: coissac
+'''
+
+def fastaIterator(lineiterator, int buffersize=100000000):
+    cdef LineBuffer lb
+    cdef str        ident
+    cdef str        definition
+    cdef dict       tags
+    cdef list       s
+    
+    if isinstance(lineiterator,(str,bytes)):
+        lineiterator=uopen(lineiterator)
+    
+    if isinstance(lineiterator, LineBuffer):
+        lb=lineiterator
+    else:
+        lb=LineBuffer(lineiterator,buffersize)
+        
+    i = iter(lb)
+    line = next(i)
+    
+    while True:
+        ident,tags,definition = parseHeader(line)
+        s = []
+        line = next(i)
+        while line[0]!='>':
+            s.append(line[0:-1])
+            line = next(i)
+        sequence  = "".join(s)
+        quality   = None
+
+        yield { "id"         : ident,
+                "definition" : definition,
+                "sequence"   : sequence,
+                "quality"    : quality,
+                "tags"       : tags,
+                "annotation" : {}
+              }
+        
+        
+        

python/obitools3/parsers/fastq.pxd

@@ -0,0 +1,8 @@
+#cython: language_level=3
+
+from .header cimport parseHeader
+from ..files.universalopener cimport uopen
+from ..files.linebuffer cimport LineBuffer
+
+
+    

python/obitools3/parsers/fastq.pyx

@@ -0,0 +1,41 @@
+#cython: language_level=3
+
+'''
+Created on 30 mars 2016
+
+@author: coissac
+'''
+
+
+
+def fastqIterator(lineiterator, int buffersize=100000000):
+    cdef LineBuffer lb
+    cdef str        ident
+    cdef str        definition
+    cdef dict       tags
+    
+    if isinstance(lineiterator,(str,bytes)):
+        lineiterator=uopen(lineiterator)
+    
+    if isinstance(lineiterator, LineBuffer):
+        lb=lineiterator
+    else:
+        lb=LineBuffer(lineiterator,buffersize)
+        
+    i = iter(lb)
+    for line in i:
+        ident,tags,definition = parseHeader(line)
+        sequence  = next(i)[0:-1]
+        next(i)
+        quality   = next(i)[0:-1]
+
+        yield { "id"         : ident,
+                "definition" : definition,
+                "sequence"   : sequence,
+                "quality"    : quality,
+                "tags"       : tags,
+                "annotation" : {}
+              }
+        
+        
+        

python/obitools3/parsers/header.pxd

@@ -0,0 +1,5 @@
+#cython: language_level=3
+
+cdef object __etag__(str x)
+
+cpdef tuple parseHeader(str header)

python/obitools3/parsers/header.pyx

@@ -0,0 +1,78 @@
+#cython: language_level=3
+
+'''
+Created on 25 mars 2016
+
+@author: coissac
+'''
+
+import re
+
+__ret__         = re.compile('''(([^ ]+)=('[^']*'|"[^"]*"|[^;]+); *)+?''')
+__re_int__      = re.compile("^[+-]?[0-9]+$")
+__re_float__    = re.compile("^[+-]?[0-9]+(\.[0-9]*)?([eE][+-]?[0-9]+)?$")
+__re_str__      = re.compile("""^"[^"]*"|'[^']*'$""")
+__re_dict__     = re.compile("""^\{\ *
+                                   (
+                                       ("[^"]*"|'[^']*')
+                                        \ *:\ *
+                                       ([^,}]+|
+                                        "[^"]*"|
+                                        '[^']*'
+                                       )
+                                   )?
+                                   (\ *,\ *
+                                       ("[^"]*"|'[^']*')
+                                        \ *:\ *
+                                       ([^,}]+|
+                                        "[^"]*"|
+                                        '[^']*'
+                                       )
+                                    )*\ *\}$""", re.VERBOSE)
+
+
+cdef object __etag__(str x):
+    if __re_int__.match(x):
+        v=int(x)
+    elif __re_float__.match(x):
+        v=float(x)
+    elif __re_str__.match(x):
+        v=x[1:-1]
+    elif x=='False':
+        v=False
+    elif x=='True':
+        v=True
+    elif __re_dict__.match(x):
+        v=eval(x)
+    else:
+        v=x
+    return v
+
+cpdef tuple parseHeader(str header):
+    cdef list m 
+    cdef dict tags 
+    cdef str definition
+    cdef str ident
+    cdef str second
+    
+    m=header[1:-1].split(maxsplit=1)
+
+    ident=m[0]
+    
+    if len(m)==1:
+        tags={}
+        definition='' 
+    else:
+        second=m[1]
+        m = __ret__.findall(second)
+        
+        if m:
+            tags = dict([(a[1],__etag__(a[2])) for a in m])
+            definition = second.split(m[-1][0],1)[1].strip()
+        else:
+            tags = {}
+            definition = second.strip() 
+        
+    return ident,tags,definition
+    
+    

python/obitools3/utils.pxd

@@ -1,5 +1,6 @@
 #cython: language_level=3
 
+
 cdef bytes str2bytes(str string)
 
 cdef str bytes2str(bytes string)

python/obitools3/utils.pyx

@@ -1,7 +1,28 @@
 #cython: language_level=3
 
+import sys
+import io
+
 cdef bytes str2bytes(str string):
+    """
+    Short cut to convert ascii encoded python string (str) to bytes 
+    which can be easily converted to C-strings. 
+    
+        @param string: the python string to be converted.
+        @type string: str
+        @return a transcoded string
+        @rtype: bytes 
+    """
     return string.encode('ascii')
 
 cdef str bytes2str(bytes string):
-    return string.decode('ascii')
+    """
+    Short cut to convert bytes (C-strings) to ascii encoded python string (str).
+    
+        @param string: the binary (C-string) string to be converted.
+        @type string: bytes
+        @return an ascii transcoded string
+        @rtype: str 
+    """
+    return string.decode('ascii')
+

src/bloom.c

@@ -26,9 +26,9 @@
 #define MAKESTRING(n) STRING(n)
 #define STRING(n) #n
 
-#ifdef __linux__
-unsigned detect_bucket_size(unsigned fallback_size);
-#endif
+//#ifdef __linux__										// TODO commented because triggers error on luke21
+//unsigned detect_bucket_size(unsigned fallback_size);
+//#endif
 
 
 static int test_bit_set_bit(unsigned char * buf, unsigned int x, int set_bit)
@@ -89,11 +89,11 @@ static void setup_buckets(struct bloom * bloom, unsigned int cache_size)
   // either compute it or use built-in default
 
   if (cache_size == 0) {
-#ifdef __linux__
-    cache_size = detect_bucket_size(BLOOM_BUCKET_SIZE_FALLBACK);
-#else
+//#ifdef __linux__													// TODO commented because triggers error on luke21
+//    cache_size = detect_bucket_size(BLOOM_BUCKET_SIZE_FALLBACK);
+//#else
     cache_size = BLOOM_BUCKET_SIZE_FALLBACK;
-#endif
+//#endif
   }
 
   bloom->buckets = (bloom->bytes / cache_size);
@@ -122,7 +122,6 @@ static void setup_buckets(struct bloom * bloom, unsigned int cache_size)
 }
 
 
-// TODO
 int bloom_filter_size(int entries, double error)
 {
 	int bytes;
@@ -185,7 +184,8 @@ int bloom_init_size(struct bloom * bloom, int entries, double error,
 
   setup_buckets(bloom, cache_size);
 
-  // TODO comment
+  // celine.mercier@metabarcoding.org :
+  // Replaced the calloc with a memset, as the memory for the bloom filter is mapped in our data structure
   memset(bloom->bf, 0, bloom->bytes);
   //bloom->bf = (unsigned char *)calloc(bloom->bytes, sizeof(unsigned char));
   //if (bloom->bf == NULL) {

src/bloom.h

@@ -9,7 +9,12 @@
 #define _BLOOM_H
 
 
-// TODO
+/** ***************************************************************************
+ * Added by celine.mercier@metabarcoding.org
+ *
+ * Bloom filter error rate wanted.
+ *
+ */
 #define BLOOM_FILTER_ERROR_RATE (0.001)
 
 
@@ -72,7 +77,13 @@ struct bloom
 typedef struct bloom bloom_t;
 
 
-// TODO
+/** ***************************************************************************
+ * Added by celine.mercier@metabarcoding.org
+ *
+ * This function computes the size needed by the bloom filter
+ * depending on the number of entries and the error rate.
+ *
+ */
 int bloom_filter_size(int entries, double error);
 
 

src/char_str_indexer.h

@@ -41,9 +41,11 @@ Obi_blob_p obi_str_to_blob(char* value);
 /**
  * @brief Converts a blob to a character string.
  *
+ * @warning The character string returned is mapped.
+ *
  * @param value_b The blob to convert.
  *
- * @returns A pointer to the character string contained in the blob.
+ * @returns A pointer on the character string contained in the blob.
  *
  * @since October 2015
  * @author Celine Mercier (celine.mercier@metabarcoding.org)
@@ -51,9 +53,33 @@ Obi_blob_p obi_str_to_blob(char* value);
 const char* obi_blob_to_str(Obi_blob_p value_b);
 
 
-// TODO doc
+/**
+ * @brief Stores a character string in an indexer and returns the index.
+ *
+ * @param indexer The indexer structure.
+ * @param value The character string to index.
+ *
+ * @returns The index referring to the stored character string in the indexer.
+ *
+ * @since April 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
 index_t obi_index_char_str(Obi_indexer_p indexer, const char* value);
 
+
+/**
+ * @brief Retrieves a character string from an indexer.
+ *
+ * @warning The character string returned is mapped.
+ *
+ * @param indexer The indexer structure.
+ * @param idx The index referring to the character string to retrieve in the indexer.
+ *
+ * @returns A pointer on the character string.
+ *
+ * @since April 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
 char* obi_retrieve_char_str(Obi_indexer_p indexer, index_t idx);
 
 

src/crc64.h

@@ -6,4 +6,9 @@
 
 #include <stdint.h>
 
+
+/**
+ * @brief Generates and returns a 64-bit Cyclic Redundancy Check from a
+ *        character string s of length l.
+ */
 uint64_t crc64(const char* s, uint64_t l);

src/dna_seq_indexer.h

@@ -53,9 +53,33 @@ Obi_blob_p obi_seq_to_blob(const char* seq);
 char* obi_blob_to_seq(Obi_blob_p value_b);
 
 
-// TODO doc
+/**
+ * @brief Stores a DNA sequence in an indexer and returns the index.
+ *
+ * @param indexer The indexer structure.
+ * @param value The DNA sequence to index.
+ *
+ * @returns The index referring to the stored DNA sequence in the indexer.
+ *
+ * @since April 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
 index_t obi_index_dna_seq(Obi_indexer_p indexer, const char* value);
 
+
+/**
+ * @brief Retrieves a DNA sequence from an indexer.
+ *
+ * @warning The DNA sequence returned must be freed by the caller.
+ *
+ * @param indexer The indexer structure.
+ * @param idx The index referring to the DNA sequence to retrieve in the indexer.
+ *
+ * @returns A pointer on the DNA sequence.
+ *
+ * @since April 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
 char* obi_retrieve_dna_seq(Obi_indexer_p indexer, index_t idx);
 
 

src/encode.c

@@ -63,17 +63,14 @@ byte_t* encode_seq_on_2_bits(const char* seq, int32_t length)
 
 	length_b = ceil((double) length / (double) 4.0);
 
-	seq_b = (byte_t*) malloc(length_b * sizeof(byte_t));
+	seq_b = (byte_t*) calloc(length_b, sizeof(byte_t));
 	if (seq_b == NULL)
 	{
-		obi_set_errno(OBI_MALLOC_ERROR);		// TODO
+		obi_set_errno(OBI_MALLOC_ERROR);
 		obidebug(1, "\nError allocating memory for an encoded DNA sequence");
 		return NULL;
 	}
 
-	// Initialize all the bits to 0
-	memset(seq_b, 0, length_b);
-
 	for (i=0; i<length; i++)
 	{
 		// Shift of 2 to make place for new nucleotide
@@ -99,7 +96,7 @@ byte_t* encode_seq_on_2_bits(const char* seq, int32_t length)
 			seq_b[i/4] |= NUC_T_2b;
 			break;
 		default:
-			obi_set_errno(OBI_ENCODE_ERROR);		// TODO
+			obi_set_errno(OBI_ENCODE_ERROR);
 			obidebug(1, "\nInvalid nucleotide base when encoding (not [atgcATGC])");
 			return NULL;
 		}
@@ -125,7 +122,7 @@ char* decode_seq_on_2_bits(byte_t* seq_b, int32_t length_seq)
 	seq = (char*) malloc((length_seq+1) * sizeof(char));
 	if (seq == NULL)
 	{
-		obi_set_errno(OBI_MALLOC_ERROR);		// TODO
+		obi_set_errno(OBI_MALLOC_ERROR);
 		obidebug(1, "\nError allocating memory for a decoded DNA sequence");
 		return NULL;
 	}
@@ -151,7 +148,7 @@ char* decode_seq_on_2_bits(byte_t* seq_b, int32_t length_seq)
 			seq[i] = 't';
 			break;
 		default:
-			obi_set_errno(OBI_DECODE_ERROR);		// TODO
+			obi_set_errno(OBI_DECODE_ERROR);
 			obidebug(1, "\nInvalid nucleotide base when decoding");
 			return NULL;
 		}
@@ -172,17 +169,14 @@ byte_t* encode_seq_on_4_bits(const char* seq, int32_t length)
 
 	length_b = ceil((double) length / (double) 2.0);
 
-	seq_b = (byte_t*) malloc(length_b * sizeof(byte_t));
+	seq_b = (byte_t*) calloc(length_b, sizeof(byte_t));
 	if (seq_b == NULL)
 	{
-		obi_set_errno(OBI_MALLOC_ERROR);		// TODO
+		obi_set_errno(OBI_MALLOC_ERROR);
 		obidebug(1, "\nError allocating memory for an encoded DNA sequence");
 		return NULL;
 	}
 
-	// Initialize all the bits to 0
-	memset(seq_b, 0, length_b);
-
 	for (i=0; i<length; i++)
 	{
 		// Shift of 4 to make place for new nucleotide
@@ -252,7 +246,7 @@ byte_t* encode_seq_on_4_bits(const char* seq, int32_t length)
 			seq_b[i/2] |= NUC_N_4b;
 			break;
 		default:
-			obi_set_errno(OBI_ENCODE_ERROR);	// TODO
+			obi_set_errno(OBI_ENCODE_ERROR);
 			obidebug(1, "\nInvalid nucleotide base when encoding (not IUPAC)");
 			return NULL;
 		}
@@ -278,7 +272,7 @@ char* decode_seq_on_4_bits(byte_t* seq_b, int32_t length_seq)
 	seq = (char*) malloc((length_seq+1) * sizeof(char));
 	if (seq == NULL)
 	{
-		obi_set_errno(OBI_MALLOC_ERROR);		// TODO
+		obi_set_errno(OBI_MALLOC_ERROR);
 		obidebug(1, "\nError allocating memory for a decoded DNA sequence");
 		return NULL;
 	}
@@ -337,7 +331,7 @@ char* decode_seq_on_4_bits(byte_t* seq_b, int32_t length_seq)
 			seq[i] = 'n';
 			break;
 		default:
-			obi_set_errno(OBI_DECODE_ERROR);	// TODO
+			obi_set_errno(OBI_DECODE_ERROR);
 			obidebug(1, "\nInvalid nucleotide base when decoding");
 			return NULL;
 		}

src/encode.h

@@ -22,10 +22,10 @@
 #include "obitypes.h"
 
 
-#define NUC_MASK_2B 0x3   									/**< Binary: 11 to use when decoding 2 bits sequences
- 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 */
-#define NUC_MASK_4B 0xF  									/**< Binary: 1111 to use when decoding 4 bits sequences
- 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 */
+#define NUC_MASK_2B 0x3   	/**< Binary: 11 to use when decoding 2 bits sequences
+ 	 	 	 	 	 	 	 */
+#define NUC_MASK_4B 0xF  	/**< Binary: 1111 to use when decoding 4 bits sequences
+ 	 	 	 	 	 	 	 */
 
 
 /**
@@ -107,7 +107,7 @@ byte_t* encode_seq_on_2_bits(const char* seq, int32_t length);
  *    10 -> t
  *    11 -> g
  *
- * @param seq The sequence to decode.
+ * @param seq_b The sequence to decode.
  * @param length_seq The initial length of the sequence before it was encoded.
  *
  * @returns The decoded sequence ended with '\0'.
@@ -169,7 +169,7 @@ byte_t* encode_seq_on_4_bits(const char* seq, int32_t length);
  *      V or v : 1110
  *      N or n : 1111
  *
- * @param seq The sequence to decode.
+ * @param seq_b The sequence to decode.
  * @param length_seq The initial length of the sequence before it was encoded.
  *
  * @returns The decoded sequence ended with '\0'.

src/murmurhash2.c

@@ -10,7 +10,7 @@
 
 // 1. It will not work incrementally.
 // 2. It will not produce the same results on little-endian and big-endian
-//    machines.
+//    machines.	->TODO
 
 unsigned int murmurhash2(const void * key, int len, const unsigned int seed)
 {

src/murmurhash2.h

@@ -2,6 +2,10 @@
 #ifndef _BLOOM_MURMURHASH2
 #define _BLOOM_MURMURHASH2
 
+
+/**
+ * @brief Generates and returns a hash code from a byte array.
+ */
 unsigned int murmurhash2(const void * key, int len, const unsigned int seed);
 
 #endif

src/obiavl.c

@@ -6,7 +6,7 @@
  * @file obiavl.c
  * @author Celine Mercier
  * @date December 3rd 2015
- * @brief Functions handling AVL trees for storing and retrieving bit arrays.
+ * @brief Functions handling AVL trees for storing and retrieving blobs.
  */
 
 
@@ -108,7 +108,8 @@ static char* build_avl_data_file_name(const char* avl_name);
 
 
 /**
- * @brief Internal function returning the size of an AVL tree header on this platform.
+ * @brief Internal function returning the size of an AVL tree header on this platform,
+ *        including the size of the bloom filter associated with the AVL tree.
  *
  * @returns The size of an AVL tree header in bytes.
  *
@@ -282,38 +283,6 @@ int add_new_avl_in_group(OBIDMS_avl_group_p avl_group);
 int maybe_in_avl(OBIDMS_avl_p avl, Obi_blob_p value);
 
 
-/**
- * @brief Internal function comparing two blobs.
- *
- * The encoding is compared first, then the length of the
- * values, then the values themselves.
- *
- * @param value_1 A pointer to the first blob structure.
- * @param value_2 A pointer to the second blob structure.
- *
- * @returns A value < 0 if value_1 < value_2,
- * 			a value > 0 if value_1 > value_2,
- * 			and 0 if value_1 == value_2.
- *
- * @since October 2015
- * @author Celine Mercier (celine.mercier@metabarcoding.org)
- */
-int blob_compare(Obi_blob_p value_1, Obi_blob_p value_2);
-
-
-/**
- * @brief Internal function calculating the size in bytes of a blob.
- *
- * @param value A pointer to the blob structure.
- *
- * @returns The size of the blob in bytes.
- *
- * @since October 2015
- * @author Celine Mercier (celine.mercier@metabarcoding.org)
- */
-int blob_sizeof(Obi_blob_p value);
-
-
 /**
  * @brief Internal function storing a value (blob) in the data array referred to by an AVL tree.
  *
@@ -586,7 +555,7 @@ size_t get_avl_header_size()
 
 	header_size = sizeof(OBIDMS_avl_header_t) + bloom_filter_size(MAX_NODE_COUNT_PER_AVL, BLOOM_FILTER_ERROR_RATE);
 
-	multiple = 	ceil((double) header_size / (double) getpagesize());
+	multiple = ceil((double) header_size / (double) getpagesize());
 
 	rounded_header_size = multiple * getpagesize();
 
@@ -982,41 +951,7 @@ int add_new_avl_in_group(OBIDMS_avl_group_p avl_group)
 
 int maybe_in_avl(OBIDMS_avl_p avl, Obi_blob_p value)
 {
-	return (bloom_check(&((avl->header)->bloom_filter), value, blob_sizeof(value)));
-}
-
-
-int blob_compare(Obi_blob_p value_1, Obi_blob_p value_2)
-{
-	int 	comp;
-	int32_t b;
-
-	if (value_1->element_size != value_2->element_size)
-		return (value_1->element_size - value_2->element_size);
-
-	if (value_1->length_encoded_value != value_2->length_encoded_value)
-		return (value_1->length_encoded_value - value_2->length_encoded_value);
-
-	if (value_1->element_size != ELEMENT_SIZE_STR) // because if so, length_decoded_value == length_encoded_value
-	{
-		if (value_1->length_decoded_value != value_2->length_decoded_value)
-			return (value_1->length_decoded_value - value_2->length_decoded_value);
-	}
-
-	b = 0;
-	comp = 0;
-	while (!comp && (b < value_1->length_encoded_value))
-	{
-		comp = *((value_1->value)+b) - *((value_2->value)+b);
-		b++;
-	}
-	return comp;
-}
-
-
-int blob_sizeof(Obi_blob_p value)
-{
-	return (sizeof(Obi_blob_t) + (value->length_encoded_value));
+	return (bloom_check(&((avl->header)->bloom_filter), value, obi_blob_sizeof(value)));
 }
 
 
@@ -1028,7 +963,7 @@ index_t avl_add_value_in_data_array(OBIDMS_avl_p avl, Obi_blob_p value)
 	value_idx = ((avl->data)->header)->data_size_used;
 
 	// Grow the data if needed
-	value_size = blob_sizeof(value);
+	value_size = obi_blob_sizeof(value);
 	while (((avl->data)->header)->data_size_max < (value_idx + value_size))
 	{
 		if (grow_avl_data(avl->data) < 0)
@@ -1067,7 +1002,7 @@ AVL_node_p avl_create_node(OBIDMS_avl_p avl, index_t node_idx)
 // Update the balance factors of the nodes from the node that will need balancing
 void avl_update_balance_factors(OBIDMS_avl_p avl)
 {
-	uint8_t n;
+	uint8_t    n;
 	AVL_node_p node;
 
 	// Update balance factors from the node where balancing might be needed
@@ -1221,8 +1156,8 @@ index_t avl_balance_node(OBIDMS_avl_p avl, AVL_node_p node, index_t node_idx)
 // Balance a given tree
 void avl_balance(OBIDMS_avl_p avl)
 {
-	index_t new_root;
-	index_t node_index;
+	index_t    new_root;
+	index_t    node_index;
 	AVL_node_p node_to_balance;
 	AVL_node_p parent_of_node_to_balance;
 
@@ -1968,7 +1903,7 @@ OBIDMS_avl_group_p obi_open_avl_group(OBIDMS_p dms, const char* avl_name)
 
 int obi_close_avl(OBIDMS_avl_p avl)
 {
-	int                 	ret_val = 0;
+	int ret_val = 0;
 
 	ret_val = close_avl_data(avl->data);
 
@@ -2043,7 +1978,7 @@ index_t obi_avl_add(OBIDMS_avl_p avl, Obi_blob_p value)
 
 	n = 0;
 	depth = 0;
-	crc = crc64((byte_t*)value, blob_sizeof(value));
+	crc = crc64((byte_t*)value, obi_blob_sizeof(value));
 
 	// Check if first node
 	if (!((avl->header)->nb_items))
@@ -2090,7 +2025,7 @@ index_t obi_avl_add(OBIDMS_avl_p avl, Obi_blob_p value)
 		if (comp == 0)
 		{ // check if really same value
 			to_compare = obi_avl_get(avl, current_node->value);
-			comp = blob_compare(to_compare, value);
+			comp = obi_blob_compare(to_compare, value);
 		}
 
 		if (comp > 0)
@@ -2172,7 +2107,7 @@ index_t obi_avl_find(OBIDMS_avl_p avl, Obi_blob_p value)
 	AVL_node_p       current_node;
 	uint64_t         crc;
 
-	crc = crc64((byte_t*)value, blob_sizeof(value));
+	crc = crc64((byte_t*)value, obi_blob_sizeof(value));
 
 	next = (avl->header)->root_idx;
 	while (next != -1)
@@ -2185,7 +2120,7 @@ index_t obi_avl_find(OBIDMS_avl_p avl, Obi_blob_p value)
 		if (comp == 0)
 		{ // Check if really same value
 			to_compare = obi_avl_get(avl, current_node->value);
-			comp = blob_compare(to_compare, value);
+			comp = obi_blob_compare(to_compare, value);
 		}
 
 		if (comp > 0)
@@ -2270,7 +2205,7 @@ index_t obi_avl_group_add(OBIDMS_avl_group_p avl_group, Obi_blob_p value)
 	}
 
 	// Add in the current AVL
-	bloom_add(&((((avl_group->sub_avls)[avl_group->current_avl_idx])->header)->bloom_filter), value, blob_sizeof(value));
+	bloom_add(&((((avl_group->sub_avls)[avl_group->current_avl_idx])->header)->bloom_filter), value, obi_blob_sizeof(value));
 	index_in_avl = (int32_t) obi_avl_add((avl_group->sub_avls)[avl_group->current_avl_idx], value);
 
 	// Build the index containing the AVL index

src/obiavl.h

@@ -6,7 +6,7 @@
  * @file obiavl.h
  * @author Celine Mercier
  * @date December 3rd 2015
- * @brief Header file for handling AVL trees for storing and retrieving blobs (i.e. coding for character strings).
+ * @brief Header file for handling AVL trees for storing and retrieving blobs.
  */
 
 
@@ -120,14 +120,13 @@ typedef struct OBIDMS_avl_header {
 	time_t		creation_date;			    	    /**< Date of creation of the file.
 												     */
 	bloom_t 	bloom_filter;			    	    /**< Bloom filter associated with the AVL tree, enabling to know if a value
-													 *   might already be stored in the data associated with the tree.
+													 *   might already be stored in the data referred to by the tree.
      	 	 	 	 	 	 	 	 	 	 	 	 */
 } OBIDMS_avl_header_t, *OBIDMS_avl_header_p;
 
 
 /**
  * @brief OBIDMS AVL tree structure.
- * 		  TODO doc
  */
 typedef struct OBIDMS_avl {
 	OBIDMS_p                	dms;			 			/**< A pointer to the OBIDMS structure to which the AVL tree belongs.
@@ -195,7 +194,7 @@ int obi_avl_exists(OBIDMS_p dms, const char* avl_name);
  * @brief Creates an AVL tree. Fails if it already exists.
  *
  * Note: An AVL tree is made of two files (referred to by two structures).
- * 		 One file contains the indices referring to the data, and the other
+ * 		 One file contains the tree referring to the data, and the other
  * 		 file contains the data itself. The AVL tree as a whole is referred
  * 		 to via the OBIDMS_avl structure. An AVL tree is stored in a directory
  * 		 with the same name, or with the base name of the AVL group if it is
@@ -203,7 +202,8 @@ int obi_avl_exists(OBIDMS_p dms, const char* avl_name);
  *
  * @param dms The OBIDMS to which the AVL tree belongs.
  * @param avl_name The name of the AVL tree.
- * @param avl_idx The index of the AVL tree if it is part of an AVL group.
+ * @param avl_idx The index of the AVL tree if it is part of an AVL group,
+ *        or -1 if it is not part of an AVL group.
  *
  * @returns A pointer to the newly created AVL tree structure.
  * @retval NULL if an error occurred.
@@ -215,16 +215,17 @@ OBIDMS_avl_p obi_create_avl(OBIDMS_p dms, const char* avl_name, int avl_idx);
 
 
 /**
- * @brief Opens an AVL tree. Fails if it does not already exist.
+ * @brief Opens an AVL tree in read-only mode. Fails if it does not already exist.
  *
  * Note: An AVL tree is made of two files (referred to by two structures).
- * 		 One file contains the indices referring to the data, and the other
+ * 		 One file contains the tree referring to the data, and the other
  * 		 file contains the data itself. The AVL tree as a whole is referred
  * 		 to via the OBIDMS_avl structure.
  *
  * @param dms The OBIDMS to which the AVL tree belongs.
  * @param avl_name The name of the AVL tree.
- * @param avl_idx The index of the AVL tree if it is part of an AVL group.
+ * @param avl_idx The index of the AVL tree if it is part of an AVL group,
+ *        or -1 if it is not part of an AVL group.
  *
  * @returns A pointer to the AVL tree structure.
  * @retval NULL if an error occurred.
@@ -272,7 +273,7 @@ OBIDMS_avl_group_p obi_create_avl_group(OBIDMS_p dms, const char* avl_name);
 
 
 /**
- * @brief Opens an AVL tree group.
+ * @brief Opens an AVL tree group in read-only mode.
  *
  * Note: An AVL tree group is composed of multiple AVL trees that all have the
  *       same base name, and an index differentiating them.
@@ -321,6 +322,7 @@ int obi_close_avl_group(OBIDMS_avl_group_p avl_group);
  * @brief Recovers a value (blob) in an AVL tree.
  *
  * @warning The blob recovered must be decoded to get the original value.
+ * @warning The blob recovered is mapped in memory.
  *
  * @param avl A pointer to the AVL tree.
  * @param index The index of the value in the data array.
@@ -334,9 +336,10 @@ Obi_blob_p obi_avl_get(OBIDMS_avl_p avl, index_t index);
 
 
 /**
- * @brief Adds a value (blob) in an AVL tree NOT checking first if it is already in it.		// TODO to discuss
+ * @brief Adds a value (blob) in an AVL tree.
  *
  * @warning The value given must be already be encoded into a blob structure (Obi_blob_t).
+ * @warning If the value is already in the AVL tree, an error will be triggered.		// TODO to discuss
  *
  * @param avl A pointer to the AVL tree.
  * @param value The blob to add in the AVL tree.
@@ -371,9 +374,13 @@ index_t obi_avl_find(OBIDMS_avl_p avl, Obi_blob_p value);
  * @brief Recovers a value (blob) in an AVL tree.
  *
  * @warning The blob recovered must be decoded to get the original value.
+ * @warning The blob recovered is mapped in memory.
  *
  * @param avl_group A pointer to the AVL tree.
- * @param index The index of the value in the data array.
+ * @param index The index of the value in the form of a 64-bit integer
+ *              with the 32 left-most bits coding for the index of the tree of
+ *              the group in which the value is stored, and the 32 right-most bits
+ *              coding for the index at which the value is stored in that AVL tree.
  *
  * @returns A pointer to the blob recovered.
  *
@@ -391,7 +398,10 @@ Obi_blob_p obi_avl_group_get(OBIDMS_avl_group_p avl_group, index_t idx);
  * @param avl_group A pointer to the AVL tree group.
  * @param value The blob to add in the AVL tree group.
  *
- * @returns The index of the value newly added in the AVL tree group.
+ * @returns The index of the value newly added in the AVL tree group, in the form of a
+ *          64-bit integer with the 32 left-most bits coding for the index of the tree
+ *          of the group in which the value is stored, and the 32 right-most bits
+ *          coding for the index at which the value is stored in that AVL tree.
  * @retval -1 if an error occurred.
  *
  * @since April 2016

src/obiblob.c

@@ -55,3 +55,37 @@ Obi_blob_p obi_blob(byte_t* encoded_value, uint8_t element_size, int32_t length_
 	return blob;
 }
 
+
+int obi_blob_compare(Obi_blob_p value_1, Obi_blob_p value_2)
+{
+	int 	comp;
+	int32_t b;
+
+	if (value_1->element_size != value_2->element_size)
+		return (value_1->element_size - value_2->element_size);
+
+	if (value_1->length_encoded_value != value_2->length_encoded_value)
+		return (value_1->length_encoded_value - value_2->length_encoded_value);
+
+	if (value_1->element_size != ELEMENT_SIZE_STR) // because if so, length_decoded_value == length_encoded_value
+	{
+		if (value_1->length_decoded_value != value_2->length_decoded_value)
+			return (value_1->length_decoded_value - value_2->length_decoded_value);
+	}
+
+	b = 0;
+	comp = 0;
+	while (!comp && (b < value_1->length_encoded_value))
+	{
+		comp = *((value_1->value)+b) - *((value_2->value)+b);
+		b++;
+	}
+	return comp;
+}
+
+
+int obi_blob_sizeof(Obi_blob_p value)
+{
+	return (sizeof(Obi_blob_t) + (value->length_encoded_value));
+}
+

src/obiblob.h

@@ -21,20 +21,19 @@
 #include "obitypes.h"
 
 
-#define ELEMENT_SIZE_STR (8)								/**< The size of an element from a value of type character string.
-                              	  	 	 	 	 	 	 	 */
-#define ELEMENT_SIZE_SEQ_2 (2)								/**< The size of an element from a value of type DNA sequence encoded on 2 bits.
-                              	  	 	 	 	 	 	 	 */
-#define ELEMENT_SIZE_SEQ_4 (4)								/**< The size of an element from a value of type DNA sequence encoded on 4 bits.
-                              	  	 	 	 	 	 	 	 */
+#define ELEMENT_SIZE_STR (8)		/**< The size of an element from a value of type character string.
+                              	  	 */
+#define ELEMENT_SIZE_SEQ_2 (2)		/**< The size of an element from a value of type DNA sequence encoded on 2 bits.
+                              	  	 */
+#define ELEMENT_SIZE_SEQ_4 (4)		/**< The size of an element from a value of type DNA sequence encoded on 4 bits.
+                              	  	 */
 
 
 /**
- * @brief Blob structure.
- *  TODO
+ * @brief Blob structure, for handling encoded values.
  */
 typedef struct Obi_blob {
-	uint8_t  element_size;		 	/**< Size in bits of one element from the value.
+	uint8_t  element_size;		 	/**< Size in bits of one element from the encoded value.
 	 	 	 	 	 	 	 	 	 */
 	int32_t  length_encoded_value;	/**< Length in bytes of the encoded value.
 	 	 	 	 	 	 	 	 	 */
@@ -46,9 +45,54 @@ typedef struct Obi_blob {
 
 
 
-// TODO doc
+/**
+ * @brief Function building a blob structure.
+ *
+ * @param encoded_value A pointer to the encoded value that will be stored in the blob.
+ * @param element_size The size in bits of one element from the encoded value.
+ * @param length_encoded_value The length in bytes of the encoded value.
+ * @param length_decoded_value The length in bytes of the decoded value.
+ *
+ * @returns A pointer to the created blob structure.
+ * @retval NULL if an error occurred.
+ *
+ * @since April 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
 Obi_blob_p obi_blob(byte_t* encoded_value, uint8_t element_size, int32_t length_encoded_value, int32_t length_decoded_value);
 
 
+/**
+ * @brief Function comparing two blobs.
+ *
+ * The encoding is compared first, then the length of the
+ * values, then the values themselves.
+ *
+ * @param value_1 A pointer to the first blob structure.
+ * @param value_2 A pointer to the second blob structure.
+ *
+ * @returns A value < 0 if value_1 < value_2,
+ * 			a value > 0 if value_1 > value_2,
+ * 			and 0 if value_1 == value_2.
+ *
+ * @since October 2015
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
+int obi_blob_compare(Obi_blob_p value_1, Obi_blob_p value_2);
+
+
+/**
+ * @brief Function calculating the size in bytes of a blob.
+ *
+ * @param value A pointer to the blob structure.
+ *
+ * @returns The size of the blob in bytes.
+ *
+ * @since October 2015
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
+int obi_blob_sizeof(Obi_blob_p value);
+
+
 #endif /* OBIBLOB_H_ */
 

src/obiblob_indexer.c

@@ -15,23 +15,43 @@
 
 #include "obiblob_indexer.h"
 #include "obidms.h"
-#include "obiavl.h"
+#include "obierrno.h"
+#include "obidebug.h"
 
 
 #define DEBUG_LEVEL 0	// TODO has to be defined somewhere else (cython compil flag?)
 
 
-inline int obi_indexer_exists(OBIDMS_p dms, const char* name);
+//inline int obi_indexer_exists(OBIDMS_p dms, const char* name);
 
-inline Obi_indexer_p obi_indexer(OBIDMS_p dms, const char* name);
+//inline Obi_indexer_p obi_indexer(OBIDMS_p dms, const char* name);
 
-inline Obi_indexer_p obi_create_indexer(OBIDMS_p dms, const char* name);
+//inline Obi_indexer_p obi_create_indexer(OBIDMS_p dms, const char* name);
 
-inline Obi_indexer_p obi_open_indexer(OBIDMS_p dms, const char* name);
+//inline Obi_indexer_p obi_open_indexer(OBIDMS_p dms, const char* name);
 
-inline int obi_close_indexer(Obi_indexer_p indexer);
+//inline int obi_close_indexer(Obi_indexer_p indexer);
 
-inline index_t obi_indexer_add(Obi_indexer_p indexer, Obi_blob_p value);
+//inline index_t obi_indexer_add(Obi_indexer_p indexer, Obi_blob_p value);
 
-inline Obi_blob_p obi_indexer_get(Obi_indexer_p indexer, index_t idx);
+//inline Obi_blob_p obi_indexer_get(Obi_indexer_p indexer, index_t idx);
+
+
+char* obi_build_indexer_name(const char* column_name, obiversion_t column_version)
+{
+	char* indexer_name;
+
+	indexer_name = (char*) malloc(INDEXER_MAX_NAME * sizeof(char));
+	if (indexer_name == NULL)
+	{
+		obi_set_errno(OBI_MALLOC_ERROR);
+		obidebug(1, "\nError allocating the memory for an indexer name");
+		return NULL;
+	}
+
+	strcpy(indexer_name, column_name);
+	sprintf(indexer_name+strlen(column_name), "_%d_indexer", column_version);
+
+	return indexer_name;
+}
 

src/obiblob_indexer.h

@@ -33,47 +33,126 @@ typedef OBIDMS_avl_group_p Obi_indexer_p; 		/**< Typedef to refer to the pointer
  	 	 	 	 	 	 	 	 	 	 	 	 */
 
 
-// TODO doc
-inline int obi_indexer_exists(OBIDMS_p dms, const char* name)
-{
-	return obi_avl_exists(dms, name);
-}
+/**
+ * @brief Checks if an indexer already exists or not.
+ *
+ * @param dms The OBIDMS to which the indexer belongs.
+ * @param name The name of the indexer.
+ *
+ * @returns A value indicating whether the indexer exists or not.
+ * @retval 1 if the indexer exists.
+ * @retval 0 if the indexer does not exist.
+ * @retval -1 if an error occurred.
+ *
+ * @since April 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
+#define obi_indexer_exists obi_avl_exists
 
 
-inline Obi_indexer_p obi_indexer(OBIDMS_p dms, const char* name)
-{
-	return obi_avl_group(dms, name);
-}
+/**
+ * @brief Opens an indexer and creates it if it does not already exist.
+ *
+ * @param dms The OBIDMS to which the indexer belongs.
+ * @param name The name of the indexer.
+ *
+ * @returns A pointer to the indexer structure.
+ * @retval NULL if an error occurred.
+ *
+ * @since April 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
+#define obi_indexer obi_avl_group
 
 
-inline Obi_indexer_p obi_create_indexer(OBIDMS_p dms, const char* name)
-{
-	return obi_create_avl_group(dms, name);
-}
+/**
+ * @brief Creates an indexer.
+ *
+ * @param dms The OBIDMS to which the indexer belongs.
+ * @param name The name of the indexer.
+ *
+ * @returns A pointer to the AVL tree group structure.
+ * @retval NULL if an error occurred.
+ *
+ * @since April 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
+#define obi_create_indexer obi_create_avl_group
 
 
-inline Obi_indexer_p obi_open_indexer(OBIDMS_p dms, const char* name)
-{
-	return obi_open_avl_group(dms, name);
-}
+/**
+ * @brief Opens an indexer.
+ *
+ * @param dms The OBIDMS to which the indexer belongs.
+ * @param name The name of the indexer.
+ *
+ * @returns A pointer to the indexer structure.
+ * @retval NULL if an error occurred.
+ *
+ * @since April 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
+#define obi_open_indexer obi_open_avl_group
 
 
-inline int obi_close_indexer(Obi_indexer_p indexer)
-{
-	return obi_close_avl_group(indexer);
-}
+/**
+ * @brief Closes an indexer.
+ *
+ * @param indexer A pointer to the indexer structure to close and free.
+ *
+ * @retval 0 if the operation was successfully completed.
+ * @retval -1 if an error occurred.
+ *
+ * @since April 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
+#define obi_close_indexer obi_close_avl_group
 
 
-inline index_t obi_indexer_add(Obi_indexer_p indexer, Obi_blob_p value)
-{
-	return obi_avl_group_add(indexer, value);
-}
+/**
+ * @brief Indexes a blob in an indexer and returns the index referring to the blob.
+ *
+ * @param indexer A pointer to the indexer.
+ * @param value The blob to index.
+ *
+ * @returns The index of the blob newly added in the indexer.
+ * @retval -1 if an error occurred.
+ *
+ * @since April 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
+#define obi_indexer_add obi_avl_group_add
 
 
-inline Obi_blob_p obi_indexer_get(Obi_indexer_p indexer, index_t idx)
-{
-	return obi_avl_group_get(indexer, idx);
-}
+/**
+ * @brief Recovers a blob from an indexer.
+ *
+ * @param indexer A pointer to the indexer.
+ * @param index The index of the blob in the indexer.
+ *
+ * @returns A pointer to the blob recovered.
+ *
+ * @since April 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
+#define obi_indexer_get obi_avl_group_get
+
+
+/**
+ * @brief Builds an indexer name in the form columnname_columnversion_indexer.
+ *
+ * @warning The returned pointer has to be freed by the caller.
+ *
+ * @param column_name The name of the column associated with the indexer.
+ * @param column_version The version of the column associated with the indexer.
+ *
+ * @returns A pointer on the indexer name built.
+ * @retval NULL if an error occurred.
+ *
+ * @since April 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
+char* obi_build_indexer_name(const char* column_name, obiversion_t column_version);
 
 
 #endif /* OBIBLOB_INDEXER_H_ */

src/obidms.c

@@ -420,7 +420,7 @@ OBIDMS_p obi_open_dms(const char* dms_name)
 	dms->opened_columns = (Opened_columns_list_p) malloc(sizeof(Opened_columns_list_t));
 	(dms->opened_columns)->nb_opened_columns = 0;
 
-	// Initialize the list of opened indexers	// TODO should be handled somewhere else?
+	// Initialize the list of opened indexers
 	dms->opened_indexers = (Opened_indexers_list_p) malloc(sizeof(Opened_indexers_list_t));
 	(dms->opened_indexers)->nb_opened_indexers = 0;
 
@@ -463,7 +463,7 @@ int obi_close_dms(OBIDMS_p dms)
 			free(dms);
 			return -1;
 		}
-		if (closedir(dms->indexer_directory) < 0)		// TODO should be handled somewhere else?
+		if (closedir(dms->indexer_directory) < 0)
 		{
 			obi_set_errno(OBI_INDEXER_ERROR);
 			obidebug(1, "\nError closing an indexer directory");
@@ -488,10 +488,10 @@ int obi_dms_is_column_name_in_list(OBIDMS_p dms, const char* column_name)
 	{
 		if (!strcmp(((*((columns_list->columns)+i))->header)->name, column_name))
 		{	// Found it
-			return 0;
+			return 1;
 		}
 	}
-	return 1;
+	return 0;
 }
 
 
@@ -512,7 +512,7 @@ OBIDMS_column_p obi_dms_get_column_from_list(OBIDMS_p dms, const char* column_na
 }
 
 
-void obi_dms_list_column(OBIDMS_p dms, OBIDMS_column_p column)
+void obi_dms_list_column(OBIDMS_p dms, OBIDMS_column_p column)	// TODO add check if column already in list?
 {
 	*(((dms->opened_columns)->columns)+((dms->opened_columns)->nb_opened_columns)) = column;
 	((dms->opened_columns)->nb_opened_columns)++;
@@ -551,7 +551,7 @@ Obi_indexer_p obi_dms_get_indexer_from_list(OBIDMS_p dms, const char* indexer_na
 
 	for (i=0; i < (indexers_list->nb_opened_indexers); i++)
 	{
-		if (!strcmp(((indexers_list->indexers)[i])->name, indexer_name))	// TODO it references something in AVL_group struct
+		if (!strcmp(((indexers_list->indexers)[i])->name, indexer_name))	// TODO get_name function indexer
 		{	// Found the indexer already opened, return it
 			return (indexers_list->indexers)[i];
 		}
@@ -577,7 +577,7 @@ int obi_dms_unlist_indexer(OBIDMS_p dms, Obi_indexer_p indexer)
 
 	for (i=0; i < indexers_list->nb_opened_indexers; i++)
 	{
-		if (!strcmp(((indexers_list->indexers)[i])->name, indexer->name))	// TODO it references something in AVL_group struct
+		if (!strcmp(((indexers_list->indexers)[i])->name, indexer->name))	// TODO get_name function indexer
 		{	// Found the indexer. Rearrange list
 			(indexers_list->nb_opened_indexers)--;
 			(indexers_list->indexers)[i] = (indexers_list->indexers)[indexers_list->nb_opened_indexers];
@@ -602,8 +602,8 @@ char* obi_dms_get_dms_path(OBIDMS_p dms)
 		return NULL;
 	}
 
-	if (getcwd(full_path, MAX_PATH_LEN) == NULL)	// TODO not sure at all about this because the DMS must be in the working directory.
-	{												// Maybe better to store when opening, but opening function seems to assume that too.
+	if (getcwd(full_path, MAX_PATH_LEN) == NULL)	// TODO store when opening
+	{
 		obi_set_errno(OBI_UTILS_ERROR);
 		obidebug(1, "\nError getting the path to a file or directory");
 		return NULL;

src/obidms.h

@@ -23,8 +23,7 @@
 #include <stdbool.h>
 
 #include "obierrno.h"
-//#include "obidmscolumn.h"
-//#include "obiblob_indexer.h"
+#include "obitypes.h"
 
 
 #define OBIDMS_MAX_NAME (2048) 	    			/**< The maximum length of an OBIDMS name.
@@ -42,26 +41,33 @@
  	 	 	 	 	 	 	 	 	 	 	 	 */
 
 
-typedef int32_t obiversion_t; 	/**< TODO double
-                                 */
+struct OBIDMS_column;							/**< Declarations to avoid circular dependencies. */
+typedef struct OBIDMS_column* OBIDMS_column_p;  /**< Declarations to avoid circular dependencies. */
 
-
-struct OBIDMS_column;		// TODO
-typedef struct OBIDMS_column* OBIDMS_column_p;
-
-typedef struct Opened_columns_list {		// TODO Handle the problem linked to columns with the same name + means only one version
-	int	 		 			nb_opened_columns;
-	OBIDMS_column_p		 	columns[MAX_NB_OPENED_COLUMNS];
+/**
+ * @brief Structure listing the columns opened in a DMS, identified by their name and version number.
+ */
+typedef struct Opened_columns_list {
+	int	 		 			nb_opened_columns;	  				/**< Number of opened columns.
+	 	   	   	   	   	   	   	   	   	   	   	   	   	   	   	 */
+	OBIDMS_column_p		 	columns[MAX_NB_OPENED_COLUMNS];	  	/**< Array of pointers on the opened columns.
+ 	   	   	   	   	   	   	 	 	 	 	 	 	 	 	 	 */
 } Opened_columns_list_t, *Opened_columns_list_p;
 
 
-struct OBIDMS_avl_group;		// TODO
-typedef struct OBIDMS_avl_group* OBIDMS_avl_group_p;
-typedef OBIDMS_avl_group_p Obi_indexer_p;
+// TODO Need to find a way to not refer to AVLs specifically
+struct OBIDMS_avl_group;								/**< Declarations to avoid circular dependencies. */
+typedef struct OBIDMS_avl_group* OBIDMS_avl_group_p;	/**< Declarations to avoid circular dependencies. */
+typedef OBIDMS_avl_group_p Obi_indexer_p;				/**< Declarations to avoid circular dependencies. */
 
+/**
+ * @brief Structure listing the indexers opened in a DMS, identified by their name.
+ */
 typedef struct Opened_indexers_list {
-	int		 		 		nb_opened_indexers;
-	Obi_indexer_p		    indexers[MAX_NB_OPENED_INDEXERS];
+	int		 		 		nb_opened_indexers;					/**< Number of opened indexers.
+	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 */
+	Obi_indexer_p		    indexers[MAX_NB_OPENED_INDEXERS];	/**< Array of pointers on the opened indexers.
+	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 */
 } Opened_indexers_list_t, *Opened_indexers_list_p;
 
 
@@ -81,10 +87,10 @@ typedef struct OBIDMS {
 	int     				dir_fd;								/**< The file descriptor of the directory entry
 																 *   usable to refer and scan the database directory.
 																 */
-	DIR*					indexer_directory;						/**< A directory entry usable to
+	DIR*					indexer_directory;					/**< A directory entry usable to
 																 *   refer and scan the indexer directory.
 																 */
-	int						indexer_dir_fd;							/**< The file descriptor of the directory entry
+	int						indexer_dir_fd;						/**< The file descriptor of the directory entry
 																 *   usable to refer and scan the indexer directory.
 																 */
 	bool					little_endian;						/**< Endianness of the database.
@@ -101,7 +107,7 @@ typedef struct OBIDMS {
  *
  * @param dms_name A pointer to a C string containing the name of the database.
  *
- * @returns An integer value indicating the status of the database
+ * @returns An integer value indicating the status of the database.
  * @retval 1 if the database exists.
  * @retval 0 if the database does not exist.
  * @retval -1 if an error occurred.
@@ -186,60 +192,138 @@ OBIDMS_p obi_dms(const char* dms_name);
 int obi_close_dms(OBIDMS_p dms);
 
 
-// TODO doc
+/**
+ * @brief Checks if a column with a given name is in the list of opened columns.
+ *
+ * @warning: Checking only the name means that the column can be of any version.
+ *
+ * @param dms The OBIDMS.
+ * @param column_name The column name that should be looked for.
+ *
+ * @returns An integer value indicating whether there is at least one column with that name
+ * 			in the list of opened columns.
+ * @retval 0 if there is at least one column with that name in the list of opened columns.
+ * @retval 1 if there is no column with that name in the list of opened columns.
+ *
+ * @since April 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
 int obi_dms_is_column_name_in_list(OBIDMS_p dms, const char* column_name);
 
 
+/**
+ * @brief Returns a column identified by its name and its version number from the list of opened columns.
+ *
+ * @param dms The OBIDMS.
+ * @param column_name The column name that should be looked for.
+ * @param version The version number of the column that should be looked for.
+ *
+ * @returns A pointer on the column if it was found in the list of opened columns.
+ * @retval NULL if the column was not found in the list of opened columns.
+ *
+ * @since April 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
 OBIDMS_column_p obi_dms_get_column_from_list(OBIDMS_p dms, const char* column_name, obiversion_t version);
 
 
+/**
+ * @brief Adds a column identified by its name and its version number in the list of opened columns.
+ *
+ * @param dms The OBIDMS.
+ * @param column A pointer on the column that should be added in the list of opened columns.
+ *
+ * @since April 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
 void obi_dms_list_column(OBIDMS_p dms, OBIDMS_column_p column);
 
 
+/**
+ * @brief Removes a column identified by its name and its version number from the list of opened columns.
+ *
+ * @param dms The OBIDMS.
+ * @param column A pointer on the column that should be removed from the list of opened columns.
+ *
+ * @since April 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
 int obi_dms_unlist_column(OBIDMS_p dms, OBIDMS_column_p column);
 
 
+/**
+ * @brief Returns an indexer identified by its name from the list of opened indexers.
+ *
+ * @param dms The OBIDMS.
+ * @param indexer_name The indexer name that should be looked for.
+ *
+ * @returns A pointer on the indexer if it was found in the list of opened indexers.
+ * @retval NULL if the indexer was not found in the list of opened indexers.
+ *
+ * @since April 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
 Obi_indexer_p obi_dms_get_indexer_from_list(OBIDMS_p dms, const char* indexer_name);
 
 
+/**
+ * @brief Adds an indexer identified by its name in the list of opened indexers.
+ *
+ * @param dms The OBIDMS.
+ * @param indexer A pointer on the indexer that should be added in the list of opened indexers.
+ *
+ * @since April 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
 void obi_dms_list_indexer(OBIDMS_p dms, Obi_indexer_p indexer);
 
 
+/**
+ * @brief Removes an indexer identified by its name from the list of opened indexers.
+ *
+ * @param dms The OBIDMS.
+ * @param column A pointer on the indexer that should be removed from the list of opened indexers.
+ *
+ * @since April 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
 int obi_dms_unlist_indexer(OBIDMS_p dms, Obi_indexer_p indexer);
 
 
+/**
+ * Function meant to disappear soon
+ */
 char* obi_dms_get_path(OBIDMS_p dms);
 
 
-/**	TODO
- * @brief Internal function getting the full path of a file or a directory from its
- *        path relative to a directory file descriptor.
+/**
+ * @brief Gets the full path of a file or a directory from its
+ *        path relative to the DMS.
  *
  * @warning The returned pointer has to be freed by the caller.
  *
- * @param directory_file_descriptor The file descriptor for the directory to which
- * 									path_name is relative.
- * @param path_name The path name for the file or directory, relative to directory_file_descriptor.
+ * @param dms The DMS to which path_name is relative.
+ * @param path_name The path name for the file or directory, relative to the DMS.
  *
  * @returns A pointer to the full path.
  * @retval NULL if an error occurred.
  *
- * @since June 2015
+ * @since April 2016
  * @author Celine Mercier (celine.mercier@metabarcoding.org)
  */
 char* obi_dms_get_full_path(OBIDMS_p dms, const char* path_name);
 
 
 /**
- * @brief Replacement function for opendirat() : open a directory relative to a directory file descriptor.
+ * @brief Opens a directory relative to the DMS.
  *
- * @param directory_file_descriptor The file descriptor for the directory in which the directory should be opened.
- * @param path_name The path name for the directory to be opened, relative to directory_file_descriptor.
+ * @param dms The DMS to which path_name is relative.
+ * @param path_name The path name for the directory to be opened, relative to the DMS.
  *
- * @returns The file descriptor of the opened directory.
+ * @returns The directory stream of the opened directory.
  * @retval NULL if an error occurred.
  *
- * @since June 2015
+ * @since April 2016
  * @author Celine Mercier (celine.mercier@metabarcoding.org)
  */
 DIR* opendir_in_dms(OBIDMS_p dms, const char* path_name);

src/obidmscolumn.c

@@ -161,7 +161,14 @@ static char* build_column_file_name(const char* column_name, obiversion_t versio
 
 	// Build the file name
 	version_number_length = (version_number == 0 ? 1 : (int)(log10(version_number)+1));
-	file_name =	(char*) malloc((strlen(column_name) + version_number_length + 6)*sizeof(char));	// TODO check the mallocs...
+	file_name =	(char*) malloc((strlen(column_name) + version_number_length + 6)*sizeof(char));
+	if (file_name == NULL)
+	{
+		obi_set_errno(OBI_MALLOC_ERROR);
+		obidebug(1, "\nError allocating the memory for a column file name");
+		return NULL;
+	}
+
 	if (sprintf(file_name,"%s@%d.odc", column_name, version_number) < 0)
 	{
 		obi_set_errno(OBICOL_MEMORY_ERROR);
@@ -179,6 +186,13 @@ static char* build_version_file_name(const char* column_name)
 
 	// Build the file name
 	file_name =	(char*) malloc((strlen(column_name) + 5)*sizeof(char));
+	if (file_name == NULL)
+	{
+		obi_set_errno(OBI_MALLOC_ERROR);
+		obidebug(1, "\nError allocating the memory for a version file name");
+		return NULL;
+	}
+
 	if (sprintf(file_name,"%s.odv", column_name) < 0)
 	{
 		obi_set_errno(OBICOL_MEMORY_ERROR);
@@ -535,7 +549,7 @@ OBIDMS_column_p obi_create_column(OBIDMS_p    dms,
 	index_t						minimum_line_count;
 	OBIType_t  					returned_data_type;
 	OBIType_t  					stored_data_type;
-	char						final_indexer_name[INDEXER_MAX_NAME];
+	char*			    		final_indexer_name;
 
 	new_column = NULL;
 
@@ -555,13 +569,38 @@ OBIDMS_column_p obi_create_column(OBIDMS_p    dms,
 		obidebug(1, "\nCan't create column because of invalid data type");
 		return NULL;
 	}
-	if (((data_type == OBI_STR) || (data_type == OBI_SEQ)) && (strcmp(indexer_name, "") == 0))
-	{	// TODO make function in obiblob_indexer.c and discuss the fact that it's here
-		strcpy(final_indexer_name, column_name);
-		strcat(final_indexer_name, "_indexer");
+
+	// Get the column directory structure associated to the column
+	column_directory = obi_column_directory(dms, column_name);
+	if (column_directory == NULL)
+	{
+		obi_set_errno(OBICOLDIR_UNKNOWN_ERROR);
+		obidebug(1, "\nError opening a column directory structure");
+		return NULL;
+	}
+
+	// Get the latest version number
+	version_number = obi_get_new_version_number(column_directory, true);
+	if (version_number < 0)
+	{
+		return NULL;
+	}
+
+	// Build the indexer name if needed
+	if ((data_type == OBI_STR) || (data_type == OBI_SEQ))
+	{
+		if (strcmp(indexer_name, "") == 0)
+		{
+			final_indexer_name = obi_build_indexer_name(column_name, version_number);
+			if (final_indexer_name == NULL)
+				return NULL;
+		}
+		else
+		{
+			final_indexer_name = (char*) malloc((strlen(indexer_name)+1)*sizeof(char));
+			strcpy(final_indexer_name, indexer_name);
+		}
 	}
-	else
-		strcpy(final_indexer_name, indexer_name);
 
 	returned_data_type = data_type;
 	if ((data_type == OBI_STR) || (data_type == OBI_SEQ))
@@ -609,27 +648,11 @@ OBIDMS_column_p obi_create_column(OBIDMS_p    dms,
 		return NULL;
 	}
 
-	// Get the column directory structure associated to the column
-	column_directory = obi_column_directory(dms, column_name);
-	if (column_directory == NULL)
-	{
-		obi_set_errno(OBICOLDIR_UNKNOWN_ERROR);
-		obidebug(1, "\nError opening a column directory structure");
-		return NULL;
-	}
-
 	// Calculate the size needed
 	header_size = obi_get_platform_header_size();
 	data_size = obi_array_sizeof(stored_data_type, nb_lines, nb_elements_per_line);
 	file_size = header_size + data_size;
 
-	// Get the latest version number
-	version_number = obi_get_new_version_number(column_directory, true);
-	if (version_number < 0)
-	{
-		return NULL;
-	}
-
 	// Get the column file name
 	column_file_name = build_column_file_name(column_name, version_number);
 	if (column_file_name == NULL)
@@ -773,7 +796,7 @@ OBIDMS_column_p obi_open_column(OBIDMS_p dms,
 	column_directory = obi_open_column_directory(dms, column_name);
 	if (column_directory == NULL)
 	{
-		//obidebug(1, "\nError opening a column directory structure");	// TODO
+		obidebug(1, "\nError opening a column directory structure");
 		return NULL;
 	}
 
@@ -809,7 +832,7 @@ OBIDMS_column_p obi_open_column(OBIDMS_p dms,
 	if (column_file_descriptor < 0)
 	{
 		obi_set_errno(OBICOL_UNKNOWN_ERROR);
-		//obidebug(1, "\nError opening column file");
+		obidebug(1, "\nError opening column file");
 		free(column_file_name);
 		return NULL;
 	}
@@ -842,7 +865,7 @@ OBIDMS_column_p obi_open_column(OBIDMS_p dms,
 	column->header 			 = mmap(NULL,
 			                  	  	header_size,
 			                  	  	PROT_READ,
-			                  	  	MAP_PRIVATE,
+			                  	  	MAP_SHARED,
 			                  	  	column_file_descriptor,
 			                  	  	0
 			                 	   );
@@ -860,7 +883,7 @@ OBIDMS_column_p obi_open_column(OBIDMS_p dms,
 	column->data   			 = mmap(NULL,
 			                 	 	(column->header)->data_size,
 			                 	 	PROT_READ,
-									MAP_PRIVATE,
+									MAP_SHARED,
 			                 	 	column_file_descriptor,
 			                 	 	header_size
 			                	   );
@@ -1251,8 +1274,18 @@ void obi_ini_to_NA_values(OBIDMS_column_p column,
 					}
 					break;
 
-	case OBI_STR:   // fallthrough
-	case OBI_SEQ:   // fallthrough
+	case OBI_STR:   for (i=start;i<end;i++)
+					{
+						*(((index_t*) (column->data)) + i) = OBIIdx_NA;
+					}
+					break;
+
+	case OBI_SEQ:   for (i=start;i<end;i++)
+					{
+						*(((index_t*) (column->data)) + i) = OBIIdx_NA;
+					}
+					break;
+
 	case OBI_IDX:   for (i=start;i<end;i++)
 					{
 						*(((index_t*) (column->data)) + i) = OBIIdx_NA;
@@ -1319,12 +1352,12 @@ OBIDMS_column_header_p obi_column_get_header_from_name(OBIDMS_p dms, const char*
 
 	// Fill the header structure
 	header = mmap(NULL,
-			 header_size,
-			 PROT_READ,
-			 MAP_SHARED,
-			 column_file_descriptor,
-			 0
-			 );
+			 	  header_size,
+				  PROT_READ,
+				  MAP_SHARED,
+				  column_file_descriptor,
+				  0
+			 	 );
 
 	if (header == MAP_FAILED)
 	{
@@ -1438,29 +1471,3 @@ int obi_column_prepare_to_get_value(OBIDMS_column_p column, index_t line_nb)
 	return 0;
 }
 
-
-char* obi_column_format_date(time_t date)	// TODO put in utils.c
-{
-	char*       formatted_time;
-	struct tm*  tmp;
-
-	formatted_time = (char*) malloc(FORMATTED_TIME_LENGTH*sizeof(char));
-
-	tmp = localtime(&date);
-	if (tmp == NULL)
-	{
-		obi_set_errno(OBICOL_UNKNOWN_ERROR);
-		obidebug(1, "\nError formatting a date");
-		return NULL;
-	}
-
-	if (strftime(formatted_time, FORMATTED_TIME_LENGTH, "%c", tmp) == 0)
-	{
-		obi_set_errno(OBICOL_UNKNOWN_ERROR);
-		obidebug(1, "\nError formatting a date");
-		return NULL;
-	}
-
-	return formatted_time;
-}
-

src/obidmscolumn.h

@@ -34,14 +34,9 @@
                                 	   	 */
 #define MAXIMUM_LINE_COUNT (1000000000) /**< The maximum line count for the data of a column. //TODO
                                 	     */
-#define FORMATTED_TIME_LENGTH (1024)    /**< The length allocated for the character string containing a formatted date.
- 	 	 	 	 	 	 	 	 	     */
 #define COMMENTS_MAX_LENGTH (2048)      /**< The maximum length for comments.
  	 	 	 	 	 	 	 	 	     */
 
-typedef int32_t obiversion_t; 	/**< Used to store the column version number
-                                 */
-
 
 /**
  * @brief OBIDMS column header structure.
@@ -329,27 +324,34 @@ int obi_close_header(OBIDMS_column_header_p header);
 index_t obi_column_get_element_index_from_name(OBIDMS_column_p column, const char* element_name);
 
 
-// TODO doc
+/**
+ * @brief Prepares a column to set a value.
+ *
+ * @param column A pointer on an OBIDMS column.
+ * @param line_nb The number of the line at which the value will be set.
+ *
+ * @retval 0 if the operation was successfully completed.
+ * @retval -1 if an error occurred.
+ *
+ * @since April 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
 int obi_column_prepare_to_set_value(OBIDMS_column_p column, index_t line_nb);
 
 
-// TODO doc
+/**
+ * @brief Prepares a column to recover a value.
+ *
+ * @param column A pointer on an OBIDMS column.
+ * @param line_nb The number of the line at which the value will be recovered.
+ *
+ * @retval 0 if the operation was successfully completed.
+ * @retval -1 if an error occurred.
+ *
+ * @since April 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
 int obi_column_prepare_to_get_value(OBIDMS_column_p column, index_t line_nb);
 
 
-/** TODO put in utils.c
- * @brief Formats a date in a way that is easy to read.
- *
- * @warning The pointer returned must be freed by the caller.
- *
- * @param date A date.
- *
- * @returns The date formatted in a way that is easy to read.
- *
- * @since October 2015
- * @author Celine Mercier (celine.mercier@metabarcoding.org)
- */
-char* obi_column_format_date(time_t date);
-
-
 #endif /* OBIDMSCOLUMN_H_ */

src/obidmscolumn_bool.c

@@ -15,11 +15,6 @@
 
 #include "obidmscolumn.h"
 #include "obitypes.h"
-#include "obierrno.h"
-#include "obidebug.h"
-
-
-#define DEBUG_LEVEL 0	// TODO has to be defined somewhere else (cython compil flag?)
 
 
 /**********************************************************************

src/obidmscolumn_char.c

@@ -15,11 +15,6 @@
 
 #include "obidmscolumn.h"
 #include "obitypes.h"
-#include "obierrno.h"
-#include "obidebug.h"
-
-
-#define DEBUG_LEVEL 0	// TODO has to be defined somewhere else (cython compil flag?)
 
 
 /**********************************************************************

src/obidmscolumn_float.c

@@ -15,11 +15,6 @@
 
 #include "obidmscolumn.h"
 #include "obitypes.h"
-#include "obierrno.h"
-#include "obidebug.h"
-
-
-#define DEBUG_LEVEL 0	// TODO has to be defined somewhere else (cython compil flag?)
 
 
 /**********************************************************************

src/obidmscolumn_idx.c

@@ -15,11 +15,6 @@
 
 #include "obidmscolumn.h"
 #include "obitypes.h"
-#include "obierrno.h"
-#include "obidebug.h"
-
-
-#define DEBUG_LEVEL 0	// TODO has to be defined somewhere else (cython compil flag?)
 
 
 /**********************************************************************

src/obidmscolumn_idx.h

@@ -7,6 +7,9 @@
  * @author Celine Mercier
  * @date February 14th 2016
  * @brief Header file for the functions handling OBIColumns containing data with the OBIType OBI_IDX.
+ *
+ * Note: OBI_IDX columns contain indices referring to data stored elsewhere
+ *		 (for example lines in other columns) and contain only one element (index) per line.
  */
 
 
@@ -20,10 +23,44 @@
 #include "obidmscolumn.h"
 #include "obitypes.h"
 
-// TODO doc
 
+/**
+ * @brief Sets a value in an OBIDMS column containing data with the type OBI_IDX.
+ *
+ *	Note: OBI_IDX columns contain indices referring to data stored elsewhere
+ *		  (for example lines in other columns) and contain only one element (index) per line.
+ *
+ * @warning Pointers returned by obi_open_column() don't allow writing.
+ *
+ * @param column A pointer as returned by obi_create_column() or obi_clone_column().
+ * @param line_nb The number of the line where the value should be set.
+ * @param value The value that should be set.
+ *
+ * @returns An integer value indicating the success of the operation.
+ * @retval 0 on success.
+ * @retval -1 if an error occurred.
+ *
+ * @since February 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
 int obi_column_set_index(OBIDMS_column_p column, index_t line_nb, index_t value);
 
+
+/**
+ * @brief Recovers a value in an OBIDMS column containing data with the type OBI_IDX.
+ *
+ *	Note: OBI_IDX columns contain indices referring to data stored elsewhere
+ *		  (for example lines in other columns) and contain only one element (index) per line.
+ *
+ * @param column A pointer as returned by obi_create_column() or obi_clone_column().
+ * @param line_nb The number of the line where the value should be recovered.
+ *
+ * @returns The recovered value.
+ * @retval OBIIdx_NA the NA value of the type if an error occurred and obi_errno is set.
+ *
+ * @since February 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
 index_t obi_column_get_index(OBIDMS_column_p column, index_t line_nb);
 
 

src/obidmscolumn_int.c

@@ -15,11 +15,6 @@
 
 #include "obidmscolumn.h"
 #include "obitypes.h"
-#include "obierrno.h"
-#include "obidebug.h"
-
-
-#define DEBUG_LEVEL 0	// TODO has to be defined somewhere else (cython compil flag?)
 
 
 /**********************************************************************

src/obidmscolumn_seq.c

@@ -15,14 +15,9 @@
 
 #include "obidmscolumn.h"
 #include "obitypes.h"
-#include "obierrno.h"
-#include "obidebug.h"
 #include "dna_seq_indexer.h"
 
 
-#define DEBUG_LEVEL 0	// TODO has to be defined somewhere else (cython compil flag?)
-
-
 /**********************************************************************
  *
  * D E F I N I T I O N   O F   T H E   P U B L I C   F U N C T I O N S

src/obidmscolumn_seq.h

@@ -19,7 +19,6 @@
 
 #include "obidmscolumn.h"
 #include "obitypes.h"
-#include "obiview.h"
 
 
 /**
@@ -52,7 +51,7 @@ int obi_column_set_obiseq_with_elt_idx(OBIDMS_column_p column, index_t line_nb,
  * @param element_idx The index of the element that should be recovered in the line.
  *
  * @returns The recovered value.
- * @retval '\0' the NA value of the type if an error occurred and obi_errno is set.
+ * @retval OBISeq_NA the NA value of the type if an error occurred and obi_errno is set.
  *
  * @since November 2015
  * @author Celine Mercier (celine.mercier@metabarcoding.org)
@@ -90,7 +89,7 @@ int obi_column_set_obiseq_with_elt_name(OBIDMS_column_p column, index_t line_nb,
  * @param element_name The name of the element that should be recovered in the line.
  *
  * @returns The recovered value.
- * @retval '\0' the NA value of the type if an error occurred and obi_errno is set.
+ * @retval OBISeq_NA the NA value of the type if an error occurred and obi_errno is set.
  *
  * @since November 2015
  * @author Celine Mercier (celine.mercier@metabarcoding.org)

src/obidmscolumn_str.c

@@ -15,14 +15,9 @@
 
 #include "obidmscolumn.h"
 #include "obitypes.h"
-#include "obierrno.h"
-#include "obidebug.h"
 #include "char_str_indexer.h"
 
 
-#define DEBUG_LEVEL 0	// TODO has to be defined somewhere else (cython compil flag?)
-
-
 /**********************************************************************
  *
  * D E F I N I T I O N   O F   T H E   P U B L I C   F U N C T I O N S

src/obidmscolumn_str.h

@@ -19,7 +19,6 @@
 
 #include "obidmscolumn.h"
 #include "obitypes.h"
-#include "obiview.h"
 
 
 /**
@@ -52,7 +51,7 @@ int obi_column_set_obistr_with_elt_idx(OBIDMS_column_p column, index_t line_nb,
  * @param element_idx The index of the element that should be recovered in the line.
  *
  * @returns The recovered value.
- * @retval '\0' the NA value of the type if an error occurred and obi_errno is set.
+ * @retval OBIStr_NA the NA value of the type if an error occurred and obi_errno is set.
  *
  * @since October 2015
  * @author Celine Mercier (celine.mercier@metabarcoding.org)
@@ -90,7 +89,7 @@ int obi_column_set_obistr_with_elt_name(OBIDMS_column_p column, index_t line_nb,
  * @param element_name The name of the element that should be recovered in the line.
  *
  * @returns The recovered value.
- * @retval '\0' the NA value of the type if an error occurred and obi_errno is set.
+ * @retval OBIStr_NA the NA value of the type if an error occurred and obi_errno is set.
  *
  * @since October 2015
  * @author Celine Mercier (celine.mercier@metabarcoding.org)

src/obitypes.c

@@ -40,8 +40,12 @@ size_t obi_sizeof(OBIType_t type)
 	case OBI_CHAR:  size = sizeof(obichar_t);
 					break;
 
-	case OBI_STR:   // fallthrough
-	case OBI_SEQ:   // fallthrough
+	case OBI_STR:	size = sizeof(index_t);
+					break;
+
+	case OBI_SEQ:	size = sizeof(index_t);
+					break;
+
 	case OBI_IDX:	size = sizeof(index_t);
 					break;
 

src/obitypes.h

@@ -21,10 +21,9 @@
 #define OBIInt_NA (INT32_MIN)		/**< NA value for the type OBI_INT */
 #define OBIIdx_NA (INT64_MIN)		/**< NA value for indices */
 #define OBIFloat_NA (float_NA())	/**< NA value for the type OBI_FLOAT */
-#define OBIChar_NA (0)				/**< NA value for the type OBI_CHAR */
-										 // TODO not sure about this one as it can be impossible to distinguish from uninitialized values
-#define OBISeq_NA ("\0")			// TODO
-#define OBIStr_NA ("\0")			// TODO
+#define OBIChar_NA (0)				/**< NA value for the type OBI_CHAR */ // TODO not sure about this one as it can be impossible to distinguish from uninitialized values
+#define OBISeq_NA ("\0")			/**< NA value for the type OBI_SEQ */  // TODO discuss
+#define OBIStr_NA ("\0")			/**< NA value for the type OBI_STR */  // TODO discuss
 
 
 /**
@@ -48,19 +47,23 @@ typedef enum OBIType {
 	OBI_CHAR,					/**< a character (C type : char) */
 	OBI_STR,				    /**< an index in a data structure (C type : int64_t) referring to a character string */
 	OBI_SEQ,					/**< an index in a data structure (C type : int64_t) referring to a DNA sequence */
-	OBI_IDX					    /**< an index referring to a line in another column (C type : int64_t) */	// TODO delete?
+	OBI_IDX					    /**< an index referring to a line in another column (C type : int64_t) */
 } OBIType_t, *OBIType_p;
 
 
-typedef int64_t index_t;
+/**
+ * Typedefs for the OBITypes.
+ */
 typedef int32_t obiint_t;
 typedef double obifloat_t;
 typedef char obichar_t;
-// TODO same for obistr_t and obiseq_t ?
+typedef int64_t index_t;
 
+typedef char byte_t;			/**< Defining byte type.
+ 	 	 	 	 	 	 	 	 */
 
-typedef char byte_t;		/**< Defining byte type.
- 	 	 	 	 	 	 	 */
+typedef int32_t obiversion_t;	/**< Defining type for version numbers.
+	 	 	 	 	 	 	 	 */
 
 
 /**
@@ -104,7 +107,7 @@ size_t obi_sizeof(OBIType_t type);
  * @brief Returns the size required to store an array of elements with an OBIType.
  *
  * The returned size is large enough to store an array large enough
- * to store all the elements but rounded at a multiple of the memory page size.				// TODO this might be redundant but still a good thing
+ * to store all the elements but rounded at a multiple of the memory page size.
  *
  * @param data_type The OBIType of the elements.
  * @param nb_lines The number of lines to be stored.

src/obiview.c

@@ -10,20 +10,15 @@
  */
 
 
-// TODO
 #include <stdlib.h>
 #include <stdio.h>
 #include <stdbool.h>
 #include <fcntl.h>
 #include <sys/mman.h>
 
-#include "obidms.h"
 #include "obiview.h"
-#include "obierrno.h"
-#include "obidebug.h"
+#include "obidms.h"
 #include "obidmscolumn.h"
-#include "utils.h"
-#include "obilittlebigman.h"
 #include "obidmscolumn_idx.h"
 #include "obidmscolumn_bool.h"
 #include "obidmscolumn_char.h"
@@ -31,6 +26,10 @@
 #include "obidmscolumn_int.h"
 #include "obidmscolumn_seq.h"
 #include "obidmscolumn_str.h"
+#include "obierrno.h"
+#include "obidebug.h"
+#include "obilittlebigman.h"
+#include "utils.h"
 
 
 #define DEBUG_LEVEL 0	// TODO has to be defined somewhere else (cython compil flag?)
@@ -59,7 +58,7 @@
 static char* build_obiview_file_name();
 
 
-/**	TODO public
+/**	TODO public?
  * Internal function creating the file containing basic informations on the OBIDMS.
  *
  * This file contains:
@@ -79,12 +78,79 @@ static char* build_obiview_file_name();
 int create_obiview_file(int dms_file_descriptor);
 
 
-// TODO doc
+/**
+ * @brief Internal function to update the line count in the context of a view.
+ *
+ * All columns of the view are enlarged to contain at least the new line count.
+ *
+ * @warning The view must be writable.
+ *
+ * @param view A pointer on the view.
+ * @param line_count The new line count.
+ *
+ * @returns A value indicating the success of the operation.
+ * @retval 0 if the operation was successfully completed.
+ * @retval -1 if an error occurred.
+ *
+ * @since February 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
+int update_lines(Obiview_p view, index_t line_count);
+
+
+/**
+ * @brief Internal function to clone a column in the context of a view.
+ *
+ * Clones with the right line selection and replaces the cloned columns with the new ones in the view.
+ * If there is a line selection, all columns have to be cloned, otherwise only the column of interest is cloned.
+ *
+ * @param view A pointer on the view.
+ * @param column_name The name of the column in the view that should be cloned.
+ *
+ * @returns A pointer on the new column.
+ * @retval NULL if an error occurred.
+ *
+ * @since February 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
+OBIDMS_column_p clone_column_in_view(Obiview_p view, const char* column_name);
+
+
+/**
+ * Internal function preparing to set a value in a column, in the context of a view.
+ *
+ * The function checks that the view is not read-only, clones the column or all columns if needed,
+ * and updates the line count if needed.
+ *
+ * @param view The view.
+ * @param column_pp A pointer on the pointer on the column, to allow replacing the column if it is cloned.
+ * @param line_nb_p A pointer on the index of the line that will be modified, to allow replacing it if needed.
+ *
+ * @retval 0 if the operation was successfully completed.
+ * @retval -1 if an error occurred.
+ *
+ * @since April 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
 int prepare_to_set_value_in_column(Obiview_p view, OBIDMS_column_p* column_pp, index_t* line_nb_p);
 
 
-// TODO doc
-int prepare_to_get_value_in_column(Obiview_p view, OBIDMS_column_p column, index_t* line_nb_p);
+/**
+ * Internal function preparing to get a value from a column, in the context of a view.
+ *
+ * The function checks that the line index is not beyond the current line count of the view,
+ * and modifies it if there is a line selection associated with the view.
+ *
+ * @param view The view.
+ * @param line_nb_p A pointer on the index of the line, to allow replacing it if needed.
+ *
+ * @retval 0 if the operation was successfully completed.
+ * @retval -1 if an error occurred.
+ *
+ * @since April 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
+int prepare_to_get_value_from_column(Obiview_p view, index_t* line_nb_p);
 
 
 /************************************************************************
@@ -99,6 +165,12 @@ static char* build_obiview_file_name()
 
 	// Build file name
 	file_name = (char*) malloc((strlen(OBIVIEW_FILE_NAME) + 1)*sizeof(char));
+	if (file_name == NULL)
+	{
+		obi_set_errno(OBI_MALLOC_ERROR);
+		obidebug(1, "\nError allocating memory for a view file name");
+		return NULL;
+	}
 	if (sprintf(file_name, OBIVIEW_FILE_NAME) < 0)
 	{
 		obi_set_errno(OBIVIEW_ERROR);
@@ -147,7 +219,6 @@ int create_obiview_file(int dms_file_descriptor)
 	char* 				file_name;
 	int 				obiview_file_descriptor;
 	size_t  			header_size;
-//	size_t  			view_size;
 	size_t  			file_size;
 	Obiviews_header_p 	header;
 
@@ -170,8 +241,7 @@ int create_obiview_file(int dms_file_descriptor)
 
 	// Truncate file to the right size
 	header_size = get_platform_header_size();
-	//view_size = get_platform_views_size(1);
-	file_size = header_size; // + view_size;
+	file_size = header_size;
 
 	if (ftruncate(obiview_file_descriptor, file_size) < 0)
 	{
@@ -217,6 +287,88 @@ int create_obiview_file(int dms_file_descriptor)
 }
 
 
+int update_lines(Obiview_p view, index_t line_count)
+{
+	int i;
+
+	for (i=0; i<(view->column_count); i++)
+	{
+		while (line_count > (((view->columns)[i])->header)->line_count)
+		{
+			// Enlarge the column
+			if (obi_enlarge_column((view->columns)[i]) < 0)
+		    	return -1;
+		}
+	}
+
+	view->line_count = line_count;
+
+	return 0;
+}
+
+
+OBIDMS_column_p clone_column_in_view(Obiview_p view, const char* column_name)
+{
+	int i;
+	OBIDMS_column_p current_line_selection = NULL;
+	OBIDMS_column_p column;
+	OBIDMS_column_p column_buffer;
+
+	// Check that the view is not read-only
+	if (view->read_only)
+	{
+		obi_set_errno(OBIVIEW_ERROR);
+		obidebug(1, "\nError trying to delete a column in a read-only view");
+		return NULL;
+	}
+
+	if (view->new_line_selection != NULL)	// TODO Probably shouldn't happen, trigger error?
+		current_line_selection = view->new_line_selection;
+	else
+		current_line_selection = view->line_selection;
+
+	for (i=0; i<(view->column_count); i++)
+	{
+		if ((current_line_selection != NULL) || (!(strcmp((((view->columns)[i])->header)->name, column_name))))
+		{ // Clone with the right line selection and replace (for all columns if there is a line selection)
+
+			// Save pointer to close column after cloning
+			column_buffer = (view->columns)[i];
+
+			// Clone and replace the column in the view
+			(view->columns)[i] = obi_clone_column(view->dms, current_line_selection, (((view->columns)[i])->header)->name, (((view->columns)[i])->header)->version, 1);
+			if ((view->columns)[i] == NULL)
+			{
+				obi_set_errno(OBIVIEW_ERROR);
+				obidebug(1, "\nError cloning a column to replace in a view");
+				return NULL;
+			}
+
+			// Close old cloned column
+			obi_close_column(column_buffer);
+
+			if (!(strcmp((((view->columns)[i])->header)->name, column_name)))
+				// Found the column to return
+				column = (view->columns)[i];
+		}
+	}
+
+	// Close old line selections
+	if (view->line_selection != NULL)
+	{
+		obi_close_column(view->line_selection);
+		view->line_selection = NULL;
+	}
+	if (view->new_line_selection != NULL)
+	{
+		obi_close_column(view->new_line_selection);
+		view->new_line_selection = NULL;
+	}
+
+	return column;
+}
+
+
 int prepare_to_set_value_in_column(Obiview_p view, OBIDMS_column_p* column_pp, index_t* line_nb_p)
 {
 	// Check that the view is not read-only
@@ -235,7 +387,7 @@ int prepare_to_set_value_in_column(Obiview_p view, OBIDMS_column_p* column_pp, i
 		if (view->line_selection != NULL)
 			(*line_nb_p) = *(((index_t*) ((view->line_selection)->data)) + (*line_nb_p));
 
-		(*column_pp) = obi_view_clone_column(view, ((*column_pp)->header)->name);
+		(*column_pp) = clone_column_in_view(view, ((*column_pp)->header)->name);
 		if ((*column_pp) == NULL)
 		{
 			obidebug(1, "\nError trying to clone a column to modify it");
@@ -245,7 +397,7 @@ int prepare_to_set_value_in_column(Obiview_p view, OBIDMS_column_p* column_pp, i
 
 	if (((*line_nb_p)+1) > view->line_count)
 	{
-		if (obi_view_update_lines(view, ((*line_nb_p)+1)) < 0)
+		if (update_lines(view, ((*line_nb_p)+1)) < 0)
 			return -1;
 	}
 
@@ -253,7 +405,7 @@ int prepare_to_set_value_in_column(Obiview_p view, OBIDMS_column_p* column_pp, i
 }
 
 
-int prepare_to_get_value_from_column(Obiview_p view, OBIDMS_column_p column, index_t* line_nb_p)
+int prepare_to_get_value_from_column(Obiview_p view, index_t* line_nb_p)
 {
 	if (((*line_nb_p)+1) > (view->line_count))
 	{
@@ -275,53 +427,6 @@ int prepare_to_get_value_from_column(Obiview_p view, OBIDMS_column_p column, ind
  *
  **********************************************************************/
 
-
-Obiview_p obi_new_view_nuc_seqs(OBIDMS_p dms, const char* view_name, Obiview_p view_to_clone, index_t* line_selection, const char* comments)
-{
-	Obiview_p view;
-
-	if (view_to_clone != NULL)
-	{ // TODO check that the view to clone is already a NUC_SEQS view (discuss possibility of transforming type of a view)
-		if (strcmp(view_to_clone->view_type, VIEW_TYPE_NUC_SEQS))
-		{
-			obi_set_errno(OBIVIEW_ERROR);
-			obidebug(1, "Trying to clone a non-NUC SEQS view to create a NUC SEQS view");
-			return NULL;
-		}
-	}
-
-	view = obi_new_view(dms, view_name, view_to_clone, line_selection, comments);
-	if (view== NULL)
-		return NULL;
-
-	strcpy(view->view_type, VIEW_TYPE_NUC_SEQS);
-
-	if (view_to_clone == NULL)
-	{
-		// Adding sequence column
-		if (obi_view_add_column(view, NUC_SEQUENCE_COLUMN, -1, OBI_SEQ, 0, 1, NUC_SEQUENCE_COLUMN, NUC_SEQUENCE_INDEXER, "Nucleotide sequences", true) < 0)
-		{
-			obidebug(1, "Error adding an obligatory column in a nucleotide sequences view");
-			return NULL;
-		}
-		// Adding id column
-		if (obi_view_add_column(view, ID_COLUMN, -1, OBI_STR, 0, 1, ID_COLUMN, ID_INDEXER, "Ids", true) < 0)
-		{
-			obidebug(1, "Error adding an obligatory column in a nucleotide sequences view");
-			return NULL;
-		}
-		// Adding description column
-		if (obi_view_add_column(view, DESCRIPTION_COLUMN, -1, OBI_STR, 0, 1, DESCRIPTION_COLUMN, DESCRIPTION_INDEXER, "Descriptions", true) < 0)
-		{
-			obidebug(1, "Error adding an obligatory column in a nucleotide sequences view");
-			return NULL;
-		}
-	}
-
-	return view;
-}
-
-
 Obiview_p obi_new_view(OBIDMS_p dms, const char* view_name, Obiview_p view_to_clone, index_t* line_selection, const char* comments)
 {
 	Obiview_p view;
@@ -329,8 +434,8 @@ Obiview_p obi_new_view(OBIDMS_p dms, const char* view_name, Obiview_p view_to_cl
 	index_t line_nb;
 	Obiviews_infos_all_p views_infos;
 
-	// Check uniqueness of name TODO but problem if view not written yet has the same name
-	views_infos = obi_read_views(dms);
+	// Check uniqueness of name TODO but problem if view not written yet has the same name. Save lists of open views in DMS ?
+	views_infos = obi_read_view_infos(dms);
 	if (views_infos != NULL)
 	{
 		for (i=0; i<((views_infos->header)->view_count); i++)
@@ -339,11 +444,11 @@ Obiview_p obi_new_view(OBIDMS_p dms, const char* view_name, Obiview_p view_to_cl
 			{
 				obi_set_errno(OBIVIEW_ERROR);
 				obidebug(1, "\nView name already exists for a previous view");
-				obi_unmap_read_views(views_infos);
+				obi_close_view_infos(views_infos);
 				return NULL;
 			}
 		}
-	obi_unmap_read_views(views_infos);
+	obi_close_view_infos(views_infos);
 	}
 
 	view = (Obiview_p) malloc(sizeof(Obiview_t));
@@ -367,13 +472,16 @@ Obiview_p obi_new_view(OBIDMS_p dms, const char* view_name, Obiview_p view_to_cl
 
 		view->dms = dms;
 		view->column_count = view_to_clone->column_count;
-		if ((view_to_clone->line_selection != NULL) && (line_selection == NULL))	// reorder conditions
+
+		// If the view to clone has an associated line selection and there is no new line selection, open the associated line selection
+		if ((view_to_clone->line_selection != NULL) && (line_selection == NULL))
 		{
 			view->line_selection = obi_open_column(dms, ((view_to_clone->line_selection)->header)->name, ((view_to_clone->line_selection)->header)->version);
 			if (view->line_selection == NULL)
 				return NULL;
 			view->line_count = view_to_clone->line_count;
 		}
+		// If there is a new line selection, build it by combining it with the one from the view to clone if there is one
 		else if (line_selection != NULL)
 		{
 			view->line_selection = obi_create_column(view->dms, LINES_COLUMN_NAME, OBI_IDX, 0, 1, LINES_COLUMN_NAME, NULL, NULL);
@@ -411,7 +519,7 @@ Obiview_p obi_new_view(OBIDMS_p dms, const char* view_name, Obiview_p view_to_cl
 				(view->line_count)++;
 			}
 		}
-		else
+		else	// If there is no line selection associated with the view to clone or the new view
 		{
 			view->line_selection = NULL;
 			view->line_count = view_to_clone->line_count;
@@ -442,7 +550,7 @@ Obiview_p obi_new_view(OBIDMS_p dms, const char* view_name, Obiview_p view_to_cl
 		view->new_line_selection = NULL;
 		(view->created_from)[0]  = '\0';
 		(view->view_type)[0]     = '\0';
-		//view->columns          = NULL; TODO
+		//view->columns          = NULL;  // TODO
 	}
 
 	strcpy(view->name, view_name);
@@ -469,6 +577,52 @@ Obiview_p obi_new_view_cloned_from_name(OBIDMS_p dms, const char* view_name, con
 }
 
 
+Obiview_p obi_new_view_nuc_seqs(OBIDMS_p dms, const char* view_name, Obiview_p view_to_clone, index_t* line_selection, const char* comments)
+{
+	Obiview_p view;
+
+	if (view_to_clone != NULL)
+	{ // Check that the view to clone is already a NUC_SEQS view (TODO discuss possibility of transforming type of a view)
+		if (strcmp(view_to_clone->view_type, VIEW_TYPE_NUC_SEQS))
+		{
+			obi_set_errno(OBIVIEW_ERROR);
+			obidebug(1, "Trying to clone a non-NUC SEQS view to create a NUC SEQS view");
+			return NULL;
+		}
+	}
+
+	view = obi_new_view(dms, view_name, view_to_clone, line_selection, comments);
+	if (view== NULL)
+		return NULL;
+
+	strcpy(view->view_type, VIEW_TYPE_NUC_SEQS);
+
+	if (view_to_clone == NULL)
+	{
+		// Adding sequence column
+		if (obi_view_add_column(view, NUC_SEQUENCE_COLUMN, -1, OBI_SEQ, 0, 1, NUC_SEQUENCE_COLUMN, NUC_SEQUENCE_INDEXER, "Nucleotide sequences", true) < 0)
+		{
+			obidebug(1, "Error adding an obligatory column in a nucleotide sequences view");
+			return NULL;
+		}
+		// Adding id column
+		if (obi_view_add_column(view, ID_COLUMN, -1, OBI_STR, 0, 1, ID_COLUMN, ID_INDEXER, "Ids", true) < 0)
+		{
+			obidebug(1, "Error adding an obligatory column in a nucleotide sequences view");
+			return NULL;
+		}
+		// Adding definition column
+		if (obi_view_add_column(view, DEFINITION_COLUMN, -1, OBI_STR, 0, 1, DEFINITION_COLUMN, DEFINITION_INDEXER, "Definitions", true) < 0)
+		{
+			obidebug(1, "Error adding an obligatory column in a nucleotide sequences view");
+			return NULL;
+		}
+	}
+
+	return view;
+}
+
+
 Obiview_p obi_new_view_nuc_seqs_cloned_from_name(OBIDMS_p dms, const char* view_name, const char* view_to_clone_name, index_t* line_selection, const char* comments)
 {
 	Obiview_p view;
@@ -488,7 +642,6 @@ Obiview_p obi_new_view_nuc_seqs_cloned_from_name(OBIDMS_p dms, const char* view_
 Obiview_p obi_open_view(OBIDMS_p dms, const char* view_name)
 {
 	Obiview_p			view;
-	bool      			first_view;
 	char*       		view_file_name;
 	int 				obiview_file_descriptor;
 	size_t  			header_size;
@@ -499,9 +652,6 @@ Obiview_p obi_open_view(OBIDMS_p dms, const char* view_name)
 	int					i;
 	int 				view_number;
 
-	// Check if 1st view : view file doesn't exist
-	first_view = 0;
-
 	view_file_name = build_obiview_file_name();
 	if (view_file_name == NULL)
 		return NULL;
@@ -560,7 +710,7 @@ Obiview_p obi_open_view(OBIDMS_p dms, const char* view_name)
 		return NULL;
 	}
 
-	// Find and open view that should be read with the line selection associated
+	// Find and open the view that should be read with the line selection associated
 	view_number = -1;
 	if (view_name == NULL)	// If view name is NULL, open the latest view TODO discuss
 		view_number = (header->view_count) - 1;
@@ -639,7 +789,7 @@ Obiview_p obi_open_view(OBIDMS_p dms, const char* view_name)
 		view->column_count++;
 	}
 
-	// Munmap and close things TODO
+	// Munmap and close things
 	munmap(views, header->views_size);
 	munmap(header, header_size);
 	close(obiview_file_descriptor);
@@ -648,113 +798,6 @@ Obiview_p obi_open_view(OBIDMS_p dms, const char* view_name)
 }
 
 
-Obiviews_infos_all_p obi_read_views(OBIDMS_p dms)
-{
-	char*       		 view_file_name;
-	int 				 obiview_file_descriptor;
-	size_t  			 header_size;
-	Obiviews_header_p 	 header;
-	Obiview_infos_p 	 view_infos;
-	Obiviews_infos_all_p views;
-
-	view_file_name = build_obiview_file_name();
-	if (view_file_name == NULL)
-		return NULL;
-
-	// Open view file, read header size and map header and views
-	obiview_file_descriptor = openat(dms->dir_fd, view_file_name, O_RDWR, 0777);
-	if (obiview_file_descriptor < 0)
-	{	// No views yet
-		free(view_file_name);
-		return NULL;
-	}
-
-	free(view_file_name);
-
-	// Read the header size
-	if (read(obiview_file_descriptor, &header_size, sizeof(size_t)) < ((ssize_t) sizeof(size_t)))
-	{
-		obi_set_errno(OBIVIEW_ERROR);
-		obidebug(1, "\nError reading the header size of an obiview file (trying to open a view when there are none?)");
-		close(obiview_file_descriptor);
-		return NULL;
-	}
-
-	// Map the header
-	header = mmap(NULL,
-				  header_size,
-				  PROT_READ | PROT_WRITE,
-				  MAP_SHARED,
-				  obiview_file_descriptor,
-				  0
- 	 	   	   	 );
-	if (header == MAP_FAILED)
-	{
-		obi_set_errno(OBIVIEW_ERROR);
-		obidebug(1, "\nError mmapping an obiview file header");
-		close(obiview_file_descriptor);
-		return NULL;
-	}
-
-	// Map the views
-	view_infos = mmap(NULL,
-				 header->views_size,
-				 PROT_READ | PROT_WRITE,
-				 MAP_SHARED,
-				 obiview_file_descriptor,
-				 header_size
- 	 	   	   	);
-	if (view_infos == MAP_FAILED)
-	{
-		obi_set_errno(OBIVIEW_ERROR);
-		obidebug(1, "\nError mmapping the views from an obiview file");
-		munmap(header, header_size);
-		close(obiview_file_descriptor);
-		return NULL;
-	}
-
-	views = (Obiviews_infos_all_p) malloc(sizeof(Obiviews_infos_all_t));
-	if (views == NULL)
-	{
-		obi_set_errno(OBIVIEW_ERROR);
-		obidebug(1, "\nError mmapping the views from an obiview file");
-		munmap(view_infos, header->views_size);
-		munmap(header, header_size);
-		close(obiview_file_descriptor);
-		return NULL;
-	}
-
-	views->header = header;
-	views->view_infos = view_infos;
-
-	close(obiview_file_descriptor);
-
-	return views;
-}
-
-
-int obi_unmap_read_views(Obiviews_infos_all_p views)
-{
-	if (munmap(views->view_infos, (views->header)->views_size) < 0)
-	{
-		obi_set_errno(OBIVIEW_ERROR);
-		obidebug(1, "\nError unmapping the views of an obiview file");
-		free(views);
-		return -1;
-	}
-	if (munmap(views->header, (views->header)->header_size) < 0)
-	{
-		obi_set_errno(OBIVIEW_ERROR);
-		obidebug(1, "\nError unmapping the header of an obiview file");
-		free(views);
-		return -1;
-
-	}
-	free(views);
-	return 0;
-}
-
-
 int obi_view_add_column(Obiview_p view,
 						const char* column_name,
 						obiversion_t version_number,
@@ -779,7 +822,7 @@ int obi_view_add_column(Obiview_p view,
 	if (view->line_count > nb_lines)
 		nb_lines = view->line_count;
 	else if (nb_lines > view->line_count)
-		obi_view_update_lines(view, nb_lines);
+		update_lines(view, nb_lines);
 
 	// Open or create the column
 	if (create)
@@ -807,61 +850,6 @@ int obi_view_add_column(Obiview_p view,
 }
 
 
-OBIDMS_column_p obi_view_clone_column(Obiview_p view, const char* column_name)
-{
-	int i;
-	OBIDMS_column_p current_line_selection = NULL;
-	OBIDMS_column_p column;
-	OBIDMS_column_p column_buffer;
-
-	// Check that the view is not read-only
-	if (view->read_only)
-	{
-		obi_set_errno(OBIVIEW_ERROR);
-		obidebug(1, "\nError trying to delete a column in a read-only view");
-		return NULL;
-	}
-
-	if (view->new_line_selection != NULL)	// TODO Probably shouldn't happen
-		current_line_selection = view->new_line_selection;
-	else
-		current_line_selection = view->line_selection;
-
-	for (i=0; i<(view->column_count); i++)
-	{
-		if ((current_line_selection != NULL) || (!(strcmp((((view->columns)[i])->header)->name, column_name))))
-		{ // Clone with the right line selection and replace
-			column_buffer = (view->columns)[i];
-			(view->columns)[i] = obi_clone_column(view->dms, current_line_selection, (((view->columns)[i])->header)->name, (((view->columns)[i])->header)->version, 1);
-			if ((view->columns)[i] == NULL)
-			{
-				obi_set_errno(OBIVIEW_ERROR);
-				obidebug(1, "\nError cloning a column to replace in a view");
-				return NULL;
-			}
-			// Found the column to return
-			if (!(strcmp((((view->columns)[i])->header)->name, column_name)))
-				column = (view->columns)[i];
-			else
-				obi_close_column(column_buffer);	// TODO weird closing after cloning but can't think of cleaner yet
-		}
-	}
-
-	if (view->line_selection != NULL)
-	{
-		obi_close_column(view->line_selection);
-		view->line_selection = NULL;
-	}
-	if (view->new_line_selection != NULL)
-	{
-		obi_close_column(view->new_line_selection);
-		view->new_line_selection = NULL;
-	}
-
-	return column;
-}
-
-
 int obi_view_delete_column(Obiview_p view, const char* column_name)
 {
 	int  i;
@@ -904,6 +892,32 @@ int obi_view_delete_column(Obiview_p view, const char* column_name)
 }
 
 
+OBIDMS_column_p obi_view_get_column(Obiview_p view, const char* column_name)
+{
+	int i;
+
+	for (i=0; i<(view->column_count); i++)
+	{
+		if (!(strcmp((((view->columns)[i])->header)->name, column_name)))
+			return (view->columns)[i];
+	}
+	return NULL;
+}
+
+
+OBIDMS_column_p* obi_view_get_pointer_on_column_in_view(Obiview_p view, const char* column_name)
+{
+	int i;
+
+	for (i=0; i<(view->column_count); i++)
+	{
+		if (!(strcmp((((view->columns)[i])->header)->name, column_name)))
+			return ((view->columns)+i);
+	}
+	return NULL;
+}
+
+
 int obi_select_line(Obiview_p view, index_t line_nb)
 {
 	// Check that the view is not read-only
@@ -914,7 +928,7 @@ int obi_select_line(Obiview_p view, index_t line_nb)
 		return -1;
 	}
 
-	// If column for line selection doesn't already exists, create it and store its informations
+	// If the column for line selection doesn't already exists, create it and store its informations
 	if ((view->new_line_selection) == NULL)
 	{
 		view->new_line_selection = obi_create_column(view->dms, LINES_COLUMN_NAME, OBI_IDX, 0, 1, LINES_COLUMN_NAME, NULL, NULL);
@@ -976,52 +990,6 @@ int obi_select_lines(Obiview_p view, index_t* line_nbs)
 }
 
 
-int obi_view_update_lines(Obiview_p view, index_t line_count)
-{
-	int i;
-
-	for (i=0; i<(view->column_count); i++)
-	{
-		while (line_count > (((view->columns)[i])->header)->line_count)
-		{
-			// Enlarge the column
-			if (obi_enlarge_column((view->columns)[i]) < 0)
-		    	return -1;
-		}
-	}
-
-	view->line_count = line_count;
-
-	return 0;
-}
-
-
-OBIDMS_column_p obi_view_get_column(Obiview_p view, const char* column_name)
-{
-	int i;
-
-	for (i=0; i<(view->column_count); i++)
-	{
-		if (!(strcmp((((view->columns)[i])->header)->name, column_name)))
-			return (view->columns)[i];
-	}
-	return NULL;
-}
-
-
-OBIDMS_column_p* obi_view_get_pointer_on_column_in_view(Obiview_p view, const char* column_name)
-{
-	int i;
-
-	for (i=0; i<(view->column_count); i++)
-	{
-		if (!(strcmp((((view->columns)[i])->header)->name, column_name)))
-			return ((view->columns)+i);
-	}
-	return NULL;
-}
-
-
 int obi_save_view(Obiview_p view)
 {
 	int i;
@@ -1204,7 +1172,7 @@ int obi_close_view(Obiview_p view)
 
 int obi_save_and_close_view(Obiview_p view)
 {
-	if (!(view->read_only))	// TODO discuss
+	if (!(view->read_only))
 	{
 		if (obi_save_view(view) < 0)
 			return -1;
@@ -1215,6 +1183,113 @@ int obi_save_and_close_view(Obiview_p view)
 }
 
 
+Obiviews_infos_all_p obi_read_view_infos(OBIDMS_p dms)
+{
+	char*       		 view_file_name;
+	int 				 obiview_file_descriptor;
+	size_t  			 header_size;
+	Obiviews_header_p 	 header;
+	Obiview_infos_p 	 view_infos;
+	Obiviews_infos_all_p views;
+
+	view_file_name = build_obiview_file_name();
+	if (view_file_name == NULL)
+		return NULL;
+
+	// Open view file, read header size and map header and views
+	obiview_file_descriptor = openat(dms->dir_fd, view_file_name, O_RDWR, 0777);
+	if (obiview_file_descriptor < 0)
+	{	// No views yet
+		free(view_file_name);
+		return NULL;
+	}
+
+	free(view_file_name);
+
+	// Read the header size
+	if (read(obiview_file_descriptor, &header_size, sizeof(size_t)) < ((ssize_t) sizeof(size_t)))
+	{
+		obi_set_errno(OBIVIEW_ERROR);
+		obidebug(1, "\nError reading the header size of an obiview file (trying to open a view when there are none?)");
+		close(obiview_file_descriptor);
+		return NULL;
+	}
+
+	// Map the header
+	header = mmap(NULL,
+				  header_size,
+				  PROT_READ | PROT_WRITE,
+				  MAP_SHARED,
+				  obiview_file_descriptor,
+				  0
+ 	 	   	   	 );
+	if (header == MAP_FAILED)
+	{
+		obi_set_errno(OBIVIEW_ERROR);
+		obidebug(1, "\nError mmapping an obiview file header");
+		close(obiview_file_descriptor);
+		return NULL;
+	}
+
+	// Map the views
+	view_infos = mmap(NULL,
+				 header->views_size,
+				 PROT_READ | PROT_WRITE,
+				 MAP_SHARED,
+				 obiview_file_descriptor,
+				 header_size
+ 	 	   	   	);
+	if (view_infos == MAP_FAILED)
+	{
+		obi_set_errno(OBIVIEW_ERROR);
+		obidebug(1, "\nError mmapping the views from an obiview file");
+		munmap(header, header_size);
+		close(obiview_file_descriptor);
+		return NULL;
+	}
+
+	views = (Obiviews_infos_all_p) malloc(sizeof(Obiviews_infos_all_t));
+	if (views == NULL)
+	{
+		obi_set_errno(OBIVIEW_ERROR);
+		obidebug(1, "\nError mmapping the views from an obiview file");
+		munmap(view_infos, header->views_size);
+		munmap(header, header_size);
+		close(obiview_file_descriptor);
+		return NULL;
+	}
+
+	views->header = header;
+	views->view_infos = view_infos;
+
+	close(obiview_file_descriptor);
+
+	return views;
+}
+
+
+int obi_close_view_infos(Obiviews_infos_all_p views)
+{
+	if (munmap(views->view_infos, (views->header)->views_size) < 0)
+	{
+		obi_set_errno(OBIVIEW_ERROR);
+		obidebug(1, "\nError unmapping the views of an obiview file");
+		free(views);
+		return -1;
+	}
+	if (munmap(views->header, (views->header)->header_size) < 0)
+	{
+		obi_set_errno(OBIVIEW_ERROR);
+		obidebug(1, "\nError unmapping the header of an obiview file");
+		free(views);
+		return -1;
+
+	}
+	free(views);
+	return 0;
+}
+
+
 /*********** FOR BOOL COLUMNS ***********/
 
 int obi_column_set_obibool_with_elt_idx_in_view(Obiview_p view, OBIDMS_column_p column, index_t line_nb, index_t element_idx, obibool_t value)
@@ -1227,7 +1302,7 @@ int obi_column_set_obibool_with_elt_idx_in_view(Obiview_p view, OBIDMS_column_p
 
 obibool_t obi_column_get_obibool_with_elt_idx_in_view(Obiview_p view, OBIDMS_column_p column, index_t line_nb, index_t element_idx)
 {
-	if (prepare_to_get_value_from_column(view, column, &line_nb) < 0)
+	if (prepare_to_get_value_from_column(view, &line_nb) < 0)
 		return OBIBool_NA;
 	return obi_column_get_obibool_with_elt_idx(column, line_nb, element_idx);
 }
@@ -1265,7 +1340,7 @@ int obi_column_set_obichar_with_elt_idx_in_view(Obiview_p view, OBIDMS_column_p
 
 obichar_t obi_column_get_obichar_with_elt_idx_in_view(Obiview_p view, OBIDMS_column_p column, index_t line_nb, index_t element_idx)
 {
-	if (prepare_to_get_value_from_column(view, column, &line_nb) < 0)
+	if (prepare_to_get_value_from_column(view, &line_nb) < 0)
 		return OBIChar_NA;
 	return obi_column_get_obichar_with_elt_idx(column, line_nb, element_idx);
 }
@@ -1303,7 +1378,7 @@ int obi_column_set_obifloat_with_elt_idx_in_view(Obiview_p view, OBIDMS_column_p
 
 obifloat_t obi_column_get_obifloat_with_elt_idx_in_view(Obiview_p view, OBIDMS_column_p column, index_t line_nb, index_t element_idx)
 {
-	if (prepare_to_get_value_from_column(view, column, &line_nb) < 0)
+	if (prepare_to_get_value_from_column(view, &line_nb) < 0)
 		return OBIFloat_NA;
 	return obi_column_get_obifloat_with_elt_idx(column, line_nb, element_idx);
 }
@@ -1341,7 +1416,7 @@ int obi_column_set_obiint_with_elt_idx_in_view(Obiview_p view, OBIDMS_column_p c
 
 obiint_t obi_column_get_obiint_with_elt_idx_in_view(Obiview_p view, OBIDMS_column_p column, index_t line_nb, index_t element_idx)
 {
-	if (prepare_to_get_value_from_column(view, column, &line_nb) < 0)
+	if (prepare_to_get_value_from_column(view, &line_nb) < 0)
 		return OBIInt_NA;
 	return obi_column_get_obiint_with_elt_idx(column, line_nb, element_idx);
 }
@@ -1379,7 +1454,7 @@ int obi_column_set_obiseq_with_elt_idx_in_view(Obiview_p view, OBIDMS_column_p c
 
 char* obi_column_get_obiseq_with_elt_idx_in_view(Obiview_p view, OBIDMS_column_p column, index_t line_nb, index_t element_idx)
 {
-	if (prepare_to_get_value_from_column(view, column, &line_nb) < 0)
+	if (prepare_to_get_value_from_column(view, &line_nb) < 0)
 		return OBISeq_NA;
 	return obi_column_get_obiseq_with_elt_idx(column, line_nb, element_idx);
 }
@@ -1417,7 +1492,7 @@ int obi_column_set_obistr_with_elt_idx_in_view(Obiview_p view, OBIDMS_column_p c
 
 const char* obi_column_get_obistr_with_elt_idx_in_view(Obiview_p view, OBIDMS_column_p column, index_t line_nb, index_t element_idx)
 {
-	if (prepare_to_get_value_from_column(view, column, &line_nb) < 0)
+	if (prepare_to_get_value_from_column(view, &line_nb) < 0)
 		return OBIStr_NA;
 	return obi_column_get_obistr_with_elt_idx(column, line_nb, element_idx);
 }

src/obiview.h

@@ -26,30 +26,82 @@
 #include "obierrno.h"
 
 
-#define OBIVIEW_NAME_MAX_LENGTH (1000)   /**< The maximum length of an OBIDMS view name.
-                                	 	  */
-#define OBIVIEW_COMMENTS_MAX_LENGTH (10000)
-
-#define OBIVIEW_FILE_NAME "obiviews"
-
-#define VIEW_TYPE_MAX_NAME (1024)
-
-#define VIEW_TYPE_NUC_SEQS "NUC_SEQS_VIEW"
-
-#define NUC_SEQUENCE_COLUMN "NUC_SEQ"
-#define NUC_SEQUENCE_INDEXER "NUC_SEQ_INDEXER"
-
-#define ID_COLUMN "ID"
-#define ID_INDEXER "ID_INDEXER"
-
-#define DESCRIPTION_COLUMN "DESCRIPTION"
-#define DESCRIPTION_INDEXER "DESCRIPTION_INDEXER"
-
-#define LINES_COLUMN_NAME "LINES"
+#define OBIVIEW_NAME_MAX_LENGTH (1000)   		/**< The maximum length of an OBIDMS view name.
+                                	 	  	  	 */
+#define OBIVIEW_COMMENTS_MAX_LENGTH (10000)		/**< The maximum length of the comments associated
+												 *   with a view.
+                                	 	  	  	 */
+#define VIEW_TYPE_MAX_LENGTH (1024)   			/**< The maximum length of the type name of a view.
+                                	 	  	  	 */
+#define OBIVIEW_FILE_NAME "obiviews"   			/**< The default name of a view file.
+                                	 	  	  	 */
+#define LINES_COLUMN_NAME "LINES"				/**< The name of the column containing the line selections
+ 	 	 	 	 	 	 	 	 	 	 	 	 *   in all views.
+                                	 	  	  	 */
+#define VIEW_TYPE_NUC_SEQS "NUC_SEQS_VIEW"   	/**< The type name of views based on nucleotide sequences
+												 *   and their metadata.
+                                	 	  	  	 */
+#define NUC_SEQUENCE_COLUMN "NUC_SEQ"			/**< The name of the column containing the nucleotide sequences
+ 	 	 	 	 	 	 	 	 	 	 	 	 *   in NUC_SEQS_VIEW views.
+                                	 	  	  	 */
+#define NUC_SEQUENCE_INDEXER "NUC_SEQ_INDEXER"	/**< The name of the indexer containing the nucleotide sequences
+ 	 	 	 	 	 	 	 	 	 	 	 	 *   in NUC_SEQS_VIEW views.
+                                	 	  	  	 */
+#define ID_COLUMN "ID"							/**< The name of the column containing the sequence identifiers
+ 	 	 	 	 	 	 	 	 	 	 	 	 *   in NUC_SEQS_VIEW views.
+                                	 	  	  	 */
+#define ID_INDEXER "ID_INDEXER"					/**< The name of the indexer containing the sequence identifiers
+ 	 	 	 	 	 	 	 	 	 	 	 	 *   in NUC_SEQS_VIEW views.
+                                	 	  	  	 */
+#define DEFINITION_COLUMN "DEFINITION"			/**< The name of the column containing the sequence definitions
+ 	 	 	 	 	 	 	 	 	 	 	 	 *   in NUC_SEQS_VIEW views.
+                                	 	  	  	 */
+#define DEFINITION_INDEXER "DEFINITION_INDEXER"	/**< The name of the indexer containing the sequence definitions
+ 	 	 	 	 	 	 	 	 	 	 	 	 *   in NUC_SEQS_VIEW views.
+                                	 	  	  	 */
 
 
 /**
- * @brief .
+ * @brief Structure for an opened view.
+ */
+typedef struct Obiview {
+
+	OBIDMS_p         	dms;    									/**< A pointer on the DMS to which the view belongs.
+	 	 	    													 */
+	char   				name[OBIVIEW_NAME_MAX_LENGTH+1];    		/**< Name of the view.
+	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 */
+	char				created_from[OBIVIEW_NAME_MAX_LENGTH+1];    /**< Name of the view from which that view was cloned if the view was cloned.
+ 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 */
+	char				view_type[VIEW_TYPE_MAX_LENGTH+1];			/**< Type of the view if there is one.
+																	 *   Types existing: NUC_SEQS_VIEW.
+ 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 */
+	bool             	read_only;    								/**< Whether the view is read-only or can be modified.
+																	 */
+	OBIDMS_column_p  	line_selection;    							/**< A pointer on the column containing the line selection
+																	 *   associated with the view if there is one.
+																	 *   This line selection is read-only, and when a line from the view is read,
+																	 *   it is this line selection that is used.
+	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 */
+	OBIDMS_column_p  	new_line_selection;    						/**< A pointer on the column containing the new line selection being built
+	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 *   to associate with the view, if there is one.
+	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 *   When a line is selected with obi_select_line() or obi_select_lines(),
+	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 *   it is recorded in this line selection.
+	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 */
+	index_t			 	line_count;    								/**< The number of lines in the view. Refers to the number of lines in each
+	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 *   column of the view if line_selection is NULL, or to the line count of
+	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 *   line_selection if it is not NULL.
+ 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 */
+	int    			 	column_count;    							/**< The number of columns in the view.
+ 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 */
+	OBIDMS_column_p  	columns[MAX_NB_OPENED_COLUMNS];    			/**< Array of pointers on all the columns of the view.
+	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 */
+	char 				comments[OBIVIEW_COMMENTS_MAX_LENGTH+1];	/**< Comments, additional informations on the view.
+	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 */
+} Obiview_t, *Obiview_p;
+
+
+/**
+ * @brief Structure referencing a column by its name and its version.
  */
 typedef struct Column_reference {
 	char 	   	 column_name[OBIDMS_COLUMN_MAX_NAME+1];    /**< Name of the column.
@@ -60,101 +112,208 @@ typedef struct Column_reference {
 
 
 /**
- * @brief .
- */
-typedef struct Obiview {
-
-	OBIDMS_p         	dms;
-
-	bool             	read_only;
-
-	OBIDMS_column_p  	line_selection;
-
-	OBIDMS_column_p  	new_line_selection;
-
-	index_t			 	line_count;
-
-	int    			 	column_count;
-
-	OBIDMS_column_p  	columns[MAX_NB_OPENED_COLUMNS];
-
-	char   				name[OBIVIEW_NAME_MAX_LENGTH+1];
-
-	char				created_from[OBIVIEW_NAME_MAX_LENGTH+1];
-
-	int					view_number;
-
-	char				view_type[VIEW_TYPE_MAX_NAME+1];
-
-	char 				comments[OBIVIEW_COMMENTS_MAX_LENGTH+1];
-
-} Obiview_t, *Obiview_p;
-
-
-/**
- * @brief .
+ * @brief Structure for a closed view stored in the view file.
+ * 		  Views are identified by their name.
+ * 		  Once a view has been written in the view file, it can not be modified and can only be read.
  */
 typedef struct Obiview_infos {
-	int					view_number;
-
-	int 				column_count;
-
-	index_t             line_count;
-
-	char   				name[OBIVIEW_NAME_MAX_LENGTH+1];
-
-	char				created_from[OBIVIEW_NAME_MAX_LENGTH+1];
-
-	time_t 		 		creation_date;
-
-	bool				all_lines;
-
-	Column_reference_t	line_selection;
-
-	Column_reference_t	column_references[MAX_NB_OPENED_COLUMNS];
-
-	char				view_type[VIEW_TYPE_MAX_NAME+1];
-
-	char 				comments[OBIVIEW_COMMENTS_MAX_LENGTH+1];
-
+	int					view_number;    							/**< Number of the view in the view file.
+	 	 	    													 */
+	time_t 		 		creation_date;    							/**< Time at which the view was written in the view file.
+		 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 */
+	char   				name[OBIVIEW_NAME_MAX_LENGTH+1];    		/**< Name of the view, used to identify it.
+	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 */
+	char				created_from[OBIVIEW_NAME_MAX_LENGTH+1];    /**< Name of the view from which that view was cloned, if it was cloned.
+	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 */
+	char				view_type[VIEW_TYPE_MAX_LENGTH+1];			/**< Type of the view if there is one.
+	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 *   Types existing: NUC_SEQS_VIEW.
+	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 */
+	bool				all_lines;									/**< Whether there is a line selection associated with the view.
+	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 */
+	Column_reference_t	line_selection;								/**< Whether there is a line selection associated with the view.
+ 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 */
+	index_t             line_count;    								/**< The number of lines in the view.
+																	 */
+	int 				column_count;    							/**< The number of columns in the view.
+	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 */
+	Column_reference_t	column_references[MAX_NB_OPENED_COLUMNS];   /**< References (name and version) for all the columns in the view.
+ 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 */
+	char 				comments[OBIVIEW_COMMENTS_MAX_LENGTH+1];	/**< Comments, additional informations on the view.
+ 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 */
 } Obiview_infos_t, *Obiview_infos_p;
 
 
+// TODO : Combine the common elements of the Obiview_infos and Obiview structures in one structure used by both?
+
+
 /**
- * @brief .
+ * @brief Structure for the header of view files.
  */
 typedef struct Obiviews_header {
-	size_t header_size;
-	size_t views_size;
-	int    view_count;
+	size_t header_size;   /**< Size of the header in bytes.
+	 	 	 	 	 	   */
+	size_t views_size;    /**< Size of the views in bytes.
+	 	   	   	   	   	   */
+	int    view_count;    /**< Number of views saved.
+	   	   	   	   	   	   */
 } Obiviews_header_t, *Obiviews_header_p;
 
 
 /**
- * @brief .
+ * @brief Structure for the information stored in view files.
  */
 typedef struct Obiviews_infos_all {
-	Obiviews_header_p	header;
-
-	Obiview_infos_p 	view_infos;
-
+	Obiviews_header_p	header;   		/**< Pointer on the header of the view file.
+	 	   	   	   	   	   	   	   	     */
+	Obiview_infos_p 	view_infos;     /**< Pointer on the beginning (first view) of the informations on views.
+	   	   	   	     	 	 	 	 	 */
 } Obiviews_infos_all_t, *Obiviews_infos_all_p;
 
 
-Obiview_p obi_new_view_nuc_seqs(OBIDMS_p dms, const char* view_name, Obiview_p view_to_clone, index_t* line_selection, const char* comments);
-
+/**
+ * @brief Creates a new view.
+ *
+ * Fails if a view with the same name already exists.
+ *
+ * @param dms A pointer on the OBIDMS.
+ * @param view_name The unique name of the view.
+ * @param view_to_clone Eventually a pointer on the opened view to clone to create the new one, if there is one. NULL if not.
+ * @param line_selection Eventually a pointer on a list of indexes corresponding to a line selection to use with the view to clone
+ * 						 if there is one. NULL if there is no line selection or no view to clone.
+ * @param comments Eventually, comments to associate with the view. NULL if not.
+ *
+ * @returns A pointer to the newly created view structure.
+ * @retval NULL if an error occurred.
+ *
+ * @since December 2015
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
 Obiview_p obi_new_view(OBIDMS_p dms, const char* view_name, Obiview_p view_to_clone, index_t* line_selection, const char* comments);
 
+
+/**
+ * @brief Creates a new view by cloning another view written in the view file and identified by its name.
+ *
+ * Note : obi_new_view can clone from a pointer on an opened view, while this function will read the view to clone
+ * 		  from the view file.
+ *
+ * Fails if a view with the same name already exists.
+ *
+ * @param dms A pointer on the OBIDMS.
+ * @param view_name The unique name of the new view.
+ * @param view_to_clone_name The name of the view to clone stored in the view file of the OBIDMS.
+ * @param line_selection Eventually a pointer on a list of indexes corresponding to a line selection to use with the view to clone
+ * 						 if there is one. NULL if there is no line selection or no view to clone.
+ * @param comments Eventually, comments to associate with the view. NULL if not.
+ *
+ * @returns A pointer to the newly created view structure.
+ * @retval NULL if an error occurred.
+ *
+ * @since December 2015
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
 Obiview_p obi_new_view_cloned_from_name(OBIDMS_p dms, const char* view_name, const char* view_to_clone_name, index_t* line_selection, const char* comments);
 
+
+/**
+ * @brief Creates a new view with the type NUC_SEQ_VIEW, based on nucleotide sequences and their metadata.
+ *
+ * Fails if a view with the same name already exists.
+ *
+ * The obligatory columns specific to the view type are created too and opened:
+ * 	- NUC_SEQUENCE_COLUMN where nucleotide sequences are stored
+ * 	- ID_COLUMN			  where sequence identifiers are stored
+ * 	- DEFINITION_COLUMN   where sequence definitions are stored
+ *
+ * @param dms A pointer on the OBIDMS.
+ * @param view_name The unique name of the view.
+ * @param view_to_clone Eventually a pointer on the opened view to clone to create the new one, if there is one. NULL if not.
+ * @param line_selection Eventually a pointer on a list of indexes corresponding to a line selection to use with the view to clone
+ * 						 if there is one. NULL if there is no line selection or no view to clone.
+ * @param comments Eventually, comments to associate with the view. NULL if not.
+ *
+ * @returns A pointer to the newly created view structure.
+ * @retval NULL if an error occurred.
+ *
+ * @since February 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
+Obiview_p obi_new_view_nuc_seqs(OBIDMS_p dms, const char* view_name, Obiview_p view_to_clone, index_t* line_selection, const char* comments);
+
+
+/**
+ * @brief Creates a new view with the type NUC_SEQ_VIEW, based on nucleotide sequences and their metadata,
+ *        by cloning another NUC_SEQ_VIEW view written in the view file and identified by its name.
+ *
+ * Note : obi_new_view_nuc_seqs can clone from a pointer on an opened view, while this function will read the view to clone
+ * 		  from the view file.
+ *
+ * Fails if a view with the same name already exists.
+ * Fails if the view to clone doesn't have the type NUC_SEQ_VIEW.
+ *
+ * The obligatory columns specific to the view type are created too and opened:
+ * 	- NUC_SEQUENCE_COLUMN where nucleotide sequences are stored
+ * 	- ID_COLUMN			  where sequence identifiers are stored
+ * 	- DEFINITION_COLUMN   where sequence definitions are stored
+ *
+ * @param dms A pointer on the OBIDMS.
+ * @param view_name The unique name of the new view.
+ * @param view_to_clone_name The name of the view to clone stored in the view file of the OBIDMS.
+ * @param line_selection Eventually a pointer on a list of indexes corresponding to a line selection to use with the view to clone
+ * 						 if there is one. NULL if there is no line selection or no view to clone.
+ * @param comments Eventually, comments to associate with the view. NULL if not.
+ *
+ * @returns A pointer to the newly created view structure.
+ * @retval NULL if an error occurred.
+ *
+ * @since February 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
 Obiview_p obi_new_view_nuc_seqs_cloned_from_name(OBIDMS_p dms, const char* view_name, const char* view_to_clone_name, index_t* line_selection, const char* comments);
 
+
+/**
+ * @brief Opens a view identified by its name stored in the view file.
+ *
+ * When opening a view, all the columns and eventually the line selection belonging to it are opened with it.
+ *
+ * @warning The opened view is read-only.
+ *
+ * @param dms A pointer on the OBIDMS.
+ * @param view_name The unique name identifying the view.
+ *
+ * @returns A pointer on the opened view structure.
+ * @retval NULL if an error occurred.
+ *
+ * @since December 2015
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
 Obiview_p obi_open_view(OBIDMS_p dms, const char* view_name);
 
-Obiviews_infos_all_p obi_read_views(OBIDMS_p dms);
-
-int obi_unmap_read_views(Obiviews_infos_all_p views);
 
+/**
+ * @brief Adds a column to a view.
+ *
+ * @warning The view must be writable.
+ *
+ * @param view A pointer on the view.
+ * @param column_name The name of the column.
+ * @param version_number The version of the column if it should be opened and not created (if -1, the latest version is retrieved).
+ * @param data_type The OBIType code of the data.
+ * @param nb_lines The number of lines to be stored.
+ * @param nb_elements_per_line The number of elements per line.
+ * @param elements_names The names of the elements with ';' as separator.
+ * @param indexer_name The name of the indexer if there is one associated with the column.
+ * @param comments Optional comments associated with the column.
+ * @param create Whether the column should be created (create == true) or opened (create == false).
+ *
+ * @returns A value indicating the success of the operation.
+ * @retval 0 if the operation was successfully completed.
+ * @retval -1 if an error occurred.
+ *
+ * @since February 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
 int obi_view_add_column(Obiview_p view,
 						const char* column_name,
 						obiversion_t version_number,
@@ -166,35 +325,193 @@ int obi_view_add_column(Obiview_p view,
 						const char* comments,
 						bool create);
 
+
+/**
+ * @brief Deletes a column from a view.
+ *
+ * @warning The view must be writable.
+ *
+ * @param view A pointer on the view.
+ * @param column_name The name of the column that should be deleted from the view.
+ *
+ * @returns A value indicating the success of the operation.
+ * @retval 0 if the operation was successfully completed.
+ * @retval -1 if an error occurred.
+ *
+ * @since February 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
 int obi_view_delete_column(Obiview_p view, const char* column_name);
 
-int obi_select_line(Obiview_p view, index_t line_nb);
-
-int obi_select_lines(Obiview_p view, index_t* line_nbs);
-
-int obi_view_update_lines(Obiview_p view, index_t line_count);
-
-OBIDMS_column_p obi_view_clone_column(Obiview_p view, const char* column_name);
-
-OBIDMS_column_p obi_view_get_column(Obiview_p view, const char* column_name);
-
-OBIDMS_column_p* obi_view_get_pointer_on_column_in_view(Obiview_p view, const char* column_name);
-
-int obi_save_view(Obiview_p view);
-
-int obi_close_view(Obiview_p view);
-
-int obi_save_and_close_view(Obiview_p view);
-
-int obi_column_set_obibool_with_elt_idx_in_view(Obiview_p view, OBIDMS_column_p column, index_t line_nb, index_t element_idx, obibool_t value);
-
 
 /**
- * @brief Sets a value in an OBIDMS column containing data with the type OBI_BOOL, using the index of the element in the line.
+ * @brief Gets the pointer on a column from its name in a view.
  *
- * @warning Pointers returned by obi_open_column() don't allow writing.
+ * @param view A pointer on the view.
+ * @param column_name The name of the column in the view.
  *
- * @param column A pointer as returned by obi_create_column() or obi_clone_column().
+ * @returns A pointer on the column.
+ * @retval NULL if an error occurred.
+ *
+ * @since February 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
+OBIDMS_column_p obi_view_get_column(Obiview_p view, const char* column_name);
+
+
+/**
+ * @brief Gets the pointer on the pointer on a column from its name in a view.
+ *
+ * Note: This is used to replace old cloned columns with new ones in views in layers above the C layer.
+ *
+ * @param view A pointer on the view.
+ * @param column_name The name of the column in the view.
+ *
+ * @returns A pointer on the pointer on the column.
+ * @retval NULL if an error occurred.
+ *
+ * @since February 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
+OBIDMS_column_p* obi_view_get_pointer_on_column_in_view(Obiview_p view, const char* column_name);
+
+
+/**
+ * @brief Selects a line in the context of a view.
+ *
+ * If no line selection exists for the view, it will be created.
+ *
+ * @warning The view must be writable.
+ * @warning The new line index will be simply added at the end of the current
+ *          new line selection.
+ *
+ * @param view A pointer on the view.
+ * @param line_nb The index of the line that should be added in the selection.
+ *
+ * @returns A value indicating the success of the operation.
+ * @retval 0 if the operation was successfully completed.
+ * @retval -1 if an error occurred.
+ *
+ * @since February 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
+int obi_select_line(Obiview_p view, index_t line_nb);
+
+
+/**
+ * @brief Selects a list of line in the context of a view.
+ *
+ * If no line selection exists for the view, it will be created.
+ *
+ * @warning The view must be writable.
+ * @warning The new line indexes will be simply added at the end of the current
+ *          new line selection.
+ *
+ * @param view A pointer on the view.
+ * @param line_nbs A pointer on an array of indexes corresponding to the lines that
+ *                 should be added in the selection.
+ *
+ * @returns A value indicating the success of the operation.
+ * @retval 0 if the operation was successfully completed.
+ * @retval -1 if an error occurred.
+ *
+ * @since February 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
+int obi_select_lines(Obiview_p view, index_t* line_nbs);
+
+
+/**
+ * @brief Saves a view, writing it in the view file.
+ *
+ * The view is written at the end of the view file, following the latest written view.
+ *
+ * @warning The view must be writable.
+ *
+ * @param view A pointer on the view.
+ *
+ * @returns A value indicating the success of the operation.
+ * @retval 0 if the operation was successfully completed.
+ * @retval -1 if an error occurred.
+ *
+ * @since February 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
+int obi_save_view(Obiview_p view);
+
+
+/**
+ * @brief Closes an opened view.
+ *
+ * @warning Use obi_save_and_close_view() to automatically save the view if it's not already saved in the view file.
+ *
+ * @param view A pointer on the view.
+ *
+ * @returns A value indicating the success of the operation.
+ * @retval 0 if the operation was successfully completed.
+ * @retval -1 if an error occurred.
+ *
+ * @since February 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
+int obi_close_view(Obiview_p view);
+
+
+/**
+ * @brief Closes an opened view, and saves it if it is not read-only (meaning it is not already saved in the view file).
+ *
+ * @param view A pointer on the view.
+ *
+ * @returns A value indicating the success of the operation.
+ * @retval 0 if the operation was successfully completed.
+ * @retval -1 if an error occurred.
+ *
+ * @since February 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
+int obi_save_and_close_view(Obiview_p view);
+
+
+/**
+ * @brief Opens the structure containing all the informations written in the view file.
+ *
+ * @param dms A pointer on the OBIDMS to which the view file belongs.
+ *
+ * @returns A pointer on the view informations structure.
+ * @retval NULL if an error occurred.
+ *
+ * @since February 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
+Obiviews_infos_all_p obi_read_view_infos(OBIDMS_p dms);
+
+
+/**
+ * @brief Closes the structure containing all the informations written in the view file.
+ *
+ * @param views A pointer on the view informations structure.
+ *
+ * @returns A value indicating the success of the operation.
+ * @retval 0 if the operation was successfully completed.
+ * @retval -1 if an error occurred.
+ *
+ * @since February 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
+int obi_close_view_infos(Obiviews_infos_all_p views);
+
+
+// TODO in following functions would it be better to use column names instead of column pointers?
+//		check if it would be a gain or loss of time
+
+/**
+ * @brief Sets a value in an OBIDMS column containing data with the type OBI_BOOL, using the index of the element in the line,
+ *        in the context of a view.
+ *
+ *  Note: If the column is read-only or if there is a line selection associated with the view (making columns non-writable), it is cloned.
+ *
+ * @param view A pointer on the opened writable view.
+ * @param column A pointer on the column.
  * @param line_nb The number of the line where the value should be set.
  * @param element_idx The index of the element that should be set in the line.
  * @param value The value that should be set.
@@ -210,9 +527,10 @@ int obi_column_set_obibool_with_elt_idx_in_view(Obiview_p view, OBIDMS_column_p
 
 
 /**
- * @brief Recovers a value in an OBIDMS column containing data with the type OBI_BOOL.
+ * @brief Recovers a value in an OBIDMS column containing data with the type OBI_BOOL, in the context of a view.
  *
- * @param column A pointer as returned by obi_create_column().
+ * @param view A pointer on the opened view.
+ * @param column A pointer on the column.
  * @param line_nb The number of the line where the value should be recovered.
  * @param element_idx The index of the element that should be recovered in the line.
  *
@@ -227,11 +545,10 @@ obibool_t obi_column_get_obibool_with_elt_idx_in_view(Obiview_p view, OBIDMS_col
 
 /**
  * @brief Sets a value in an OBIDMS column containing data with the type OBI_BOOL,
- *        using the name of the element in the line.
+ *        using the name of the element in the line, in the context of a view.
  *
- * @warning Pointers returned by obi_open_column() don't allow writing.
- *
- * @param column A pointer as returned by obi_create_column() or obi_clone_column().
+ * @param view A pointer on the opened writable view.
+ * @param column A pointer on the column.
  * @param line_nb The number of the line where the value should be set.
  * @param element_name The name of the element that should be set in the line.
  * @param value The value that should be set.
@@ -248,9 +565,10 @@ int obi_column_set_obibool_with_elt_name_in_view(Obiview_p view, OBIDMS_column_p
 
 /**
  * @brief Recovers a value in an OBIDMS column containing data with the type OBI_BOOL,
- *        using the name of the element in the line.
+ *        using the name of the element in the line, in the context of a view.
  *
- * @param column A pointer as returned by obi_create_column() or obi_clone_column().
+ * @param view A pointer on the opened view.
+ * @param column A pointer on the column.
  * @param line_nb The number of the line where the value should be recovered.
  * @param element_name The name of the element that should be recovered in the line.
  *
@@ -264,11 +582,13 @@ obibool_t obi_column_get_obibool_with_elt_name_in_view(Obiview_p view, OBIDMS_co
 
 
 /**
- * @brief Sets a value in an OBIDMS column containing data with the type OBI_CHAR, using the index of the element in the line.
+ * @brief Sets a value in an OBIDMS column containing data with the type OBI_CHAR, using the index of the element in the line,
+ *        in the context of a view.
  *
- * @warning Pointers returned by obi_open_column() don't allow writing.
+ *  Note: If the column is read-only or if there is a line selection associated with the view (making columns non-writable), it is cloned.
  *
- * @param column A pointer as returned by obi_create_column() or obi_clone_column().
+ * @param view A pointer on the opened writable view.
+ * @param column A pointer on the column.
  * @param line_nb The number of the line where the value should be set.
  * @param element_idx The index of the element that should be set in the line.
  * @param value The value that should be set.
@@ -284,9 +604,10 @@ int obi_column_set_obichar_with_elt_idx_in_view(Obiview_p view, OBIDMS_column_p
 
 
 /**
- * @brief Recovers a value in an OBIDMS column containing data with the type OBI_CHAR.
+ * @brief Recovers a value in an OBIDMS column containing data with the type OBI_CHAR, in the context of a view.
  *
- * @param column A pointer as returned by obi_create_column().
+ * @param view A pointer on the opened view.
+ * @param column A pointer on the column.
  * @param line_nb The number of the line where the value should be recovered.
  * @param element_idx The index of the element that should be recovered in the line.
  *
@@ -301,11 +622,10 @@ obichar_t obi_column_get_obichar_with_elt_idx_in_view(Obiview_p view, OBIDMS_col
 
 /**
  * @brief Sets a value in an OBIDMS column containing data with the type OBI_CHAR,
- *        using the name of the element in the line.
+ *        using the name of the element in the line, in the context of a view.
  *
- * @warning Pointers returned by obi_open_column() don't allow writing.
- *
- * @param column A pointer as returned by obi_create_column() or obi_clone_column().
+ * @param view A pointer on the opened writable view.
+ * @param column A pointer on the column.
  * @param line_nb The number of the line where the value should be set.
  * @param element_name The name of the element that should be set in the line.
  * @param value The value that should be set.
@@ -322,9 +642,10 @@ int obi_column_set_obichar_with_elt_name_in_view(Obiview_p view, OBIDMS_column_p
 
 /**
  * @brief Recovers a value in an OBIDMS column containing data with the type OBI_CHAR,
- *        using the name of the element in the line.
+ *        using the name of the element in the line, in the context of a view.
  *
- * @param column A pointer as returned by obi_create_column() or obi_clone_column().
+ * @param view A pointer on the opened view.
+ * @param column A pointer on the column.
  * @param line_nb The number of the line where the value should be recovered.
  * @param element_name The name of the element that should be recovered in the line.
  *
@@ -338,11 +659,13 @@ obichar_t obi_column_get_obichar_with_elt_name_in_view(Obiview_p view, OBIDMS_co
 
 
 /**
- * @brief Sets a value in an OBIDMS column containing data with the type OBI_FLOAT, using the index of the element in the line.
+ * @brief Sets a value in an OBIDMS column containing data with the type OBI_FLOAT, using the index of the element in the line,
+ *        in the context of a view.
  *
- * @warning Pointers returned by obi_open_column() don't allow writing.
+ *  Note: If the column is read-only or if there is a line selection associated with the view (making columns non-writable), it is cloned.
  *
- * @param column A pointer as returned by obi_create_column() or obi_clone_column().
+ * @param view A pointer on the opened writable view.
+ * @param column A pointer on the column.
  * @param line_nb The number of the line where the value should be set.
  * @param element_idx The index of the element that should be set in the line.
  * @param value The value that should be set.
@@ -358,9 +681,10 @@ int obi_column_set_obifloat_with_elt_idx_in_view(Obiview_p view, OBIDMS_column_p
 
 
 /**
- * @brief Recovers a value in an OBIDMS column containing data with the type OBI_FLOAT.
+ * @brief Recovers a value in an OBIDMS column containing data with the type OBI_FLOAT, in the context of a view.
  *
- * @param column A pointer as returned by obi_create_column().
+ * @param view A pointer on the opened view.
+ * @param column A pointer on the column.
  * @param line_nb The number of the line where the value should be recovered.
  * @param element_idx The index of the element that should be recovered in the line.
  *
@@ -375,11 +699,10 @@ obifloat_t obi_column_get_obifloat_with_elt_idx_in_view(Obiview_p view, OBIDMS_c
 
 /**
  * @brief Sets a value in an OBIDMS column containing data with the type OBI_FLOAT,
- *        using the name of the element in the line.
+ *        using the name of the element in the line, in the context of a view.
  *
- * @warning Pointers returned by obi_open_column() don't allow writing.
- *
- * @param column A pointer as returned by obi_create_column() or obi_clone_column().
+ * @param view A pointer on the opened writable view.
+ * @param column A pointer on the column.
  * @param line_nb The number of the line where the value should be set.
  * @param element_name The name of the element that should be set in the line.
  * @param value The value that should be set.
@@ -396,9 +719,10 @@ int obi_column_set_obifloat_with_elt_name_in_view(Obiview_p view, OBIDMS_column_
 
 /**
  * @brief Recovers a value in an OBIDMS column containing data with the type OBI_FLOAT,
- *        using the name of the element in the line.
+ *        using the name of the element in the line, in the context of a view.
  *
- * @param column A pointer as returned by obi_create_column() or obi_clone_column().
+ * @param view A pointer on the opened view.
+ * @param column A pointer on the column.
  * @param line_nb The number of the line where the value should be recovered.
  * @param element_name The name of the element that should be recovered in the line.
  *
@@ -412,11 +736,13 @@ obifloat_t obi_column_get_obifloat_with_elt_name_in_view(Obiview_p view, OBIDMS_
 
 
 /**
- * @brief Sets a value in an OBIDMS column containing data with the type OBI_INT, using the index of the element in the line.
+ * @brief Sets a value in an OBIDMS column containing data with the type OBI_INT, using the index of the element in the line,
+ *        in the context of a view.
  *
- * @warning Pointers returned by obi_open_column() don't allow writing.
+ *  Note: If the column is read-only or if there is a line selection associated with the view (making columns non-writable), it is cloned.
  *
- * @param column A pointer as returned by obi_create_column() or obi_clone_column().
+ * @param view A pointer on the opened writable view.
+ * @param column A pointer on the column.
  * @param line_nb The number of the line where the value should be set.
  * @param element_idx The index of the element that should be set in the line.
  * @param value The value that should be set.
@@ -432,14 +758,15 @@ int obi_column_set_obiint_with_elt_idx_in_view(Obiview_p view, OBIDMS_column_p c
 
 
 /**
- * @brief Recovers a value in an OBIDMS column containing data with the type OBI_INT.
+ * @brief Recovers a value in an OBIDMS column containing data with the type OBI_INT, in the context of a view.
  *
- * @param column A pointer as returned by obi_create_column().
+ * @param view A pointer on the opened view.
+ * @param column A pointer on the column.
  * @param line_nb The number of the line where the value should be recovered.
  * @param element_idx The index of the element that should be recovered in the line.
  *
  * @returns The recovered value.
- * @retval OBIBool_NA the NA value of the type if an error occurred and obi_errno is set.
+ * @retval OBIInt_NA the NA value of the type if an error occurred and obi_errno is set.
  *
  * @since February 2016
  * @author Celine Mercier (celine.mercier@metabarcoding.org)
@@ -449,11 +776,10 @@ obiint_t obi_column_get_obiint_with_elt_idx_in_view(Obiview_p view, OBIDMS_colum
 
 /**
  * @brief Sets a value in an OBIDMS column containing data with the type OBI_INT,
- *        using the name of the element in the line.
+ *        using the name of the element in the line, in the context of a view.
  *
- * @warning Pointers returned by obi_open_column() don't allow writing.
- *
- * @param column A pointer as returned by obi_create_column() or obi_clone_column().
+ * @param view A pointer on the opened writable view.
+ * @param column A pointer on the column.
  * @param line_nb The number of the line where the value should be set.
  * @param element_name The name of the element that should be set in the line.
  * @param value The value that should be set.
@@ -470,14 +796,15 @@ int obi_column_set_obiint_with_elt_name_in_view(Obiview_p view, OBIDMS_column_p
 
 /**
  * @brief Recovers a value in an OBIDMS column containing data with the type OBI_INT,
- *        using the name of the element in the line.
+ *        using the name of the element in the line, in the context of a view.
  *
- * @param column A pointer as returned by obi_create_column() or obi_clone_column().
+ * @param view A pointer on the opened view.
+ * @param column A pointer on the column.
  * @param line_nb The number of the line where the value should be recovered.
  * @param element_name The name of the element that should be recovered in the line.
  *
  * @returns The recovered value.
- * @retval OBIBool_NA the NA value of the type if an error occurred and obi_errno is set.
+ * @retval OBIInt_NA the NA value of the type if an error occurred and obi_errno is set.
  *
  * @since February 2016
  * @author Celine Mercier (celine.mercier@metabarcoding.org)
@@ -486,12 +813,13 @@ obiint_t obi_column_get_obiint_with_elt_name_in_view(Obiview_p view, OBIDMS_colu
 
 
 /**
- * @brief Sets a value in an OBIDMS column containing data in the form of indices referring
- * to DNA sequences handled by an indexer, using the index of the element in the line.
+ * @brief Sets a value in an OBIDMS column containing data with the type OBI_SEQ, using the index of the element in the line,
+ *        in the context of a view.
  *
- * @warning Pointers returned by obi_open_column() don't allow writing.
+ *  Note: If the column is read-only or if there is a line selection associated with the view (making columns non-writable), it is cloned.
  *
- * @param column A pointer as returned by obi_create_column() or obi_clone_column().
+ * @param view A pointer on the opened writable view.
+ * @param column A pointer on the column.
  * @param line_nb The number of the line where the value should be set.
  * @param element_idx The index of the element that should be set in the line.
  * @param value The value that should be set.
@@ -507,15 +835,15 @@ int obi_column_set_obiseq_with_elt_idx_in_view(Obiview_p view, OBIDMS_column_p c
 
 
 /**
- * @brief Recovers a value in an OBIDMS column containing data in the form of indices referring
- * to DNA sequences handled by an indexer, using the index of the element in the line.
+ * @brief Recovers a value in an OBIDMS column containing data with the type OBI_SEQ, in the context of a view.
  *
- * @param column A pointer as returned by obi_create_column().
+ * @param view A pointer on the opened view.
+ * @param column A pointer on the column.
  * @param line_nb The number of the line where the value should be recovered.
  * @param element_idx The index of the element that should be recovered in the line.
  *
  * @returns The recovered value.
- * @retval '\0' the NA value of the type if an error occurred and obi_errno is set.
+ * @retval OBISeq_NA the NA value of the type if an error occurred and obi_errno is set.
  *
  * @since February 2016
  * @author Celine Mercier (celine.mercier@metabarcoding.org)
@@ -524,12 +852,11 @@ char* obi_column_get_obiseq_with_elt_idx_in_view(Obiview_p view, OBIDMS_column_p
 
 
 /**
- * @brief Sets a value in an OBIDMS column containing data in the form of indices referring
- * to DNA sequences handled by an indexer, using the name of the element in the line.
+ * @brief Sets a value in an OBIDMS column containing data with the type OBI_SEQ,
+ *        using the name of the element in the line, in the context of a view.
  *
- * @warning Pointers returned by obi_open_column() don't allow writing.
- *
- * @param column A pointer as returned by obi_create_column() or obi_clone_column().
+ * @param view A pointer on the opened writable view.
+ * @param column A pointer on the column.
  * @param line_nb The number of the line where the value should be set.
  * @param element_name The name of the element that should be set in the line.
  * @param value The value that should be set.
@@ -545,15 +872,16 @@ int obi_column_set_obiseq_with_elt_name_in_view(Obiview_p view, OBIDMS_column_p
 
 
 /**
- * @brief Recovers a value in an OBIDMS column containing data in the form of indices referring
- * to DNA sequences handled by an indexer, using the name of the element in the line.
+ * @brief Recovers a value in an OBIDMS column containing data with the type OBI_SEQ,
+ *        using the name of the element in the line, in the context of a view.
  *
- * @param column A pointer as returned by obi_create_column() or obi_clone_column().
+ * @param view A pointer on the opened view.
+ * @param column A pointer on the column.
  * @param line_nb The number of the line where the value should be recovered.
  * @param element_name The name of the element that should be recovered in the line.
  *
  * @returns The recovered value.
- * @retval '\0' the NA value of the type if an error occurred and obi_errno is set.
+ * @retval OBISeq_NA the NA value of the type if an error occurred and obi_errno is set.
  *
  * @since February 2016
  * @author Celine Mercier (celine.mercier@metabarcoding.org)
@@ -562,12 +890,13 @@ char* obi_column_get_obiseq_with_elt_name_in_view(Obiview_p view, OBIDMS_column_
 
 
 /**
- * @brief Sets a value in an OBIDMS column containing data in the form of indices referring
- * to character strings handled by an indexer, using the index of the element in the line.
+ * @brief Sets a value in an OBIDMS column containing data with the type OBI_STR, using the index of the element in the line,
+ *        in the context of a view.
  *
- * @warning Pointers returned by obi_open_column() don't allow writing.
+ *  Note: If the column is read-only or if there is a line selection associated with the view (making columns non-writable), it is cloned.
  *
- * @param column A pointer as returned by obi_create_column() or obi_clone_column().
+ * @param view A pointer on the opened writable view.
+ * @param column A pointer on the column.
  * @param line_nb The number of the line where the value should be set.
  * @param element_idx The index of the element that should be set in the line.
  * @param value The value that should be set.
@@ -583,15 +912,15 @@ int obi_column_set_obistr_with_elt_idx_in_view(Obiview_p view, OBIDMS_column_p c
 
 
 /**
- * @brief Recovers a value in an OBIDMS column containing data in the form of indices referring
- * to character strings handled by an indexer, using the index of the element in the line.
+ * @brief Recovers a value in an OBIDMS column containing data with the type OBI_STR, in the context of a view.
  *
- * @param column A pointer as returned by obi_create_column().
+ * @param view A pointer on the opened view.
+ * @param column A pointer on the column.
  * @param line_nb The number of the line where the value should be recovered.
  * @param element_idx The index of the element that should be recovered in the line.
  *
  * @returns The recovered value.
- * @retval '\0' the NA value of the type if an error occurred and obi_errno is set.
+ * @retval OBIStr_NA the NA value of the type if an error occurred and obi_errno is set.
  *
  * @since February 2016
  * @author Celine Mercier (celine.mercier@metabarcoding.org)
@@ -600,12 +929,11 @@ const char* obi_column_get_obistr_with_elt_idx_in_view(Obiview_p view, OBIDMS_co
 
 
 /**
- * @brief Sets a value in an OBIDMS column containing data in the form of indices referring
- * to character strings handled by an indexer, using the name of the element in the line.
+ * @brief Sets a value in an OBIDMS column containing data with the type OBI_STR,
+ *        using the name of the element in the line, in the context of a view.
  *
- * @warning Pointers returned by obi_open_column() don't allow writing.
- *
- * @param column A pointer as returned by obi_create_column() or obi_clone_column().
+ * @param view A pointer on the opened writable view.
+ * @param column A pointer on the column.
  * @param line_nb The number of the line where the value should be set.
  * @param element_name The name of the element that should be set in the line.
  * @param value The value that should be set.
@@ -621,15 +949,16 @@ int obi_column_set_obistr_with_elt_name_in_view(Obiview_p view, OBIDMS_column_p
 
 
 /**
- * @brief Recovers a value in an OBIDMS column containing data in the form of indices referring
- * to character strings handled by an indexer, using the name of the element in the line.
+ * @brief Recovers a value in an OBIDMS column containing data with the type OBI_STR,
+ *        using the name of the element in the line, in the context of a view.
  *
- * @param column A pointer as returned by obi_create_column() or obi_clone_column().
+ * @param view A pointer on the opened view.
+ * @param column A pointer on the column.
  * @param line_nb The number of the line where the value should be recovered.
  * @param element_name The name of the element that should be recovered in the line.
  *
  * @returns The recovered value.
- * @retval '\0' the NA value of the type if an error occurred and obi_errno is set.
+ * @retval OBIStr_NA the NA value of the type if an error occurred and obi_errno is set.
  *
  * @since February 2016
  * @author Celine Mercier (celine.mercier@metabarcoding.org)

src/utils.c

@@ -17,11 +17,11 @@
 #include <stdlib.h>
 #include <dirent.h>
 #include <unistd.h>
+#include <time.h>
 
 #include "utils.h"
 #include "obidebug.h"
 #include "obierrno.h"
-#include "obidms.h"
 
 
 #define DEBUG_LEVEL 0	// TODO has to be defined somewhere else (cython compil flag?)
@@ -35,17 +35,17 @@
  **********************************************************************/
 
 
-int count_dir(char *dir)
+int count_dir(char* dir_path)
 {
-	struct dirent *dp;
-	DIR *fd;
-	int count;
+	struct dirent* dp;
+	DIR*           fd;
+	int            count;
 
 	count = 0;
-	if ((fd = opendir(dir)) == NULL)
+	if ((fd = opendir(dir_path)) == NULL)
 	{
 		obi_set_errno(OBI_UTILS_ERROR);
-		obidebug(1, "Error opening a directory: %s\n", dir);
+		obidebug(1, "Error opening a directory: %s\n", dir_path);
 		return -1;
 	}
 	while ((dp = readdir(fd)) != NULL)
@@ -54,5 +54,33 @@ int count_dir(char *dir)
 			continue;
 		count++;
 	}
+
 	return count;
 }
+
+
+char* obi_format_date(time_t date)
+{
+	char*       formatted_time;
+	struct tm*  tmp;
+
+	formatted_time = (char*) malloc(FORMATTED_TIME_LENGTH*sizeof(char));
+
+	tmp = localtime(&date);
+	if (tmp == NULL)
+	{
+		obi_set_errno(OBICOL_UNKNOWN_ERROR);
+		obidebug(1, "\nError formatting a date");
+		return NULL;
+	}
+
+	if (strftime(formatted_time, FORMATTED_TIME_LENGTH, "%c", tmp) == 0)
+	{
+		obi_set_errno(OBICOL_UNKNOWN_ERROR);
+		obidebug(1, "\nError formatting a date");
+		return NULL;
+	}
+
+	return formatted_time;
+}
+

src/utils.h

@@ -19,14 +19,38 @@
 #include "obidms.h"
 
 
+#define FORMATTED_TIME_LENGTH (1024)    /**< The length allocated for the character string containing a formatted date.
+ 	 	 	 	 	 	 	 	 	     */
 #define ONE_IF_ZERO(x) (((x)==0)?1:(x))	/**< If x is equal to 0, x takes the value 1.
                                 	   	 */
 
 
-/*
- * TODO
+/**
+ * @brief Counts the number of files and directories in a directory.
+ *
+ * @param dir_path The absolute path of the directory.
+ *
+ * @returns The number of files and directories in the directory.
+ *
+ * @since April 2016
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
  */
-int count_dir(char *dir);
+int count_dir(char* dir_path);
+
+
+/**
+ * @brief Formats a date in a way that is easy to read.
+ *
+ * @warning The pointer returned must be freed by the caller.
+ *
+ * @param date A date.
+ *
+ * @returns The date formatted in a way that is easy to read.
+ *
+ * @since October 2015
+ * @author Celine Mercier (celine.mercier@metabarcoding.org)
+ */
+char* obi_format_date(time_t date);
 
 
 #endif /* UTILS_H_ */