Commented a loose print

any class where values are referenced by keys with an iterator

LICENSE

@@ -0,0 +1,518 @@
+
+  CeCILL FREE SOFTWARE LICENSE AGREEMENT
+
+Version 2.1 dated 2013-06-21
+
+
+    Notice
+
+This Agreement is a Free Software license agreement that is the result
+of discussions between its authors in order to ensure compliance with
+the two main principles guiding its drafting:
+
+  * firstly, compliance with the principles governing the distribution
+    of Free Software: access to source code, broad rights granted to users,
+  * secondly, the election of a governing law, French law, with which it
+    is conformant, both as regards the law of torts and intellectual
+    property law, and the protection that it offers to both authors and
+    holders of the economic rights over software.
+
+The authors of the CeCILL (for Ce[a] C[nrs] I[nria] L[ogiciel] L[ibre]) 
+license are: 
+
+Commissariat à l'énergie atomique et aux énergies alternatives - CEA, a
+public scientific, technical and industrial research establishment,
+having its principal place of business at 25 rue Leblanc, immeuble Le
+Ponant D, 75015 Paris, France.
+
+Centre National de la Recherche Scientifique - CNRS, a public scientific
+and technological establishment, having its principal place of business
+at 3 rue Michel-Ange, 75794 Paris cedex 16, France.
+
+Institut National de Recherche en Informatique et en Automatique -
+Inria, a public scientific and technological establishment, having its
+principal place of business at Domaine de Voluceau, Rocquencourt, BP
+105, 78153 Le Chesnay cedex, France.
+
+
+    Preamble
+
+The purpose of this Free Software license agreement is to grant users
+the right to modify and redistribute the software governed by this
+license within the framework of an open source distribution model.
+
+The exercising of this right is conditional upon certain obligations for
+users so as to preserve this status for all subsequent redistributions.
+
+In consideration of access to the source code and the rights to copy,
+modify and redistribute granted by the license, users are provided only
+with a limited warranty and the software's author, the holder of the
+economic rights, and the successive licensors only have limited liability.
+
+In this respect, the risks associated with loading, using, modifying
+and/or developing or reproducing the software by the user are brought to
+the user's attention, given its Free Software status, which may make it
+complicated to use, with the result that its use is reserved for
+developers and experienced professionals having in-depth computer
+knowledge. Users are therefore encouraged to load and test the
+suitability of the software as regards their requirements in conditions
+enabling the security of their systems and/or data to be ensured and,
+more generally, to use and operate it in the same conditions of
+security. This Agreement may be freely reproduced and published,
+provided it is not altered, and that no provisions are either added or
+removed herefrom.
+
+This Agreement may apply to any or all software for which the holder of
+the economic rights decides to submit the use thereof to its provisions.
+
+Frequently asked questions can be found on the official website of the
+CeCILL licenses family (http://www.cecill.info/index.en.html) for any 
+necessary clarification.
+
+
+    Article 1 - DEFINITIONS
+
+For the purpose of this Agreement, when the following expressions
+commence with a capital letter, they shall have the following meaning:
+
+Agreement: means this license agreement, and its possible subsequent
+versions and annexes.
+
+Software: means the software in its Object Code and/or Source Code form
+and, where applicable, its documentation, "as is" when the Licensee
+accepts the Agreement.
+
+Initial Software: means the Software in its Source Code and possibly its
+Object Code form and, where applicable, its documentation, "as is" when
+it is first distributed under the terms and conditions of the Agreement.
+
+Modified Software: means the Software modified by at least one
+Contribution.
+
+Source Code: means all the Software's instructions and program lines to
+which access is required so as to modify the Software.
+
+Object Code: means the binary files originating from the compilation of
+the Source Code.
+
+Holder: means the holder(s) of the economic rights over the Initial
+Software.
+
+Licensee: means the Software user(s) having accepted the Agreement.
+
+Contributor: means a Licensee having made at least one Contribution.
+
+Licensor: means the Holder, or any other individual or legal entity, who
+distributes the Software under the Agreement.
+
+Contribution: means any or all modifications, corrections, translations,
+adaptations and/or new functions integrated into the Software by any or
+all Contributors, as well as any or all Internal Modules.
+
+Module: means a set of sources files including their documentation that
+enables supplementary functions or services in addition to those offered
+by the Software.
+
+External Module: means any or all Modules, not derived from the
+Software, so that this Module and the Software run in separate address
+spaces, with one calling the other when they are run.
+
+Internal Module: means any or all Module, connected to the Software so
+that they both execute in the same address space.
+
+GNU GPL: means the GNU General Public License version 2 or any
+subsequent version, as published by the Free Software Foundation Inc.
+
+GNU Affero GPL: means the GNU Affero General Public License version 3 or
+any subsequent version, as published by the Free Software Foundation Inc.
+
+EUPL: means the European Union Public License version 1.1 or any
+subsequent version, as published by the European Commission.
+
+Parties: mean both the Licensee and the Licensor.
+
+These expressions may be used both in singular and plural form.
+
+
+    Article 2 - PURPOSE
+
+The purpose of the Agreement is the grant by the Licensor to the
+Licensee of a non-exclusive, transferable and worldwide license for the
+Software as set forth in Article 5 <#scope> hereinafter for the whole
+term of the protection granted by the rights over said Software.
+
+
+    Article 3 - ACCEPTANCE
+
+3.1 The Licensee shall be deemed as having accepted the terms and
+conditions of this Agreement upon the occurrence of the first of the
+following events:
+
+  * (i) loading the Software by any or all means, notably, by
+    downloading from a remote server, or by loading from a physical medium;
+  * (ii) the first time the Licensee exercises any of the rights granted
+    hereunder.
+
+3.2 One copy of the Agreement, containing a notice relating to the
+characteristics of the Software, to the limited warranty, and to the
+fact that its use is restricted to experienced users has been provided
+to the Licensee prior to its acceptance as set forth in Article 3.1
+<#accepting> hereinabove, and the Licensee hereby acknowledges that it
+has read and understood it.
+
+
+    Article 4 - EFFECTIVE DATE AND TERM
+
+
+      4.1 EFFECTIVE DATE
+
+The Agreement shall become effective on the date when it is accepted by
+the Licensee as set forth in Article 3.1 <#accepting>.
+
+
+      4.2 TERM
+
+The Agreement shall remain in force for the entire legal term of
+protection of the economic rights over the Software.
+
+
+    Article 5 - SCOPE OF RIGHTS GRANTED
+
+The Licensor hereby grants to the Licensee, who accepts, the following
+rights over the Software for any or all use, and for the term of the
+Agreement, on the basis of the terms and conditions set forth hereinafter.
+
+Besides, if the Licensor owns or comes to own one or more patents
+protecting all or part of the functions of the Software or of its
+components, the Licensor undertakes not to enforce the rights granted by
+these patents against successive Licensees using, exploiting or
+modifying the Software. If these patents are transferred, the Licensor
+undertakes to have the transferees subscribe to the obligations set
+forth in this paragraph.
+
+
+      5.1 RIGHT OF USE
+
+The Licensee is authorized to use the Software, without any limitation
+as to its fields of application, with it being hereinafter specified
+that this comprises:
+
+ 1. permanent or temporary reproduction of all or part of the Software
+    by any or all means and in any or all form.
+
+ 2. loading, displaying, running, or storing the Software on any or all
+    medium.
+
+ 3. entitlement to observe, study or test its operation so as to
+    determine the ideas and principles behind any or all constituent
+    elements of said Software. This shall apply when the Licensee
+    carries out any or all loading, displaying, running, transmission or
+    storage operation as regards the Software, that it is entitled to
+    carry out hereunder.
+
+
+      5.2 ENTITLEMENT TO MAKE CONTRIBUTIONS
+
+The right to make Contributions includes the right to translate, adapt,
+arrange, or make any or all modifications to the Software, and the right
+to reproduce the resulting software.
+
+The Licensee is authorized to make any or all Contributions to the
+Software provided that it includes an explicit notice that it is the
+author of said Contribution and indicates the date of the creation thereof.
+
+
+      5.3 RIGHT OF DISTRIBUTION
+
+In particular, the right of distribution includes the right to publish,
+transmit and communicate the Software to the general public on any or
+all medium, and by any or all means, and the right to market, either in
+consideration of a fee, or free of charge, one or more copies of the
+Software by any means.
+
+The Licensee is further authorized to distribute copies of the modified
+or unmodified Software to third parties according to the terms and
+conditions set forth hereinafter.
+
+
+        5.3.1 DISTRIBUTION OF SOFTWARE WITHOUT MODIFICATION
+
+The Licensee is authorized to distribute true copies of the Software in
+Source Code or Object Code form, provided that said distribution
+complies with all the provisions of the Agreement and is accompanied by:
+
+ 1. a copy of the Agreement,
+
+ 2. a notice relating to the limitation of both the Licensor's warranty
+    and liability as set forth in Articles 8 and 9,
+
+and that, in the event that only the Object Code of the Software is
+redistributed, the Licensee allows effective access to the full Source
+Code of the Software for a period of at least three years from the
+distribution of the Software, it being understood that the additional
+acquisition cost of the Source Code shall not exceed the cost of the
+data transfer.
+
+
+        5.3.2 DISTRIBUTION OF MODIFIED SOFTWARE
+
+When the Licensee makes a Contribution to the Software, the terms and
+conditions for the distribution of the resulting Modified Software
+become subject to all the provisions of this Agreement.
+
+The Licensee is authorized to distribute the Modified Software, in
+source code or object code form, provided that said distribution
+complies with all the provisions of the Agreement and is accompanied by:
+
+ 1. a copy of the Agreement,
+
+ 2. a notice relating to the limitation of both the Licensor's warranty
+    and liability as set forth in Articles 8 and 9,
+
+and, in the event that only the object code of the Modified Software is
+redistributed,
+
+ 3. a note stating the conditions of effective access to the full source
+    code of the Modified Software for a period of at least three years
+    from the distribution of the Modified Software, it being understood
+    that the additional acquisition cost of the source code shall not
+    exceed the cost of the data transfer.
+
+
+        5.3.3 DISTRIBUTION OF EXTERNAL MODULES
+
+When the Licensee has developed an External Module, the terms and
+conditions of this Agreement do not apply to said External Module, that
+may be distributed under a separate license agreement.
+
+
+        5.3.4 COMPATIBILITY WITH OTHER LICENSES
+
+The Licensee can include a code that is subject to the provisions of one
+of the versions of the GNU GPL, GNU Affero GPL and/or EUPL in the
+Modified or unmodified Software, and distribute that entire code under
+the terms of the same version of the GNU GPL, GNU Affero GPL and/or EUPL.
+
+The Licensee can include the Modified or unmodified Software in a code
+that is subject to the provisions of one of the versions of the GNU GPL,
+GNU Affero GPL and/or EUPL and distribute that entire code under the
+terms of the same version of the GNU GPL, GNU Affero GPL and/or EUPL.
+
+
+    Article 6 - INTELLECTUAL PROPERTY
+
+
+      6.1 OVER THE INITIAL SOFTWARE
+
+The Holder owns the economic rights over the Initial Software. Any or
+all use of the Initial Software is subject to compliance with the terms
+and conditions under which the Holder has elected to distribute its work
+and no one shall be entitled to modify the terms and conditions for the
+distribution of said Initial Software.
+
+The Holder undertakes that the Initial Software will remain ruled at
+least by this Agreement, for the duration set forth in Article 4.2 <#term>.
+
+
+      6.2 OVER THE CONTRIBUTIONS
+
+The Licensee who develops a Contribution is the owner of the
+intellectual property rights over this Contribution as defined by
+applicable law.
+
+
+      6.3 OVER THE EXTERNAL MODULES
+
+The Licensee who develops an External Module is the owner of the
+intellectual property rights over this External Module as defined by
+applicable law and is free to choose the type of agreement that shall
+govern its distribution.
+
+
+      6.4 JOINT PROVISIONS
+
+The Licensee expressly undertakes:
+
+ 1. not to remove, or modify, in any manner, the intellectual property
+    notices attached to the Software;
+
+ 2. to reproduce said notices, in an identical manner, in the copies of
+    the Software modified or not.
+
+The Licensee undertakes not to directly or indirectly infringe the
+intellectual property rights on the Software of the Holder and/or
+Contributors, and to take, where applicable, vis-à-vis its staff, any
+and all measures required to ensure respect of said intellectual
+property rights of the Holder and/or Contributors.
+
+
+    Article 7 - RELATED SERVICES
+
+7.1 Under no circumstances shall the Agreement oblige the Licensor to
+provide technical assistance or maintenance services for the Software.
+
+However, the Licensor is entitled to offer this type of services. The
+terms and conditions of such technical assistance, and/or such
+maintenance, shall be set forth in a separate instrument. Only the
+Licensor offering said maintenance and/or technical assistance services
+shall incur liability therefor.
+
+7.2 Similarly, any Licensor is entitled to offer to its licensees, under
+its sole responsibility, a warranty, that shall only be binding upon
+itself, for the redistribution of the Software and/or the Modified
+Software, under terms and conditions that it is free to decide. Said
+warranty, and the financial terms and conditions of its application,
+shall be subject of a separate instrument executed between the Licensor
+and the Licensee.
+
+
+    Article 8 - LIABILITY
+
+8.1 Subject to the provisions of Article 8.2, the Licensee shall be
+entitled to claim compensation for any direct loss it may have suffered
+from the Software as a result of a fault on the part of the relevant
+Licensor, subject to providing evidence thereof.
+
+8.2 The Licensor's liability is limited to the commitments made under
+this Agreement and shall not be incurred as a result of in particular:
+(i) loss due the Licensee's total or partial failure to fulfill its
+obligations, (ii) direct or consequential loss that is suffered by the
+Licensee due to the use or performance of the Software, and (iii) more
+generally, any consequential loss. In particular the Parties expressly
+agree that any or all pecuniary or business loss (i.e. loss of data,
+loss of profits, operating loss, loss of customers or orders,
+opportunity cost, any disturbance to business activities) or any or all
+legal proceedings instituted against the Licensee by a third party,
+shall constitute consequential loss and shall not provide entitlement to
+any or all compensation from the Licensor.
+
+
+    Article 9 - WARRANTY
+
+9.1 The Licensee acknowledges that the scientific and technical
+state-of-the-art when the Software was distributed did not enable all
+possible uses to be tested and verified, nor for the presence of
+possible defects to be detected. In this respect, the Licensee's
+attention has been drawn to the risks associated with loading, using,
+modifying and/or developing and reproducing the Software which are
+reserved for experienced users.
+
+The Licensee shall be responsible for verifying, by any or all means,
+the suitability of the product for its requirements, its good working
+order, and for ensuring that it shall not cause damage to either persons
+or properties.
+
+9.2 The Licensor hereby represents, in good faith, that it is entitled
+to grant all the rights over the Software (including in particular the
+rights set forth in Article 5 <#scope>).
+
+9.3 The Licensee acknowledges that the Software is supplied "as is" by
+the Licensor without any other express or tacit warranty, other than
+that provided for in Article 9.2 <#good-faith> and, in particular,
+without any warranty as to its commercial value, its secured, safe,
+innovative or relevant nature.
+
+Specifically, the Licensor does not warrant that the Software is free
+from any error, that it will operate without interruption, that it will
+be compatible with the Licensee's own equipment and software
+configuration, nor that it will meet the Licensee's requirements.
+
+9.4 The Licensor does not either expressly or tacitly warrant that the
+Software does not infringe any third party intellectual property right
+relating to a patent, software or any other property right. Therefore,
+the Licensor disclaims any and all liability towards the Licensee
+arising out of any or all proceedings for infringement that may be
+instituted in respect of the use, modification and redistribution of the
+Software. Nevertheless, should such proceedings be instituted against
+the Licensee, the Licensor shall provide it with technical and legal
+expertise for its defense. Such technical and legal expertise shall be
+decided on a case-by-case basis between the relevant Licensor and the
+Licensee pursuant to a memorandum of understanding. The Licensor
+disclaims any and all liability as regards the Licensee's use of the
+name of the Software. No warranty is given as regards the existence of
+prior rights over the name of the Software or as regards the existence
+of a trademark.
+
+
+    Article 10 - TERMINATION
+
+10.1 In the event of a breach by the Licensee of its obligations
+hereunder, the Licensor may automatically terminate this Agreement
+thirty (30) days after notice has been sent to the Licensee and has
+remained ineffective.
+
+10.2 A Licensee whose Agreement is terminated shall no longer be
+authorized to use, modify or distribute the Software. However, any
+licenses that it may have granted prior to termination of the Agreement
+shall remain valid subject to their having been granted in compliance
+with the terms and conditions hereof.
+
+
+    Article 11 - MISCELLANEOUS
+
+
+      11.1 EXCUSABLE EVENTS
+
+Neither Party shall be liable for any or all delay, or failure to
+perform the Agreement, that may be attributable to an event of force
+majeure, an act of God or an outside cause, such as defective
+functioning or interruptions of the electricity or telecommunications
+networks, network paralysis following a virus attack, intervention by
+government authorities, natural disasters, water damage, earthquakes,
+fire, explosions, strikes and labor unrest, war, etc.
+
+11.2 Any failure by either Party, on one or more occasions, to invoke
+one or more of the provisions hereof, shall under no circumstances be
+interpreted as being a waiver by the interested Party of its right to
+invoke said provision(s) subsequently.
+
+11.3 The Agreement cancels and replaces any or all previous agreements,
+whether written or oral, between the Parties and having the same
+purpose, and constitutes the entirety of the agreement between said
+Parties concerning said purpose. No supplement or modification to the
+terms and conditions hereof shall be effective as between the Parties
+unless it is made in writing and signed by their duly authorized
+representatives.
+
+11.4 In the event that one or more of the provisions hereof were to
+conflict with a current or future applicable act or legislative text,
+said act or legislative text shall prevail, and the Parties shall make
+the necessary amendments so as to comply with said act or legislative
+text. All other provisions shall remain effective. Similarly, invalidity
+of a provision of the Agreement, for any reason whatsoever, shall not
+cause the Agreement as a whole to be invalid.
+
+
+      11.5 LANGUAGE
+
+The Agreement is drafted in both French and English and both versions
+are deemed authentic.
+
+
+    Article 12 - NEW VERSIONS OF THE AGREEMENT
+
+12.1 Any person is authorized to duplicate and distribute copies of this
+Agreement.
+
+12.2 So as to ensure coherence, the wording of this Agreement is
+protected and may only be modified by the authors of the License, who
+reserve the right to periodically publish updates or new versions of the
+Agreement, each with a separate number. These subsequent versions may
+address new issues encountered by Free Software.
+
+12.3 Any Software distributed under a given version of the Agreement may
+only be subsequently distributed under the same version of the Agreement
+or a subsequent version, subject to the provisions of Article 5.3.4
+<#compatibility>.
+
+
+    Article 13 - GOVERNING LAW AND JURISDICTION
+
+13.1 The Agreement is governed by French law. The Parties agree to
+endeavor to seek an amicable solution to any disagreements or disputes
+that may arise during the performance of the Agreement.
+
+13.2 Failing an amicable solution within two (2) months as from their
+occurrence, and unless emergency proceedings are necessary, the
+disagreements or disputes shall be referred to the Paris Courts having
+jurisdiction, by the more diligent Party.

README.md

@@ -0,0 +1,40 @@
+The `OBITools3`: A package for the management of analyses and data in DNA metabarcoding   
+---------------------------------------------
+
+
+
+DNA metabarcoding offers new perspectives for biodiversity research [1]. This approach of ecosystem studies relies heavily on the use of Next-Generation Sequencing (NGS), and consequently requires the ability to to treat large volumes of data. The `OBITools` package  satisfies this requirement thanks to a set of programs specifically designed for analyzing NGS data in a DNA metabarcoding context [2] - <http://metabarcoding.org/obitools>. Their capacity to filter and edit sequences while taking into account taxonomic annotation helps to setup tailored-made analysis pipelines for a broad range of DNA metabarcoding applications, including biodiversity surveys or diet analyses.   
+
+
+
+**The `OBITools3`.** This new version of the `OBITools` looks to significantly improve the storage efficiency and the data processing speed. To this end, the `OBITools3` rely on an ad hoc database system, inside which all the data that a DNA metabarcoding experiment must consider is stored: the sequences, the metadata (describing for instance the samples), the database containing the reference sequences used for the taxonomic annotation, as well as the taxonomic databases. Besides the gain in efficiency, this new structure allows an easier access to all the data associated with an experiment.   
+
+
+
+**Column-oriented storage.** An analysis pipeline corresponds to a succession of commands, each computing one step of the analysis, and where the result of the command *n* is used by the command *n+1*. DNA metabarcoding data can easily be represented in the form of tables, and each command can be regarded as an operation transforming one or several 'input' tables into one or several 'output' tables, which can be used by the next command. Many of the basic operations in a pipeline copy without modification an important part of the input tables to the result tables, and use for their calculations only a small part of the input data. In the original `OBITools`, those tables are kept in the form of annotated sequence files in the FASTA or FASTQ format. This has two consequences: i) keeping the transitional results of the analysis pipeline means using disk space for an important volume of redundant data, ii) The coding and decoding of informations that are not actually used represent an important part of the treatment process. The new database system used by the `OBITools3` (called DMS for Data Management System) relies on column-oriented storage. The columns are immutable and can be assembled in views representing the data tables. This way, the data not modified by a command in an input table can easily be associated to the result table without duplicating any information ; and the data not used at all by a command can be associated with the result table without being read. This strategy results in a gain in disk space efficiency by limiting data redundancy, as well as a gain in execution time by limiting data reading, writing and conversion operations. Finally, as a mean to optimize data access, each column is stored in a binary file directly mapped in memory for reading and writing operations.   
+
+
+
+**Storage optimization.** DNA metabarcoding data is intrinsically very redundant. For example, the same sequence corresponding to a species will be present several thousand times across all samples. In order to limit the disk space used and make comparison operations more efficient, data in the form of character strings is stored in columns using a complex indexing structure, efficient on millions of values, coupling hash functions, Bloom filters and AVL trees. Finally, DNA sequences are compressed by encoding each nucleotide on two or four bits depending on whether the sequences contain only the four nucleotides (A, C, G, T) or use the IUPAC codes.   
+
+
+
+**Saving the data processing history.** The totality of the informations used by the `OBITools3` is stored in immutable data structures in the DMS. If a command has to modify a column used as input to produce its result, a new version of that column is created, leaving the initial version intact. This storage system enables to keep, at minimal cost, the totality of the transitional results produced by the pipeline. The storage of metadata describing all the operations that have produced a view (a result table) in the DMS makes possible the creation of an oriented hypergraph, where each node corresponds to a view and each arrow to an operation. By retracing the dependency relationships in this hypergraph, it is possible to rebuild *a posteriori* the entirety of the process that has produced a result table.   
+
+
+
+**Tools.** The `OBITools3` offer the same tools as the original `OBITools`. Eventually, new versions of `ecoPrimers` (PCR primer design) [3], `ecoPCR` (*in silico* PCR) [4], as well as `Sumatra` (sequence alignment) and `Sumaclust` (sequence alignment and clustering) [5] will be added, taking advantage of the database structure developed for the `OBITools3`.    
+
+
+
+**Implementation and disponibility.** The lower layers managing the DMS as well as all the compute-intensive functions are coded in `C99` for efficiency reasons. A `Cython` (<http://www.cython.org>) object layer allows for a simple but efficient implementation of the `OBITools3` commands in `Python 3.5`. The `OBITools3` are still in development, and the first functional versions are expected for autumn 2016.    
+
+
+
+**References.**
+
+1. Taberlet P, Coissac E, Hajibabaei M, Rieseberg LH: Environmental DNA. Mol Ecol 2012:1789–1793.
+2. Boyer F, Mercier C, Bonin A, Le Bras Y, Taberlet P, Coissac E: OBITools: a Unix-inspired software package for DNA metabarcoding. Mol Ecol Resour 2015:n/a–n/a.
+3. Riaz T, Shehzad W, Viari A, Pompanon F, Taberlet P, Coissac E: ecoPrimers: inference of new DNA barcode markers from whole genome sequence analysis. Nucleic Acids Res 2011, 39:e145.
+4. Ficetola GF, Coissac E, Zundel S, Riaz T, Shehzad W, Bessière J, Taberlet P, Pompanon F: An in silico approach for the evaluation of DNA barcodes. BMC Genomics 2010, 11:434.
+5. Mercier C, Boyer F, Bonin A, Coissac E (2013) SUMATRA and SUMACLUST: fast and exact comparison and clustering of sequences. Available: <http://metabarcoding.org/sumatra> and <http://metabarcoding.org/sumaclust>

distutils.ext/obidistutils/command/build_ext.py

@@ -7,107 +7,109 @@ Created on 13 fevr. 2014
 from distutils import log
 import os
 
-from Cython.Distutils import build_ext  as ori_build_ext  # @UnresolvedImport
-
-from Cython.Compiler import Options as cython_options  # @UnresolvedImport
 
 from distutils.errors import DistutilsSetupError
 
-class build_ext(ori_build_ext):
+try:
-
+    from Cython.Distutils import build_ext  as ori_build_ext  # @UnresolvedImport
+    from Cython.Compiler import Options as cython_options  # @UnresolvedImport
+    class build_ext(ori_build_ext):
     
-    def modifyDocScripts(self):
-        build_dir_file=open("doc/sphinx/build_dir.txt","w")
-        print(self.build_lib,file=build_dir_file)
-        build_dir_file.close()
         
-    def initialize_options(self):
+        def modifyDocScripts(self):
-        ori_build_ext.initialize_options(self)  # @UndefinedVariable
+            build_dir_file=open("doc/sphinx/build_dir.txt","w")
-        self.littlebigman = None
+            print(self.build_lib,file=build_dir_file)
-        self.built_files = None
+            build_dir_file.close()
-
-    
-    def finalize_options(self):
-        ori_build_ext.finalize_options(self)  # @UndefinedVariable
-
-        self.set_undefined_options('littlebigman',
-                                   ('littlebigman',  'littlebigman'))
-        
-        self.set_undefined_options('build_files',
-                                   ('files',  'built_files'))
-
-        self.cython_c_in_temp = 1
-                   
-        if self.littlebigman =='-DLITTLE_END':
-            if self.define is None:
-                self.define=[('LITTLE_END',None)]
-            else:
-                self.define.append('LITTLE_END',None)
-        
-    def substitute_sources(self,exe_name,sources):
-        """
-        Substitutes source file name starting by an @ by the actual
-        name of the built file (see --> build_files)
-        """
-        sources = list(sources)
-        for i in range(len(sources)):
-            message = "%s :-> %s" % (exe_name,sources[i])
-            if sources[i][0]=='@':
-                try:
-                    filename = self.built_files[sources[i][1:]]
-                except KeyError:
-                    tmpfilename = os.path.join(self.build_temp,sources[i][1:])
-                    if os.path.isfile       (tmpfilename):
-                        filename = tmpfilename
-                    else:
-                        raise DistutilsSetupError(
-                             'The %s filename declared in the source '
-                             'files of the program %s have not been '
-                             'built by the installation process' % (sources[i],
-                                                                     exe_name))
-                sources[i]=filename
-                log.info("%s changed to %s",message,filename)
-            else:
-                log.info("%s ok",message)
-
-        return sources
-
-    def build_extensions(self):
-        # First, sanity-check the 'extensions' list
-        
-        for ext in self.extensions:
-            ext.sources = self.substitute_sources(ext.name,ext.sources)
             
-        self.check_extensions_list(self.extensions)
+        def initialize_options(self):
-
+            ori_build_ext.initialize_options(self)  # @UndefinedVariable
-        for ext in self.extensions:
+            self.littlebigman = None
-            log.info("%s :-> %s",ext.name,ext.sources)
+            self.built_files = None
-            ext.sources = self.cython_sources(ext.sources, ext)
-            self.build_extension(ext)
-
-        
-    def run(self):
-        self.modifyDocScripts()
-
-        for cmd_name in self.get_sub_commands():
-            self.run_command(cmd_name)
-
-        cython_options.annotate = True
-        ori_build_ext.run(self)  # @UndefinedVariable
-        
-
-    def has_files(self):
-        return self.distribution.has_files()
-
-    def has_executables(self):
-        return self.distribution.has_executables()
     
-    sub_commands = [('build_files',has_files),
-                    ('build_cexe', has_executables)
-                    ] + \
-                   ori_build_ext.sub_commands
-
         
+        def finalize_options(self):
+            ori_build_ext.finalize_options(self)  # @UndefinedVariable
+    
+            self.set_undefined_options('littlebigman',
+                                       ('littlebigman',  'littlebigman'))
+            
+            self.set_undefined_options('build_files',
+                                       ('files',  'built_files'))
+    
+            self.cython_c_in_temp = 1
+                       
+            if self.littlebigman =='-DLITTLE_END':
+                if self.define is None:
+                    self.define=[('LITTLE_END',None)]
+                else:
+                    self.define.append('LITTLE_END',None)
+            
+        def substitute_sources(self,exe_name,sources):
+            """
+            Substitutes source file name starting by an @ by the actual
+            name of the built file (see --> build_files)
+            """
+            sources = list(sources)
+            for i in range(len(sources)):
+                message = "%s :-> %s" % (exe_name,sources[i])
+                if sources[i][0]=='@':
+                    try:
+                        filename = self.built_files[sources[i][1:]]
+                    except KeyError:
+                        tmpfilename = os.path.join(self.build_temp,sources[i][1:])
+                        if os.path.isfile       (tmpfilename):
+                            filename = tmpfilename
+                        else:
+                            raise DistutilsSetupError(
+                                 'The %s filename declared in the source '
+                                 'files of the program %s have not been '
+                                 'built by the installation process' % (sources[i],
+                                                                         exe_name))
+                    sources[i]=filename
+                    log.info("%s changed to %s",message,filename)
+                else:
+                    log.info("%s ok",message)
+    
+            return sources
+    
+        def build_extensions(self):
+            # First, sanity-check the 'extensions' list
+            
+            for ext in self.extensions:
+                ext.sources = self.substitute_sources(ext.name,ext.sources)
+                
+            self.check_extensions_list(self.extensions)
+    
+            for ext in self.extensions:
+                log.info("%s :-> %s",ext.name,ext.sources)
+                ext.sources = self.cython_sources(ext.sources, ext)
+                self.build_extension(ext)
+    
+            
+        def run(self):
+            self.modifyDocScripts()
+    
+            for cmd_name in self.get_sub_commands():
+                self.run_command(cmd_name)
+    
+            cython_options.annotate = True
+            ori_build_ext.run(self)  # @UndefinedVariable
+            
+    
+        def has_files(self):
+            return self.distribution.has_files()
+    
+        def has_executables(self):
+            return self.distribution.has_executables()
+        
+        sub_commands = [('build_files',has_files),
+                        ('build_cexe', has_executables)
+                        ] + \
+                       ori_build_ext.sub_commands
+
+except ImportError:
+    from distutils.command import build_ext  # @UnusedImport
+           
 
 
         

distutils.ext/obidistutils/core.py

@@ -40,6 +40,7 @@ def findPackage(root,base=None):
     
 def findCython(root,base=None,pyrexs=None):
     setupdir = os.path.dirname(sys.argv[0])
+    csourcedir = os.path.join(setupdir,"src")
     pyrexs=[]
 
     if base is None:
@@ -53,6 +54,7 @@ def findCython(root,base=None,pyrexs=None):
                                     [pyrex]
                                     )
                           )
+            pyrexs[-1].include_dirs.append(csourcedir)
             try:
                 cfiles = os.path.splitext(pyrex)[0]+".cfiles"
                 cfilesdir = os.path.dirname(cfiles)

distutils.ext/obidistutils/serenity/bootstrappip.py

@@ -0,0 +1,36 @@
+'''
+Created on 22 janv. 2016
+
+@author: coissac
+'''
+
+import sys
+from urllib import request
+import os.path
+
+from obidistutils.serenity.util import get_serenity_dir
+from obidistutils.serenity.rerun import rerun_with_anothe_python
+from obidistutils.serenity.checkpython import is_a_virtualenv_python
+
+getpipurl="https://bootstrap.pypa.io/get-pip.py"
+
+def bootstrap():
+
+    getpipfile=os.path.join(get_serenity_dir(),"get-pip.py")
+    
+    with request.urlopen(getpipurl) as getpip:
+        with open(getpipfile,"wb") as out:
+            for l in getpip:
+                out.write(l)
+                
+    python = sys.executable
+    
+    if is_a_virtualenv_python():
+        command= "%s %s" % (python,getpipfile)        
+    else:
+        command= "%s %s --user" % (python,getpipfile)
+    
+    os.system(command)
+    
+    rerun_with_anothe_python(python)
+    

distutils.ext/obidistutils/serenity/checkpackage.py

@@ -7,8 +7,13 @@ Created on 2 oct. 2014
 import re
 import os
 
-import pip                                              # @UnresolvedImport
+try:
-from pip.utils import get_installed_distributions       # @UnresolvedImport
+    import pip                                              # @UnresolvedImport
+    from pip.utils import get_installed_distributions       # @UnresolvedImport
+except ImportError:
+    from .bootstrappip import bootstrap
+    bootstrap()
+
 from distutils.version import StrictVersion             # @UnusedImport
 from distutils.errors import DistutilsError
 from distutils import log

distutils.ext/obidistutils/serenity/virtual.py

@@ -59,7 +59,7 @@ def serenity_virtualenv(envname,package,version,minversion='3.4',maxversion=None
                     clear=True, 
                     symlinks=False, 
                     with_pip=True)
-        
+                
         # check the newly created virtualenv
         return serenity_virtualenv(envname,package,version)
     

doc/conf.py

@@ -36,7 +36,7 @@ extensions = [
     'sphinx.ext.pngmath',
     'sphinx.ext.ifconfig',
     'sphinx.ext.viewcode',
-#    'breathe',
+    'breathe',
 ]
 
 # Add any paths that contain templates here, relative to this directory.
@@ -295,4 +295,6 @@ texinfo_documents = [
 sys.path.append( "breathe/" )
 breathe_projects = { "OBITools3": "doxygen/xml/" }
 breathe_default_project = "OBITools3"
-
+#breathe_projects_source = {
+#     "auto" : ( "../src", ["obidms.h", "obiavl.h"] )
+#     }

doc/source/index.rst

@@ -11,7 +11,7 @@ OBITools3 documentation
 
    Programming guidelines <guidelines>
    Data structures <data>
-
+   Code documentation <code_doc/codedoc>
 
 Indices and tables
 ------------------

doc/source/types.rst

@@ -4,6 +4,7 @@ OBITypes
 
     
 .. image:: ./UML/OBITypes_UML.png
+
 :download:`html version of the OBITypes UML file <UML/OBITypes_UML.class.violet.html>`
 
 

doc/sphinx/build_dir.txt

@@ -1 +1 @@
-build/lib.macosx-10.6-intel-3.5
+build/lib.macosx-10.6-intel-3.4

python/obi.py

@@ -0,0 +1,52 @@
+#!/usr/local/bin/python3.4
+'''
+obi -- shortdesc
+
+obi is a description
+
+It defines classes_and_methods
+
+@author:     user_name
+
+@copyright:  2014 organization_name. All rights reserved.
+
+@license:    license
+
+@contact:    user_email
+@deffield    updated: Updated
+'''
+
+
+default_config = { 'software'       : "The OBITools",
+                   'log'            : False,
+                   'loglevel'       : 'INFO',
+                   'progress'       : True,
+                   'defaultdms'     : None,
+                   'inputview'      : None,
+                   'outputview'     : None
+                  }
+
+root_config_name='obi'
+
+from obitools3.apps.config import getConfiguration     # @UnresolvedImport
+from obitools3.version import version
+
+__all__     = []
+__version__ = version
+__date__    = '2014-09-28'
+__updated__ = '2014-09-28'
+
+DEBUG = 1
+TESTRUN = 0
+PROFILE = 0
+
+
+if __name__ =="__main__":
+    
+    config = getConfiguration(root_config_name,
+                              default_config)    
+                
+    config[root_config_name]['module'].run(config)
+    
+    
+    

python/obitools3/apps/arguments.pxd

@@ -0,0 +1,3 @@
+#cython: language_level=3
+
+cpdef buildArgumentParser(str configname, str softname)

python/obitools3/apps/arguments.pyx

@@ -0,0 +1,61 @@
+#cython: language_level=3
+
+'''
+Created on 27 mars 2016
+
+@author: coissac
+'''
+
+import argparse
+import sys
+
+
+from .command import getCommandsList
+
+class ObiParser(argparse.ArgumentParser): 
+    def error(self, message):
+        sys.stderr.write('error: %s\n' % message)
+        self.print_help()
+        sys.exit(2)
+
+cpdef buildArgumentParser(str configname, 
+                          str softname):
+    parser = ObiParser()
+    
+    parser.add_argument('--version',   dest='%s:version' % configname, 
+                                       action='store_true', 
+                                       default=False, 
+                        help='Print the version of %s' % softname)
+
+    parser.add_argument('--log',       dest='%s:log' % configname, 
+                                       action='store',
+                                       type=str,
+                                       default=None, 
+                        help='Create a logfile')
+
+    parser.add_argument('--no-progress', dest='%s:progress' % configname, 
+                                       action='store_false', 
+                                       default=None, 
+                        help='Do not print the progress bar during analyzes')
+
+    subparsers = parser.add_subparsers(title='subcommands',
+                                       description='valid subcommands',
+                                       help='additional help')
+    
+    commands = getCommandsList()
+    
+    for c in commands:
+        module = commands[c]
+        
+        if hasattr(module, "run"):
+            if hasattr(module, "__title__"):
+                sub = subparsers.add_parser(c,help=module.__title__)
+            else:
+                sub = subparsers.add_parser(c)
+    
+            if hasattr(module, "addOptions"):
+                module.addOptions(sub)
+            
+            sub.set_defaults(**{'%s:module'  % configname : module})
+                        
+    return parser

python/obitools3/apps/command.pxd

@@ -0,0 +1,3 @@
+#cython: language_level=3
+
+cdef object loadCommand(str name,loader)

python/obitools3/apps/command.pyx

@@ -0,0 +1,44 @@
+#cython: language_level=3
+
+'''
+Created on 27 mars 2016
+
+@author: coissac
+'''
+
+import pkgutil
+
+from obitools3 import commands
+
+cdef object loadCommand(str name,loader):
+    '''
+    Load a command module from its name and an ImpLoader
+    
+    This function is for internal use
+    
+    @param name:   name of the module
+    @type name: str 
+    @param loader: the module loader
+    @type loader: ImpLoader
+    
+    @return the loaded module
+    @rtype: module 
+    '''
+    
+    module = loader.find_module(name).load_module(name)
+    return module
+
+def getCommandsList():
+    '''
+    Returns the list of sub-commands available to the main `obi` command
+    
+    @return: a dict instance with key corresponding to each command and
+             value corresponding to the module
+             
+    @rtype: dict
+    '''
+    
+    cdef dict cmds = dict((x[1],loadCommand(x[1],x[0])) 
+                           for x in pkgutil.iter_modules(commands.__path__) 
+                           if not x[2])
+    return cmds

python/obitools3/apps/config.pxd

@@ -0,0 +1,10 @@
+#cython: language_level=3
+
+cpdef str setRootConfigName(str rootname)
+cpdef str getRootConfigName()
+
+cdef dict buildDefaultConfiguration(str root_config_name,
+                                    dict  config)
+
+cpdef dict getConfiguration(str root_config_name=?,
+                                    dict  config=?)

python/obitools3/apps/config.pyx

@@ -0,0 +1,103 @@
+#cython: language_level=3
+
+'''
+Created on 27 mars 2016
+
+@author: coissac
+'''
+
+import sys
+
+from .command   import  getCommandsList
+from .logging   cimport getLogger
+from .arguments cimport buildArgumentParser
+
+from ..version import version
+from _curses import version
+
+cdef dict __default_config__ = {}
+
+
+cpdef str setRootConfigName(str rootname):
+    global __default_config__
+    if '__root_config__' in __default_config__:
+        if __default_config__["__root_config__"] in __default_config__:
+            __default_config__[rootname]=__default_config__[__default_config__["__root_config__"]]
+            del __default_config__[__default_config__["__root_config__"]]
+    __default_config__['__root_config__']=rootname
+    return rootname
+
+cpdef str getRootConfigName():
+    global __default_config__
+    return __default_config__.get('__root_config__',None)
+
+cdef dict buildDefaultConfiguration(str root_config_name,
+                                    dict  config):
+    global __default_config__
+    
+    __default_config__.clear()
+    setRootConfigName(root_config_name)    
+
+    __default_config__[root_config_name]=config
+    
+    config['version']=version
+    
+    commands = getCommandsList()
+    
+    for c in commands:
+        module = commands[c]
+    
+        assert hasattr(module, "run")
+        
+        if hasattr(module, 'default_config'):
+            __default_config__[c]=module.default_config
+        else:
+            __default_config__[c]={}
+                        
+    return __default_config__
+        
+
+cpdef dict getConfiguration(str root_config_name="__default__",
+                            dict  config={}):
+    global __default_config__
+
+    if '__done__' in __default_config__:
+        return __default_config__
+    
+    if root_config_name=="__default__":
+        raise RuntimeError("No root_config_name specified")
+    
+    if not config:
+        raise RuntimeError("Base configuration is empty")
+    
+    
+    
+    config =  buildDefaultConfiguration(root_config_name,
+                                        config)
+    
+    parser = buildArgumentParser(root_config_name,
+                                 config[root_config_name]['software'])
+    
+    options = vars(parser.parse_args())
+
+    if options['%s:version' % root_config_name]:
+        print("%s - Version %s" % (config[root_config_name]['software'],
+                                   config[root_config_name]['version']))
+        sys.exit(0)
+    
+    for k in options:
+        section,key = k.split(':')
+        s = config[section]
+        if options[k] is not None:
+            s[key]=options[k]
+            
+    if not 'module' in config[root_config_name]:
+        print('\nError: No command specified',file=sys.stderr)
+        parser.print_help()
+        sys.exit(2)
+                
+    getLogger(config)
+    
+    config['__done__']=True
+            
+    return config

python/obitools3/apps/logging.pxd

@@ -0,0 +1,3 @@
+#cython: language_level=3
+
+cpdef getLogger(dict config)

python/obitools3/apps/logging.pyx

@@ -0,0 +1,46 @@
+#cython: language_level=3
+
+'''
+Created on 27 mars 2016
+
+@author: coissac
+'''
+
+import logging
+import sys
+
+cpdef getLogger(dict config):
+    '''
+    Returns the logger as defined by the command line option
+    or by the config file
+    :param config:
+    '''
+    
+    root = config["__root_config__"]
+ 
+    level  = config[root]['loglevel'] 
+    logfile= config[root]['log'] 
+    
+    rootlogger   = logging.getLogger()
+    logFormatter = logging.Formatter("%(asctime)s [%(levelname)-5.5s]  %(message)s")
+
+    stderrHandler = logging.StreamHandler(sys.stderr)
+    stderrHandler.setFormatter(logFormatter)
+
+    rootlogger.addHandler(stderrHandler)
+    
+    if logfile:
+        fileHandler = logging.FileHandler(logfile)
+        fileHandler.setFormatter(logFormatter)
+        rootlogger.addHandler(fileHandler)
+    
+    try:
+        loglevel = getattr(logging, level) 
+    except:
+        loglevel = logging.INFO
+        
+    rootlogger.setLevel(loglevel)
+    
+    config[root]['logger']=rootlogger
+    
+    return rootlogger

python/obitools3/apps/progress.pxd

@@ -0,0 +1,64 @@
+#cython: language_level=3
+
+cdef extern from "stdio.h":
+    struct FILE
+    int fprintf(FILE *stream, char *format, ...)
+    int fputs(char *string, FILE *stream)
+    FILE* stderr
+    ctypedef unsigned int off_t "unsigned long long"
+    
+cdef extern from "unistd.h":
+    int fsync(int fd);
+
+cdef extern from "time.h":
+    struct tm :
+        int tm_yday 
+        int tm_hour
+        int tm_min
+        int tm_sec
+
+    enum: CLOCKS_PER_SEC
+    
+    ctypedef int time_t
+    ctypedef int clock_t
+    ctypedef int suseconds_t
+    
+    struct timeval:
+        time_t      tv_sec     #  seconds */
+        suseconds_t tv_usec    #  microseconds */
+
+
+    struct timezone :
+        int tz_minuteswest;    # minutes west of Greenwich
+        int tz_dsttime;        # type of DST correction 
+
+        
+    int gettimeofday(timeval *tv, timezone *tz)
+
+
+    tm *gmtime_r(time_t *clock, tm *result)
+    time_t time(time_t *tloc)
+    clock_t clock()
+
+cdef class ProgressBar:
+    cdef off_t   maxi
+    cdef clock_t starttime
+    cdef clock_t lasttime
+    cdef clock_t tickcount
+    cdef int freq
+    cdef int cycle
+    cdef int arrow
+    cdef int lastlog
+    cdef bint ontty
+    cdef int fd
+    
+    cdef bytes _head
+    cdef char *chead
+    
+    cdef object logger
+
+    cdef char  *wheel
+    cdef char  *spaces
+    cdef char*  diese 
+    
+    cdef clock_t clock(self)

python/obitools3/apps/progress.pyx

@@ -0,0 +1,151 @@
+#cython: language_level=3
+
+'''
+Created on 27 mars 2016
+
+@author: coissac
+'''
+
+from ..utils cimport str2bytes, bytes2str
+from .config cimport getConfiguration 
+import sys
+
+cdef class ProgressBar:
+    cdef clock_t clock(self):
+        cdef clock_t t
+        cdef timeval tp
+        cdef clock_t s
+        
+        <void> gettimeofday(&tp,NULL)
+        s = <clock_t> (<double> tp.tv_usec * 1.e-6 * <double> CLOCKS_PER_SEC)
+        t = tp.tv_sec * CLOCKS_PER_SEC + s 
+        
+        return t
+
+    def __init__(self,
+                 off_t maxi,
+                 dict  config={},
+                 str head="",
+                 double seconde=0.1):
+        self.starttime = self.clock()
+        self.lasttime  = self.starttime
+        self.tickcount = <clock_t> (seconde * CLOCKS_PER_SEC)
+        self.freq      = 1
+        self.cycle     = 0
+        self.arrow     = 0
+        self.lastlog   = 0
+        
+        if not config:
+            config=getConfiguration()
+            
+        
+        self.ontty = sys.stderr.isatty()
+        
+        if (maxi<=0):
+            maxi=1
+            
+        self.maxi = maxi
+        self.head = head
+        self.chead= self._head 
+        
+        
+        self.logger=config[config["__root_config__"]]["logger"]
+        self.wheel =  '|/-\\'
+        self.spaces='          ' \
+                    '          ' \
+                    '          ' \
+                    '          ' \
+                    '          '
+        self.diese ='##########' \
+                    '##########' \
+                    '##########' \
+                    '##########' \
+                    '##########'  
+                           
+    def __call__(self,object pos):
+        cdef off_t    ipos
+        cdef clock_t  elapsed
+        cdef clock_t  newtime
+        cdef clock_t  delta
+        cdef clock_t  more 
+        cdef double   percent 
+        cdef tm remain
+        cdef int days,hour,minu,sec
+        cdef off_t fraction
+        cdef int twentyth
+        
+        self.cycle+=1
+    
+        if self.cycle % self.freq == 0:
+            self.cycle=1
+            newtime  = self.clock()
+            delta         = newtime - self.lasttime
+            self.lasttime = newtime
+            elapsed       = newtime - self.starttime
+#            print(" ",delta,elapsed,elapsed/CLOCKS_PER_SEC,self.tickcount)
+            
+            if   delta < self.tickcount / 5 :
+                self.freq*=2
+            elif delta > self.tickcount * 5 and self.freq>1:
+                self.freq/=2
+                
+            
+            if callable(pos):
+                ipos=pos()
+            else:
+                ipos=pos
+                
+            if ipos==0:
+                ipos=1                
+
+            percent = <double>ipos/<double>self.maxi
+            more = <time_t>((<double>elapsed / percent * (1. - percent))/CLOCKS_PER_SEC)
+            <void>gmtime_r(&more, &remain)
+            days  = remain.tm_yday 
+            hour  = remain.tm_hour
+            minu  = remain.tm_min
+            sec   = remain.tm_sec
+                
+            if self.ontty:
+                fraction=<int>(percent * 50.)
+                self.arrow=(self.arrow+1) % 4
+            
+                if days:
+                    <void>fprintf(stderr,b'\r%s %5.1f %% |%.*s%c%.*s] remain : %d days %02d:%02d:%02d',
+                                    self.chead,
+                                    percent*100,
+                                    fraction,self.diese,
+                                    self.wheel[self.arrow],
+                                    50-fraction,self.spaces,
+                                    days,hour,minu,sec)
+                else:
+                    <void>fprintf(stderr,b'\r%s %5.1f %% |%.*s%c%.*s] remain : %02d:%02d:%02d',
+                                    self.chead,
+                                    percent*100.,
+                                    fraction,self.diese,
+                                    self.wheel[self.arrow],
+                                    50-fraction,self.spaces,
+                                    hour,minu,sec)
+
+            tenth = int(percent * 10)
+            if tenth != self.lastlog:
+                
+                if self.ontty:
+                    <void>fputs(b'\n',stderr)
+                    
+                self.logger.info('%s %5.1f %% remain : %02d:%02d:%02d' % (
+                                        bytes2str(self._head),
+                                        percent*100.,
+                                        hour,minu,sec))
+                self.lastlog=tenth
+        else:
+            self.cycle+=1
+
+    property head:
+    
+        def __get__(self):
+            return self._head
+        
+        def __set__(self,str value):
+            self._head=str2bytes(value)
+            self.chead=self._head

python/obitools3/commands/check.pyx

@@ -0,0 +1,68 @@
+#cython: language_level=3
+
+from obitools3.dms.dms import DMS       # TODO cimport doesn't work
+from obitools3.dms.view.view import View    # TODO cimport doesn't work
+ 
+ 
+__title__="Print a preview of a DMS, view, column...."
+ 
+default_config = {   'inputview'    : None,
+                 }
+ 
+
+# TODO make it work with URIs
+ 
+def addOptions(parser):
+ 
+    # TODO put this common group somewhere else but I don't know where
+    group=parser.add_argument_group('DMS and view options')
+ 
+    group.add_argument('--default-dms','-d', 
+                       action="store", dest="obi:defaultdms",
+                       metavar='<DMS NAME>',
+                       default=None,
+                       type=str,
+                       help="Name of the default DMS for reading and writing data.")
+ 
+    group.add_argument('--view','-v',
+                       action="store", dest="obi:view",
+                       metavar='<VIEW NAME>',
+                       default=None,
+                       type=str,
+                       help="Name of the view.")
+  
+ 
+#    group=parser.add_argument_group('obi check specific options')
+ 
+#     group.add_argument('--print',
+#                      action="store", dest="less:print",
+#                      metavar='<N>',
+#                      default=None,
+#                      type=int,
+#                      help="Print N sequences (default: 10)")
+ 
+     
+def run(config):
+     
+    # Open DMS
+    d = DMS.open(config['obi']['defaultdms'])
+     
+    # Open input view uif there is one 
+    if config['obi']['inputview'] is not None :
+        iview = View.open(d, config['obi']['inputview'])
+        print(repr(iview))
+    
+    else :
+        for v in d :
+            print(repr(v))
+     
+    d.close()
+     
+     
+     
+ 
+     
+     
+     
+     
+         

python/obitools3/commands/count.py

@@ -0,0 +1,44 @@
+'''
+Created on 8 mars 2016
+
+@author: coissac
+'''
+
+from obitools3.apps.progress import ProgressBar  # @UnresolvedImport
+import time
+
+__title__="Counts sequences in a sequence set"
+
+
+default_config = {   'countmode'       : None
+                 }
+
+def addOptions(parser):
+    parser.add_argument(dest='obi:input',     metavar='obi:input', 
+                                              nargs='?', 
+                                              default=None,
+                        help='input data set' )
+
+    group=parser.add_argument_group('Obicount specific options')
+    group.add_argument('-s','--sequence',
+                             action="store_true", dest="count:sequence",
+                             default=False,
+                             help="Prints only the number of sequence records."
+                             )
+ 
+    group.add_argument('-a','--all',
+                             action="store_true", dest="count:all",
+                             default=False,
+                             help="Prints only the total count of sequence records (if a sequence has no `count` attribute, its default count is 1) (default: False)."
+                             )
+
+    
+    
+def run(config):
+    # The code of my command
+    pb = ProgressBar(1000,config,seconde=1)
+    
+    for i in range(1,1001):
+        pb(i)
+        time.sleep(0.01)
+        

python/obitools3/commands/export.pyx

@@ -0,0 +1,109 @@
+# from obitools3.apps.progress cimport ProgressBar  # @UnresolvedImport
+# from obitools3.dms.dms import OBIDMS       # TODO cimport doesn't work
+# from obitools3.utils cimport bytes2str
+# 
+# import time
+# import re
+
+
+def run(config):
+    pass
+
+# __title__="Export a NUC_SEQS view to a fasta or fastq file"
+# 
+# 
+# default_config = {   'inputview'    : None,
+#                  }
+# 
+# def addOptions(parser):
+# 
+#     # TODO put this common group somewhere else but I don't know where
+#     group=parser.add_argument_group('DMS and view options')
+# 
+#     group.add_argument('--default-dms','-d', 
+#                        action="store", dest="obi:defaultdms",
+#                        metavar='<DMS NAME>',
+#                        default=None,
+#                        type=str,
+#                        help="Name of the default DMS for reading and writing data.")
+# 
+#     group.add_argument('--input-view','-i',
+#                        action="store", dest="obi:inputview",
+#                        metavar='<INPUT VIEW NAME>',
+#                        default=None,
+#                        type=str,
+#                        help="Name of the input view, either raw if the view is in the default DMS,"
+#                             " or in the form 'dms:view' if it is in another DMS.")
+# 
+#     group=parser.add_argument_group('obi export specific options')
+# 
+#     group.add_argument('--format','-f',
+#                        action="store", dest="export:format",
+#                        metavar='<FORMAT>',
+#                        default="fasta",
+#                        type=str,
+#                        help="Export in the format <FORMAT>, 'fasta' or 'fastq'. Default: 'fasta'.")         # TODO export in csv
+# 
+# def run(config):
+#     
+#     # TODO import doesn't work
+#     NUC_SEQUENCE_COLUMN = "NUC_SEQ"
+#     ID_COLUMN = "ID"
+#     DEFINITION_COLUMN = "DEFINITION"
+#     QUALITY_COLUMN = "QUALITY"    
+#     
+#     special_columns = [NUC_SEQUENCE_COLUMN, ID_COLUMN, DEFINITION_COLUMN, QUALITY_COLUMN]
+#     
+#     # Open DMS
+#     d = OBIDMS(config['obi']['defaultdms'])
+#     
+#     # Open input view
+#     iview = d.open_view(config['obi']['inputview'])
+#     
+#     print(iview.type)
+#     
+#     # TODO check that the view has the type NUC_SEQS
+#     if ((config['export']['format'] == "fasta") or (config['export']['format'] == "fastq")) and (iview.type != "NUC_SEQS_VIEW") :   # TODO find a way to import those macros
+#         raise Exception("Error: the view to export in fasta or fastq format is not a NUC_SEQS view")
+#     
+#     # Initialize the progress bar
+#     pb = ProgressBar(len(iview), config, seconde=5)
+#     
+#     i=0
+#     for seq in iview :
+#         pb(i)
+#         
+#         toprint = ">"+seq.id+" "
+#         
+#         for col_name in seq :
+#             if col_name not in special_columns :
+#                 toprint = toprint + col_name + "=" + str(seq[col_name]) + "; "
+#         
+#         if DEFINITION_COLUMN in seq :
+#             toprint = toprint + seq.definition
+#         
+#         nucseq = bytes2str(seq.nuc_seq)
+#         
+#         if config['export']['format'] == "fasta" :
+#             nucseq = re.sub("(.{60})", "\\1\n", nucseq, 0, re.DOTALL)
+#         
+#         toprint = toprint + "\n" + nucseq
+#         
+#         if config['export']['format'] == "fastq" :
+#             toprint = toprint + "\n" + "+" + "\n" + seq.get_str_quality()
+#             
+#         print(toprint)
+#         i+=1
+#         
+#     iview.close()
+#     d.close()
+#     
+#     print("Done.")
+#     
+#     
+# 
+#     
+#     
+#     
+#     
+#         

python/obitools3/commands/grep.pyx

@@ -0,0 +1,96 @@
+#cython: language_level=3
+
+from obitools3.apps.progress cimport ProgressBar  # @UnresolvedImport
+from obitools3.dms.dms import DMS       # TODO cimport doesn't work
+from obitools3.dms.view.view import View, Line_selection    # TODO cimport doesn't work
+ 
+from functools import reduce
+import time
+ 
+__title__="Grep view lines that match the given predicates"
+ 
+default_config = {   'inputview'    : None,
+                     'outputview'   : None
+                 }
+ 
+def addOptions(parser):
+ 
+    # TODO put this common group somewhere else but I don't know where
+    group=parser.add_argument_group('DMS and view options')
+ 
+    group.add_argument('--default-dms','-d', 
+                       action="store", dest="obi:defaultdms",
+                       metavar='<DMS NAME>',
+                       default=None,
+                       type=str,
+                       help="Name of the default DMS for reading and writing data.")
+ 
+    group.add_argument('--input-view','-i',
+                       action="store", dest="obi:inputview",
+                       metavar='<INPUT VIEW NAME>',
+                       default=None,
+                       type=str,
+                       help="Name of the input view, either raw if the view is in the default DMS,"
+                            " or in the form 'dms:view' if it is in another DMS.")
+ 
+    group.add_argument('--output-view','-o',
+                       action="store", dest="obi:outputview",
+                       metavar='<OUTPUT VIEW NAME>',
+                       default=None,
+                       type=str,
+                       help="Name of the output view, either raw if the view is in the default DMS,"
+                            " or in the form 'dms:view' if it is in another DMS.")
+ 
+ 
+    group=parser.add_argument_group('obi grep specific options')
+ 
+    group.add_argument('--predicate','-p',
+                       action="append", dest="grep:predicates",
+                       metavar='<PREDICATE>',
+                       default=None,
+                       type=str,
+                       help="Grep lines that match the given python expression on <line> or <sequence>.")
+ 
+     
+def run(config):
+     
+    # Open DMS
+    d = DMS.open(config['obi']['defaultdms'])
+     
+    # Open input view 1
+    iview = View.open(d, config['obi']['inputview'])
+     
+    # Initialize the progress bar
+    pb = ProgressBar(len(iview), config, seconde=5)
+         
+    # Apply filter
+    selection = Line_selection(iview)
+    for i in range(len(iview)) :
+        pb(i)
+        line = iview[i]
+                 
+        loc_env = {'sequence': line, 'line': line}      # TODO add taxonomy
+         
+        good = (reduce(lambda bint x, bint y: x and y,
+                (bool(eval(p, loc_env, line))
+                for p in config['grep']['predicates']), True))
+         
+        if good :
+            selection.append(i)
+    
+    # Create output view with the line selection
+    oview = selection.materialize(config['obi']['outputview'], comments="obi grep: "+str(config['grep']['predicates'])+"\n")
+ 
+    print("\n")
+    print(repr(oview))
+ 
+    d.close()
+     
+     
+     
+ 
+     
+     
+     
+     
+         

python/obitools3/commands/import.pyx

@@ -0,0 +1,284 @@
+#cython: language_level=3
+
+# TODO cimport generate errors with argument numbers, but without them some variables can't be declared
+
+from obitools3.apps.progress cimport ProgressBar  # @UnresolvedImport
+from obitools3.files.universalopener cimport uopen
+from obitools3.parsers.fasta import fastaIterator
+from obitools3.parsers.fastq import fastqIterator
+from obitools3.dms.dms import DMS       # TODO cimport doesn't work
+from obitools3.dms.view.view cimport View
+from obitools3.dms.view.typed_view.view_NUC_SEQS import View_NUC_SEQS    # TODO cimport doesn't work
+from obitools3.dms.column.column cimport Column
+
+from obitools3.utils cimport tobytes, \
+                             get_obitype, \
+                             update_obitype
+
+from obitools3.dms.capi.obitypes cimport obitype_t, \
+                                         OBI_VOID
+
+from obitools3.dms.capi.obierrno cimport obi_errno
+
+import time
+
+import pickle
+
+
+__title__="Imports sequences from different formats into a DMS"
+ 
+ 
+default_config = {   'destview'     : None,
+                     'skip'         : 0,
+                     'only'         : None,
+                     'skiperror'    : False,
+                     'seqinformat'  : None,
+                     'moltype'      : 'nuc',
+                     'filename'     : None
+                 }
+
+def addOptions(parser):
+    parser.add_argument(dest='import:filename',     
+                        metavar='<FILENAME>', 
+                        nargs='?', 
+                        default=None,
+                        help='Name of the sequence file to import' )
+
+    group=parser.add_argument_group('obi import specific options')
+
+    group.add_argument('--default-dms','-d',
+                     action="store", dest="obi:defaultdms",
+                     metavar='<DMS NAME>',
+                     default=None,
+                     type=str,
+                     help="Name of the default DMS for reading and writing data")
+
+    group.add_argument('--destination-view','-v',
+                     action="store", dest="import:destview",
+                     metavar='<VIEW NAME>',
+                     default=None,
+                     type=str,
+                     required=True,
+                     help="Name of the default DMS for reading and writing data")
+
+    group.add_argument('--skip',
+                     action="store", dest="import:skip",
+                     metavar='<N>',
+                     default=0,
+                     type=int,
+                     help="Skip the N first sequences")
+
+    group.add_argument('--only',
+                     action="store", dest="import:only",
+                     metavar='<N>',
+                     default=None,
+                     type=int,
+                     help="Treat only N sequences")
+
+    group.add_argument('--skip-on-error',
+                     action="store_true", dest="import:skiperror",
+                     default=None,
+                     help="Skip sequence entries with parse error")
+
+    group.add_argument('--fasta',
+                     action="store_const", dest="import:seqinformat",
+                     default=None,
+                     const='fasta',
+                     help="Input file is in fasta nucleic format (including obitools fasta extentions)")
+
+    group.add_argument('--fastq',
+                     action="store_const", dest="import:seqinformat",
+                     default=None,
+                     const='fastq',
+                     help="Input file is in sanger fastq nucleic format (standard fastq)")
+
+    group.add_argument('--nuc',
+                     action="store_const", dest="import:moltype",
+                     default=None,
+                     const='nuc',
+                     help="Input file contains nucleic sequences")
+
+    group.add_argument('--prot',
+                     action="store_const", dest="import:moltype",
+                     default=None,
+                     const='pep',
+                     help="Input file contains protein sequences")
+
+
+# TODO: Handling of NA values. Check None. Specify in doc? None or NA? Possiblity to specify in option?
+# look in R read.table option to specify NA value
+def run(config):
+    
+    cdef   int         i
+    cdef   type        value_type
+    cdef   obitype_t   value_obitype
+    cdef   obitype_t   old_type
+    cdef   obitype_t   new_type
+    cdef   bint        get_quality
+    cdef   bint        NUC_SEQS_view
+    cdef   int         nb_elts
+    cdef   object      d
+    cdef   View        view
+    cdef   object      iseq
+    cdef   object      seq
+    cdef   object      inputs
+    cdef   Column      id_col
+    cdef   Column      def_col
+    cdef   Column      seq_col
+    cdef   Column      qual_col
+    cdef   Column      old_column
+    cdef   bint        rewrite
+    cdef   dict        dcols
+    cdef   int         skipping
+    cdef   str         tag
+    cdef   object      value
+    cdef   list        elt_names
+    cdef   int         old_nb_elements_per_line
+    cdef   int         new_nb_elements_per_line
+    cdef   list        old_elements_names
+    cdef   list        new_elements_names
+    cdef   ProgressBar pb
+    global             obi_errno
+    
+    pb = ProgressBar(1000000, config, seconde=5)   # TODO should be number of records in file
+       
+    inputs = uopen(config['import']['filename'])
+
+    # Create or open DMS
+    try:
+        d = DMS.test_open(config['obi']['defaultdms'])
+    except :
+        d = DMS.new(config['obi']['defaultdms'])
+
+    get_quality = False
+    NUC_SEQS_view = False
+    if config['import']['seqinformat']=='fasta':
+        get_quality = False
+        NUC_SEQS_view = True
+        iseq = fastaIterator(inputs)
+        view = View_NUC_SEQS.new(d, config['import']['destview'], quality=get_quality)
+    elif config['import']['seqinformat']=='fastq':
+        get_quality = True
+        NUC_SEQS_view = True
+        iseq = fastqIterator(inputs)
+        view = View_NUC_SEQS.new(d, config['import']['destview'], quality=get_quality)
+    else:
+        raise RuntimeError('File format not handled')
+        
+    # Save basic columns in variables for optimization
+    if NUC_SEQS_view :
+        id_col = view["ID"]
+        def_col = view["DEFINITION"]
+        seq_col = view["NUC_SEQ"]
+        if get_quality :
+            qual_col = view["QUALITY"]
+    
+    dcols = {}
+    
+    skipping = 0
+    i = 0
+    for seq in iseq :
+        if skipping < config['import']['skip'] :    # TODO not efficient because sequences are parsed
+            skipping+=1
+        elif i == config['import']['only'] :
+            break
+        else :
+            pb(i)
+            if NUC_SEQS_view :
+                id_col[i] = seq['id']
+                def_col[i] = seq['definition']
+                seq_col[i] = seq['sequence']
+                if get_quality :
+                    qual_col[i] = seq['quality']
+            
+            for tag in seq['tags'] :
+    
+                value = seq['tags'][tag]
+                
+                if tag not in dcols :
+                    
+                    value_type = type(value)
+                    nb_elts = 1
+                    value_obitype = OBI_VOID
+                    
+                    if value_type == dict or value_type == list :
+                        nb_elts = len(value)
+                        elt_names = list(value)
+                    else :
+                        nb_elts = 1
+                        elt_names = None
+                    
+                    value_obitype = get_obitype(value)
+                    
+                    if value_obitype != OBI_VOID :
+                        dcols[tag] = (Column.new_column(view, tag, value_obitype, nb_elements_per_line=nb_elts, elements_names=elt_names), value_obitype)
+
+                        # Fill value
+                        dcols[tag][0][i] = value
+                    
+                    # TODO else log error?
+
+                else :
+        
+                    rewrite = False
+
+                    # Check type adequation
+                    old_type = dcols[tag][1]
+                    new_type = OBI_VOID
+                    new_type = update_obitype(old_type, value)
+                    if old_type != new_type :
+                        rewrite = True
+
+                    try:
+                        # Fill value
+                        dcols[tag][0][i] = value
+                    
+                    except IndexError :
+                        
+                        value_type = type(value)
+                        old_column = dcols[tag][0]
+                        old_nb_elements_per_line = old_column.nb_elements_per_line
+                        new_nb_elements_per_line = 0
+                        old_elements_names = old_column.elements_names
+                        new_elements_names = None
+    
+                        #####################################################################
+                        
+                        # Check the length and keys of column lines if needed
+                        if value_type == dict :    # Check dictionary keys
+                            for k in value :
+                                if k not in old_elements_names :
+                                    new_elements_names = list(value)
+                                    rewrite = True
+                                    break
+                        
+                        elif value_type == list or value_type == tuple :  # Check vector length
+                            if old_nb_elements_per_line < len(value) :
+                                new_nb_elements_per_line = len(value)
+                                rewrite = True
+                        
+                        #####################################################################
+                        
+                        if rewrite :
+                            if new_nb_elements_per_line == 0 and new_elements_names is not None :
+                                new_nb_elements_per_line = len(new_elements_names)
+                            
+                            dcols[tag] = (view.rewrite_column_with_diff_attributes(old_column.name, 
+                                                                                   new_data_type=new_type, 
+                                                                                   new_nb_elements_per_line=new_nb_elements_per_line,
+                                                                                   new_elements_names=new_elements_names), 
+                                          value_obitype)
+                            
+                            # Reset obierrno
+                            obi_errno = 0
+                            
+                            # Fill value
+                            dcols[tag][0][i] = value
+        
+            i+=1
+
+    print("\n")
+    print(view.__repr__())
+ 
+    d.close()
+

python/obitools3/commands/lcs.cfiles

@@ -0,0 +1,65 @@
+../../../src/bloom.h
+../../../src/bloom.c
+../../../src/char_str_indexer.h
+../../../src/char_str_indexer.c
+../../../src/crc64.h
+../../../src/crc64.c
+../../../src/dna_seq_indexer.h
+../../../src/dna_seq_indexer.c
+../../../src/encode.h
+../../../src/encode.c
+../../../src/hashtable.h
+../../../src/hashtable.c
+../../../src/murmurhash2.h
+../../../src/murmurhash2.c
+../../../src/obi_align.h
+../../../src/obi_align.c
+../../../src/obiavl.h
+../../../src/obiavl.c
+../../../src/obiblob_indexer.h
+../../../src/obiblob_indexer.c
+../../../src/obiblob.h
+../../../src/obiblob.c
+../../../src/obidebug.h
+../../../src/obidms_taxonomy.h
+../../../src/obidms_taxonomy.c
+../../../src/obidms.h
+../../../src/obidms.c
+../../../src/obidmscolumn_blob.c
+../../../src/obidmscolumn_blob.h
+../../../src/obidmscolumn_bool.c
+../../../src/obidmscolumn_bool.h
+../../../src/obidmscolumn_char.c
+../../../src/obidmscolumn_char.h
+../../../src/obidmscolumn_float.c
+../../../src/obidmscolumn_float.h
+../../../src/obidmscolumn_idx.h
+../../../src/obidmscolumn_idx.c
+../../../src/obidmscolumn_int.c
+../../../src/obidmscolumn_int.h
+../../../src/obidmscolumn_qual.h
+../../../src/obidmscolumn_qual.c
+../../../src/obidmscolumn_seq.c
+../../../src/obidmscolumn_seq.h
+../../../src/obidmscolumn_str.c
+../../../src/obidmscolumn_str.h
+../../../src/obidmscolumn.h
+../../../src/obidmscolumn.c
+../../../src/obidmscolumndir.h
+../../../src/obidmscolumndir.c
+../../../src/obierrno.h
+../../../src/obierrno.c
+../../../src/obilittlebigman.h
+../../../src/obilittlebigman.c
+../../../src/obitypes.h
+../../../src/obitypes.c
+../../../src/obiview.h
+../../../src/obiview.c
+../../../src/sse_banded_LCS_alignment.h
+../../../src/sse_banded_LCS_alignment.c
+../../../src/uint8_indexer.h
+../../../src/uint8_indexer.c
+../../../src/upperband.h
+../../../src/upperband.c
+../../../src/utils.h
+../../../src/utils.c

python/obitools3/commands/lcs.pyx

@@ -0,0 +1,236 @@
+#cython: language_level=3
+# 
+# from obitools3.apps.progress cimport ProgressBar  # @UnresolvedImport
+# from obitools3.dms.dms import OBIDMS       # TODO cimport doesn't work
+# from obitools3.utils cimport str2bytes
+# 
+# from obitools3.dms.capi.obialign cimport obi_lcs_align_one_column, \
+#                                          obi_lcs_align_two_columns
+# 
+# 
+# import time
+# 
+# __title__="Aligns one sequence column with itself or two sequence columns"
+# 
+# 
+# default_config = {   'inputview'    : None,
+#                  }
+# 
+# def addOptions(parser):
+# 
+#     # TODO put this common group somewhere else but I don't know where.
+#     # Also some options should probably be in another group
+#     group=parser.add_argument_group('DMS and view options')
+# 
+#     group.add_argument('--default-dms', '-d', 
+#                        action="store", dest="obi:defaultdms",
+#                        metavar='<DMS NAME>',
+#                        default=None,
+#                        type=str,
+#                        help="Name of the default DMS for reading and writing data.")
+# 
+#     group.add_argument('--input-view-1', '-i',
+#                        action="store", dest="obi:inputview1",
+#                        metavar='<INPUT VIEW NAME>',
+#                        default=None,
+#                        type=str,
+#                        help="Name of the (first) input view.")
+# 
+#     group.add_argument('--input-view-2', '-I',
+#                        action="store", dest="obi:inputview2",
+#                        metavar='<INPUT VIEW NAME>',
+#                        default="",
+#                        type=str,
+#                        help="Eventually, the name of the second input view.")
+# 
+#     group.add_argument('--input-column-1', '-c',
+#                        action="store", dest="obi:inputcolumn1",
+#                        metavar='<INPUT COLUMN NAME>',
+#                        default="",
+#                        type=str,
+#                        help="Name of the (first) input column. "
+#                             " Default: the default nucleotide sequence column of the view if there is one.")
+# 
+#     group.add_argument('--input-column-2', '-C',
+#                        action="store", dest="obi:inputcolumn2",
+#                        metavar='<INPUT COLUMN NAME>',
+#                        default="",
+#                        type=str,
+#                        help="Eventually, the name of the second input column.")
+# 
+#     group.add_argument('--input-elt-1', '-e',
+#                        action="store", dest="obi:inputelement1",
+#                        metavar='<INPUT ELEMENT NAME>',
+#                        default="",
+#                        type=str,
+#                        help="If the first input column has multiple elements per line, name of the element referring to the sequence to align. "
+#                             " Default: the first element of the line.")
+# 
+#     group.add_argument('--input-elt-2', '-E',
+#                        action="store", dest="obi:inputelement2",
+#                        metavar='<INPUT ELEMENT NAME>',
+#                        default="",
+#                        type=str,
+#                        help="If the second input column has multiple elements per line, name of the element referring to the sequence to align. "
+#                             " Default: the first element of the line.")
+# 
+#     group.add_argument('--id-column-1', '-f',
+#                        action="store", dest="obi:idcolumn1",
+#                        metavar='<ID COLUMN NAME>',
+#                        default="",
+#                        type=str,
+#                        help="Name of the (first) column containing the identifiers of the sequences to align. "
+#                             " Default: the default ID column of the view if there is one.")
+# 
+#     group.add_argument('--id-column-2', '-F',
+#                        action="store", dest="obi:idcolumn2",
+#                        metavar='<ID COLUMN NAME>',
+#                        default="",
+#                        type=str,
+#                        help="Eventually, the name of the second ID column.")
+# 
+#     group.add_argument('--output-view', '-o',
+#                        action="store", dest="obi:outputview",
+#                        metavar='<OUTPUT VIEW NAME>',
+#                        default=None,
+#                        type=str,
+#                        help="Name of the output view.")
+# 
+# 
+#     group=parser.add_argument_group('obi lcs specific options')
+#  
+#     group.add_argument('--threshold','-t',
+#                        action="store", dest="align:threshold",
+#                        metavar='<THRESHOLD>',
+#                        default=0.0,
+#                        type=float,
+#                        help="Score threshold. If the score is normalized and expressed in similarity (default),"
+#                             " it is an identity, e.g. 0.95 for an identity of 95%%. If the score is normalized"
+#                             " and expressed in distance, it is (1.0 - identity), e.g. 0.05 for an identity of 95%%."
+#                             " If the score is not normalized and expressed in similarity, it is the length of the"
+#                             " Longest Common Subsequence. If the score is not normalized and expressed in distance,"
+#                             " it is (reference length - LCS length)."
+#                             " Only sequence pairs with a similarity above <THRESHOLD> are printed. Default: 0.00"
+#                             " (no threshold).")
+# 
+#     group.add_argument('--longest-length','-L',
+#                        action="store_const", dest="align:reflength",
+#                        default=0,
+#                        const=1,
+#                        help="The reference length is the length of the longest sequence."
+#                             " Default: the reference length is the length of the alignment.")
+# 
+#     group.add_argument('--shortest-length','-l',
+#                        action="store_const", dest="align:reflength",
+#                        default=0,
+#                        const=2,
+#                        help="The reference length is the length of the shortest sequence."
+#                             " Default: the reference length is the length of the alignment.")
+# 
+#     group.add_argument('--raw','-r',
+#                        action="store_false", dest="align:normalize",
+#                        default=True,
+#                        help="Raw score, not normalized. Default: score is normalized with the reference sequence length.")
+# 
+#     group.add_argument('--distance','-D',
+#                        action="store_false", dest="align:similarity",
+#                        default=True,
+#                        help="Score is expressed in distance. Default: score is expressed in similarity.")
+# 
+#     group.add_argument('--print-seq','-s',
+#                        action="store_true", dest="align:printseq",
+#                        default=False,
+#                        help="The nucleotide sequences are written in the output view. Default: they are not written.")
+# 
+#     group.add_argument('--print-count','-n',
+#                        action="store_true", dest="align:printcount",
+#                        default=False,
+#                        help="Sequence counts are written in the output view. Default: they are not written.")
+# 
+#     group.add_argument('--thread-count','-p',   # TODO should probably be in a specific option group
+#                        action="store", dest="align:threadcount",
+#                        metavar='<THREAD COUNT>',
+#                        default=1,
+#                        type=int,
+#                        help="Number of threads to use for the computation. Default: one.")
+# 
+# 
+# # cpdef align(str dms_n, 
+# #             str input_view_1_n, str output_view_n,
+# #             str input_view_2_n="",
+# #             str input_column_1_n="", str input_column_2_n="",
+# #             str input_elt_1_n="", str input_elt_2_n="",
+# #             str id_column_1_n="", str id_column_2_n="",
+# #             double threshold=0.0, bint normalize=True, 
+# #             int reference=0, bint similarity_mode=True,
+# #             bint print_seq=False, bint print_count=False,
+# #             comments="",
+# #             int thread_count=1) :
+# #                  
+# #     cdef OBIDMS d         
+# #     d = OBIDMS(dms_n)
+# # 
+# #     if input_view_2_n == "" and input_column_2_n == "" :
+# #         if obi_lcs_align_one_column(d._pointer, \
+# #                                     str2bytes(input_view_1_n), \
+# #                                     str2bytes(input_column_1_n), \
+# #                                     str2bytes(input_elt_1_n), \
+# #                                     str2bytes(id_column_1_n), \
+# #                                     str2bytes(output_view_n), \
+# #                                     str2bytes(comments), \
+# #                                     print_seq, \
+# #                                     print_count, \
+# #                                     threshold, normalize, reference, similarity_mode,
+# #                                     thread_count) < 0 :
+# #             raise Exception("Error aligning sequences")
+# #     else :
+# #         if obi_lcs_align_two_columns(d._pointer, \
+# #                                      str2bytes(input_view_1_n), \
+# #                                      str2bytes(input_view_2_n), \
+# #                                      str2bytes(input_column_1_n), \
+# #                                      str2bytes(input_column_2_n), \
+# #                                      str2bytes(input_elt_1_n), \
+# #                                      str2bytes(input_elt_2_n), \
+# #                                      str2bytes(id_column_1_n), \
+# #                                      str2bytes(id_column_2_n), \
+# #                                      str2bytes(output_view_n), \
+# #                                      str2bytes(comments), \
+# #                                      print_seq, \
+# #                                      print_count, \
+# #                                      threshold, normalize, reference, similarity_mode) < 0 :
+# #             raise Exception("Error aligning sequences")        
+# #         
+# #     d.close()
+# # 
+# # 
+def run(config):
+    pass
+    # TODO: Build formatted comments with all parameters etc
+#     comments = "Obi align"
+#      
+#     # Call cython alignment function
+#     align(config['obi']['defaultdms'],  \
+#           config['obi']['inputview1'],  \
+#           config['obi']['outputview'],  \
+#           input_view_2_n   = config['obi']['inputview2'],  \
+#           input_column_1_n = config['obi']['inputcolumn1'],  \
+#           input_column_2_n = config['obi']['inputcolumn2'], \
+#           input_elt_1_n    = config['obi']['inputelement1'],  \
+#           input_elt_2_n    = config['obi']['inputelement2'], \
+#           id_column_1_n    = config['obi']['idcolumn1'],  \
+#           id_column_2_n    = config['obi']['idcolumn2'], \
+#           threshold        = config['align']['threshold'], \
+#           normalize        = config['align']['normalize'],  \
+#           reference        = config['align']['reflength'],  \
+#           similarity_mode  = config['align']['similarity'],  \
+#           print_seq        = config['align']['printseq'],  \
+#           print_count      = config['align']['printcount'], \
+#           comments         = comments, \
+#           thread_count     = config['align']['threadcount'])
+#  
+#     print("Done.")
+# # 
+# #     
+# #     
+# #     
+# #     

python/obitools3/commands/less.pyx

@@ -0,0 +1,57 @@
+#cython: language_level=3
+
+from obitools3.dms.dms import DMS       # TODO cimport doesn't work
+from obitools3.dms.view.view import View    # TODO cimport doesn't work
+
+
+# TODO with URIs
+ 
+__title__="Less equivalent"
+ 
+default_config = {   'inputview'    : None,
+                 }
+ 
+def addOptions(parser):
+ 
+    # TODO put this common group somewhere else but I don't know where
+    group=parser.add_argument_group('DMS and view options')
+ 
+    group.add_argument('--default-dms','-d', 
+                       action="store", dest="obi:defaultdms",
+                       metavar='<DMS NAME>',
+                       default=None,
+                       type=str,
+                       help="Name of the default DMS for reading and writing data.")
+ 
+    group.add_argument('--view','-v',
+                       action="store", dest="obi:view",
+                       metavar='<VIEW NAME>',
+                       default=None,
+                       type=str,
+                       help="Name of the view to print.")
+  
+ 
+    group=parser.add_argument_group('obi less specific options')
+ 
+    group.add_argument('--print', '-n',
+                     action="store", dest="less:print",
+                     metavar='<N>',
+                     default=10,
+                     type=int,
+                     help="Print N sequences (default: 10)")
+ 
+
+def run(config):
+     
+    # Open DMS
+    d = DMS.open(config['obi']['defaultdms'])
+     
+    # Open input view
+    iview = View.open(d, config['obi']['inputview'])
+              
+    # Print
+    for i in range(config['less']['print']) :
+        print(repr(iview[i]))
+     
+    d.close()
+

python/obitools3/commands/test.pyx

@@ -0,0 +1,437 @@
+#cython: language_level=3
+
+from obitools3.apps.progress cimport ProgressBar  # TODO I absolutely don't understand why it doesn't work without that line
+from obitools3.dms.view.view import View, Line_selection
+from obitools3.dms.view.typed_view.view_NUC_SEQS import View_NUC_SEQS
+from obitools3.dms.dms import DMS
+from obitools3.dms.column import Column
+from obitools3.dms.taxo.taxo import OBI_Taxonomy
+from obitools3.utils cimport str2bytes
+from obitools3.dms.capi.obitypes cimport OBI_INT, \
+                                         OBI_FLOAT, \
+                                         OBI_BOOL, \
+                                         OBI_CHAR, \
+                                         OBI_STR, \
+                                         OBI_SEQ
+import shutil
+import string
+import random
+
+
+VIEW_TYPES = ["", "NUC_SEQS_VIEW"]
+COL_TYPES = [OBI_INT, OBI_FLOAT, OBI_BOOL, OBI_CHAR, OBI_STR, OBI_SEQ]
+NUC_SEQUENCE_COLUMN = "NUC_SEQ"
+ID_COLUMN = "ID"
+DEFINITION_COLUMN = "DEFINITION"
+QUALITY_COLUMN = "QUALITY"    
+SPECIAL_COLUMNS = [NUC_SEQUENCE_COLUMN, ID_COLUMN, DEFINITION_COLUMN, QUALITY_COLUMN]
+
+#TAXDUMP = "" TODO path=?
+TAXTEST = "taxtest"
+ 
+NAME_MAX_LEN = 200
+COL_COMMENTS_MAX_LEN = 2048
+MAX_INT = 2147483647    # used to generate random float values
+ 
+ 
+__title__="Tests if the obitools are working properly"
+
+
+default_config = {
+                 }
+
+
+def test_taxo(config, infos):
+    tax1 = OBI_Taxonomy.open(infos['dms'], config['obi']['taxo'], taxdump=True)
+    tax1.write(TAXTEST)
+    tax2 = OBI_Taxonomy.open(infos['dms'], TAXTEST, taxdump=False)
+    assert len(tax1) == len(tax2), "Length of written taxonomy != length of read taxdump : "+str(len(tax2))+" != "+str(len(tax1))
+    
+    i = 0
+    for x in range(config['test']['nbtests']):
+        idx = random.randint(0, len(tax1)-1)
+        t1 = tax1.get_taxon_by_idx(idx)
+        t2 = tax2.get_taxon_by_idx(idx)
+        assert t1 == t2, "Taxon gotten from written taxonomy != taxon read from taxdump : "+str(t2)+" != "+str(t1)
+        i+=1
+        if (i%(config['test']['nbtests']/10)) == 0 :
+            print("Testing taxonomy functions......"+str(i*100/config['test']['nbtests'])+"%")
+
+    tax1.close()
+    tax2.close()
+
+
+def random_length(max_len):
+    return random.randint(1, max_len)
+
+
+def random_bool(config):
+    return random.choice([True, False])
+
+
+def random_char(config):
+    return str2bytes(random.choice(string.ascii_lowercase))
+
+
+def random_float(config):
+    return random.randint(0, MAX_INT) + random.random()
+
+
+def random_int(config):
+    return random.randint(0, config['test']['maxlinenb'])
+
+
+def random_seq(config):
+    return str2bytes(''.join(random.choice(['a','t','g','c']) for i in range(random_length(config['test']['seqmaxlen']))))
+
+
+def random_bytes(config):
+    return random_bytes_with_max_len(config['test']['strmaxlen'])
+
+
+def random_str_with_max_len(max_len):
+    return ''.join(random.choice(string.ascii_lowercase) for i in range(random_length(max_len)))
+
+
+def random_bytes_with_max_len(max_len):
+    return str2bytes(''.join(random.choice(string.ascii_lowercase) for i in range(random_length(max_len))))
+
+ 
+def random_column(infos):
+    return random.choice(sorted(list(infos['view'].keys())))
+ 
+ 
+def random_unique_name(infos):
+    name = ""
+    while name == "" or name in infos['unique_names'] :
+        name = random_str_with_max_len(NAME_MAX_LEN)
+    infos['unique_names'].append(name)
+    return name
+ 
+ 
+def random_unique_element_name(config, infos):
+    name = ""
+    while name == "" or name in infos['unique_names'] :
+        name = random_str_with_max_len(config['test']['elt_name_max_len'])
+    infos['unique_names'].append(name)
+    return name
+ 
+ 
+def print_test(config, sentence):
+    if config['test']['verbose'] :
+        print(sentence)
+ 
+ 
+def test_set_and_get(config, infos):
+    print_test(config, ">>> Set and get test")
+    col_name = random_column(infos)
+    col = infos['view'][col_name]
+    element_names = col.elements_names
+    data_type = col.data_type
+    if data_type == "OBI_QUAL" :
+        print_test(config, "-")
+        return
+    idx = random_int(config)
+    value = random.choice([None, infos['random_generator'][data_type](config)])
+    if col.nb_elements_per_line > 1 :
+        elt = random.choice(element_names)
+        col[idx][elt] = value
+        assert col[idx][elt] == value, "Column: "+repr(col)+"\nSet value != gotten value "+str(value)+" != "+str(col[idx][elt])
+    else:
+        col[idx] = value
+        assert col[idx] == value, "Column: "+repr(col)+"\nSet value != gotten value "+str(value)+" != "+str(col[idx])
+    
+    print_test(config, ">>> Set and get test OK")
+
+
+def test_add_col(config, infos):
+    print_test(config, ">>> Add column test")
+    #existing_col = random_bool(config)    # TODO doesn't work because of line count problem. See obiview.c line 1737
+    #if existing_col and infos["view_names"] != [] :
+    #    random_view = infos['dms'].open_view(random.choice(infos["view_names"]))
+    #    random_column = random_view[random.choice(sorted(list(random_view.columns))]
+    #    random_column_refs = random_column.refs
+    #    if random_column_refs['name'] in infos['view'] :
+    #        alias = random_unique_name(infos)
+    #    else :
+    #        alias = ''
+    #    infos['view'].add_column(random_column_refs['name'], version_number=random_column_refs['version'], alias=alias, create=False)
+    #    random_view.close()
+    #else :
+    create_random_column(config, infos)
+    print_test(config, ">>> Add column test OK")
+
+
+def test_delete_col(config, infos):
+    print_test(config, ">>> Delete column test")
+    if len(list(infos['view'].keys())) <= 1 :
+        print_test(config, "-")
+        return
+    col_name = random_column(infos)
+    if col_name in SPECIAL_COLUMNS :
+        print_test(config, "-")
+        return
+    infos['view'].delete_column(col_name)
+    print_test(config, ">>> Delete column test OK")
+
+
+def test_col_alias(config, infos):
+    print_test(config, ">>> Changing column alias test")
+    col_name = random_column(infos)
+    if col_name in SPECIAL_COLUMNS :
+        print_test(config, "-")
+        return
+    infos['view'][col_name].name = random_unique_name(infos)
+    print_test(config, ">>> Changing column alias test OK")
+         
+ 
+def test_new_view(config, infos):
+    print_test(config, ">>> New view test")
+    random_new_view(config, infos)
+    print_test(config, ">>> New view test OK")
+ 
+ 
+def random_test(config, infos):
+    return random.choice(infos['tests'])(config, infos)
+ 
+ 
+def random_view_type():
+    return random.choice(VIEW_TYPES)
+ 
+ 
+def random_col_type():
+    return random.choice(COL_TYPES)    
+ 
+ 
+def fill_column(config, infos, col) :
+    data_type = col.data_type
+    element_names = col.elements_names
+     
+    if len(element_names) > 1 :
+        for i in range(random_int(config)) :
+            for j in range(len(element_names)) :
+                col[i][element_names[j]] = random.choice([None, infos['random_generator'][data_type](config)])
+    else :
+        for i in range(random_int(config)) :
+            col[i] = random.choice([None, infos['random_generator'][data_type](config)])
+
+
+def create_random_column(config, infos) :
+    alias = random.choice(['', random_unique_name(infos)])
+    nb_elements_per_line=random.randint(1, config['test']['maxelts'])
+    elements_names = []
+    for i in range(nb_elements_per_line) :
+        elements_names.append(random_unique_element_name(config, infos))
+    elements_names = random.choice([None, elements_names])
+    name = random_unique_name(infos)
+    data_type = random_col_type()
+    
+    column = Column.new_column(infos['view'], 
+                               name, 
+                               data_type, 
+                               nb_elements_per_line=nb_elements_per_line,
+                               elements_names=elements_names,
+                               comments=random_str_with_max_len(COL_COMMENTS_MAX_LEN),
+                               alias=alias
+                               )   
+    
+    if alias != '' :
+        assert infos['view'][alias] == column
+    else :
+        assert infos['view'][name] == column
+
+    return column
+ 
+ 
+def fill_view(config, infos):
+    for i in range(random.randint(1, config['test']['maxinicolcount'])) :
+        col = create_random_column(config, infos)
+        fill_column(config, infos, col)
+ 
+ 
+def random_new_view(config, infos, first=False):
+    v_to_clone = None
+    line_selection = None
+    quality_col = False     # TODO
+    if not first:
+        infos['view_names'].append(infos['view'].name)
+        infos['view'].close()
+        v_to_clone = View.open(infos['dms'], random.choice(infos["view_names"]))
+        v_type = ""
+        print_test(config, "View to clone: ")
+        print_test(config, repr(v_to_clone))
+        create_line_selection = random_bool(config)
+        if create_line_selection and v_to_clone.line_count > 0:
+            print_test(config, "New view with new line selection.")
+            line_selection = Line_selection(v_to_clone)
+            for i in range(random.randint(1, v_to_clone.line_count)) :
+                line_selection.append(random.randint(0, v_to_clone.line_count-1))
+            #print_test(config, "New line selection: "+str(line_selection))
+    else :
+        v_type = random_view_type()
+     
+    if line_selection is not None :
+        infos['view'] = line_selection.materialize(random_unique_name(infos), comments=random_str_with_max_len(config['test']['commentsmaxlen']))
+    elif v_to_clone is not None :
+        infos['view'] = v_to_clone.clone(random_unique_name(infos), comments=random_str_with_max_len(config['test']['commentsmaxlen']))
+    else :
+        if v_type == "NUC_SEQS_VIEW" :
+            infos['view'] = View_NUC_SEQS.new(infos['dms'], random_unique_name(infos), comments=random_str_with_max_len(config['test']['commentsmaxlen']))   # TODO quality column
+        else :
+            infos['view'] = View.new(infos['dms'], random_unique_name(infos), comments=random_str_with_max_len(config['test']['commentsmaxlen']))   # TODO quality column
+            
+    print_test(config, repr(infos['view']))
+    if v_to_clone is not None :
+        if line_selection is None:
+            assert v_to_clone.line_count == infos['view'].line_count, "New view and cloned view don't have the same line count : "+str(v_to_clone.line_count)+" (view to clone line count) != "+str(infos['view'].line_count)+" (new view line count)"
+        else :
+            assert len(line_selection) == infos['view'].line_count, "New view with new line selection does not have the right line count : "+str(len(line_selection))+" (line selection length) != "+str(infos['view'].line_count)+" (new view line count)"
+        v_to_clone.close()
+    if first :
+        fill_view(config, infos)
+
+
+def create_test_obidms(config, infos):
+    infos['dms'] = DMS.new(config['obi']['defaultdms'])
+
+
+def ini_dms_and_first_view(config, infos):
+    create_test_obidms(config, infos)
+    random_new_view(config, infos, first=True)
+    infos['view_names'] = []
+
+
+def addOptions(parser):
+     
+    # TODO put this common group somewhere else but I don't know where
+    group=parser.add_argument_group('DMS and view options')
+ 
+    group.add_argument('--default-dms','-d', 
+                       action="store", dest="obi:defaultdms",
+                       metavar='<DMS NAME>',
+                       default="/tmp/test_dms",
+                       type=str,
+                       help="Name of the default DMS for reading and writing data. "
+                            "Default: /tmp/test_dms")
+    
+    group.add_argument('--taxo','-t',     # TODO I don't understand why the option is not registered if it is not set
+                       action="store", dest="obi:taxo",
+                       metavar='<TAXDUMP PATH>',
+                       default='',  # TODO not None because if it's None, the option is not entered in the option dictionary.
+                       type=str,
+                       help="Path to a taxdump to test the taxonomy.")
+
+     
+    group=parser.add_argument_group('obi test specific options')
+ 
+    group.add_argument('--nb_tests','-n',
+                       action="store", dest="test:nbtests",
+                       metavar='<NB_TESTS>',
+                       default=1000,
+                       type=int,
+                       help="Number of tests to carry out. "
+                            "Default: 1000")
+ 
+    group.add_argument('--seq_max_len','-s',
+                       action="store", dest="test:seqmaxlen",
+                       metavar='<SEQ_MAX_LEN>',
+                       default=200,
+                       type=int,
+                       help="Maximum length of DNA sequences. "
+                            "Default: 200")
+ 
+    group.add_argument('--str_max_len','-r',
+                       action="store", dest="test:strmaxlen",
+                       metavar='<STR_MAX_LEN>',
+                       default=200,
+                       type=int,
+                       help="Maximum length of character strings. "
+                            "Default: 200")
+ 
+    group.add_argument('--comments_max_len','-c',
+                       action="store", dest="test:commentsmaxlen",
+                       metavar='<COMMENTS_MAX_LEN>',
+                       default=10000,
+                       type=int,
+                       help="Maximum length of view comments. "
+                            "Default: 10000")
+ 
+    group.add_argument('--max_ini_col_count','-o',
+                       action="store", dest="test:maxinicolcount",
+                       metavar='<MAX_INI_COL_COUNT>',
+                       default=10,
+                       type=int,
+                       help="Maximum number of columns in the initial view. "
+                            "Default: 10")
+ 
+    group.add_argument('--max_line_nb','-l',
+                       action="store", dest="test:maxlinenb",
+                       metavar='<MAX_LINE_NB>',
+                       default=10000,
+                       type=int,
+                       help="Maximum number of lines in a column. "
+                            "Default: 10000")
+ 
+    group.add_argument('--max_elts_per_line','-e',
+                       action="store", dest="test:maxelts",
+                       metavar='<MAX_ELTS_PER_LINE>',
+                       default=20,
+                       type=int,
+                       help="Maximum number of elements per line in a column. "
+                            "Default: 20")
+ 
+    group.add_argument('--verbose','-v',
+                       action="store_true", dest="test:verbose",
+                       default=False,
+                       help="Print the tests. "
+                            "Default: Don't print the tests")
+ 
+    group.add_argument('--seed','-g',
+                       action="store", dest="test:seed",
+                       metavar='<SEED>',
+                       default=None,
+                       help="Seed (use for reproducible tests). "
+                            "Default: Seed is determined by Python")
+ 
+def run(config):
+     
+    if 'seed' in config['test'] :
+        random.seed(config['test']['seed'])
+     
+    infos = {'dms': None, 
+             'view': None, 
+             'view_names': None, 
+             'unique_names': [],
+             'random_generator': {b"OBI_BOOL": random_bool, b"OBI_CHAR": random_char, b"OBI_FLOAT": random_float, b"OBI_INT": random_int, b"OBI_SEQ": random_seq, b"OBI_STR": random_bytes},
+             'tests': [test_set_and_get, test_add_col, test_delete_col, test_col_alias, test_new_view]
+            }
+ 
+    config['test']['elt_name_max_len'] = int((COL_COMMENTS_MAX_LEN - config['test']['maxelts']) / config['test']['maxelts'])
+ 
+    print("Initializing the DMS and the first view...")
+     
+    shutil.rmtree(config['obi']['defaultdms']+'.obidms', ignore_errors=True)
+    
+    ini_dms_and_first_view(config, infos)
+    print_test(config, repr(infos['view']))
+    
+    i = 0
+    for t in range(config['test']['nbtests']):
+        random_test(config, infos)
+        print_test(config, repr(infos['view']))
+        i+=1
+        if (i%(config['test']['nbtests']/10)) == 0 :
+            print("Testing......"+str(i*100/config['test']['nbtests'])+"%")
+ 
+    #print(infos)
+    
+    if config['obi']['taxo'] != '' :
+        test_taxo(config, infos)
+    
+    infos['view'].close()
+    infos['dms'].close()
+    shutil.rmtree(config['obi']['defaultdms']+'.obidms', ignore_errors=True)
+     
+    print("Done.")
+
+        

python/obitools3/dms/__init__.py

@@ -0,0 +1,2 @@
+from .dms import DMS  # @UnresolvedImport
+

python/obitools3/dms/capi/obialign.pxd

@@ -0,0 +1,41 @@
+#cython: language_level=3
+
+from obitools3.dms.capi.obidms       cimport OBIDMS_p
+from obitools3.dms.capi.obitypes     cimport const_char_p
+
+
+cdef extern from "obi_align.h" nogil:
+
+    int obi_lcs_align_one_column(OBIDMS_p dms, 
+                                 const_char_p seq_view_name, 
+                                 const_char_p seq_column_name, 
+                                 const_char_p seq_elt_name,
+                                 const_char_p id_column_name, 
+                                 const_char_p output_view_name, 
+                                 const_char_p output_view_comments,
+                                 bint print_seq, 
+                                 bint print_count,
+                                 double threshold, 
+                                 bint normalize, 
+                                 int reference, 
+                                 bint similarity_mode,
+                                 int thread_count)
+
+
+    int obi_lcs_align_two_columns(OBIDMS_p dms,
+                                  const_char_p seq1_view_name,
+                                  const_char_p seq2_view_name,
+                                  const_char_p seq1_column_name,
+                                  const_char_p seq2_column_name,
+                                  const_char_p seq1_elt_name,
+                                  const_char_p seq2_elt_name,
+                                  const_char_p id1_column_name,
+                                  const_char_p id2_column_name,
+                                  const_char_p output_view_name, 
+                                  const_char_p output_view_comments,
+                                  bint print_seq, 
+                                  bint print_count,
+                                  double threshold, 
+                                  bint normalize, 
+                                  int reference, 
+                                  bint similarity_mode)

python/obitools3/dms/capi/obidms.pxd

@@ -0,0 +1,19 @@
+#cython: language_level=3
+
+from .obitypes cimport const_char_p
+
+cdef extern from "obidms.h" nogil:
+    struct OBIDMS_t:
+        const_char_p dms_name
+
+    ctypedef OBIDMS_t* OBIDMS_p 
+
+    OBIDMS_p obi_dms(const_char_p dms_name)
+    OBIDMS_p obi_open_dms(const_char_p dms_path)
+    OBIDMS_p obi_test_open_dms(const_char_p dms_path)
+    OBIDMS_p obi_create_dms(const_char_p dms_path)
+    int obi_close_dms(OBIDMS_p dms)
+    char* obi_dms_get_dms_path(OBIDMS_p dms)
+    char* obi_dms_get_full_path(OBIDMS_p dms, const_char_p path_name)
+
+

python/obitools3/dms/capi/obidmscolumn.pxd

@@ -0,0 +1,62 @@
+#cython: language_level=3
+
+from ..capi.obidms   cimport OBIDMS_p
+from ..capi.obitypes cimport const_char_p, \
+                             OBIType_t, \
+                             obiversion_t, \
+                             obiint_t, \
+                             obibool_t, \
+                             obichar_t, \
+                             obifloat_t, \
+                             index_t, \
+                             time_t
+
+from libc.stdint cimport uint8_t
+
+
+cdef extern from "obidmscolumn.h" nogil:
+    
+    struct Column_reference_t :
+        const_char_p  column_name
+        obiversion_t  version
+    
+    ctypedef Column_reference_t* Column_reference_p
+
+    struct OBIDMS_column_header_t:
+        size_t              header_size
+        size_t              data_size
+        index_t             line_count
+        index_t             lines_used
+        index_t             nb_elements_per_line
+        const_char_p        elements_names
+        OBIType_t           returned_data_type
+        OBIType_t           stored_data_type
+        time_t              creation_date
+        obiversion_t        version
+        obiversion_t        cloned_from
+        const_char_p        name
+        const_char_p        indexer_name
+        Column_reference_t  associated_column
+        const_char_p        comments
+    
+    ctypedef OBIDMS_column_header_t* OBIDMS_column_header_p
+
+    struct OBIDMS_column_t:
+        OBIDMS_p               dms
+        OBIDMS_column_header_p header
+        bint                   writable
+        
+    ctypedef OBIDMS_column_t* OBIDMS_column_p
+        
+    int obi_close_column(OBIDMS_column_p column)
+    
+    obiversion_t obi_column_get_latest_version_from_name(OBIDMS_p dms, 
+                                                         const_char_p column_name)
+        
+    OBIDMS_column_header_p obi_column_get_header_from_name(OBIDMS_p dms, 
+                                                           const_char_p column_name,
+                                                           obiversion_t version_number)
+    
+    int obi_close_header(OBIDMS_column_header_p header)
+    
+    char* obi_get_elements_names(OBIDMS_column_p column)

python/obitools3/dms/capi/obierrno.pxd

@@ -0,0 +1,8 @@
+#cython: language_level=3
+
+
+cdef extern from "obierrno.h":
+    int obi_errno
+
+    extern int OBI_LINE_IDX_ERROR
+    extern int OBI_ELT_IDX_ERROR

python/obitools3/dms/capi/obitaxonomy.pxd

@@ -7,21 +7,38 @@ from libc.stdint   cimport int32_t
 
 cdef extern from "obidms_taxonomy.h" nogil:
 
-    struct OBIDMS_taxonomy_t
-    ctypedef OBIDMS_taxonomy_t* OBIDMS_taxonomy_p
-    
     struct ecotxnode :
         int32_t           taxid
         int32_t           rank
         int32_t           farest
         ecotxnode*        parent
         char*             name
-    
+        char*             preferred_name
+
     ctypedef ecotxnode ecotx_t
 
 
-    OBIDMS_taxonomy_p obi_read_taxonomy(OBIDMS_p dms, const_char_p taxonomy_name, bint read_alternative_names)
+    struct ecotxidx_t :   
+        int32_t  count
+        int32_t  max_taxid
+        int32_t  buffer_size
+        ecotx_t* taxon
+
+
+    struct OBIDMS_taxonomy_t :
+#        ecorankidx_t* ranks
+#        econameidx_t* names
+        ecotxidx_t*   taxa
+
+    ctypedef OBIDMS_taxonomy_t* OBIDMS_taxonomy_p
     
+
+    OBIDMS_taxonomy_p obi_read_taxonomy(OBIDMS_p dms, const_char_p taxonomy_name, bint read_alternative_names)
+
+    OBIDMS_taxonomy_p obi_read_taxdump(const_char_p taxdump)
+    
+    int obi_write_taxonomy(OBIDMS_p dms, OBIDMS_taxonomy_p tax, const_char_p tax_name)
+
     int obi_close_taxonomy(OBIDMS_taxonomy_p taxonomy)
     
     ecotx_t* obi_taxo_get_parent_at_rank(ecotx_t* taxon, int32_t rankidx)
@@ -40,3 +57,9 @@ cdef extern from "obidms_taxonomy.h" nogil:
     
     ecotx_t* obi_taxo_get_superkingdom(ecotx_t* taxon, OBIDMS_taxonomy_p taxonomy)
     
+    int obi_taxo_add_local_taxon(OBIDMS_taxonomy_p tax, const char* name, const char* rank_name, int32_t parent_taxid, int32_t min_taxid)
+
+    int obi_taxo_add_preferred_name_with_taxid(OBIDMS_taxonomy_p tax, int32_t taxid, const char* preferred_name)
+
+    int obi_taxo_add_preferred_name_with_taxon(OBIDMS_taxonomy_p tax, ecotx_t* taxon, const char* preferred_name)
+

python/obitools3/dms/capi/obitypes.pxd

@@ -1,20 +1,19 @@
 #cython: language_level=3
 
 
-from libc.stdint cimport int32_t, int64_t
+from libc.stdint cimport int32_t, int64_t, uint8_t
+
 from posix.types cimport time_t
 
 
 cdef extern from *:
     ctypedef char* const_char_p "const char*"
 
+
 cdef extern from "encode.h" nogil:
     bint only_ATGC(const_char_p seq)
-
+    bint only_IUPAC_DNA(const_char_p seq)
-
+    bint is_a_DNA_seq(const_char_p seq)
-cdef extern from "obidmscolumn.h" nogil:
-
-    ctypedef int32_t obiversion_t
 
 
 cdef extern from "obitypes.h" nogil:
@@ -25,6 +24,7 @@ cdef extern from "obitypes.h" nogil:
         OBI_FLOAT,
         OBI_BOOL,
         OBI_CHAR,
+        OBI_QUAL,
         OBI_STR,
         OBI_SEQ,
         OBI_IDX
@@ -37,10 +37,12 @@ cdef extern from "obitypes.h" nogil:
     
     ctypedef OBIBool obibool_t
     ctypedef int32_t obiint_t
-    ctypedef double obifloat_t
+    ctypedef double  obifloat_t
-    ctypedef char obichar_t  
+    ctypedef char    obichar_t  
     ctypedef int64_t index_t
 
+    ctypedef int32_t obiversion_t
+
     extern obiint_t      OBIInt_NA
     extern index_t       OBIIdx_NA
     extern obifloat_t    OBIFloat_NA
@@ -48,5 +50,9 @@ cdef extern from "obitypes.h" nogil:
     extern obibool_t     OBIBool_NA
     extern const_char_p  OBISeq_NA
     extern const_char_p  OBIStr_NA
+    extern const_char_p  OBIQual_char_NA
+    extern uint8_t*      OBIQual_int_NA
 
     const_char_p name_data_type(int data_type)
+
+ctypedef OBIType_t obitype_t

python/obitools3/dms/capi/obiutils.pxd

@@ -0,0 +1,12 @@
+#cython: language_level=3
+
+
+from posix.types cimport time_t
+
+from ..capi.obitypes cimport const_char_p
+
+
+cdef extern from "utils.h" nogil:
+
+    const_char_p obi_format_date(time_t date)
+

python/obitools3/dms/capi/obiview.pxd

@@ -0,0 +1,298 @@
+#cython: language_level=3
+
+from .obitypes           cimport const_char_p, \
+                                 OBIType_t, \
+                                 obiversion_t, \
+                                 obiint_t, \
+                                 obibool_t, \
+                                 obichar_t, \
+                                 obifloat_t, \
+                                 index_t, \
+                                 time_t
+from ..capi.obidms       cimport OBIDMS_p
+from ..capi.obidmscolumn cimport OBIDMS_column_p, \
+                                 Column_reference_t, \
+                                 Column_reference_p
+
+from libc.stdint cimport uint8_t
+
+
+cdef extern from "obiview.h" nogil:
+
+    extern const_char_p VIEW_TYPE_NUC_SEQS
+    extern const_char_p NUC_SEQUENCE_COLUMN
+    extern const_char_p ID_COLUMN
+    extern const_char_p DEFINITION_COLUMN
+    extern const_char_p QUALITY_COLUMN
+
+
+    struct Alias_column_pair_t :
+        Column_reference_t    column_refs
+        const_char_p          alias
+    
+    ctypedef Alias_column_pair_t* Alias_column_pair_p
+
+
+    struct Obiview_infos_t :
+        time_t              creation_date
+        const_char_p        name
+        const_char_p        created_from
+        const_char_p        view_type
+        bint                all_lines
+        Column_reference_t  line_selection
+        index_t             line_count
+        int                 column_count
+        Alias_column_pair_p column_references
+        const_char_p        comments
+
+    ctypedef Obiview_infos_t* Obiview_infos_p
+
+
+    struct Obiview_t :
+        Obiview_infos_p     infos
+        OBIDMS_p            dms
+        bint                read_only
+        OBIDMS_column_p     line_selection
+        OBIDMS_column_p     columns
+        int                 nb_predicates
+        # TODO declarations for column dictionary and predicate function array?
+        
+    ctypedef Obiview_t* Obiview_p
+
+
+    Obiview_p obi_new_view_nuc_seqs(OBIDMS_p dms, const_char_p view_name, Obiview_p view_to_clone, index_t* line_selection, const_char_p comments, bint quality_column)
+
+    Obiview_p obi_new_view(OBIDMS_p dms, const_char_p view_name, Obiview_p view_to_clone, index_t* line_selection, const_char_p comments)
+    
+    Obiview_p obi_new_view_cloned_from_name(OBIDMS_p dms, const_char_p view_name, const_char_p view_to_clone_name, index_t* line_selection, const_char_p comments)
+
+    Obiview_p obi_new_view_nuc_seqs_cloned_from_name(OBIDMS_p dms, const_char_p view_name, const_char_p view_to_clone_name, index_t* line_selection, const_char_p comments)
+    
+    Obiview_p obi_clone_view(OBIDMS_p dms, Obiview_p view_to_clone, const char* view_name, index_t* line_selection, const char* comments)
+
+    Obiview_p obi_clone_view_from_name(OBIDMS_p dms, const char* view_to_clone_name, const char* view_name, index_t* line_selection, const char* comments)
+    
+    Obiview_infos_p obi_view_map_file(OBIDMS_p dms, const char* view_name, bint finished)
+
+    int obi_view_unmap_file(OBIDMS_p dms, Obiview_infos_p view_infos)
+
+    Obiview_p obi_open_view(OBIDMS_p dms, const_char_p view_name)
+    
+    int obi_view_add_column(Obiview_p view,
+                            const_char_p column_name,
+                            obiversion_t version_number,
+                            const_char_p alias,
+                            OBIType_t data_type,
+                            index_t nb_lines,
+                            index_t nb_elements_per_line,
+                            char* elements_names,
+                            const_char_p indexer_name,
+                            const_char_p associated_column_name,
+                            obiversion_t associated_column_version,
+                            const_char_p comments,
+                            bint create)
+
+    int obi_view_delete_column(Obiview_p view, const_char_p column_name)
+            
+    OBIDMS_column_p obi_view_get_column(Obiview_p view, const_char_p column_name)
+
+    OBIDMS_column_p* obi_view_get_pointer_on_column_in_view(Obiview_p view, const_char_p column_name)
+
+    int obi_view_create_column_alias(Obiview_p view, const_char_p current_name, const_char_p alias)    
+    
+    int obi_save_and_close_view(Obiview_p view)
+
+
+    # OBI_INT
+    int obi_set_int_with_elt_name_and_col_p_in_view(Obiview_p view,
+                                                    OBIDMS_column_p column_p, 
+                                                    index_t line_nb, 
+                                                    const_char_p element_name, 
+                                                    obiint_t value)
+
+    int obi_set_int_with_elt_idx_and_col_p_in_view(Obiview_p view,
+                                                   OBIDMS_column_p column_p, 
+                                                   index_t line_nb, 
+                                                   index_t element_idx, 
+                                                   obiint_t value)
+    
+    obiint_t obi_get_int_with_elt_name_and_col_p_in_view(Obiview_p view,
+                                                         OBIDMS_column_p column_p, 
+                                                         index_t line_nb, 
+                                                         const_char_p element_name)
+    
+    obiint_t obi_get_int_with_elt_idx_and_col_p_in_view(Obiview_p view,
+                                                        OBIDMS_column_p column_p, 
+                                                        index_t line_nb, 
+                                                        index_t element_idx)
+
+
+    # OBI_BOOL
+    int obi_set_bool_with_elt_name_and_col_p_in_view(Obiview_p view,
+                                                     OBIDMS_column_p column_p, 
+                                                     index_t line_nb, 
+                                                     const_char_p element_name, 
+                                                     obibool_t value)
+
+    int obi_set_bool_with_elt_idx_and_col_p_in_view(Obiview_p view,
+                                                    OBIDMS_column_p column_p, 
+                                                    index_t line_nb, 
+                                                    index_t element_idx, 
+                                                    obibool_t value)
+    
+    obibool_t obi_get_bool_with_elt_name_and_col_p_in_view(Obiview_p view,
+                                                           OBIDMS_column_p column_p, 
+                                                           index_t line_nb, 
+                                                           const_char_p element_name)
+    
+    obibool_t obi_get_bool_with_elt_idx_and_col_p_in_view(Obiview_p view,
+                                                          OBIDMS_column_p column_p, 
+                                                          index_t line_nb, 
+                                                          index_t element_idx)
+
+
+    # OBI_CHAR
+    int obi_set_char_with_elt_name_and_col_p_in_view(Obiview_p view,
+                                                     OBIDMS_column_p column_p, 
+                                                     index_t line_nb, 
+                                                     const_char_p element_name, 
+                                                     obichar_t value)
+
+    int obi_set_char_with_elt_idx_and_col_p_in_view(Obiview_p view,
+                                                    OBIDMS_column_p column_p, 
+                                                    index_t line_nb, 
+                                                    index_t element_idx, 
+                                                    obichar_t value)
+    
+    obichar_t obi_get_char_with_elt_name_and_col_p_in_view(Obiview_p view,
+                                                           OBIDMS_column_p column_p, 
+                                                           index_t line_nb, 
+                                                           const_char_p element_name)
+    
+    obichar_t obi_get_char_with_elt_idx_and_col_p_in_view(Obiview_p view,
+                                                          OBIDMS_column_p column_p, 
+                                                          index_t line_nb, 
+                                                          index_t element_idx)
+
+
+    # OBI_FLOAT
+    int obi_set_float_with_elt_name_and_col_p_in_view(Obiview_p view,
+                                                      OBIDMS_column_p column_p, 
+                                                      index_t line_nb, 
+                                                      const_char_p element_name, 
+                                                      obifloat_t value)
+
+    int obi_set_float_with_elt_idx_and_col_p_in_view(Obiview_p view,
+                                                     OBIDMS_column_p column_p, 
+                                                     index_t line_nb, 
+                                                     index_t element_idx, 
+                                                     obifloat_t value)
+    
+    obifloat_t obi_get_float_with_elt_name_and_col_p_in_view(Obiview_p view,
+                                                             OBIDMS_column_p column_p, 
+                                                             index_t line_nb, 
+                                                             const_char_p element_name)
+    
+    obifloat_t obi_get_float_with_elt_idx_and_col_p_in_view(Obiview_p view,
+                                                            OBIDMS_column_p column_p, 
+                                                            index_t line_nb, 
+                                                            index_t element_idx)
+
+
+    # OBI_QUAL
+    int obi_set_qual_char_with_elt_idx_and_col_p_in_view(Obiview_p view, 
+                                                         OBIDMS_column_p column_p, 
+                                                         index_t line_nb, 
+                                                         index_t element_idx, 
+                                                         const char* value)
+
+    int obi_set_qual_int_with_elt_idx_and_col_p_in_view(Obiview_p view, 
+                                                        OBIDMS_column_p column_p, 
+                                                        index_t line_nb, 
+                                                        index_t element_idx, 
+                                                        const uint8_t* value, 
+                                                        int value_length)
+
+    char* obi_get_qual_char_with_elt_idx_and_col_p_in_view(Obiview_p view, 
+                                                           OBIDMS_column_p column_p, 
+                                                           index_t line_nb, 
+                                                           index_t element_idx)
+
+    const uint8_t* obi_get_qual_int_with_elt_idx_and_col_p_in_view(Obiview_p view, 
+                                                                   OBIDMS_column_p column_p, 
+                                                                   index_t line_nb, 
+                                                                   index_t element_idx, 
+                                                                   int* value_length)
+
+    int obi_set_qual_char_with_elt_name_and_col_p_in_view(Obiview_p view, 
+                                                          OBIDMS_column_p column_p, 
+                                                          index_t line_nb, 
+                                                          const char* element_name, 
+                                                          const char* value)
+
+    int obi_set_qual_int_with_elt_name_and_col_p_in_view(Obiview_p view, 
+                                                         OBIDMS_column_p column_p, 
+                                                         index_t line_nb, 
+                                                         const char* element_name, 
+                                                         const uint8_t* value, 
+                                                         int value_length)
+
+    char* obi_get_qual_char_with_elt_name_and_col_p_in_view(Obiview_p view, 
+                                                            OBIDMS_column_p column_p, 
+                                                            index_t line_nb, 
+                                                            const char* element_name)
+
+    const uint8_t* obi_get_qual_int_with_elt_name_and_col_p_in_view(Obiview_p view, 
+                                                                    OBIDMS_column_p column_p, 
+                                                                    index_t line_nb, 
+                                                                    const char* element_name, 
+                                                                    int* value_length)
+
+
+    # OBI_STR
+    int obi_set_str_with_elt_name_and_col_p_in_view(Obiview_p view,
+                                                    OBIDMS_column_p column_p, 
+                                                    index_t line_nb, 
+                                                    const_char_p element_name, 
+                                                    const_char_p value)
+
+    int obi_set_str_with_elt_idx_and_col_p_in_view(Obiview_p view,
+                                                   OBIDMS_column_p column_p, 
+                                                   index_t line_nb, 
+                                                   index_t element_idx, 
+                                                   const_char_p value)
+    
+    const_char_p obi_get_str_with_elt_name_and_col_p_in_view(Obiview_p view,
+                                                           OBIDMS_column_p column_p, 
+                                                           index_t line_nb, 
+                                                           const_char_p element_name)
+    
+    const_char_p obi_get_str_with_elt_idx_and_col_p_in_view(Obiview_p view,
+                                                            OBIDMS_column_p column_p, 
+                                                            index_t line_nb, 
+                                                            index_t element_idx)
+
+
+    # OBI_SEQ
+    int obi_set_seq_with_elt_name_and_col_p_in_view(Obiview_p view,
+                                                    OBIDMS_column_p column_p, 
+                                                    index_t line_nb, 
+                                                    const_char_p element_name, 
+                                                    const_char_p value)
+
+    int obi_set_seq_with_elt_idx_and_col_p_in_view(Obiview_p view,
+                                                   OBIDMS_column_p column_p, 
+                                                   index_t line_nb, 
+                                                   index_t element_idx, 
+                                                   const_char_p value)
+    
+    char* obi_get_seq_with_elt_name_and_col_p_in_view(Obiview_p view,
+                                                      OBIDMS_column_p column_p, 
+                                                      index_t line_nb, 
+                                                      const_char_p element_name)
+    
+    char* obi_get_seq_with_elt_idx_and_col_p_in_view(Obiview_p view,
+                                                     OBIDMS_column_p column_p, 
+                                                     index_t line_nb, 
+                                                     index_t element_idx)
+

python/obitools3/dms/column/__init__.py

@@ -0,0 +1 @@
+from .column import Column  # @UnresolvedImport

python/obitools3/dms/column/column.pxd

@@ -0,0 +1,52 @@
+#cython: language_level=3
+
+
+from ..capi.obitypes     cimport index_t, \
+                                 obitype_t
+                                 
+from ..capi.obidmscolumn cimport OBIDMS_column_p
+                                                                 
+from ..view.view cimport View
+ 
+from ..object cimport OBIWrapper                          
+
+ 
+cdef dict __OBIDMS_COLUMN_CLASS__
+
+
+cdef class Column(OBIWrapper) :
+ 
+    cdef View  _view
+    cdef bytes _alias
+    
+    cdef inline OBIDMS_column_p pointer(self)
+    cpdef close(self)
+ 
+    @staticmethod
+    cdef type get_column_class(obitype_t obitype, bint multi_elts)
+
+    @staticmethod
+    cdef type get_python_type(obitype_t obitype, bint multi_elts)
+
+
+cdef class Column_multi_elts(Column) :
+
+    # The type of [values] can be dict, Column_line, or any other class with values referenced by keys with an iterator [for key in values]
+    cpdef set_line(self, index_t line_nb, object values)
+
+
+cdef class Column_line:
+ 
+    cdef Column    _column
+    cdef index_t   _index
+
+    cpdef update(self, data)
+     
+    
+cdef register_column_class(obitype_t obitype,
+                           bint multi_elts,
+                           type obiclass, 
+                           type python)
+
+cdef register_all_column_classes()
+

python/obitools3/dms/column/column.pyx

@@ -0,0 +1,366 @@
+#cython: language_level=3
+
+from obitools3.dms.column import typed_column
+ 
+
+__OBIDMS_COLUMN_CLASS__ = {}
+
+
+from ..capi.obitypes cimport name_data_type, \
+                             obitype_t, \
+                             OBI_BOOL
+ 
+from ..capi.obidmscolumn cimport OBIDMS_column_header_p, \
+                                 obi_close_column, \
+                                 obi_get_elements_names
+                                     
+from ..capi.obiutils cimport obi_format_date
+ 
+from ..capi.obiview cimport obi_view_add_column, \
+                            obi_view_get_pointer_on_column_in_view, \
+                            Obiview_p
+
+from ..object cimport OBIObjectClosedInstance
+
+from obitools3.utils cimport tobytes, \
+                             bytes2str, \
+                             str2bytes
+ 
+from obitools3.dms.column import typed_column
+ 
+import importlib
+import inspect
+import pkgutil
+
+cdef class Column(OBIWrapper) :
+    '''
+    The obitools3.dms.column.Column class wraps a C instance of a column in the context of a View 
+    '''
+
+    cdef inline OBIDMS_column_p pointer(self) :
+        return <OBIDMS_column_p>(<OBIDMS_column_p*>(self._pointer))[0]
+
+
+    @staticmethod
+    cdef type get_column_class(obitype_t obitype, bint multi_elts):
+        '''
+        Internal function returning the python class representing 
+        a column for a given obitype.
+        '''
+        return __OBIDMS_COLUMN_CLASS__[(obitype, multi_elts)][0]
+
+
+    @staticmethod
+    cdef type get_python_type(obitype_t obitype, bint multi_elts):  # TODO
+        '''
+        Internal function returning the python type representing 
+        an instance for a given obitype.
+        '''
+        return __OBIDMS_COLUMN_CLASS__[(obitype, multi_elts)][1]
+
+
+    @staticmethod
+    def new_column(View      view,
+                   object    column_name,
+                   obitype_t data_type,
+                   index_t   nb_elements_per_line=1,
+                   list      elements_names=None,
+                   object    comments=b"",
+                   object    alias=b""):
+        # TODO indexer_name?
+        
+        cdef bytes column_name_b = tobytes(column_name)
+        cdef bytes alias_b = tobytes(alias)
+        cdef bytes comments_b = tobytes(comments)
+        cdef bytes elements_names_b
+        cdef char* elements_names_p
+
+        if not view.active() :
+            raise OBIObjectClosedInstance()
+ 
+        if alias_b == b"" :
+            alias_b = column_name_b
+ 
+        if elements_names is not None:
+            elements_names_b = b';'.join([tobytes(x) for x in elements_names])
+            elements_names_p = elements_names_b
+        else:
+            elements_names_p = NULL
+                                
+        if (obi_view_add_column(view                      = view.pointer(),
+                                column_name               = column_name_b,
+                                version_number            = -1,
+                                alias                     = alias_b,
+                                data_type                 = <obitype_t>data_type,
+                                nb_lines                  = len(view),
+                                nb_elements_per_line      = nb_elements_per_line,
+                                elements_names            = elements_names_p,
+                                indexer_name              = NULL,
+                                associated_column_name    = NULL,
+                                associated_column_version = -1,
+                                comments                  = comments_b,
+                                create                    = True)<0):
+            raise RuntimeError("Cannot create column %s in view %s" % (bytes2str(column_name_b),
+                                                                       bytes2str(view.name)))
+
+        return Column.open(view, alias_b)
+ 
+ 
+    @staticmethod
+    def open(View view,
+             object column_name):
+        cdef bytes            column_name_b = tobytes(column_name)
+        cdef OBIDMS_column_p* column_pp
+        cdef OBIDMS_column_p  column_p
+        cdef Column           column
+        cdef obitype_t        column_type
+        cdef type             column_class
+        
+        if not view.active() :
+            raise OBIObjectClosedInstance()
+        
+        column_pp = obi_view_get_pointer_on_column_in_view(view.pointer(), 
+                                                           column_name_b)
+         
+        if column_pp == NULL:
+            raise KeyError("Cannot access to column %s in view %s" % (
+                    bytes2str(column_name_b),
+                    bytes2str(view.name)
+                ))
+        
+        column_p = column_pp[0]
+        column_type = column_p.header.returned_data_type 
+        column_class = Column.get_column_class(column_type, (column_p.header.nb_elements_per_line > 1))
+        column = OBIWrapper.new(column_class, column_pp)
+        
+        column._view = view
+        column._alias = column_name_b
+        view.register(column)
+
+        return column
+
+
+    def add_to_view(self,
+                    View view,
+                    object column_name=None) :
+        
+        cdef bytes alias   
+        cdef OBIDMS_column_p column_p = self.pointer()
+        
+        if not view.active() :
+            raise OBIObjectClosedInstance()
+        
+        if (column_name is None):
+            alias = self._alias
+        else:
+            alias = tobytes(column_name)
+                       
+        if (obi_view_add_column(view                      = view.pointer(),
+                                column_name               = column_p.header.name,
+                                version_number            = column_p.header.version,
+                                alias                     = alias,
+                                data_type                 = <obitype_t>0,
+                                nb_lines                  = -1,
+                                nb_elements_per_line      = -1,
+                                elements_names            = NULL,
+                                indexer_name              = NULL,
+                                associated_column_name    = NULL,
+                                associated_column_version = -1,
+                                comments                  = NULL,
+                                create                    = False) < 0):
+            raise RuntimeError("Cannot insert column %s (%s@%d) into view %s" %
+                               ( bytes2str(alias),
+                                 bytes2str(column_p.header.name),
+                                 column_p.header.version,
+                                 bytes2str(view.name)
+                               ))
+            
+        view.register(self)
+        
+ 
+    def __len__(self):
+        '''
+        implements the len() function for the Column class
+         
+            @rtype: `int`
+        '''
+        return self.lines_used
+     
+     
+    def __sizeof__(self):
+        '''
+        returns the size of the C object wrapped by the Column instance
+        '''
+        cdef OBIDMS_column_header_p header = self.pointer().header
+        return header.header_size + header.data_size
+     
+     
+    def __iter__(self):
+        cdef index_t line_nb  
+ 
+        for line_nb in range(self.lines_used):
+            yield self[line_nb]
+
+
+    def __setitem__(self, index_t line_nb, object value):
+        self.set_line(line_nb, value)
+
+
+    def __getitem__(self, index_t line_nb):
+        return self.get_line(line_nb)
+  
+                 
+    def __str__(self) :
+        cdef str         to_print
+        cdef Column_line line
+        to_print = ''
+        for line in self :
+            to_print = to_print + str(line) + "\n"
+        return to_print
+ 
+ 
+    def __repr__(self) :
+        cdef bytes s
+        s = self._alias + b", original name: " + self.original_name + b", version " + str2bytes(str(self.version)) + b", data type: " + self.data_type
+        return bytes2str(s)     # TODO can't return bytes
+
+         
+    cpdef close(self):  # TODO discuss, can't be called bc then bug when closing view that tries to close it in C
+    
+        cdef OBIDMS_column_p pointer
+
+        if self.active() :
+            pointer = self.pointer()
+            self._view.unregister(self)
+            OBIWrapper.close(self)      
+            #if obi_close_column(pointer) < 0 :
+            #    raise Exception("Problem closing column %s" % bytes2str(self.name))
+ 
+ 
+    # Column alias property getter and setter
+    @property
+    def name(self):
+        return self._alias
+    @name.setter
+    def name(self, new_alias):  # @DuplicatedSignature
+        self._view.rename_column(self._alias, new_alias)
+ 
+    # elements_names property getter
+    @property
+    def elements_names(self):
+        return obi_get_elements_names(self.pointer()).split(b';')
+ 
+    # nb_elements_per_line property getter
+    @property
+    def nb_elements_per_line(self):
+        return self.pointer().header.nb_elements_per_line
+ 
+    # data_type property getter
+    @property
+    def data_type(self):
+        return name_data_type(self.pointer().header.returned_data_type)
+ 
+    # original_name property getter
+    @property
+    def original_name(self):
+        return self.pointer().header.name
+ 
+    # version property getter
+    @property
+    def version(self):
+        return self.pointer().header.version
+ 
+    # lines_used property getter
+    @property
+    def lines_used(self):
+        return self.pointer().header.lines_used
+ 
+    # comments property getter
+    @property
+    def comments(self):
+        return self.pointer().header.comments
+     
+    # creation_date property getter
+    @property
+    def creation_date(self):
+        return obi_format_date(self.pointer().header.creation_date)
+     
+ 
+######################################################################################################
+ 
+
+cdef class Column_multi_elts(Column) :
+    
+    def __getitem__(self, index_t line_nb):
+        return Column_line(self, line_nb)
+    
+    cpdef set_line(self, index_t line_nb, object values):
+        for element_name in values :
+            self.set_item(line_nb, element_name, values[element_name])
+
+
+######################################################################################################
+
+
+cdef class Column_line :
+ 
+    def __init__(self, Column column, index_t line_nb) :
+        self._index = line_nb
+        self._column = column
+         
+
+    def __getitem__(self, object elt_id) :
+        return self._column.get_item(self._index, elt_id)
+
+
+    def __setitem__(self, object elt_id, object value):
+        self._column.set_item(self._index, elt_id, value)
+
+
+    def __contains__(self, str element_name):
+        return (element_name in self._column.elements_names)
+ 
+ 
+    def __repr__(self) :
+        return str(self._column.get_line(self._index))
+ 
+     
+    cpdef update(self, data):       # TODO ?????
+        if isinstance(data, dict):
+            data=data.items()
+        for key,value in data:
+            if key in self:
+                self[key]=value
+
+ 
+######################################################################################################
+
+
+cdef register_column_class(obitype_t obitype,
+                           bint multi_elts,
+                           type obiclass,
+                           type python_type):
+    '''
+    Each sub class of `OBIDMS_column` needs to be registered after its declaration
+    to declare its relationship with an `OBIType_t`
+    '''
+    global __OBIDMS_COLUMN_CLASS__
+    
+    assert issubclass(obiclass, Column)
+    
+    __OBIDMS_COLUMN_CLASS__[(obitype, multi_elts)] = (obiclass, python_type)
+
+
+cdef register_all_column_classes() :
+    
+    x = list(pkgutil.walk_packages(typed_column.__path__, prefix="obitools3.dms.column.typed_column."))
+    all_modules = [importlib.import_module(a[1]) for a in x]
+    for mod in all_modules :
+        getattr(mod, 'register_class')()
+
+
+register_all_column_classes()
+
+
+
+    

python/obitools3/dms/column/typed_column/__init__.py

@@ -0,0 +1,3 @@
+#from .bool import Column_bool
+#from .int import Column_int
+# TODO why is this needed?

python/obitools3/dms/column/typed_column/bool.pxd

@@ -0,0 +1,44 @@
+# #cython: language_level=3
+
+
+from ...capi.obitypes cimport index_t
+
+from ..column cimport Column, \
+                      Column_multi_elts
+
+
+cdef class Column_bool(Column) :
+
+    cpdef object get_line(self, index_t line_nb)
+    cpdef set_line(self, index_t line_nb, object value)
+
+
+cdef class Column_multi_elts_bool(Column_multi_elts) :
+
+    cpdef object get_item(self, index_t line_nb, object elt_id)
+    cpdef object get_line(self, index_t line_nb)
+    cpdef set_item(self, index_t line_nb, object elt_id, object value)
+    
+    
+    
+    
+    
+    
+    
+    
+# cdef class Column_line_bool(Column_line) :
+#         
+#     @staticmethod
+#     cdef bool obibool_t2bool(obibool_t value)
+#     
+#     @staticmethod
+#     cdef bool2obibool_t(bool value)
+#     
+#     cpdef bool get_bool_item_by_name(self,bytes element_name)
+#     cpdef bool get_bool_item_by_idx(self,index_t index)
+#     cpdef set_bool_item_by_name(self,bytes element_name,bool value)
+#     cpdef set_bool_item_by_idx(self,index_t index,bool value)
+# 
+#       
+# #    cdef obibool_t [:]    _data_view
+#

python/obitools3/dms/column/typed_column/bool.pyx

@@ -0,0 +1,313 @@
+#cython: language_level=3
+
+
+from ..column cimport register_column_class
+
+from ...view.view cimport View
+
+from obitools3.utils cimport tobytes, \
+                             obi_errno_to_exception
+
+from ...capi.obiview cimport obi_get_bool_with_elt_name_and_col_p_in_view, \
+                             obi_get_bool_with_elt_idx_and_col_p_in_view, \
+                             obi_set_bool_with_elt_name_and_col_p_in_view, \
+                             obi_set_bool_with_elt_idx_and_col_p_in_view
+
+from ...capi.obitypes     cimport OBI_BOOL, OBIBool_NA, obibool_t
+
+from cpython.bool cimport PyBool_FromLong
+
+
+cdef class Column_bool(Column):
+
+    @staticmethod
+    def new(View    view,
+            object  column_name,
+            index_t nb_elements_per_line=1,
+            object  elements_names=None,
+            object  comments=b""):
+        return Column.new_column(view, column_name, OBI_BOOL,
+                          nb_elements_per_line=nb_elements_per_line, 
+                          elements_names=elements_names, 
+                          comments=comments)
+                
+
+    cpdef object get_line(self, index_t line_nb):
+        cdef   obibool_t value
+        cdef   object    result
+        value = obi_get_bool_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, 0)
+        obi_errno_to_exception(line_nb=line_nb, elt_id=None, error_message="Problem getting a value from a column")
+        if value == OBIBool_NA :
+            result = None
+        else :
+            result = PyBool_FromLong(value)
+        return result
+    
+    
+    cpdef set_line(self, index_t line_nb, object value):
+        if value is None :
+            value = OBIBool_NA
+        if obi_set_bool_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, 0, <obibool_t> value) < 0 :
+            obi_errno_to_exception(line_nb=line_nb, elt_id=None, error_message="Problem setting a value in a column")
+ 
+ 
+cdef class Column_multi_elts_bool(Column_multi_elts):
+ 
+    cpdef object get_item(self, index_t line_nb, object elt_id) :
+        cdef   obibool_t value
+        cdef   object    result
+        cdef   bytes     elt_name
+        if type(elt_id) == int :
+            value = obi_get_bool_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, elt_id)
+        else :
+            elt_name = tobytes(elt_id)
+            value = obi_get_bool_with_elt_name_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, elt_name)
+        obi_errno_to_exception(line_nb=line_nb, elt_id=elt_id, error_message="Problem getting a value from a column")
+        if value == OBIBool_NA :
+            result = None
+        else :
+            result = PyBool_FromLong(value)
+        return result
+    
+    cpdef object get_line(self, index_t line_nb) :
+        cdef   obibool_t value
+        cdef   object    value_in_result
+        cdef   dict      result
+        cdef   index_t   i
+        cdef   bint      all_NA
+        result = {}
+        all_NA = True
+        for i in range(self.nb_elements_per_line) :
+            value = obi_get_bool_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, i)
+            obi_errno_to_exception(line_nb=line_nb, elt_id=i, error_message="Problem getting a value from a column")
+            if value == OBIBool_NA :
+                value_in_result = None
+            else :
+                value_in_result = PyBool_FromLong(value)   
+            result[self.elements_names[i]] = value_in_result
+            if all_NA and (value_in_result is not None) :
+                all_NA = False
+        if all_NA :
+            result = None
+        return result
+    
+    cpdef set_item(self, index_t line_nb, object elt_id, object value) :
+        cdef   bytes elt_name
+        if value is None :
+            value = OBIBool_NA
+        if type(elt_id) == int :
+            if obi_set_bool_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, elt_id, <obibool_t> value) < 0 :
+                obi_errno_to_exception(line_nb=line_nb, elt_id=elt_id, error_message="Problem setting a value in a column")
+        else :
+            elt_name = tobytes(elt_id)
+            if obi_set_bool_with_elt_name_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, elt_name, <obibool_t> value) < 0 :
+                obi_errno_to_exception(line_nb=line_nb, elt_id=elt_id, error_message="Problem setting a value in a column")
+
+
+def register_class() :
+    register_column_class(OBI_BOOL, False, Column_bool, bool)
+    register_column_class(OBI_BOOL, True, Column_multi_elts_bool, bool)
+
+
+
+
+
+
+
+
+# cdef class Column_line_bool(Column_line) :
+# 
+#     cdef update_pointer(self):
+#         """
+#         Checks if the obicolumn address changed since the last call and update
+#         if need the `_column_p` and `_data_view` data structure fields.
+#         """
+#         cdef OBIDMS_column_p* column_pp
+#         column_pp = <OBIDMS_column_p*>self._pointer
+#         cdef OBIDMS_column_p column_p = column_pp[0]
+#         
+#         if column_p != self._column_p:
+#             self._column_p  = column_p
+#             self._data_view = (<obibool_t*> (column_p.data)) + \
+#                               self._index * column_p.header.nb_elements_per_line
+#     
+#     @staticmethod                      
+#     cdef bool obibool_t2bool(obibool_t value):
+#         cdef bool      result
+#         
+#         if value == OBIBool_NA :
+#             result = None
+#         else :
+#             result = PyBool_FromLong(value)
+#             
+#         return result
+#     
+#     @staticmethod                      
+#     cdef bool2obibool_t(bool value):
+#         cdef obibool_t result
+#         
+#         if value is None:
+#             result=OBIBool_NA 
+#         else:
+#             result= <obibool_t> <int> value
+#             
+#         return result
+#         
+# 
+#     def __init__(self, Column column, index_t line_nb) :
+#         """
+#         Creates a new `OBIDMS_column_line_bool`
+#         
+#             @param column: an OBIDMS_column instance
+#             @param line_nb: the line in the column 
+#         """
+#         
+#         Column_line.__init__(self, column, line_nb)
+#         self.update_pointer()
+# 
+#         
+#         
+#     cpdef bool get_bool_item_by_name(self, bytes element_name) :
+#         """
+#         Returns the value associated to the name `element_name` of the current line
+#         
+#             @param element_name: a `bytes` instance containing the name of the element
+#             
+#             @return: the `bool` value corresponding to the name
+#         """
+#         cdef char*     cname = element_name
+#         cdef obibool_t value 
+#         global         obi_errno
+#         
+#         self.update_pointer()
+#         
+#         cdef OBIDMS_column_p* column_pp
+#         column_pp = <OBIDMS_column_p*>self._pointer
+#         cdef OBIDMS_column_p column_p = column_pp[0]
+# 
+#         value = obi_column_get_obibool_with_elt_name(column_p,
+#                                                      self._index, 
+#                                                      cname)
+#         
+#         if obi_errno > 0 :
+#             obi_errno = 0
+#             raise KeyError("Cannot access to key %s" % bytes2str(element_name))
+#                     
+#         return Column_line_bool.obibool_t2bool(value)
+# 
+# 
+#     cpdef bool get_bool_item_by_idx(self,index_t index):
+#         """
+#         Returns the value associated to the name `element_name` of the current line
+#         
+#             @param index: a `int` instance containing the index of the element
+#             
+#             @return: the `bool` value corresponding to the name
+#         """
+#         cdef obibool_t value  # @DuplicatedSignature
+#         global obi_errno
+# 
+#         cdef OBIDMS_column_p* column_pp
+#         column_pp = <OBIDMS_column_p*>self._pointer
+#         cdef OBIDMS_column_p column_p = column_pp[0]
+# 
+#         self.update_pointer()
+#         
+#         value = obi_column_get_obibool_with_elt_idx(column_p,
+#                                                     self._index, 
+#                                                     index)
+#         
+#         if obi_errno > 0 :
+#             obi_errno = 0
+#             raise IndexError("Cannot access to element %d" % index)
+#                     
+#         return Column_line_bool.obibool_t2bool(value)
+# 
+#         
+#     def __getitem__(self, object element_name) :
+#         cdef bytes     name
+#         cdef int   cindex
+#         cdef obibool_t value        
+#         cdef type      typearg = type(element_name)
+#         cdef bool      result
+#         
+# 
+#         if typearg == int:
+#             cindex=element_name
+#             if cindex < 0:
+#                 cindex = self._len - cindex
+#             result=self.get_bool_item_by_idx(cindex)
+#         elif typearg == bytes:
+#             result=self.get_bool_item_by_name(element_name)
+#         elif typearg == str:
+#             name = str2bytes(element_name)
+#             result=self.get_bool_item_by_name(name)
+#         
+#         return result
+#     
+#     cpdef set_bool_item_by_name(self,bytes element_name,bool value):
+#         """
+#         Sets the value associated to the name `element_name` of the current line
+#         
+#             @param element_name: a `bytes` instance containing the name of the element
+#             @param value: a `bool` instance of the new value
+#             
+#             @return: the `bool` value corresponding to the name
+#         """
+#         cdef char*     cname = element_name
+#         cdef obibool_t cvalue 
+#         
+#         self.update_pointer()
+#         cvalue = OBIDMS_column_line_bool.bool2obibool_t(value)
+# 
+#         if ( obi_column_set_obibool_with_elt_name((<OBIDMS_column_p*>self._pointer)[0], 
+#                                                   self._index, 
+#                                                   cname,
+#                                                   cvalue) < 0 ):
+#             raise KeyError("Cannot access to key %s" % bytes2str(element_name))
+#                     
+#     cpdef set_bool_item_by_idx(self,index_t index,bool value):
+#         """
+#         Sets the value associated to the name `element_name` of the current line
+#         
+#             @param index: a `int` instance containing the index of the element
+#             @param value: a `bool` instance of the new value
+#             
+#             @return: the `bool` value corresponding to the name
+#         """
+#         cdef obibool_t cvalue  # @DuplicatedSignature
+#         
+#         self.update_pointer()
+#         cvalue = OBIDMS_column_line_bool.bool2obibool_t(value)
+# 
+#         if ( obi_column_set_obibool_with_elt_idx((<OBIDMS_column_p*>self._pointer)[0],
+#                                                  self._index, 
+#                                                  index,
+#                                                  cvalue) < 0 ):
+#             raise IndexError("Cannot access to item index %d" % index)
+#                     
+# 
+#              
+#     def __setitem__(self, object element_name, object value):
+#         cdef bytes     name
+#         cdef int   cindex
+#         cdef type      typearg = type(element_name)
+#         cdef bool      result
+#         
+# 
+#         if typearg == int:
+#             cindex=element_name
+#             if cindex < 0:
+#                 cindex = self._len - cindex
+#             self.set_bool_item_by_idx(cindex,value)
+#         elif typearg == bytes:
+#             self.set_bool_item_by_name(element_name,value)
+#         elif typearg == str:
+#             name = str2bytes(element_name)
+#             self.set_bool_item_by_name(name,value)
+# 
+#     def __repr__(self) :
+#         return str(self._column.get_line(self._index))
+#     
+#     def __len__(self):
+#         return self._len

python/obitools3/dms/column/typed_column/char.pxd

@@ -0,0 +1,20 @@
+# #cython: language_level=3
+ 
+
+from ...capi.obitypes cimport index_t
+
+from ..column cimport Column, \
+                      Column_multi_elts
+
+
+cdef class Column_char(Column) :
+
+    cpdef object get_line(self, index_t line_nb)
+    cpdef set_line(self, index_t line_nb, object value)
+
+
+cdef class Column_multi_elts_char(Column_multi_elts) :
+
+    cpdef object get_item(self, index_t line_nb, object elt_id)
+    cpdef object get_line(self, index_t line_nb)
+    cpdef set_item(self, index_t line_nb, object elt_id, object value)

python/obitools3/dms/column/typed_column/char.pyx

@@ -0,0 +1,114 @@
+#cython: language_level=3
+
+
+from ..column cimport register_column_class
+
+from ...view.view cimport View
+
+from obitools3.utils cimport tobytes, \
+                             obi_errno_to_exception
+
+from ...capi.obiview cimport obi_get_char_with_elt_name_and_col_p_in_view, \
+                             obi_get_char_with_elt_idx_and_col_p_in_view, \
+                             obi_set_char_with_elt_name_and_col_p_in_view, \
+                             obi_set_char_with_elt_idx_and_col_p_in_view
+
+from ...capi.obitypes     cimport OBI_CHAR, OBIChar_NA, obichar_t
+
+
+
+cdef class Column_char(Column):
+
+    @staticmethod
+    def new(View    view,
+            object  column_name,
+            index_t nb_elements_per_line=1,
+            object  elements_names=None,
+            object  comments=b""):
+        
+        return Column.new_column(view, column_name, OBI_CHAR,
+                                 nb_elements_per_line=nb_elements_per_line, 
+                                 elements_names=elements_names, 
+                                 comments=comments)
+
+    cpdef object get_line(self, index_t line_nb):
+        cdef   obichar_t value
+        cdef   object    result
+        value = obi_get_char_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, 0)
+        obi_errno_to_exception(line_nb=line_nb, elt_id=None, error_message="Problem getting a value from a column")
+        if value == OBIChar_NA :
+            result = None
+        else :
+            result = <bytes>value  # TODO return bytes or str?
+        return result
+    
+    cpdef set_line(self, index_t line_nb, object value):
+        cdef   obichar_t value_b
+        if value is None :
+            value_b = OBIChar_NA
+        else :
+            value_b = <obichar_t> tobytes(value)[0]
+        if obi_set_char_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, 0, value_b) < 0:
+            obi_errno_to_exception(line_nb=line_nb, elt_id=None, error_message="Problem setting a value in a column")
+ 
+ 
+cdef class Column_multi_elts_char(Column_multi_elts):
+ 
+    cpdef object get_item(self, index_t line_nb, object elt_id) :
+        cdef   obichar_t value
+        cdef   object    result
+        cdef   bytes     elt_name
+        if type(elt_id) == int :
+            value = obi_get_char_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, elt_id)
+        else :
+            elt_name = tobytes(elt_id)
+            value = obi_get_char_with_elt_name_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, elt_name)
+        obi_errno_to_exception(line_nb=line_nb, elt_id=elt_id, error_message="Problem getting a value from a column")
+        if value == OBIChar_NA :
+            result = None
+        else :
+            result = <bytes>value  # TODO return bytes or str?
+        return result
+    
+    cpdef object get_line(self, index_t line_nb) :
+        cdef obichar_t value
+        cdef object    value_in_result
+        cdef dict      result
+        cdef index_t   i
+        cdef bint      all_NA
+        result = {}
+        all_NA = True
+        for i in range(self.nb_elements_per_line) :
+            value = obi_get_char_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, i)
+            obi_errno_to_exception(line_nb=line_nb, elt_id=i, error_message="Problem getting a value from a column")
+            if value == OBIChar_NA :
+                value_in_result = None
+            else :
+                value_in_result = <bytes>value     # TODO return bytes or str?
+            result[self.elements_names[i]] = value_in_result
+            if all_NA and (value_in_result is not None) :
+                all_NA = False
+        if all_NA :
+            result = None
+        return result
+    
+    cpdef set_item(self, index_t line_nb, object elt_id, object value) :
+        cdef bytes elt_name
+        cdef obichar_t value_b
+        if value is None :
+            value_b = OBIChar_NA
+        else :
+            value_b = <obichar_t> tobytes(value)[0]
+        if type(elt_id) == int :
+            if obi_set_char_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, elt_id, value_b) < 0 :
+                obi_errno_to_exception(line_nb=line_nb, elt_id=elt_id, error_message="Problem setting a value in a column")
+        else :
+            elt_name = tobytes(elt_id)
+            if obi_set_char_with_elt_name_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, elt_name, value_b) < 0 :
+                obi_errno_to_exception(line_nb=line_nb, elt_id=elt_id, error_message="Problem setting a value in a column")
+
+
+def register_class():
+    register_column_class(OBI_CHAR, False, Column_char, bytes)  # TODO bytes or str?
+    register_column_class(OBI_CHAR, True, Column_multi_elts_char, bytes)  # TODO bytes or str?
+

python/obitools3/dms/column/typed_column/float.pxd

@@ -0,0 +1,21 @@
+# #cython: language_level=3
+ 
+
+from ...capi.obitypes cimport index_t
+
+from ..column cimport Column, \
+                      Column_multi_elts
+
+
+cdef class Column_float(Column) :
+
+    cpdef object get_line(self, index_t line_nb)
+    cpdef set_line(self, index_t line_nb, object value)
+
+
+cdef class Column_multi_elts_float(Column_multi_elts) :
+
+    cpdef object get_item(self, index_t line_nb, object elt_id)
+    cpdef object get_line(self, index_t line_nb)
+    cpdef set_item(self, index_t line_nb, object elt_id, object value)
+    

python/obitools3/dms/column/typed_column/float.pyx

@@ -0,0 +1,110 @@
+#cython: language_level=3
+
+
+from ..column cimport register_column_class
+
+from ...view.view cimport View
+
+from obitools3.utils cimport tobytes, \
+                             obi_errno_to_exception
+
+from ...capi.obiview cimport obi_get_float_with_elt_name_and_col_p_in_view, \
+                             obi_get_float_with_elt_idx_and_col_p_in_view, \
+                             obi_set_float_with_elt_name_and_col_p_in_view, \
+                             obi_set_float_with_elt_idx_and_col_p_in_view
+
+from ...capi.obitypes     cimport OBI_FLOAT, OBIFloat_NA, obifloat_t
+
+
+
+cdef class Column_float(Column):
+
+    @staticmethod
+    def new(View    view,
+            object  column_name,
+            index_t nb_elements_per_line=1,
+            object  elements_names=None,
+            object  comments=b""):
+        
+        return Column.new_column(view, column_name, OBI_FLOAT,
+                                 nb_elements_per_line=nb_elements_per_line, 
+                                 elements_names=elements_names, 
+                                 comments=comments)
+
+
+    cpdef object get_line(self, index_t line_nb):
+        cdef obifloat_t value
+        cdef object     result 
+        value = obi_get_float_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, 0)
+        obi_errno_to_exception(line_nb=line_nb, elt_id=None, error_message="Problem getting a value from a column")
+        if value == OBIFloat_NA :
+            result = None
+        else :
+            result = <double> value
+        return result
+    
+    
+    cpdef set_line(self, index_t line_nb, object value):
+        if value is None :
+            value = OBIFloat_NA
+        if obi_set_float_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, 0, <obifloat_t> value) < 0:
+            obi_errno_to_exception(line_nb=line_nb, elt_id=None, error_message="Problem setting a value in a column")
+ 
+ 
+cdef class Column_multi_elts_float(Column_multi_elts):
+ 
+    cpdef object get_item(self, index_t line_nb, object elt_id) :
+        cdef obifloat_t value
+        cdef object     result
+        cdef bytes      elt_name
+        if type(elt_id) == int :
+            value = obi_get_float_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, elt_id)
+        else :
+            elt_name = tobytes(elt_id)
+            value = obi_get_float_with_elt_name_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, elt_name)
+        obi_errno_to_exception(line_nb=line_nb, elt_id=elt_id, error_message="Problem getting a value from a column")
+        if value == OBIFloat_NA :
+            result = None
+        else :
+            result = <double> value
+        return result
+    
+    cpdef object get_line(self, index_t line_nb) :
+        cdef obifloat_t value
+        cdef object     value_in_result
+        cdef dict result
+        cdef index_t i
+        cdef bint all_NA
+        result = {}
+        all_NA = True
+        for i in range(self.nb_elements_per_line) :
+            value = obi_get_float_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, i)
+            obi_errno_to_exception(line_nb=line_nb, elt_id=i, error_message="Problem getting a value from a column")
+            if value == OBIFloat_NA :
+                value_in_result = None
+            else :
+                value_in_result = <double> value   
+            result[self.elements_names[i]] = value_in_result
+            if all_NA and (value_in_result is not None) :
+                all_NA = False
+        if all_NA :
+            result = None
+        return result
+    
+    cpdef set_item(self, index_t line_nb, object elt_id, object value) :
+        cdef bytes elt_name
+        if value is None :
+            value = OBIFloat_NA
+        if type(elt_id) == int :
+            if obi_set_float_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, elt_id, <obifloat_t> value) < 0 :
+                obi_errno_to_exception(line_nb=line_nb, elt_id=elt_id, error_message="Problem setting a value in a column")
+        else :
+            elt_name = tobytes(elt_id)
+            if obi_set_float_with_elt_name_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, elt_name, <obifloat_t> value) < 0 :
+                obi_errno_to_exception(line_nb=line_nb, elt_id=elt_id, error_message="Problem setting a value in a column")
+
+
+def register_class():
+    register_column_class(OBI_FLOAT, False, Column_float, float)    # TODO why not double?
+    register_column_class(OBI_FLOAT, True, Column_multi_elts_float, float)    # TODO why not double?
+

python/obitools3/dms/column/typed_column/int.pxd

@@ -0,0 +1,21 @@
+# #cython: language_level=3
+ 
+
+from ...capi.obitypes cimport index_t
+
+from ..column cimport Column, \
+                      Column_multi_elts
+
+
+cdef class Column_int(Column) :
+
+    cpdef object get_line(self, index_t line_nb)
+    cpdef set_line(self, index_t line_nb, object value)
+
+
+cdef class Column_multi_elts_int(Column_multi_elts) :
+
+    cpdef object get_item(self, index_t line_nb, object elt_id)
+    cpdef object get_line(self, index_t line_nb)
+    cpdef set_item(self, index_t line_nb, object elt_id, object value)
+    

python/obitools3/dms/column/typed_column/int.pyx

@@ -0,0 +1,111 @@
+#cython: language_level=3
+
+
+from ..column cimport register_column_class
+
+from ...view.view cimport View
+
+from obitools3.utils cimport tobytes, \
+                             obi_errno_to_exception
+
+from ...capi.obiview cimport obi_get_int_with_elt_name_and_col_p_in_view, \
+                             obi_get_int_with_elt_idx_and_col_p_in_view, \
+                             obi_set_int_with_elt_name_and_col_p_in_view, \
+                             obi_set_int_with_elt_idx_and_col_p_in_view
+
+from ...capi.obitypes     cimport OBI_INT, OBIInt_NA, obiint_t
+
+from cpython.int cimport PyInt_FromLong
+
+
+cdef class Column_int(Column):
+
+    @staticmethod
+    def new(View    view,
+            object  column_name,
+            index_t nb_elements_per_line=1,
+            object  elements_names=None,
+            object  comments=b""):
+        
+        return Column.new_column(view, column_name, OBI_INT,
+                                 nb_elements_per_line=nb_elements_per_line, 
+                                 elements_names=elements_names, 
+                                 comments=comments)
+
+
+    cpdef object get_line(self, index_t line_nb):
+        cdef obiint_t value
+        cdef object result 
+        value = obi_get_int_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, 0)
+        obi_errno_to_exception(line_nb=line_nb, elt_id=None, error_message="Problem getting a value from a column")
+        if value == OBIInt_NA :
+            result = None
+        else :
+            result = PyInt_FromLong(value)
+        return result
+    
+    
+    cpdef set_line(self, index_t line_nb, object value):
+        if value is None :
+            value = OBIInt_NA
+        if obi_set_int_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, 0, <obiint_t> value) < 0:
+            obi_errno_to_exception(line_nb=line_nb, elt_id=None, error_message="Problem setting a value in a column")
+ 
+ 
+cdef class Column_multi_elts_int(Column_multi_elts):
+ 
+    cpdef object get_item(self, index_t line_nb, object elt_id) :
+        cdef obiint_t  value
+        cdef object    result
+        cdef bytes     elt_name
+        if type(elt_id) == int :
+            value = obi_get_int_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, elt_id)
+        else :
+            elt_name = tobytes(elt_id)
+            value = obi_get_int_with_elt_name_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, elt_name)
+        obi_errno_to_exception(line_nb=line_nb, elt_id=elt_id, error_message="Problem getting a value from a column")
+        if value == OBIInt_NA :
+            result = None
+        else :
+            result = PyInt_FromLong(value)
+        return result
+    
+    cpdef object get_line(self, index_t line_nb) :
+        cdef obiint_t value
+        cdef object   value_in_result
+        cdef dict result
+        cdef index_t i
+        cdef bint all_NA
+        result = {}
+        all_NA = True
+        for i in range(self.nb_elements_per_line) :
+            value = obi_get_int_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, i)
+            obi_errno_to_exception(line_nb=line_nb, elt_id=i, error_message="Problem getting a value from a column")
+            if value == OBIInt_NA :
+                value_in_result = None
+            else :
+                value_in_result = PyInt_FromLong(value)   
+            result[self.elements_names[i]] = value_in_result
+            if all_NA and (value_in_result is not None) :
+                all_NA = False
+        if all_NA :
+            result = None
+        return result
+    
+    cpdef set_item(self, index_t line_nb, object elt_id, object value) :
+        cdef bytes elt_name
+        if value is None :
+            value = OBIInt_NA
+        if type(elt_id) == int :
+            if obi_set_int_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, elt_id, <obiint_t> value) < 0 :
+                obi_errno_to_exception(line_nb=line_nb, elt_id=elt_id, error_message="Problem setting a value in a column")
+        else :
+            elt_name = tobytes(elt_id)
+            if obi_set_int_with_elt_name_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, elt_name, <obiint_t> value) < 0 :
+                obi_errno_to_exception(line_nb=line_nb, elt_id=elt_id, error_message="Problem setting a value in a column")
+
+
+def register_class():
+    register_column_class(OBI_INT, False, Column_int, int)
+    register_column_class(OBI_INT, True, Column_multi_elts_int, int)
+

python/obitools3/dms/column/typed_column/qual.pxd

@@ -0,0 +1,26 @@
+# #cython: language_level=3
+ 
+
+from ...capi.obitypes cimport index_t
+
+from ..column cimport Column, \
+                      Column_multi_elts
+
+
+cdef class Column_qual(Column) :
+
+    cpdef object get_line(self, index_t line_nb)
+    cpdef object get_str_line(self, index_t line_nb)
+    cpdef set_line(self, index_t line_nb, object value)
+    cpdef set_str_line(self, index_t line_nb, object value)
+
+
+cdef class Column_multi_elts_qual(Column_multi_elts) :
+
+    cpdef object get_item(self, index_t line_nb, object elt_id)
+    cpdef object get_str_item(self, index_t line_nb, object elt_id)
+    cpdef object get_line(self, index_t line_nb)
+    cpdef object get_str_line(self, index_t line_nb)
+    cpdef set_item(self, index_t line_nb, object elt_id, object value)
+    cpdef set_str_item(self, index_t line_nb, object elt_id, object value)
+    

python/obitools3/dms/column/typed_column/qual.pyx

@@ -0,0 +1,236 @@
+#cython: language_level=3
+
+
+from ..column cimport register_column_class
+
+from ...view.view cimport View
+
+from obitools3.utils cimport tobytes, bytes2str, \
+                             obi_errno_to_exception
+
+from ...capi.obiview cimport obi_get_qual_char_with_elt_name_and_col_p_in_view, \
+                             obi_get_qual_char_with_elt_idx_and_col_p_in_view, \
+                             obi_set_qual_char_with_elt_name_and_col_p_in_view, \
+                             obi_set_qual_char_with_elt_idx_and_col_p_in_view, \
+                             obi_get_qual_int_with_elt_name_and_col_p_in_view, \
+                             obi_get_qual_int_with_elt_idx_and_col_p_in_view, \
+                             obi_set_qual_int_with_elt_name_and_col_p_in_view, \
+                             obi_set_qual_int_with_elt_idx_and_col_p_in_view
+
+from ...capi.obitypes     cimport OBI_QUAL, OBIQual_char_NA, OBIQual_int_NA, const_char_p
+
+from libc.stdlib cimport free
+from libc.stdint cimport uint8_t
+from libc.stdlib cimport malloc
+
+
+cdef class Column_qual(Column):
+
+    @staticmethod
+    def new(View    view,
+            object  column_name,
+            index_t nb_elements_per_line=1,
+            object  elements_names=None,
+            object  comments=b""):
+        
+        return Column.new_column(view, column_name, OBI_QUAL,
+                                 nb_elements_per_line=nb_elements_per_line, 
+                                 elements_names=elements_names, 
+                                 comments=comments)
+
+
+    cpdef object get_line(self, index_t line_nb):
+        cdef const uint8_t* value
+        cdef int            value_length
+        cdef object         result 
+        value = obi_get_qual_int_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, 0, &value_length)
+        obi_errno_to_exception(line_nb=line_nb, elt_id=None, error_message="Problem getting a value from a column")
+        if value == OBIQual_int_NA :
+            result = None
+        else :
+            result = []
+            for i in range(value_length) :
+                result.append(<int>value[i])            
+        return result
+
+    
+    cpdef object get_str_line(self, index_t line_nb):
+        cdef   char*  value
+        cdef   object result
+        cdef   int    i
+        value = obi_get_qual_char_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, 0)
+        obi_errno_to_exception(line_nb=line_nb, elt_id=None, error_message="Problem getting a value from a column")
+        if value == OBIQual_char_NA :
+            result = None
+        else :  # TODO discuss
+            result = bytes2str(value)
+            free(value)
+        return result
+    
+    
+    cpdef set_line(self, index_t line_nb, object value):
+        cdef uint8_t* value_b
+        cdef int      value_length
+        if value is None :
+            if obi_set_qual_int_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, 0, OBIQual_int_NA, 0) < 0:
+                obi_errno_to_exception(line_nb=line_nb, elt_id=None, error_message="Problem setting a value in a column")
+        else :
+            value_length = len(value)
+            value_b = <uint8_t*> malloc(value_length * sizeof(uint8_t))
+            for i in range(value_length) :
+                value_b[i] = <uint8_t>value[i]
+            if obi_set_qual_int_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, 0, value_b, value_length) < 0:
+                obi_errno_to_exception(line_nb=line_nb, elt_id=None, error_message="Problem setting a value in a column")
+            free(value_b)
+ 
+ 
+    cpdef set_str_line(self, index_t line_nb, object value):
+        cdef bytes value_b        
+        if value is None :
+            if obi_set_qual_char_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, 0, OBIQual_char_NA) < 0:
+                obi_errno_to_exception(line_nb=line_nb, elt_id=None, error_message="Problem setting a value in a column")
+        else :
+            value_b = tobytes(value)
+            if obi_set_qual_char_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, 0, value_b) < 0:
+                obi_errno_to_exception(line_nb=line_nb, elt_id=None, error_message="Problem setting a value in a column")
+
+ 
+cdef class Column_multi_elts_qual(Column_multi_elts):
+ 
+    cpdef object get_item(self, index_t line_nb, object elt_id):
+        cdef const uint8_t*  value
+        cdef int             value_length
+        cdef object          result
+        cdef int             i
+        if type(elt_id) == int :
+            value = obi_get_qual_int_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, elt_id, &value_length)
+        else :
+            elt_name = tobytes(elt_id)
+            value = obi_get_qual_int_with_elt_name_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, elt_name, &value_length)
+        obi_errno_to_exception(line_nb=line_nb, elt_id=elt_id, error_message="Problem getting a value from a column")
+        if value == OBIQual_int_NA :
+            result = None
+        else :
+            result = []
+            for i in range(value_length) :
+                result.append(<int>value[i])            
+        return result
+
+    
+    cpdef object get_str_item(self, index_t line_nb, object elt_id):
+        cdef char*  value
+        cdef object result
+        if type(elt_id) == int :
+            value = obi_get_qual_char_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, elt_id)
+        else :
+            elt_name = tobytes(elt_id)
+            value = obi_get_qual_char_with_elt_name_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, elt_name)
+        obi_errno_to_exception(line_nb=line_nb, elt_id=elt_id, error_message="Problem getting a value from a column")
+        if value == OBIQual_char_NA :
+            result = None
+        else :
+            result = bytes2str(value)   # TODO return bytes?
+            free(value)
+        return result
+
+    
+    cpdef object get_line(self, index_t line_nb) :
+        cdef const uint8_t*  value
+        cdef int             value_length
+        cdef object          value_in_result
+        cdef dict            result
+        cdef index_t         i
+        cdef int             j
+        cdef bint            all_NA
+        result = {}
+        all_NA = True
+        for i in range(self.nb_elements_per_line) :
+            value = obi_get_qual_int_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, i, &value_length)
+            obi_errno_to_exception(line_nb=line_nb, elt_id=i, error_message="Problem getting a value from a column")
+            if value == OBIQual_int_NA :
+                value_in_result = None
+            else :
+                value_in_result = []
+                for j in range(value_length) :
+                    value_in_result.append(<int>value[j])            
+            result[self.elements_names[i]] = value_in_result
+            if all_NA and (value_in_result is not None) :
+                all_NA = False
+        if all_NA :
+            result = None
+        return result
+
+
+    cpdef object get_str_line(self, index_t line_nb) :
+        cdef char*   value
+        cdef object  value_in_result
+        cdef dict    result
+        cdef index_t i
+        cdef bint    all_NA
+        result = {}
+        all_NA = True
+        for i in range(self.nb_elements_per_line) :
+            value = obi_get_qual_char_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, i)
+            obi_errno_to_exception(line_nb=line_nb, elt_id=i, error_message="Problem getting a value from a column")
+            if value == OBIQual_char_NA :
+                value_in_result = None
+            else :
+                value_in_result = bytes2str(value)
+                free(value)
+            result[self.elements_names[i]] = value_in_result
+            if all_NA and (value_in_result is not None) :
+                all_NA = False
+        if all_NA :
+            result = None
+        return result
+    
+    
+    cpdef set_item(self, index_t line_nb, object elt_id, object value):
+        cdef uint8_t* value_b
+        cdef int      value_length
+        cdef bytes    elt_name
+        
+        if value is None :
+            value_b = OBIQual_int_NA
+            value_length = 0
+        else :
+            value_length = len(value)
+            value_b = <uint8_t*> malloc(value_length * sizeof(uint8_t))
+            for i in range(value_length) :
+                value_b[i] = <uint8_t>value[i]           
+   
+        if type(elt_id) == int :
+            if obi_set_qual_int_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, elt_id, value_b, value_length) < 0 :
+                obi_errno_to_exception(line_nb=line_nb, elt_id=elt_id, error_message="Problem setting a value in a column")
+        else :
+            elt_name = tobytes(elt_id)
+            if obi_set_qual_int_with_elt_name_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, elt_name, value_b, value_length) < 0:
+                obi_errno_to_exception(line_nb=line_nb, elt_id=elt_id, error_message="Problem setting a value in a column")
+        
+        if value is not None :
+            free(value_b)
+    
+
+    cpdef set_str_item(self, index_t line_nb, object elt_id, object value):
+        cdef bytes value_b
+        cdef bytes elt_name
+        
+        if value is None :
+            value_b = OBIQual_char_NA
+        else :
+            value_b = tobytes(value)
+   
+        if type(elt_id) == int :
+            if obi_set_qual_char_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, elt_id, value_b) < 0 :
+                obi_errno_to_exception(line_nb=line_nb, elt_id=elt_id, error_message="Problem setting a value in a column")
+        else :
+            elt_name = tobytes(elt_id)
+            if obi_set_qual_char_with_elt_name_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, elt_name, value_b) < 0:
+                obi_errno_to_exception(line_nb=line_nb, elt_id=elt_id, error_message="Problem setting a value in a column")
+
+
+
+def register_class():
+    register_column_class(OBI_QUAL, False, Column_qual, bytes)    # TODO str? int?
+    register_column_class(OBI_QUAL, True, Column_multi_elts_qual, bytes)    # TODO str? int?
+

python/obitools3/dms/column/typed_column/seq.pxd

@@ -0,0 +1,21 @@
+# #cython: language_level=3
+ 
+
+from ...capi.obitypes cimport index_t
+
+from ..column cimport Column, \
+                      Column_multi_elts
+
+
+cdef class Column_seq(Column) :
+
+    cpdef object get_line(self, index_t line_nb)
+    cpdef set_line(self, index_t line_nb, object value)
+
+
+cdef class Column_multi_elts_seq(Column_multi_elts) :
+
+    cpdef object get_item(self, index_t line_nb, object elt_id)
+    cpdef object get_line(self, index_t line_nb)
+    cpdef set_item(self, index_t line_nb, object elt_id, object value)
+    

python/obitools3/dms/column/typed_column/seq.pyx

@@ -0,0 +1,137 @@
+#cython: language_level=3
+
+
+from ..column cimport register_column_class
+
+from ...view.view cimport View
+
+from obitools3.utils cimport tobytes, \
+                             obi_errno_to_exception
+
+from ...capi.obiview cimport obi_get_seq_with_elt_name_and_col_p_in_view, \
+                             obi_get_seq_with_elt_idx_and_col_p_in_view, \
+                             obi_set_seq_with_elt_name_and_col_p_in_view, \
+                             obi_set_seq_with_elt_idx_and_col_p_in_view
+
+from ...capi.obitypes     cimport OBI_SEQ, OBISeq_NA
+
+from libc.stdlib cimport free
+
+
+cdef class Column_seq(Column):
+
+    @staticmethod
+    def new(View    view,
+            object  column_name,
+            index_t nb_elements_per_line=1,
+            object  elements_names=None,
+            object  comments=b""):
+        
+        return Column.new_column(view, column_name, OBI_SEQ,
+                                 nb_elements_per_line=nb_elements_per_line, 
+                                 elements_names=elements_names, 
+                                 comments=comments)
+
+
+    cpdef object get_line(self, index_t line_nb):
+        cdef char*  value
+        cdef object result
+        value = obi_get_seq_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, 0)
+        obi_errno_to_exception(line_nb=line_nb, elt_id=None, error_message="Problem getting a value from a column")
+        if value == OBISeq_NA :
+            result = None
+        else :   # TODO
+            try:
+                result = <bytes> value
+            finally:
+                free(value)
+        return result
+
+
+    cpdef set_line(self, index_t line_nb, object value):
+        cdef char* value_b
+        cdef bytes value_bytes
+        
+        if value is None :
+            value_b = <char*>OBISeq_NA
+        else :
+            value_bytes = tobytes(value)
+            value_b = <char*>value_bytes
+        
+        if obi_set_seq_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, 0, value_b) < 0:
+            obi_errno_to_exception(line_nb=line_nb, elt_id=None, error_message="Problem setting a value in a column")
+ 
+ 
+cdef class Column_multi_elts_seq(Column_multi_elts):
+ 
+    cpdef object get_item(self, index_t line_nb, object elt_id) :
+        cdef char*  value
+        cdef object result
+        if type(elt_id) == int :
+            value = obi_get_seq_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, elt_id)
+        else :
+            elt_name = tobytes(elt_id)
+            value = obi_get_seq_with_elt_name_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, elt_name)
+        obi_errno_to_exception(line_nb=line_nb, elt_id=elt_id, error_message="Problem getting a value from a column")
+        if value == OBISeq_NA :
+            result = None
+        else :
+            try:
+                result = <bytes> value
+            finally:
+                free(value)
+        return result
+
+
+    cpdef object get_line(self, index_t line_nb) :
+        cdef char*   value
+        cdef object  value_in_result
+        cdef dict    result
+        cdef index_t i
+        cdef bint    all_NA
+        result = {}
+        all_NA = True
+        for i in range(self.nb_elements_per_line) :
+            value = obi_get_seq_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, i)
+            obi_errno_to_exception(line_nb=line_nb, elt_id=i, error_message="Problem getting a value from a column")
+            if value == OBISeq_NA :
+                value_in_result = None
+            else :
+                try:
+                    value_in_result = <bytes> value
+                finally:
+                    free(value)    
+            result[self.elements_names[i]] = value_in_result
+            if all_NA and (value_in_result is not None) :
+                all_NA = False
+        if all_NA :
+            result = None
+        return result
+
+
+    cpdef set_item(self, index_t line_nb, object elt_id, object value):
+        
+        cdef bytes elt_name
+        cdef char* value_b
+        cdef bytes value_bytes
+        
+        if value is None :
+            value_b = <char*>OBISeq_NA
+        else :
+            value_bytes = tobytes(value)
+            value_b = <char*>value_bytes
+        
+        if type(elt_id) == int :
+            if obi_set_seq_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, elt_id, value_b) < 0:
+                obi_errno_to_exception(line_nb=line_nb, elt_id=elt_id, error_message="Problem setting a value in a column")
+        
+        else :
+            elt_name = tobytes(elt_id)
+            if obi_set_seq_with_elt_name_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, elt_name, value_b) < 0:
+                obi_errno_to_exception(line_nb=line_nb, elt_id=elt_id, error_message="Problem setting a value in a column")
+
+
+def register_class():
+    register_column_class(OBI_SEQ, False, Column_seq, bytes)    # TODO str?
+    register_column_class(OBI_SEQ, True, Column_multi_elts_seq, bytes)    # TODO str?
+

python/obitools3/dms/column/typed_column/str.pxd

@@ -0,0 +1,21 @@
+# #cython: language_level=3
+ 
+
+from ...capi.obitypes cimport index_t
+
+from ..column cimport Column, \
+                      Column_multi_elts
+
+
+cdef class Column_str(Column) :
+
+    cpdef object get_line(self, index_t line_nb)
+    cpdef set_line(self, index_t line_nb, object value)
+
+
+cdef class Column_multi_elts_str(Column_multi_elts) :
+
+    cpdef object get_item(self, index_t line_nb, object elt_id)
+    cpdef object get_line(self, index_t line_nb)
+    cpdef set_item(self, index_t line_nb, object elt_id, object value)
+    

python/obitools3/dms/column/typed_column/str.pyx

@@ -0,0 +1,124 @@
+#cython: language_level=3
+
+
+from ..column cimport register_column_class
+
+from ...view.view cimport View
+
+from obitools3.utils cimport tobytes, \
+                             obi_errno_to_exception
+
+from ...capi.obiview cimport obi_get_str_with_elt_name_and_col_p_in_view, \
+                             obi_get_str_with_elt_idx_and_col_p_in_view, \
+                             obi_set_str_with_elt_name_and_col_p_in_view, \
+                             obi_set_str_with_elt_idx_and_col_p_in_view
+
+from ...capi.obitypes     cimport OBI_STR, OBIStr_NA, const_char_p
+
+
+cdef class Column_str(Column):
+
+    @staticmethod
+    def new(View    view,
+            object  column_name,
+            index_t nb_elements_per_line=1,
+            object  elements_names=None,
+            object  comments=b""):
+        
+        return Column.new_column(view, column_name, OBI_STR,
+                                 nb_elements_per_line=nb_elements_per_line, 
+                                 elements_names=elements_names, 
+                                 comments=comments)
+
+
+    cpdef object get_line(self, index_t line_nb):
+        cdef const_char_p value
+        cdef object       result
+        value = obi_get_str_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, 0)
+        obi_errno_to_exception(line_nb=line_nb, elt_id=None, error_message="Problem getting a value from a column")
+        if value == OBIStr_NA :
+            result = None
+        else :
+            result = <bytes> value      # NOTE: value is not freed because the pointer points to a mmapped region in an AVL data file.
+        return result
+
+
+    cpdef set_line(self, index_t line_nb, object value):
+        cdef char* value_b
+        cdef bytes value_bytes
+        
+        if value is None :
+            value_b = <char*>OBIStr_NA
+        else :
+            value_bytes = tobytes(value)
+            value_b = <char*>value_bytes
+
+        if obi_set_str_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, 0, value_b) < 0:
+            obi_errno_to_exception(line_nb=line_nb, elt_id=None, error_message="Problem setting a value in a column")
+ 
+ 
+cdef class Column_multi_elts_str(Column_multi_elts):
+ 
+    cpdef object get_item(self, index_t line_nb, object elt_id) :
+        cdef const_char_p value
+        cdef object       result
+        if type(elt_id) == int :
+            value = obi_get_str_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, elt_id)
+        else :
+            elt_name = tobytes(elt_id)
+            value = obi_get_str_with_elt_name_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, elt_name)
+        obi_errno_to_exception(line_nb=line_nb, elt_id=elt_id, error_message="Problem getting a value from a column")
+        if value == OBIStr_NA :
+            result = None
+        else :
+            result = <bytes> value      # NOTE: value is not freed because the pointer points to a mmapped region in an AVL data file.
+        return result
+
+
+    cpdef object get_line(self, index_t line_nb) :
+        cdef const_char_p value
+        cdef object       value_in_result
+        cdef dict         result
+        cdef index_t      i
+        cdef bint         all_NA
+        result = {}
+        all_NA = True
+        for i in range(self.nb_elements_per_line) :
+            value = obi_get_str_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, i)
+            obi_errno_to_exception(line_nb=line_nb, elt_id=i, error_message="Problem getting a value from a column")
+            if value == OBIStr_NA :
+                value_in_result = None
+            else :
+                value_in_result = <bytes> value    
+            result[self.elements_names[i]] = value_in_result
+            if all_NA and (value_in_result is not None) :
+                all_NA = False
+        if all_NA :
+            result = None
+        return result
+
+
+    cpdef set_item(self, index_t line_nb, object elt_id, object value):
+        cdef bytes elt_name
+        cdef char* value_b
+        cdef bytes value_bytes
+
+        if value is None :
+            value_b = <char*>OBIStr_NA
+        else :
+            value_bytes = tobytes(value)
+            value_b = <char*>value_bytes
+                
+        if type(elt_id) == int :
+            if obi_set_str_with_elt_idx_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, elt_id, <char*>value_b) < 0:
+                obi_errno_to_exception(line_nb=line_nb, elt_id=elt_id, error_message="Problem setting a value in a column")
+        else :
+            elt_name = tobytes(elt_id)
+            if obi_set_str_with_elt_name_and_col_p_in_view(self._view.pointer(), self.pointer(), line_nb, elt_name, <char*>value_b) < 0:
+                obi_errno_to_exception(line_nb=line_nb, elt_id=elt_id, error_message="Problem setting a value in a column")
+
+
+def register_class():
+    register_column_class(OBI_STR, False, Column_str, bytes)    # TODO str?
+    register_column_class(OBI_STR, True, Column_multi_elts_str, bytes)    # TODO str?
+

python/obitools3/dms/dms.cfiles

@@ -0,0 +1,33 @@
+../../../src/bloom.c
+../../../src/char_str_indexer.c
+../../../src/crc64.c
+../../../src/dna_seq_indexer.c
+../../../src/encode.c
+../../../src/hashtable.c
+../../../src/linked_list.c
+../../../src/murmurhash2.c
+../../../src/obi_align.c
+../../../src/obiavl.c
+../../../src/obiblob_indexer.c
+../../../src/obiblob.c
+../../../src/obidms_taxonomy.c
+../../../src/obidms.c
+../../../src/obidmscolumn_blob.c
+../../../src/obidmscolumn_bool.c
+../../../src/obidmscolumn_char.c
+../../../src/obidmscolumn_float.c
+../../../src/obidmscolumn_idx.c
+../../../src/obidmscolumn_int.c
+../../../src/obidmscolumn_qual.c
+../../../src/obidmscolumn_seq.c
+../../../src/obidmscolumn_str.c
+../../../src/obidmscolumn.c
+../../../src/obidmscolumndir.c
+../../../src/obierrno.c
+../../../src/obilittlebigman.c
+../../../src/obitypes.c
+../../../src/obiview.c
+../../../src/sse_banded_LCS_alignment.c
+../../../src/uint8_indexer.c
+../../../src/upperband.c
+../../../src/utils.c

python/obitools3/dms/dms.pxd

@@ -0,0 +1,11 @@
+#cython: language_level=3
+                                
+from .object cimport OBIWrapper
+
+from .capi.obidms cimport OBIDMS_p
+
+
+cdef class DMS(OBIWrapper):
+
+    cdef inline OBIDMS_p pointer(self)
+    cpdef int view_count(self)

python/obitools3/dms/dms.pyx

@@ -0,0 +1,152 @@
+#cython: language_level=3
+
+
+from libc.stdlib  cimport free
+from cpython.list cimport PyList_Size
+
+from .capi.obidms cimport obi_open_dms, \
+                          obi_test_open_dms, \
+                          obi_create_dms, \
+                          obi_close_dms, \
+                          obi_dms_get_full_path
+                                                      
+from .capi.obitypes cimport const_char_p
+                            
+from obitools3.utils cimport bytes2str, \
+                             str2bytes, \
+                             tobytes, \
+                             tostr
+                             
+from .object cimport OBIObjectClosedInstance
+
+from pathlib import Path
+
+from .view import view
+
+
+cdef class DMS(OBIWrapper):    
+    
+    cdef inline OBIDMS_p pointer(self):
+        return <OBIDMS_p>(self._pointer)
+    
+    
+    @staticmethod
+    def new(object dms_name) :
+        cdef OBIDMS_p pointer
+        cdef DMS dms
+        cdef bytes dms_name_b = tobytes(dms_name)
+        pointer = obi_create_dms(<const_char_p> dms_name_b)      
+        if pointer == NULL :
+            raise Exception("Failed creating an OBIDMS")
+        dms = OBIWrapper.new(DMS, pointer)
+        return dms
+
+
+    @staticmethod
+    def open(object dms_name) :
+        cdef OBIDMS_p pointer
+        cdef DMS dms
+        cdef bytes dms_name_b = tobytes(dms_name)
+        pointer = obi_open_dms(<const_char_p> dms_name_b)      
+        if pointer == NULL :
+            raise Exception("Failed opening an OBIDMS")
+        dms = OBIWrapper.new(DMS, pointer)
+        return dms
+   
+
+    @staticmethod
+    def test_open(object dms_name) :
+        cdef OBIDMS_p pointer
+        cdef DMS dms
+        cdef bytes dms_name_b = tobytes(dms_name)
+        pointer = obi_test_open_dms(<const_char_p> dms_name_b)      
+        if pointer == NULL :
+            raise Exception("Failed opening an OBIDMS")
+        dms = OBIWrapper.new(DMS, pointer)
+        return dms
+    
+    
+    def close(self) :
+        '''
+        Closes the DMS instance and free the associated memory
+        
+        The `close` method is automatically called by the object destructor.
+        '''
+        cdef OBIDMS_p pointer = self.pointer()
+        
+        if self.active() :
+            OBIWrapper.close(self)
+            if (obi_close_dms(pointer)) < 0 :
+                raise Exception("Problem closing an OBIDMS")
+
+
+    # name property getter
+    @property
+    def name(self) :
+        '''
+        Returns the name of the DMS instance
+        
+            @rtype: bytes
+        '''
+        return <bytes> self.pointer().dms_name
+
+
+    def keys(self) :
+        
+        cdef const_char_p path = obi_dms_get_full_path(self.pointer(), b"VIEWS")
+        
+        if path == NULL:
+            raise RuntimeError("Cannot retrieve the view database path")
+        
+        p = Path(bytes2str(path))
+        
+        free(path)
+        
+        for v in p.glob("*.obiview") :
+            yield str2bytes(v.stem)
+
+
+    def values(self) :
+        cdef bytes view_name
+        for view_name in self.keys():
+            yield self.get_view(view_name)
+    
+    
+    def items(self) :
+        cdef bytes view_name
+        for view_name in self.keys():
+            yield (view_name, self.get_view(view_name))
+    
+    
+    def __contains__(self, key) :
+        
+        cdef str key_s = tostr(key)
+        
+        cdef const_char_p path = obi_dms_get_full_path(self.pointer(), b"VIEWS")
+        p = Path(bytes2str(path),key_s)
+
+        free(path)
+        
+        return p.with_suffix(".obiview").is_file()
+    
+    
+    cpdef int view_count(self) :
+        return PyList_Size(list(self.keys()))
+    
+    
+    def __len__(self) :
+        return self.view_count()
+    
+    
+    def __getitem__(self, object view_name):
+        return self.get_view(view_name)
+    
+    
+    def __iter__(self) :
+        return self.keys()
+    
+    
+    def get_view(self, object view_name) :
+        return view.View.open(self, view_name)
+    
+    

python/obitools3/dms/obiseq.pxd

@@ -0,0 +1,22 @@
+#cython: language_level=3
+
+from .view.view cimport Line
+
+
+cdef class Seq(dict) :
+
+    cdef str    _id
+    cdef object _seq
+    cdef str    _definition
+    
+    
+cdef class Nuc_Seq(Seq) :
+
+    cdef object _quality   
+    #cpdef object reverse_complement(self)    
+    
+    
+cdef class Nuc_Seq_Stored(Line) :
+
+    cpdef object get_str_quality(self)
+    #cpdef object reverse_complement(self)

python/obitools3/dms/obiseq.pyx

@@ -0,0 +1,130 @@
+#cython: language_level=3
+
+from obitools3.utils cimport bytes2str, str2bytes
+
+from .capi.obiview cimport NUC_SEQUENCE_COLUMN, \
+                           ID_COLUMN, \
+                           DEFINITION_COLUMN, \
+                           QUALITY_COLUMN
+
+
+NUC_SEQUENCE_COLUMN_str = bytes2str(NUC_SEQUENCE_COLUMN)
+ID_COLUMN_str           = bytes2str(ID_COLUMN)
+DEFINITION_COLUMN_str   = bytes2str(DEFINITION_COLUMN)
+QUALITY_COLUMN_str      = bytes2str(QUALITY_COLUMN)
+
+
+cdef class Seq(dict) :
+    def __init__(self, str id, object seq, object definition=None) :
+        self.id = id
+        self.seq = seq
+        if definition is not None :
+            self.definition = definition
+            
+    # sequence id property getter and setter
+    @property
+    def id(self):  # @ReservedAssignment
+        return self._id
+    
+    @id.setter
+    def id(self, str new_id):  # @ReservedAssignment @DuplicatedSignature
+        self._id = new_id
+        self[ID_COLUMN] = new_id
+        
+    # sequence property getter and setter
+    @property
+    def seq(self):
+        return self._seq
+    
+    @seq.setter
+    def seq(self, object new_seq):  # @DuplicatedSignature
+        self._seq = new_seq
+        self["SEQ"] = new_seq   # TODO discuss
+        
+    # sequence definition property getter and setter
+    @property
+    def definition(self):
+        return self._definition
+    
+    @definition.setter
+    def definition(self, object new_definition):  # @DuplicatedSignature
+        self._definition = new_definition
+        self[DEFINITION_COLUMN_str] = new_definition
+        
+        
+cdef class Nuc_Seq(Seq) :
+    
+    # nuc sequence property getter and setter
+    @property
+    def seq(self):
+        return self._seq
+    
+    @seq.setter
+    def seq(self, object new_seq):  # @DuplicatedSignature
+        self._seq = new_seq
+        self[NUC_SEQUENCE_COLUMN_str] = new_seq
+        
+    # sequence quality property getter and setter
+    @property
+    def quality(self):
+        return self._quality
+    
+    @quality.setter
+    def quality(self, object new_quality):  # @DuplicatedSignature
+        self._quality = new_quality
+        self[QUALITY_COLUMN_str] = new_quality
+        
+#    cpdef str reverse_complement(self) :    TODO in C ?
+#        pass
+
+
+cdef class Nuc_Seq_Stored(Line) :
+    
+    # sequence id property getter and setter
+    @property
+    def id(self):               # @ReservedAssignment @DuplicatedSignature
+        return self[ID_COLUMN_str]
+    
+    @id.setter
+    def id(self, str new_id):   # @ReservedAssignment @DuplicatedSignature
+        self[ID_COLUMN_str] = new_id
+        
+    # sequence definition property getter and setter
+    @property
+    def definition(self):
+        return self[DEFINITION_COLUMN_str]
+    
+    @definition.setter
+    def definition(self, str new_def): # @DuplicatedSignature
+        self[DEFINITION_COLUMN_str] = new_def
+        
+    # nuc_seq property getter and setter
+    @property
+    def nuc_seq(self):
+        return self[NUC_SEQUENCE_COLUMN_str]
+    
+    @nuc_seq.setter
+    def nuc_seq(self, object new_seq): # @DuplicatedSignature
+        self[NUC_SEQUENCE_COLUMN_str] = new_seq
+        
+    # quality property getter and setter
+    @property
+    def quality(self):
+        return self[QUALITY_COLUMN_str]
+    
+    @quality.setter
+    def quality(self, object new_qual): # @DuplicatedSignature
+        if (type(new_qual) == list) or (new_qual is None) : # TODO check that quality column exists
+            self[QUALITY_COLUMN_str] = new_qual
+        else :  # Quality is in str form
+            self._view.get_column(QUALITY_COLUMN_str).set_str_line(self._index, new_qual)
+            
+    cpdef object get_str_quality(self) :    # TODO not ideal. Make quality_int and quality_str properties
+        return self._view.get_column(QUALITY_COLUMN_str).get_str_line(self._index)
+    
+#    cpdef str reverse_complement(self) :    TODO in C ?
+#        pass
+
+    # TODO static method to import OBI_Nuc_Seq to OBI_Nuc_Seq_Stored ?
+    
+    

python/obitools3/dms/object.pxd

@@ -0,0 +1,28 @@
+#cython: language_level=3
+
+
+cdef dict __c_cython_mapping__
+
+
+cdef class OBIObject:
+
+    cdef dict _dependent_objects
+   
+    cdef register(self, OBIObject object)
+    cdef unregister(self, OBIObject object)
+    cpdef close(self)
+
+
+cdef class OBIWrapper(OBIObject):
+    
+    cdef void*  _pointer
+    
+    cdef inline size_t cid(self)    
+    cdef inline bint active(self)
+    
+    @staticmethod
+    cdef object new(type constructor, void* pointer)
+
+
+cdef class OBIObjectClosedInstance(Exception):
+    pass

python/obitools3/dms/object.pyx

@@ -0,0 +1,89 @@
+#cython: language_level=3
+
+
+__c_cython_mapping__ = {}
+
+
+cdef class OBIObject:
+
+    def __init__(self, __internalCall__) :
+                
+        if __internalCall__ != 987654 or \
+           type(self) == OBIObject or \
+           type(self) == OBIWrapper or \
+           not isinstance(self, OBIObject) :
+            
+            raise RuntimeError('OBIObject constructor can not be called directly')
+        
+        self._dependent_objects = {}
+
+
+    cdef register(self, OBIObject object):
+        self._dependent_objects[id(object)] = object
+
+        
+    cdef unregister(self, OBIObject object):
+        del self._dependent_objects[id(object)]
+        
+
+    cpdef close(self):
+    
+        cdef OBIObject object
+        cdef list      to_close = list((self._dependent_objects).values())
+        
+        for object in to_close:
+            object.close()
+        
+        assert len(self._dependent_objects.values()) == 0
+
+
+cdef class OBIWrapper(OBIObject) :
+    '''
+    The OBIWrapper class enables to wrap a C object representing a DMS or an element from a DMS.
+    '''
+    
+    cdef inline size_t cid(self) :
+        return <size_t>(self._pointer)
+    
+
+    cdef inline bint active(self) :
+        return self._pointer != NULL
+
+
+    cpdef close(self):
+        if (self._pointer != NULL):
+            OBIObject.close(self)
+            del __c_cython_mapping__[<size_t>self._pointer]
+            self._pointer = NULL
+            
+        assert len(self._dependent_objects.values()) == 0
+
+
+    def __dealloc__(self):
+        '''
+        Destructor of any OBI instance.
+        
+        The destructor automatically calls the `close` method and
+        therefore closes and frees all associated objects and memory.
+        '''
+        self.close()
+
+    
+    @staticmethod
+    cdef object new(type constructor, void* pointer) :
+    
+        cdef OBIWrapper o
+        
+        if (<size_t>pointer in __c_cython_mapping__):
+            #print("Pointer already in cython dict")
+            return __c_cython_mapping__[<size_t>pointer]
+        else:
+            o = constructor(987654)
+            o._pointer = pointer
+            __c_cython_mapping__[<size_t>pointer] = o
+            return o
+
+
+cdef class OBIDeactivatedInstanceError(Exception):
+    pass
+

python/obitools3/dms/taxo/taxo.pxd

@@ -0,0 +1,23 @@
+#cython: language_level=3
+
+from ..capi.obitaxonomy cimport ecotx_t, OBIDMS_taxonomy_p
+
+from ..dms cimport DMS
+
+from ..object cimport OBIWrapper                          
+
+
+cdef class OBI_Taxonomy(OBIWrapper) :
+    cdef str  _name     # TODO keep as bytes?
+    cdef DMS  _dms
+    
+    cdef inline OBIDMS_taxonomy_p pointer(self)
+
+    cpdef get_taxon_by_idx(self, int idx)
+    cpdef write(self, str prefix)
+    cpdef int add_taxon(self, str name, str rank_name, int parent_taxid, int min_taxid=*)
+    cpdef close(self)
+ 
+cdef class OBI_Taxon :
+    cdef ecotx_t*      _pointer
+    cdef OBI_Taxonomy  _tax

python/obitools3/dms/taxo/taxo.pyx

@@ -0,0 +1,196 @@
+#cython: language_level=3
+
+from obitools3.utils cimport str2bytes, bytes2str, tobytes
+
+from ..capi.obitaxonomy cimport obi_read_taxonomy, \
+                                obi_read_taxdump, \
+                                obi_write_taxonomy, \
+                                obi_close_taxonomy, \
+                                obi_taxo_get_taxon_with_taxid, \
+                                obi_taxo_add_local_taxon, \
+                                obi_taxo_add_preferred_name_with_taxon, \
+                                ecotx_t                
+                                           
+from cpython.pycapsule cimport PyCapsule_New, PyCapsule_GetPointer
+
+
+cdef class OBI_Taxonomy(OBIWrapper) :
+    # TODO function to import taxonomy?
+    
+    cdef inline OBIDMS_taxonomy_p pointer(self) :
+        return <OBIDMS_taxonomy_p>(self._pointer)        
+
+
+    @staticmethod
+    def open(DMS dms, str name, bint taxdump=False) :
+        
+        cdef void*        pointer
+        cdef OBI_Taxonomy taxo
+        
+        if taxdump :
+            pointer = <void*>obi_read_taxdump(tobytes(name))
+        else :
+            pointer = <void*>obi_read_taxonomy(dms.pointer(), tobytes(name), True)    # TODO discuss
+        # TODO if not found in DMS, try to import?
+        
+        if pointer == NULL :
+            raise RuntimeError("Error : Cannot read taxonomy %s" 
+                               % name)
+        
+        taxo = OBIWrapper.new(OBI_Taxonomy, pointer)
+
+        dms.register(taxo)
+
+        taxo._dms = dms
+        taxo._name = name
+        
+        return taxo
+
+    
+    def __getitem__(self, object ref):
+        
+        cdef ecotx_t* taxon_p
+        cdef object   taxon_capsule
+        
+        if type(ref) == int :
+            taxon_p = obi_taxo_get_taxon_with_taxid(self.pointer(), ref)
+            if taxon_p == NULL :
+                raise Exception("Taxon not found")
+            taxon_capsule = PyCapsule_New(taxon_p, NULL, NULL)
+            return OBI_Taxon(taxon_capsule, self)
+        else :
+            raise Exception("Not implemented")
+
+
+    cpdef get_taxon_by_idx(self, int idx):
+    
+        cdef ecotx_t* taxa
+        cdef ecotx_t* taxon_p
+        cdef object   taxon_capsule
+ 
+        taxa = self.pointer().taxa.taxon
+        taxon_p = <ecotx_t*> (taxa+idx)
+        taxon_capsule = PyCapsule_New(taxon_p, NULL, NULL)
+        return OBI_Taxon(taxon_capsule, self)
+
+    
+    def __len__(self):
+        return self.pointer().taxa.count
+
+
+    def __iter__(self):
+         
+        cdef ecotx_t* taxa
+        cdef ecotx_t* taxon_p
+        cdef object   taxon_capsule
+        cdef int      t
+ 
+        taxa = self.pointer().taxa.taxon
+ 
+        # Yield each taxid
+        for t in range(self.pointer().taxa.count):
+            taxon_p = <ecotx_t*> (taxa+t)
+            taxon_capsule = PyCapsule_New(taxon_p, NULL, NULL)
+            yield OBI_Taxon(taxon_capsule, self)
+
+
+    cpdef write(self, str prefix) :
+        if obi_write_taxonomy(self._dms.pointer(), self.pointer(), tobytes(prefix)) < 0 :
+            raise Exception("Error writing the taxonomy to binary files")
+    
+    
+    cpdef int add_taxon(self, str name, str rank_name, int parent_taxid, int min_taxid=10000000) :
+        cdef int taxid
+        taxid = obi_taxo_add_local_taxon(self.pointer(), tobytes(name), tobytes(rank_name), parent_taxid, min_taxid)
+        if taxid < 0 :
+            raise Exception("Error adding a new taxon to the taxonomy")
+        else :
+            return taxid
+
+
+    cpdef close(self) :
+        
+        cdef OBIDMS_taxonomy_p pointer = self.pointer()
+        
+        if self.active() :
+            self._dms.unregister(self)
+            OBIWrapper.close(self)      
+            if (obi_close_taxonomy(self.pointer()) < 0) :
+                raise Exception("Problem closing the taxonomy %s" % 
+                                self._name)
+
+
+    # name property getter
+    @property
+    def name(self):
+        return self._name
+
+    
+cdef class OBI_Taxon :    # TODO dict subclass?
+    
+    def __init__(self, object taxon_capsule, OBI_Taxonomy tax) :
+        self._pointer = <ecotx_t*> PyCapsule_GetPointer(taxon_capsule, NULL)
+        if self._pointer == NULL :
+            raise Exception("Error reading a taxon (NULL pointer)")
+        self._tax = tax
+    
+    
+    # To test equality
+    def __richcmp__(self, OBI_Taxon taxon2, int op):
+        return (self.name == taxon2.name) and \
+               (self.taxid == taxon2.taxid) and \
+               (self.rank == taxon2.rank) and \
+               (self.farest == taxon2.farest) and \
+               (self.parent.taxid == taxon2.parent.taxid) and \
+               (self.preferred_name == taxon2.preferred_name)
+        
+        
+    # name property getter
+    @property
+    def name(self):
+        return bytes2str(self._pointer.name)
+
+    # taxid property getter
+    @property
+    def taxid(self):
+        return self._pointer.taxid
+
+    # rank property getter
+    @property
+    def rank(self):
+        return self._pointer.rank
+
+    # farest property getter
+    @property
+    def farest(self):
+        return self._pointer.farest
+
+    # parent property getter
+    @property
+    def parent(self):
+        cdef object parent_capsule
+        parent_capsule = PyCapsule_New(self._pointer.parent, NULL, NULL)
+        return OBI_Taxon(parent_capsule, self._tax)
+
+    # preferred name property getter and setter
+    @property
+    def preferred_name(self):
+        if self._pointer.preferred_name != NULL :
+            return bytes2str(self._pointer.preferred_name)
+
+    @preferred_name.setter
+    def preferred_name(self, str new_preferred_name) :  # @DuplicatedSignature
+        if (obi_taxo_add_preferred_name_with_taxon(self._tax.pointer(), self._pointer, tobytes(new_preferred_name)) < 0) :
+            raise Exception("Error adding a new preferred name to a taxon")
+
+    def __repr__(self):
+        d = {}
+        d['taxid']          = self.taxid
+        d['name']           = self.name
+        d['rank']           = self.rank
+        d['preferred name'] = self.preferred_name
+        d['parent']         = self.parent.taxid
+        d['farest']         = self.farest
+        return str(d)
+    
+    

python/obitools3/dms/view/__init__.py

@@ -0,0 +1,3 @@
+#from .view import View                 # @UnresolvedImport
+#from .view import Line_selection       # @UnresolvedImport
+#from .view import Line                 # @UnresolvedImport

python/obitools3/dms/view/typed_view/view_NUC_SEQS.pxd

@@ -0,0 +1,8 @@
+#cython: language_level=3
+
+
+from ..view cimport View
+
+
+cdef class View_NUC_SEQS(View):
+    pass

python/obitools3/dms/view/typed_view/view_NUC_SEQS.pyx

@@ -0,0 +1,79 @@
+#cython: language_level=3
+
+from obitools3.dms.capi.obiview cimport obi_new_view_nuc_seqs, \
+                             obi_new_view_nuc_seqs_cloned_from_name, \
+                             VIEW_TYPE_NUC_SEQS
+
+from ..view cimport register_view_class
+
+from obitools3.dms.obiseq cimport Nuc_Seq, Nuc_Seq_Stored
+
+from obitools3.dms.dms cimport DMS
+
+from obitools3.dms.capi.obitypes cimport index_t
+
+from obitools3.utils cimport tobytes, bytes2str
+
+from obitools3.dms.capi.obidms cimport OBIDMS_p
+
+from obitools3.dms.object cimport OBIWrapper                          
+
+
+
+cdef class View_NUC_SEQS(View):
+    
+    @staticmethod
+    def new(DMS dms, 
+            object view_name, 
+            object comments=None,
+            bint quality=False):
+
+        cdef bytes view_name_b = tobytes(view_name)
+        cdef bytes comments_b
+        cdef str message
+        cdef void* pointer
+
+        cdef View_NUC_SEQS view
+
+        if comments is not None:
+            comments_b = tobytes(comments)
+        else:
+            comments_b = b''
+
+        pointer = <void*>obi_new_view_nuc_seqs(<OBIDMS_p>dms._pointer, 
+                                               view_name_b, 
+                                               NULL, 
+                                               NULL, 
+                                               comments_b,
+                                               quality)
+        
+        if pointer == NULL :
+            message = "Error : Cannot create view %s" % bytes2str(view_name_b)
+            raise RuntimeError(message)
+        
+        view = OBIWrapper.new(View_NUC_SEQS, pointer)
+        view._dms = dms
+        dms.register(view)
+
+        return view
+
+
+# TODO
+    def __getitem__(self, object item) :
+        if type(item) == int :
+            return Nuc_Seq_Stored(self, item)
+        else :  # TODO assume str or bytes for optimization?
+            return self.get_column(item)    # TODO hyper lent dans la pratique
+    
+    
+    def __setitem__(self, index_t line_idx, Nuc_Seq sequence_obj) :
+        for key in sequence_obj :
+            self[line_idx][key] = sequence_obj[key]
+
+
+# TODO make properties for id, seq, def columns etc
+
+
+def register_class() :
+    register_view_class(VIEW_TYPE_NUC_SEQS, View_NUC_SEQS)
+

python/obitools3/dms/view/view.pxd

@@ -0,0 +1,65 @@
+#cython: language_level=3
+
+
+from ..capi.obiview cimport Obiview_p
+
+from ..capi.obitypes cimport index_t, obitype_t
+
+from ..object cimport OBIWrapper                          
+
+from ..dms cimport DMS
+
+from ..column.column cimport Column
+
+
+cdef dict __OBIDMS_VIEW_CLASS__
+
+
+cdef class View(OBIWrapper):
+    
+    cdef DMS _dms
+    
+    cdef inline Obiview_p pointer(self)   
+     
+    cpdef delete_column(self, 
+                        object column_name)
+    
+    cpdef rename_column(self, 
+                        object current_name, 
+                        object new_name)
+    
+    cpdef Column rewrite_column_with_diff_attributes(self,
+                                                     object    column_name,
+                                                     obitype_t new_data_type=*,
+                                                     index_t   new_nb_elements_per_line=*,
+                                                     list      new_elements_names=*)
+
+    cpdef Line_selection new_selection(self,
+                                       list lines=*)
+
+    @staticmethod
+    cdef type get_view_class(bytes view_type)
+
+
+cdef class Line_selection(list):
+    
+    cdef View  _view
+    cdef bytes _view_name
+    
+    cdef index_t* __build_binary_list__(self)
+
+    cpdef View materialize(self,
+                           object view_name,
+                           object comments=*)
+
+
+cdef class Line :
+
+    cdef index_t _index
+    cdef View    _view
+    
+
+cdef register_view_class(bytes view_type_name,
+                         type view_class)
+
+cdef register_all_view_classes()

python/obitools3/dms/view/view.pyx

@@ -0,0 +1,556 @@
+#cython: language_level=3
+
+
+cdef dict __VIEW_CLASS__= {}
+
+
+from libc.stdlib  cimport malloc
+
+from ..capi.obiview cimport Alias_column_pair_p, \
+                            obi_new_view, \
+                            obi_open_view, \
+                            obi_clone_view, \
+                            obi_save_and_close_view, \
+                            obi_view_get_pointer_on_column_in_view, \
+                            obi_view_delete_column, \
+                            obi_view_create_column_alias
+                           
+from ..capi.obidmscolumn cimport OBIDMS_column_p
+from ..capi.obidms cimport OBIDMS_p
+
+from obitools3.utils cimport tobytes, \
+                             str2bytes, \
+                             bytes2str
+   
+from ..object cimport OBIObjectClosedInstance
+
+from obitools3.dms.view import typed_view
+
+from ..capi.obitypes cimport is_a_DNA_seq, \
+                             OBI_VOID, \
+                             OBI_BOOL, \
+                             OBI_CHAR, \
+                             OBI_FLOAT, \
+                             OBI_INT, \
+                             OBI_QUAL, \
+                             OBI_SEQ, \
+                             OBI_STR
+
+import importlib
+import inspect
+import pkgutil
+  
+
+cdef class View(OBIWrapper) :
+
+    cdef inline Obiview_p pointer(self) :
+        return <Obiview_p>(self._pointer)        
+    
+    @staticmethod
+    cdef type get_view_class(bytes view_type):
+        global __VIEW_CLASS__
+        return __VIEW_CLASS__.get(view_type, View) 
+     
+    
+    @staticmethod
+    def new(DMS dms, 
+            object view_name, 
+            object comments=None):
+
+        cdef bytes view_name_b = tobytes(view_name)
+        cdef bytes comments_b
+        cdef str message
+        cdef void* pointer
+
+        cdef View view  # @DuplicatedSignature
+
+        if comments is not None:
+            comments_b  = tobytes(comments)
+        else:
+            comments_b  = b''
+
+        pointer = <void*>obi_new_view(<OBIDMS_p>dms._pointer, 
+                                      view_name_b, 
+                                      NULL, 
+                                      NULL, 
+                                      comments_b)
+        
+        if pointer == NULL :
+            message = "Error : Cannot create view %s" % bytes2str(view_name_b)
+            raise RuntimeError(message)
+        
+        view = OBIWrapper.new(View, pointer)
+        view._dms = dms
+        dms.register(view)
+
+        return view
+    
+          
+    def clone(self, 
+              object view_name, 
+              object comments=None):
+                          
+                          
+        cdef bytes view_name_b = tobytes(view_name)
+        cdef bytes comments_b
+        cdef void* pointer
+        cdef View view
+        
+        if not self.active() :
+            raise OBIObjectClosedInstance()
+
+        if comments is not None:
+            comments_b  = tobytes(comments)
+        else:
+            comments_b  = b''
+                                
+        pointer = <void*> obi_clone_view(self._dms.pointer(), 
+                                         self.pointer(), 
+                                         view_name_b, 
+                                         NULL, 
+                                         comments_b)
+        
+        if pointer == NULL :
+            raise RuntimeError("Error : Cannot clone view %s into view %s" 
+                               % (str(self.name),
+                                  bytes2str(view_name_b))
+                              )
+            
+        view = OBIWrapper.new(type(self), pointer)
+        view._dms = self._dms
+        self._dms.register(view)
+                                
+        return view
+    
+        
+    @staticmethod
+    def open(DMS dms,               # @ReservedAssignment
+             object view_name):
+
+        cdef bytes view_name_b = tobytes(view_name)
+        cdef void* pointer
+        cdef View  view
+        cdef type  view_class
+        
+        pointer = <void*> obi_open_view(dms.pointer(), 
+                                        view_name_b)                
+                
+        if pointer == NULL :
+            raise RuntimeError("Error : Cannot open view %s" % bytes2str(view_name_b))
+        
+        view_class = View.get_view_class((<Obiview_p>pointer).infos.view_type)
+        view = OBIWrapper.new(view_class, pointer)
+    
+        view._dms = dms
+        dms.register(view)
+                
+        return view
+        
+    
+    cpdef close(self):
+        cdef Obiview_p pointer = self.pointer()
+        
+        if self.active() :
+            self._dms.unregister(self)
+            OBIWrapper.close(self)      
+            if obi_save_and_close_view(pointer) < 0 :
+                raise Exception("Problem closing view %s" % 
+                                bytes2str(self.name))
+    
+    
+    def __repr__(self) :
+        # TODO check everywhere
+        if not self.active() :
+            raise OBIObjectClosedInstance()
+         
+        cdef str s = "{name:s}\n{comments:s}\n{line_count:d} lines\n".format(name       = str(self.name),
+                                                                             comments   = str(self.comments),
+                                                                             line_count = self.line_count)
+ 
+        for column_name in self.keys() :
+            s = s + repr(self[column_name]) + '\n'
+         
+        return s
+    
+     
+    def keys(self):
+        cdef int i
+        cdef Obiview_p pointer = self.pointer()
+        cdef int nb_column = pointer.infos.column_count
+        cdef Alias_column_pair_p column_p =  pointer.infos.column_references
+        
+        if not self.active() :
+            raise OBIObjectClosedInstance()
+
+        for i in range(nb_column) :
+            col_alias = bytes2str(pointer.infos.column_references[i].alias)
+            yield col_alias
+        
+       
+    def get_column(self,
+                   object column_name):
+
+        if not self.active() :
+            raise OBIObjectClosedInstance()
+                
+        return Column.open(self, column_name)
+    
+    
+    cpdef delete_column(self, 
+                        object column_name) :
+
+        cdef bytes column_name_b = tobytes(column_name)
+
+        if not self.active() :
+            raise OBIObjectClosedInstance()
+ 
+        # Close the cython instance first
+        col = self[column_name]
+        col.close()
+        
+        # Remove the column from the view which closes the C structure
+        if obi_view_delete_column(self.pointer(), column_name_b) < 0 :
+            raise Exception("Problem deleting column %s from a view",
+                            bytes2str(column_name_b))
+
+
+    cpdef rename_column(self, 
+                        object current_name, 
+                        object new_name):
+                        
+        cdef Column column
+        cdef bytes current_name_b = tobytes(current_name)
+        cdef bytes new_name_b     = tobytes(new_name)
+        
+        if not self.active() :
+            raise OBIObjectClosedInstance()
+        
+        if (obi_view_create_column_alias(self.pointer(), 
+                                         tobytes(current_name_b), 
+                                         tobytes(new_name_b)) < 0) :
+            raise Exception("Problem in renaming column %s to %s" % (
+                                bytes2str(current_name_b),
+                                bytes2str(new_name_b)))
+
+
+    # TODO warning, not multithreading compliant
+    cpdef Column rewrite_column_with_diff_attributes(self,
+                                                     object    column_name,
+                                                     obitype_t new_data_type=<obitype_t>OBI_VOID,
+                                                     index_t   new_nb_elements_per_line=0,
+                                                     list      new_elements_names=None) :
+        
+        cdef Column  old_column                                             
+        cdef Column  new_column
+        cdef index_t length = len(self)
+        
+        old_column = self.get_column(column_name)
+
+        if new_data_type == 0 :
+            new_data_type = old_column.data_type
+        
+        if new_nb_elements_per_line == 0 :
+            new_nb_elements_per_line = old_column.nb_elements_per_line
+        
+        if new_elements_names is None :
+            new_elements_names = old_column.elements_names
+
+        new_column = Column.new_column(self, old_column.pointer().header.name, new_data_type, 
+                                       nb_elements_per_line=new_nb_elements_per_line, elements_names=new_elements_names, 
+                                       comments=old_column.comments, alias=tobytes(column_name)+tobytes('___new___'))
+        
+        for i in range(length) :
+            new_column[i] = old_column[i]
+
+        # Remove old column from view
+        self.delete_column(column_name)
+
+        # Rename new
+        new_column.name = column_name
+        
+        return new_column
+
+
+    cpdef Line_selection new_selection(self,list lines=None):
+        return Line_selection(self, lines)
+
+
+    def __iter__(self):
+        # Iteration on each line of all columns
+        
+        # Declarations
+        cdef index_t line_nb
+        cdef Line    line
+        
+        # Yield each line    
+        for line_nb in range(self.line_count) :
+            line = self[line_nb]
+            yield line
+
+
+    def __getitem__(self, object item) :
+        if type(item) == int :
+            return Line(self, item)
+        else :  # TODO assume str or bytes for optimization?
+            return self.get_column(item)    # TODO hyper lent dans la pratique            
+
+
+    def __contains__(self, str column_name):
+        return (column_name in self.keys())
+    
+    
+    def __len__(self):
+        return(self.line_count)
+
+
+    def __str__(self) :
+        cdef Line line
+        cdef str to_print
+        to_print = ""
+        for line in self :
+            to_print = to_print + str(line) + "\n"
+        return to_print
+    
+    
+    @property
+    def dms(self):
+        return self._dms
+    
+    
+    # line_count property getter
+    @property
+    def line_count(self):
+        return self.pointer().infos.line_count
+
+
+    # name property getter
+    @property
+    def name(self):
+        return <bytes> self.pointer().infos.name
+
+
+    # view type property getter
+    @property
+    def type(self):  # @ReservedAssignment
+        return <bytes> self.pointer().infos.view_type
+
+
+    # comments property getter
+    @property
+    def comments(self):
+        return <bytes> self.pointer().infos.comments
+    # TODO setter that concatenates new comments?
+
+
+cdef class Line :
+
+    def __init__(self, View view, index_t line_nb) :
+        self._index = line_nb
+        self._view = view
+    
+    
+    def __getitem__(self, str column_name) :
+        return (self._view)[column_name][self._index]
+
+
+    def __setitem__(self, str column_name, object value):   # TODO discuss
+        # TODO detect multiple elements (dict type)? put somewhere else? but more risky (in get)
+        # TODO OBI_QUAL ?
+        cdef type       value_type
+        cdef obitype_t  value_obitype
+        cdef bytes      value_b
+                 
+        if column_name not in self._view :
+            if value == None :
+                raise Exception("Trying to create a column from a None value (can't guess type)")
+            value_type = type(value)
+            if value_type == int :
+                value_obitype = OBI_INT
+            elif value_type == float :
+                value_obitype = OBI_FLOAT
+            elif value_type == bool :
+                value_obitype = OBI_BOOL           
+            elif value_type == str or value_type == bytes :
+                if value_type == str :
+                    value_b = str2bytes(value)
+                else :
+                    value_b = value
+                if is_a_DNA_seq(value_b) :
+                    value_obitype = OBI_SEQ
+                elif len(value) == 1 :
+                    value_obitype = OBI_CHAR
+                elif (len(value) > 1) :
+                    value_obitype = OBI_STR
+            else :
+                raise Exception("Could not guess the type of a value to create a new column")
+            
+            Column.new_column(self._view, column_name, value_obitype)
+ 
+        (self._view)[column_name][self._index] = value
+ 
+ 
+    def __iter__(self):
+        for column_name in (self._view).keys() :
+            yield self[column_name]
+ 
+ 
+    def __contains__(self, str column_name):
+        return (column_name in self._view.keys())
+
+
+    def __repr__(self):
+        cdef dict line
+        cdef str column_name
+        line = {}
+        for column_name in self._view.keys() :
+            line[column_name] = self[column_name]
+        return str(line)
+
+
+#     cpdef dict get_view_infos(self, str view_name) :
+#         
+#         cdef Obiview_infos_p      view_infos_p
+#         cdef dict                 view_infos_d
+#         cdef Alias_column_pair_p  column_refs
+#         cdef int                  i, j
+#         cdef str                  column_name
+# 
+#         view_infos_p = obi_view_map_file(self._pointer, 
+#                                          tobytes(view_name))
+#         view_infos_d = {}
+#         view_infos_d["name"] = bytes2str(view_infos_p.name)
+#         view_infos_d["comments"] = bytes2str(view_infos_p.comments)
+#         view_infos_d["view_type"] = bytes2str(view_infos_p.view_type)
+#         view_infos_d["column_count"] = <int> view_infos_p.column_count
+#         view_infos_d["line_count"] = <int> view_infos_p.line_count
+#         view_infos_d["created_from"] = bytes2str(view_infos_p.created_from)
+#         view_infos_d["creation_date"] = bytes2str(obi_format_date(view_infos_p.creation_date))
+#         if (view_infos_p.all_lines) :
+#             view_infos_d["line_selection"] = None
+#         else :
+#             view_infos_d["line_selection"] = {}
+#             view_infos_d["line_selection"]["column_name"] = bytes2str((view_infos_p.line_selection).column_name)
+#             view_infos_d["line_selection"]["version"] = <int> (view_infos_p.line_selection).version
+#         view_infos_d["column_references"] = {}
+#         column_references = view_infos_p.column_references
+#         for j in range(view_infos_d["column_count"]) :
+#             column_name = bytes2str((column_references[j]).alias)
+#             view_infos_d["column_references"][column_name] = {}
+#             view_infos_d["column_references"][column_name]["original_name"] = bytes2str((column_references[j]).column_refs.column_name)
+#             view_infos_d["column_references"][column_name]["version"] = (column_references[j]).column_refs.version
+#         
+#         obi_view_unmap_file(self._pointer, view_infos_p)
+# 
+#         return view_infos_d
+
+
+
+cdef class Line_selection(list):
+    
+    def __init__(self, View view, lines=None) :
+        if view._pointer == NULL:
+            raise Exception("Error: trying to create a line selection with an invalidated view")
+        self._view = view
+        self._view_name = view.name
+        
+        if lines is not None:
+            self.extend(lines)
+        
+        
+    def extend(self, iterable):
+        cdef index_t i
+        cdef index_t max_i = self._view.line_count
+        
+        for i in iterable:  # TODO this is already checked in C
+            if i > max_i:
+                raise RuntimeError("Error: trying to select line %d beyond the line count %d of view %s" %
+                                   (i,
+                                    max_i,
+                                    self._view_name)
+                                   )
+            list.append(self,i)
+        
+        
+    def append(self, index_t idx) :
+        if idx >= self._view.line_count :
+            raise IndexError("Error: trying to select line %d beyond the line count %d of view %s" %
+                               (idx,
+                                self._view.line_count,
+                                bytes2str(self.name))
+                               )
+        list.append(self,idx)
+        
+        
+    cdef index_t* __build_binary_list__(self):
+        cdef index_t*        line_selection_p = NULL
+        cdef int             i
+        cdef size_t          l_selection = len(self)
+                
+        line_selection_p = <index_t*> malloc((l_selection + 1) * sizeof(index_t))   # +1 for the -1 flagging the end of the array
+        for i in range(l_selection) :
+                line_selection_p[i] = self[i]
+        line_selection_p[l_selection] = -1      # flagging the end of the array
+
+        return line_selection_p
+        
+        
+    cpdef View materialize(self,
+                           object view_name,
+                           object comments=""):
+
+        cdef bytes view_name_b = tobytes(view_name)
+        cdef bytes comments_b
+        cdef Obiview_p pointer
+        cdef View view
+                
+        if not self._view.active() :
+            raise OBIObjectClosedInstance()
+
+        if comments is not None:
+            comments_b  = tobytes(comments)
+        else:
+            comments_b  = b''
+    
+        pointer = obi_clone_view(self._view._dms.pointer(), 
+                                 self._view.pointer(), 
+                                 view_name_b, 
+                                 self.__build_binary_list__(), 
+                                 comments_b)
+
+        if pointer == NULL :
+            raise RuntimeError("Error : Cannot clone view %s into view %s with new line selection" 
+                               % (str(self._view.name),
+                                  bytes2str(view_name_b))
+                              )
+        
+        view = OBIWrapper.new(type(self._view), pointer)
+        view._dms = self._view._dms
+        view._dms.register(view)
+                
+        return view
+
+
+#############################################################
+
+
+cdef register_view_class(bytes view_type_name,
+                         type view_class):
+    '''
+    Each subclass of `dms.view` needs to be registered after its declaration
+    '''
+    global __VIEW_CLASS__
+
+    assert issubclass(view_class, View)
+    
+    __VIEW_CLASS__[view_type_name] = view_class
+
+
+cdef register_all_view_classes() :
+    
+    x = list(pkgutil.walk_packages(typed_view.__path__, prefix="obitools3.dms.view.typed_view."))
+    all_modules = [importlib.import_module(a[1]) for a in x]
+    for mod in all_modules :
+        getattr(mod, 'register_class')()
+
+
+register_all_view_classes()

python/obitools3/files/linebuffer.pxd

@@ -0,0 +1,6 @@
+#cython: language_level=3
+
+
+cdef class LineBuffer:
+    cdef object fileobj
+    cdef int    size

python/obitools3/files/linebuffer.pyx

@@ -0,0 +1,23 @@
+#cython: language_level=3
+
+'''
+Created on 30 mars 2016
+
+@author: coissac
+'''
+
+cdef class LineBuffer:
+
+    def __init__(self,object fileobj,int size=100000000):
+        self.fileobj=fileobj 
+        self.size=size
+    
+    def __iter__(self):
+        cdef list buff = self.fileobj.readlines(self.size)
+        cdef str  l
+        
+        while buff:
+            for l in buff:
+                yield l
+            buff = self.fileobj.readlines(self.size)
+    

python/obitools3/files/uncompress.pxd

@@ -0,0 +1,18 @@
+#cython: language_level=3
+
+cdef class MagicKeyFile:
+    cdef object stream
+    cdef str    stream_mode
+    cdef object binary
+    cdef bytes  key 
+    cdef int    keylength 
+    cdef int    pos
+    
+    cpdef bytes read(self,int size=?)
+    cpdef bytes read1(self,int size=?)
+    cpdef int tell(self)
+
+    
+cdef class CompressedFile:
+    cdef object accessor
+    

python/obitools3/files/uncompress.pyx

@@ -0,0 +1,117 @@
+#cython: language_level=3
+
+'''
+Created on 28 mars 2016
+
+@author: coissac
+'''
+
+import zipfile
+import bz2
+import gzip
+
+import io
+
+cdef class MagicKeyFile:
+    def __init__(self,stream,length=2):
+        
+        binary=stream
+        self.stream = stream
+        self.stream_mode = None 
+        if hasattr(stream, "mode"):
+            self.stream_mode = stream.mode
+            if (not 'b' in stream.mode and 
+                hasattr(stream, "buffer") and 
+                'b' in stream.buffer.mode):
+                binary=stream.buffer
+
+        if (self.stream_mode is None and 
+            not (hasattr(stream, 'headers') and 
+                 hasattr(stream.headers, "keys") and
+                 'Content-type' in stream.headers)):
+            raise TypeError("stream does not present the good interface")
+        
+        self.binary=binary
+        self.key=binary.read(length)
+        self.keylength=length
+        self.pos=0
+        
+    cpdef bytes read(self,int size=-1):
+        cdef bytes r
+        
+        if self.pos < self.keylength:
+            if size > (self.keylength - self.pos):
+                size = size - self.keylength + self.pos
+                r = self.key[self.pos:] + self.binary.read(size)
+                self.pos=self.keylength + 1
+            elif size >=0 :
+                r = self.key[self.pos:(self.pos+size)]
+                self.pos+=size
+            else:
+                r = self.key[self.pos:] + self.binary.read(size)
+                self.pos=self.keylength + 1
+        else:
+            r = self.binary.read(size)
+            
+        return r
+
+    cpdef bytes read1(self,int size=-1):
+        return self.read(size)
+
+    cpdef int tell(self):
+        cdef int p
+        
+        if self.pos < self.keylength:
+            p = self.pos 
+        else:
+            p = self.binary.tell()
+        
+        return p
+    
+    def __getattr__(self,name):
+        return getattr(self.binary, name)
+        
+
+    
+cdef class CompressedFile:
+    
+    
+    def __init__(self,stream):
+        cdef int            keylength
+        cdef MagicKeyFile   magic
+        cdef str            compressor
+        cdef bytes          k 
+        cdef object         c
+        
+        cdef dict compress = { 'zip' : (b'\x50\x4b\x03\x04',zipfile.ZipFile),
+                               'bz2' : (b'\x42\x5a\x68',bz2.BZ2File),
+                               'gz'  : (b'\x1f\x8b\x08',gzip.open)
+                   }
+        
+        keylength = max([len(x[0]) for x in compress.values()])
+        magic=MagicKeyFile(stream,keylength)
+        
+        self.accessor = None
+        
+        for compressor in compress:
+            k,c = compress[compressor]
+            if magic.key.startswith(k):
+                self.accessor = c(magic)
+        
+        if self.accessor is None:
+            self.accessor = magic
+        
+        if ((hasattr(stream, 'headers') and 
+             hasattr(stream.headers, "keys") and
+             'Content-type' in stream.headers and
+             stream.headers['Content-type'].startswith('text/')) or
+            'b' not in magic.stream_mode):
+            self.accessor = io.TextIOWrapper(self.accessor)
+        
+    
+    def __getattr__(self,name):
+        return getattr(self.accessor, name)
+    
+    def __iter__(self):
+        for x in self.accessor:
+            yield x 

python/obitools3/files/universalopener.pxd

@@ -0,0 +1,5 @@
+#cython: language_level=3
+
+from .uncompress cimport CompressedFile
+
+cpdef CompressedFile uopen(str name, mode=?)

python/obitools3/files/universalopener.pyx

@@ -0,0 +1,23 @@
+#cython: language_level=3
+
+'''
+Created on 25 mars 2016
+
+@author: coissac
+'''
+
+from urllib.request import urlopen
+
+
+cpdef CompressedFile uopen(str name, mode='r'):
+    cdef CompressedFile c
+    
+    try:
+        f = urlopen(name)
+    except:
+        f = open(name,mode) 
+        
+    c = CompressedFile(f)
+    
+    return c
+    

python/obitools3/format/fasta.pxd

@@ -0,0 +1,10 @@
+from ..utils cimport bytes2str
+from .header cimport HeaderFormat
+from cython.view cimport array as cvarray
+
+cdef class FastaFormat:
+
+    cdef HeaderFormat headerFormater
+    
+    cdef size_t  sequenceBufferLength
+    cdef char*   sequenceBuffer

python/obitools3/format/fasta.pyx

@@ -0,0 +1,30 @@
+cimport cython
+
+cdef class FastaFormat:
+	
+	def __init__(self, list tags=[], bint printNAKeys=False):
+		self.headerFormater = HeaderFormat(True,
+										   tags,
+										   printNAKeys)
+		
+	@cython.boundscheck(False)	
+	def __call__(self, dict data):
+		cdef bytes brawseq      = data['sequence']
+		cdef size_t lseq        = len(brawseq)
+		cdef size_t k=0
+		cdef list   lines       = []
+		
+		for k in range(0,lseq,60):
+			lines.append(brawseq[k:(k+60)])
+			
+		brawseq = b'\n'.join(lines)
+		
+		return "%s\n%s" % (self.headerFormater(data),bytes2str(brawseq))
+					
+		
+				
+			
+		
+		
+		
+		

python/obitools3/format/header.pxd

@@ -0,0 +1,7 @@
+cdef class HeaderFormat:
+
+    cdef str    start
+    cdef set    tags
+    cdef bint   printNaKeys
+    cdef size_t headerBufferLength
+    

python/obitools3/format/header.pyx

@@ -0,0 +1,60 @@
+
+cdef class HeaderFormat:
+    
+    def __init__(self, bint fastaHeader=True, list tags=[], bint printNAKeys=False):
+        '''
+        
+            @param fastaHeader: 
+            @type  fastaHeader: `bool`
+            
+            @param tags:
+            @type  tags: `list` of `bytes`
+            
+            @param printNAKeys: 
+            @type  printNAKeys: `bool` 
+        '''
+        
+        self.tags           = set(tags)
+        self.printNaKeys    = printNAKeys
+        
+        if fastaHeader:
+            self.start=">"
+        else:
+            self.start="@"
+        
+        self.headerBufferLength = 1000
+        #self.headerBuffer = []
+        
+    def __call__(self, dict data):
+        cdef str header
+        cdef dict tags = data['tags']
+        cdef set  ktags
+        cdef list lines = [""]
+        cdef str tagline
+        
+        if self.tags is not None and self.tags:
+            ktags = self.tags
+        else:
+            ktags = set(tags.keys())
+            
+        for k in ktags:
+            if k in tags:
+                value = tags[k]
+                if value is not None or self.printNaKeys:
+                    lines.append("%s=%s;" % (k,tags[k]))
+                    
+        if len(lines) > 1:
+            tagline=" ".join(lines)
+        else:
+            tagline=""
+            
+        if data['definition'] is not None:
+            header = "%s%s%s %s" % (self.start,data['id'],
+                                    tagline,
+                                    data['definition'])
+        else:
+            header = "%s%s%s" % (self.start,data['id'],
+                                 tagline)
+            
+        return header
+        

python/obitools3/obidms/_obidms.cfiles

@@ -1,28 +0,0 @@
-../../../src/obidms.h
-../../../src/obidms.c
-../../../src/obiview.h
-../../../src/obiview.c
-../../../src/obidmscolumn.h
-../../../src/obidmscolumn.c
-../../../src/obidmscolumndir.h
-../../../src/obidmscolumndir.c
-../../../src/obierrno.h
-../../../src/obierrno.c
-../../../src/obilittlebigman.h
-../../../src/obilittlebigman.c
-../../../src/obitypes.h
-../../../src/obitypes.c
-../../../src/private_at_functions.h
-../../../src/private_at_functions.c
-../../../src/obiavl.h
-../../../src/obiavl.c
-../../../src/encode.h
-../../../src/encode.c
-../../../src/obidmscolumn_idx.h
-../../../src/obidmscolumn_idx.c
-../../../src/obidms_taxonomy.c
-../../../src/obidms_taxonomy.h
-../../../src/bloom.c
-../../../src/bloom.h
-../../../src/MurmurHash2.c
-../../../src/murmurhash2.h

python/obitools3/obidms/_obidms.pxd

@@ -1,94 +0,0 @@
-#cython: language_level=3
-
-from .capi.obidms       cimport OBIDMS_p
-from .capi.obidmscolumn cimport OBIDMS_column_p
-from .capi.obiview      cimport Obiview_p
-from .capi.obitypes     cimport obiversion_t, OBIType_t, index_t
-from ._obitaxo          cimport OBI_Taxonomy
-
-cdef class OBIDMS_column:
-
-    cdef  OBIDMS_column_p*   pointer
-    cdef  OBIDMS             dms
-    cdef  Obiview_p          view
-    cdef  str                data_type
-    cdef  str                dms_name
-    cdef  str                column_name
-    cdef  index_t            nb_elements_per_line
-    cdef  list               elements_names
-        
-    cpdef         update_pointer(self)
-    cpdef list    get_elements_names(self)
-    cpdef str     get_data_type(self)
-    cpdef index_t get_nb_lines_used(self)
-    cpdef str     get_creation_date(self)
-    cpdef str     get_comments(self)
-    cpdef         close(self)
-    @staticmethod
-    cdef  object  get_subclass_type(OBIDMS_column_p column_p)
-
-
-cdef class OBIDMS_column_multi_elts(OBIDMS_column):
-    
-    cpdef set_line(self, index_t line_nb, dict values)
-
-
-cdef class OBIDMS_column_line:
-
-    cdef OBIDMS_column column
-    cdef index_t index
-
-
-cdef class OBIView:
-    
-    cdef Obiview_p pointer
-    cdef str       name
-    cdef str       comments
-    cdef dict      columns
-    cdef dict      columns_pp    # TODO this dict might be unnecessary
-    cdef OBIDMS    dms
-    
-    cpdef delete_column(self, str column_name)
-    cpdef add_column(self, 
-                     str column_name, 
-                     obiversion_t version_number=*, 
-                     str type=*,
-                     index_t nb_lines=*, 
-                     index_t nb_elements_per_line=*,
-                     list elements_names=*,
-                     str avl_name=*,
-                     str comments=*,
-                     bint create=*
-                    )
-    cpdef select_line(self, index_t line_nb)
-    cpdef select_lines(self, list line_selection)
-    cpdef save_and_close(self)
-
-
-cdef class OBIView_NUC_SEQS(OBIView):
-
-    cdef OBIDMS_column ids
-    cdef OBIDMS_column sequences
-    cdef OBIDMS_column descriptions
-    
-    cpdef delete_column(self, str column_name)
-    
-
-cdef class OBIView_line :
-
-    cdef index_t index
-    cdef OBIView view
-
-
-cdef class OBIDMS:
-
-    cdef OBIDMS_p pointer
-    cdef str dms_name
-    
-    cpdef close(self)
-    cpdef OBI_Taxonomy open_taxonomy(self, str taxo_name)
-    cpdef OBIView open_view(self, str view_name)
-    cpdef OBIView new_view(self, str view_name, object view_to_clone=*, list line_selection=*, str view_type=*, str comments=*)
-    cpdef dict read_view_infos(self, str view_name)
-    cpdef dict read_views(self)
-

python/obitools3/obidms/_obidms.pyx

@@ -1,740 +0,0 @@
-#cython: language_level=3
-
-from obitools3.utils cimport bytes2str, str2bytes
-
-from .capi.obidms cimport obi_dms, \
-                          obi_close_dms
-                          
-from .capi.obidmscolumn cimport obi_truncate_and_close_column, \
-                                obi_column_format_date, \
-                                OBIDMS_column_p, \
-                                OBIDMS_column_header_p
-                                
-from .capi.obitypes cimport const_char_p, \
-                            OBIType_t, \
-                            OBI_INT, \
-                            OBI_FLOAT, \
-                            OBI_BOOL, \
-                            OBI_CHAR, \
-                            OBI_STR, \
-                            OBI_SEQ, \
-                            name_data_type, \
-                            only_ATGC    # discuss
-                            
-from ._obidms cimport OBIDMS, \
-                      OBIDMS_column, \
-                      OBIView, \
-                      OBIView_line
-
-from ._obitaxo cimport OBI_Taxonomy
-
-from ._obiseq cimport OBI_Nuc_Seq, OBI_Nuc_Seq_Stored
-
-from ._obidmscolumn_int cimport OBIDMS_column_int, \
-                                OBIDMS_column_multi_elts_int
-                                
-from ._obidmscolumn_float cimport OBIDMS_column_float, \
-                                  OBIDMS_column_multi_elts_float
-
-from ._obidmscolumn_bool cimport OBIDMS_column_bool, \
-                                 OBIDMS_column_multi_elts_bool
-
-from ._obidmscolumn_char cimport OBIDMS_column_char, \
-                                 OBIDMS_column_multi_elts_char
-
-from ._obidmscolumn_str cimport OBIDMS_column_str, \
-                                OBIDMS_column_multi_elts_str
-
-from ._obidmscolumn_seq cimport OBIDMS_column_seq, \
-                                OBIDMS_column_multi_elts_seq
-
-from .capi.obiview cimport Obiview_p, \
-                           Obiviews_infos_all_p, \
-                           Obiview_infos_p, \
-                           Column_reference_p, \
-                           obi_new_view_nuc_seqs, \
-                           obi_new_view, \
-                           obi_new_view_cloned_from_name, \
-                           obi_new_view_nuc_seqs_cloned_from_name, \
-                           obi_open_view,  \
-                           obi_read_views, \
-                           obi_unmap_read_views, \
-                           obi_view_delete_column, \
-                           obi_view_add_column, \
-                           obi_view_get_column, \
-                           obi_view_get_pointer_on_column_in_view, \
-                           obi_select_line, \
-                           obi_select_lines, \
-                           obi_save_and_close_view, \
-                           VIEW_TYPE_NUC_SEQS, \
-                           NUC_SEQUENCE_COLUMN, \
-                           ID_COLUMN, \
-                           DESCRIPTION_COLUMN
-                           
-from libc.stdlib cimport malloc
-from cpython.pycapsule cimport PyCapsule_New, PyCapsule_GetPointer
-
-
-cdef class OBIDMS_column :
-
-    # Should only be initialized through a subclass
-    def __init__(self, OBIView view, str column_name):
-        
-        cdef OBIDMS_column_p  column_p
-        cdef OBIDMS_column_p* column_pp
-        
-        column_pp = <OBIDMS_column_p*> PyCapsule_GetPointer(((view.columns_pp)[column_name]), NULL) # or use C function
-        column_p = column_pp[0]     # TODO ugly cython dereferencing but can't find better
-        
-        # Fill structure
-        self.pointer = column_pp
-        self.dms = view.dms
-        self.view = view.pointer    # TODO pointer or instance?
-        self.data_type = bytes2str(name_data_type((column_p.header).returned_data_type))
-        self.column_name = bytes2str((column_p.header).name)
-        self.nb_elements_per_line = (column_p.header).nb_elements_per_line
-        self.elements_names = (bytes2str((column_p.header).elements_names)).split(';')
-
-    def __setitem__(self, index_t line_nb, object value):
-        self.set_line(line_nb, value)
-
-    def __getitem__(self, index_t line_nb):
-        return self.get_line(line_nb)
-
-    def __len__(self):
-        return (self.pointer)[0].header.lines_used
-    
-    def __sizeof__(self):
-        return ((self.pointer)[0].header.header_size + (self.pointer)[0].header.data_size)
-    
-    def __iter__(self):
-    
-        # Declarations
-        cdef index_t lines_used
-        cdef index_t line_nb
-        
-        # Yield each line
-        lines_used = (self.pointer)[0].header.lines_used
-        for line_nb in range(lines_used):
-            yield self.get_line(line_nb)
-    
-    cpdef update_pointer(self):
-        self.pointer = <OBIDMS_column_p*> obi_view_get_pointer_on_column_in_view(self.view, str2bytes(self.column_name))
-
-    cpdef list get_elements_names(self):
-        return self.elements_names
-         
-    cpdef str get_data_type(self):
-        return self.data_type
-    
-    cpdef index_t get_nb_lines_used(self):
-        return (self.pointer)[0].header.lines_used
-        
-    cpdef str get_creation_date(self):
-        return bytes2str(obi_column_format_date((self.pointer)[0].header.creation_date))
-    
-    cpdef str get_comments(self):
-        return bytes2str((self.pointer)[0].header.comments)
-    
-    def __repr__(self) :
-        cdef str to_print
-        to_print = ''
-        for line in self :
-            to_print = to_print + str(line) + "\n"
-        return to_print
-        
-    cpdef close(self):
-        if obi_truncate_and_close_column((self.pointer)[0]) < 0 :
-            raise Exception("Problem closing a column")
-
-
-    @staticmethod
-    cdef object get_subclass_type(OBIDMS_column_p column_p) :
-        
-        cdef object subclass
-        cdef OBIDMS_column_header_p header
-        cdef OBIType_t col_type
-        cdef bint col_writable
-        cdef bint col_one_element_per_line
-
-        header = column_p.header        
-        col_type = header.returned_data_type
-        col_writable = column_p.writable
-        col_one_element_per_line = ((header.nb_elements_per_line) == 1)
-        
-        if col_type == OBI_INT :
-            if col_one_element_per_line :
-                subclass = OBIDMS_column_int
-            else :
-                subclass = OBIDMS_column_multi_elts_int
-        elif col_type == OBI_FLOAT :
-            if col_one_element_per_line :
-                subclass = OBIDMS_column_float
-            else :
-                subclass = OBIDMS_column_multi_elts_float
-        elif col_type == OBI_BOOL :
-            if col_one_element_per_line :
-                subclass = OBIDMS_column_bool
-            else :
-                subclass = OBIDMS_column_multi_elts_bool
-        elif col_type == OBI_CHAR :
-            if col_one_element_per_line :
-                subclass = OBIDMS_column_char
-            else :
-                subclass = OBIDMS_column_multi_elts_char
-        elif col_type == OBI_STR :
-            if col_one_element_per_line :
-                subclass = OBIDMS_column_str
-            else :
-                subclass = OBIDMS_column_multi_elts_str
-        elif col_type == OBI_SEQ :
-            if col_one_element_per_line :
-                subclass = OBIDMS_column_seq
-            else :
-                subclass = OBIDMS_column_multi_elts_seq
-        else :
-            raise Exception("Problem with the data type")
-
-        return subclass
-
-    
-######################################################################################################
-
-cdef class OBIDMS_column_multi_elts(OBIDMS_column) :
-    
-    def __getitem__(self, index_t line_nb):
-        return OBIDMS_column_line(self, line_nb)
-
-    cpdef set_line(self, index_t line_nb, dict values):
-        for element_name in values :
-            self.set_item(line_nb, element_name, values[element_name])
-
-######################################################################################################
-
-cdef class OBIDMS_column_line :
-
-    def __init__(self, OBIDMS_column column, index_t line_nb) :
-        self.index = line_nb
-        self.column = column
-        
-    def __getitem__(self, str element_name) :
-        return self.column.get_item(self.index, element_name)
-     
-    def __setitem__(self, str element_name, object value):
-        self.column.set_item(self.index, element_name, value)
-
-    def __contains__(self, str element_name):
-        return (element_name in self.column.elements_names)
-
-    def __repr__(self) :
-        return str(self.column.get_line(self.index))
-
-##########################################
-
-
-cdef class OBIView :
-    
-    def __init__(self, OBIDMS dms, str view_name, bint new=False, object view_to_clone=None, list line_selection=None, str comments=""): 
-        
-        cdef Obiview_p view = NULL
-        cdef int i
-        cdef list col_list
-        cdef str col_name
-        cdef OBIDMS_column column
-        cdef OBIDMS_column_p column_p
-        cdef OBIDMS_column_p* column_pp
-        cdef OBIDMS_column_header_p header
-        cdef index_t* line_selection_p
-        cdef object col_capsule
-        
-        self.dms = dms
-        
-        if line_selection is not None :
-            line_selection_p = <index_t*> malloc((len(line_selection) + 1) * sizeof(index_t))
-            for i in range(len(line_selection)) :
-                line_selection_p[i] = line_selection[i]     # TODO type problem?
-            line_selection_p[len(line_selection)] = -1
-        else :
-            line_selection_p = NULL
-       
-        if new :
-            if view_to_clone is not None :
-                if type(view_to_clone) == str :
-                    view = obi_new_view_cloned_from_name(dms.pointer, str2bytes(view_name), str2bytes(view_to_clone), line_selection_p, str2bytes(comments))
-                else :
-                    view = obi_new_view(dms.pointer, str2bytes(view_name), (<OBIView> view_to_clone).pointer, line_selection_p, str2bytes(comments))
-            elif view_to_clone is None :
-                view = obi_new_view(dms.pointer, str2bytes(view_name), NULL, line_selection_p, str2bytes(comments))
-        elif not new :
-            if view_name is not None :
-                view = obi_open_view(dms.pointer, str2bytes(view_name))
-            elif view_name is None :
-                view = obi_open_view(dms.pointer, NULL)
-        
-        if view == NULL :
-            raise Exception("Error creating/opening view")
-
-        self.pointer = view
-        self.name = bytes2str(view.name)
-        
-        # go through columns to build list and open python object (TODO make separate function?)
-        self.columns = {}
-        self.columns_pp = {}
-                
-        i = 0
-        while i < view.column_count :
-            column_pp = <OBIDMS_column_p*> ((view.columns)+i)
-            column_p = <OBIDMS_column_p> (view.columns)[i]
-            header = (column_p).header
-            
-            col_name = bytes2str(header.name)
-            col_capsule = PyCapsule_New(column_pp, NULL, NULL)   # TODO discuss
-            (self.columns_pp)[col_name] = col_capsule
-            
-            subclass = OBIDMS_column.get_subclass_type(column_p)
-            self.columns[col_name] = subclass(self, col_name)
-            
-            i+=1
-
-
-    def __repr__(self) :
-        
-        cdef str s
-        cdef OBIDMS_column column
-        cdef OBIDMS_column_p column_p
-        
-        s = self.name
-        s = s + ", " + self.comments + ", " + str(self.pointer.line_count) + " lines" 
-        for column_name in self.columns :   # TODO make function in OBIDMS_column class
-            column = self.columns[column_name]
-            column_p = (column.pointer)[0]
-            s = s + "\n" + column_name + ", version " + str(column_p.header.version) + ", data type: " + column.data_type
-        return s
-        
-    
-    cpdef delete_column(self, str column_name) :
-        
-        cdef int i
-        cdef Obiview_p view
-        cdef OBIDMS_column column
-        cdef OBIDMS_column_p column_p
-        cdef OBIDMS_column_p* column_pp
-        cdef OBIDMS_column_header_p header
-        cdef str column_n
- 
-        view = self.pointer
- 
-        if obi_view_delete_column(view, str2bytes(column_name)) < 0 :
-            raise Exception("Problem deleting a column from a view")
-        
-        # Update the dictionaries of column pointers and column objects, and update pointers in column objects (make function?):
-        (self.columns).pop(column_name)
-        (self.columns_pp).pop(column_name)
-        
-        i = 0
-        while i < view.column_count :
-            column_pp = <OBIDMS_column_p*> ((view.columns)+i)
-            column_p = <OBIDMS_column_p> (view.columns)[i]
-            header = (column_p).header
-            col_name = bytes2str(header.name)
-            col_capsule = PyCapsule_New(column_pp, NULL, NULL)
-            (self.columns_pp)[col_name] = col_capsule
-            i+=1
-
-        for column_n in self.columns :
-            (self.columns[column_n]).update_pointer()
-
-    
-    cpdef add_column(self, 
-                     str column_name, 
-                     obiversion_t version_number=-1, 
-                     str type='',
-                     index_t nb_lines=0, 
-                     index_t nb_elements_per_line=1,    # TODO 1?
-                     list elements_names=None,
-                     str avl_name="default_AVL_tree",
-                     str comments="",
-                     bint create=True   # TODO
-                     ) :
-        
-        cdef bytes column_name_b
-        cdef bytes elements_names_b
-        cdef object subclass
-        cdef OBIDMS_column_p* column_pp
-        cdef OBIDMS_column_p column_p
-        
-        column_name_b = str2bytes(column_name)
-        
-        if nb_elements_per_line > 1 :
-            elements_names_b = str2bytes(';'.join(elements_names))
-        elif nb_elements_per_line == 1 :
-            elements_names_b = column_name_b
-    
-        if type :    
-            if type == 'OBI_INT' :
-                data_type = OBI_INT
-            elif type == 'OBI_FLOAT' :
-                data_type = OBI_FLOAT
-            elif type == 'OBI_BOOL' :
-                data_type = OBI_BOOL
-            elif type == 'OBI_CHAR' :
-                data_type = OBI_CHAR
-            elif type == 'OBI_STR' :
-                data_type = OBI_STR
-            elif type == 'OBI_SEQ' :
-                data_type = OBI_SEQ
-            else :
-                raise Exception("Invalid provided data type")
-        
-        if (obi_view_add_column(self.pointer, column_name_b, version_number,     # should return pointer on column?
-                                data_type, nb_lines, nb_elements_per_line, 
-                                elements_names_b, str2bytes(avl_name), 
-                                str2bytes(comments), create) < 0) :
-            raise Exception("Problem adding a column in a view")
-        
-        # Store the column pointer
-        column_pp = obi_view_get_pointer_on_column_in_view(self.pointer, column_name_b)
-        if column_pp == NULL :
-            raise Exception("Problem getting a column in a view")
-        
-        col_capsule = PyCapsule_New(column_pp, NULL, NULL)   # TODO
-        (self.columns_pp)[column_name] = col_capsule
-        
-        # Open and store the subclass
-        column_p = column_pp[0]     # TODO ugly cython dereferencing
-        subclass = OBIDMS_column.get_subclass_type(column_p)
-        (self.columns)[column_name] = subclass(self, column_name)
-                
-    
-    cpdef save_and_close(self) :
-        if (obi_save_and_close_view(self.pointer) < 0) :
-            raise Exception("Problem closing a view")
-    
-
-    def __iter__(self):
-        # iter on each line of all columns
-        
-        # Declarations
-        cdef index_t lines_used
-        cdef index_t line_nb
-        cdef OBIView_line line  # TODO for NUC SEQS View
-        
-        # Yield each line TODO line class
-        lines_used = (self.pointer).line_count
-        
-        for line_nb in range(lines_used) :
-            line = self[line_nb]
-            yield line
-
-
-    def __getitem__(self, object item) :
-        if type(item) == str :
-            return (self.columns)[item]
-        elif type(item) == int :     # TODO int?
-            return OBIView_line(self, item)
-
-
-    cpdef select_line(self, index_t line_nb) :
-        if obi_select_line(self.pointer, line_nb) < 0 :
-            raise Exception("Problem selecting a line")
-
-    
-    cpdef select_lines(self, list line_selection) :
-        cdef index_t* line_selection_p
-        line_selection_p = <index_t*> malloc((len(line_selection) + 1) * sizeof(index_t))
-        for i in range(len(line_selection)) :
-            line_selection_p[i] = line_selection[i]     # TODO type problem?
-        line_selection_p[len(line_selection)] = -1
-        if obi_select_lines(self.pointer, line_selection_p) < 0 :
-            raise Exception("Problem selecting a list of lines")
-    
-    
-    def __contains__(self, str column_name):
-        return (column_name in self.columns)
-    
-    
-    def __str__(self) :
-        cdef OBIView_line line
-        cdef str to_print
-        to_print = ""
-        for line in self.__iter__() :
-            to_print = to_print + str(line) + "\n"
-        return to_print
-
-#############################################
-
-cdef class OBIView_NUC_SEQS(OBIView):
-    
-    def __init__(self, OBIDMS dms, str view_name, bint new=False, object view_to_clone=None, list line_selection=None, str comments=""): 
-        
-        cdef Obiview_p view = NULL
-        cdef int i
-        cdef list col_list
-        cdef str col_name
-        cdef OBIDMS_column column
-        cdef OBIDMS_column_p column_p
-        cdef OBIDMS_column_p* column_pp
-        cdef OBIDMS_column_header_p header
-        cdef index_t* line_selection_p
-                
-        self.dms = dms
-        
-        if line_selection is not None :
-            line_selection_p = <index_t*> malloc((len(line_selection) + 1) * sizeof(index_t))
-            for i in range(len(line_selection)) :
-                line_selection_p[i] = line_selection[i]     # TODO type problem?
-            line_selection_p[len(line_selection)] = -1
-        else :
-            line_selection_p = NULL
-       
-        if new :
-            if view_to_clone is not None :
-                if type(view_to_clone) == str :
-                    view = obi_new_view_nuc_seqs_cloned_from_name(dms.pointer, str2bytes(view_name), str2bytes(view_to_clone), line_selection_p, str2bytes(comments))
-                else :
-                    view = obi_new_view_nuc_seqs(dms.pointer, str2bytes(view_name), (<OBIView> view_to_clone).pointer, line_selection_p, str2bytes(comments))
-            elif view_to_clone is None :
-                view = obi_new_view_nuc_seqs(dms.pointer, str2bytes(view_name), NULL, line_selection_p, str2bytes(comments))
-        elif not new :
-            if view_name is not None :
-                view = obi_open_view(dms.pointer, str2bytes(view_name))
-            elif view_name is None :
-                view = obi_open_view(dms.pointer, NULL)
-        
-        if view == NULL :
-            raise Exception("Error creating/opening view")
-
-        self.pointer = view
-        self.name = bytes2str(view.name)
-        self.comments = bytes2str(view.comments)
-        
-        # go through columns to build list and open python object (TODO make separate function?)
-        self.columns = {}
-        self.columns_pp = {}
-                
-        i = 0
-        while i < view.column_count :
-            column_pp = <OBIDMS_column_p*> ((view.columns)+i)
-            column_p = <OBIDMS_column_p> (view.columns)[i]
-            header = (column_p).header
-            
-            col_name = bytes2str(header.name)
-            col_capsule = PyCapsule_New(column_pp, NULL, NULL)   # TODO discuss
-            (self.columns_pp)[col_name] = col_capsule
-            
-            subclass = OBIDMS_column.get_subclass_type(column_p)
-            self.columns[col_name] = subclass(self, col_name)
-            
-            i+=1
-        
-        self.ids = self.columns[bytes2str(ID_COLUMN)]
-        self.sequences = self.columns[bytes2str(NUC_SEQUENCE_COLUMN)]
-        self.descriptions = self.columns[bytes2str(DESCRIPTION_COLUMN)]
-
-
-    cpdef delete_column(self, str column_name) :
-        
-        cdef int i
-        cdef Obiview_p view
-        cdef OBIDMS_column column
-        cdef OBIDMS_column_p column_p
-        cdef OBIDMS_column_p* column_pp
-        cdef OBIDMS_column_header_p header
-        cdef str column_n
- 
-        if ((column_name == bytes2str(ID_COLUMN)) or (column_name == bytes2str(NUC_SEQUENCE_COLUMN)) or (column_name == bytes2str(DESCRIPTION_COLUMN))) :
-            raise Exception("Can't delete an obligatory column from a NUC_SEQS view")
-        
-        view = self.pointer
- 
-        if obi_view_delete_column(view, str2bytes(column_name)) < 0 :
-            raise Exception("Problem deleting a column from a view")
-        
-        # Update the dictionaries of column pointers and column objects, and update pointers in column objects (make function?):
-        (self.columns).pop(column_name)
-        (self.columns_pp).pop(column_name)
-        
-        i = 0
-        while i < view.column_count :
-            column_pp = <OBIDMS_column_p*> ((view.columns)+i)
-            column_p = <OBIDMS_column_p> (view.columns)[i]
-            header = (column_p).header
-            col_name = bytes2str(header.name)
-            col_capsule = PyCapsule_New(column_pp, NULL, NULL)
-            (self.columns_pp)[col_name] = col_capsule
-            i+=1
-
-        for column_n in self.columns :
-            (self.columns[column_n]).update_pointer()
-
-            
-    def __getitem__(self, object item) :
-        if type(item) == str :
-            return (self.columns)[item]
-        elif type(item) == int :     # TODO int?
-            return OBI_Nuc_Seq_Stored(self, item)
-    
-    
-    def __setitem__(self, index_t line_idx, OBI_Nuc_Seq sequence_obj) :
-        for key in sequence_obj :
-            self[line_idx][key] = sequence_obj[key]
-            
-
-#############################################
-
-cdef class OBIView_line :
-
-    def __init__(self, OBIView view, index_t line_nb) :
-        self.index = line_nb
-        self.view = view
-    
-    def __getitem__(self, str column_name) :
-        return ((self.view).columns)[column_name][self.index]
-    
-    def __setitem__(self, str column_name, object value):
-        # TODO detect multiple elements (dict type)? put somewhere else? but more risky (in get)
-        cdef type value_type
-        cdef str value_obitype
-        if column_name not in self.view :
-            if value == None :
-                raise Exception("Trying to create a column from a None value (can't guess type)")
-            value_type = type(value)
-            if value_type == int :
-                value_obitype = 'OBI_INT'
-            elif value_type == float :
-                value_obitype = 'OBI_FLOAT'
-            elif value_type == bool :
-                value_obitype = 'OBI_BOOL'            
-            elif value_type == str :
-                if only_ATGC(str2bytes(value)) :    # TODO
-                    value_obitype = 'OBI_SEQ'
-                elif len(value) == 1 :
-                    value_obitype = 'OBI_CHAR'
-                elif (len(value) > 1) :
-                    value_obitype = 'OBI_STR'
-            else :
-                raise Exception("Could not guess the type of a value to create a new column")
-            self.view.add_column(column_name, type=value_obitype)
-
-        (((self.view).columns)[column_name]).set_line(self.index, value)
-
-    def __contains__(self, str column_name):
-        return (column_name in self.view)
-        
-    def __repr__(self):
-        cdef dict line
-        cdef str column_name
-        line = {}
-        for column_name in self.view.columns :
-            line[column_name] = self[column_name]
-        return str(line)
-
-##########################################
-
-cdef class OBIDMS :
-    
-    def __init__(self, str dms_name) :
-        
-        # Declarations
-        cdef bytes dms_name_b
-        
-        # Format the character string to send to C function
-        dms_name_b = str2bytes(dms_name)
-        
-        # Fill structure and create or open the DMS
-        self.dms_name = dms_name
-        self.pointer = obi_dms(<const_char_p> dms_name_b)
-        if self.pointer == NULL :
-            raise Exception("Failed opening or creating an OBIDMS")
-
-
-    cpdef close(self) :
-        if (obi_close_dms(self.pointer)) < 0 :
-            raise Exception("Problem closing an OBIDMS")
-
-    
-    cpdef OBI_Taxonomy open_taxonomy(self, str taxo_name) :
-        return OBI_Taxonomy(self, taxo_name)
-    
-
-    cpdef OBIView open_view(self, str view_name) :
-        
-        cdef object view_class
-        cdef dict view_infos
-        
-        view_infos = self.read_view_infos(view_name)
-        
-        if view_infos["view_type"] == bytes2str(VIEW_TYPE_NUC_SEQS) :
-            view_class = OBIView_NUC_SEQS
-        else :
-            view_class = OBIView
-        
-        return view_class(self, view_name)
-    
-
-    cpdef OBIView new_view(self, str view_name, object view_to_clone=None, list line_selection=None, str view_type=None, str comments="") :
-        
-        cdef object view_class
-        
-        if view_type is not None :
-            if view_type == bytes2str(VIEW_TYPE_NUC_SEQS) :
-                view_class = OBIView_NUC_SEQS
-        else :
-            view_class = OBIView
-        
-        return view_class(self, view_name, new=True, view_to_clone=view_to_clone, line_selection=line_selection, comments=comments)
-
-
-    cpdef dict read_view_infos(self, str view_name) :
-        all_views = self.read_views()
-        return all_views[view_name]
-
-
-    cpdef dict read_views(self) :    # TODO function that prints the dic nicely and function that prints 1 view. Add column type in col ref
-        
-        cdef Obiviews_infos_all_p all_views_p
-        cdef Obiview_infos_p view_p
-        cdef Column_reference_p column_refs
-        cdef int nb_views
-        cdef int i, j
-        cdef str view_name
-        cdef str column_name
-        cdef dict views
-        cdef bytes name_b
-        
-        views = {}
-        all_views_p = obi_read_views(self.pointer)
-        if all_views_p == NULL :
-            raise Exception("No views to read")
-        nb_views = <int> (all_views_p.header).view_count
-        for i in range(nb_views) :
-            view_p = (<Obiview_infos_p> (all_views_p.view_infos)) + i
-            view_name = bytes2str(view_p.name)
-            views[view_name] = {}
-            views[view_name]["comments"] = bytes2str(view_p.comments)
-            views[view_name]["view_type"] = bytes2str(view_p.view_type)
-            views[view_name]["column_count"] = <int> view_p.column_count
-            views[view_name]["line_count"] = <int> view_p.line_count
-            views[view_name]["view_number"] = <int> view_p.view_number
-            views[view_name]["created_from"] = bytes2str(view_p.created_from)
-            views[view_name]["creation_date"] = bytes2str(obi_column_format_date(view_p.creation_date))     # TODO move this function in utils or somethings
-            if (view_p.all_lines) :
-                views[view_name]["line_selection"] = None
-            else :
-                views[view_name]["line_selection"] = {}
-                views[view_name]["line_selection"]["column_name"] = bytes2str((view_p.line_selection).column_name)
-                views[view_name]["line_selection"]["version"] = <int> (view_p.line_selection).version
-            views[view_name]["column_references"] = {}
-            column_refs = view_p.column_references
-            for j in range(views[view_name]["column_count"]) :
-                column_name = bytes2str((column_refs[j]).column_name)
-                views[view_name]["column_references"][column_name] = {}
-                views[view_name]["column_references"][column_name]["version"] = column_refs[j].version
-        
-        obi_unmap_read_views(all_views_p);
-        
-        return views
-    
-    
-    
-    

python/obitools3/obidms/_obidmscolumn_bool.cfiles

@@ -1,20 +0,0 @@
-../../../src/obidmscolumn_bool.c
-../../../src/obidmscolumn_bool.h
-../../../src/obiview.h
-../../../src/obiview.c
-../../../src/obidmscolumn.h
-../../../src/obidmscolumn.c
-../../../src/obidmscolumndir.h
-../../../src/obidmscolumndir.c
-../../../src/obidms.h
-../../../src/obidms.c
-../../../src/obierrno.h
-../../../src/obierrno.c
-../../../src/obilittlebigman.h
-../../../src/obilittlebigman.c
-../../../src/obitypes.h
-../../../src/obitypes.c
-../../../src/private_at_functions.h
-../../../src/private_at_functions.c
-../../../src/obiavl.h
-../../../src/obiavl.c

python/obitools3/obidms/_obidmscolumn_bool.pxd

@@ -1,14 +0,0 @@
-#cython: language_level=3
-
-from .capi.obitypes cimport index_t
-from ._obidms       cimport OBIDMS_column, OBIDMS_column_multi_elts
-
-
-cdef class OBIDMS_column_bool(OBIDMS_column):
-    cpdef object get_line(self, index_t line_nb)
-    cpdef set_line(self, index_t line_nb, object value)
-
-cdef class OBIDMS_column_multi_elts_bool(OBIDMS_column_multi_elts):
-    cpdef object get_item(self, index_t line_nb, str element_name)
-    cpdef object get_line(self, index_t line_nb)
-    cpdef set_item(self, index_t line_nb, str element_name, object value)

python/obitools3/obidms/_obidmscolumn_bool.pyx

@@ -1,78 +0,0 @@
-#cython: language_level=3
- 
-from .capi.obidmscolumn cimport obi_truncate_and_close_column, \
-                                obi_column_get_obibool_with_elt_name_in_view, \
-                                obi_column_get_obibool_with_elt_idx_in_view, \
-                                obi_column_set_obibool_with_elt_name_in_view, \
-                                obi_column_set_obibool_with_elt_idx_in_view
-from .capi.obierrno     cimport obi_errno
-from .capi.obitypes     cimport OBIBool_NA, obibool_t
-
-from obitools3.utils cimport str2bytes
-
-from cpython.bool cimport PyBool_FromLong
-
-
-cdef class OBIDMS_column_bool(OBIDMS_column):
-
-    cpdef object get_line(self, index_t line_nb):
-        cdef obibool_t value
-        cdef object result
-        value = obi_column_get_obibool_with_elt_idx_in_view(self.view, (self.pointer)[0], line_nb, 0)
-        if obi_errno > 0 :
-            raise IndexError(line_nb)
-        if value == OBIBool_NA :
-            result = None
-        else :
-            result = PyBool_FromLong(value)
-        return result
-    
-    cpdef set_line(self, index_t line_nb, object value):
-        if value is None :
-            value = OBIBool_NA
-        if obi_column_set_obibool_with_elt_idx_in_view(self.view, (self.pointer)[0], line_nb, 0, <obibool_t> value) < 0:
-            raise Exception("Problem setting a value in a column")
-
-
-cdef class OBIDMS_column_multi_elts_bool(OBIDMS_column_multi_elts):
-
-    cpdef object get_item(self, index_t line_nb, str element_name):
-        cdef obibool_t value
-        cdef object result
-        value = obi_column_get_obibool_with_elt_name_in_view(self.view, (self.pointer)[0], line_nb, str2bytes(element_name))
-        if obi_errno > 0 :
-            raise IndexError(line_nb, element_name)
-        if value == OBIBool_NA :
-            result = None
-        else :
-            result = PyBool_FromLong(value)
-        return result
-    
-    cpdef object get_line(self, index_t line_nb) :
-        cdef obibool_t value
-        cdef object value_in_result
-        cdef dict result
-        cdef index_t i
-        cdef bint all_NA
-        result = {}
-        all_NA = True
-        for i in range(self.nb_elements_per_line) :
-            value = obi_column_get_obibool_with_elt_idx_in_view(self.view, (self.pointer)[0], line_nb, i)
-            if obi_errno > 0 :
-                raise IndexError(line_nb)
-            if value == OBIBool_NA :
-                value_in_result = None
-            else :
-                value_in_result = PyBool_FromLong(value)   
-            result[self.elements_names[i]] = value_in_result
-            if all_NA and (value_in_result is not None) :
-                all_NA = False
-        if all_NA :
-            result = None
-        return result
-
-    cpdef set_item(self, index_t line_nb, str element_name, object value):
-        if value is None :
-            value = OBIBool_NA
-        if obi_column_set_obibool_with_elt_name_in_view(self.view, (self.pointer)[0], line_nb, str2bytes(element_name), <obibool_t> value) < 0:
-            raise Exception("Problem setting a value in a column")

python/obitools3/obidms/_obidmscolumn_char.cfiles

@@ -1,20 +0,0 @@
-../../../src/obidmscolumn_char.c
-../../../src/obidmscolumn_char.h
-../../../src/obiview.h
-../../../src/obiview.c
-../../../src/obidmscolumn.h
-../../../src/obidmscolumn.c
-../../../src/obidmscolumndir.h
-../../../src/obidmscolumndir.c
-../../../src/obidms.h
-../../../src/obidms.c
-../../../src/obierrno.h
-../../../src/obierrno.c
-../../../src/obilittlebigman.h
-../../../src/obilittlebigman.c
-../../../src/obitypes.h
-../../../src/obitypes.c
-../../../src/private_at_functions.h
-../../../src/private_at_functions.c
-../../../src/obiavl.h
-../../../src/obiavl.c

python/obitools3/obidms/_obidmscolumn_char.pxd

@@ -1,14 +0,0 @@
-#cython: language_level=3
-
-from .capi.obitypes cimport index_t
-from ._obidms       cimport OBIDMS_column, OBIDMS_column_multi_elts
-
-
-cdef class OBIDMS_column_char(OBIDMS_column):
-    cpdef object get_line(self, index_t line_nb)
-    cpdef set_line(self, index_t line_nb, object value)
-
-cdef class OBIDMS_column_multi_elts_char(OBIDMS_column_multi_elts):
-    cpdef object get_item(self, index_t line_nb, str element_name)
-    cpdef object get_line(self, index_t line_nb)
-    cpdef set_item(self, index_t line_nb, str element_name, object value)

python/obitools3/obidms/_obidmscolumn_char.pyx

@@ -1,77 +0,0 @@
-#cython: language_level=3
- 
-from .capi.obidmscolumn cimport obi_truncate_and_close_column, \
-                                obi_column_get_obichar_with_elt_name_in_view, \
-                                obi_column_get_obichar_with_elt_idx_in_view, \
-                                obi_column_set_obichar_with_elt_name_in_view, \
-                                obi_column_set_obichar_with_elt_idx_in_view
-from .capi.obierrno     cimport obi_errno
-from .capi.obitypes     cimport OBIChar_NA, obichar_t
-
-from obitools3.utils cimport str2bytes, bytes2str
-
-
-cdef class OBIDMS_column_char(OBIDMS_column):
-
-    cpdef object get_line(self, index_t line_nb):
-        cdef obichar_t value
-        cdef object result
-        value = obi_column_get_obichar_with_elt_idx_in_view(self.view, (self.pointer)[0], line_nb, 0)
-        if obi_errno > 0 :
-            raise IndexError(line_nb)
-        if value == OBIChar_NA :
-            result = None
-        else :
-            result = bytes2str(value)
-        return result
-
-    cpdef set_line(self, index_t line_nb, object value):
-        if value is None :
-            value = OBIChar_NA
-        if obi_column_set_obichar_with_elt_idx_in_view(self.view, (self.pointer)[0], line_nb, 0, str2bytes(value)[0]) < 0:
-            raise Exception("Problem setting a value in a column")
-
-
-cdef class OBIDMS_column_multi_elts_char(OBIDMS_column_multi_elts):
-
-    cpdef object get_item(self, index_t line_nb, str element_name):
-        cdef obichar_t value
-        cdef object result
-        value = obi_column_get_obichar_with_elt_name_in_view(self.view, (self.pointer)[0], line_nb, str2bytes(element_name))
-        if obi_errno > 0 :
-            raise IndexError(line_nb, element_name)
-        if value == OBIChar_NA :
-            result = None
-        else :
-            result = bytes2str(value)
-        return result
-
-    cpdef object get_line(self, index_t line_nb) :
-        cdef obichar_t value
-        cdef object value_in_result
-        cdef dict result
-        cdef index_t i
-        cdef bint all_NA
-        result = {}
-        all_NA = True
-        for i in range(self.nb_elements_per_line) :
-            value = obi_column_get_obichar_with_elt_idx_in_view(self.view, (self.pointer)[0], line_nb, i)
-            if obi_errno > 0 :
-                raise IndexError(line_nb)
-            if value == OBIChar_NA :
-                value_in_result = None
-            else :
-                value_in_result = bytes2str(value)   
-            result[self.elements_names[i]] = value_in_result
-            if all_NA and (value_in_result is not None) :
-                all_NA = False
-        if all_NA :
-            result = None
-        return result
-
-    cpdef set_item(self, index_t line_nb, str element_name, object value):
-        if value is None :
-            value = OBIChar_NA
-        if obi_column_set_obichar_with_elt_name_in_view(self.view, (self.pointer)[0], line_nb, str2bytes(element_name), str2bytes(value)[0]) < 0:
-            raise Exception("Problem setting a value in a column")
-

python/obitools3/obidms/_obidmscolumn_float.cfiles

@@ -1,20 +0,0 @@
-../../../src/obidmscolumn_float.c
-../../../src/obidmscolumn_float.h
-../../../src/obiview.h
-../../../src/obiview.c
-../../../src/obidmscolumn.h
-../../../src/obidmscolumn.c
-../../../src/obidmscolumndir.h
-../../../src/obidmscolumndir.c
-../../../src/obidms.h
-../../../src/obidms.c
-../../../src/obierrno.h
-../../../src/obierrno.c
-../../../src/obilittlebigman.h
-../../../src/obilittlebigman.c
-../../../src/obitypes.h
-../../../src/obitypes.c
-../../../src/private_at_functions.h
-../../../src/private_at_functions.c
-../../../src/obiavl.h
-../../../src/obiavl.c