Commit Graph

63 Commits

Author SHA1 Message Date
3e45c34491 import: now imports and adds taxids for SILVA and RDP files, added
import of lists, fixed skipping of errors (was not overwriting), and
fixed --no-progress-bar option
2021-06-01 09:21:07 +12:00
e099a16624 small fixes 2021-04-07 10:28:02 +12:00
169b6514b4 small doc fixes 2021-03-29 13:07:48 +13:00
3e93cfff7b import: Columns are now rewritten in OBI_FLOAT if a value is > INT32_MAX 2021-03-29 09:00:52 +13:00
ad1fd3c341 Now handling dictionaries with one key 2021-03-10 16:50:30 +13:00
b9b4cec5b5 import: now can import SILVA fasta files 2020-10-30 10:43:04 +01:00
50b292b489 obi import: added --space-priority option to import a view line by line 2020-07-26 17:36:52 +02:00
8aa455ad8a Python: made all commands handle output to buffer object (e.g. stdout) 2020-07-24 16:09:48 +02:00
2a2c233936 obi import: fixed a bug when skipping an entry 2020-05-29 21:19:42 +02:00
6094ce2bbc obi import: skip on error more robust 2020-05-28 20:40:36 +02:00
f13f8f6165 obi import: minor doc/display improvements 2020-05-20 11:46:29 +02:00
fa834e4b8b obi import: small bug fix 2020-05-18 17:06:58 +02:00
773b36ec37 obi import: fixed the import of old obitools files with premerged
informations, and other minor improvements
2020-05-07 17:03:04 +02:00
bb72682f7d obi import: new option --preread to do a first readthrough of the
dataset if it contains huge dictionaries for a much faster import.
2020-02-02 21:12:34 +01:00
c67d668989 obi import: fixed a bug when the first entry would contain a dictionary
with one key. Switch to beta8
2020-01-29 20:23:39 +01:00
71276537a6 obi import: fixed bug when importing a taxdump 2019-09-22 16:45:30 +02:00
ec0737a600 Added signal catching and handling in C and Cython 2019-09-21 16:47:22 +02:00
06f9d6da60 obi import: importing a view to a DMS now uses the C API (more efficient
and imports all metadata)
2019-09-21 12:49:29 +02:00
7c3fa14789 obi import: fixed bug when reading output URI 2019-09-20 20:43:48 +02:00
eb6c59dc1e obi import: proper check for taxonomy name already existing in DMS when
importing a taxdump
2019-09-17 13:41:49 +02:00
f296517716 Various display improvements 2019-09-03 21:46:39 +02:00
3cfe3a9b00 Improved progress display when importing files in a DMS 2019-08-29 10:12:06 +02:00
a3e6b7d913 obi import: fixed import of View_NUC_SEQS to another DMS 2019-03-31 15:42:07 +02:00
ce686e9569 obi import: progress bar fixed when using --only option 2019-03-30 15:16:57 +01:00
8e70bf1ee1 obi import: fixed bug when rewriting a column (keeping wrong type in
import module)
2019-03-26 14:56:18 +01:00
802a3f5933 data import: entries now counted if there are multiple files 2019-03-18 18:16:39 +01:00
e8090a44c9 Fixed the ultimate bug with embl (and genbank) parsers: raising any
exception in a python generator makes it unable to resume. So now,
exceptions are not raised but printed, then functions return None and
that's handled at higher level.
2019-03-15 16:06:06 +01:00
2a4f1b8feb obi import: now properly uses macros for column names 2019-03-15 11:39:21 +01:00
d88390c6d8 Cython API: when importing a file in a DMS, its length is computed
beforehand for the progress bar
2019-03-13 18:35:32 +01:00
b45c2ee653 Cython API: cleaner column rewriting API 2019-03-13 11:13:55 +01:00
2a6a112d29 obi import: fixed writing quality in views when appropriate (but still
not a satisfying solution)
2018-12-11 19:33:55 +01:00
4d51f4f015 obi import: better checking of whether to import quality 2018-11-19 11:23:39 +01:00
e18b762d81 Weird buggy Eclipse commit with nothing changed 2018-10-21 17:35:18 +02:00
da0e3d4043 Cython: added full handling of NA strings when importing files 2018-10-17 16:41:15 +02:00
58589e04be Cython: rearranged input and output format options to have both and
updated commands accordingly
2018-10-17 11:19:48 +02:00
e9a41c5b97 Commands: updated for JSON formatted comments with history 2018-10-07 19:10:34 +02:00
b98880b7fa Various non-important fixes and comments 2018-07-28 17:07:17 +02:00
895d09b133 obi import: 'taxid' columns are imported as 'TAXID' to fit view
predicates, and fixed taxdump import and DMS closing
2018-07-28 17:03:00 +02:00
4a815785c4 obi import: added basic taxdump import 2018-03-21 16:35:44 +01:00
15e43bb9a1 Cython API: obi import can now import ngsfilter files and tabular files 2018-03-12 18:10:43 +01:00
c8097e14e1 obi import: removed old traces 2018-02-12 14:36:56 +01:00
428c4eb5e6 obi import: fixed creation of quality columns (to discuss) 2017-12-19 11:07:00 +01:00
1ae634d56b Added atexit command to obi import, obi uniq and obi less 2017-10-16 11:09:55 +02:00
532d8e9cd7 obi import: small efficiency improvement when dealing with NA values 2017-10-04 15:44:48 +02:00
6a2759eee6 obi import with new input/ouput API 2017-08-20 17:58:36 +02:00
407f61a408 Add the possibility to create temporary objects like a temporary
directory and a temporary DMS
2017-07-28 16:33:19 +02:00
b9c65a871f Patch decoding of URL 2017-07-28 12:41:28 +02:00
adf5cbef97 Added DMS method to create a DMS if it doesn't already exists, otherwise
opens it
2017-07-28 09:55:43 +02:00
9482c663c0 minor comments and changes 2017-07-27 19:46:34 +02:00
ced9a268a1 obi import: added an option to specify the NA value in the input file
(default is 'NA', same as in R's read.table function)
2017-07-11 12:10:33 +02:00