obitools3/doc/source/formats.rst

#######
Formats
#######


*********************************************
The OBItools3 Data Management System (OBIDMS)
*********************************************

An OBIDMS directory consists of :
	* OBIDMS column files
	* OBIDMS view descriptions
	* an OBIDMS history file


OBIDMS column files
===================

Each OBIDMS column file contains :
	* a header of a size equal to a multiple of PAGESIZE (PAGESIZE being equal to 4096 bytes
	  on most systems) containing metadata
	* one column of data of the same type

OBIDMS column files are read-only.


File name
---------

Each file is named with the attribute associated to the data it contains, and the number of
its version, separated by an underscore.

Example : ``count_0003``

.. todo::
	Filename extension?


Header
------

The header of an OBIDMS column contains :

	* Endian byte order
	* PAGESIZE value / Size of the header

	* Number of lines (total or without the header?)
	* Data type (int, str...)
	* Date of creation
	* Version of the file
	* Eventual comments


Data
----

A column of data of the same type.


Versioning
----------

Since OBIDMS column files are read-only, any modification leads to the creation of a new version
of the column file.

The first version of a column file is numbered 0001, and each new version increments that
number by 1.


Mandatory columns
-----------------

Some columns must exist in an OBIDMS directory :
	* sequence identifiers column


OBIDMS history file
===================

An OBIDMS history file consists of data that can be represented in the form of a directed acyclic
graph presenting the history of all the operations ever done in the OBIDMS directory.


OBIDMS views
============

An OBIDMS view corresponds to a list of OBIDMS columns and lines. A view includes one version
of each mandatory column. Only one version of each column is included. All the columns of
one view contain the same number of lines in the same order.


OBIDMS ULM
==========

.. image:: ./images/OBIDMS_ULM.png
:download:`html version of the OBIDMS ULM file </ObiDMS_ULM.class.violet.html>`


OBIIntColumn header file
========================

.. doxygenfile:: obiintcolumn.h