The Storage Manager Subsystem

EYEDB is based on a client/server architecture. The server kernel is the storage manager subsystem providing the
following main services:
- persistent raw data management,
- transactional services,
- recovery system,
- B-tree and hash indexes,
- multi-volume database management.

The storage manager can be used independently from EYEDB.

Raw Data Management

The central concept of the storage manager is the raw object. A raw object is a piece of persistent raw data tied to an object identifier named oid .
An oid identifies a raw object in a unique way within a set of databases. It is generated by the storage manager at raw object creation.
An oid is composed of three fields: the storage index, the database identifier and a random generated magic number.
The first field identifies the object physical location within a database volume. The second one identifies a database and the last one ensures more security in the object identification process.

The storage manager is responsible for the management of raw objects:

• raw object creation (storage allocation, oid allocation, object storage),
• raw object update (contents modification),
• raw object reading,
• raw object deleting (oid deallocation, storage deallocation),
• raw object resizing (storage reallocation, object moving),
• raw object locking and unlocking (share locking, exclusive locking, private locking),
• raw object access control

Memory Mapped Architecture

The storage manager is based on a memory mapped architecture. Database volumes are mapped within the server virtual memory space.
Due to some 32-bit system limitations, the databases greater than 2Gb cannot be mapped as a whole.

The storage manager implements a segment-based mapping algorithm: when reading an object, the storage manager checks if the corresponding storage piece is mapped within its virtual memory space. If it is not mapped, it maps a large segment of data around the raw object, eventually extending a neighboring segment.
If the total mapped size is more than the system maximum, it unmaps the less recent used mapped segment. On 64-bit system, this algorithm is not needed as databases up to several Tb (i.e. tera-bytes) can be mapped at whole within the virtual memory space.
Currently, the storage manager can deal with databases up to one Tb.

Transactional Services

The storage manager provides standard transaction services which guarantees atomicity, consistency, isolation and integrity within a database.
Its transaction unit is based on a two-phase locking protocol. The protocol requires that each transaction issues lock and unlock requests in two phases:

• growing phase: a transaction may obtain locks but may not release any lock.
• shrinking phase: a transaction may release locks but may not obtain any new lock.

Initially, a transaction is in the growing phase. The transaction acquires locks as needed. Once the transaction releases a lock, it enters the shrinking phase and no more lock requests may be issued.

The storage manager provides different transaction locking modes: read and write shared, read shared and write exclusive, read and write exclusive or database exclusive.
It provides immediate deadlock detection.

The Recovery System

The storage manager provides a simple but efficient recovery system against failures:

• client failure: the transaction is automatically aborted by the server.
• server failure or operating system failure: the current transactions will be automatically aborted on the next database opening.
• the disk failure recovery is not supported: this is a deliberate choice of simplicity since storage consistency can rely on the RAID technology or transactional file systems now available on modern operating systems.

B-Tree and Hash Indexes

The storage manager provides support for B-Tree and Hash indexes.
The B-Tree index provides fixed size raw data indexation, efficient exact match query and range query.
The Hash index provides variable size raw data indexation and efficient exact match query. The hash key function can be provided by the client.

Multi-Volume Database Management

The database storage unit is the volume files. A database can contains up to 512 volumes each one up to 2Gb on a 32-bit file system interface, or up to several tera-bytes on a 64-bit file system interface. The storage manager provides facilities to add, move, resize and reorganize database volumes.