NuoDB System Architecture

The administration layer consists of Admin Processes or APs. The AP is a Java application called nuoadmin. On a bare-metal or VM-based implementation, every node (host) must have an AP running on it. In a containerized implementation, such as under Kubernetes, far fewer APs are typically deployed and they run in their own separate pods.

The APs also provide a connection brokering and load-balancing service to clients - see Client Connections below.

Transaction Engines (TEs) run in this layer and perform the parsing, optimization and execution of queries. They fetch data as required, either from another TE or an SM. All changes are passed to the SMs to be committed to disk. Once data has been loaded by a TE, it is stored in its cache for faster access next time. For more information, see Data Caching.

This layer is where Atomicity, Consistency and Isolation are implemented. NuoDB offers both SQL compliance and ACID transactions even though the database is distributed. This is in marked contrast to many NoSQL databases that typically have their own custom "SQL-like" query languages and usually do not offer ACID guarantees.

Multiple TEs are recommended both for high-availability (redundancy) and throughput (to handle the SQL load on the database).

Storage Managers (SMs) are responsible for persistence (Durability). Data is stored in files (ideally on a local disk) and reread as necessary. NuoDB calls the directory on disk, and all its contents, an archive, unlike other databases where an archive is a backup). Only TEs communicate with SMs either to request data or to have data stored. A redundant set of SMs (typically 3 or 5) is recommended.

By default, every SM manages a complete copy of the entire database, each is effectively a backup for all the others. NuoDB uses this to easily restore individual SMs in the event of a planned or unplanned outage - for more information, see Archive Synchronization.

When a TE needs changes to be committed, it sends the data to all the running SMs. The TE then waits for them all to acknowledge the commit before acknowledging the commit to its client. For performance and resilience, like most database systems, the SM writes all changes to a Journal, only periodically writing changes to its disk. Once changes in the journal have been written to disk, they can be removed from the journal (a process termed reaping).

In terms of the CAP Theorem, NuoDB prioritizes consistency over availability. Therefore a client may have to wait while a data item is being updated. However, all running SMs are always up to date.

Subdividing a database across subsets of SMs is possible using Data Partitioning. Partitioning allows each SM to hold only part of the archive, not all of it.

For a form of Eventual Consistency (data is always available, but may be slightly out of date) use Asynchronous SMs.

The set of APs, and any databases they manage, are referred to as a NuoDB Domain. Databases are unique to their domain and the TEs and SMs for a database are unique to that database. TEs and SMs are never shared by more than one database.

NuoDB provides a command-line utility, nuocmd, for domain and database management. The utility communicates with APs via a REST-style interface over HTTP or HTTPS, and the APs, in turn carry out the requests. nuocmd can be used to create archives and databases, start and stop database processes, fetch runtime and debugging information, and much more. The REST API is documented and can be used by your own scripts or applications.

Processes may come and go in a distributed environment, for example due to scaling in or out. To avoid clients having to keep track of the processes in the database, clients always connect via an AP (any AP, it does not matter which). All the APs have complete knowledge of all processes in the domain at all times and can return the address of a suitable TE for the client to talk to.

Once connected to a TE, the client will run SQL directly against that TE. They only ever need to talk to an AP again if they need new connections. Clients never talk to SMs.

See Connection Brokering for more details.

Because both TEs and SMs share common-code, in particular, data caching, they are both implemented by the same C++ application called nuodb in NUODB_HOME/bin. When nuodb starts, a run-time option tells it to run as either a TE or an SM. The nuodb application cannot be invoked directly. It must be started by the domain.