Extending the Database Across Multiple Hosts (Scaling Out)

NuoDB is a distributed, peer-to-peer, database system. Each host server in the NuoDB domain runs one NuoDB Admin (nuoadmin). The NuoDB Command (nuocmd) commands in this section can be entered from any host in the domain running a nuoadmin service.

Step 1 - Configuring the initial host

If a NuoDB database will span more than one host, the configuration file on the initial host must be adjusted to include the explicit hostname of the initial host, rather than $(hostname). Go back to nuoadmin.conf on the initial host and check altAddr and intialMembership properties. For example:

  "altAddr": "$(hostname)",
  "initialMembership": {
    "nuoadmin-0": { "transport": "$(hostname):48005", "version": "0:10000" }
  }

Make sure that you substitute the hostname of the host server in place of $(hostname) in the above nuoadmin.conf configuration text.

If you have already started your NuoDB domain on your initial server using an incorrect value for altAddr, you may have to shut down all database and Admin Processes, delete the NuoDB Admin raftlog file in $NUODB_VARDIR, and restart all processes after replacing the altAddr value with the correct hostname or address. See Environment Variables for more information on how NUODB_VARDIR is resolved.

Step 2 - Installing NuoDB on Another Host

As part of configuring a multi-host database, install NuoDB on a second host machine. For information on installing NuoDB, see Installing NuoDB.

Step 3 - Configuring NuoDB Admin on a Second Host

On each host that will participate in the NuoDB domain, edit the nuoadmin.config configuration file for that host machine. For information on setting properties for an Admin Process (AP), see Host Properties (nuoadmin.conf).

Below is an excerpt from a sample nuoadmin.conf file, on a second host that defines an additional AP (nuoadmin-1) that will run on the second host machine (host2) as defined by the value set for ThisServerId. Edits made to nuoadmin.conf (see examples in bold below) ensure that the AP named nuoadmin-1 on host machine host2 peers into, and gains membership to, the initial NuoDB domain established by nuoadmin-0.

  "initialMembership": {
    "nuoadmin-0" : { "transport" : "host1:48005", "version" : "0:10000" }
  },
  "ThisServerId" : "nuoadmin-1",
  "altAddr" : "host2",
  ...

For information on additional properties to be set for an AP, see Host Properties (nuoadmin.conf).

Step 4 - Starting NuoDB Admin on a Second Host

When you have configured NuoDB Admin on a second host, then start NuoDB Admin for that host.

sudo systemctl start nuoadmin

For more information, see Starting NuoDB Admin.

Once the nuoadmin-1 AP is started on a second host (host2), nuoadmin-1 will be successfully added as a FOLLOWER to the NuoDB domain.

Step 5 - Verifying that a Second Host is Active

To verify that a second host has been successfully installed, is running, and is connected to the domain, run the nuocmd show domain command:

nuocmd show domain

server version: 6.0-1-fc6a857de9, server license: Enterprise
server time: 2023-09-08T06:21:58.175000, client token: f37640883d151b9a034970300ceb37ae46fecbad
Servers:
  [nuoadmin-0] host1:48005 (LEADER, Leader=nuoadmin-0) ACTIVE:Connected
  [nuoadmin-1] host2:48005 (FOLLOWER, Leader=nuoadmin-0) ACTIVE:Connected *
Databases:
  hockey [RUNNING]
    [SM] host1:48007 [start_id = 0] [server_id = nuoadmin-0] [pid = 40] [node_id = 1] MONITORED:RUNNING
    [TE] host1:48006 [start_id = 1] [server_id = nuoadmin-0] [pid = 39] [node_id = 2] MONITORED:RUNNING

The nuocmd show domain command is issued using NuoDB Command (nuocmd). For more information on NuoDB Command and other command line tools, see Command Line Tools.

Step 6 - Starting Database Processes on a Second Host

If planning to start a Storage Manager (SM) process on a second host, first use the nuocmd create archive command to create the required database archive. For details, see Creating an Archive.

Example:

nuocmd create archive --db-name hockey --server-id nuoadmin-1 --archive-path /var/opt/nuodb/demo-archives/hockey

To start the SM process on the database archive, use the nuocmd start database command with --incremental option. Alternatively, you may use the nuocmd start process command.

For our example, to start an SM and a Transaction Engine (TE) process for the hockey database on the second host (nuoadmin-1), use the following commands:

nuocmd start database --db-name hockey --incremental --te-server-id nuoadmin-1

--te-server-id nuoadmin-1 starts a new TE on the second host. Running this command again will start a second TE on the second host. In addition, where an archive is created, an SM process is automatically started on the available archive.

If planning to start a TE process for an existing database on another host, use the following command:

nuocmd start process --server-id <id of the host> --db-name <database name> --engine-type TE

For example:

nuocmd start process --server-id nuoadmin-1 --db-name hockey --engine-type TE

Step 7 - Verifying that Database Processes Have Started on a Second Host

To confirm that database processes have been successfully started on a second host, run the nuocmd show domain command:

nuocmd show domain

server version: 6.0-1-fc6a857de9, server license: Enterprise
server time: 2023-09-08T06:22:06.165012, client token: f37640883d151b9a034970300ceb37ae46fecbad
Servers:
  [nuoadmin-0] host1:48005 (LEADER, Leader=nuoadmin-0) ACTIVE:Connected *
  [nuoadmin-1] host2:48005 (FOLLOWER, Leader=nuoadmin-0) ACTIVE:Connected
Databases:
  hockey [RUNNING]
    [SM] host1:48007 [start_id = 0] [server_id = nuoadmin-0] [pid = 40] [node_id = 1] MONITORED:RUNNING
    [TE] host1:48006 [start_id = 1] [server_id = nuoadmin-0] [pid = 39] [node_id = 2] MONITORED:RUNNING
    [SM] host2:48007 [start_id = 2] [server_id = nuoadmin-1] [pid = 42] [node_id = 3] MONITORED:RUNNING
    [TE] host2:48006 [start_id = 3] [server_id = nuoadmin-1] [pid = 41] [node_id = 4] MONITORED:RUNNING

The NuoDB Limited Use License supports creating only one Storage Manager (SM). To deploy a database with two or more SMs, the installation of a NuoDB Enterprise License is required. For more information, see Obtain and Install a Product License.