Rolling Upgrade: Databases Remain Available

A rolling upgrade moves a NuoDB domain, which might include multiple databases, to a new NuoDB release without shutting down the databases. Rolling upgrades ensure continuous database availability throughout the upgrade process.

All processes on a given host must be stopped to upgrade the software; this means that no-downtime upgrade is only possible if all databases in the domain have sufficient redundancy:

  • Every database must have more than one SM in every Storage Group, and these SMs must be on different hosts.

  • Every database must have TEs on more than one host.

  • There must be more than one NuoDB Admin Process (AP), on different hosts.

If you prefer to stop the database while upgrading, see Upgrading With Databases Down.

Maintaining Admin Process Quorum in a Two-Admin Process Domain

It is best practice to keep a NuoDB Admin process (AP) in the durable domain configuration while its host is being upgraded. However in a two-AP domain, in the unusual event that an AP does not restart, AP quorum is lost. Without AP quorum, operations such as the addition of a new host are not allowed. See Admin Process Quorum.

In this situation, remove the AP from domain membership and regain AP quorum.

Rolling Upgrade Instructions

During a rolling upgrade, the NuoDB release software is updated on one host in the domain at a time so that at no time are all hosts in a database offline simultaneously.

The rolling upgrade procedure consists of executing the following steps on each host: . Shutting down NuoDB processes and services . Installing the latest release of NuoDB . Restarting the NuoDB processes and services

It does not matter in which order you choose to upgrade the hosts. There is a small advantage in restarting the engines with the highest start IDs first and going down; however this is not critical.

Upgrade Each Host

For each host in the domain, execute the following steps:

  1. Shut down the NuoDB database processes
    Shut down all NuoDB Storage Managers (SMs) and Transaction Engines (TEs) for all databases running on the host machine using the nuocmd shutdown process command. For more information on how to run this command, see NuoDB Command Reference.

    The nuocmd shutdown process command default is to shutdown the database processes "gracefully" which allows client connections to successfully complete their transactions. If a processes refuses to shutdown, use nuocmd shutdown process --kill or nuocmd shutdown process --evict on them specifically.
  2. Shut down the NuoDB Admin Process
    For more information, see the Domain Operations section, Starting and Stopping NuoDB Admin.

  3. Install the new NuoDB release software
    Install the new NuoDB release software; see the instructions in

  4. Restart NuoDB Admin Services
    Start the AP. For more information, see Starting and Stopping NuoDB Admin.

  5. Start the NuoDB database processes (SMs and TEs)
    Start all NuoDB SMs and TEs for all databases running on the host using the nuocmd start process command. For more information on using this command, see NuoDB Command Reference.

Repeat the above steps for each host in the domain.

Verifying the Rolling Upgrade

During the rolling upgrade process, you may monitor version information using the show database-versions NuoDB Command operation; see NuoDB Command Reference. Here is an example after one TE has been updated:

nuocmd show database-versions --db-name mydb
effective version ID: 1376256, effective version: 4.1|4.1.1|4.1.2, max version ID: 1376256
Available versions:
Process versions:
  version ID: 1376256, version: 4.1|4.1.1|4.1.2, release: 4.1.2-6-a9cf2cd579
    [SM] host1:48006 [start_id = 0] [server_id = nuoadmin-0] [pid = 98234] [node_id = 1] [last_ack =  4.83] MONITORED:RUNNING
    [SM] host2:48006 [start_id = 1] [server_id = nuoadmin-1] [pid = 45632] [node_id = 2] [last_ack =  8.80] MONITORED:RUNNING
    [TE] host3:48006 [start_id = 2] [server_id = nuoadmin-2] [pid = 10873] [node_id = 3] [last_ack =  3.22] MONITORED:RUNNING
  version ID: 1441792, version: 4.2, release: 4.2-1-c96c3d1a26
    [TE] host4:48006 [start_id = 4] [server_id = nuoadmin-3] [pid = 82572] [node_id = 5] [last_ack =  2.92] MONITORED:RUNNING

At the end of the rolling upgrade, show database-versions reveals that all database processes have been updated and, in this case, a database protocol upgrade is also needed:

nuocmd show database-versions --db-name mydb
effective version ID: 1376256, effective version: 4.1|4.1.1|4.1.2, max version ID: 1376256
Available versions:
  version ID: 1441792, version: 4.2
Process versions:
  version ID: 1441792, version: 4.2, release: 4.2-1-c96c3d1a26
    [SM] host1:48006 [start_id = 5] [server_id = nuoadmin-0] [pid = 25439] [node_id = 6] [last_ack =  4.12] MONITORED:RUNNING
    [SM] host2:48006 [start_id = 6] [server_id = nuoadmin-1] [pid = 65232] [node_id = 7] [last_ack =  3.76] MONITORED:RUNNING
    [TE] host3:48006 [start_id = 7] [server_id = nuoadmin-2] [pid = 86244] [node_id = 8] [last_ack =  5.43] MONITORED:RUNNING
    [TE] host4:48006 [start_id = 4] [server_id = nuoadmin-3] [pid = 82572] [node_id = 5] [last_ack =  2.11] MONITORED:RUNNING
After upgrading the database software, it may be necessary to upgrade the database protocol.
For more information, see Upgrading the Database Protocol .