Rolling Upgrade: Databases Remain Available
A rolling upgrade moves a NuoDB domain, which might include multiple databases, to a new NuoDB release without shutting down the databases. Rolling upgrades ensure continuous database availability throughout the upgrade process.
All processes on a given host must be stopped to upgrade the software. Therefore, a no-downtime upgrade is only possible if all databases in the domain have sufficient redundancy:
Every database must have more than one SM in every Storage Group, and these SMs must be on different hosts.
Every database must have TEs on more than one host.
There must be more than one NuoDB Admin Process (AP), on different hosts.
If you prefer to stop the database while upgrading, see Upgrading With Databases Down.
It is best practice to keep a NuoDB AP in the durable domain configuration while its host is being upgraded. However in a two-AP domain, in the unusual event that an AP does not restart, AP quorum is lost. Without AP quorum, operations such as the addition of a new host are not allowed. See Admin Process (AP) Quorum.
In this situation, remove the AP from domain membership and regain AP quorum.
During a rolling upgrade, the NuoDB release software is updated on one host in the domain at a time so that at no time are all hosts in a database offline simultaneously.
The rolling upgrade procedure consists of executing the following steps on each host:
Shutting down NuoDB processes and services
Installing the latest release of NuoDB
Restarting the NuoDB processes and services
|It does not matter in which order you choose to upgrade the hosts. There is a small advantage in restarting the engines with the highest start IDs first and going down; however, this is not critical.|
For each host in the domain, execute the following steps:
Shut down the NuoDB database processes
Shut down all NuoDB Storage Managers (SMs) and Transaction Engines (TEs) for all databases running on the host machine using the
nuocmd shutdown processcommand. For more information on how to run this command, see NuoDB Command Reference.
nuocmd shutdown processcommand default is to shutdown the database processes "gracefully" which allows client connections to successfully complete their transactions. If a processes refuses to shutdown, use
nuocmd shutdown process --killor
nuocmd shutdown process --evicton them specifically.
Shut down the NuoDB AP
For more information, see the Domain Operations section, Starting and Stopping NuoDB Admin.
Install the new NuoDB release software
Install the new NuoDB release software; see the instructions in
Prepare NuoDB configuration files
The new NuoDB release might introduce new properties in the sample configuration files that are distributed with each release. To take advantage of the new features or changes to features, the files need to be manually merged so that the new or changed configuration parameters are moved into the existing configuration files. If NuoDB is installed using an RPM package manager utility, the old configuration files won’t be replaced with those included in the new package but instead. Some of the new configuration files might be created with a
.rpmnewsuffix by the RPM package manager. If NuoDB is installed using a TAR package, the
$NUODB_HOMEfor the new release might point to a different location. In such a case, all configuration files must be merged and copied to the correct directories under the new
$NUODB_HOMEdirectory. For more information, see Pre-Upgrade Best Practices.
Retain NuoDB Domain state
The NuoDB domain keeps track of current state information for both NuoDB Admin and its databases under management. This state information is persisted in the
$NUODB_VARDIR/raftlogfile. If NuoDB is installed using a TAR package, the
$NUODB_HOMEfor the new release might point to a different location. In such a case, the
raftlogfile must be copied to the new
Restart NuoDB Admin Services
Start the AP. For more information, see Starting and Stopping NuoDB Admin.
Start the NuoDB database processes (SMs and TEs)
Start all NuoDB SMs and TEs for all databases running on the host using the
nuocmd start processcommand. For more information on using this command, see NuoDB Command Reference.
|Repeat the above steps for each host in the domain.|
During the rolling upgrade process, you may monitor version information using the
show database-versions NuoDB Command operation; see NuoDB Command Reference.
Here is an example after one TE has been updated:
nuocmd show database-versions --db-name mydb
effective version ID: 1376256, effective version: 4.1|4.1.1|4.1.2, max version ID: 1376256 Available versions: Process versions: version ID: 1376256, version: 4.1|4.1.1|4.1.2, release: 4.1.2-6-a9cf2cd579 [SM] host1:48006 [start_id = 0] [server_id = nuoadmin-0] [pid = 98234] [node_id = 1] [last_ack = 4.83] MONITORED:RUNNING [SM] host2:48006 [start_id = 1] [server_id = nuoadmin-1] [pid = 45632] [node_id = 2] [last_ack = 8.80] MONITORED:RUNNING [TE] host3:48006 [start_id = 2] [server_id = nuoadmin-2] [pid = 10873] [node_id = 3] [last_ack = 3.22] MONITORED:RUNNING version ID: 1441792, version: 4.2, release: 4.2-1-c96c3d1a26 [TE] host4:48006 [start_id = 4] [server_id = nuoadmin-3] [pid = 82572] [node_id = 5] [last_ack = 2.92] MONITORED:RUNNING
At the end of the rolling upgrade,
show database-versions reveals that all database processes have been updated and, in this case, a database protocol upgrade is also needed:
nuocmd show database-versions --db-name mydb
effective version ID: 1376256, effective version: 4.1|4.1.1|4.1.2, max version ID: 1376256 Available versions: version ID: 1441792, version: 4.2 Process versions: version ID: 1441792, version: 4.2, release: 4.2-1-c96c3d1a26 [SM] host1:48006 [start_id = 5] [server_id = nuoadmin-0] [pid = 25439] [node_id = 6] [last_ack = 4.12] MONITORED:RUNNING [SM] host2:48006 [start_id = 6] [server_id = nuoadmin-1] [pid = 65232] [node_id = 7] [last_ack = 3.76] MONITORED:RUNNING [TE] host3:48006 [start_id = 7] [server_id = nuoadmin-2] [pid = 86244] [node_id = 8] [last_ack = 5.43] MONITORED:RUNNING [TE] host4:48006 [start_id = 4] [server_id = nuoadmin-3] [pid = 82572] [node_id = 5] [last_ack = 2.11] MONITORED:RUNNING
After upgrading the database software, it may be necessary to upgrade the database protocol.
For more information, see Upgrading the Database Protocol .