Using Incremental Hot Copy

An incremental hot copy creates a space-efficient, transactionally-consistent copy of an SM which stores only those atoms that have changed since the previous full or incremental hot copy in a backup set containing hot copies of that SM. Like full hot copy, incremental hot copy includes every storage group the hot copied SM serves. Because an incremental hot copy is relative to the most recent full or incremental hot copy in a backup set (rather than always being relative to the full hot copy), incremental hot copies do not become inflated over time as the database accumulates more changes relative to the full hot copy in a backup set. When restoring using incremental hot copies, all incremental hot copies in a backup set will need to be read to reconstruct the latest state.

Note: An incremental hot copy of an SM is approximately the size of all changed atoms in the SM being hot copied at the time the hot copy finishes, plus the size of the journal of the SM at the time the hot copy finishes. Note that when an atom is changed, the entire atom is copied into the incremental hot copy.

Note: If you are using table partitions and storage groups, and have two or more SMs serving disjoint sets of storage groups, the recommend restore approach is to use point-in-time restore with journal hot copy.

How to use incremental hot copy

To request an incremental hot copy, specify type incremental when executing the hot copy and specify an existing backupSetDirectory that contains a completed full hot copy. Incremental hot copy requires the use of backup sets.

Note: You must use the most-recently created backup set when initiating incremental hot copy. Incremental hot copy can only be executed once the full hot copy used to create that backup set has completed.

Incremental hot copies in a backup set must be restored into a new archive in order to use them to start an SM. Restoring an archive from an incremental hot copy in a backup set is accomplished via the NuoDB Archive utility (see NuoDB Archive). This will extract the latest incremental hot copy from the backup set and copy it to a new archive directory.

There is no option to restore only a subset of incremental hot copies within a backup set, restore of full and incremental hot copies always restores the latest incremental hot copy. Using journal hot copy enables point-in-time restore with backup sets, allowing restore to chosen points in time instead of restoring only the latest hot copy in the backup set.

Restore an archive from one or more incremental hot copies

Restore an archive from a backup set using NuoDB Archive (nuoarchive restore) with the --restore-dir option.

NuoDB Archive options relevant to restoring one or more incremental hot copies from a backup set.

nuoarchive restore [<restore options>] [<nuodb options>] <location>
restore options are:
--restore-dir Restore into this destination directory

Examples: running incremental hot copy into a backup set

Before incremental hot copy can be executed, a backup set containing a full hot copy must exist.

nuocmd hotcopy database --db-name test --type full --default-backup-dir /tmp/hotcopy/2017-12-18

If we inspect the file system, we can see that the backup set directory is populated with a full hot copy.

$ ls -At1 /tmp/hotcopy/2017-12-18

To make the incremental hot copy interesting, we will insert a table row using autocommit into a table we already created prior to the full hot copy. The foo table has a single integer column.

SQL> insert into foo values(1);

Now we can perform a hot copy with type set to incremental. It will copy only those atoms changed since the full hot copy.

nuocmd hotcopy database --db-name test --type incremental --default-backup-dir /tmp/hotcopy/2017-12-18

The backup set now contains an incremental hot copy.

$ ls -At1 /tmp/hotcopy/2017-12-18

We can compare what atoms were copied in the full and incremental hot copies. The incremental copied fewer atoms.

$ find /tmp/hotcopy/2017-12-18/full/data -name '*.atm' | wc --lines
$ find /tmp/hotcopy/2017-12-18/ -name '*.atm' | wc --lines

Incremental hot copy can be executed again. If no changes have been made in the database, no atom files will be copied.

nuodb [test] > hotcopy database test host localhost pid 23588 backupSetDirectory /tmp/hotcopy/2017-12-18 type incremental
$ find /tmp/hotcopy/2017-12-18/ -name '*.atm' | wc --lines

Restoring the latest incremental hot copy from a backup set

When restoring from a backup set, specify the directory to copy the restored archive to using with the --restore-dir option. The directory provided for --restore-dir must not exist, or must be empty. For more information, see Restoring to Specific Backup Elements and Restoring Entire Backup Set.