Examples of Hot Copy Errors

Hot copy can fail in different ways. Failure of hot copy can be remedied by the user. Here, we list the most common errors and what they mean.

Caution: If there is a failure during execution of a hot copy then you must start a new hot copy operation. You cannot resume the failed operation. A failure can possibly result in an unexpected termination of the SM or SSM, or a disk failure.

/hotcopyResults directory does not exist

Suppose that the /hotcopyResults directory does not exist. Execution of the hotcopy command with an archive destination of /hotcopyResults/test_archive would fail and you would see a message such as the following:

nuodb [domain] > show process status host localhost pid 16689

NodeID: 1  NodeState: RUNNING  Clients: 0 
Hot Copy: Hot copy to /hotcopyResults/test_archive 
failed: Unable to create archive directory “/hotcopyResults/test_archive.incomplete" 
(No such file or directory)

In the previous error message, notice that the name of the archive directory that could not be created is /hotcopyResults/test_archive.incomplete. During hot copy execution, NuoDB adds the .incomplete extension to indicate that the hot copy is in progress. The extension is removed when the hot copy successfully completes. However, in the situation where the hot copy operation fails, you might find instances of the .incomplete extension in your file system. It is up to you to remove any directories and files with a .incomplete extension. NuoDB does not store the location of .incomplete directories and files.

No storage manager could be found matching search criteria

You can search for an SM to hot copy based on some search criteria (host, host:port, and pid). If a unique SM cannot be found, you could get an error message like this, meaning that no SM with any pid could be found at the specified host connected to the specified broker port.

nuodb [test] > hotcopy database test host localhost:48006 destinationArchiveDirectory /tmp/hotcopy8
Snapshot to copy, or firstSnapshot-lastSnapshot (optional):
Album to copy, or firstAlbum-lastAlbum, 0 means current state (optional):
Destination journal directory (optional):
Destination snapshot directory (optional):
Storage Manager process id (optional):
Timeout (ms/s/m/h/d/w) (optional):
hotcopy: Database has no storage managers: localhost:48006:<any>

You could also specify the pid of a non-existent host or a host that is not a valid target for hot copy.

nuodb [test] > hotcopy database test host localhost:48005 pid 33816 destinationArchiveDirectory /tmp/hotcopy8
Snapshot to copy, or firstSnapshot-lastSnapshot (optional):
Album to copy, or firstAlbum-lastAlbum, 0 means current state (optional):
Destination journal directory (optional):
Destination snapshot directory (optional):
Timeout (ms/s/m/h/d/w) (optional):
hotcopy: Process is not an SM and does not support hot copy functionality: localhost:48005:33816: host localhost:48005 pid 33816

Bad Hot Copy Location

Hot copy can fail because of a permission error.

nuodb [test] > hotcopy database test host localhost:48006 destinationArchiveDirectory /bad-location
Snapshot to copy, or firstSnapshot-lastSnapshot (optional):
Album to copy, or firstAlbum-lastAlbum, 0 means current state (optional):
Destination journal directory (optional):
Destination snapshot directory (optional):
Storage Manager process id (optional):
Timeout (ms/s/m/h/d/w) (optional):
Hot copy control node: host JohnDoe-MacBook-Pro-3.local:48006 pid 34355
hotcopy: Hot copy to Node 3 (host JohnDoe-MacBook-Pro-3.local:48006 pid 34355) /bad-location failed: Hot copy failed: Node 3: /bad-location.incomplete: 
Directory createdir failed 
Permission denied: archive: Hot copy failed: Node 3: /bad-location.incomplete: 
Directory createdir failed: Permission denied: archive

Failure of SM during Hot Copy

Hotcopy will fail if the SM engine process fails for any reason. Below is a sample:

nuodb [domain] > hotcopy database test host localhost pid 40399 destinationArchiveDirectory /tmp/hotcopy8
Snapshot to copy, or firstSnapshot-lastSnapshot (optional):
Album to copy, or firstAlbum-lastAlbum, 0 means current state (optional):
Destination journal directory (optional):
Destination snapshot directory (optional):
Timeout (ms/s/m/h/d/w) (optional):
Hot copy control node: host nuodbce pid 40399
hotcopy: Request failed with status running: Stage1: Begin: empty management response for Query Summary
nuodb [domain] >

Wrong database ID

Hot copy will fail if the SM engine process attempts to create a new backup on another database backup set directory. Below is an example of this failure:

nuodb [domain] > hotcopy database test host localhost pid 19553 backupSetDirectory 
'/var/opt/nuodb/production-archives/BackupSets/12-10-2017' type incremental
Snapshot to copy, or firstSnapshot-lastSnapshot (optional):
Album to copy, or firstAlbum-lastAlbum, 0 means current state (optional):
Hot copy plan (optional):
Destination directory (optional):
Destination journal directory (optional):
Destination snapshot directory (optional):
Storage Manager process id (optional):
Timeout (ms/s/m/h/d/w) (optional):
Hot copy control node: host nuodbce pid 19553
hotcopy: /var/opt/nuodb/production-archives/BackupSetsTest/full/control/state.xml: Element database ID is
 aba12f78-b035-9440-1581-2721e3330db1, expected 4f11b68a-0dfd-b543-0aab-1168b0782476
nuodb [domain] >

Using both old Mechanism and Backup Sets

Hot copy will fail if the command you run tries to use both backupSetDirectory and destinationArchiveDirectory parameters. Below is a sample:

nuodb [domain] > hotcopy database test host localhost backupSetDirectory pid 19553 
'/var/opt/nuodb/production-archives/BackupSetsTest/12-10-2017' destinationArchiveDirectory 
/var/opt/nuodb/production-archives/testbck type incremental
Snapshot to copy, or firstSnapshot-lastSnapshot (optional):
Album to copy, or firstAlbum-lastAlbum, 0 means current state (optional):
Destination journal directory (optional):
Destination snapshot directory (optional):
Storage Manager process id (optional):
Timeout (ms/s/m/h/d/w) (optional):
hotcopy: You cannot specify both backupSetDirectory and destinationArchiveDirectory parameters
nuodb [domain] >

Trying to run Incremental Backup before invoking Full Backup

Hot Copy will fail if an Incremental Backup is invoked before a Full Backup has been completed:

nuodb [domain] > hotcopy database test host localhost pid 19553 backupSetDirectory 
'/var/opt/nuodb/production-archives/BackupSets/12-10-2017' type incremental
Snapshot to copy, or firstSnapshot-lastSnapshot (optional):
Album to copy, or firstAlbum-lastAlbum, 0 means current state (optional):
Hot copy plan (optional):
Destination directory (optional):
Destination journal directory (optional):
Destination snapshot directory (optional):
Storage Manager process id (optional):
Timeout (ms/s/m/h/d/w) (optional):
Hot copy control node: host nuodbce pid 19553
hotcopy: Hot copy to Node 1 (host nuodbce pid 19553) /var/opt/nuodb/production-archives/BackupSets/12-10-2017 failed: 
Hot copy failed: Node 1: Could not find a valid archive at the 
given location: /var/opt/nuodb/production-archives/BackupSets/12-10-2017/full/control: 
Hot copy failed: Node 1: Could not find a valid archive at the 
given location: /var/opt/nuodb/production-archives/BackupSets/12-10-2017/full/control
nuodb [domain] >

Trying to run Journal Backup before invoking Full Backup

Likewise, Hot Copy will fail if Journal Backup is invoked before a Full Backup has been completed:

nuodb [domain] > hotcopy database errortest host localhost pid 19553 backupSetDirectory 
'/var/opt/nuodb/production-archives/BackupSets/12-10-2017' type journal
Snapshot to copy, or firstSnapshot-lastSnapshot (optional):
Album to copy, or firstAlbum-lastAlbum, 0 means current state (optional):
Hot copy plan (optional):
Destination directory (optional):
Destination journal directory (optional):
Destination snapshot directory (optional):
Storage Manager process id (optional):
Timeout (ms/s/m/h/d/w) (optional):
Hot copy control node: host nuodbce pid 19553
hotcopy: Cannot create backup set /var/opt/nuodb/production-archives/BackupSets/12-10-2017: invalid content tmp
nuodb [domain] >