Migrate from File Archives to Log-Structured Archives (LSA)

LSA provides better performance than File Archive for write-intensive workloads. For more information on archive types, see Archive.

This section describes the steps to migrate from File archive to LSA.

During the migration, archives are replaced one at a time. To avoid reducing the resilience of the database during the migration, a new archive of type LSA is created and an additional Storage Manager (SM) is added to the database to perform archive sync before an SM is shut down and File archive deleted. So, these migration steps require resources for one additional archive and SM to be provisioned prior to the migration. These additional resources can be deprovisioned after migration is complete.

Steps to Migrate from File Archive to LSA

To migrate from File Archive to archive of type LSA, follow these steps for each archive.

View the details of the archives.
```
nuocmd show archives --db-name <db_name>
```
For archives of type File, the archive location may have a prefix file: or the prefix may be empty. Select an archive of type File and note the archive ID. This File archive will be replaced by an archive of type LSA.
Create a new archive of type LSA with a new archive location.
```
nuocmd create archive --db-name <db_name> --server-id <server id> --archive-path lsa:<path>
```
An existing archive or journal path must not be used as the archive path for LSA.

Start an SM on the newly-created archive.

nuocmd start database --db-name test --incremental

Wait for the new SM to reach the RUNNING state.
Shut down the SM running on the File archive selected in step 1. If there are no SMs running on this archive, skip to the step 6.
```
nuocmd shutdown process --start-id <start_id>
```

Delete the File Archive.

nuocmd delete archive --archive-id <archive-id> --purge

[Optional] Delete the storage associated with the archive deleted in step 6.
Repeat step 1 to step 7 until all the File Archives are replaced by archives of type LSA.

Example: Migrate from Two File Archives to Two Archives of Type LSA

This example shows the steps to replace two File archives with two archives of type LSA.

Setup:

A running database called test.
Two SMs and one Transaction Engine (TE) in the RUNNING state.
All the SMs run on the same host.
Each SM runs on a File Archive.

Assumption: There is free storage space at the location where the new archive is created to store a third copy of the database. For more information, see note.

First, list the processes running in the database:

nuocmd show domain

server version: 6.0-beta-2-946711ad9f, server license: Enterprise
server time: 2024-05-13T15:28:37.533, client token: f5ff3f9fe27d16664849e47d19beb73a0ee101c9
Servers:
  [nuoadmin1] nuoadmin1:48005 [last_ack = 0.32] ACTIVE (LEADER, Leader=nuoadmin1, log=0/23/23) Connected *
Databases:
  test [state = RUNNING]
    [SM] nuoadmin1:48007 [start_id = 3] [server_id = nuoadmin1] [pid = 190] [node_id = 2] [last_ack =  6.05] MONITORED:RUNNING
    [SM] nuoadmin1:48006 [start_id = 4] [server_id = nuoadmin1] [pid = 197] [node_id = 1] [last_ack =  7.14] MONITORED:RUNNING
    [TE] nuoadmin1:48008 [start_id = 5] [server_id = nuoadmin1] [pid = 243] [node_id = 3] [last_ack =  2.42] MONITORED:RUNNING

View the details of the archives used by existing SMs, use nuocmd show archives.

For example :

nuocmd show archives --db-name test

[0] nuoadmin1 : /var/opt/nuodb/archive/file1 @ test [journal_path = ] [observer_storage_groups = []] RUNNING
  [SM] nuoadmin1:48007 [start_id = 3] [server_id = nuoadmin1] [pid = 190] [node_id = 2] [last_ack =  4.98] MONITORED:RUNNING
[1] nuoadmin1 : /var/opt/nuodb/archive/file2 @ test [journal_path = ] [observer_storage_groups = []] RUNNING
  [SM] nuoadmin1:48006 [start_id = 4] [server_id = nuoadmin1] [pid = 197] [node_id = 1] [last_ack =  6.07] MONITORED:RUNNING

There are two SMs each running on a File Archive.

The output shows that the archive used by the SMs are File Archives located at /var/opt/nuodb/archive/file1 and /var/opt/nuodb/archive/file2.

Create a new archive of type LSA.

nuocmd create archive --db-name test --server-id nuoadmin1 --archive-path lsa:/var/opt/nuodb/archive/testlsa1

Archive(archive_path=lsa:/var/opt/nuodb/archive/testlsa1, db_name=test, id=2, is_observer=False, observer_storage_groups=[], server_id=nuoadmin1, state=PROVISIONED)

To view the details of the archives:

nuocmd show archives --db-name test

[0] nuoadmin1 : /var/opt/nuodb/archive/file1 @ test [journal_path = ] [observer_storage_groups = []] RUNNING
  [SM] nuoadmin1:48007 [start_id = 3] [server_id = nuoadmin1] [pid = 190] [node_id = 2] [last_ack =  3.81] MONITORED:RUNNING
[1] nuoadmin1 : /var/opt/nuodb/archive/file2 @ test [journal_path = ] [observer_storage_groups = []] RUNNING
  [SM] nuoadmin1:48006 [start_id = 4] [server_id = nuoadmin1] [pid = 197] [node_id = 1] [last_ack =  4.89] MONITORED:RUNNING
[2] nuoadmin1 : lsa:/var/opt/nuodb/archive/testlsa1 @ test [journal_path = ] [observer_storage_groups = []] PROVISIONED

Archive 2 is the newly created archive with the location lsa:/var/opt/nuodb/archive/testlsa1.

Start an SM on the newly-created archive.

nuocmd start database with the incremental option starts a new SM on the newly provisioned archive of type LSA.

nuocmd start database --db-name test --incremental

STARTING: StartProcessRequest(archive_id=2, db_name=test, engine_type=SM, expected_incarnation_major=1, expected_incarnation_minor=0, labels={}, options={}, server_id=nuoadmin1)

To view the processes running on the domain:

nuocmd show domain

server version: 6.0-beta-2-946711ad9f, server license: Enterprise
server time: 2024-05-13T15:30:31.446, client token: 14c01f10c990b33954666f27de55f964e54c50a0
Servers:
  [nuoadmin1] nuoadmin1:48005 [last_ack = 0.04] ACTIVE (LEADER, Leader=nuoadmin1, log=0/31/31) Connected *
Databases:
  test [state = RUNNING]
    [SM] nuoadmin1:48007 [start_id = 3] [server_id = nuoadmin1] [pid = 190] [node_id = 2] [last_ack =  6.56] MONITORED:RUNNING
    [SM] nuoadmin1:48006 [start_id = 4] [server_id = nuoadmin1] [pid = 197] [node_id = 1] [last_ack =  0.99] MONITORED:RUNNING
    [TE] nuoadmin1:48008 [start_id = 5] [server_id = nuoadmin1] [pid = 243] [node_id = 3] [last_ack =  6.29] MONITORED:RUNNING
    [SM] nuoadmin1:48009 [start_id = 6] [server_id = nuoadmin1] [pid = 293] [node_id = 4] [last_ack =  1.55] MONITORED:SYNCING

To view the details of the archives:

nuocmd show archives --db-name test

[0] nuoadmin1 : /var/opt/nuodb/archive/file1 @ test [journal_path = ] [observer_storage_groups = []] RUNNING
  [SM] nuoadmin1:48007 [start_id = 3] [server_id = nuoadmin1] [pid = 190] [node_id = 2] [last_ack =  0.15] MONITORED:RUNNING
[1] nuoadmin1 : /var/opt/nuodb/archive/file2 @ test [journal_path = ] [observer_storage_groups = []] RUNNING
  [SM] nuoadmin1:48006 [start_id = 4] [server_id = nuoadmin1] [pid = 197] [node_id = 1] [last_ack =  1.23] MONITORED:RUNNING
[2] nuoadmin1 : lsa:/var/opt/nuodb/archive/testlsa1 @ test [journal_path = ] [observer_storage_groups = []] RUNNING
  [SM] nuoadmin1:48009 [start_id = 6] [server_id = nuoadmin1] [pid = 293] [node_id = 4] [last_ack =  5.63] MONITORED:SYNCING

The output shows that there are three archives. Archives 0 and 1 are of type File, and archive 2 is of type LSA.

Wait for the new SM to reach the RUNNING state.

Shut down the SM running on archive 0.

nuocmd shutdown process --start-id 3

nuocmd show archives --db-name test

[0] nuoadmin1 : /var/opt/nuodb/archive/file1 @ test [journal_path = ] [observer_storage_groups = []] NOT_RUNNING
  [SM] nuoadmin1:48007 [start_id = 3] [server_id = nuoadmin1] [pid = 190] [node_id = 2] EXITED(REQUESTED_SHUTDOWN:SHUTTING_DOWN):(2024-05-13T15:31:26.281+0000) Gracefully shutdown engine (0)
[1] nuoadmin1 : /var/opt/nuodb/archive/file2 @ test [journal_path = ] [observer_storage_groups = []] RUNNING
  [SM] nuoadmin1:48006 [start_id = 4] [server_id = nuoadmin1] [pid = 197] [node_id = 1] [last_ack =  3.12] MONITORED:RUNNING
[2] nuoadmin1 : lsa:/var/opt/nuodb/archive/testlsa1 @ test [journal_path = ] [observer_storage_groups = []] RUNNING
  [SM] nuoadmin1:48009 [start_id = 6] [server_id = nuoadmin1] [pid = 293] [node_id = 4] [last_ack =  7.24] MONITORED:RUNNING

SM 3 is shut down gracefully.

Delete the archive 0.

Delete the archive using nuocmd delete archive with the purge option.

nuocmd delete archive --archive-id 0 --purge

To view the details of the remaining archives:

nuocmd show archives --db-name test

[1] nuoadmin1 : /var/opt/nuodb/archive/file2 @ test [journal_path = ] [observer_storage_groups = []] RUNNING
  [SM] nuoadmin1:48006 [start_id = 4] [server_id = nuoadmin1] [pid = 197] [node_id = 1] [last_ack =  3.99] MONITORED:RUNNING
[2] nuoadmin1 : lsa:/var/opt/nuodb/archive/testlsa1 @ test [journal_path = ] [observer_storage_groups = []] RUNNING
  [SM] nuoadmin1:48009 [start_id = 6] [server_id = nuoadmin1] [pid = 293] [node_id = 4] [last_ack =  8.40] MONITORED:RUNNING

Delete the archive directory /var/opt/nuodb/archive/file1 from disk.

Create a new archive of type LSA.

nuocmd create archive --db-name test --server-id nuoadmin1 --archive-path lsa:/var/opt/nuodb/archive/testlsa2

Archive(archive_path=lsa:/var/opt/nuodb/archive/testlsa2, db_name=test, id=3, is_observer=False, observer_storage_groups=[], server_id=nuoadmin1, state=PROVISIONED)

To view the details of the archives:

nuocmd show archives --db-name test

[1] nuoadmin1 : /var/opt/nuodb/archive/file2 @ test [journal_path = ] [observer_storage_groups = []] RUNNING
  [SM] nuoadmin1:48006 [start_id = 4] [server_id = nuoadmin1] [pid = 197] [node_id = 1] [last_ack =  0.91] MONITORED:RUNNING
[2] nuoadmin1 : lsa:/var/opt/nuodb/archive/testlsa1 @ test [journal_path = ] [observer_storage_groups = []] RUNNING
  [SM] nuoadmin1:48009 [start_id = 6] [server_id = nuoadmin1] [pid = 293] [node_id = 4] [last_ack =  5.34] MONITORED:RUNNING
[3] nuoadmin1 : lsa:/var/opt/nuodb/archive/testlsa2 @ test [journal_path = ] [observer_storage_groups = []] PROVISIONED

Archive 1 is of type File, and archives 2 and 3 are of type LSA.

To view the details of the processes running in the domain:

nuocmd show domain

server version: 6.0-beta-2-946711ad9f, server license: Enterprise
server time: 2024-05-13T15:33:17.923, client token: daef3d4de0f316e73c3b2078eda9cd783d103ad5
Servers:
  [nuoadmin1] nuoadmin1:48005 [last_ack = 0.24] ACTIVE (LEADER, Leader=nuoadmin1, log=0/37/37) Connected *
Databases:
  test [state = RUNNING]
    [SM] nuoadmin1:48006 [start_id = 4] [server_id = nuoadmin1] [pid = 197] [node_id = 1] [last_ack =  7.39] MONITORED:RUNNING
    [TE] nuoadmin1:48008 [start_id = 5] [server_id = nuoadmin1] [pid = 243] [node_id = 3] [last_ack =  2.67] MONITORED:RUNNING
    [SM] nuoadmin1:48009 [start_id = 6] [server_id = nuoadmin1] [pid = 293] [node_id = 4] [last_ack =  1.81] MONITORED:RUNNING

Start an SM on the newly-created archive.

nuocmd start database with the incremental option starts a new SM on the newly provisioned archive of type LSA.

nuocmd start database --db-name test --incremental

STARTING: StartProcessRequest(archive_id=3, db_name=test, engine_type=SM, expected_incarnation_major=1, expected_incarnation_minor=1, labels={}, options={}, server_id=nuoadmin1)

To view the processes running on the domain:

nuocmd show domain

server version: 6.0-beta-2-946711ad9f, server license: Enterprise
server time: 2024-05-13T15:33:58.256, client token: ae68e0ef272753816c08220d7523f00627b3ff45
Servers:
  [nuoadmin1] nuoadmin1:48005 [last_ack = 0.49] ACTIVE (LEADER, Leader=nuoadmin1, log=0/43/43) Connected *
Databases:
  test [state = RUNNING]
    [SM] nuoadmin1:48006 [start_id = 4] [server_id = nuoadmin1] [pid = 197] [node_id = 1] [last_ack =  1.32] MONITORED:RUNNING
    [TE] nuoadmin1:48008 [start_id = 5] [server_id = nuoadmin1] [pid = 243] [node_id = 3] [last_ack =  1.32] MONITORED:RUNNING
    [SM] nuoadmin1:48009 [start_id = 6] [server_id = nuoadmin1] [pid = 293] [node_id = 4] [last_ack =  1.32] MONITORED:RUNNING
    [SM] nuoadmin1:48007 [start_id = 7] [server_id = nuoadmin1] [pid = 377] [node_id = 5] [last_ack =  0.14] MONITORED:SYNCING

nuocmd show archives --db-name test

[1] nuoadmin1 : /var/opt/nuodb/archive/file2 @ test [journal_path = ] [observer_storage_groups = []] RUNNING
  [SM] nuoadmin1:48006 [start_id = 4] [server_id = nuoadmin1] [pid = 197] [node_id = 1] [last_ack =  4.59] MONITORED:RUNNING
[2] nuoadmin1 : lsa:/var/opt/nuodb/archive/testlsa1 @ test [journal_path = ] [observer_storage_groups = []] RUNNING
  [SM] nuoadmin1:48009 [start_id = 6] [server_id = nuoadmin1] [pid = 293] [node_id = 4] [last_ack =  8.21] MONITORED:RUNNING
[3] nuoadmin1 : lsa:/var/opt/nuodb/archive/testlsa2 @ test [journal_path = ] [observer_storage_groups = []] RUNNING
  [SM] nuoadmin1:48007 [start_id = 7] [server_id = nuoadmin1] [pid = 377] [node_id = 5] [last_ack =  7.02] MONITORED:SYNCING

The output shows that there are three archives. Archive 1 is of type File, and archives 2 and 3 are of type LSA.

Wait for the new SM to reach the RUNNING state.

Shut down the SM running on archive 1.

nuocmd shutdown process --start-id 4

To view the details of the archives:

nuocmd show archives --db-name test

[1] nuoadmin1 : /var/opt/nuodb/archive/file2 @ test [journal_path = ] [observer_storage_groups = []] NOT_RUNNING
  [SM] nuoadmin1:48006 [start_id = 4] [server_id = nuoadmin1] [pid = 197] [node_id = 1] EXITED(REQUESTED_SHUTDOWN:SHUTTING_DOWN):(2024-05-13T15:34:39.764+0000) Gracefully shutdown engine (0)
[2] nuoadmin1 : lsa:/var/opt/nuodb/archive/testlsa1 @ test [journal_path = ] [observer_storage_groups = []] RUNNING
  [SM] nuoadmin1:48009 [start_id = 6] [server_id = nuoadmin1] [pid = 293] [node_id = 4] [last_ack =  0.27] MONITORED:RUNNING
[3] nuoadmin1 : lsa:/var/opt/nuodb/archive/testlsa2 @ test [journal_path = ] [observer_storage_groups = []] RUNNING
  [SM] nuoadmin1:48007 [start_id = 7] [server_id = nuoadmin1] [pid = 377] [node_id = 5] [last_ack =  6.57] MONITORED:RUNNING

The SM that was running on archive 1 was shut down gracefully.

Delete the archive 1.

nuocmd delete archive --archive-id  1 --purge

To view the details of the remaining archives:

nuocmd show archives --db-name test

[2] nuoadmin1 : lsa:/var/opt/nuodb/archive/testlsa1 @ test [journal_path = ] [observer_storage_groups = []] RUNNING
  [SM] nuoadmin1:48009 [start_id = 6] [server_id = nuoadmin1] [pid = 293] [node_id = 4] [last_ack =  6.28] MONITORED:RUNNING
[3] nuoadmin1 : lsa:/var/opt/nuodb/archive/testlsa2 @ test [journal_path = ] [observer_storage_groups = []] RUNNING
  [SM] nuoadmin1:48007 [start_id = 7] [server_id = nuoadmin1] [pid = 377] [node_id = 5] [last_ack =  4.31] MONITORED:RUNNING

To view the details of the domain:

nuocmd show domain

server version: 6.0-beta-2-946711ad9f, server license: Enterprise
server time: 2024-05-13T15:33:58.256, client token: ae68e0ef272753816c08220d7523f00627b3ff45
Servers:
  [nuoadmin1] nuoadmin1:48005 [last_ack = 0.49] ACTIVE (LEADER, Leader=nuoadmin1, log=0/43/43) Connected *
Databases:
  test [state = RUNNING]
    [TE] nuoadmin1:48008 [start_id = 5] [server_id = nuoadmin1] [pid = 243] [node_id = 3] [last_ack =  1.32] MONITORED:RUNNING
    [SM] nuoadmin1:48009 [start_id = 6] [server_id = nuoadmin1] [pid = 293] [node_id = 4] [last_ack =  1.32] MONITORED:RUNNING
    [SM] nuoadmin1:48007 [start_id = 7] [server_id = nuoadmin1] [pid = 377] [node_id = 5] [last_ack =  0.14] MONITORED:RUNNING

The database now has two archives of type LSA (and none of type File), and there is an SM running on each archive.

Delete the archive directory /var/opt/nuodb/archive/file2 from disk.