NuoDB Archive - Checking

The NuoDB Archive check (nuoarchive check) command is used to perform the following tasks:

  • Clean the archive

    It removes non-referenced database objects from the archive.

  • Validate and report on non-dropped references

    For database references that are not marked as dropped, nuoarchive check validates and reports on their correctness. It loads each schema and traverses all the objects referenced by it. It validates that all items referenced by the object are present and accessible.

    Each object is checked for the following:

    • Referential integrity of the object itself

    • Correctness of type

    • Validity of the objects it references (such as presence and type of data atoms)

    • Validity of internal indexing

    • MVCC (Multi-Version Concurrency Control) version integrity

  • Validate the entire archive

    When all atoms are loaded by nuoarchive check, the checksum of every atom in the archive is validated and every atom is expanded into memory to ensure it is not corrupted. The command then reports on the incorrect atoms.

    Validation consists of testing and reporting except in the following cases:

    • Use of an index might result in the index being dropped and re-created

    • MVCC versions are corrected if needed

    • Any incorrect references are fixed

  • Report on unique constraint violations or duplicate records on a unique index

    The nuoarchive command reports unique constraint violations when it reports or repairs indexes. For example, if a unique index is missing valid records and the records violate the unique constraint when added back to the index, the following error message is reported:

    [ERROR] nuoarchive: Catalog test.te1 ID 6 Index C1 ID 3: Index is missing recordId 2
Ignoring unique dup exception while repairing indexes: Uniqueness violation encountered while populating index C1, key = 'foo1' for recordId 2

    To repair this and restore uniqueness, start the database and delete the records that duplicate the recordId indicated in the error message.

While safe commit helps to prevent the creation of data inconsistencies, such as errors that manifest as a "null descriptor" failure, it does not correct inconsistencies that are already written to cold atoms in the archive. nuoarchive check finds and repairs issues such as a missing descriptor. If your database uses more than one Storage Manager (SM), run nuoarchive check on the archive of a single stopped SM to look for issues without bringing the database down.

Since other SM(s) remain running, the database remains available and the initial check does not have to result in downtime. Also, repair can be done on individual tables, so the check does not necessarily have to be slow.

See also: Validating Databases.

Syntax

nuoarchive check [--help|--version]
nuoarchive check [--repair|--dry-run] [--show-stats] [--rewrite] [--list-snapshots] [--quiet] [--full]
                 [--check-tables <table>[,<table>...]] [--check-table-ids <tableId>[,<tableId>...]]
                 [--skip-tables <table>[,<table>...]] [--skip-table-ids <tableId>[,<tableId>...]]
                 [--recalculate-index-stats]
                 [--journal-dir <dir>]
                 [--<nuodb_option_name> <option_argument>]
                 <location>

Arguments

The arguments for nuoarchive check are the following:

--help:
Displays a list of options.

--version:
Shows the nuoarchive command version.

--repair:
Repairs issues in the archive. The default behavior is that the utility shows errors but does not modify the archive. The NuoDB Archive check command performs clean-up operations before it creates the new archive.

--dry-run:
Report the changes without applying them (default).

--show-stats:
Provide quantities of each type of atom in the archive and a size histogram of each atom type.

--rewrite:
Rewrite older atoms at the effective version.

--list-snapshots:
List restorable transactions and the album they reside in from newest to oldest.

--quiet:
Does not generate any output.

--full:
Continue after the first error.

--check-tables:
Check only the table with the specified table name(s).

--check-table-ids:
Check only the table with the specified table ID(s) or catalog ID(s).

--skip-tables:
Skip the table(s) with the specified table name(s).

--skip-table-ids:
Skip the table(s) with the specified table ID(s) or catalog ID(s).

--recalculate-index-stats:
Recalculate and update the statistics stored in each root index atom, and print the indexes where the statistics are incorrect by a significant margin—​for example, where the KEYCOUNT value is off by 15%. Enabling this switch does not affect any other task performed by NuoDB Archive. In --dry-run mode, indexes with wrong stats are printed but nothing is updated.

--journal-dir:
Provide the location of the journal directory, in case nuoarchive check cannot find it. This is needed when the journal is kept separately from the archive and is moved from the location where the archive expects it.

nuodb_option_name:
For more information, see Database Options.

location:
Fully qualified path to the archive directory that will be checked for errors. It is not necessary to specify the location option with the --help and --version options. To get the location of the archive, run nuocmd get archives.

Usage

  • Before performing nuoarchive check, ensure no SMs are using the archive that is the source for the nuoarchive check utility.

  • You must be logged in as the owner of the archive storage.

  • You must have access to the archive storage as a local directory:

    • In bare metal environments, use ssh <archive_owner>@<archive_host>

    • In containerized environments such as Docker, use docker exec -it <archive_container_name> bash

    • In Kubernetes environments, use kubectl exec -it <archive_container_name> bash

The nuoarchive check command must be invoked by the operating system user account that owns the archive directory. If another user invokes nuoarchive check, the following error message is generated:

nuoarchive: Permissions error: archive owned by system account 'other_user'

To run nuoarchive check on an encrypted archive (see Transparent Data Encryption) you must provide the storage password that was current when the archive was last in use. By default, nuoarchive check will ask for the storage password. The password is not displayed on the screen.

To provide the password non-interactively, use a Linux pipe command to send the password through standard input or set the storage password in the NUODB_STORAGE_PASSWORD environment variable.

If the password is not correct, nuoarchive check will fail.

To validate an archive, run the nuoarchive check <location> command. The following code shows a sample result.

nuoarchive check /var/opt/nuodb/archive
[INFO ] check: Validating Schema SYSTEM ID 4
[INFO ] check: Validating Ids HOCKEY.HOCKEY$IDENTITY_SEQUENCE ID 103
[INFO ] check: Validating Table HOCKEY.HOCKEY ID 101
[INFO ] check: Validating Table HOCKEY.PLAYERS ID 104
[INFO ] check: Validating Table HOCKEY.SCORING ID 108
[INFO ] check: Validating Table HOCKEY.TEAMS ID 106
[INFO ] check: Archive verification found no issues.

An error message, "[ERROR] check: Could not find a valid archive at the given location: <location>", indicates a mistake in the <location> specified or that the nuoarchive command was not run on the disk volume the SM was using. Check for errors in the specified <location> or connect to the appropriate host in your deployment environment (see Note) and re-run nuoarchive check.

To repair any issues, run the nuoarchive check --repair command. If an SM is using the archive and you run nuoarchive check --repair, nuoarchive check stops with an error message, "[ERROR] check: Archive is already in use: <location>". Similarly, if nuoarchive check is running on an archive (in any mode), then any attempt to start an SM on that archive fails. The SM does not start and reports that the archive is locked.

A few of the limitations of executing nuoarchive check on containerized deployments are the following:

  1. The nuoarchive check --repair command will return an error, as nuoacrchive cannot write to the archive while an SM is using the archive.

  2. Since the archive state changes while nuoarchive is running, there are chances for false error reports. Do not run nuoarchive if an SM is using the archive you want to validate.

After running nuoarchive check --repair, re-run it to ensure no new issues are found. If you find more issues being repaired when running nuoarchive check the second time, contact NuoDB Support.

Exit Codes

The NuoDB Archive check command uses the following exit codes:

Exit Code Description

0

NuoDB Archive has not found anything in the archive that requires change.

All messages returned are [INFO] level.

1

NuoDB Archive has not found any inconsistencies in the archive. However, nuoarchive check has performed a clean-up of the archive, for example, cleaning dropped objects, failed transactions, and so on. This clean-up reduces disk usage and improves performance and has no impact on the consistency or correctness of the archive.

All messages returned are [WARN] level.

2

NuoDB Archive has found inconsistencies in the archive. All issues found were repairable. If nuoarchive check was run with the --repair option, no further action is required.

All messages returned are [WARN] or [ERROR] level. The more severe message takes precedence; if there are WARN and ERROR level messages, the exit code is 2 (or higher).

3

The NuoDB Archive has found inconsistencies in the archive that require further action for resolution. Information on the required actions is displayed. You must perform these actions when the database is running.

All messages returned are [ERROR] level.

123

The NuoDB Archive has discovered issues in the archive that it could not resolve. Do not use the NuoDB Archive command again. Contact NuoDB Support.

All messages returned are [ERROR] level.

124

Critical error. NuoDB Archive could not complete due to an unexpected error trying to operate on the archive. Do not use the NuoDB Archive command again. Contact NuoDB Support.

125

Invalid command line arguments. For example, the specification of non-existent, empty, or invalid archive or journal directories.

  • The exit codes described above are used when nuoarchive check is run with the --repair option or the --dry-run option. When running nuoarchive check with the --dry-run option, the required actions are just reported and with the --repair option, the required actions were performed.

  • If more than one type of issue is found, the highest applicable exit code is displayed.

Examples

The following examples use the sample Ice Hockey database used in Quick Start Guides.

Example 1: Use nuoarchive check
nuoarchive check /var/opt/nuodb/archive
[INFO ] nuoarchive check: Validating Schema SYSTEM
[INFO ] nuoarchive check: Validating Table HOCKEY.HOCKEY ID 59
[INFO ] nuoarchive check: Validating Table HOCKEY.PLAYERS ID 61
[INFO ] nuoarchive check: Validating Table HOCKEY.TEAMS ID 63
[INFO ] nuoarchive check: Validating Table HOCKEY.SCORING ID 65
[INFO ] nuoarchive check: Archive verification found no issues.
In containerized environments such as Docker, run the docker exec command to make sure that the NuoDB Archive is running on the disk volume that the SM is using.
Example 2: Run nuoarchive check with the --repair option
nuoarchive check --repair /var/opt/nuodb/archive
[INFO ] nuoarchive check: Validating Schema SYSTEM
[INFO ] nuoarchive check: Validating Table HOCKEY.HOCKEY ID 59
[INFO ] nuoarchive check: Validating Table HOCKEY.PLAYERS ID 61
[INFO ] nuoarchive check: Validating Table HOCKEY.TEAMS ID 63
[INFO ] nuoarchive check: Table ID 65 was dropped
[INFO ] nuoarchive check: Catalog ID 66 was dropped
nuoarchive check /var/opt/nuodb/archive
[INFO ] nuoarchive check: Validating Schema SYSTEM
[INFO ] nuoarchive check: Validating Table HOCKEY.HOCKEY ID 59
[INFO ] nuoarchive check: Validating Table HOCKEY.PLAYERS ID 61
[INFO ] nuoarchive check: Validating Table HOCKEY.TEAMS ID 63
[INFO ] nuoarchive check: Archive verification found no issues.
Example 3: Run nuoarchive check with the --check-table option
nuoarchive check --check-table hockey.scoring /var/opt/nuodb/archive
[INFO ] nuoarchive check: Validating Table HOCKEY.SCORING ID 120
[INFO ] nuoarchive check: Cleanup skipped due to --check-table flag.
[INFO ] nuoarchive check: Archive verification found no issues.
Example 4: Run nuoarchive check to repair a table
nuoarchive check --repair --check-tables hockey.scoring /var/opt/nuodb/archive
[INFO ] nuoarchive check: Validating Table HOCKEY.SCORING ID 120
[INFO ] nuoarchive check: Cleanup skipped due to --check-table flag.
[INFO ] nuoarchive check: Archive verification found no issues.