NuoDB Check

NuoDB Check (nuochk) is a command line utility that can do the following:

Description

Dropping Non-Referenced Database Objects

NuoDB Check traverses each database object (data, sequence, index, and so on) in the specified archive. Any atoms in the archive that are no longer referenced by any database object are dropped from the database and the relevant atom file is removed from the archive. NuoDB Check reclaims space attributed to dropped indexes, tables, sequences and also blobs.

Note that NuoDB automatically reclaims unused space within each atom at runtime, without the need for NuoDB Check. In the schema, any references to these atoms are then removed as well. This frees disk space and allows the database to operate more efficiently. Dropped items occur normally during the course of operations.

Validating and Reporting on Non-Dropped References

For database items that are not marked as dropped, NuoDB Check validates and reports on their correctness. It does this by loading each schema and traversing all objects referenced by it. Each object is checked for the following:

Implicit Whole Archive Validation

Implicit validation of the entire archive is done when all atoms are loaded by NuoDB Check. NuoDB Check then reports on those atoms that are incorrect. Validation consists of testing and reporting except in these cases:

If you are invoking NuoDB Check to restore to a snapshot then the utility validates and cleans up the source archive before it creates the new archive.

Duplicate Records on a Unique Index

NuoDB check also reports unique constraint violations when it reports/ repairs indexes.

Note: The --verbose flag is not required to report this information.

For example, if a unique index is missing valid records and if those records violate the unique constraint when added back to the index, a message similar to the following will be printed to stdout:

[ERROR] nuochk: Catalog test.te1 ID 6 Index C1 ID 3: Index is missing recordId 2
Ignoring unique dup exception while repairing indexes: Uniqueness violation encountered while populating index C1, key = 'foo1' for recordId 2

This indicates that the unique index has been violated, i.e. there are duplicate records in a table. To repair, you must select which duplicate records to delete in order to restore uniqueness.

Report timestamps

NuoDB Check supports an option called --report-timestamps that can be used to report every transaction that can be restored in a backup set, along with the timestamp at which it was committed. This timestamp was chosen by the TE that performed the commit, and it is chosen at pre-commit time. Timestamps will not appear in strict order when the database contains multiple TEs, instead they will be partially ordered by committing TE.

You may optionally specify a time window of interest to restrict the number of transactions output by this tool. To restrict the lower bound of timestamps, use the --start-time option. To restrict the upper bound of timestamps, use the --end-time option.

Note: Timestamps output by --report-timestamps are in GMT. The timestamps input to --start-time and --end-time must be in GMT and must be in the strict ISO 8601 combined date and time representation, that is, “YYYY-MM-DDTHH:MM:SS”.

NuoDB Check options relevant to --report-timestamps.

Usage: nuochk [--repair] [--quiet] [--full] [<nuodb options>] <location>
nuochk Options are:
<location> Archive or backup set location
--report-timestamps Report timestamp/transaction ID mappings
--end-time Upper end of time range for which to display timestamp/transaction mappings
--start-time Lower end of time range for which to display timestamp/transaction mappings

Restore snapshot

NuoDB Check supports an option called --restore-snapshot that can be used in conjunction with --restore-dir to restore a transaction. Pass the transaction id of the transaction to restore to --restore-snapshot and the directory into which the restored archive should be constructed to --restore-dir.

NuoDB Check options relevant to --restore-snapshot.

Usage: nuochk [--repair] [--quiet] [--full] [<nuodb options>] <location>
nuochk Options are:
--restore-dir Restore into this destination directory
--restore-snapshot Restore this snapshot (identified by transaction id)

Efficient point-in-time restore

It is important to give NuoDB Check enough memory during point-in-time restore, especially if the database being restored has large indexes. A simple rule of thumb is to restore on a host that is not currently running a database, and give NuoDB Checkas much memory as is normally given to an SM. Adjust the amount of memory given to NuoDB Check with the --mem option. See the reference guide for more details on NuoDB Check.

Using Safe Commit versus running NuoDB Check

While safe commit helps to prevent the creation of data inconsistencies (for example, errors that manifest as the "null descriptor" failure), it does not correct inconsistencies that are already written to cold atoms in the archive.

Running NuoDB Check finds and repairs issues such as a missing descriptor.

NuoDB Check can be run on the archive of a stopped storage manager while the database is running to look for issues without bringing the database down, so the initial check does not have to result in downtime. Repair can be done on individual tables, so the repair does not necessarily have to be slow.

See also: Validating Databases.

Syntax

nuochk [option [...] ] [archive_dir]

Arguments

Options

In addition to NuoDB Check command line options, NuoDB Check also accepts nuodb command line options. The more useful ones are listed here.

--snapshot-archive snapshot_archive_dir

Required if you are restoring to a snapshot. Specify the directory that contains the snapshot you want to restore to.

Usage

Ensure there are no storage managers or snapshot storage managers using the archive that is the source for tNuoDB Check. To validate an archive, you may want to start by running the command by itself without options to see the output. Any issues found will be reported but no changes will be made to the archive. A success message is printed if no issues are detected.

If necessary, you can run the command again with the --repair option to repair issues. If an SM or SSM is using that archive and NuoDB Check is run with --repair, NuoDB Check execution immediately stops with an error saying that an SM or SSM is using the archive. Similarly, if NuoDB Check is running on an archive (in any mode) then any attempt to start an SM or SSM on that archive fails. The SM or SSM does not start and reports that the archive is locked.

Caution: After running NuoDB Check with --repair, it is advisable to run it a second time to ensure no new issues are found. If you do find more issues being repaired when running NuoDB Check a second time, contact support@nuodb.com.

NuoDB Check must be invoked by the operating system user account that owns the archive directory. If NuoDB Check is invoked by another user, an error similar to the following error message is generated:

nuochk: Permissions error: archive owned by system account 'other_user'

Do not run NuoDB Check if a running SM or SSM is using the archive you want to validate.

NuoDB Check uses the following exit codes:

Exit Code Description
0 NuoDB Check has not found anything that requires change in the archive. All messages returned are INFO level.
1 NuoDB Check has not found any inconsistencies in the archive. However NuoDB Check has performed a clean-up of the archive, for example cleaning dropped objects, failed transactions and so on. This clean-up reduces disk usage and may improve performance, but has no impact on the consistency or correctness of the archive. All messages returned are WARN level.
2

NuoDB Check has found inconsistencies in the archive. All issues found were repairable; if run in repair mode then no further action is required. If problems are found, they are almost always in this category. Messages returned may be WARN or ERROR level. The more severe message takes precedence; if there are WARN and ERROR level messages, the exit code is 2 (or higher).

3 NuoDB Check has found inconsistencies in the archive that require further action for resolution; information on actions required is displayed. You must perform these actions, for example rebuilding of indexes, when the database is running. All messages returned are ERROR level.
123 NuoDB Check has discovered issues in the archive that it could not resolve. Do not use the archive again without contacting support@nuodb.com first. All messages returned are ERROR level.
124 Critical error. NuoDB Check could not complete due to an unexpected error trying to operate on the archive. Do not use the archive again without contacting support@nuodb.com first.
125 Invalid command line arguments. For example, the specification of non-existent, empty, or invalid archive or journal directories.

Note: The exit codes described above are used whether you run NuoDB Check in --repair mode or --dry-run mode. However in --repair mode, these codes indicate that operations required were actually performed, as opposed to --dry-run mode where the actions required are simply reported. If more than one type of issue is found, the highest applicable exit code is displayed.

Examples

This command invokes NuoDB Check on a valid SQL QuickStart database. See Running the SQL QuickStart.

$ nuochk /var/opt/nuodb/production-archives/test
[INFO ] nuochk: Validating Schema SYSTEM
[INFO ] nuochk: Validating Table HOCKEY.HOCKEY ID 59
[INFO ] nuochk: Validating Table HOCKEY.PLAYERS ID 61
[INFO ] nuochk: Validating Table HOCKEY.TEAMS ID 63
[INFO ] nuochk: Validating Table HOCKEY.SCORING ID 65
[INFO ] nuochk: Archive verification found no issues.

After artificially injecting some staleness in the SQL QuickStart database and running NuoDB Check again, the following information is provided:

$ nuochk /var/opt/nuodb/production-archives/test
[INFO ] nuochk: Validating Schema SYSTEM
[INFO ] nuochk: Validating Table HOCKEY.HOCKEY ID 59
[INFO ] nuochk: Validating Table HOCKEY.PLAYERS ID 61
[INFO ] nuochk: Validating Table HOCKEY.TEAMS ID 63
[INFO ] nuochk: Table ID 65 can be dropped
[INFO ] nuochk: Catalog ID 66 can be dropped

Running NuoDB Check again but with the --repair option has the following result:

$ nuochk --repair /var/opt/nuodb/production-archives/test
[INFO ] nuochk: Validating Schema SYSTEM
[INFO ] nuochk: Validating Table HOCKEY.HOCKEY ID 59
[INFO ] nuochk: Validating Table HOCKEY.PLAYERS ID 61
[INFO ] nuochk: Validating Table HOCKEY.TEAMS ID 63
[INFO ] nuochk: Table ID 65 was dropped
[INFO ] nuochk: Catalog ID 66 was dropped

$ nuochk /var/opt/nuodb/production-archives/test
[INFO ] nuochk: Validating Schema SYSTEM
[INFO ] nuochk: Validating Table HOCKEY.HOCKEY ID 59
[INFO ] nuochk: Validating Table HOCKEY.PLAYERS ID 61
[INFO ] nuochk: Validating Table HOCKEY.TEAMS ID 63
[INFO ] nuochk: Archive verification found no issues.

Running NuoDB Check again but with the --check-table option (to check only the hockey.scoring table) has the following result:

$ nuochk --check-table hockey.scoring /var/opt/nuodb/production-archives/test
[INFO ] nuochk: Validating Table HOCKEY.SCORING ID 120
[INFO ] nuochk: Cleanup skipped due to --check-table flag.
[INFO ] nuochk: Archive verification found no issues.

You may then add the --repair option to repair only the table defined.