Open topic with navigation
NuoDB Check is a command line utility that can do the following:
nuochkutility can create an archive only when you are restoring to a snapshot or transaction.
NuoDB Check traverses each database object (data, sequence, index, and so on) in the specified archive. Any atoms in the archive that are no longer referenced by any database object are dropped from the database and the relevant atom file is removed from the archive. NuoDB Check reclaims space attributed to dropped indexes, tables, sequences and also blobs.
Note that NuoDB automatically reclaims unused space within each atom at runtime, without the need for NuoDB Check. In the schema, any references to these atoms are then removed as well. This frees disk space and allows the database to operate more efficiently. Dropped items occur normally during the course of operations.
For database items that are not marked as dropped, NuoDB Check validates and reports on their correctness. It does this by loading each schema and traversing all objects referenced by it. Each object is checked for the following:
Implicit validation of the entire archive is done when all atoms are loaded by NuoDB Check. NuoDB Check then reports on those atoms that are incorrect. Validation consists of testing and reporting except in these cases:
If you are invoking
nuochk to restore to a snapshot then the utility validates and cleans up the source archive before it creates the new archive.
NuoDB check also reports unique constraint violations when it reports/ repairs indexes.
--verbose flag is not required to report this information.
For example, if a unique index is missing valid records and if those records violate the unique constraint when added back to the index, a message similar to the following will be printed to stdout:
[ERROR] nuochk: Catalog test.te1 ID 6 Index C1 ID 3: Index is missing recordId 2 Ignoring unique dup exception while repairing indexes: Uniqueness violation encountered while populating index C1, key = 'foo1' for recordId 2
This indicates that the unique index has been violated, i.e. there are duplicate records in a table. To repair, you must select which duplicate records to delete in order to restore uniqueness.
NuoDB Check (nuochk) supports an option called --report-timestamps that can be used to report every transaction that can be restored in a backup set, along with the timestamp at which it was committed. This timestamp was chosen by the TE that performed the commit, and it is chosen at pre-commit time. Timestamps will not appear in strict order when the database contains multiple TEs, instead they will be partially ordered by committing TE.
You may optionally specify a time window of interest to restrict the number of transactions output by this tool. To restrict the lower bound of timestamps, use the option --start-time. To restrict the upper bound of timestamps, use the option --end-time.
Note: Timestamps output by --report-timestamps are in GMT.
The timestamps input to --start-time and --end-time must be in GMT and must be in the strict ISO 8601 combined date and time representation, i.e., “YYYY-MM-DDTHH:MM:SS”.
NuoDB Check options relevant to --report-timestamps.
Usage: nuochk [--repair] [--quiet] [--full] [<nuodb options>] <location>
nuochk Options are:
<location> Archive or backup set location
--report-timestamps Report timestamp/transaction ID mappings
--end-time Upper end of time range for which to display timestamp/transaction mappings
--start-time Lower end of time range for which to display timestamp/transaction mappings
NuoDB Check (nuochk) supports an option called --restore-snapshot that can be used in conjunction with --restore-dir to restore a transaction. Pass the transaction id of the transaction to restore to --restore-snapshot and the directory into which the restored archive should be constructed to --restore-dir.
NuoDB Check options relevant to --restore-snapshot.
Usage: nuochk [--repair] [--quiet] [--full] [<nuodb options>] <location>
nuochk Options are:
--restore-dir Restore into this destination directory
--restore-snapshot Restore this snapshot (identified by transaction id)
It is important to give NuoDB Check (nuochk) enough memory during point-in-time restore, especially if the database being restored has large indexes. A simple rule of thumb is to restore on a host that is not currently running a database, and give nuochk as much memory as is normally given to an SM. Adjust the amount of memory given to nuochk with the --mem option. See the reference guide for more details on nuochk.
While safe commit helps to prevent the creation of data inconsistencies (e.g., errors that manifest as the "null descriptor" failure), it does not correct inconsistencies that are already written to cold atoms in the archive.
Running nuochk should find and repair issues such as a missing descriptor.
Nuochk can be run on the archive of a stopped storage manager while the database is running to look for issues without bringing the database down, so the initial check does not have to result in downtime. Repair can be done on individual tables, so the repair does not necessarily have to be slow.
See also: Validating Databases.
option[...] ] [
Fully qualified path to the archive directory that will be checked for errors. It is not necessary to specify the
archive_dir option with the
If you are restoring to a snapshot then this option specifies the path of the archive directory that contains the snapshot you are restoring to. This archive directory can be the result of a hot copy on a snapshot storage manager or the result of a copy operation on a stopped snapshot storage manager.
In addition to
nuochk command line options, the
nuochk utility also accepts
nuodb command line options. The more useful ones are listed here.
Continues processing after encountering the first error.
Displays a list of options.
Required if you are restoring to a snapshot. Specification of this option instructs the
nuochk utility to read the journal from the directory specified for the
This is required if this option was specified when the storage manager was started. The same
journal_location must be used.
If you are restoring to a snapshot then this option specifies the path of the journal directory that was the result of the operation that provided the snapshot you are restoring to.
Limits the amount of memory that
nuochk will consume, specified as megabytes (m) or gigabytes (g). The default is 2 gigabytes.
nuochk utility will show only errors detected.
nuochk utility will repair issues in the archive. The default behavior is that the utility shows errors but does not modify the archive.
Required if you are restoring to a snapshot. The
nuochk utility performs clean up operations before it creates the new archive.
Required if you are restoring to a snapshot. Specify the file system directory that will contain the new archive. This directory must not exist.
Displays a time stamp along with a snapshot id and album number. This matches the output seen when querying system.snapshots.
Required if you are restoring to a snapshot. Specify the ID of the snapshot to restore to. You must have obtained this snapshot by executing the
nuodbmgr hotcopy command on a database's snapshot storage manager or by copying current state from a snapshot storage manager.
Required if you are restoring to a snapshot. Specify the directory that contains the snapshot you want to restore to.
Enables you to take a table name (and/or ID) and perform checks only on that table. Running this command followed by SCHEMA.TABLE will cause nuochk to run validateDB only on that table in the given schema
Provides a version number.
Displays the most recent startup details for internal and support use.
Displays the historical startup details for internal and support use.
--report-timestamps --start-time "yyyy-mm-ddThh:mm:ss" --end-time "yyyy-mm-ddThh:mm:ss" --format simple /path/to/archive
The tool walks through the journal files and prints out transactionId/timestamp pairs that fall within the time window.
The --start-time and --end-time are optional parameters. Start time defaults to January 1st 1970 (0 in UTC) and end time defaults to January 19th 2038 (UINT32_MAX in Posix time).
Make sure no storage managers or snapshot storage managers are using the archive that is the source for the
nuochk utility. To validate an archive, you may want to start by running the command by itself without options to see the output. Any issues found will be reported but no changes will be made to the archive. A success message is printed if no issues are detected.
If necessary, you can run the command again with the
--repair option to repair issues. If an SM or SSM is using that archive and
nuochk is run with
nuochk execution immediately stops with an error saying that an SM or SSM is using the archive. Similarly, if
nuochk is running on an archive (in any mode) then any attempt to start an SM or SSM on that archive fails. The SM or SSM does not start and reports that the archive is locked.
--repair, it is advisable to run it a second time to ensure no new issues are found. If you do find more issues being repaired when running
nuochk the second time, contact NuoDB support at firstname.lastname@example.org.
nuochk utility must be invoked by the operating system user account that owns the archive directory. If
nuochk is invoked by another user, an error similar to the following error message is generated:
nuochk: Permissions error: archive owned by system account 'other_user'
Do not run
nuochk if a running SM or SSM is using the archive you want to validate.
nuochk utility uses the following exit codes:
|0||No issues were detected with the archive and no changes needed to be made. The archive is valid and consistent.|
|1||Some changes needed to be made in the archive (for example, unused atoms were detected). Those changes were made and the result is a valid and consistent archive.|
The utility failed to complete and validate the archive. For example, this could be because of an invalid command line argument or an unrecoverable archive format.
This command invokes nuochk on a valid SQL QuickStart database. See Running the SQL QuickStart.
nuochk /var/opt/nuodb/production-archives/testValidating Schema SYSTEM Validating Table HOCKEY.HOCKEY ID 59 Validating Table HOCKEY.PLAYERS ID 61 Validating Table HOCKEY.TEAMS ID 63 Validating Table HOCKEY.SCORING ID 65 Archive verification found no issues.
After artificially injecting some staleness in the SQL QuickStart database and running
nuochk again, the following information is provided:
nuochk /var/opt/nuodb/production-archives/testValidating Schema SYSTEM Validating Table HOCKEY.HOCKEY ID 59 Validating Table HOCKEY.PLAYERS ID 61 Validating Table HOCKEY.TEAMS ID 63 Table ID 65 can be dropped Catalog ID 66 can be dropped
nuochk again but with the
--repair option has the following result:
nuochk --repair /var/opt/nuodb/production-archives/testValidating Schema SYSTEM Validating Table HOCKEY.HOCKEY ID 59 Validating Table HOCKEY.PLAYERS ID 61 Validating Table HOCKEY.TEAMS ID 63 Table ID 65 was dropped Catalog ID 66 was dropped $
nuochk /var/opt/nuodb/production-archives/testValidating Schema SYSTEM Validating Table HOCKEY.HOCKEY ID 59 Validating Table HOCKEY.PLAYERS ID 61 Validating Table HOCKEY.TEAMS ID 63 Archive verification found no issues.