Open topic with navigation
NuoDB Check (
nuochk) is a command line utility that can do the following:
NuoDB Check traverses each database object (data, sequence, index, and so on) in the specified archive. Any atoms in the archive that are no longer referenced by any database object are dropped from the database and the relevant atom file is removed from the archive. NuoDB Check reclaims space attributed to dropped indexes, tables, sequences and also blobs.
Note that NuoDB automatically reclaims unused space within each atom at runtime, without the need for NuoDB Check. In the schema, any references to these atoms are then removed as well. This frees disk space and allows the database to operate more efficiently. Dropped items occur normally during the course of operations.
For database items that are not marked as dropped, NuoDB Check validates and reports on their correctness. It does this by loading each schema and traversing all objects referenced by it. Each object is checked for the following:
Implicit validation of the entire archive is done when all atoms are loaded by NuoDB Check. NuoDB Check then reports on those atoms that are incorrect. Validation consists of testing and reporting except in these cases:
If you are invoking NuoDB Check to restore to a snapshot then the utility validates and cleans up the source archive before it creates the new archive.
NuoDB check also reports unique constraint violations when it reports/ repairs indexes.
--verbose flag is not required to report this information.
For example, if a unique index is missing valid records and if those records violate the unique constraint when added back to the index, a message similar to the following will be printed to stdout:
[ERROR] nuochk: Catalog test.te1 ID 6 Index C1 ID 3: Index is missing recordId 2 Ignoring unique dup exception while repairing indexes: Uniqueness violation encountered while populating index C1, key = 'foo1' for recordId 2
This indicates that the unique index has been violated, i.e. there are duplicate records in a table. To repair, you must select which duplicate records to delete in order to restore uniqueness.
NuoDB Check supports an option called
--report-timestamps that can be used to report every transaction that can be restored in a backup set, along with the timestamp at which it was committed. This timestamp was chosen by the TE that performed the commit, and it is chosen at pre-commit time. Timestamps will not appear in strict order when the database contains multiple TEs, instead they will be partially ordered by committing TE.
You may optionally specify a time window of interest to restrict the number of transactions output by this tool. To restrict the lower bound of timestamps, use the
--start-time option. To restrict the upper bound of timestamps, use the
Note: Timestamps output by --report-timestamps are in GMT. The timestamps input to
--end-time must be in GMT and must be in the strict ISO 8601 combined date and time representation, that is, “YYYY-MM-DDTHH:MM:SS”.
NuoDB Check options relevant to
Usage: nuochk [--repair] [--quiet] [--full] [<nuodb options>] <location>
nuochk Options are:
<location> Archive or backup set location
--report-timestamps Report timestamp/transaction ID mappings
--end-time Upper end of time range for which to display timestamp/transaction mappings
--start-time Lower end of time range for which to display timestamp/transaction mappings
NuoDB Check supports an option called --restore-snapshot that can be used in conjunction with --restore-dir to restore a transaction. Pass the transaction id of the transaction to restore to --restore-snapshot and the directory into which the restored archive should be constructed to --restore-dir.
NuoDB Check options relevant to --restore-snapshot.
Usage: nuochk [--repair] [--quiet] [--full] [<nuodb options>] <location>
nuochk Options are:
--restore-dir Restore into this destination directory
--restore-snapshot Restore this snapshot (identified by transaction id)
It is important to give NuoDB Check enough memory during point-in-time restore, especially if the database being restored has large indexes. A simple rule of thumb is to restore on a host that is not currently running a database, and give NuoDB Checkas much memory as is normally given to an SM. Adjust the amount of memory given to NuoDB Check with the --mem option. See the reference guide for more details on NuoDB Check.
While safe commit helps to prevent the creation of data inconsistencies (for example, errors that manifest as the "null descriptor" failure), it does not correct inconsistencies that are already written to cold atoms in the archive.
Running NuoDB Check finds and repairs issues such as a missing descriptor.
NuoDB Check can be run on the archive of a stopped storage manager while the database is running to look for issues without bringing the database down, so the initial check does not have to result in downtime. Repair can be done on individual tables, so the repair does not necessarily have to be slow.
See also: Validating Databases.
option[...] ] [
Fully qualified path to the archive directory that will be checked for errors. It is not necessary to specify the
archive_dir option with the
If you are restoring to a snapshot then this option specifies the path of the archive directory that contains the snapshot you are restoring to. This archive directory can be the result of a hot copy on a snapshot storage manager or the result of a copy operation on a stopped snapshot storage manager.
In addition to NuoDB Check command line options, NuoDB Check also accepts
nuodb command line options. The more useful ones are listed here.
Displays a list of options.
Provides a version number.
Displays the most recent startup details for internal and support use.
Enables you to take a table name and perform checks only on that table. Running this command followed by SCHEMA.TABLE causes NuoDB Check to run validateDB only on that table in the given schema. You may also specify just a schema to be checked, that is, check all tables in the schema defined).
"." must be used. For example
"schema."for all tables in the defined schema, and
".table"for tables with the name defined across all schemas).
Enables you to take a table ID and perform checks only on that table.
This is required if this option was specified when the storage manager was started. The same
journal_location must be used.
If you are restoring to a snapshot then this option specifies the path of the journal directory that was the result of the operation that provided the snapshot you are restoring to.
Continues processing after encountering the first error.
Limits the amount of memory that NuoDB Check can consume, specified as megabytes (m) or gigabytes (g). The default is 2 gigabytes.
NuoDB Check will show only errors detected.
Recalculates statistics in each root index atom. Enabling this switch does not affect any other task performed by NuoDB Check. In
--dry-run mode, indexes where the stats are wrong by a significant margin (for example where the KEYCOUNT value is out by 15%).
Reports changes without applying (default option).
NuoDB Check repairs issues in the archive. The default behavior is that the utility shows errors but does not modify the archive.
Required if you are restoring to a snapshot. NuoDB Check performs clean up operations before it creates the new archive.
Required if you are restoring to a snapshot. Specify the file system directory that will contain the new archive. This directory must not exist.
Required if you are restoring to a snapshot. Specify the ID of the snapshot to restore to. You must have obtained this snapshot by executing the
nuodbmgr hotcopy command on a database's snapshot storage manager or by copying current state from a snapshot storage manager.
Displays a timestamp, a snapshot ID and an album number. This matches the output seen when querying system.snapshots.
Required if you are restoring to a snapshot. Specify the directory that contains the snapshot you want to restore to.
Ensure there are no storage managers or snapshot storage managers using the archive that is the source for tNuoDB Check. To validate an archive, you may want to start by running the command by itself without options to see the output. Any issues found will be reported but no changes will be made to the archive. A success message is printed if no issues are detected.
If necessary, you can run the command again with the
--repair option to repair issues. If an SM or SSM is using that archive and NuoDB Check is run with
--repair, NuoDB Check execution immediately stops with an error saying that an SM or SSM is using the archive. Similarly, if NuoDB Check is running on an archive (in any mode) then any attempt to start an SM or SSM on that archive fails. The SM or SSM does not start and reports that the archive is locked.
After running NuoDB Check with
--repair, it is advisable to run it a second time to ensure no new issues are found. If you do find more issues being repaired when running NuoDB Check a second time, contact firstname.lastname@example.org.
NuoDB Check must be invoked by the operating system user account that owns the archive directory. If NuoDB Check is invoked by another user, an error similar to the following error message is generated:
nuochk: Permissions error: archive owned by system account 'other_user'
Do not run NuoDB Check if a running SM or SSM is using the archive you want to validate.
NuoDB Check uses the following exit codes:
|0||NuoDB Check has not found anything that requires change in the archive. All messages returned are INFO level.|
|1||NuoDB Check has not found any inconsistencies in the archive. However NuoDB Check has performed a clean-up of the archive, for example cleaning dropped objects, failed transactions and so on. This clean-up reduces disk usage and may improve performance, but has no impact on the consistency or correctness of the archive. All messages returned are WARN level.|
NuoDB Check has found inconsistencies in the archive. All issues found were repairable; if run in repair mode then no further action is required. If problems are found, they are almost always in this category. Messages returned may be WARN or ERROR level. The more severe message takes precedence; if there are WARN and ERROR level messages, the exit code is 2 (or higher).
|3||NuoDB Check has found inconsistencies in the archive that require further action for resolution; information on actions required is displayed. You must perform these actions, for example rebuilding of indexes, when the database is running. All messages returned are ERROR level.|
|125||NuoDB Check has discovered issues in the archive that it could not resolve. Do not use the archive again without contacting email@example.com first. All messages returned are ERROR level.|
|126||Critical error. NuoDB Check could not complete due to an unexpected error trying to operate on the archive. Do not use the archive again without contacting firstname.lastname@example.org first.|
|127||Invalid command line arguments. For example, the specification of non-existent, empty, or invalid archive or journal directories.|
Note: The exit codes described above are used whether you run NuoDB Check in
--repair mode or
--dry-run mode. However in
--repair mode, these codes indicate that operations required were actually performed, as opposed to
--dry-run mode where the actions required are simply reported. If more than one type of issue is found, the highest applicable exit code is displayed.
This command invokes NuoDB Check on a valid SQL QuickStart database. See Running the SQL QuickStart.
nuochk /var/opt/nuodb/production-archives/test[INFO ] nuochk: Validating Schema SYSTEM [INFO ] nuochk: Validating Table HOCKEY.HOCKEY ID 59 [INFO ] nuochk: Validating Table HOCKEY.PLAYERS ID 61 [INFO ] nuochk: Validating Table HOCKEY.TEAMS ID 63 [INFO ] nuochk: Validating Table HOCKEY.SCORING ID 65 [INFO ] nuochk: Archive verification found no issues.
After artificially injecting some staleness in the SQL QuickStart database and running NuoDB Check again, the following information is provided:
nuochk /var/opt/nuodb/production-archives/test[INFO ] nuochk: Validating Schema SYSTEM [INFO ] nuochk: Validating Table HOCKEY.HOCKEY ID 59 [INFO ] nuochk: Validating Table HOCKEY.PLAYERS ID 61 [INFO ] nuochk: Validating Table HOCKEY.TEAMS ID 63 [INFO ] nuochk: Table ID 65 can be dropped [INFO ] nuochk: Catalog ID 66 can be dropped
Running NuoDB Check again but with the
--repair option has the following result:
nuochk --repair /var/opt/nuodb/production-archives/test[INFO ] nuochk: Validating Schema SYSTEM [INFO ] nuochk: Validating Table HOCKEY.HOCKEY ID 59 [INFO ] nuochk: Validating Table HOCKEY.PLAYERS ID 61 [INFO ] nuochk: Validating Table HOCKEY.TEAMS ID 63 [INFO ] nuochk: Table ID 65 was dropped [INFO ] nuochk: Catalog ID 66 was dropped $
nuochk /var/opt/nuodb/production-archives/test[INFO ] nuochk: Validating Schema SYSTEM [INFO ] nuochk: Validating Table HOCKEY.HOCKEY ID 59 [INFO ] nuochk: Validating Table HOCKEY.PLAYERS ID 61 [INFO ] nuochk: Validating Table HOCKEY.TEAMS ID 63 [INFO ] nuochk: Archive verification found no issues.
Running NuoDB Check again but with the
--check-table option (to check only the
hockey.scoring table) has the following result:
nuochk --check-table hockey.scoring /var/opt/nuodb/production-archives/test[INFO ] nuochk: Validating Table HOCKEY.SCORING ID 120 [INFO ] nuochk: Cleanup skipped due to --check-table flag. [INFO ] nuochk: Archive verification found no issues.
You may then add the
--repair option to repair only the table defined.