About the Safe Commit Protocol

When you start a Transaction Engine (TE), the safe commit protocol is enabled by default. NuoDB strongly recommends that you continue to use the safe commit protocol.

Note: The commit protocol is set using the “--commit” option. If you have changed the default commit protocol and would like to revert to using the safe commit protocol, restart the TE(s) using “--commit”>safe<. Alternatively, open the database's capture file to change the “--commit” option to >safe< then restart the database.

Note: The capture file omits configuration options whose value is not the default. If you have never defined a commit protocol, the capture file does not define it.

See also: Start Process and Restart Database.

What the Safe Commit Protocol Does

For commits that insert, delete or update data, the safe commit protocol works as follows:

  1. The client requests a commit.
  2. Pre-commit is only sent to storage managers that are leader candidates for storage groups modified by the transaction.
  3. Each available storage manager acknowledges the pre-commit.
  4. The transaction engine sends a commit message to all nodes.

Note: If a storage group goes offline, an ongoing transaction that modifies it is not be resolved until that storage group comes back online (or is deleted). If the commit fails after all modified storage groups come back online, the following error message is returned:

Transaction NNNNN failed because storage groups X, Y, Z went offline during commit.

Note: Even though a database uses the safe commit protocol, transactions may still be unsuccessful. As with all commit protocols, acknowledgment of the pre-commit indicates that the transaction is visible to other clients. With safe commit, this acknowledgment also guarantees that the transaction commit is durable.

Durability Under Safe Commit

Failure of a database process (TE or SM) is typically transient in that it can be restarted. For example, if a SM fails (perhaps due to power failure), you can usually resolve the failure (restore power) and restart the SM. In rare cases, storage media can suffer permanent failure that prevents the SM from being restarted. This results in the permanent loss of an archive or journal. A permanently failed database process cannot be restarted and so cannot resume serving the database. In the event that a database process has permanently failed, the solution is to replace the failed process with a running database process. For example replacing a failed SM with a running SM.

An archive that permanently fails is no longer available. An unavailable archive may prevent a database being able to enforce durability. This can happen if you need to perform a cold restart of the database and the missing, failed archive is the only archive that contains one or more updates.

Note: Where a database process fails and nodes in the domain can no longer communicate with each other, this may lead to a network partition condition. For information on network partition, see Examples of Failure Scenarios.

Examples of Safe Commit Behavior

Consider the following database configuration:

Scenario 1 - Permanent Storage Loss without Cold Restart

In this scenario, there is a permanent storage loss but a cold restartFor a given database, all processes are shut down before the database is restarted. is not needed. The database continues to run and durability is not violated.

  1. TE1 commits Transaction 1 (T1).
  2. SM1 crashes but the disk is not lost. SM2 continues running.
  3. TE1 commits Transaction 2 (T2).
    This is allowed because when max-lost-archives is set to 0 only one SM needs to be running for insert, delete or update transactions t o be committed. At this time, only SM2 has T2.
  4. SM1 is restarted by the DBA.
  5. SM1 synchronizes with SM2.
    At this time, both SM1 and SM2 have T2.
  6. SM2 crashes and cannot be restarted.
  7. SM3 is started by the DBA.
  8. SM3 synchronizes with SM1.
    At this time, both SM1 and SM2 have T2.

Scenario 2 - Permanent Storage Loss with Cold Restart

In this scenario, a sequence of failures (perhaps due to a power outage) has occurred. The permanent failure of SM2 has resulted in the loss of T2, therefore a cold restart is required. Durability is violated because T2 was committed on only one archive (SM2) and that archive was permanently lost before another archive could synchronize with it.

  1. TE1 commits T1.
  2. SM1 crashes but the disk is not lost. SM2 continues running.
  3. TE1 commits transaction T2 which modifies the database.
    At this time, only SM2 has T2.
  4. SM2 crashes and cannot be restarted.
    As the only TE has failed, the database is down.
  5. SM1 is restarted by the DBA.
    SM1's archive defines the database.
  6. A TE is started by the DBA.
    T2 is lost.

Scenario 3 - Permanent Storage Loss with Cold Restart (and max-lost-archives set to 1)

The details of this scenario are the same as with scenario 2 but the "--max-lost-archives"option has been set to 1 instead of 0:

  1. TE1 commits T1.
  2. SM1 crashes but the disk is not lost. SM2 continues running.
  3. TE1 attempts to commit T2, which would modify the database.
    NuoDB does not allow this commit because when the "--max-lost-archives"option is set to 1, two SMs must be running in order to commit a transaction that updates, inserts or deletes data.
  4. SM2 crashes and cannot be restarted.
    As the only TE has failed, the database is down.
  5. SM1 is restarted by the DBA.
    This archive defines the database.
  6. A TE is started by the DBA.
    No transactions are lost.

In this scenario, durability is not violated. No transactions were committed when there was only one SM running. A cold restart was required and durability was maintained.

Comparing the Safe Commit Protocol with the Remote:N Commit Protocol

With the safe commit protocol, you can add or remove SMs without changing the database's configuration, and durability continues to be guaranteed. The safe commit protocol always requires a commit acknowledgment from each running SM.

With the remote:n commit protocol, you can replace n with the number of the database's storage managers, for example remote:2, and achieve the same durability guarantee as the safe commit protocol. You then have the same durability guarantee as if the database was using the safe commit protocol. To continue to enforce the durability guarantee after you add a storage manager, do the following:

  1. Shut down the running TEs.
  2. Restart the TEs with the “--commit” option set to remote:3.
  3. Add/start the third SM.

Comparing use of the Safe Compare Commit to running NuoBD Check

While the safe commit protocol helps avoid data inconsistencies, for example errors that manifest as the "null descriptor" failure, it does not correct inconsistencies already written to cold atoms in the archive. Therefore, if a database was running with a non-safe commit mode, and a corruption was introduced into the archive, switching to safe commit does not resolve the issue. To resolve the issue, you must first run nuochk, then run nuochk again using --repair to locate and repair the data inconsistency issues, such as missing descriptors. For more information on NuoDB Check, see Command Line Tools.