Setting the ping-timeout Database Option

Engine-level failure detection

To enable failure detection, set the ping-timeout n database option, where n is the timeout interval in seconds. NuoDB recommends an interval timeout of 60 seconds or greater. For more information on ping-timeout, see Database Options.

With failure detection enabled using ping-timeout, when network communication with a process times out or the network connection is shut down unexpectedly, the process is evicted from the database by its peers. See Setting Up Failure Detection.

Broker-level failure detection

Brokers and agents ping their connected peer(s) periodically. A peer sends a warning to its agent log if it does not receive acknowledgment of its initial ping after 10 seconds, and then continually after that.

By default, heartbeatPeerTimeout is set to 45, which means a peer waits 45 seconds for a ping acknowledgment before disconnecting a non-responsive peer. For a peer to automatically disconnect a non-responsive peer, set heartbeatPeerTimeout to a value greater than the value set for the RaftMaxElectionTimeout property. The default setting for RaftMaxElectionTimeout is 8300 (milliseconds).

Note: Although set to 45 seconds by default, NuoDB recommends a minimum setting of 20 seconds for the heartbeatPeerTimeout property.

When heartbeatPeerTimeout is set to a value greater than 0 and that length of time elapses before this peer receives a ping acknowledgement from another peer then this peer closes the session and disconnects with the non-responsive peer.

A broker reports disconnected peers to all reachable brokers in the domain. Disconnected peers periodically try to reconnect. A broker tries to peer back into the domain according to the durable domain configuration. A broker tries to reconnect to another broker in the domain.

On this peer’s host, execution of the NuoDB Manager show domain hosts command indicates that the non-responsive peer and any database processes (TEs, SMs, SSMs) associated with the non-responsive peer are UNREACHABLE. The status of a peer as unreachable does not change the durable domain configuration. See NuoDB Manager Show Domain