Host Properties (default.properties)

The default.properties file sets configuration options for the NuoDB broker or agent. It resides in the NUODB_HOME/etc directory.

The domain and domainPassword properties are required. Brokers and agents do not start unless these properties are set. The other properties in default.properties are optional.

Leading and trailing whitespace in the setting of all properties in default.properties is ignored.

Caution: For Windows users, the default is that you do not have write access to the NuoDB installation directory. If you try to save the default.properties file without write permissions, you receive an error message that states that another program has access to the file. To resolve this, grant write permissions to the NUODB_HOME/etc directory:
  1. In Windows Explorer, right-click C:\Program Files\NuoDB.
  2. Select Properties > Security > Advanced.
  3. In the Advanced Security Settings for NuoDB dialog, click Change Permissions to display the Change Permission dialog.
  4. In Permission entries, select the user category to be granted write access, and click Edit.
  5. In the Permission Entry for NuoDB dialog, select Allow for Write attributes.
  6. Click OK in each dialog to close them all.
Name Default Usage

advertiseAlt={true|false}

false

This flag specifies whether the alternate address should be advertised instead of the locally observed network addresses (as determined by a call to java.net.InetAddress.getLocalHost()). This is meaningful only for brokers, since only brokers advertise addresses to clients and other brokers. See also altAddr below.

When a broker advertises its own alternate address, it also advertises the alternate address of each broker to which it is peered and which also has altAddr set. This enables a domain to be set up such that:

  • One set of brokers runs within a region using locally observed network addresses that are not visible to clients that try to connection from outside the region.
  • A second set of two or more brokers advertise their alternate addresses and are available for client connections.

At startup, a broker or agent logs the address that is returned from InetAddress.getLocalHost() to the agent log as in the following example:

2015-01-19T15:33:33.833+0000 INFO PeerContainerImpl.<init> (main) peer 
unstableId=[8343c7db-d60a-4382-a40c-6c0c68d7ac3e], 
peer=[54.149.94.33:48004], port=48004, raft=true, 
address=[ip-172-31-5-193/172.31.5.193:48004], addrToAdvertise=[54.148.81.156]

agentPeerTimeout=n

-1

Note that in this release, each host in a domain runs a broker.

This property applies only to the initial entry of a broker into the domain. By default, a broker shuts down if its attempt to peer into the domain fails. By setting this property, a broker continues to try to peer into the domain until this timeout (in milliseconds) has elapsed. If the timeout elapses and the broker has not been able to peer into the domain then the broker shuts down.

To preserve backward compatibility, the default is to not retry (-1).

This option is not related to agent reconnect when its peer broker disconnects.

altAddr=alt_addr

Not set

An alternate address to use in identifying this host. To advertise this address, set the advertiseAlt property to true.

automationTemplate=template_name Single Host

If enableSystemDatabase is set to true then use this property to specify the template to use to manage the nuodb_system database.

balancer=policy

ChainableModBalancer

A property specifying the SQL connection load balancer that this broker should use. The balancer determines how the broker chooses which transaction engine (TE) to use for a SQL client connection. This property has no effect on an agent.

Set the balancer property to one or more load balancer policies separated by commas. For more information, see Balancing Database Load Across Hosts.

Available load balancers are:

  • RoundRobinBalancer
  • ChainableModBalancer
  • ChainableRegionBalancer
  • ChainableReqGroupBalancer
  • ChainableTagBalancer
  • ChainableLocalityBalancer
  • ModBalancer (deprecated)
  • RegionBalancer (deprecated)
  • AffinityBalancer (deprecated)

binDir=directory

Not set

The location of the directory that holds the nuodb executable. This is needed only if you are using a non-standard NuoDB distribution directory structure.

broker={true|false}

true

Specifies whether this agent should be run as a broker. In this release, this should always be true because the use of agents is not supported.

chainableReqGroupBalancerPropName=name LBReq

Specifies the name of the client connection property that specifies the process group that you want the broker to use when ChainableReqGroupBalancer is in the value of the host's balancer property.

Setting the chainableReqGroupBalancerPropName host property to the name of any other connection property overrides the LBReq connection property in client connection requests.

See Balancing Database Load Across Hosts and Connection Properties.

domain=domain_name

domain

The name used to identify the domain to which this broker or agent belongs. This property must be set for the broker or agent to start.

domainPassword=domain_password

Not set

Password used by this broker or agent to join the domain. This property must be set for the broker or agent to start.

Every broker and agent in a domain uses the same password to peer into the domain. In other words, the domainPassword property is set to the same value on every host in the domain. This enables:

  • Every broker and agent to securely connect.
  • Bootstrapping creation of the initial domain administrator account. See About Domain Credentials.

For security reasons, there is no default password. As with all properties in default.properties, leading and trailing whitespace is ignored.

enableResync={true|false} true If this property is set to false, the state reconciliation behavior is disabled, which means that the DSMResync barrier is disabled. Setting this property to false may result in stale last known state if processes were shut down while this broker was offline. See How Creation of Extra Processes is Avoided.

enableSystemDatabase={true|false}

false

If this property is set to true then when the broker on this host starts it provisions the optional nuodb_system database. If the automationTemplate property is also set then the nuodb_system database is created and managed according to the template specified by the automationTemplate property.

However, if the nuodb_system database has already been provisioned by another broker in the domain then the setting of this property is ignored.

Among other things, the nuodb_system database stores metrics for hosts and database processes. These metrics are available in the Automation Console and by means of REST API calls. Currently, the NuoDB Manager utility does not provide access to these metrics.

Typically, you provision the nuodb_system database when you create the domain by starting the first broker. But you can also run a domain without provisioning the nuodb_system database. If enableSystemDatabase is set to false (or if the nuodb_system database has already been provisioned) then the nuodb_system database is not provisioned when this broker starts. At any time, to provision the nuodb_system database, set the enableSystemDatabase property to true and optionally specify a template for creating and managing it. Then restart the broker.

Note: If you are using Transport Layer Security, there is no domain account. enableSystemDatabase propertyuses the hardcoded domain account when creating the nuodb_system database. In a TLS domain, this database has to be created manually specifying a named user/pass pair.

heartbeatPeerTimeout=n 45 seconds

Brokers and agents ping their connected peer(s) periodically. A peer sends a warning to its agent log if it does not receive acknowledgement of its initial ping after 10 seconds and then continually after that.

The default behavior, heartbeatPeerTimeout=45, is that a peer waits for a ping acknowledgement for up to 45 seconds. If an acknowledgement is not received before 45 seconds elapses then the non-responsive peer is disconnected.

A setting of heartbeatPeerTimeout=0 means that a peer waits for a ping acknowledgement forever and does not disconnect a non-responsive peer.

For a peer to disconnect a non-responsive peer, the heartbeatPeerTimeout property must be set to a value greater than the setting of the RaftMaxElectionTimeout property. The default setting for that property is 8300 milliseconds. NuoDB recommends a minimum setting of 20 seconds for the heartbeatPeerTimeout property.

A broker reports disconnected peers to all reachable brokers in the domain. Disconnected peers periodically try to reconnect. A broker tries to peer back into the domain according to the durable domain configuration. An agent tries to reconnect to another broker in the domain.

On this peer's host, execution of the NuoDB Manager show domain hosts command indicates that any non-responsive peer and any database processes (TEs, SMs, SSMs) associated with the non-responsive peer are UNREACHABLE. The status of a peer as unreachable does not change the durable domain configurationThe durable domain configuration provides domain configuration information that is stored consistently on each broker in the domain by means of a Raft log..

hostTags=tag=value[, tag=value]...

Not set

Set this property to inject host tags into the broker or agent. The format is a comma-separated list of key=value pairs, with each string token being trimmed. It is also possible to specify a key without a value as in "tag" or "tag=". Examples:

hostTags = tag1=value1, tag2, tag3=value3

hostTags = tag1= value1, tag2=, tag3=value3

localBindAddress Not set

For internal use only. Do not set this property.

log=level INFO

The log level for the broker or agent log output. Valid levels are, from most verbose to least verbose:

ALL, FINEST, FINER, FINE, CONFIG, INFO, WARNING, SEVERE, OFF

metricsPurgeAge=n 12h

Remove metrics by age. Specify an integer followed by d for day, h for hour, or m for minute.

nodeStartTimeoutSec 30 seconds The number of seconds that NuoDB waits for a process (TE, SM, SSM) to start. You should never set this property to be less than 30 seconds.

peer=peer[,peer]...

Not set

Specifies the host name and optionally a port for an existing broker, already running in the domain, to which this broker or agent should connect when joining the domain. A broker accepts only a single peer specification. An agent accepts a comma-separated list of peers.

Be sure to peer all brokers to the same broker. On restart, a broker tries to connect to the broker specified by its peer setting. If that peer is unreachable, the broker falls back to its last-known state of brokers. If you are using non-default port settings, be sure to include the port number with the name or address of the peer.

Caution: Do not set this property when using the NuoDB Community Edition.

peerWaitInterval=n 10 seconds When a broker starts it tries to connect to the broker specified by the peer property. If the initial try to connect fails then this broker tries to connect again every peerWaitInterval seconds until connection is successful or the period of time specified by the peerWaitTimeout property elapses.
peerWaitTimeout=n 120 seconds When a broker starts it must connect to the domain peer specified by the peer property before the period of time specified by the peerWaitTimeout property elapses. If the broker cannot connect during that period of time then the broker shuts down.

port=port_number

48004

The port that this broker or agent listens on for all incoming connections. If no value is specified, the broker or agent randomly selects an available port on which to listen. Enable this setting when you need to limit the openings in your firewall and AWS security policies.

portRange= start[,end]

48005

A range of port numbers that nuodb executables should use.

Specifying a start without an end indicates that process TCP/IP ports are assigned incrementally from the start without limit.

Each new process (transaction engine, storage manager, snapshot storage manager) that is started on a machine is communicated with by means of an assigned TCP/IP port that is specified through this property. Ensure firewall rules allow access from other machines.

procssStartWaitBarrier All

Enables the ProcessReconnect barrier and the DSMResync barrier for all databases (managed and unmanaged). This prevents split-braining a database.

Upon restart, a broker or agent runs a Domain State Machine resync protocol to remove process entries for any database processes that shut down while the local broker or agent was not running. Also, the restarted broker or agent waits for processes to reconnect and then reconciles the process ID list with the process entries in the DSM. Starting a database process is allowed only after the process reconnect window and the DSM resync. It can be as much as 10 seconds for any local process to reconnect. See also How Creation of Extra Processes is Avoided.

raft true You must not set this internal-use-only property to false.
RaftHeartbeatTimeout=n 500 milliseconds

These properties affect election of a leader broker.

The leader broker sends a heartbeat to other brokers in the domain at intervals specified by the setting of the RaftHeartbeatTimeout property. If the leader broker becomes unreachable, then the remaining brokers elect a new leader. Each broker attempts to go from follower to candidate to leader.

If messages in the agent log indicate that leadership is not stable, you might want to adjust the settings of these properties according to the following rules:

  • RaftHeartbeatTimeout should be significantly higher than the latency.
  • RaftMinElectionTimeout should be significantly higher than the RaftHeartbeatTimeout.
  • RaftMaxElectionTimeout should be higher than RaftMinElectionTimeout.

While maintaining these rules, these adjustments might be helpful:

  • If elections are being triggered too often then try increasing the RaftHeartbeatTimeout and/or the RaftMinElectionTimeout.
  • If elections take too many rounds to be resolved (there are many Converting to Candidate messages with no Converting to Leader message) then try increasing RaftMaxElectionTimeout, which increases the difference between RaftMinElectionTimeout and RaftMaxElectionTimeout.

All timeout specifications are in milliseconds. For an understanding of the behavior specified by the settings of the RaftMinElectionTimeout and RaftMaxElectionTimeout properties, see the Raft paper, In Search of an Understandable Consensus Algorithm.

Also also About Brokers and Peering for more information about the leader broker.

RaftMinElectionTimeout=n 4100 milliseconds
RaftMaxElectionTimeout=n 8300 milliseconds
RaftLogCompactionThreshold=n 512 entries

Every update to the broker's durable domain configuration causes a Raft log entry. This property lets you set a maximum number of entries to be stored in the log. When the broker starts, it reads all log entries to compute the current set of state machines. This can get expensive if you have a large number of entries, which are also held in memory.

RaftMembershipCmdMaxRetry 60 For internal use only. Do not change the setting.
region=region_name Not set User-supplied name of this broker or agent's region. This property is useful for Region distributed domains. The region of a host should not be changed after it has been set. In general, the region name should reflect the actual physical location of the host itself. For example, region=US_East.
regionBalancerLocalFallback false

Use this property with the ChainableRegionBalancer policy. When regionBalancerLocalFallback is set to false (the default) and a connection request does not specify the LBRegion connection property then it is a connection error. This is regardless of whether or not there is a subsequent balancer policy in a specified chain.

When regionBalancerLocalFallback is set to true:

  • When a connection request does not specify the LBRegion connection property then the region of the broker is used.

  • When a connection request specifies the LBRegion connection property then the specified region is used.

See Balancing Database Load Across Hosts.

removeSelfOnShutdown={true|false}

false

When set to true, this broker or agent removes itself from the durable domain configuration when it terminates.

When set to false, this broker or agent does not remove itself from the durable domain configuration when it terminates.

An alternative to using this host property is to invoke the NuoDB Manager shutdown host command. This command has an optional parameter for removing the specified host from the domain.

Not recommended for use in Production.

requireConnectKey={true|false}

false

This property specifies whether database processes (TEs, SMs, SSMs) can be started only through this broker or agent or can also be started directly by starting a nuodb executable on the host. The default behavior is that database processes can be started directly.

When set to true then a connection key is required to start all nuodb processes. A connection key is available only if the process was started through a request to the broker or agent. This is a best practice flag for locking down a system.

runEnforcerOnEveryEvent={true|false} true

The default behavior is that most domain events cause the enforcer to run, regardless of settings for schEnforcerPeriodSec and schEnforcerInitDelaySec. Examples of events that cause the enforcer to run include: broker or agent joins domain, broker or agent leaves domain, transaction engine process joins, TE leaves, update to a database template. This behavior is useful in development/demonstration projects but should be disabled in production.

schEnforcerPeriodSec=n 10

This broker runs the enforcer every n seconds.

When the enforcer runs, one of the results is an indication of whether this broker is the leader broker.

Every broker runs the enforcer according to this property setting. However, it is only the leader broker that does anything as a result of running the enforcer.

schEnforcerInitDelaySec=n 15 After this broker starts, it runs the enforcer for the first time after n seconds. Thereafter, this broker runs the enforcer every schEnforcerPeriodSec seconds.

singleHostDbRestart={true|false}

false

The default behavior is that single host databases are not automatically restarted when this broker or its host restarts.

When set to true, this property enables auto-restart of single host databases upon broker or host restart. To accomplish this, the broker writes a marker file into the var directory, for example, /var/opt/nuodb/demo-archives, when a database starts. When the host or the broker restarts, the broker starts any database for which a marker exists. A database shutdown operation removes the marker.

This property applies to only unmanaged databases. That is, databases that were created without a template and so are not governed by the enforcer.

The default is false, which preserves backward compatibility.

statsIntervalSeconds 20

For internal use only. You should not change the setting.

trackProcessOnReconnect={true| false} false

This property is not included in the default.properties file but you can add it.

The default behavior is that reconnecting processes are already logged in the durable domain configuration because they would have been started by this broker. If the trackProcessOnReconnect property is set to true, the broker tries to add database processes to the durable domain configuration when the broker peers back into the domain. This should normally not be needed, but might be needed in the case where the management tier was shut down and re-provisioned. For example, the broker was restarted with ‑‑reset-broker-state while processes were still running.

Note: For information about the propertyProvider property, see PropertiesProvider Plugin Class Customization.