Host Properties (nuoadmin.conf)

To configure a host, set the required values for properties in nuoadmin.conf. You can find nuoadmin.conf in the following locations:

Installation type Location

RPM / DEB

/etc/nuodb

TAR

$NUODB_HOME/etc

Windows MSI

%NUODB_HOME%/etc

Setting nuoadmin.conf properties

Using a text editor, define admin process (AP) settings using nuoadmin.conf properties. Some of the nuoadmin.conf properties are described in the below table. All properties are described in the configuration file shipped with the product or in the sample configuration file located at $NUODB_HOME/etc/nuoadmin.conf.sample.

Property Description Comments

initialMembership

A map of servers and details for APs that are allowed to create/bootstrap the domain. Each key is a server ID, with values such as that seen in Example 1 below. If one of the keys matches the value of ThisServerId, then this server can create/bootstrap the domain. If not, then it attempts to enter the domain indicated via a server ID in the initial membership. Alternatively, the peer property may be used to specify an entry peer. All members must have the same initial membership, unless peer is used to specify the entry peer.

The associated transport property is a host name or IP address and port number following the format ipaddr:port.

The initialMembership is immutable during the domain lifecycle. Changing it requires destroying the domain and bootstrapping a new one.

peer

An endpoint (host:port) through which new admin servers can join the membership. This can be the address of a specific admin server in the membership, or a load-balancer that is routing traffic to existing admin servers.

ThisServerId

A unique identifier for this admin process that is permanently assigned, for example server0.

Used by TEs to communicate with the APs, and used by SQL clients to obtain a SQL connection. It should be stable and can’t be changed without removing the durable Raft state. The raftlog file is located in $NUODB_VARDIR.

agentPort

The port used by SQL clients to obtain a connection from an AP and also used by database processes to communicate with an AP.

Default value is 48004.

adminPort

The port that APs use to communicate with each other.

Default value is 48005.

rest.port

The port used by the REST API service. If rest is not specified in otherServices, then this property is ignored.

Default is 8888.

portRange

The port range or the first port that engines should listen on. For each additional engine per host, a unique port number is assigned within the range. For example, a range can be specified using, 48006, 48010. Alternatively, 48006 can be specified as a starting port.

Default value is 48006 (indicating that the port range starts at 48006).

altAddr

The address which the AP stores in the raft membership, and the address that all other APs use to communicate with it. If it is not specified, an AP stores its hostname in the durable membership (which other APs can then use to communicate with it).

When set, the value for altAddr also displays for the host machine value when running the nuocmd show domain command.

If a host cannot be reached using the altAddr value, the server ID on that host shows as "Disconnected"

ssl

Used to enable or disable SSL (TLS) domain connection and membership authentication.

Default value is true. To disable TLS, set this property to false.

domainEntryClient.peerWaitTimeout

The maximum amount of time to wait, for a request for a peer to join an admin domain to complete.

Default is 60000 (60 seconds).

leaderAssignmentTimeout

The maximum amount of time to wait for all initialized archives to be started when assigning storage group leaders for a database.

Default is 60000 (60 seconds).

pendingProcessTimeout

The maximum amount of time to wait for a TE to connect to the AP after the TE is started.

Default is 5000 (5 seconds).

pendingReconnectTimeout

The maximum amount of time to wait for a TE to reconnect to the AP after the AP is restarted.

Default is 10000 (10 seconds).

broadcastLatencyMicros

The maximum period of time (microseconds) it would take the raft leader to send a heartbeat to all followers; the heartbeat period and election timeout range are both calculated as multiples of the broadcastLatencyMicros value, with the election timeout being an order of magnitude larger than the heartbeat period.

When working with large clusters (20 or more admin servers/processes), or high-latency environments, consider increasing the value set for this property.

The value set for broadcastLatencyMicros is an approximation of the time it would take for an AP to communicate with all of its peers in the admin server membership. It is used to heuristically derive parameters for the Raft algorithm, specifically the raft.heartbeatPeriodMillis, raft.minElectionTimeoutMillis, and raft.maxElectionTimeoutMillis properties, which can also be specified directly. For example, if there are ten APs spread evenly across two regions, and the intra-region server-to-server latency is 500 microseconds and the inter-region server-to-server latency is 1500 microseconds, a reasonable setting for broadcastLatencyMicros would be 9500 (4 * 500 + 5 * 1500).

(raft.heartbeatPeriodMillis is the period at which the leader sends heartbeat messages to followers to assert its leadership status. raft.minElectionTimeoutMillis and raft.maxElectionTimeoutMillis specify the range of the randomly-generated timeout values used by followers to start an election in the absence of a leader.)

generatedKeyAlgorithm

If the Admin is TLS-enabled and has a certificate with signing rights (i.e. the certificate has the attribute BasicConstraint:CA=true), then it will generate a new key-pair and certificate for every engine process that it starts, which it signs with its own key. generatedKeyAlgorithm controls the asymmetric encryption algorithm for the key-pair and can be either RSA or EC.

Default is RSA.

generatedKeyStrength

The strength of the generated key, which corresponds to a particular size in bits for each algorithm. The key strengths are WEAK, MEDIUM, STRONG, and VERY_STRONG. See the shipped nuoadmin.conf.sample for the corresponding key sizes for each algorithm type.

Default is MEDIUM (512 bits for RSA, 192 bites for EC).

Notes on Setting the peer and initialMembership Properties

Physical and VMware deployments

So there is no restriction to the endpoints that a new server can join through, the peer property can be set to a load-balancer in front of the admin servers currently running. However, when setting peer, a disadvantage is that there is a race condition on who the initial member is, which could lead to different servers bootstrapping different domains. This is solved by setting both peer and initialMembership. If the domain has not been bootstrapped yet, then it can only be bootstrapped by a majority of the servers displayed in initialMembership. If a server is not displayed in initialMembership, then it must join through the peer endpoint.

For NuoDB Admin step by step configuration guidelines, see Configuring NuoDB Admin.

OpenShift Deployments

In containerized environments, the docker entrypoint script to start the AP injects values into nuoadmin.conf based on environment variables. The NUODB_DOMAIN_ENTRYPOINT variable becomes the peer and the NUODB_BOOTSTRAP_SERVERID variable becomes the lone member in the initial membership.

For more information, see Admin Process.

Example 1

The following nuoadmin.conf code sample provides settings for an admin service defined on host server0 that is also the initial member:

    ...
    "initialMembership": {
        "server0" : { "transport" : "server0:48005", "version" : "0:10000" }
    },
    "ThisServerId": "server0",
    "altAddr": "server0",
    ...
As this is the first host, the value of ThisServerId appears in the initialMembership. The altAddr property is the address that is advertised by a particular admin process. The default value of altAddr is $(hostname), which expands to the hostname of the server. The ThisServerId property is an user-defined unique identifier for an admin process, and is typically chosen to match the altAddr, assuming that all hosts have stable and unique hostnames.
Once the domain has been bootstrapped (service started), you can’t change the value for initialMembership. In the event that you need to change this value, you must first remove the raftlog file (located in /var/opt/nuodb) and restart NuoDB Admin. This may be required in a scenario where you began with a single host configuration (using localhost), and later decided to change it to reflect a real host name.

Example 2

The use of the peer property enables late binding of the initial membership.

peer and initialMembership are only used if there is no durable Raft state. If there is durable Raft state, the membership from the recovered Raft state is used.

The peer property specifies an address (host and port) to query for the initial membership to use when entering the domain. Unlike the addresses in the initial membership, there is no requirement that the address specified by peer actually belongs to a server in the membership, which means that it could actually be the address of a load balancer (for example Nginx or an Elastic Load Balancer) that is brokering the existing set of admin servers.

    ...
    "peer": "server0",
    "ThisServerId" : "server0",
    "altAddr" : "server0",
    ...

Expansion of Files and Variables

It is possible to specify configuration properties in nuoadmin.conf that are not bound until the admin process is started.

The syntax ${VAR} can be used evaluate a JVM property or an environment variable. For example, the following entry can be used to specify the password used to protect an AP’s keystore:

    ...
    "keystore-password": "${NUODB_KEYSTORE_PASSWORD}",
    ...

This will evaluate to the JVM property NUODB_KEYSTORE_PASSWORD or the environment variable NUODB_KEYSTORE_PASSWORD. JVM properties take precedence over environment variables.

It is also possible to expand the contents of a file as a configuration property, using the syntax $(</path/to/file). For example, the following entry can be used to specify the password used to protect an AP’s keystore:

    ...
    "keystore-password": "$(</etc/nuodb/keys/keystore-password)",
    ...

This will expand the contents of /etc/nuodb/keys/keystore-password, removing any leading or trailing whitespace.

Variable and file expansion can be performed for the value of a key-value entry, but not the key. In other words, these expressions can only appear on right-hand side of a key-value entry.