Adding a Process Group to a Built-in Template

You can add a process group to a built-in template. A process group always contains transaction engines or storage managers. This can be useful, for example, when some hosts are used for basic transaction processing while other hosts are best used for extract, transform and load (ETL) processing.

For example, you could define a process group that contains transaction engines to be used only for ETL. You can then assign a host tag, for example ETL, to hosts that have TEs to be used for ETL. A client connection request would specify the ETL host tag and NuoDB would direct the request to an ETL host.

When you add a process group to a built-in template you are changing the default behavior of databases that use that template. Any databases that already use that template and any databases that are created with that template include the new process group in their configuration.

Before you add a process group to a built-in template, ensure that the hosts for the new process group are provisioned. See Provisioning Hosts.

Information about adding a process group is organized as follows:

Instructions for Adding a Process Group

To add a process group to a built-in template:

  1. Invoke the NuoDB Manager show template command, specify the templateName parameter with the name of the built-in template you want to update (if it contains a space then enclose it in single quotes) and specify the file parameter with the name of an output file. For example:

    show template templateName 'Multi Host' file MultiHostTemplate

  2. Copy the built-in template to another file so that you have a copy of the built-in template before you update it. For example, copy it to MultHostTemplateOriginal. Since you are changing NuoDB defaults, a best practice is to have a copy of the provided, built-in template in case you want to fall back.
  3. In a JSON editor or a text editor, open the template you want to update. For example, if you saved the provided Multi Host template in MultiHostTemplateOriginal, then you would edit the MultHostTemplate file.
  4. In the editor, under the processes array, copy and paste either the SMs or TEs element.
  5. Change the name value in the copied element. For example, you could specify ETL.
  6. Update the new process group as needed. For example, you might want to apply mutually exclusive tag constraints to two process groups. One process group could use hosts that have the specified tag (ex) and the other process group could use hosts that do not have the specified tag (nx).
  7. Depending on how you specify the new process group, you might need to edit an existing process group. For example, you might need to specify that the TEs group can use only those hosts that do not have the ETL tag.
  8. Save the updated template.
  9. Invoke the NuoDB Manager update template command, specify the templateName parameter with the name of the built-in template you are updating and then specify the file parameter with the path of the file that contains the updated template. For example:

    update template templateName 'Multi Host' file MultiHostTemplate

In the Minimally Redundant, Multi Host and Region Distributed templates, the hostLimit database option is set to false. This means that on each of a database's hosts, NuoDB can start one process for each process group defined in the database's template. If hostLimit is set to true, only one database process is allowed per host per database.

Example of a New Process Group

Here is an example that adds the ETL process group to the Multi Host template. The additions are highlighted in bold, including a comment that describes the update.

"name" : "Multi Host",
"version" : 5,
"summary" : "This template starts up to two Storage Managers and as many Transaction 
     Engines as possible. Each host will have at most one SM and one TE, so this 
     template can be used with any number of hosts. A process will be restarted on 
     a new host if one goes offline and another is available.
     11-07-2015: We added a process group for TEs dedicated to ETL.",
"options" : {
  "commit" : "${COMMIT:remote:1}",
  "backoff.reqMinUptime" : "30000",
  "hostLimit" : "${HOST_LIMIT:false}"
},
"requirements" : [ {
  "region" : "${REGION}",
  "processes" : [ {
    "name" : "SMs",
    "type" : "SM",
    "min" : "${SM_MIN:1}",
    "max" : "${SM_MAX:2}",
    "options" : { },
    "tags" : { }
  }, {
    "name" : "TEs",
    "type" : "TE",
    "min" : "${TE_MIN:1}",
    "max" : "${TE_MAX:}",
    "options" : { },
    "tags" : { "ETL": "nx:" }
  }, {
    "name" : "ETL",
    "type" : "TE",
    "min" : "${TE_MIN:1}",
    "max" : "${TE_MAX:}",
    "options" : { },
    "tags" : { "ETL": "ex:"}
  } ]
} ]

Confirming That Processes in the Added Group are Running

After you add a process group, you can invoke the NuoDB Manager show database processes command to confirm that processes in the added group are running. Specify the reqGroupName option. For example:

nuodb [domain] > show database processes database foo inclOptions mem,reqGroupName
[SM] ip-172-31-34-74/54.88.137.63:60341 (us-west) [ pid = 18953 ] 
   [ db = foo ] [ nodeId = 2 ] [ mem = 1073741824 ] [ reqGroupName = SMs ] RUNNING
[TE] ip-172-31-34-74/54.88.137.63:60344 (us-west) [ pid = 18954 ] 
   [ db = foo ] [ nodeId = 3 ] [ mem = 1073741824 ] [ reqGroupName = TEs ] RUNNING
[TE] ip-172-31-34-74/54.88.137.63:60358 (us-west) [ pid = 18958 ] 
   [ db = foo ] [ nodeId = 5 ] [ mem = 1073741824 ] [ reqGroupName = ETL ] RUNNING
[SM] ip-172-31-34-74/54.88.10.23:60339 (us-west) [ pid = 18952 ] 
   [ db = foo ] [ nodeId = 1 ] [ mem = 1073741824 ] [ reqGroupName = SMs ] RUNNING
[TE] ip-172-31-34-74/54.88.10.23:60349 (us-west) [ pid = 18956 ] 
   [ db = foo ] [ nodeId = 4 ] [ mem = 1073741824 ] [ reqGroupName = TEs ] RUNNING
[TE] ip-172-31-34-74/54.88.10.23:60364 (us-west) [ pid = 18959 ] 
   [ db = foo ] [ nodeId = 6 ] [ mem = 1073741824 ] [ reqGroupName = ETL ] RUNNING

nuodb [domain] > 

In Automation Console and the REST API, it is a known limitation that process group names do not appear in runtime information displays. This is relevant only when you add a process group to a built-in template. If you add a process group then use the NuoDB Manager show database processes command to view the process configuration. Specify the user-data field so that you can see the group that each process belongs to. For example:

nuodb [domain] > show database processes database nuodb_system inclOptions user-data
[SM] Nuodb-2.local/192.168.2:60614 (us-west) [ pid = 35063 ] [ db = nuodb_system ] 
     [ nodeId = 1 ] [ user-data = reqGroupName=SMs ] RUNNING
[TE] Nuodb-2.local/192.168.2:60616 (us-west) [ pid = 35064 ] [ db = nuodb_system ] 
     [ nodeId = 2 ] [ user-data = reqGroupName=TEs ] RUNNING
[SM] Nuodb-1.local/192.168.1.102:60762 (us-west) [ pid = 35073 ] [ db = nuodb_system ] 
     [ nodeId = 3 ] [ user-data = reqGroupName=SMs ] RUNNING
[TE] Nuodb-1.local/192.168.1.102:60766 (us-west) [ pid = 35074 ] [ db = nuodb_system ] 
     [ nodeId = 4 ] [ user-data = reqGroupName=TEs ] RUNNING

Restoring an Original Template

To return to using the NuoDB-provided version of a built-in template, invoke the NuoDB Manager update template command, specify the templateName parameter with the name of the template you want to restore and specify the file parameter with the name of the file that contains the original built-in template. For example:

update template templateName 'Multi Host' file MultiHostTemplateOriginal

To confirm that the domain is now using the original, built-in template, invoke the NuoDB Manager show template command. For example:

show template templateName 'Multi Host' file MultiHostTemplateInUse

In the MultiHostTemplateInUse output file, you should now see only the TEs process group and the SMs process group.