Notes on Using Transparent Huge Pages

Transparent Huge Pages (THP) is a Linux feature that reduces the overhead of using large amounts of memory. Like other databases, NuoDB uses memory in such a way that enabling THP results in a slow but steady increase in Resident Set Size (RSS). This decreases the performance of NuoDB Transaction Engines (TEs) and Storage Managers (SMs) to a point where the system eventually runs out of memory and kills those processes using significant amounts of memory.

Caution: If you install NuoDB on a system where THP is enabled, NuoDB Transaction Engine (TE) and Storage Manager (SM) processes do not start and NuoDB sends an error message to its agent log. Such error messages are similar to the following:

2015-01-01T01:01:01.000-0000 [1600] NuoDB will not run with transparent hugepages enabled, please disable or set to [madvise]
2015-01-01T01:01:01.000-0000 [1600] See http://doc.nuodb.com/display/doc/Linux+Installation#LinuxInstallation-THP for details	

Determining Whether THP is Enabled

To determine whether THP is enabled on a system, run the following command:

$ cat /sys/kernel/mm/transparent_hugepage/enabled

If the content of the file is either madvise or never, THP is already disabled. If the content is [always] madvise never (as shown below), THP is enabled and you must disable it to run NuoDB.

$ cat /sys/kernel/mm/transparent_hugepage/enabled
[always] madvise never

Disabling THP

For information on disabling THP, see the following sections:

Disabling THP Until the Next Restart

To disable THP until the next system restart, run the following commands:

$ echo madvise | sudo tee -a /sys/kernel/mm/transparent_hugepage/enabled
$ echo madvise | sudo tee -a /sys/kernel/mm/transparent_hugepage/defrag

Note: While NuoDB recommends that you specify madvise, specifying never also enables NuoDB to work correctly.

If the response to the above (disabling) commands is similar to:

cannot create /sys/kernel/mm/transparent_hugepage/enabled: Read-only file system

then refer to Disabling THP on Read-Only File Systems.

Disabling THP Permanently

To permanently disable THP, do one of the following:

For either of these changes to take effect, you must restart your system. If it is not convenient to restart immediately after performing one of the above actions, see Disabling THP Until the Next Restart.

Editing the Kernel Bootloader

If you are using a bootloader, for example /etc/grub.conf, add the following to the end of the kernel boot line:

transparent_hugepage=never            

Caution: Before editing your bootloader configuration, refer to your system documentation.

Editing Your System Startup Script

First, identify your system startup script. This is likely to be one of the following:

After identifying your system startup script, edit the system-startup-file to add the following lines:

test -f /sys/kernel/mm/transparent_hugepage/enabled \
	&& echo madvise > /sys/kernel/mm/transparent_hugepage/enabled
test -f /sys/kernel/mm/transparent_hugepage/defrag \
	&& echo madvise > /sys/kernel/mm/transparent_hugepage/defrag

Note: You musts run the above commands as a root user.

To ensure that the startup script is executed when the system reboots, change the permissions of your system-startup-file with:

$ chmod +x system-startup-file

Disabling THP on Read-Only File Systems

On some hosts, the /sys directory is mounted as a read-only file system; the disabling of THP requires additional steps on a read-only file system. To determine whether /sys is mounted as a read-only file system, run the following command:

$ grep sysfs /proc/mounts        

You will see a line such as the following.

sysfs /sys sysfs ro,nosuid,nodev,noexec,relatime 0 0

If the first option is ro, as in this example, then the /sys directory is mounted as a read-only file system.

To disable THP on a read-only file system, do the following:

  1. Remount the /sys directory as read-write:

    $  sudo mount -o rw,remount /sys                    

  2. As a root user, change permissions as follows:

    $ chmod o+w /sys/kernel/mm/transparent_hugepage/enabled
    $ chmod o+w /sys/kernel/mm/transparent_hugepage/defrag				

Note: The steps documented in this section disable THP only until the next host restart. For information on permanently disabling THP, see Disabling THP Permanently.