Data Caching

Whenever an engine (TE or SM) loads an atom, it caches it for future use. The typical scenario is as follows.

The first time data is requested (typically a SELECT against a table), the necessary rows are requested by a TE from one of he SMs. The atoms corresponding to those rows are fetched from disk, sent across the network from the SM to the TE and the TE caches them in memory. Subsequent queries against the same table can reuse the cached data avoiding network and disk overhead.

If another TE requires the same data, it can choose to fetch it from either the first TE or from an SM. Fetching from a TE is preferred since the SM may have to read it from disk. Internal metadata (in the Catalog atoms) keeps track of which TEs and SMs have a copy of any given piece of data. Each TE also keeps track of how fast the other engines in the database respond - see ping-timeout property. In this way the TE attempts to fetch data from the fastest source.

When a cached data item is changed by another TE (that also has it cached), the same metadata ensures that every engine is sent a copy of the change. Copies of the data in other TEs are effectively locked to prevent concurrent updates (it’s not quite that simple due to NuoDB’s isolation modes). The SMs commit the change (see architecture/system-architecture.adoc#database-writes) and then any TE with the data cached updates its copy also.

When an engine starts it appears to use all the memory assigned to it, because it initializes its cache. Even though the cache is mostly empty, the process appears to be using lots of memory. Process memory size is, therefore, not an indicator of how busy the process is.

A Least Recently Used (LRU) algorithm is used to control the cache when it starts to get full. Certain key atoms such as the Master Catalog and Transaction Manager are locked into the cache but other atoms may be evicted as needed.