Gaffer for pipeline – part 3 – Compute cache size

Like many other applications that calculate data on-demand, Gaffer has a memory cache that can increase performance in UI sessions if you have RAM to spare.

The default size (1GB) is somewhat conservative as to ensure we don’t use up all the resources of more modest machines when Gaffer is used out-the-box. It is easy to increase the limit though, which can bring performance gains in real-world production scenarios.

Gaffer’s compute cache is managed by Gaffer.ValuePlug (as it’s the class that introduces the concept of values to plugs). The Gaffer UI provides a preference for users to adjust values locally for interactive sessions, but should you want to adjust the default cache size across the board in your facility (a good idea), it can be adjusted as follows, in a startup script:

import Gaffer
# The limit is specified in bytes, the following sets an 8GB cache
Gaffer.ValuePlug.setCacheMemoryLimit( 8 * 1024 * 1024 * 1024 )

This is generally done in the Gaffer module startup sub-directory so it applies to all gaffer apps. You shouldn’t set this in a gui script (see below):

The ValuePlug API also allows the cache limit to be queried and the cache cleared if desired:

print( "Using {used} of {limit} bytes".format(
   used = Gaffer.ValuePlug.cacheMemoryUsage(),
   limit = Gaffer.ValuePlug.getCacheMemoryLimit()
) )

Gaffer.ValuePlug.clearCache()

User preferences

The gui app adds a preference to the application that allows users to override the cache size for the UI session only. This is available via Gaffer > Preferences > Cache.

Changes to this value are sorted in the users preferences file (which is just a startup script: ~/gaffer/startup/gui/preferences.py).

As the user’s ~/gaffer directory is always added to the end of $GAFFER_STARTUP_PATHS at startup, the user’s preferences will take precedence over other scripts that call setCacheMemoryLimit.

If you wish to make UI-only changes as a facility default, then its best to set this in a GafferUI startup script instead, as this will be run before the gui scripts.