Runtime

The runtime configuration specifies the location of the local file system tree that contains re-loadable configuration elements. If runtime is not configured, a “null” provider is used which has the effect of using all defaults built into the code.

File system layout

Various sections of the configuration guide describe the runtime settings that are available. For example, here are the runtime settings for upstream clusters.

Assume that the folder /srv/runtime/v1 points to the actual file system path where global runtime configurations are stored. The following would be a typical configuration setting for runtime:

  • symlink_root: /srv/runtime/current
  • subdirectory: envoy
  • override_subdirectory: envoy_override

Where /srv/runtime/current is a symbolic link to /srv/runtime/v1.

Each ‘.’ in a runtime key indicates a new directory in the hierarchy, rooted at symlink_root + subdirectory. For example, the health_check.min_interval key would have the following full file system path (using the symbolic link):

/srv/runtime/current/envoy/health_check/min_interval

The terminal portion of a path is the file. The contents of the file constitute the runtime value. When reading numeric values from a file, spaces and new lines will be ignored.

The override_subdirectory is used along with the --service-cluster CLI option. Assume that --service-cluster has been set to my-cluster. Envoy will first look for the health_check.min_interval key in the following full file system path:

/srv/runtime/current/envoy_override/my-cluster/health_check/min_interval

If found, the value will override any value found in the primary lookup path. This allows the user to customize the runtime values for individual clusters on top of global defaults.

Comments

Lines starting with # as the first character are treated as comments.

Comments can be used to provide context on an existing value. Comments are also useful in an otherwise empty file to keep a placeholder for deployment in a time of need.

Updating runtime values via symbolic link swap

There are two steps to update any runtime value. First, create a hard copy of the entire runtime tree and update the desired runtime values. Second, atomically swap the symbolic link root from the old tree to the new runtime tree, using the equivalent of the following command:

  1. /srv/runtime:~$ ln -s /srv/runtime/v2 new && mv -Tf new current

It’s beyond the scope of this document how the file system data is deployed, garbage collected, etc.

Statistics

The file system runtime provider emits some statistics in the runtime. namespace.

NameTypeDescription
load_errorCounterTotal number of load attempts that resulted in an error
override_dir_not_existsCounterTotal number of loads that did not use an override directory
override_dir_existsCounterTotal number of loads that did use an override directory
load_successCounterTotal number of load attempts that were successful
num_keysGaugeNumber of keys currently loaded