Consul-Terraform-Sync Architecture

Consul-Terraform-Sync (CTS) is a service-oriented tool for managing network infrastructure near real-time. CTS runs as a daemon and integrates the network topology maintained by your Consul cluster with your network infrastructure to dynamically secure and connect services.

CTS workflow

The following diagram shows the CTS workflow as it monitors the Consul service catalog for updates.

1. CTS monitors the state of Consul’s service catalog and its KV store. This process is described in Watcher and views.
2. CTS detects a change.
3. CTS prompts Terraform to update the state of the infrastructure.

Watcher and views

CTS uses Consul’s blocking queries functionality to monitor Consul for updates. If an endpoint does not support blocking queries, CTS uses polling to watch for changes. These mechanisms are referred to in CTS as watchers.

The watcher maintains a separate thread for each value monitored and runs any tasks that depend on the watched value whenever it is updated. These threads are referred to as views. For example, a thread may run a task to update a proxy when the watcher detects that an instance has become unhealthy .

Tasks

A task is the action triggered by the updated data monitored in Consul. It takes that dynamic service data and translates it into a call to the infrastructure application to configure it with the updates. It uses a driver to push out these updates, the initial driver being a local Terraform run. An example of a task is to automate a firewall security policy rule with discovered IP addresses for a set of Consul services.

Drivers

A driver encapsulates the resources required to communicate the updates to the network infrastructure. The following drivers are supported:

• Terraform driver
• Terraform Cloud driverEnterpriseEnterprise

Each driver includes a set of providers that enables support for a wide variety of infrastructure applications.

State storage and persistence

The following types of state information are associated with CTS.

Terraform state information

By default, CTS stores Terraform state data in the Consul KV, but you can specify where this information is stored by configuring the backend setting in the Terraform driver configuration. The data persists if CTS stops and the backend is configured to a remote location.

CTS task and event data

By default, CTS stores task and event data in memory. This data is transient and does not persist. If you configure CTS to run with high availability enabled, CTS stores the data in the Consul KV. High availability is an enterprise feature that promotes CTS resiliency. When high availability is enabled, CTS stores and persists task changes and events that occur when an instance stops.

The data stored when operating in high availability mode includes task changes made using the task API or CLI. Examples of task changes include creating a new task, deleting a task, and enabling or disabling a task. You can empty the leader’s stored state information by starting CTS with the -reset-storage flag.

Instance compatibility checks (high availability)

If you run CTS with high availability enabled, CTS performs instance compatibility checks to ensure that all instances in the cluster behave consistently. Consistent instance behavior enables CTS to properly perform automations configured in the state storage.

The CTS instance compatibility check reports an error if the task module is configured with a local module, but the module does not exist on the CTS instance. Refer to the Terraform documentation for additional information about module sources. Example log:

[ERROR] ha.compat: error="compatibility check failure: stat ./example-module: no such file or directory"

Refer to Error Messages for additional information.

CTS instances perform a compatibility check on start-up based on the stored state and every five minutes after starting. If the check detects an incompatible CTS instance, it generates a log so that an operator can address it.

CTS logs the error message and continues to run when it finds an incompatibility. CTS can still elect an incompatible instance to be the leader, but tasks affected by the incompatibility do not run successfully. This can happen when all active CTS instances enter once-mode and run the tasks once when initially elected.

Security guidelines

We recommend following the network security guidelines described in the Secure Consul-Terraform-Sync for Production tutorial. The tutorial contains a checklist of best practices to secure your CTS installation for a production environment.