Built-In Certificate Authority for Service Mesh

Consul ships with a built-in CA system so that service mesh can be easily enabled out of the box. The built-in CA generates and stores the root certificate and private key on Consul servers. It can also be configured with a custom certificate and private key if needed.

If service mesh is enabled and no CA provider is specified, the built-in CA is the default provider used. The provider can be updated and rotated at any point to migrate to a new provider.

This page documents the specifics of the built-in CA provider. Please read the certificate management overview page first to understand how Consul manages certificates with configurable CA providers.

Configuration

The built-in CA provider has no required configuration. Enabling service mesh alone will configure the built-in CA provider, and will automatically generate a root certificate and private key:

Built-In CA - 图1

/etc/consul.d/config.hcl

  1. # ...
  2. connect {
  3.   enabled = true
  4. }

The configuration options are listed below.

Note: The first key is the value used in API calls, and the second key (after the /) is used if you are adding the configuration to the agent’s configuration file.

  • PrivateKey / private_key (string: "") - A PEM-encoded private key for signing operations. This must match the private key used for the root certificate if it is manually specified. If this is blank, a private key is automatically generated.

  • RootCert / root_cert (string: "") - A PEM-encoded root certificate to use. If this is blank, a root certificate is automatically generated using the private key specified. If this is specified, the certificate must be a valid SPIFFE SVID signing certificate and the URI in the SAN must match the cluster identifier created at bootstrap with the “.consul” TLD. The cluster identifier can be found using the CA List Roots endpoint.

Common CA Config Options

The following configuration options are supported by all CA providers:

  • CSRMaxConcurrent / csr_max_concurrent (int: 0) - Sets a limit on the number of Certificate Signing Requests that can be processed concurrently. Defaults to 0 (disabled). This is useful when you want to limit the number of CPU cores available to the server for certificate signing operations. For example, on an 8 core server, setting this to 1 will ensure that no more than one CPU core will be consumed when generating or rotating certificates. Setting this is recommended instead of csr_max_per_second when you want to limit the number of cores consumed since it is simpler to reason about limiting CSR resources this way without artificially slowing down rotations. Added in 1.4.1.

  • CSRMaxPerSecond / csr_max_per_second (float: 50) - Sets a rate limit on the maximum number of Certificate Signing Requests (CSRs) the servers will accept. This is used to prevent CA rotation from causing unbounded CPU usage on servers. It defaults to 50 which is conservative – a 2017 MacBook can process about 100 per second using only ~40% of one CPU core – but sufficient for deployments up to ~1500 service instances before the time it takes to rotate is impacted. For larger deployments we recommend increasing this based on the expected number of server instances and server resources, or use csr_max_concurrent instead if servers have more than one CPU core. Setting this to zero disables rate limiting. Added in 1.4.1.

  • LeafCertTTL / leaf_cert_ttl (duration: "72h") - The upper bound on the lease duration of a leaf certificate issued for a service. In most cases a new leaf certificate will be requested by a proxy before this limit is reached. This is also the effective limit on how long a server outage can last (with no leader) before network connections will start being rejected. Defaults to 72h. This value cannot be lower than 1 hour or higher than 1 year.

    This value is also used when rotating out old root certificates from the cluster. When a root certificate has been inactive (rotated out) for more than twice the current leaf_cert_ttl, it will be removed from the trusted list.

  • RootCertTTL / root_cert_ttl (duration: "87600h") The time to live (TTL) for a root certificate. Defaults to 10 years as 87600h. This value, if provided, needs to be higher than the intermediate certificate TTL.

    This setting applies to all Consul CA providers.

    For the Vault provider, this value is only used if the backend is not initialized at first.

  • IntermediateCertTTL / intermediate_cert_ttl (duration: "8760h") The time to live (TTL) for any intermediate certificates signed by root certificate of the primary datacenter. This field is only valid in the primary datacenter. Defaults to 1 year as 8760h.

    This setting applies to all Consul CA providers.

    For the Vault provider, this value is only used if the backend is not initialized at first.

  • PrivateKeyType / private_key_type (string: "ec") - The type of key to generate for this CA. This is only used when the provider is generating a new key. If private_key is set for the Consul provider, or existing root or intermediate PKI paths given for Vault then this will be ignored. Currently supported options are ec or rsa. Default is ec.

    It is required that all servers in a datacenter have the same config for the CA. It is recommended that servers in different datacenters use the same key type and size, although the built-in CA and Vault provider will both allow mixed CA key types.

    Some CA providers (currently Vault) will not allow cross-signing a new CA certificate with a different key type. This means that if you migrate from an RSA-keyed Vault CA to an EC-keyed CA from any provider, you may have to proceed without cross-signing which risks temporary connection issues for workloads during the new certificate rollout. We highly recommend testing this outside of production to understand the impact, and suggest sticking to same key type where possible.

    Note: This only affects CA keys generated by the provider. Leaf certificate keys are always EC 256 regardless of the CA configuration.

  • PrivateKeyBits / private_key_bits (string: "") - The length of key to generate for this CA. This is only used when the provider is generating a new key. If private_key is set for the Consul provider, or existing root or intermediate PKI paths given for Vault then this will be ignored.

    Currently supported values are:

Specifying a Custom Private Key and Root Certificate

By default, a root certificate and private key will be automatically generated during the cluster’s bootstrap. It is possible to configure the Consul CA provider to use a specific private key and root certificate. This is particularly useful if you have an external PKI system that doesn’t currently integrate with Consul directly.

To view the current CA configuration, use the Get CA Configuration endpoint:

  1. $ curl localhost:8500/v1/connect/ca/configuration
  2. {
  3.     "Provider": "consul",
  4.     "Config": {
  5.         "LeafCertTTL": "72h",
  6.         "IntermediateCertTTL": "8760h"
  7.     },
  8.     "CreateIndex": 5,
  9.     "ModifyIndex": 5
  10. }

This is the default service mesh CA configuration if nothing is explicitly set when service mesh is enabled - the PrivateKey and RootCert fields have not been set, so those have been generated (as seen above in the roots list).

There are two ways to have the Consul CA use a custom private key and root certificate: either through the ca_config section of the Agent configuration (which can only be used during the cluster’s initial bootstrap) or through the Update CA Configuration endpoint.

Currently Consul requires that root certificates are valid SPIFFE SVID Signing certificates and that the URI encoded in the SAN is the cluster identifier created at bootstrap with the “.consul” TLD. In this example, we will set the URI SAN to spiffe://36cb52cd-4058-f811-0432-6798a240c5d3.consul.

In order to use the Update CA Configuration HTTP endpoint, the private key and certificate must be passed via JSON:

  1. $ jq --null-input --rawfile key root.key --rawfile cert root.crt '
  2. {
  3.     "Provider": "consul",
  4.     "Config": {
  5.         "LeafCertTTL": "72h",
  6.         "PrivateKey": $key | sub("\\n$"; ""),
  7.         "RootCert": $cert | sub("\\n$"; ""),
  8.         "IntermediateCertTTL": "8760h"
  9.     }
  10. }' > ca_config.json

The resulting ca_config.json file can then be used to update the active root certificate:

  1. $ cat ca_config.json
  2. {
  3.   "Provider": "consul",
  4.   "Config": {
  5.     "LeafCertTTL": "72h",
  6.     "PrivateKey": "-----BEGIN RSA PRIVATE KEY-----\nMIIEpAIBAAKCAQEArqiy1c3pbT3cSkjdEM1APALUareU...",
  7.     "RootCert": "-----BEGIN CERTIFICATE-----\nMIIDijCCAnKgAwIBAgIJAOFZ66em1qC7MA0GCSqGSIb3...",
  8.     "IntermediateCertTTL": "8760h"
  9.   }
  10. }
  12. $ curl --request PUT --data @ca_config.json localhost:8500/v1/connect/ca/configuration
  14. ...
  16. [INFO] connect: CA rotated to new root under provider "consul"

The cluster is now using the new private key and root certificate. Updating the CA config this way also triggered a certificate rotation.