Traffic rate limiting overview

This topic provides overview information about the traffic rates limits you can configure for Consul datacenters.

Introduction

Configuring rate limits on RPC and gRPC traffic mitigates the risks to Consul servers when client agents or services send excessive read or write requests to Consul resources. A read request is defined as any request that does not modify Consul internal state. A write request is defined as any request that modifies Consul internal state. Configure read and write request limits independently.

Workflow

You can set global limits on the rate of read and write requests that affect individual servers in the datacenter. You can set limits for all source IP addresses, which enables you to specify a budget for read and write requests to prevent any single source IP from overwhelming the Consul server and negatively affecting the network. The following steps describe the general process for setting global read and write rate limits:

  1. Set arbitrary limits to begin understanding the upper boundary of RPC and gRPC loads in your network. Refer to Initialize rate limit settings for additional information.

  2. Monitor the metrics and logs and readjust the initial configurations as necessary. Refer to Monitor rate limit data

  3. Define your final operational limits based on your observations. If you are defining global rate limits, refer to Set global traffic rate limits for additional information. For information about setting limits per source IP address, refer to Limit traffic rates for a source IP.

EnterpriseOverview - 图1Enterprise

Setting limits per source IP requires Consul Enterprise.

Order of operations

You can define request rate limits in the agent configuration and in the control plane request limit configuration entry. The configuration entry also supports rate limit configurations for Consul resources. Consul perfroms the following order of operations when determing request rate limits:

  1. Parse request.
  2. Does the request reach a global server limit?
    • No: Proceed to the next stage.
    • Yes: Return an error that the requested resource has been exhausted.
  3. Does the request reach a limit associated with its source IP address?
    • No: Proceed to the next stage.
    • Yes: Return an error that the requested resource has been exhausted.
  4. Resolve the Consul Enterprise metadata.
  5. Does the request reach a limit associated with the source partition?
    • No: Proceed to the next stage.
    • Yes: Return an error that the requested resource has been exhausted.
  6. Does the request reach a limit associated with the source namespace?
    • No: Proceed to the next stage.
    • Yes: Return an error that the requested resource has been exhausted.
  7. Resolve the ACL identity associated with the request.
  8. Does the request reach a limit associated with its identity?
    • No: Proceed to the next stage.
    • Yes: Return an error that the requested resource has been exhausted.
  9. Handle the request.

Kubernetes

To define global rate limits, configure the request_limits settings in the Consul Helm chart. Refer to the Helm chart reference for additional information. Refer to the control plane request limit configuration entry reference for information about applying a CRD for limiting traffic rates from source IPs.