InFlightReq

Limiting the Number of Simultaneous In-Flight Requests

InFlightReq

To proactively prevent services from being overwhelmed with high load, a limit on the number of simultaneous in-flight requests can be applied.

Configuration Examples

  1. labels:
  2.   - "traefik.http.middlewares.test-inflightreq.inflightreq.amount=10"

  1. apiVersion: traefik.containo.us/v1alpha1
  2. kind: Middleware
  3. metadata:
  4.   name: test-inflightreq
  5. spec:
  6.   inFlightReq:
  7.     amount: 10

  1. # Limiting to 10 simultaneous connections
  2. - "traefik.http.middlewares.test-inflightreq.inflightreq.amount=10"

  1. "labels": {
  2.   "traefik.http.middlewares.test-inflightreq.inflightreq.amount": "10"
  3. }

  1. # Limiting to 10 simultaneous connections
  2. labels:
  3.   - "traefik.http.middlewares.test-inflightreq.inflightreq.amount=10"

  1. # Limiting to 10 simultaneous connections
  2. [http.middlewares]
  3.   [http.middlewares.test-inflightreq.inFlightReq]
  4.     amount = 10 

  1. # Limiting to 10 simultaneous connections
  2. http:
  3.   middlewares:
  4.     test-inflightreq:
  5.       inFlightReq:
  6.         amount: 10

Configuration Options

amount

The amount option defines the maximum amount of allowed simultaneous in-flight request.The middleware will return an HTTP 429 Too Many Requests if there are already amount requests in progress (based on the same sourceCriterion strategy).

  1. labels:
  2.   - "traefik.http.middlewares.test-inflightreq.inflightreq.amount=10"

  1. apiVersion: traefik.containo.us/v1alpha1
  2. kind: Middleware
  3. metadata:
  4.   name: test-inflightreq
  5. spec:
  6.   inFlightReq:
  7.     amount: 10

  1. # Limiting to 10 simultaneous connections
  2. - "traefik.http.middlewares.test-inflightreq.inflightreq.amount=10"

  1. "labels": {
  2.   "traefik.http.middlewares.test-inflightreq.inflightreq.amount": "10"
  3. }

  1. # Limiting to 10 simultaneous connections
  2. labels:
  3.   - "traefik.http.middlewares.test-inflightreq.inflightreq.amount=10"

  1. # Limiting to 10 simultaneous connections
  2. [http.middlewares]
  3.   [http.middlewares.test-inflightreq.inFlightReq]
  4.     amount = 10 

  1. # Limiting to 10 simultaneous connections
  2. http:
  3.   middlewares:
  4.     test-inflightreq:
  5.       inFlightReq:
  6.         amount: 10

sourceCriterion

SourceCriterion defines what criterion is used to group requests as originating from a common source.The precedence order is ipStrategy, then requestHeaderName, then requestHost.If none are set, the default is to use the requestHost.

sourceCriterion.ipStrategy

The ipStrategy option defines two parameters that sets how Traefik will determine the client IP: depth, and excludedIPs.

ipStrategy.depth

The depth option tells Traefik to use the X-Forwarded-For header and take the IP located at the depth position (starting from the right).

  • If depth is greater than the total number of IPs in X-Forwarded-For, then the client IP will be empty.
  • depth is ignored if its value is lesser than or equal to 0.

Example of Depth & X-Forwarded-For

If depth was equal to 2, and the request X-Forwarded-For header was "10.0.0.1,11.0.0.1,12.0.0.1,13.0.0.1" then the "real" client IP would be "10.0.0.1" (at depth 4) but the IP used as the criterion would be "12.0.0.1" (depth=2).

X-Forwarded-FordepthclientIP
"10.0.0.1,11.0.0.1,12.0.0.1,13.0.0.1"1"13.0.0.1"
"10.0.0.1,11.0.0.1,12.0.0.1,13.0.0.1"3"11.0.0.1"
"10.0.0.1,11.0.0.1,12.0.0.1,13.0.0.1"5""

  1. labels:
  2.   - "traefik.http.middlewares.test-inflightreq.inflightreq.sourcecriterion.ipstrategy.depth=2"

  1. apiVersion: traefik.containo.us/v1alpha1
  2. kind: Middleware
  3. metadata:
  4.   name: test-inflightreq
  5. spec:
  6.   inFlightReq:
  7.     sourceCriterion:
  8.       ipStrategy:
  9.         depth: 2

  1. - "traefik.http.middlewares.test-inflightreq.inflightreq.sourcecriterion.ipstrategy.depth=2"

  1. "labels": {
  2.   "traefik.http.middlewares.test-inflightreq.inflightreq.sourcecriterion.ipstrategy.depth": "2"
  3. }

  1. labels:
  2.   - "traefik.http.middlewares.test-inflightreq.inflightreq.sourcecriterion.ipstrategy.depth=2"

  1. [http.middlewares]
  2.   [http.middlewares.test-inflightreq.inflightreq]
  3.     [http.middlewares.test-inflightreq.inFlightReq.sourceCriterion.ipStrategy]
  4.       depth = 2

  1. http:
  2.   middlewares:
  3.     test-inflightreq:
  4.       inFlightReq:
  5.         sourceCriterion:
  6.           ipStrategy:
  7.             depth: 2
ipStrategy.excludedIPs

excludedIPs tells Traefik to scan the X-Forwarded-For header and pick the first IP not in the list.

If depth is specified, excludedIPs is ignored.

Example of ExcludedIPs & X-Forwarded-For

X-Forwarded-ForexcludedIPsclientIP
"10.0.0.1,11.0.0.1,12.0.0.1,13.0.0.1""12.0.0.1,13.0.0.1""11.0.0.1"
"10.0.0.1,11.0.0.1,12.0.0.1,13.0.0.1""15.0.0.1,13.0.0.1""12.0.0.1"
"10.0.0.1,11.0.0.1,12.0.0.1,13.0.0.1""10.0.0.1,13.0.0.1""12.0.0.1"
"10.0.0.1,11.0.0.1,12.0.0.1,13.0.0.1""15.0.0.1,16.0.0.1""13.0.0.1"
"10.0.0.1,11.0.0.1""10.0.0.1,11.0.0.1"""

  1. labels:
  2.   - "traefik.http.middlewares.test-inflightreq.inflightreq.sourcecriterion.ipstrategy.excludedips=127.0.0.1/32, 192.168.1.7"

  1. apiVersion: traefik.containo.us/v1alpha1
  2. kind: Middleware
  3. metadata:
  4.   name: test-inflightreq
  5. spec:
  6.   inFlightReq:
  7.     sourceCriterion:
  8.       ipStrategy:
  9.         excludedIPs:
  10.         - 127.0.0.1/32
  11.         - 192.168.1.7

  1. - "traefik.http.middlewares.test-inflightreq.inflightreq.sourcecriterion.ipstrategy.excludedips=127.0.0.1/32, 192.168.1.7"

  1. "labels": {
  2.   "traefik.http.middlewares.test-inflightreq.inflightreq.sourcecriterion.ipstrategy.excludedips": "127.0.0.1/32, 192.168.1.7"
  3. }

  1. labels:
  2.   - "traefik.http.middlewares.test-inflightreq.inflightreq.sourcecriterion.ipstrategy.excludedips=127.0.0.1/32, 192.168.1.7"

  1. [http.middlewares]
  2.   [http.middlewares.test-inflightreq.inflightreq]
  3.     [http.middlewares.test-inflightreq.inFlightReq.sourceCriterion.ipStrategy]
  4.       excludedIPs = ["127.0.0.1/32", "192.168.1.7"]

  1. http:
  2.   middlewares:
  3.     test-inflightreq:
  4.       inFlightReq:
  5.         sourceCriterion:
  6.           ipStrategy:
  7.             excludedIPs:
  8.               - "127.0.0.1/32"
  9.               - "192.168.1.7"

sourceCriterion.requestHeaderName

Requests having the same value for the given header are grouped as coming from the same source.

  1. labels:
  2.   - "traefik.http.middlewares.test-inflightreq.inflightreq.sourcecriterion.requestheadername=username"

  1. apiVersion: traefik.containo.us/v1alpha1
  2. kind: Middleware
  3. metadata:
  4.   name: test-inflightreq
  5. spec:
  6.   inFlightReq:
  7.     sourceCriterion:
  8.       requestHeaderName: username

  1. - "traefik.http.middlewares.test-inflightreq.inflightreq.sourcecriterion.requestheadername=username"

  1. "labels": {
  2.   "traefik.http.middlewares.test-inflightreq.inflightreq.sourcecriterion.requestheadername": "username"
  3. }

  1. labels:
  2.   - "traefik.http.middlewares.test-inflightreq.inflightreq.sourcecriterion.requestheadername=username"

  1. [http.middlewares]
  2.   [http.middlewares.test-inflightreq.inflightreq]
  3.     [http.middlewares.test-inflightreq.inFlightReq.sourceCriterion]
  4.       requestHeaderName = "username"

  1. http:
  2.   middlewares:
  3.     test-inflightreq:
  4.       inFlightReq:
  5.         sourceCriterion:
  6.           requestHeaderName: username

sourceCriterion.requestHost

Whether to consider the request host as the source.

  1. labels:
  2.   - "traefik.http.middlewares.test-inflightreq.inflightreq.sourcecriterion.requesthost=true"

  1. apiVersion: traefik.containo.us/v1alpha1
  2. kind: Middleware
  3. metadata:
  4.   name: test-inflightreq
  5. spec:
  6.   inFlightReq:
  7.     sourceCriterion:
  8.       requestHost: true

  1. - "traefik.http.middlewares.test-inflightreq.inflightreq.sourcecriterion.requesthost=true"

  1. "labels": {
  2.   "traefik.http.middlewares.test-inflightreq.inflightreq.sourcecriterion.requesthost": "true"
  3. }

  1. labels:
  2.   - "traefik.http.middlewares.test-inflightreq.inflightreq.sourcecriterion.requesthost=true"

  1. [http.middlewares]
  2.   [http.middlewares.test-inflightreq.inflightreq]
  3.     [http.middlewares.test-inflightreq.inFlightReq.sourceCriterion]
  4.       requestHost = true

  1. http:
  2.   middlewares:
  3.     test-inflightreq:
  4.       inFlightReq:
  5.         sourceCriterion:
  6.           requestHost: true