Request Timeouts

This task shows you how to set up request timeouts in Envoy using Istio.

Istio includes beta support for the Kubernetes Gateway API and intends to make it the default API for traffic management in the future. The following instructions allow you to choose to use either the Gateway API or the Istio configuration API when configuring traffic management in the mesh. Follow instructions under either the Gateway API or Istio APIs tab, according to your preference.

This document configures internal mesh (east-west) traffic that requires Gateway API features that are either experimental or Istio specific. Before using the Gateway API instructions, make sure to:

  1. Install the experimental version of the Gateway API CRDs:

    1. $ kubectl kustomize "github.com/kubernetes-sigs/gateway-api/config/crd/experimental?ref=444631bfe06f3bcca5d0eadf1857eac1d369421d" | kubectl apply -f -

  2. Configure Istio to read the alpha Gateway API resources by setting the PILOT_ENABLE_ALPHA_GATEWAY_API environment variable to true when installing Istio:

    1. $ istioctl install --set values.pilot.env.PILOT_ENABLE_ALPHA_GATEWAY_API=true --set profile=minimal -y

Before you begin

Request timeouts

A timeout for HTTP requests can be specified using a timeout field in a route rule. By default, the request timeout is disabled, but in this task you override the reviews service timeout to half a second. To see its effect, however, you also introduce an artificial 2 second delay in calls to the ratings service.

  1. Route requests to v2 of the reviews service, i.e., a version that calls the ratings service:
  1. $ kubectl apply -f - <<EOF
  2. apiVersion: networking.istio.io/v1alpha3
  3. kind: VirtualService
  4. metadata:
  5.   name: reviews
  6. spec:
  7.   hosts:
  8.     - reviews
  9.   http:
  10.   - route:
  11.     - destination:
  12.         host: reviews
  13.         subset: v2
  14. EOF
  1. $ kubectl apply -f - <<EOF
  2. apiVersion: gateway.networking.k8s.io/v1beta1
  3. kind: HTTPRoute
  4. metadata:
  5.   name: reviews
  6. spec:
  7.   parentRefs:
  8.   - group: ""
  9.     kind: Service
  10.     name: reviews
  11.     port: 9080
  12.   rules:
  13.   - backendRefs:
  14.     - name: reviews-v2
  15.       port: 9080
  16. EOF
  1. Add a 2 second delay to calls to the ratings service:
  1. $ kubectl apply -f - <<EOF
  2. apiVersion: networking.istio.io/v1alpha3
  3. kind: VirtualService
  4. metadata:
  5.   name: ratings
  6. spec:
  7.   hosts:
  8.   - ratings
  9.   http:
  10.   - fault:
  11.       delay:
  12.         percentage:
  13.           value: 100
  14.         fixedDelay: 2s
  15.     route:
  16.     - destination:
  17.         host: ratings
  18.         subset: v1
  19. EOF

Gateway API does not support fault injection yet, so we need to use an Istio VirtualService to add the delay for now:

  1. $ kubectl apply -f - <<EOF
  2. apiVersion: networking.istio.io/v1alpha3
  3. kind: VirtualService
  4. metadata:
  5.   name: ratings
  6. spec:
  7.   hosts:
  8.   - ratings
  9.   http:
  10.   - fault:
  11.       delay:
  12.         percentage:
  13.           value: 100
  14.         fixedDelay: 2s
  15.     route:
  16.     - destination:
  17.         host: ratings
  18. EOF

  1. Open the Bookinfo URL http://$GATEWAY_URL/productpage in your browser, where $GATEWAY_URL is the External IP address of the ingress, as explained in the Bookinfo doc.

    You should see the Bookinfo application working normally (with ratings stars displayed), but there is a 2 second delay whenever you refresh the page.

  2. Now add a half second request timeout for calls to the reviews service:

  1. $ kubectl apply -f - <<EOF
  2. apiVersion: networking.istio.io/v1alpha3
  3. kind: VirtualService
  4. metadata:
  5.   name: reviews
  6. spec:
  7.   hosts:
  8.   - reviews
  9.   http:
  10.   - route:
  11.     - destination:
  12.         host: reviews
  13.         subset: v2
  14.     timeout: 0.5s
  15. EOF
  1. $ kubectl apply -f - <<EOF
  2. apiVersion: gateway.networking.k8s.io/v1beta1
  3. kind: HTTPRoute
  4. metadata:
  5.   name: reviews
  6. spec:
  7.   parentRefs:
  8.   - group: ""
  9.     kind: Service
  10.     name: reviews
  11.     port: 9080
  12.   rules:
  13.   - backendRefs:
  14.     - name: reviews-v2
  15.       port: 9080
  16.     timeouts:
  17.       request: 500ms
  18. EOF

  1. Refresh the Bookinfo web page.

    You should now see that it returns in about 1 second, instead of 2, and the reviews are unavailable.

    The reason that the response takes 1 second, even though the timeout is configured at half a second, is because there is a hard-coded retry in the productpage service, so it calls the timing out reviews service twice before returning.

Understanding what happened

In this task, you used Istio to set the request timeout for calls to the reviews microservice to half a second. By default the request timeout is disabled. Since the reviews service subsequently calls the ratings service when handling requests, you used Istio to inject a 2 second delay in calls to ratings to cause the reviews service to take longer than half a second to complete and consequently you could see the timeout in action.

You observed that instead of displaying reviews, the Bookinfo product page (which calls the reviews service to populate the page) displayed the message: Sorry, product reviews are currently unavailable for this book. This was the result of it receiving the timeout error from the reviews service.

If you examine the fault injection task, you’ll find out that the productpage microservice also has its own application-level timeout (3 seconds) for calls to the reviews microservice. Notice that in this task you used an Istio route rule to set the timeout to half a second. Had you instead set the timeout to something greater than 3 seconds (such as 4 seconds) the timeout would have had no effect since the more restrictive of the two takes precedence. More details can be found here.

One more thing to note about timeouts in Istio is that in addition to overriding them in route rules, as you did in this task, they can also be overridden on a per-request basis if the application adds an x-envoy-upstream-rq-timeout-ms header on outbound requests. In the header, the timeout is specified in milliseconds instead of seconds.

Cleanup

  • Remove the application routing rules:

Zip

  1. $ kubectl delete -f @samples/bookinfo/networking/virtual-service-all-v1.yaml@
  1. $ kubectl delete httproute reviews
  2. $ kubectl delete virtualservice ratings
  • If you are not planning to explore any follow-on tasks, see the Bookinfo cleanup instructions to shutdown the application.