Multi-cluster Traffic Management

Within a multicluster mesh, traffic rules specific to the cluster topology may be desirable. This document describes a few ways to manage traffic in a multicluster mesh. Before reading this guide:

  1. Read Deployment Models
  2. Make sure your deployed services follow the concept of namespace sameness.

Keeping traffic in-cluster

In some cases the default cross-cluster load balancing behavior is not desirable. To keep traffic “cluster-local” (i.e. traffic sent from cluster-a will only reach destinations in cluster-a), mark hostnames or wildcards as clusterLocal using MeshConfig.serviceSettings.

For example, you can enforce cluster-local traffic for an individual service, all services in a particular namespace, or globally for all services in the mesh, as follows:

per-service per-namespace global

  1. serviceSettings:
  2. - settings:
  3.     clusterLocal: true
  4.   hosts:
  5.   - "mysvc.myns.svc.cluster.local"
  1. serviceSettings:
  2. - settings:
  3.     clusterLocal: true
  4.   hosts:
  5.   - "*.myns.svc.cluster.local"
  1. serviceSettings:
  2. - settings:
  3.     clusterLocal: true
  4.   hosts:
  5.   - "*"

Partitioning Services

DestinationRule.subsets allows partitioning a service by selecting labels. These labels can be the labels from Kubernetes metadata, or from built-in labels. One of these built-in labels, topology.istio.io/cluster, in the subset selector for a DestinationRule allows creating per-cluster subsets.

  1. apiVersion: networking.istio.io/v1beta1
  2. kind: DestinationRule
  3. metadata:
  4.   name: mysvc-per-cluster-dr
  5. spec:
  6.   host: mysvc.myns.svc.cluster.local
  7.   subsets:
  8.   - name: cluster-1
  9.     labels:
  10.       topology.istio.io/cluster: cluster-1
  11.   - name: cluster-2
  12.     labels:
  13.       topology.istio.io/cluster: cluster-2

Using these subsets you can create various routing rules based on the cluster such as mirroring or shifting.

This provides another option to create cluster-local traffic rules by restricting the destination subset in a VirtualService:

  1. apiVersion: networking.istio.io/v1beta1
  2. kind: VirtualService
  3. metadata:
  4.   name: mysvc-cluster-local-vs
  5. spec:
  6.   hosts:
  7.   - mysvc.myns.svc.cluster.local
  8.   http:
  9.   - name: "cluster-1-local"
  10.     match:
  11.     - sourceLabels:
  12.         topology.istio.io/cluster: "cluster-1"
  13.     route:
  14.     - destination:
  15.         host: mysvc.myns.svc.cluster.local
  16.         subset: cluster-1
  17.   - name: "cluster-2-local"
  18.     match:
  19.     - sourceLabels:
  20.         topology.istio.io/cluster: "cluster-2"
  21.     route:
  22.     - destination:
  23.         host: mysvc.myns.svc.cluster.local
  24.         subset: cluster-2

Using subset-based routing this way to control cluster-local traffic, as opposed to MeshConfig.serviceSettings, has the downside of mixing service-level policy with topology-level policy. For example, a rule that sends 10% of traffic to v2 of a service will need twice the number of subsets (e.g., cluster-1-v2, cluster-2-v2). This approach is best limited to situations where more granular control of cluster-based routing is needed.