Configure an external gateway through a secondary network interface

Configure an external gateway through a secondary network interface

As a cluster administrator, you can configure an external gateway on a secondary network.

This feature offers the following benefits:

Granular control over egress traffic on a per-namespace basis
Flexible configuration of static and dynamic external gateway IP addresses
Support for both IPv4 and IPv6 address families

Prerequisites

Your cluster uses the OVN-Kubernetes network plugin.
Your infrastructure is configured to route traffic from the secondary external gateway.

How OKD determines the external gateway IP address

You configure a secondary external gateway with the AdminPolicyBasedExternalRoute custom resource from the the k8s.ovn.org API group. The custom resource (CR) supports static and dynamic approaches to specifying an external gateway’s IP address. Each namespace that a AdminPolicyBasedExternalRoute CR targets cannot be selected by any other AdminPolicyBasedExternalRoute CR. A namespace cannot have concurrent secondary external gateways.

Static assignment

You specify an IP address directly.

Dynamic assignment

You specify an IP address indirectly, with namespace and pod selectors, and an optional network attachment definition.

If the name of a network attachment definition is provided, the external gateway IP address of the network attachment is used.
If the name of a network attachment definition is not provided, the external gateway IP address for the pod itself is used. However, this approach works only if the pod is configured with hostNetwork set to true.

AdminPolicyBasedExternalRoute object configuration

You can define an AdminPolicyBasedExternalRoute object, which is cluster scoped, with the following properties. A namespace can be selected by only one AdminPolicyBasedExternalRoute CR at a time.

Table 1. `AdminPolicyBasedExternalRoute` object
Field	Type	Description
`metadata.name`	`string`	Specifies the name of the `AdminPolicyBasedExternalRoute` object.
`spec.from`	`string`	Specifies a namespace selector that the routing polices apply to. Only `namespaceSelector` is supported for external traffic. For example: `from:` `namespaceSelector:` `matchLabels:` `kubernetes.io/metadata.name: novxlan-externalgw-ecmp-4059` A namespace can be targeted by only one `AdminPolicyBasedExternalRoute` CR. If a namespace is selected by more than one `AdminPolicyBasedExternalRoute` CR, a `failed` error status occurs on the second and subsequent CRs targeting the same namespace.
`spec.nextHops`	`object`	Specifies the destinations where the packets are forwarded to. Must be either or both of `static` and `dynamic`. You must have at least one next hop defined.

Table 2. `nextHops` object
Field	Type	Description
`static`	`array`	Specifies an array of static IP addresses.
`dynamic`	`array`	Specifies an array of pod selectors corresponding to pods configured with a network attachment definition to use as the external gateway target.

Table 3. `nextHops.static` object
Field	Type	Description
`ip`	`string`	Specifies either an IPv4 or IPv6 address of the next destination hop.
`bfdEnabled`	`boolean`	Optional: Specifies whether Bi-Directional Forwarding Detection (BFD) is supported by the network. The default value is `false`.

Table 4. `nextHops.dynamic` object
Field	Type	Description
`podSelector`	`string`	Specifies a set-based label selector to filter the pods in the namespace that match this network configuration.
`namespaceSelector`	`string`	Specifies a `set-based` selector to filter the namespaces that the `podSelector` applies to. You must specify a value for this field.
`bfdEnabled`	`boolean`	Optional: Specifies whether Bi-Directional Forwarding Detection (BFD) is supported by the network. The default value is `false`.
`networkAttachmentName`	`string`	Optional: Specifies the name of a network attachment definition. The name must match the list of logical networks associated with the pod. If this field is not specified, the host network of the pod is used. However, the pod must be configure as a host network pod to use the host network.

Example secondary external gateway configurations

In the following example, the AdminPolicyBasedExternalRoute object configures two static IP addresses as external gateways for pods in namespaces with the kubernetes.io/metadata.name: novxlan-externalgw-ecmp-4059 label.

apiVersion: k8s.ovn.org/v1
kind: AdminPolicyBasedExternalRoute
metadata:
  name: default-route-policy
spec:
  from:
    namespaceSelector:
      matchLabels:
        kubernetes.io/metadata.name: novxlan-externalgw-ecmp-4059
  nextHops:
    static:
    - ip: "172.18.0.8"
    - ip: "172.18.0.9"

In the following example, the AdminPolicyBasedExternalRoute object configures a dynamic external gateway. The IP addresses used for the external gateway are derived from the additional network attachments associated with each of the selected pods.

apiVersion: k8s.ovn.org/v1
kind: AdminPolicyBasedExternalRoute
metadata:
  name: shadow-traffic-policy
spec:
  from:
    namespaceSelector:
      matchLabels:
        externalTraffic: ""
  nextHops:
    dynamic:
    - podSelector:
        matchLabels:
          gatewayPod: ""
      namespaceSelector:
        matchLabels:
          shadowTraffic: ""
      networkAttachmentName: shadow-gateway
    - podSelector:
        matchLabels:
          gigabyteGW: ""
      namespaceSelector:
        matchLabels:
          gatewayNamespace: ""
      networkAttachmentName: gateway

In the following example, the AdminPolicyBasedExternalRoute object configures both static and dynamic external gateways.

apiVersion: k8s.ovn.org/v1
kind: AdminPolicyBasedExternalRoute
metadata:
  name: multi-hop-policy
spec:
  from:
    namespaceSelector:
      matchLabels:
        trafficType: "egress"
  nextHops:
    static:
    - ip: "172.18.0.8"
    - ip: "172.18.0.9"
    dynamic:
    - podSelector:
        matchLabels:
          gatewayPod: ""
      namespaceSelector:
        matchLabels:
          egressTraffic: ""
      networkAttachmentName: gigabyte

Configure a secondary external gateway

You can configure a secondary external gateway for a namespace in your cluster.

Prerequisites

You installed the OpenShift CLI (oc).
You are logged in to the cluster with a user with cluster-admin privileges.

Procedure

Create a YAML file that contains an AdminPolicyBasedExternalRoute object.
To create an admin policy based external route, enter the following command:
```
$ oc create -f <file>.yaml
```
where:

<file>

Specifies the name of the YAML file that you created in the previous step.

Example output
```
adminpolicybasedexternalroute.k8s.ovn.org/default-route-policy created
```

To confirm that the admin policy based external route was created, enter the following command:

$ oc describe apbexternalroute <name> | tail -n 6

where:

<name>

Specifies the name of the AdminPolicyBasedExternalRoute object.

Example output

Status:
  Last Transition Time:  2023-04-24T15:09:01Z
  Messages:
  Configured external gateway IPs: 172.18.0.8
  Status:  Success
Events:  <none>

Additional resources

For more information about additional network attachments, see Understanding multiple networks