Configuring an egress service

As a cluster administrator, you can configure egress traffic for pods behind a load balancer service by using an egress service.

Egress service is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.

You can use the EgressService custom resource (CR) to manage egress traffic in the following ways:

  • Assign a load balancer service IP address as the source IP address for egress traffic for pods behind the load balancer service.

    Assigning the load balancer IP address as the source IP address in this context is useful to present a single point of egress and ingress. For example, in some scenarios, an external system communicating with an application behind a load balancer service can expect the source and destination IP address for the application to be the same.

    When you assign the load balancer service IP address to egress traffic for pods behind the service, OVN-Kubernetes restricts the ingress and egress point to a single node. This limits the load balancing of traffic that MetalLB typically provides.

  • Assign the egress traffic for pods behind a load balancer to a different network than the default node network.

    This is useful to assign the egress traffic for applications behind a load balancer to a different network than the default network. Typically, the different network is implemented by using a VRF instance associated with a network interface.

Egress service custom resource

Define the configuration for an egress service in an EgressService custom resource. The following YAML describes the fields for the configuration of an egress service:

  1. apiVersion: k8s.ovn.org/v1
  2. kind: EgressService
  3. metadata:
  4. name: <egress_service_name> (1)
  5. namespace: <namespace> (2)
  6. spec:
  7. sourceIPBy: <egress_traffic_ip> (3)
  8. nodeSelector: (4)
  9. matchLabels:
  10. node-role.kubernetes.io/<role>: ""
  11. network: <egress_traffic_network> (5)
1Specify the name for the egress service. The name of the EgressService resource must match the name of the load-balancer service that you want to modify.
2Specify the namespace for the egress service. The namespace for the EgressService must match the namespace of the load-balancer service that you want to modify. The egress service is namespace-scoped.
3Specify the source IP address of egress traffic for pods behind a service. Valid values are LoadBalancerIP or Network. Use the LoadBalancerIP value to assign the LoadBalancer service ingress IP address as the source IP address for egress traffic. Specify Network to assign the network interface IP address as the source IP address for egress traffic.
4Optional: If you use the LoadBalancerIP value for the sourceIPBy specification, a single node handles the LoadBalancer service traffic. Use the nodeSelector field to limit which node can be assigned this task. When a node is selected to handle the service traffic, OVN-Kubernetes labels the node in the following format: egress-service.k8s.ovn.org/<svc-namespace>-<svc-name>: “”. When the nodeSelector field is not specified, any node can manage the LoadBalancer service traffic.
5Optional: Specify the routing table for egress traffic. If you do not include the network specification, the egress service uses the default host network.

Example egress service specification

  1. apiVersion: k8s.ovn.org/v1
  2. kind: EgressService
  3. metadata:
  4. name: test-egress-service
  5. namespace: test-namespace
  6. spec:
  7. sourceIPBy: "LoadBalancerIP"
  8. nodeSelector:
  9. matchLabels:
  10. vrf: "true"
  11. network: "2"

Deploying an egress service

You can deploy an egress service to manage egress traffic for pods behind a LoadBalancer service.

The following example configures the egress traffic to have the same source IP address as the ingress IP address of the LoadBalancer service.

Prerequisites

  • Install the OpenShift CLI (oc).

  • Log in as a user with cluster-admin privileges.

  • You configured MetalLB BGPPeer resources.

Procedure

  1. Create an IPAddressPool CR with the desired IP for the service:

    1. Create a file, such as ip-addr-pool.yaml, with content like the following example:

      1. apiVersion: metallb.io/v1beta1
      2. kind: IPAddressPool
      3. metadata:
      4. name: example-pool
      5. namespace: metallb-system
      6. spec:
      7. addresses:
      8. - 172.19.0.100/32
    2. Apply the configuration for the IP address pool by running the following command:

      1. $ oc apply -f ip-addr-pool.yaml
  2. Create Service and EgressService CRs:

    1. Create a file, such as service-egress-service.yaml, with content like the following example:

      1. apiVersion: v1
      2. kind: Service
      3. metadata:
      4. name: example-service
      5. namespace: example-namespace
      6. annotations:
      7. metallb.universe.tf/address-pool: example-pool (1)
      8. spec:
      9. selector:
      10. app: example
      11. ports:
      12. - name: http
      13. protocol: TCP
      14. port: 8080
      15. targetPort: 8080
      16. type: LoadBalancer
      17. ---
      18. apiVersion: k8s.ovn.org/v1
      19. kind: EgressService
      20. metadata:
      21. name: example-service
      22. namespace: example-namespace
      23. spec:
      24. sourceIPBy: "LoadBalancerIP" (2)
      25. nodeSelector: (3)
      26. matchLabels:
      27. node-role.kubernetes.io/worker: ""
      1The LoadBalancer service uses the IP address assigned by MetalLB from the example-pool IP address pool.
      2This example uses the LoadBalancerIP value to assign the ingress IP address of the LoadBalancer service as the source IP address of egress traffic.
      3When you specify the LoadBalancerIP value, a single node handles the LoadBalancer service’s traffic. In this example, only nodes with the worker label can be selected to handle the traffic. When a node is selected, OVN-Kubernetes labels the node in the following format egress-service.k8s.ovn.org/<svc-namespace>-<svc-name>: “”.

      If you use the sourceIPBy: “LoadBalancerIP” setting, you must specify the load-balancer node in the BGPAdvertisement custom resource (CR).

    2. Apply the configuration for the service and egress service by running the following command:

      1. $ oc apply -f service-egress-service.yaml
  3. Create a BGPAdvertisement CR to advertise the service:

    1. Create a file, such as service-bgp-advertisement.yaml, with content like the following example:

      1. apiVersion: metallb.io/v1beta1
      2. kind: BGPAdvertisement
      3. metadata:
      4. name: example-bgp-adv
      5. namespace: metallb-system
      6. spec:
      7. ipAddressPools:
      8. - example-pool
      9. nodeSelector:
      10. - matchLabels:
      11. egress-service.k8s.ovn.org/example-namespace-example-service: "" (1)
      1In this example, the EgressService CR configures the source IP address for egress traffic to use the load-balancer service IP address. Therefore, you must specify the load-balancer node for return traffic to use the same return path for the traffic originating from the pod.

Verification

  1. Verify that you can access the application endpoint of the pods running behind the MetalLB service by running the following command:

    1. $ curl <external_ip_address>:<port_number> (1)
    1Update the external IP address and port number to suit your application endpoint.
  2. If you assigned the LoadBalancer service’s ingress IP address as the source IP address for egress traffic, verify this configuration by using tools such as tcpdump to analyze packets received at the external client.

Additional resources