Configuring MetalLB BGP peers

As a cluster administrator, you can add, modify, and delete Border Gateway Protocol (BGP) peers. The MetalLB Operator uses the BGP peer custom resources to identify which peers that MetalLB speaker pods contact to start BGP sessions. The peers receive the route advertisements for the load-balancer IP addresses that MetalLB assigns to services.

About the BGP peer custom resource

The fields for the BGP peer custom resource are described in the following table.

Table 1. MetalLB BGP peer custom resource
FieldTypeDescription

metadata.name

string

Specifies the name for the BGP peer custom resource.

metadata.namespace

string

Specifies the namespace for the BGP peer custom resource.

spec.myASN

integer

Specifies the Autonomous System number for the local end of the BGP session. Specify the same value in all BGP peer custom resources that you add. The range is 0 to 4294967295.

spec.peerASN

integer

Specifies the Autonomous System number for the remote end of the BGP session. The range is 0 to 4294967295.

spec.peerAddress

string

Specifies the IP address of the peer to contact for establishing the BGP session.

spec.sourceAddress

string

Optional: Specifies the IP address to use when establishing the BGP session. The value must be an IPv4 address.

spec.peerPort

integer

Optional: Specifies the network port of the peer to contact for establishing the BGP session. The range is 0 to 16384.

spec.holdTime

string

Optional: Specifies the duration for the hold time to propose to the BGP peer. The minimum value is 3 seconds (3s). The common units are seconds and minutes, such as 3s, 1m, and 5m30s. To detect path failures more quickly, also configure BFD.

spec.keepaliveTime

string

Optional: Specifies the maximum interval between sending keep-alive messages to the BGP peer. If you specify this field, you must also specify a value for the holdTime field. The specified value must be less than the value for the holdTime field.

spec.routerID

string

Optional: Specifies the router ID to advertise to the BGP peer. If you specify this field, you must specify the same value in every BGP peer custom resource that you add.

spec.password

string

Optional: Specifies the MD5 password to send to the peer for routers that enforce TCP MD5 authenticated BGP sessions.

spec.passwordSecret

string

Optional: Specifies name of the authentication secret for the BGP Peer. The secret must live in the metallb namespace and be of type basic-auth.

spec.bfdProfile

string

Optional: Specifies the name of a BFD profile.

spec.nodeSelectors

object[]

Optional: Specifies a selector, using match expressions and match labels, to control which nodes can connect to the BGP peer.

spec.ebgpMultiHop

boolean

Optional: Specifies that the BGP peer is multiple network hops away. If the BGP peer is not directly connected to the same network, the speaker cannot establish a BGP session unless this field is set to true. This field applies to external BGP. External BGP is the term that is used to describe when a BGP peer belongs to a different Autonomous System.

The passwordSecret field is mutually exclusive with the password field, and contains a reference to a secret containing the password to use. Setting both fields results in a failure of the parsing.

Configuring a BGP peer

As a cluster administrator, you can add a BGP peer custom resource to exchange routing information with network routers and advertise the IP addresses for services.

Prerequisites

  • Install the OpenShift CLI (oc).

  • Log in as a user with cluster-admin privileges.

  • Configure MetalLB with a BGP advertisement.

Procedure

  1. Create a file, such as bgppeer.yaml, with content like the following example:

    1. apiVersion: metallb.io/v1beta2
    2. kind: BGPPeer
    3. metadata:
    4. namespace: metallb-system
    5. name: doc-example-peer
    6. spec:
    7. peerAddress: 10.0.0.1
    8. peerASN: 64501
    9. myASN: 64500
    10. routerID: 10.10.10.10
  2. Apply the configuration for the BGP peer:

    1. $ oc apply -f bgppeer.yaml

Configure a specific set of BGP peers for a given address pool

This procedure illustrates how to:

  • Configure a set of address pools (pool1 and pool2).

  • Configure a set of BGP peers (peer1 and peer2).

  • Configure BGP advertisement to assign pool1 to peer1 and pool2 to peer2.

Prerequisites

  • Install the OpenShift CLI (oc).

  • Log in as a user with cluster-admin privileges.

Procedure

  1. Create address pool pool1.

    1. Create a file, such as ipaddresspool1.yaml, with content like the following example:

      1. apiVersion: metallb.io/v1beta1
      2. kind: IPAddressPool
      3. metadata:
      4. namespace: metallb-system
      5. name: pool1
      6. spec:
      7. addresses:
      8. - 4.4.4.100-4.4.4.200
      9. - 2001:100:4::200-2001:100:4::400
    2. Apply the configuration for the IP address pool pool1:

      1. $ oc apply -f ipaddresspool1.yaml
  2. Create address pool pool2.

    1. Create a file, such as ipaddresspool2.yaml, with content like the following example:

      1. apiVersion: metallb.io/v1beta1
      2. kind: IPAddressPool
      3. metadata:
      4. namespace: metallb-system
      5. name: pool2
      6. spec:
      7. addresses:
      8. - 5.5.5.100-5.5.5.200
      9. - 2001:100:5::200-2001:100:5::400
    2. Apply the configuration for the IP address pool pool2:

      1. $ oc apply -f ipaddresspool2.yaml
  3. Create BGP peer1.

    1. Create a file, such as bgppeer1.yaml, with content like the following example:

      1. apiVersion: metallb.io/v1beta2
      2. kind: BGPPeer
      3. metadata:
      4. namespace: metallb-system
      5. name: peer1
      6. spec:
      7. peerAddress: 10.0.0.1
      8. peerASN: 64501
      9. myASN: 64500
      10. routerID: 10.10.10.10
    2. Apply the configuration for the BGP peer:

      1. $ oc apply -f bgppeer1.yaml
  4. Create BGP peer2.

    1. Create a file, such as bgppeer2.yaml, with content like the following example:

      1. apiVersion: metallb.io/v1beta2
      2. kind: BGPPeer
      3. metadata:
      4. namespace: metallb-system
      5. name: peer2
      6. spec:
      7. peerAddress: 10.0.0.2
      8. peerASN: 64501
      9. myASN: 64500
      10. routerID: 10.10.10.10
    2. Apply the configuration for the BGP peer2:

      1. $ oc apply -f bgppeer2.yaml
  5. Create BGP advertisement 1.

    1. Create a file, such as bgpadvertisement1.yaml, with content like the following example:

      1. apiVersion: metallb.io/v1beta1
      2. kind: BGPAdvertisement
      3. metadata:
      4. name: bgpadvertisement-1
      5. namespace: metallb-system
      6. spec:
      7. ipAddressPools:
      8. - pool1
      9. peers:
      10. - peer1
      11. communities:
      12. - 65535:65282
      13. aggregationLength: 32
      14. aggregationLengthV6: 128
      15. localPref: 100
    2. Apply the configuration:

      1. $ oc apply -f bgpadvertisement1.yaml
  6. Create BGP advertisement 2.

    1. Create a file, such as bgpadvertisement2.yaml, with content like the following example:

      1. apiVersion: metallb.io/v1beta1
      2. kind: BGPAdvertisement
      3. metadata:
      4. name: bgpadvertisement-2
      5. namespace: metallb-system
      6. spec:
      7. ipAddressPools:
      8. - pool2
      9. peers:
      10. - peer2
      11. communities:
      12. - 65535:65282
      13. aggregationLength: 32
      14. aggregationLengthV6: 128
      15. localPref: 100
    2. Apply the configuration:

      1. $ oc apply -f bgpadvertisement2.yaml

Exposing a service through a network VRF

You can expose a service through a virtual routing and forwarding (VRF) instance by associating a VRF on a network interface with a BGP peer.

Exposing a service through a VRF on a BGP peer is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.

By using a VRF on a network interface to expose a service through a BGP peer, you can segregate traffic to the service, configure independent routing decisions, and enable multi-tenancy support on a network interface.

By establishing a BGP session through an interface belonging to a network VRF, MetalLB can advertise services through that interface and enable external traffic to reach the service through this interface. However, the network VRF routing table is different from the default VRF routing table used by OVN-Kubernetes. Therefore, the traffic cannot reach the OVN-Kubernetes network infrastructure.

To enable the traffic directed to the service to reach the OVN-Kubernetes network infrastructure, you must configure routing rules to define the next hops for network traffic. See the NodeNetworkConfigurationPolicy resource in “Managing symmetric routing with MetalLB” in the Additional resources section for more information.

These are the high-level steps to expose a service through a network VRF with a BGP peer:

  1. Define a BGP peer and add a network VRF instance.

  2. Specify an IP address pool for MetalLB.

  3. Configure a BGP route advertisement for MetalLB to advertise a route using the specified IP address pool and the BGP peer associated with the VRF instance.

  4. Deploy a service to test the configuration.

Prerequisites

  • You installed the OpenShift CLI (oc).

  • You logged in as a user with cluster-admin privileges.

  • You defined a NodeNetworkConfigurationPolicy to associate a Virtual Routing and Forwarding (VRF) instance with a network interface. For more information about completing this prerequisite, see the Additional resources section.

  • You installed MetalLB on your cluster.

Procedure

  1. Create a BGPPeer custom resources (CR):

    1. Create a file, such as frrviavrf.yaml, with content like the following example:

      1. apiVersion: metallb.io/v1beta2
      2. kind: BGPPeer
      3. metadata:
      4. name: frrviavrf
      5. namespace: metallb-system
      6. spec:
      7. myASN: 100
      8. peerASN: 200
      9. peerAddress: 192.168.130.1
      10. vrf: ens4vrf (1)
      1Specifies the network VRF instance to associate with the BGP peer. MetalLB can advertise services and make routing decisions based on the routing information in the VRF.

      You must configure this network VRF instance in a NodeNetworkConfigurationPolicy CR. See the Additional resources for more information.

    2. Apply the configuration for the BGP peer by running the following command:

      1. $ oc apply -f frrviavrf.yaml
  2. Create an IPAddressPool CR:

    1. Create a file, such as first-pool.yaml, with content like the following example:

      1. apiVersion: metallb.io/v1beta1
      2. kind: IPAddressPool
      3. metadata:
      4. name: first-pool
      5. namespace: metallb-system
      6. spec:
      7. addresses:
      8. - 192.169.10.0/32
    2. Apply the configuration for the IP address pool by running the following command:

      1. $ oc apply -f first-pool.yaml
  3. Create a BGPAdvertisement CR:

    1. Create a file, such as first-adv.yaml, with content like the following example:

      1. apiVersion: metallb.io/v1beta1
      2. kind: BGPAdvertisement
      3. metadata:
      4. name: first-adv
      5. namespace: metallb-system
      6. spec:
      7. ipAddressPools:
      8. - first-pool
      9. peers:
      10. - frrviavrf (1)
      1In this example, MetalLB advertises a range of IP addresses from the first-pool IP address pool to the frrviavrf BGP peer.
    2. Apply the configuration for the BGP advertisement by running the following command:

      1. $ oc apply -f first-adv.yaml
  4. Create a Namespace, Deployment, and Service CR:

    1. Create a file, such as deploy-service.yaml, with content like the following example:

      1. apiVersion: v1
      2. kind: Namespace
      3. metadata:
      4. name: test
      5. ---
      6. apiVersion: apps/v1
      7. kind: Deployment
      8. metadata:
      9. name: server
      10. namespace: test
      11. spec:
      12. selector:
      13. matchLabels:
      14. app: server
      15. template:
      16. metadata:
      17. labels:
      18. app: server
      19. spec:
      20. containers:
      21. - name: server
      22. image: registry.redhat.io/ubi9/ubi
      23. ports:
      24. - name: http
      25. containerPort: 30100
      26. command: ["/bin/sh", "-c"]
      27. args: ["sleep INF"]
      28. ---
      29. apiVersion: v1
      30. kind: Service
      31. metadata:
      32. name: server1
      33. namespace: test
      34. spec:
      35. ports:
      36. - name: http
      37. port: 30100
      38. protocol: TCP
      39. targetPort: 30100
      40. selector:
      41. app: server
      42. type: LoadBalancer
    2. Apply the configuration for the namespace, deployment, and service by running the following command:

      1. $ oc apply -f deploy-service.yaml

Verification

  1. Identify a MetalLB speaker pod by running the following command:

    1. $ oc get -n metallb-system pods -l component=speaker

    Example output

    1. NAME READY STATUS RESTARTS AGE
    2. speaker-c6c5f 6/6 Running 0 69m
  2. Verify that the state of the BGP session is Established in the speaker pod by running the following command, replacing the variables to match your configuration:

    1. $ oc exec -n metallb-system <speaker_pod> -c frr -- vtysh -c "show bgp vrf <vrf_name> neigh"

    Example output

    1. BGP neighbor is 192.168.30.1, remote AS 200, local AS 100, external link
    2. BGP version 4, remote router ID 192.168.30.1, local router ID 192.168.30.71
    3. BGP state = Established, up for 04:20:09
    4. ...
  3. Verify that the service is advertised correctly by running the following command:

    1. $ oc exec -n metallb-system <speaker_pod> -c frr -- vtysh -c "show bgp vrf <vrf_name> ipv4"

Additional resources

Example BGP peer configurations

Example: Limit which nodes connect to a BGP peer

You can specify the node selectors field to control which nodes can connect to a BGP peer.

  1. apiVersion: metallb.io/v1beta2
  2. kind: BGPPeer
  3. metadata:
  4. name: doc-example-nodesel
  5. namespace: metallb-system
  6. spec:
  7. peerAddress: 10.0.20.1
  8. peerASN: 64501
  9. myASN: 64500
  10. nodeSelectors:
  11. - matchExpressions:
  12. - key: kubernetes.io/hostname
  13. operator: In
  14. values: [compute-1.example.com, compute-2.example.com]

Example: Specify a BFD profile for a BGP peer

You can specify a BFD profile to associate with BGP peers. BFD compliments BGP by providing more rapid detection of communication failures between peers than BGP alone.

  1. apiVersion: metallb.io/v1beta2
  2. kind: BGPPeer
  3. metadata:
  4. name: doc-example-peer-bfd
  5. namespace: metallb-system
  6. spec:
  7. peerAddress: 10.0.20.1
  8. peerASN: 64501
  9. myASN: 64500
  10. holdTime: "10s"
  11. bfdProfile: doc-example-bfd-profile-full

Deleting the bidirectional forwarding detection (BFD) profile and removing the bfdProfile added to the border gateway protocol (BGP) peer resource does not disable the BFD. Instead, the BGP peer starts using the default BFD profile. To disable BFD from a BGP peer resource, delete the BGP peer configuration and recreate it without a BFD profile. For more information, see BZ#2050824.

Example: Specify BGP peers for dual-stack networking

To support dual-stack networking, add one BGP peer custom resource for IPv4 and one BGP peer custom resource for IPv6.

  1. apiVersion: metallb.io/v1beta2
  2. kind: BGPPeer
  3. metadata:
  4. name: doc-example-dual-stack-ipv4
  5. namespace: metallb-system
  6. spec:
  7. peerAddress: 10.0.20.1
  8. peerASN: 64500
  9. myASN: 64500
  10. ---
  11. apiVersion: metallb.io/v1beta2
  12. kind: BGPPeer
  13. metadata:
  14. name: doc-example-dual-stack-ipv6
  15. namespace: metallb-system
  16. spec:
  17. peerAddress: 2620:52:0:88::104
  18. peerASN: 64500
  19. myASN: 64500

Next steps