Configuring an SR-IOV Ethernet network attachment

You can configure an Ethernet network attachment for an Single Root I/O Virtualization (SR-IOV) device in the cluster.

Ethernet device configuration object

You can configure an Ethernet network device by defining an SriovNetwork object.

The following YAML describes an SriovNetwork object:

  1. apiVersion: sriovnetwork.openshift.io/v1
  2. kind: SriovNetwork
  3. metadata:
  4. name: <name> (1)
  5. namespace: openshift-sriov-network-operator (2)
  6. spec:
  7. resourceName: <sriov_resource_name> (3)
  8. networkNamespace: <target_namespace> (4)
  9. vlan: <vlan> (5)
  10. spoofChk: "<spoof_check>" (6)
  11. ipam: |- (7)
  12. {}
  13. linkState: <link_state> (8)
  14. maxTxRate: <max_tx_rate> (9)
  15. minTxRate: <min_tx_rate> (10)
  16. vlanQoS: <vlan_qos> (11)
  17. trust: "<trust_vf>" (12)
  18. capabilities: <capabilities> (13)
1A name for the object. The SR-IOV Network Operator creates a NetworkAttachmentDefinition object with same name.
2The namespace where the SR-IOV Network Operator is installed.
3The value for the spec.resourceName parameter from the SriovNetworkNodePolicy object that defines the SR-IOV hardware for this additional network.
4The target namespace for the SriovNetwork object. Only pods in the target namespace can attach to the additional network.
5Optional: A Virtual LAN (VLAN) ID for the additional network. The integer value must be from 0 to 4095. The default value is 0.
6Optional: The spoof check mode of the VF. The allowed values are the strings “on” and “off”.

You must enclose the value you specify in quotes or the object is rejected by the SR-IOV Network Operator.

7A configuration object for the IPAM CNI plug-in as a YAML block scalar. The plug-in manages IP address assignment for the attachment definition.
8Optional: The link state of virtual function (VF). Allowed value are enable, disable and auto.
9Optional: A maximum transmission rate, in Mbps, for the VF.
10Optional: A minimum transmission rate, in Mbps, for the VF. This value must be less than or equal to the maximum transmission rate.

Intel NICs do not support the minTxRate parameter. For more information, see BZ#1772847.

11Optional: An IEEE 802.1p priority level for the VF. The default value is 0.
12Optional: The trust mode of the VF. The allowed values are the strings “on” and “off”.

You must enclose the value that you specify in quotes, or the SR-IOV Network Operator rejects the object.

13Optional: The capabilities to configure for this additional network. You can specify “{ “ips”: true }” to enable IP address support or “{ “mac”: true }” to enable MAC address support.

Configuration for ipam CNI plug-in

The ipam Container Network Interface (CNI) plug-in provides IP address management (IPAM) for other CNI plug-ins.

You can use the following methods for IP address assignment:

  • Static assignment.

  • Dynamic assignment through a DHCP server. The DHCP server you specify must be reachable from the additional network.

  • Dynamic assignment through the Whereabouts IPAM CNI plug-in.

Static IP address assignment configuration

The following JSON describes the configuration for static IP address assignment:

Static assignment configuration

  1. {
  2. "ipam": {
  3. "type": "static",
  4. "addresses": [ (1)
  5. {
  6. "address": "<address>", (2)
  7. "gateway": "<gateway>" (3)
  8. }
  9. ],
  10. "routes": [ (4)
  11. {
  12. "dst": "<dst>", (5)
  13. "gw": "<gw>" (6)
  14. }
  15. ],
  16. "dns": { (7)
  17. "nameservers": ["<nameserver>"], (8)
  18. "domain": "<domain>", (9)
  19. "search": ["<search_domain>"] (10)
  20. }
  21. }
  22. }
1An array describing IP addresses to assign to the virtual interface. Both IPv4 and IPv6 IP addresses are supported.
2An IP address and network prefix that you specify. For example, if you specify 10.10.21.10/24, then the additional network is assigned an IP address of 10.10.21.10 and the netmask is 255.255.255.0.
3The default gateway to route egress network traffic to.
4An array describing routes to configure inside the pod.
5The IP address range in CIDR format, such as 192.168.17.0/24, or 0.0.0.0/0 for the default route.
6The gateway where network traffic is routed.
7Optional: DNS configuration.
8An of array of one or more IP addresses for to send DNS queries to.
9The default domain to append to a hostname. For example, if the domain is set to example.com, a DNS lookup query for example-host is rewritten as example-host.example.com.
10An array of domain names to append to an unqualified hostname, such as example-host, during a DNS lookup query.

Dynamic IP address assignment configuration

The following JSON describes the configuration for dynamic IP address address assignment with DHCP.

Renewal of DHCP leases

A pod obtains its original DHCP lease when it is created. The lease must be periodically renewed by a minimal DHCP server deployment running on the cluster.

The SR-IOV Network Operator does not create a DHCP server deployment; The Cluster Network Operator is responsible for creating the minimal DHCP server deployment.

To trigger the deployment of the DHCP server, you must create a shim network attachment by editing the Cluster Network Operator configuration, as in the following example:

Example shim network attachment definition
  1. apiVersion: operator.openshift.io/v1
  2. kind: Network
  3. metadata:
  4. name: cluster
  5. spec:
  6. additionalNetworks:
  7. - name: dhcp-shim
  8. namespace: default
  9. type: Raw
  10. rawCNIConfig: |-
  11. {
  12. name”: dhcp-shim”,
  13. cniVersion”: 0.3.1”,
  14. type”: bridge”,
  15. ipam”: {
  16. type”: dhcp
  17. }
  18. }

DHCP assignment configuration

  1. {
  2. "ipam": {
  3. "type": "dhcp"
  4. }
  5. }

Dynamic IP address assignment configuration with Whereabouts

The Whereabouts CNI plug-in allows the dynamic assignment of an IP address to an additional network without the use of a DHCP server.

The following JSON describes the configuration for dynamic IP address assignment with Whereabouts:

Whereabouts assignment configuration

  1. {
  2. "ipam": {
  3. "type": "whereabouts",
  4. "range": "<range>", (1)
  5. "exclude": ["<exclude_part>, ..."], (2)
  6. }
  7. }
1Specify an IP address and range in CIDR notation. IP addresses are assigned from within this range of addresses.
2Optional: Specify a list of IP addresses and ranges in CIDR notation. IP addresses within an excluded address range are not assigned.

Static IP address assignment configuration example

You can configure ipam for static IP address assignment:

  1. {
  2. "ipam": {
  3. "type": "static",
  4. "addresses": [
  5. {
  6. "address": "191.168.1.7"
  7. }
  8. ]
  9. }
  10. }

Dynamic IP address assignment configuration example using DHCP

You can configure ipam for DHCP:

  1. {
  2. "ipam": {
  3. "type": "dhcp"
  4. }
  5. }

Dynamic IP address assignment configuration example using Whereabouts

You can configure ipam to use Whereabouts:

  1. {
  2. "ipam": {
  3. "type": "whereabouts",
  4. "range": "192.0.2.192/27",
  5. "exclude": [
  6. "192.0.2.192/30",
  7. "192.0.2.196/32"
  8. ]
  9. }
  10. }

Configuring SR-IOV additional network

You can configure an additional network that uses SR-IOV hardware by creating a SriovNetwork object. When you create a SriovNetwork object, the SR-IOV Operator automatically creates a NetworkAttachmentDefinition object.

Do not modify or delete a SriovNetwork object if it is attached to any pods in the running state.

Prerequisites

  • Install the OpenShift CLI (oc).

  • Log in as a user with cluster-admin privileges.

Procedure

  1. Create a SriovNetwork object, and then save the YAML in the <name>.yaml file, where <name> is a name for this additional network. The object specification might resemble the following example:

    1. apiVersion: sriovnetwork.openshift.io/v1
    2. kind: SriovNetwork
    3. metadata:
    4. name: attach1
    5. namespace: openshift-sriov-network-operator
    6. spec:
    7. resourceName: net1
    8. networkNamespace: project2
    9. ipam: |-
    10. {
    11. "type": "host-local",
    12. "subnet": "10.56.217.0/24",
    13. "rangeStart": "10.56.217.171",
    14. "rangeEnd": "10.56.217.181",
    15. "gateway": "10.56.217.1"
    16. }
  2. To create the object, enter the following command:

    1. $ oc create -f <name>.yaml

    where <name> specifies the name of the additional network.

  3. Optional: To confirm that the NetworkAttachmentDefinition object that is associated with the SriovNetwork object that you created in the previous step exists, enter the following command. Replace <namespace> with the networkNamespace you specified in the SriovNetwork object.

    1. $ oc get net-attach-def -n <namespace>

Next steps

Additional resources