Placing pods on specific nodes using node selectors

A node selector specifies a map of key-value pairs. The rules are defined using custom labels on nodes and selectors specified in pods.

For the pod to be eligible to run on a node, the pod must have the indicated key-value pairs as the label on the node.

If you are using node affinity and node selectors in the same pod configuration, see the important considerations below.

Using node selectors to control pod placement

You can use node selectors on pods and labels on nodes to control where the pod is scheduled. With node selectors, OKD schedules the pods on nodes that contain matching labels.

You add labels to a node, a machine set, or a machine config. Adding the label to the machine set ensures that if the node or machine goes down, new nodes have the label. Labels added to a node or machine config do not persist if the node or machine goes down.

To add node selectors to an existing pod, add a node selector to the controlling object for that pod, such as a ReplicaSet object, DaemonSet object, StatefulSet object, Deployment object, or DeploymentConfig object. Any existing pods under that controlling object are recreated on a node with a matching label. If you are creating a new pod, you can add the node selector directly to the Pod spec.

You cannot add a node selector directly to an existing scheduled pod.

Prerequisites

To add a node selector to existing pods, determine the controlling object for that pod. For example, the router-default-66d5cf9464-m2g75 pod is controlled by the router-default-66d5cf9464 replica set:

  1. $ oc describe pod router-default-66d5cf9464-7pwkc
  2. Name: router-default-66d5cf9464-7pwkc
  3. Namespace: openshift-ingress
  4. ....
  5. Controlled By: ReplicaSet/router-default-66d5cf9464

The web console lists the controlling object under ownerReferences in the pod YAML:

  1. ownerReferences:
  2. - apiVersion: apps/v1
  3. kind: ReplicaSet
  4. name: router-default-66d5cf9464
  5. uid: d81dd094-da26-11e9-a48a-128e7edf0312
  6. controller: true
  7. blockOwnerDeletion: true

Procedure

  1. Add labels to a node by using a machine set or editing the node directly:

    • Use a MachineSet object to add labels to nodes managed by the machine set when a node is created:

      1. Run the following command to add labels to a MachineSet object:

        1. $ oc patch MachineSet <name> --type='json' -p='[{"op":"add","path":"/spec/template/spec/metadata/labels", "value":{"<key>"="<value>","<key>"="<value>"}}]' -n openshift-machine-api

        For example:

        1. $ oc patch MachineSet abc612-msrtw-worker-us-east-1c --type='json' -p='[{"op":"add","path":"/spec/template/spec/metadata/labels", "value":{"type":"user-node","region":"east"}}]' -n openshift-machine-api
      2. Verify that the labels are added to the MachineSet object by using the oc edit command:

        For example:

        1. $ oc edit MachineSet abc612-msrtw-worker-us-east-1c -n openshift-machine-api

        Example MachineSet object

        1. apiVersion: machine.openshift.io/v1beta1
        2. kind: MachineSet
        3. ....
        4. spec:
        5. ...
        6. template:
        7. metadata:
        8. ...
        9. spec:
        10. metadata:
        11. labels:
        12. region: east
        13. type: user-node
        14. ....
    • Add labels directly to a node:

      1. Edit the Node object for the node:

        1. $ oc label nodes <name> <key>=<value>

        For example, to label a node:

        1. $ oc label nodes ip-10-0-142-25.ec2.internal type=user-node region=east
      2. Verify that the labels are added to the node:

        1. $ oc get nodes -l type=user-node,region=east

        Example output

        1. NAME STATUS ROLES AGE VERSION
        2. ip-10-0-142-25.ec2.internal Ready worker 17m v1.18.3+002a51f
  1. Add the matching node selector to a pod:

    • To add a node selector to existing and future pods, add a node selector to the controlling object for the pods:

      Example ReplicaSet object with labels

      1. kind: ReplicaSet
      2. ....
      3. spec:
      4. ....
      5. template:
      6. metadata:
      7. creationTimestamp: null
      8. labels:
      9. ingresscontroller.operator.openshift.io/deployment-ingresscontroller: default
      10. pod-template-hash: 66d5cf9464
      11. spec:
      12. nodeSelector:
      13. beta.kubernetes.io/os: linux
      14. node-role.kubernetes.io/worker: ''
      15. type: user-node (1)
      1Add the node selector.
    • To add a node selector to a specific, new pod, add the selector to the Pod object directly:

      Example Pod object with a node selector

      1. apiVersion: v1
      2. kind: Pod
      3. ....
      4. spec:
      5. nodeSelector:
      6. region: east
      7. type: user-node

      You cannot add a node selector directly to an existing scheduled pod.