Placing pods on specific nodes using node selectors

A node selector specifies a map of key-value pairs. The rules are defined using custom labels on nodes and selectors specified in pods.

For the pod to be eligible to run on a node, the pod must have the indicated key-value pairs as the label on the node.

If you are using node affinity and node selectors in the same pod configuration, see the important considerations below.

Using node selectors to control pod placement

You can use node selectors on pods and labels on nodes to control where the pod is scheduled. With node selectors, OKD schedules the pods on nodes that contain matching labels.

You add labels to a node, a compute machine set, or a machine config. Adding the label to the compute machine set ensures that if the node or machine goes down, new nodes have the label. Labels added to a node or machine config do not persist if the node or machine goes down.

To add node selectors to an existing pod, add a node selector to the controlling object for that pod, such as a ReplicaSet object, DaemonSet object, StatefulSet object, Deployment object, or DeploymentConfig object. Any existing pods under that controlling object are recreated on a node with a matching label. If you are creating a new pod, you can add the node selector directly to the pod spec. If the pod does not have a controlling object, you must delete the pod, edit the pod spec, and recreate the pod.

You cannot add a node selector directly to an existing scheduled pod.

Prerequisites

To add a node selector to existing pods, determine the controlling object for that pod. For example, the router-default-66d5cf9464-m2g75 pod is controlled by the router-default-66d5cf9464 replica set:

  1. $ oc describe pod router-default-66d5cf9464-7pwkc

Example output

  1. kind: Pod
  2. apiVersion: v1
  3. metadata:
  4. # ...
  5. Name: router-default-66d5cf9464-7pwkc
  6. Namespace: openshift-ingress
  7. # ...
  8. Controlled By: ReplicaSet/router-default-66d5cf9464
  9. # ...

The web console lists the controlling object under ownerReferences in the pod YAML:

  1. apiVersion: v1
  2. kind: Pod
  3. metadata:
  4. name: router-default-66d5cf9464-7pwkc
  5. # ...
  6. ownerReferences:
  7. - apiVersion: apps/v1
  8. kind: ReplicaSet
  9. name: router-default-66d5cf9464
  10. uid: d81dd094-da26-11e9-a48a-128e7edf0312
  11. controller: true
  12. blockOwnerDeletion: true
  13. # ...

Procedure

  1. Add labels to a node by using a compute machine set or editing the node directly:

    • Use a MachineSet object to add labels to nodes managed by the compute machine set when a node is created:

      1. Run the following command to add labels to a MachineSet object:

        1. $ oc patch MachineSet <name> --type='json' -p='[{"op":"add","path":"/spec/template/spec/metadata/labels", "value":{"<key>"="<value>","<key>"="<value>"}}]' -n openshift-machine-api

        For example:

        1. $ oc patch MachineSet abc612-msrtw-worker-us-east-1c --type='json' -p='[{"op":"add","path":"/spec/template/spec/metadata/labels", "value":{"type":"user-node","region":"east"}}]' -n openshift-machine-api

        You can alternatively apply the following YAML to add labels to a compute machine set:

        1. apiVersion: machine.openshift.io/v1beta1
        2. kind: MachineSet
        3. metadata:
        4. name: xf2bd-infra-us-east-2a
        5. namespace: openshift-machine-api
        6. spec:
        7. template:
        8. spec:
        9. metadata:
        10. labels:
        11. region: east
        12. type: user-node
        13. # …
      2. Verify that the labels are added to the MachineSet object by using the oc edit command:

        For example:

        1. $ oc edit MachineSet abc612-msrtw-worker-us-east-1c -n openshift-machine-api

        Example MachineSet object

        1. apiVersion: machine.openshift.io/v1beta1
        2. kind: MachineSet
        3. # ...
        4. spec:
        5. # ...
        6. template:
        7. metadata:
        8. # ...
        9. spec:
        10. metadata:
        11. labels:
        12. region: east
        13. type: user-node
        14. # ...
    • Add labels directly to a node:

      1. Edit the Node object for the node:

        1. $ oc label nodes <name> <key>=<value>

        For example, to label a node:

        1. $ oc label nodes ip-10-0-142-25.ec2.internal type=user-node region=east

        You can alternatively apply the following YAML to add labels to a node:

        1. kind: Node
        2. apiVersion: v1
        3. metadata:
        4. name: hello-node-6fbccf8d9
        5. labels:
        6. type: user-node
        7. region: east
        8. # …
      2. Verify that the labels are added to the node:

        1. $ oc get nodes -l type=user-node,region=east

        Example output

        1. NAME STATUS ROLES AGE VERSION
        2. ip-10-0-142-25.ec2.internal Ready worker 17m v1.28.5
  1. Add the matching node selector to a pod:

    • To add a node selector to existing and future pods, add a node selector to the controlling object for the pods:

      Example ReplicaSet object with labels

      1. kind: ReplicaSet
      2. apiVersion: apps/v1
      3. metadata:
      4. name: hello-node-6fbccf8d9
      5. # ...
      6. spec:
      7. # ...
      8. template:
      9. metadata:
      10. creationTimestamp: null
      11. labels:
      12. ingresscontroller.operator.openshift.io/deployment-ingresscontroller: default
      13. pod-template-hash: 66d5cf9464
      14. spec:
      15. nodeSelector:
      16. kubernetes.io/os: linux
      17. node-role.kubernetes.io/worker: ''
      18. type: user-node (1)
      19. # ...
      1Add the node selector.
    • To add a node selector to a specific, new pod, add the selector to the Pod object directly:

      Example Pod object with a node selector

      1. apiVersion: v1
      2. kind: Pod
      3. metadata:
      4. name: hello-node-6fbccf8d9
      5. # ...
      6. spec:
      7. nodeSelector:
      8. region: east
      9. type: user-node
      10. # ...

      You cannot add a node selector directly to an existing scheduled pod.