Understanding Windows container workloads

Windows Container Support for Red Hat OpenShift provides built-in support for running Microsoft Windows Server containers on OKD. For those that administer heterogeneous environments with a mix of Linux and Windows workloads, OKD allows you to deploy Windows workloads running on Windows Server containers while also providing traditional Linux workloads hosted on Fedora CoreOS (FCOS) or Fedora.

Multi-tenancy for clusters that have Windows nodes is not supported. Hostile multi-tenant usage introduces security concerns in all Kubernetes environments. Additional security features like pod security policies, or more fine-grained role-based access control (RBAC) for nodes, make exploits more difficult. However, if you choose to run hostile multi-tenant workloads, a hypervisor is the only security option you should use. The security domain for Kubernetes encompasses the entire cluster, not an individual node. For these types of hostile multi-tenant workloads, you should use physically isolated clusters.

Windows Server Containers provide resource isolation using a shared kernel but are not intended to be used in hostile multitenancy scenarios. Scenarios that involve hostile multitenancy should use Hyper-V Isolated Containers to strongly isolate tenants.

Windows Machine Config Operator prerequisites

The following information details the supported platform versions, Windows Server versions, and networking configurations for the Windows Machine Config Operator. See the vSphere documentation for any information that is relevant to only that platform.

Supported platforms based on OKD and WMCO versions

PlatformSupported OKD versionSupported WMCO versionInstaller-provisioned infrastructure installation supportUser-provisioned infrastructure installation support

Amazon Web Services (AWS)

4.6+

WMCO 1.0+

GA

Tech Preview

Microsoft Azure

4.6+

WMCO 1.0+

GA

Tech Preview

VMware vSphere

4.7+

WMCO 2.0+

GA

Tech Preview

Supported platforms for Bring-Your-Own-Host (BYOH) instances based on OKD and WMCO versions

PlatformSupported OKD versionSupported WMCO versionBYOH for installer-provisioned infrastructure installation supportBYOH for user-provisioned infrastructure installation support

Amazon Web Services (AWS)

4.8+

WMCO 3.1+

GA

Tech Preview

Microsoft Azure

4.8+

WMCO 3.1+

GA

Tech Preview

VMware vSphere

4.8+

WMCO 3.1+

GA

GA[1]

bare metal

4.8+

WMCO 3.1+

GA[1]

  1. This installation type is only supported when the platform: none field is set in the install-config.yaml file during cluster installation.

Supported Windows Server versions

The following table lists the supported Windows Server version based on the applicable platform. Any unlisted Windows Server version is not supported and will cause errors. To prevent these errors, only use the appropriate version according to the platform in use.

PlatformSupported Windows Server version

Amazon Web Services (AWS)

Windows Server Long-Term Servicing Channel (LTSC): Windows Server 2019

Microsoft Azure

Windows Server Long-Term Servicing Channel (LTSC): Windows Server 2019

VMware vSphere

Windows Server Semi-Annual Channel (SAC): Windows Server 2004 and 20H2

Supported networking

Hybrid networking with OVN-Kubernetes is the only supported networking configuration. See the additional resources below for more information on this functionality. The following tables outline the type of networking configuration and Windows Server versions to use based on your platform. You must specify the network configuration when you install the cluster. Be aware that OpenShift SDN networking is the default network for OKD clusters. However, OpenShift SDN is not supported by WMCO.

Table 1. Platform networking support
PlatformSupported networking

Amazon Web Services (AWS)

Hybrid networking with OVN-Kubernetes

Microsoft Azure

Hybrid networking with OVN-Kubernetes

VMware vSphere

Hybrid networking with OVN-Kubernetes with a custom VXLAN port

Table 2. Hybrid OVN-Kubernetes Windows Server support
Hybrid networking with OVN-KubernetesSupported Windows Server version

Default VXLAN port

Windows Server Long-Term Servicing Channel (LTSC): Windows Server 2019

Custom VXLAN port

Windows Server Semi-Annual Channel (SAC): Windows Server 2004 and 20H2

Supported installation method

The installer-provisioned infrastructure installation method is the only supported installation method. This is consistent across all supported platforms. The user-provisioned infrastructure installation method is Tech Preview for all supported platforms.

Additional resources

Windows workload management

To run Windows workloads in your cluster, you must first install the Windows Machine Config Operator (WMCO). The WMCO is a Linux-based Operator that runs on Linux-based control plane and compute nodes. The WMCO orchestrates the process of deploying and managing Windows workloads on a cluster.

WMCO workflow

Figure 1. WMCO design

Before deploying Windows workloads, you must create a Windows compute node and have it join the cluster. The Windows node hosts the Windows workloads in a cluster, and can run alongside other Linux-based compute nodes. You can create a Windows compute node by creating a Windows machine set to host Windows Server compute machines. You must apply a Windows-specific label to the machine set that specifies a Windows OS image that has the Docker-formatted container runtime add-on enabled.

Currently, the Docker-formatted container runtime is used in Windows nodes. Kubernetes is deprecating Docker as a container runtime; you can reference the Kubernetes documentation for more information in Docker deprecation. Containerd will be the new supported container runtime for Windows nodes in a future release of Kubernetes.

The WMCO watches for machines with the Windows label. After a Windows machine set is detected and its respective machines are provisioned, the WMCO configures the underlying Windows virtual machine (VM) so that it can join the cluster as a compute node.

Mixed Windows and Linux workloads

Figure 2. Mixed Windows and Linux workloads

The WMCO expects a predetermined secret in its namespace containing a private key that is used to interact with the Windows instance. WMCO checks for this secret during boot up time and creates a user data secret which you must reference in the Windows MachineSet object that you created. Then the WMCO populates the user data secret with a public key that corresponds to the private key. With this data in place, the cluster can connect to the Windows VM using an SSH connection.

After the cluster establishes a connection with the Windows VM, you can manage the Windows node using similar practices as you would a Linux-based node.

The OKD web console provides most of the same monitoring capabilities for Windows nodes that are available for Linux nodes. However, the ability to monitor workload graphs for pods running on Windows nodes is not available at this time.

Scheduling Windows workloads to a Windows node can be done with typical pod scheduling practices like taints, tolerations, and node selectors; alternatively, you can differentiate your Windows workloads from Linux workloads and other Windows-versioned workloads by using a RuntimeClass object.

Windows node services

The following Windows-specific services are installed on each Windows node:

ServiceDescription

kubelet

Registers the Windows node and manages its status.

Container Network Interface (CNI) plug-ins

Exposes networking for Windows nodes.

Windows Machine Config Bootstrapper (WMCB)

Configures the kubelet and CNI plug-ins.

hybrid-overlay

Creates the OKD Host Network Service (HNS).

kube-proxy

Maintains network rules on nodes allowing outside communication.