Kata Containers with Cilium

Kata Containers is an open source project that provides a secure container runtime with lightweight virtual machines that feel and perform like containers, but provide stronger workload isolation using hardware virtualization technology as a second layer of defense. Kata Containers implements OCI runtime spec, just like runc that is used by Docker. Cilium can be used along with Kata Containers, using both enables higher degree of security. Kata Containers enhances security in the compute layer, while Cilium provides policy and observability in the networking layer.

This guide shows how to install Cilium along with Kata Containers. It assumes that you have already followed the official Kata Containers installation user guide to get the Kata Containers runtime up and running on your platform of choice but that you haven’t yet setup Kubernetes.

Note

This guide has been validated by following the Kata Containers guide for Google Compute Engine (GCE) and using Ubuntu 18.04 LTS with the packaged version of Kata Containers, CRI-containerd and Kubernetes 1.18.3.

Setup Kubernetes with CRI

Kata Containers runtime is an OCI compatible runtime and cannot directly interact with the CRI API level. For this reason, it relies on a CRI implementation to translate CRI into OCI. At the time of writing this guide, there are two supported ways called CRI-O and CRI-containerd. It is up to you to choose the one that you want, but you have to pick one.

Refer to the section Requirements for detailed instruction on how to prepare your Kubernetes environment and make sure to use Kubernetes >= 1.12. Then, follow the official guide to run Kata Containers with Kubernetes.

Note

Minimum version of kubernetes 1.12 is required to use the RuntimeClass Feature for Kata Container runtime described below.

With your Kubernetes cluster ready, you can now proceed to deploy Cilium.

Deploy Cilium

Note

First, make sure you have Helm 3 installed.

If you have (or planning to have) Helm 2 charts (and Tiller) in the same cluster, there should be no issue as both version are mutually compatible in order to support gradual migration. Cilium chart is targeting Helm 3 (v3.0.3 and above).

Setup Helm repository:

  1. helm repo add cilium https://helm.cilium.io/

Deploy Cilium release via Helm:

Using CRI-O

Using CRI-containerd

  1. helm install cilium cilium/cilium --version 1.8.10 \
  2.   --namespace kube-system \
  3.   --set global.containerRuntime.integration=crio
  1. helm install cilium cilium/cilium --version 1.8.10 \
  2.   --namespace kube-system \
  3.   --set global.containerRuntime.integration=containerd

Validate the Installation

You can monitor as Cilium and all required components are being installed:

  1. kubectl -n kube-system get pods --watch
  2. NAME                                    READY   STATUS              RESTARTS   AGE
  3. cilium-operator-cb4578bc5-q52qk         0/1     Pending             0          8s
  4. cilium-s8w5m                            0/1     PodInitializing     0          7s
  5. coredns-86c58d9df4-4g7dd                0/1     ContainerCreating   0          8m57s
  6. coredns-86c58d9df4-4l6b2                0/1     ContainerCreating   0          8m57s

It may take a couple of minutes for all components to come up:

  1. cilium-operator-cb4578bc5-q52qk         1/1     Running   0          4m13s
  2. cilium-s8w5m                            1/1     Running   0          4m12s
  3. coredns-86c58d9df4-4g7dd                1/1     Running   0          13m
  4. coredns-86c58d9df4-4l6b2                1/1     Running   0          13m

Deploy the connectivity test

You can deploy the “connectivity-check” to test connectivity between pods. It is recommended to create a separate namespace for this.

  1. kubectl create ns cilium-test

Deploy the check with:

  1. kubectl apply -n cilium-test -f https://raw.githubusercontent.com/cilium/cilium/v1.8/examples/kubernetes/connectivity-check/connectivity-check.yaml

It will deploy a series of deployments which will use various connectivity paths to connect to each other. Connectivity paths include with and without service load-balancing and various network policy combinations. The pod name indicates the connectivity variant and the readiness and liveness gate indicates success or failure of the test:

  1. $ kubectl get pods -n cilium-test
  2. NAME                                                    READY   STATUS    RESTARTS   AGE
  3. echo-a-6788c799fd-42qxx                                 1/1     Running   0          69s
  4. echo-b-59757679d4-pjtdl                                 1/1     Running   0          69s
  5. echo-b-host-f86bd784d-wnh4v                             1/1     Running   0          68s
  6. host-to-b-multi-node-clusterip-585db65b4d-x74nz         1/1     Running   0          68s
  7. host-to-b-multi-node-headless-77c64bc7d8-kgf8p          1/1     Running   0          67s
  8. pod-to-a-allowed-cnp-87b5895c8-bfw4x                    1/1     Running   0          68s
  9. pod-to-a-b76ddb6b4-2v4kb                                1/1     Running   0          68s
  10. pod-to-a-denied-cnp-677d9f567b-kkjp4                    1/1     Running   0          68s
  11. pod-to-b-intra-node-nodeport-8484fb6d89-bwj8q           1/1     Running   0          68s
  12. pod-to-b-multi-node-clusterip-f7655dbc8-h5bwk           1/1     Running   0          68s
  13. pod-to-b-multi-node-headless-5fd98b9648-5bjj8           1/1     Running   0          68s
  14. pod-to-b-multi-node-nodeport-74bd8d7bd5-kmfmm           1/1     Running   0          68s
  15. pod-to-external-1111-7489c7c46d-jhtkr                   1/1     Running   0          68s
  16. pod-to-external-fqdn-allow-google-cnp-b7b6bcdcb-97p75   1/1     Running   0          68s

Note

If you deploy the connectivity check to a single node cluster, pods that check multi-node functionalities will remain in the Pending state. This is expected since these pods need at least 2 nodes to be scheduled successfully.

Run Kata Containers with Cilium CNI

Now that your Kubernetes cluster is configured with the Kata Containers runtime and Cilium as the CNI, you can run a sample workload by following these instructions.