Quick Installation

This guide will walk you through the quick default installation. It will automatically detect and use the best configuration possible for the Kubernetes distribution you are using. All state is stored using Kubernetes CRDs.

This is the best installation method for most use cases. For large environments (> 500 nodes) or if you want to run specific datapath modes, refer to the Advanced Installation guide.

Should you encounter any issues during the installation, please refer to the Troubleshooting section and / or seek help on the Slack channel.

Create the Cluster

If you don’t have a Kubernetes Cluster yet, you can use the instructions below to create a Kubernetes cluster locally or using a managed Kubernetes service:

GKE

AKS

EKS

kind

minikube

The following command creates a Kubernetes cluster using Google Kubernetes Engine. See Installing Google Cloud SDK for instructions on how to install gcloud and prepare your account.

  1. export NAME="$(whoami)-$RANDOM"
  2. # Create the node pool with the following taint to guarantee that
  3. # Pods are only scheduled in the node when Cilium is ready.
  4. gcloud container clusters create "${NAME}" \
  5.  --node-taints node.cilium.io/agent-not-ready=true:NoSchedule \
  6.  --zone us-west2-a
  7. gcloud container clusters get-credentials "${NAME}" --zone us-west2-a

The following command creates a Kubernetes cluster using Azure Kubernetes Service. See Azure Cloud CLI for instructions on how to install az and prepare your account.

  1. export NAME="$(whoami)-$RANDOM"
  2. export AZURE_RESOURCE_GROUP="aks-cilium-group"
  3. az group create --name "${AZURE_RESOURCE_GROUP}" -l westus2
  5. # Details: Basic load balancers are not supported with multiple node
  6. # pools. Create a cluster with standard load balancer selected to use
  7. # multiple node pools, learn more at aka.ms/aks/nodepools.
  8. az aks create \
  9. --resource-group "${AZURE_RESOURCE_GROUP}" \
  10. --name "${NAME}" \
  11. --network-plugin azure \
  12. --load-balancer-sku standard
  14. # Get the name of the node pool that was just created since it will
  15. # be deleted after Cilium is installed.
  16. nodepool_to_delete=$(az aks nodepool list --cluster-name "${NAME}" -g "${AZURE_RESOURCE_GROUP}" -o json | jq -r '.[0].name')
  18. # Create a node pool with 'mode=system' as it is the same mode used
  19. # for the default nodepool on cluster creation also this new node
  20. # pool will have the taint 'node.cilium.io/agent-not-ready=true:NoSchedule'
  21. # which will guarantee that pods will only be scheduled on that node
  22. # once Cilium is ready.
  23. az aks nodepool add \
  24.   --name "nodepool2" \
  25.   --cluster-name "${NAME}" \
  26.   --resource-group "${AZURE_RESOURCE_GROUP}" \
  27.   --node-count 2 \
  28.   --mode system \
  29.   --node-taints node.cilium.io/agent-not-ready=true:NoSchedule
  31. # Get the credentials to access the cluster with kubectl
  32. az aks get-credentials --name "${NAME}" --resource-group "${AZURE_RESOURCE_GROUP}"
  34. # We can only delete the first node pool after Cilium is installed
  35. # because some pods have Pod Disruption Budgets set. If we try to
  36. # delete the first node pool without the second node pool being ready,
  37. # AKS will not succeed with the pool deletion because some Deployments
  38. # can't cease to exist in the cluster.
  39. #
  40. # NOTE: Only delete the nodepool after deploying Cilium
  41. az aks nodepool delete --name ${nodepool_to_delete} \
  42.   --cluster-name "${NAME}" \
  43.   --resource-group "${AZURE_RESOURCE_GROUP}"

Attention

Do NOT specify the --network-policy flag when creating the cluster, as this will cause the Azure CNI plugin to install unwanted iptables rules.

The following command creates a Kubernetes cluster with eksctl using Amazon Elastic Kubernetes Service. See eksctl Installation for instructions on how to install eksctl and prepare your account.

  1. export NAME="$(whoami)-$RANDOM"
  2. cat <<EOF >eks-config.yaml
  3. apiVersion: eksctl.io/v1alpha5
  4. kind: ClusterConfig
  6. metadata:
  7.   name: ${NAME}
  8.   region: eu-west-1
  10. managedNodeGroups:
  11. - name: ng-1
  12.   desiredCapacity: 2
  13.   privateNetworking: true
  14.   # taint nodes so that application pods are
  15.   # not scheduled until Cilium is deployed.
  16.   taints:
  17.    - key: "node.cilium.io/agent-not-ready"
  18.      value: "true"
  19.      effect: "NoSchedule"
  20. EOF
  21. eksctl create cluster -f ./eks-config.yaml

Install kind >= v0.7.0 per kind documentation: Installation and Usage

  1. curl -LO https://raw.githubusercontent.com/cilium/cilium/v1.10/Documentation/gettingstarted/kind-config.yaml
  2. kind create cluster --config=kind-config.yaml

Install minikube >= v1.5.2 as per minikube documentation: Install Minikube.

  1. minikube start --network-plugin=cni

Install the Cilium CLI

Install the latest version of the Cilium CLI. The Cilium CLI can be used to install Cilium, inspect the state of a Cilium installation, and enable/disable various features (e.g. clustermesh, Hubble).

Linux

macOS

Other

  1. curl -L --remote-name-all https://github.com/cilium/cilium-cli/releases/latest/download/cilium-linux-amd64.tar.gz{,.sha256sum}
  2. sha256sum --check cilium-linux-amd64.tar.gz.sha256sum
  3. sudo tar xzvfC cilium-linux-amd64.tar.gz /usr/local/bin
  4. rm cilium-linux-amd64.tar.gz{,.sha256sum}
  1. curl -L --remote-name-all https://github.com/cilium/cilium-cli/releases/latest/download/cilium-darwin-amd64.tar.gz{,.sha256sum}
  2. shasum -a 256 -c cilium-darwin-amd64.tar.gz.sha256sum
  3. sudo tar xzvfC cilium-darwin-amd64.tar.gz /usr/local/bin
  4. rm cilium-darwin-amd64.tar.gz{,.sha256sum}

See the full page of releases.

Install Cilium

You can install Cilium on any Kubernetes cluster. Pick one of the options below:

Generic

GCP/GKE

Azure/AKS

AWS/EKS

OpenShift

RKE

k3s

These are the generic instructions on how to install Cilium into any Kubernetes cluster. The installer will attempt to automatically pick the best configuration options for you. Please see the other tabs for distribution/platform specific instructions which also list the ideal default configuration for particular platforms.

Requirements:

Tip

See System Requirements for more details on the system requirements.

Install Cilium

Install Cilium into the Kubernetes cluster pointed to by your current kubectl context:

  1. cilium install

To install Cilium on Google Kubernetes Engine (GKE), perform the following steps:

Default Configuration:

DatapathIPAMDatastore
Direct RoutingKubernetes PodCIDRKubernetes CRD

Requirements:

  • The cluster must be created with the taint node.cilium.io/agent-not-ready=true:NoSchedule using --node-taints option.

Install Cilium:

Install Cilium into the GKE cluster:

  1. cilium install

To install Cilium on Azure Kubernetes Service (AKS), perform the following steps:

Default Configuration:

DatapathIPAMDatastore
Direct RoutingAzure IPAMKubernetes CRD

Tip

If you want to chain Cilium on top of the Azure CNI, refer to the guide Azure CNI.

Requirements:

  • The AKS cluster must be created with --network-plugin azure for compatibility with Cilium. The Azure network plugin will be replaced with Cilium by the installer.
  • Node pools must also be created with the taint node.cilium.io/agent-not-ready=true:NoSchedule using --node-taints option.

Limitations:

  • All VMs and VM scale sets used in a cluster must belong to the same resource group.

Install Cilium:

Install Cilium into the AKS cluster:

  1. cilium install --azure-resource-group "${AZURE_RESOURCE_GROUP}"

To install Cilium on Amazon Elastic Kubernetes Service (EKS), perform the following steps:

Default Configuration:

DatapathIPAMDatastore
Direct Routing (ENI)AWS ENIKubernetes CRD

For more information on AWS ENI mode, see AWS ENI.

Tip

If you want to chain Cilium on top of the AWS CNI, refer to the guide AWS VPC CNI plugin.

The following command creates a Kubernetes cluster with eksctl using Amazon Elastic Kubernetes Service. See eksctl Installation for instructions on how to install eksctl and prepare your account.

  1. export NAME="$(whoami)-$RANDOM"
  2. cat <<EOF >eks-config.yaml
  3. apiVersion: eksctl.io/v1alpha5
  4. kind: ClusterConfig
  6. metadata:
  7.   name: ${NAME}
  8.   region: eu-west-1
  10. managedNodeGroups:
  11. - name: ng-1
  12.   desiredCapacity: 2
  13.   privateNetworking: true
  14.   # taint nodes so that application pods are
  15.   # not scheduled until Cilium is deployed.
  16.   taints:
  17.    - key: "node.cilium.io/agent-not-ready"
  18.      value: "true"
  19.      effect: "NoSchedule"
  20. EOF
  21. eksctl create cluster -f ./eks-config.yaml

Limitations:

  • The AWS ENI integration of Cilium is currently only enabled for IPv4. If you want to use IPv6, use a datapath/IPAM mode other than ENI.

Install Cilium:

Install Cilium into the EKS cluster.

  1. cilium install
  2. cilium status --wait

To install Cilium on OpenShift, perform the following steps:

Default Configuration:

DatapathIPAMDatastore
EncapsulationCluster PoolKubernetes CRD

Requirements:

  • OpenShift 4.x

Install Cilium:

Cilium is a Certified OpenShift CNI Plugin and is best installed when an OpenShift cluster is created using the OpenShift installer. Please refer to Installation on OpenShift OKD for more information.

To install Cilium on Rancher Kubernetes Engine (RKE), perform the following steps:

Note

If you are using RKE2, Cilium has been directly integrated. Please see Using Cilium in the RKE2 documentation. You can use either method.

Default Configuration:

DatapathIPAMDatastore
EncapsulationCluster PoolKubernetes CRD

Requirements:

  • Follow the RKE Installation Guide with the below change:

    From:

    1. network:
    2.   options:
    3.     flannel_backend_type: "vxlan"
    4.   plugin: "canal"

    To:

    1. network:
    2.   plugin: none

Install Cilium:

Install Cilium into your newly created RKE cluster:

  1. cilium install

To install Cilium on k3s, perform the following steps:

Default Configuration:

DatapathIPAMDatastore
EncapsulationCluster PoolKubernetes CRD

Requirements:

  • Install your k3s cluster as you would normally would but pass in --flannel-backend=none so you can install Cilium on top:
  1. curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC='--flannel-backend=none' sh -

Install Cilium:

Install Cilium into your newly created Kubernetes cluster:

  1. cilium install

If the installation fails for some reason, run cilium status to retrieve the overall status of the Cilium deployment and inspect the logs of whatever pods are failing to be deployed.

Tip

You may be seeing cilium install print something like this:

  1. ♻️  Restarted unmanaged pod kube-system/event-exporter-gke-564fb97f9-rv8hg
  2. ♻️  Restarted unmanaged pod kube-system/kube-dns-6465f78586-hlcrz
  3. ♻️  Restarted unmanaged pod kube-system/kube-dns-autoscaler-7f89fb6b79-fsmsg
  4. ♻️  Restarted unmanaged pod kube-system/l7-default-backend-7fd66b8b88-qqhh5
  5. ♻️  Restarted unmanaged pod kube-system/metrics-server-v0.3.6-7b5cdbcbb8-kjl65
  6. ♻️  Restarted unmanaged pod kube-system/stackdriver-metadata-agent-cluster-level-6cc964cddf-8n2rt

This indicates that your cluster was already running some pods before Cilium was deployed and the installer has automatically restarted them to ensure all pods get networking provided by Cilium.

Validate the Installation

To validate that Cilium has been properly installed, you can run

  1. $ cilium status --wait
  2.    /¯¯\
  3. /¯¯\__/¯¯\    Cilium:         OK
  4. \__/¯¯\__/    Operator:       OK
  5. /¯¯\__/¯¯\    Hubble:         disabled
  6. \__/¯¯\__/    ClusterMesh:    disabled
  7.    \__/
  9. DaemonSet         cilium             Desired: 2, Ready: 2/2, Available: 2/2
  10. Deployment        cilium-operator    Desired: 2, Ready: 2/2, Available: 2/2
  11. Containers:       cilium-operator    Running: 2
  12.                   cilium             Running: 2
  13. Image versions    cilium             quay.io/cilium/cilium:v1.9.5: 2
  14.                   cilium-operator    quay.io/cilium/operator-generic:v1.9.5: 2

Run the following command to validate that your cluster has proper network connectivity:

  1. $ cilium connectivity test
  2. ℹ️  Monitor aggregation detected, will skip some flow validation steps
  3. ✨ [k8s-cluster] Creating namespace for connectivity check...
  4. (...)
  5. ---------------------------------------------------------------------------------------------------------------------
  6. 📋 Test Report
  7. ---------------------------------------------------------------------------------------------------------------------
  8. ✅ 69/69 tests successful (0 warnings)

Congratulations! You have a fully functional Kubernetes cluster with Cilium. 🎉

Next Steps