Installation

Since v1.0.0 (alpha/beta), OpenKruise requires Kubernetes version >= 1.16.

Install with helm

Kruise can be simply installed by helm v3.5+, which is a simple command-line tool and you can get it from here.

  1. # Firstly add openkruise charts repository if you haven't do this.
  2. $ helm repo add openkruise https://openkruise.github.io/charts/
  3. # [Optional]
  4. $ helm repo update
  5. # Install the latest version.
  6. $ helm install kruise openkruise/kruise --version 1.3.0

Upgrade with helm

  1. # Firstly add openkruise charts repository if you haven't do this.
  2. $ helm repo add openkruise https://openkruise.github.io/charts/
  3. # [Optional]
  4. $ helm repo update
  5. # Upgrade to the latest version.
  6. $ helm upgrade kruise openkruise/kruise --version 1.3.0 [--force]

Note that:

  1. Before upgrade, you must firstly read the Change Log to make sure that you have understand the breaking changes in the new version.
  2. If you want to drop the chart parameters you configured for the old release or set some new parameters, it is recommended to add --reset-values flag in helm upgrade command. Otherwise you should use --reuse-values flag to reuse the last release’s values.
  3. If you are upgrading Kruise from 0.x to 1.x, you must add --force for upgrade command. Otherwise it is an optional flag.

Optional: download charts manually

If you have problem with connecting to https://openkruise.github.io/charts/ in production, you might need to download the chart from here manually and install or upgrade with it.

  1. $ helm install/upgrade kruise /PATH/TO/CHART

Options

Note that installing this chart directly means it will use the default template values for Kruise.

You may have to set your specific configurations if it is deployed into a production cluster, or you want to configure feature-gates.

Optional: chart parameters

The following table lists the configurable parameters of the chart and their default values.

ParameterDescriptionDefault
featureGatesFeature gates for Kruise, empty string means all by default
installation.namespacenamespace for kruise installationkruise-system
installation.createNamespaceWhether to create the installation.namespacetrue
manager.log.levelLog level that kruise-manager printed4
manager.replicasReplicas of kruise-controller-manager deployment2
manager.image.repositoryRepository for kruise-manager imageopenkruise/kruise-manager
manager.image.tagTag for kruise-manager imagev1.2.0
manager.resources.limits.cpuCPU resource limit of kruise-manager container200m
manager.resources.limits.memoryMemory resource limit of kruise-manager container512Mi
manager.resources.requests.cpuCPU resource request of kruise-manager container100m
manager.resources.requests.memoryMemory resource request of kruise-manager container256Mi
manager.metrics.portPort of metrics served8080
manager.webhook.portPort of webhook served9443
manager.nodeAffinityNode affinity policy for kruise-manager pod{}
manager.nodeSelectorNode labels for kruise-manager pod{}
manager.tolerationsTolerations for kruise-manager pod[]
daemon.log.levelLog level that kruise-daemon printed4
daemon.portPort of metrics and healthz that kruise-daemon served10221
daemon.resources.limits.cpuCPU resource limit of kruise-daemon container50m
daemon.resources.limits.memoryMemory resource limit of kruise-daemon container128Mi
daemon.resources.requests.cpuCPU resource request of kruise-daemon container0
daemon.resources.requests.memoryMemory resource request of kruise-daemon container0
daemon.affinityAffinity policy for kruise-daemon pod{}
daemon.socketLocationLocation of the container manager control socket/var/run
daemon.socketFileSpecify the socket file name in socketLocation (if you are not using containerd/docker/pouch/cri-o)
webhookConfiguration.failurePolicy.podsThe failurePolicy for pods in mutating webhook configurationIgnore
webhookConfiguration.timeoutSecondsThe timeoutSeconds for all webhook configuration30
crds.managedKruise will not install CRDs with chart if this is falsetrue
manager.resyncPeriodResync period of informer kruise-manager, defaults no resync0
manager.hostNetworkWhether kruise-manager pod should run with hostnetworkfalse
imagePullSecretsThe list of image pull secrets for kruise imagefalse

Specify each parameter using the --set key=value[,key=value] argument to helm install or helm upgrade.

Optional: feature-gate

Feature-gate controls some influential features in Kruise:

NameDescriptionDefaultEffect (if closed)
PodWebhookWhether to open a webhook for Pod createtrueSidecarSet/KruisePodReadinessGate disabled
KruiseDaemonWhether to deploy kruise-daemon DaemonSettrueImagePulling/ContainerRecreateRequest disabled
DaemonWatchingPodShould each kruise-daemon watch pods on the same nodetrueFor in-place update with same imageID or env from labels/annotations
CloneSetShortHashEnables CloneSet controller only set revision hash name to pod labelfalseCloneSet name can not be longer than 54 characters
KruisePodReadinessGateEnables Kruise webhook to inject ‘KruisePodReady’ readiness-gate to all Pods during creationfalseThe readiness-gate will only be injected to Pods created by Kruise workloads
PreDownloadImageForInPlaceUpdateEnables CloneSet controller to create ImagePullJobs to pre-download images for in-place updatefalseNo image pre-download for in-place update
CloneSetPartitionRollbackEnables CloneSet controller to rollback Pods to currentRevision when number of updateRevision pods is bigger than (replicas - partition)falseCloneSet will only update Pods to updateRevision
ResourcesDeletionProtectionEnables protection for resources deletionfalseNo protection for resources deletion
TemplateNoDefaultsWhether to disable defaults injection for pod/pvc template in workloadsfalseShould not close this feature if it has open
PodUnavailableBudgetDeleteGateEnables PodUnavailableBudget for pod deletion, evictionfalseNo protection for pod deletion, eviction
PodUnavailableBudgetUpdateGateEnables PodUnavailableBudget for pod.Spec updatefalseNo protection for in-place update
WorkloadSpreadEnables WorkloadSpread to manage multi-domain and elastic deployfalseWorkloadSpread disabled
InPlaceUpdateEnvFromMetadataEnables Kruise to in-place update a container in Pod when its env from labels/annotations changed and pod is in-place updatingfalseOnly container image can be in-place update
StatefulSetAutoDeletePVCEnables policies controlling deletion of PVCs created by a StatefulSetfalseNo deletion of PVCs by StatefulSet
PreDownloadImageForDaemonSetUpdateEnables DaemonSet controller to create ImagePullJobs to pre-download images for in-place updatefalseNo image pre-download for in-place update
PodProbeMarkerGateWhether to turn on PodProbeMarker abilityfalsePodProbeMarker disabled
SidecarSetPatchPodMetadataDefaultsAllowedAllow SidecarSet patch any annotations to Pod Object, no more whitelist checksfalseAnnotations are not allowed to patch randomly and need to be configured via SidecarSet_PatchPodMetadata_WhiteList

If you want to configure the feature-gate, just set the parameter when install or upgrade. Such as:

  1. $ helm install kruise https://... --set featureGates="ResourcesDeletionProtection=true\,PreDownloadImageForInPlaceUpdate=true"

If you want to enable all feature-gates, set the parameter as featureGates=AllAlpha=true.

Optional: the local image for China

If you are in China and have problem to pull image from official DockerHub, you can use the registry hosted on Alibaba Cloud:

  1. $ helm install kruise https://... --set manager.image.repository=openkruise-registry.cn-shanghai.cr.aliyuncs.com/openkruise/kruise-manager

Best Practices

Installation parameters for K3s

Usually K3s has the different runtime path from the default /var/run. So you have to set daemon.socketLocation to the real runtime socket path on your K3s node (e.g. /run/k3s or /var/run/k3s/).

Installation parameters for AWS EKS

When using a custom CNI (such as Weave or Calico) on EKS, the webhook cannot be reached by default. This happens because the control plane cannot be configured to run on a custom CNI on EKS, so the CNIs differ between control plane and worker nodes.

To address this, the webhook can be run in the host network so it can be reached, by setting --set manager.hostNetwork=true when use helm install or upgrade.

Uninstall

Note that this will lead to all resources created by Kruise, including webhook configurations, services, namespace, CRDs, CR instances and Pods managed by Kruise controller, to be deleted!

Please do this ONLY when you fully understand the consequence.

To uninstall kruise if it is installed with helm charts:

  1. $ helm uninstall kruise
  2. release "kruise" uninstalled