PodUnavailableBudget

FEATURE STATE: Kruise v0.10.0

Kubernetes offers Pod Disruption Budget to help you run highly available applications even when you introduce frequent voluntary disruptions. PDB limits the number of Pods of a replicated application that are down simultaneously from voluntary disruptions. However, it can only constrain the voluntary disruption triggered by the Eviction API. For example, when you run kubectl drain, the tool tries to evict all of the Pods on the Node you’re taking out of service.

In the following voluntary disruption scenarios, there are still business disruption or SLA degradation situations:

  1. The application owner update deployment’s pod template for general upgrading, while cluster administrator drain nodes to scale the cluster down(learn about Cluster Autoscaling).
  2. The middleware team is using SidecarSet to rolling upgrade the sidecar containers of the cluster, e.g. ServiceMesh envoy, while HPA triggers the scale-down of business applications.
  3. The application owner and middleware team release the same Pods at the same time based on OpenKruise cloneSet, sidecarSet in-place upgrades

In voluntary disruption scenarios, PodUnavailableBudget can achieve the effect of preventing application disruption or SLA degradation, which greatly improves the high availability of application services.

A sample PodUnavailableBudget yaml looks like following:

  1. apiVersion: apps.kruise.io/v1alpha1
  2. kind: PodUnavailableBudget
  3. metadata:
  4.   name: web-server-pub
  5.   namespace: web
  6. spec:
  7.   targetRef:
  8.     apiVersion: apps.kruise.io/v1alpha1
  9.     # cloneset, deployment, statefulset etc.
  10.     kind: CloneSet
  11.     name: web-server
  12.   # selector label query over pods managed by the budget
  13.   # selector and TargetReference are mutually exclusive, targetRef is priority to take effect.
  14.   # selector is commonly used in scenarios where applications are deployed using multiple workloads,
  15.   # and targetRef is used for protection against a single workload.
  16. # selector:
  17. #   matchLabels:
  18. #     app: web-server
  19.   # maximum number of Pods unavailable for the current cloneset, the example is cloneset.replicas(5) * 60% = 3
  20.   # maxUnavailable and minAvailable are mutually exclusive, maxUnavailable is priority to take effect
  21.   maxUnavailable: 60%
  22.   # Minimum number of Pods available for the current cloneset, the example is cloneset.replicas(5) * 40% = 2
  23. # minAvailable: 40%
  24. -----------------------
  26. apiVersion: apps.kruise.io/v1alpha1
  27. kind: CloneSet
  28. metadata:
  29.   labels:
  30.     app: web-server
  31.   name: web-server
  32.   namespace: web
  33. spec:
  34.   replicas: 5
  35.   selector:
  36.     matchLabels:
  37.       app: web-server
  38.   template:
  39.     metadata:
  40.       labels:
  41.         app: web-server
  42.     spec:
  43.       containers:
  44.       - name: nginx
  45.         image: nginx:alpine

Implementation

This program customizes the PodUnavailableBudget (later referred to as PUB) CRD resource to describe the desired state of the application, and the working mechanism is shown below:

PodUnavailableBudget

Comparison with Kubernetes native PDB

Kubernetes PodDisruptionBudget implements protection against Pod Eviction based on the EvictionREST interface, while PodUnavailableBudget intercepts all pod modification requests through the admission webhook validating mechanism (Many voluntary disruption scenarios can be summarized as modifications to Pod resources), and reject the request if the modification does not satisfy the desired state of the PUB.

Pub contains all the protection capabilities of kubernetes PDB, you can use both, or use pub independently to implement your application protection (Recommend).

feature-gates

PodUnavailableBudget protection against Pods is turned off by default, if you want to turn it on set feature-gates PodUnavailableBudgetDeleteGate and PodUnavailableBudgetUpdateGate.

  1. $ helm install kruise https://... --set featureGates="PodUnavailableBudgetDeleteGate=true\,PodUnavailableBudgetUpdateGate=true"

PodUnavailableBudget Status

  1. # kubectl describe podunavailablebudgets web-server-pub
  2. Name:         web-server-pub
  3. Kind:         PodUnavailableBudget
  4. Status:
  5.   unavailableAllowed:   3   # unavailableAllowed number of pod unavailable that are currently allowed
  6.   currentAvailable:     5   # currentAvailable current number of available pods
  7.   desiredAvailable:     2   # desiredAvailable minimum desired number of available pods
  8.   totalReplicas:        5   # totalReplicas total number of pods counted by this PUB