Custom Resource Annotations in Helm-based Operators

Use custom resource annotations to configure how reconciliation works.

helm.sdk.operatorframework.io/upgrade-force

This annotation can be set to "true" on custom resources to enable the chart to be upgraded with the helm upgrade --force option. For more info see the Helm Upgrade documentation and this explanation of --force behavior.

Example

  1. apiVersion: example.com/v1alpha1
  2. kind: Nginx
  3. metadata:
  4. name: nginx-sample
  5. annotations:
  6. helm.sdk.operatorframework.io/upgrade-force: "true"
  7. spec:
  8. replicaCount: 2
  9. service:
  10. port: 8080

Setting this annotation to true and making a change to trigger an upgrade (e.g. setting spec.replicaCount: 3) will cause the custom resource to be reconciled and upgraded with the force option. This can be verified in the log message when an upgrade succeeds:

  1. {"level":"info","ts":1591198931.1703992,"logger":"helm.controller","msg":"Upgraded release","namespace":"helm-nginx","name":"example-nginx","apiVersion":"cache.example.com/v1alpha1","kind":"Nginx","release":"example-nginx","force":true}

helm.sdk.operatorframework.io/uninstall-wait

This annotation can be set to "true" on custom resources to enable the deletion to wait until all the resources in the status.deployedRelease.manifest are deleted.

Example

  1. apiVersion: example.com/v1alpha1
  2. kind: Nginx
  3. metadata:
  4. name: nginx-sample
  5. annotations:
  6. helm.sdk.operatorframework.io/uninstall-wait: "true"
  7. spec:
  8. ...
  9. status:
  10. ...
  11. deployedRelease:
  12. manifest: |
  13. ---
  14. # Source: nginx/templates/serviceaccount.yaml
  15. apiVersion: v1
  16. kind: ServiceAccount
  17. metadata:
  18. name: nginx-sample
  19. labels:
  20. helm.sh/chart: nginx-0.1.0
  21. app.kubernetes.io/name: nginx
  22. app.kubernetes.io/instance: nginx-sample
  23. app.kubernetes.io/version: "1.16.0"
  24. app.kubernetes.io/managed-by: Helm
  25. ---
  26. # Source: nginx/templates/service.yaml
  27. apiVersion: v1
  28. kind: Service
  29. metadata:
  30. name: nginx-sample
  31. labels:
  32. helm.sh/chart: nginx-0.1.0
  33. app.kubernetes.io/name: nginx
  34. app.kubernetes.io/instance: nginx-sample
  35. app.kubernetes.io/version: "1.16.0"
  36. app.kubernetes.io/managed-by: Helm
  37. spec:
  38. type: ClusterIP
  39. ports:
  40. - port: 80
  41. targetPort: http
  42. protocol: TCP
  43. name: http
  44. selector:
  45. app.kubernetes.io/name: nginx
  46. app.kubernetes.io/instance: nginx-sample
  47. ---
  48. # Source: nginx/templates/deployment.yaml
  49. apiVersion: apps/v1
  50. kind: Deployment
  51. metadata:
  52. name: nginx-sample
  53. labels:
  54. helm.sh/chart: nginx-0.1.0
  55. app.kubernetes.io/name: nginx
  56. app.kubernetes.io/instance: nginx-sample
  57. app.kubernetes.io/version: "1.16.0"
  58. app.kubernetes.io/managed-by: Helm
  59. spec:
  60. replicas: 1
  61. selector:
  62. matchLabels:
  63. app.kubernetes.io/name: nginx
  64. app.kubernetes.io/instance: nginx-sample
  65. template:
  66. metadata:
  67. labels:
  68. app.kubernetes.io/name: nginx
  69. app.kubernetes.io/instance: nginx-sample
  70. spec:
  71. serviceAccountName: nginx-sample
  72. securityContext:
  73. {}
  74. containers:
  75. - name: nginx
  76. securityContext:
  77. {}
  78. image: "nginx:1.16.0"
  79. imagePullPolicy: IfNotPresent
  80. ports:
  81. - name: http
  82. containerPort: 80
  83. protocol: TCP
  84. livenessProbe:
  85. httpGet:
  86. path: /
  87. port: http
  88. readinessProbe:
  89. httpGet:
  90. path: /
  91. port: http
  92. resources:
  93. {}

Setting this annotation to true and deleting the custom resource will cause the custom resource to be reconciled continuously until all the resources in status.deployedRelease.manifest are deleted. This can be verified in the log message when a delete has been triggered:

  1. {"level":"info","ts":1612294054.5845876,"logger":"helm.controller","msg":"Uninstall wait","namespace":"default","name":"nginx-sample","apiVersion":"example.com/v1alpha1","kind":"Nginx","release":"nginx-sample"}

helm.sdk.operatorframework.io/reconcile-period

While running a Helm-based operator, the reconcile-period can be specified through the custom resource’s annotations under the helm.sdk.operatorframework.io/reconcile-period key. This feature guarantees that an operator will get reconciled, at minimum, in the specified interval of time. In other words, it ensures that the cluster will not go longer than the specified reconcile-period without being reconciled. However, the cluster may be reconciled at any moment if there are changes detected in the desired state.

The reconcile period can be specified in the custom resource’s annotations in the following manner:

  1. ...
  2. metadata:
  3. name: nginx-sample
  4. annotations:
  5. helm.sdk.operatorframework.io/reconcile-period: 5s
  6. ...

The value that is present under this key must be in the h/m/s format. For example, 1h2m4s, 3m0s, 4s are all valid values, but 1x3m9s is invalid.

NOTE: This is just one way of specifying the reconcile period for Helm-based operators. There are two other ways: using the --reconcile-period command-line flag and under the ‘reconcilePeriod’ key in the watches.yaml file. If these three methods are used simultaneously to specify reconcile period (which they should not be), the order of precedence is as follows: Custom Resource Annotations > watches.yaml > command-line flag.

Last modified March 16, 2022: moved docs, created reconcile result object (#5599) (f5e15be6)