Generating CRDs

Kubebuilder provides a tool named controller-gen to generate manifests for CustomResourceDefinitions. The tool resides in the controller-tools repository and is installed through a Makefile target called controller-gen.

If you examine the Makefile in your project, you will see a target named manifests for generating manifests. manifests target is also listed as prerequisite for other targets like run, tests, deploy etc to ensure CRD manifests are regenerated when needed.

  1. # Generate manifests e.g. CRD, RBAC etc.
  2. manifests: controller-gen
  3.     $(CONTROLLER_GEN) $(CRD_OPTIONS) rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
  5. # find or download controller-gen
  6. # download controller-gen if necessary
  7. controller-gen:
  8. ifeq (, $(shell which controller-gen))
  9.     go get sigs.k8s.io/controller-tools/cmd/controller-gen@v0.2.0-beta.2
  10. CONTROLLER_GEN=$(GOBIN)/controller-gen
  11. else
  12. CONTROLLER_GEN=$(shell which controller-gen)
  13. endif

When you run make manifests, you should see generated CRDs are under config/crd/bases directory.

controller-gen generates manifests for RBAC as well, but this section covers the generation of CRD manifests.

controller-gen reads kubebuilder markers of the form // +kubebuilder:something… defined as Go comments in the <your-api-kind>_types.go file under apis/… to produce the CRD manifests. Sections below describe various supported annotations.

Validation

CRDs support validation by definining (OpenAPI v3 schema) in the validation section. To learn more about the validation feature, refer to the original docs here. One can specify validation for a field by annotating the field with kubebuilder marker which is of the form// +kubebuilder:validation:<key=value>. If you want to specify multiple validations for a field, you can add multiple such markers as demonstrated in the example below.

Currently, supporting keys are Maximum, Minimum, MaxLength, MinLength, MaxItems, MinItems, UniqueItems, Enum, Pattern, ExclusiveMaximum,ExclusiveMinimum, MultipleOf, Format. The // +kubebuilder:validation:Pattern=.+:.+ annotation specifies the Pattern validation requiring that the Image field match the regular expression .+:.+

Example:

  1. type ToySpec struct {
  3.     // +kubebuilder:validation:Maximum=100
  4.     // +kubebuilder:validation:Minimum=1
  5.     // +kubebuilder:validation:ExclusiveMinimum=true
  6.     Power  float32 `json:"power,omitempty"`
  8.     Bricks int32   `json:"bricks,omitempty"`
  9.     // +kubebuilder:validation:MaxLength=15
  10.     // +kubebuilder:validation:MinLength=1
  11.     Name string `json:"name,omitempty"`
  13.     // +kubebuilder:validation:MaxItems=500
  14.     // +kubebuilder:validation:MinItems=1
  15.     // +kubebuilder:validation:UniqueItems=false
  16.     Knights []string `json:"knights,omitempty"`
  18.     // +kubebuilder:validation:Enum=Lion;Wolf;Dragon
  19.     Alias string `json:"alias,omitempty"`
  21.     // +kubebuilder:validation:Enum=1;2;3
  22.     Rank    int    `json:"rank"`
  23. }

Additional printer columns

Starting with Kubernetes 1.11, kubectl uses server-side printing. The serverdecides which columns are shown by the kubectl get command. You cancustomize these columns using a CustomResourceDefinition.To add an additional column, add a comment with the following marker formatjust above the struct definition of the Kind.

Format: // +kubebuilder:printcolumn:name="Name",type="type",JSONPath="json-path",description="desc",priority="priority",format="format"

Note that description, priority and format are optional. Refer to theadditonal printer columns docsto learn more about the values of name, type, JsonPath, description, priority and format.

The following example adds the Spec, Replicas, and Age columns.

  1. // +kubebuilder:printcolumn:name="Spec",type="integer",JSONPath=".spec.cronSpec",description="status of the kind"
  2. // +kubebuilder:printcolumn:name="Replicas",type="integer",JSONPath=".spec.replicas",description="The number of jobs launched by the CronJob"
  3. // +kubebuilder:printcolumn:name="Age",type="date",JSONPath=".metadata.creationTimestamp"
  4. type CronTab struct {
  5.     metav1.TypeMeta   `json:",inline"`
  6.     metav1.ObjectMeta `json:"metadata,omitempty"`
  8.     Spec   CronTabSpec   `json:"spec,omitempty"`
  9.     Status CronTabStatus `json:"status,omitempty"`
  10. }

Subresource

Custom resources support /status and /scale subresources as of kubernetes1.13 release. You can learn more about the subresources here.

1. Status

To enable /status subresource, annotate the kind with // +kubebuilder:subresource:status marker.

2. Scale

To enable /scale subresource, annotate the kind with // +kubebuilder:subresource:scale:specpath=<jsonpath>,statuspath=<jsonpath>,selectorpath=<jsonpath> marker.

Scale subresource marker contains three fields: specpath, statuspath and selectorpath.

  • specpath refers to specReplicasPath attribute of Scale object, and value jsonpath defines the JSONPath inside of a custom resource that corresponds to Scale.Spec.Replicas. This is a required field.
  • statuspath refers to statusReplicasPath attribute of Scale object. and the jsonpath value of it defines the JSONPath inside of a custom resource that corresponds to Scale.Status.Replicas. This is a required field.
  • selectorpath refers to labelSelectorPath attribute of Scale object, and the value jsonpath defines the JSONPath inside of a custom resource that corresponds to Scale.Status.Selector. This is an optional field.

Example:

  1. type ToySpec struct {
  2.     Replicas *int32 `json:"replicas"` // Add this field in Toy Spec, so the jsonpath to this field is `.spec.replicas`
  3. }
  5. // ToyStatus defines the observed state of Toy
  6. type ToyStatus struct {
  7.     Replicas int32 `json:"replicas"` // Add this field in Toy Status, so the jsonpath to this field is `.status.replicas`
  8. }
  11. // Toy is the Schema for the toys API
  12. // +kubebuilder:subresource:status
  13. // +kubebuilder:subresource:scale:specpath=.spec.replicas,statuspath=.status.replicas
  14. type Toy struct {
  15.     metav1.TypeMeta   `json:",inline"`
  16.     metav1.ObjectMeta `json:"metadata,omitempty"`
  18.     Spec   ToySpec   `json:"spec,omitempty"`
  19.     Status ToyStatus `json:"status,omitempty"`
  20. }

In order to enable scale subresource in type definition file, you have to apply the scale subresource right before the kind struct definition, with correct jsonpath values according to the spec and status. And then make sure the jsonpaths are already defined in the Spec and Status struct. Finally, update the <kind>_types_test.go files according to the types Spec and Status changes.

In the above example for the type Toy, we added // +kubebuilder:subresource:scale:specpath=.spec.replicas,statuspath=.status.replicas comment before Toy struct definition. .spec.replicas refers to the josnpath of Spec struct field (ToySpec.Replicas). And jsonpath .status.healthyReplicas refers to Status struct field (ToyStatus.Replicas).

Multiple Versions

If you are defining multiple versions of a kind in your project, you need to dothe following:

  • Set CRD_OPTIONS ?= "crd:trivialVersions=false" in the Makefile
  • Annotate the Go struct with marker // +kubebuilder:storageversion for theindicating the storage version.