Adding Admission Webhooks to an Ansible-based Operator

Background

For general background on what admission webhooks are, why to use them, and how to build them, please refer to the official Kubernetes documentation on Extensible Admission Controllers

This guide will assume that you understand the above content, and that you have an existing admission webhook server. You will likely need to make a few modifications to the webhook server container.

When integrating an admission webhook server into your Ansible-based Operator, we recommend that you deploy it as a sidecar container alongside your operator. This allows you to make use of the proxy server that the operator deploys, as well as the cache that backs it. The sidecar will be defined in the deploy/operator.yaml and it will look like:

  1. ...
  2. # This deploys the webhook
  3.         - name: webhook
  4.           # Replace this with the built image name
  5.           image: "REPLACE_WEBHOOK_IMAGE"
  6.           imagePullPolicy: "Always"
  7.           volumeMounts:
  8.           - mountPath: /etc/tls/
  9.             name: webhook-cert
  10. ...            
  12. ## Ensuring the webhook server uses the caching proxy
  14. When an Ansible-based Operator runs, it creates a Kubernetes proxy server and serves it on
  15. `http://localhost:8888`. This proxy server does not require any authorization, so all you need to
  16. do to make use of the proxy is ensure that your Kubernetes client is pointing at `http://localhost:8888`
  17. and that it does not attempt to verify SSL. If you use the default in-cluster configuration, you will
  18. be hitting the real API server and will not get caching for free.
  20. ## Deploying the webhook server
  22. To deploy the webhook server as a sidecar alongside your operator, all you need to do is add the container
  23. specification to your `deploy/operator.yaml`. You may also need to add a volume for mounting in TLS secrets,
  24. as your webhook server is required to have a valid SSL configuration. Below is a sample updated container
  25. specification that deploys a webhook:
  27. ```yaml
  28.       containers:
  29.         - name: my-operator
  30.           # Replace this with the built image name
  31.           image: "REPLACE_IMAGE"
  32.           imagePullPolicy: "Always"
  33.           volumeMounts:
  34.           - mountPath: /tmp/ansible-operator/runner
  35.             name: runner
  36.           env:
  37.             - name: WATCH_NAMESPACE
  38.               valueFrom:
  39.                 fieldRef:
  40.                   fieldPath: metadata.namespace
  41.             - name: POD_NAME
  42.               valueFrom:
  43.                 fieldRef:
  44.                   fieldPath: metadata.name
  45.             - name: OPERATOR_NAME
  46.               value: "validating-operator"
  47.             - name: ANSIBLE_GATHERING
  48.               value: explicit
  49.         # This deploys the webhook
  50.         - name: webhook
  51.           # Replace this with the built image name
  52.           image: "REPLACE_WEBHOOK_IMAGE"
  53.           imagePullPolicy: "Always"
  54.           volumeMounts:
  55.           - mountPath: /etc/tls/
  56.             name: webhook-cert
  57.       volumes:
  58.         - name: runner
  59.           emptyDir: {}
  60.         # This assumes there is a secret called webhook-cert containing TLS certificates
  61.         # Projects like cert-manager can create these certificates
  62.         - name: webhook-cert
  63.           secret:
  64.             secretName: webhook-cert

This will run your webhook server alongside the operator, but Kubernetes will not yet call the webhooks before resources can be created. In order to let Kubernetes know about your webhooks, you must create specific API resources.

Making Kubernetes call your webhooks

In order to make your webhooks callable at all, first you must create a Service that points at your webhook server. Below is a sample service that creates a Service named my-operator-webhook, that will send traffic on port 443 to port 5000 in a Pod that matches the selector name=my-operator. Modify these values to match your environment.

  1. apiVersion: v1
  2. kind: Service
  3. metadata:
  4.   name: my-operator-webhook
  5. spec:
  6.   ports:
  7.   - name: webhook
  8.     port: 443
  9.     protocol: TCP
  10.     # Change targetPort to match the port your server is listening on
  11.     targetPort: 5000
  12.   selector:
  13.     # Change this selector to match the labels on your operator pod
  14.     name: my-operator
  15.   type: ClusterIP

Now that you have a Service directing traffic to your webhook server, you will need to create MutatingWebhookConfiguration or ValidatingWebhookConfiguration objects (depending on what type of webhook you have deployed), which will tell Kubernetes to send certain API requests through your webhooks before writing to etcd.

Below are examples of both MutatingWebhookConfiguration and ValidatingWebhookConfiguration objects, which will tell Kubernetes to call the my-operator-webhook service when samples.example.com Example resources are created. The mutating webhook is served on the /mutating path in my example webhook server, and the validating webhook is served on /validating. Update these values as needed to reflect your environment and desired behavior. These objects are thoroughly documented in the official Kubernetes documentation on Extensible Admission Controllers

  1. ---
  2. apiVersion: admissionregistration.k8s.io/v1
  3. kind: MutatingWebhookConfiguration
  4. metadata:
  5.   name: mutating.example.com
  6. webhooks:
  7. - name: "mutating.example.com"
  8.   rules:
  9.   - apiGroups:   ["samples.example.com"]
  10.     apiVersions: ["*"]
  11.     operations:  ["CREATE"]
  12.     resources:   ["examples"]
  13.     scope:       "Namespaced"
  14.   clientConfig:
  15.     service:
  16.       # Replace this with the namespace your service is in
  17.       namespace: REPLACE_NAMESPACE
  18.       name: my-operator-webhook
  19.       path: /mutating
  20.   admissionReviewVersions: ["v1"]
  21.   sideEffects: None
  22. ---
  23. apiVersion: admissionregistration.k8s.io/v1
  24. kind: ValidatingWebhookConfiguration
  25. metadata:
  26.   name: validating.example.com
  27. webhooks:
  28. - name: validating.example.com
  29.   rules:
  30.   - apiGroups:   ["samples.example.com"]
  31.     apiVersions: ["*"]
  32.     operations:  ["CREATE"]
  33.     resources:   ["examples"]
  34.     scope:       "Namespaced"
  35.   clientConfig:
  36.     service:
  37.       # Replace this with the namespace your service is in
  38.       namespace: REPLACE_NAMESPACE
  39.       name: my-operator-webhook
  40.       path: /validating
  41.   admissionReviewVersions: ["v1"]
  42.   failurePolicy: Fail
  43.   sideEffects: None

If these resources are configured properly you will now have an admissions webhook that can reject or mutate incoming resources before they are written to the Kubernetes database.

Summary

To deploy an existing admissions webhook to validate or mutate your Kubernetes resources alongside an Ansible-based Operator, you must

  1. Configure your admissions webhook to use the proxy server running on http://localhost:8888 in the operator pod
  2. Add the webhook container to your operator deployment
  3. Create a Service pointing to your webhook
  4. Make sure your webhook is reachable via the Service over https
  5. Create MutatingWebhookConfiguration or ValidatingWebhookConfiguration mapping the resource you want to mutate/validate to the Service you created

Last modified January 1, 0001