Kubernetes RBAC

In KubeVela v1.4, Authentication & Authorization mechanism is introduced. This allows applications to dispatch and manage resources using the identity of the application’s creator/modifier. With this feature, it will be easy to limit the access of KubeVela users/applications and isolate their living spaces, which will make your KubeVela system safer.

To enable Authentication & Authorization in your KubeVela system, you need to do the following steps

  1. Delete the ClusterRoleBinding ends with vela-core:manager-rolebinding. Usually you can do it through:

    1. kubectl delete ClusterRoleBinding kubevela-vela-core:manager-rolebinding
  2. Upgrade the controller, and wait for the installation finished:

    1. helm upgrade --install kubevela kubevela/vela-core --create-namespace -n vela-system --set authentication.enabled=true --set authentication.withUser=true --wait
  3. Make sure your version Vela CLI v1.4.1+, refer to the installation guide.

  4. (Optional) Install vela-prism through running the following commands, which will allow you to enjoy the advanced API extensions in KubeVela.

    1. helm repo add prism https://charts.kubevela.net/prism
    2. helm repo update
    3. helm install vela-prism prism/vela-prism -n vela-system
  5. Before we start, assume we already have two managed clusters joined in KubeVela, called c2 and c3. You can refer to the multi-cluster document and see how to join managed clusters into KubeVela control plane.

  1. $ vela cluster list
  2. NAME ALIAS CREDENTIAL_TYPE ENDPOINT ACCEPTED LABELS
  3. local Internal - true
  4. c3 X509Certificate <c3 apiserver url> true
  5. c2 X509Certificate <c2 apiserver url> true

The following procedures are illustrated in the following figure. The operations below are associated with different users, which are identified by switching KUBECONFIG environment variable. (Each KUBECONFIG corresponds to one user.)

auth-procedure

  1. Let’s start with a new coming user named Alice. As the system administrator, you can assign a KubeConfig for Alice to use.
  1. $ vela auth gen-kubeconfig --user alice > alice.kubeconfig
  2. Private key generated.
  3. Certificate request generated.
  4. Certificate signing request alice generated.
  5. Certificate signing request alice approved.
  6. Signed certificate retrieved.
  1. Now alice is unable to do anything in the cluster with the given KubeConfig. We can grant her the privileges of Read/Write resources in the dev namespace of the control plane and managed cluster c2.
  1. $ vela auth grant-privileges --user alice --for-namespace dev --for-cluster=local,c2 --create-namespace
  2. ClusterRole kubevela:writer created in local.
  3. RoleBinding dev/kubevela:writer:binding created in local.
  4. ClusterRole kubevela:writer created in c2.
  5. RoleBinding dev/kubevela:writer:binding created in c2.
  6. Privileges granted.
  1. We can check the privileges of Alice by the following command
  1. $ vela auth list-privileges --user alice --cluster local,c2
  2. User=alice
  3. ├── [Cluster] local
  4. └── [ClusterRole] kubevela:writer
  5. ├── [Scope]
  6. └── [Namespaced] dev (RoleBinding kubevela:writer:binding)
  7. └── [PolicyRules]
  8. ├── APIGroups: *
  9. Resources: *
  10. Verb: get, list, watch, create, update, patch, delete
  11. └── NonResourceURLs: *
  12. Verb: get, list, watch, create, update, patch, delete
  13. └── [Cluster] c2
  14. └── [ClusterRole] kubevela:writer
  15. ├── [Scope]
  16. └── [Namespaced] dev (RoleBinding kubevela:writer:binding)
  17. └── [PolicyRules]
  18. ├── APIGroups: *
  19. Resources: *
  20. Verb: get, list, watch, create, update, patch, delete
  21. └── NonResourceURLs: *
  22. Verb: get, list, watch, create, update, patch, delete
  1. Alice can create an application in the dev namespace now. The application can also dispatch resources into the dev namespace of cluster c2.
  1. cat <<EOF | KUBECONFIG=alice.kubeconfig vela up -f -
  2. apiVersion: core.oam.dev/v1beta1
  3. kind: Application
  4. metadata:
  5. name: podinfo
  6. namespace: dev
  7. spec:
  8. components:
  9. - name: podinfo
  10. type: webservice
  11. properties:
  12. image: stefanprodan/podinfo:6.0.1
  13. policies:
  14. - type: topology
  15. name: topology
  16. properties:
  17. clusters: ["c2"]
  18. EOF
  1. Alice can see the the application is successfully deployed.
  1. $ KUBECONFIG=alice.kubeconfig vela status podinfo -n dev
  2. About:
  3. Name: podinfo
  4. Namespace: dev
  5. Created at: 2022-05-31 17:06:14 +0800 CST
  6. Status: running
  7. Workflow:
  8. mode: DAG
  9. finished: true
  10. Suspend: false
  11. Terminated: false
  12. Steps
  13. - id:rk3npcpycl
  14. name:deploy-topology
  15. type:deploy
  16. phase:succeeded
  17. message:
  18. Services:
  19. - Name: podinfo
  20. Cluster: c2 Namespace: dev
  21. Type: webservice
  22. Healthy Ready:1/1
  23. No trait applied
  1. If Alice wants to access resources outside the dev namespace, she will be forbidden to do so.
  1. KUBECONFIG=alice.kubeconfig kubectl get pod -n vela-system
  2. Error from server (Forbidden): pods is forbidden: User "alice" cannot list resource "pods" in API group "" in the namespace "vela-system"
  1. If Alice try to create a application in the dev namespace and the application intends to dispatch resources in c3 cluster (which Alice does not have the privileges), the application will not be able to do so.

Alice create the application podinfo-bad

  1. $ cat <<EOF | KUBECONFIG=alice.kubeconfig vela up -f -
  2. apiVersion: core.oam.dev/v1beta1
  3. kind: Application
  4. metadata:
  5. name: podinfo-bad
  6. namespace: dev
  7. spec:
  8. components:
  9. - name: podinfo-bad
  10. type: webservice
  11. properties:
  12. image: stefanprodan/podinfo:6.0.1
  13. policies:
  14. - type: topology
  15. name: topology
  16. properties:
  17. clusters: ["c3"]
  18. EOF

Alice checks the status of podinfo-bad, the error message will be shown.

  1. $ KUBECONFIG=alice.kubeconfig vela status podinfo-bad -n dev
  2. About:
  3. Name: podinfo-bad
  4. Namespace: dev
  5. Created at: 2022-05-31 17:09:16 +0800 CST
  6. Status: runningWorkflow
  7. Workflow:
  8. mode: DAG
  9. finished: false
  10. Suspend: false
  11. Terminated: false
  12. Steps
  13. - id:tw539smx7m
  14. name:deploy-topology
  15. type:deploy
  16. phase:failed
  17. message:step deploy: run step(provider=multicluster,do=deploy): Found 1 errors. [(error encountered in cluster c3: HandleComponentsRevision: controllerrevisions.apps is forbidden: User "alice" cannot list resource "controllerrevisions" in API group "apps" in the namespace "dev")]
  18. Services:
  1. Let’s create a new KubeConfig for another new User Bob. Bob will only be granted with the readonly privileges for the resources in the dev namespace of the control plane and cluster c2.
  1. $ vela auth gen-kubeconfig --user bob > bob.kubeconfig
  2. Private key generated.
  3. Certificate request generated.
  4. Certificate signing request bob generated.
  5. Certificate signing request bob approved.
  6. Signed certificate retrieved.
  7. $ vela auth grant-privileges --user bob --for-namespace dev --for-cluster=local,c2 --readonly
  8. ClusterRole kubevela:reader created in local.
  9. RoleBinding dev/kubevela:reader:binding created in local.
  10. ClusterRole kubevela:reader created in c2.
  11. RoleBinding dev/kubevela:reader:binding created in c2.
  12. Privileges granted.
  1. User Bob can see the applications and their status under the namespace of dev.
  1. $ KUBECONFIG=bob.kubeconfig vela ls -n dev
  2. APP COMPONENT TYPE TRAITS PHASE HEALTHY STATUS CREATED-TIME
  3. podinfo podinfo webservice running healthy Ready:1/1 2022-05-31 17:06:14 +0800 CST
  4. podinfo-bad podinfo-bad webservice workflowTerminated 2022-05-31 17:09:16 +0800 CST
  5. $ KUBECONFIG=bob.kubeconfig vela status podinfo -n dev
  6. About:
  7. Name: podinfo
  8. Namespace: dev
  9. Created at: 2022-05-31 17:06:14 +0800 CST
  10. Status: running
  11. Workflow:
  12. mode: DAG
  13. finished: true
  14. Suspend: false
  15. Terminated: false
  16. Steps
  17. - id:rk3npcpycl
  18. name:deploy-topology
  19. type:deploy
  20. phase:succeeded
  21. message:
  22. Services:
  23. - Name: podinfo
  24. Cluster: local Namespace: dev
  25. Type: webservice
  26. Healthy Ready:1/1
  27. No trait applied
  28. - Name: podinfo
  29. Cluster: c2 Namespace: dev
  30. Type: webservice
  31. Healthy Ready:1/1
  32. No trait applied
  1. But he is forbidden to do any mutating actions, such as deleting application in dev namespace.
  1. $ KUBECONFIG=bob.kubeconfig vela delete podinfo-bad -n dev
  2. Deleting Application "podinfo-bad"
  3. Error: delete application err: applications.core.oam.dev "podinfo-bad" is forbidden: User "bob" cannot delete resource "applications" in API group "core.oam.dev" in the namespace "dev"
  4. 2022/05/31 17:17:51 delete application err: applications.core.oam.dev "podinfo-bad" is forbidden: User "bob" cannot delete resource "applications" in API group "core.oam.dev" in the namespace "dev"
  1. Instead, User Alice can delete application.
  1. $ KUBECONFIG=alice.kubeconfig vela delete podinfo-bad -n dev
  2. application.core.oam.dev "podinfo-bad" deleted
  1. (Optional) After vela-prism installed, you will be able to list resources of the application through the following command
  1. $ KUBECONFIG=bob.kubeconfig vela status podinfo -n dev --tree --detail
  2. CLUSTER NAMESPACE RESOURCE STATUS APPLY_TIME DETAIL
  3. c2 ─── dev ─── Deployment/podinfo updated 2022-05-31 17:06:14 Ready: 1/1 Up-to-date: 1 Available: 1 Age: 13m
  4. local ─── dev ─── Deployment/podinfo updated 2022-05-31 17:06:14 Ready: 1/1 Up-to-date: 1 Available: 1 Age: 13m

Notice that if vela-prism is not installed, Alice and Bob will be forbidden to run this listing command.

The guide above demonstrates how system operators can grant limited privileges for users and therefore restrict the access of their created applications. For more detail explanation on how this capability is achieved, read the Underlying Mechanism article.

As the platform builder, you may want to bind KubeVela application with your customized identity. For example, using a manual specified ServiceAccount for the application. If you want to do so, it is not mandatory to enable the Authentication feature flag in KubeVela. Read the System Integration for more details.

Last updated on Aug 4, 2023 by Daniel Higuero