准备证书

声明:本文使用的证书为自签名,不能用于商用环境

部署集群前,需要生成集群各组件之间通信所需的证书。本文使用开源 CFSSL 作为验证部署工具,以便用户了解证书的配置和集群组件之间证书的关联关系。用户可以根据实际情况选择合适的工具,例如 OpenSSL 。

编译安装 CFSSL

编译安装 CFSSL 的参考命令如下(需要互联网下载权限,需要配置代理的请先完成配置),

  1. $ wget --no-check-certificate https://github.com/cloudflare/cfssl/archive/v1.5.0.tar.gz
  2. $ tar -zxf v1.5.0.tar.gz
  3. $ cd cfssl-1.5.0/
  4. $ make -j6
  5. $ cp bin/* /usr/local/bin/

生成根证书

编写 CA 配置文件,例如 ca-config.json:

  1. $ cat ca-config.json | jq
  2. {
  3. "signing": {
  4. "default": {
  5. "expiry": "8760h"
  6. },
  7. "profiles": {
  8. "kubernetes": {
  9. "usages": [
  10. "signing",
  11. "key encipherment",
  12. "server auth",
  13. "client auth"
  14. ],
  15. "expiry": "8760h"
  16. }
  17. }
  18. }
  19. }

编写 CA CSR 文件,例如 ca-csr.json:

  1. $ cat ca-csr.json | jq
  2. {
  3. "CN": "Kubernetes",
  4. "key": {
  5. "algo": "rsa",
  6. "size": 2048
  7. },
  8. "names": [
  9. {
  10. "C": "CN",
  11. "L": "HangZhou",
  12. "O": "openEuler",
  13. "OU": "WWW",
  14. "ST": "BinJiang"
  15. }
  16. ]
  17. }

生成 CA 证书和密钥:

  1. $ cfssl gencert -initca ca-csr.json | cfssljson -bare ca

得到如下证书:

  1. ca.csr ca-key.pem ca.pem

生成 admin 账户证书

admin 是 K8S 用于系统管理的一个账户,编写 admin 账户的 CSR 配置,例如 admin-csr.json:

  1. cat admin-csr.json | jq
  2. {
  3. "CN": "admin",
  4. "key": {
  5. "algo": "rsa",
  6. "size": 2048
  7. },
  8. "names": [
  9. {
  10. "C": "CN",
  11. "L": "HangZhou",
  12. "O": "system:masters",
  13. "OU": "Containerum",
  14. "ST": "BinJiang"
  15. }
  16. ]
  17. }

生成证书:

  1. $ cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes admin-csr.json | cfssljson -bare admin

结果如下:

  1. admin.csr admin-key.pem admin.pem

生成 service-account 账户证书

编写 service-account 账户的 CSR 配置文件,例如 service-account-csr.json:

  1. cat service-account-csr.json | jq
  2. {
  3. "CN": "service-accounts",
  4. "key": {
  5. "algo": "rsa",
  6. "size": 2048
  7. },
  8. "names": [
  9. {
  10. "C": "CN",
  11. "L": "HangZhou",
  12. "O": "Kubernetes",
  13. "OU": "openEuler k8s install",
  14. "ST": "BinJiang"
  15. }
  16. ]
  17. }

生成证书:

  1. $ cfssl gencert -ca=../ca/ca.pem -ca-key=../ca/ca-key.pem -config=../ca/ca-config.json -profile=kubernetes service-account-csr.json | cfssljson -bare service-account

结果如下:

  1. service-account.csr service-account-key.pem service-account.pem

生成 kube-controller-manager 组件证书

编写 kube-controller-manager 的 CSR 配置:

  1. {
  2. "CN": "system:kube-controller-manager",
  3. "key": {
  4. "algo": "rsa",
  5. "size": 2048
  6. },
  7. "names": [
  8. {
  9. "C": "CN",
  10. "L": "HangZhou",
  11. "O": "system:kube-controller-manager",
  12. "OU": "openEuler k8s kcm",
  13. "ST": "BinJiang"
  14. }
  15. ]
  16. }

生成证书:

  1. $ cfssl gencert -ca=../ca/ca.pem -ca-key=../ca/ca-key.pem -config=../ca/ca-config.json-profile=kubernetes kube-controller-manager-csr.json | cfssljson -bare kube-controller-manager

结果如下:

  1. kube-controller-manager.csr kube-controller-manager-key.pem kube-controller-manager.pem

生成 kube-proxy 证书

编写 kube-proxy 的 CSR 配置:

  1. {
  2. "CN": "system:kube-proxy",
  3. "key": {
  4. "algo": "rsa",
  5. "size": 2048
  6. },
  7. "names": [
  8. {
  9. "C": "CN",
  10. "L": "HangZhou",
  11. "O": "system:node-proxier",
  12. "OU": "openEuler k8s kube proxy",
  13. "ST": "BinJiang"
  14. }
  15. ]
  16. }

生成证书:

  1. $ cfssl gencert -ca=../ca/ca.pem -ca-key=../ca/ca-key.pem -config=../ca/ca-config.json -profile=kubernetes kube-proxy-csr.json | cfssljson -bare kube-proxy

结果如下:

  1. kube-proxy.csr kube-proxy-key.pem kube-proxy.pem

生成 kube-scheduler 证书

编写 kube-scheduler 的 CSR 配置:

  1. {
  2. "CN": "system:kube-scheduler",
  3. "key": {
  4. "algo": "rsa",
  5. "size": 2048
  6. },
  7. "names": [
  8. {
  9. "C": "CN",
  10. "L": "HangZhou",
  11. "O": "system:kube-scheduler",
  12. "OU": "openEuler k8s kube scheduler",
  13. "ST": "BinJiang"
  14. }
  15. ]
  16. }

生成证书:

  1. $ cfssl gencert -ca=../ca/ca.pem -ca-key=../ca/ca-key.pem -config=../ca/ca-config.json -profile=kubernetes kube-scheduler-csr.json | cfssljson -bare kube-scheduler

结果如下:

  1. kube-scheduler.csr kube-scheduler-key.pem kube-scheduler.pem

生成 kubelet 证书

由于证书涉及到 kubelet 所在机器的 hostname 和 IP 地址信息,因此每个 node 节点配置不尽相同,所以编写脚本完成,生成脚本如下:

  1. $ cat node_csr_gen.bash
  2. #!/bin/bash
  3. nodes=(k8snode1 k8snode2 k8snode3)
  4. IPs=("192.168.122.157" "192.168.122.158" "192.168.122.159")
  5. for i in "${!nodes[@]}"; do
  6. cat > "${nodes[$i]}-csr.json" <<EOF
  7. {
  8. "CN": "system:node:${nodes[$i]}",
  9. "key": {
  10. "algo": "rsa",
  11. "size": 2048
  12. },
  13. "names": [
  14. {
  15. "C": "CN",
  16. "L": "HangZhou",
  17. "O": "system:nodes",
  18. "OU": "openEuler k8s kubelet",
  19. "ST": "BinJiang"
  20. }
  21. ]
  22. }
  23. EOF
  24. # generate ca
  25. echo "generate: ${nodes[$i]} ${IPs[$i]}"
  26. cfssl gencert -ca=../ca/ca.pem -ca-key=../ca/ca-key.pem -config=../ca/ca-config.json -hostname=${nodes[$i]},${IPs[$i]}-profile=kubernetes ${nodes[$i]}-csr.json | cfssljson -bare ${nodes[$i]}
  27. done

说明:如果节点存在多个 IP 或者其他别名,-hostname 可以增加其他的 IP 或者 hostname

结果如下:

  1. k8snode1.csr k8snode1.pem k8snode2-key.pem k8snode3-csr.json
  2. k8snode1-csr.json k8snode2.csr k8snode2.pem k8snode3-key.pem
  3. k8snode1-key.pem k8snode2-csr.json k8snode3.csr k8snode3.pem

CSR 配置信息,以 k8snode1 为例如下:

  1. $ cat k8snode1-csr.json
  2. {
  3. "CN": "system:node:k8snode1",
  4. "key": {
  5. "algo": "rsa",
  6. "size": 2048
  7. },
  8. "names": [
  9. {
  10. "C": "CN",
  11. "L": "HangZhou",
  12. "O": "system:nodes",
  13. "OU": "openEuler k8s kubelet",
  14. "ST": "BinJiang"
  15. }
  16. ]
  17. }

注意:由于每个 node 所属的账户组为 system:node,因此 CSR 的 CN 字段都为 system:node 加上hostname

生成 kube-apiserver 证书

编写 kube api server 的 CSR 配置文件:

  1. $ cat kubernetes-csr.json | jq
  2. {
  3. "CN": "kubernetes",
  4. "key": {
  5. "algo": "rsa",
  6. "size": 2048
  7. },
  8. "names": [
  9. {
  10. "C": "CN",
  11. "L": "HangZhou",
  12. "O": "Kubernetes",
  13. "OU": "openEuler k8s kube api server",
  14. "ST": "BinJiang"
  15. }
  16. ]
  17. }

生成证书和密钥:

  1. cfssl gencert -ca=../ca/ca.pem -ca-key=../ca/ca-key.pem -config=../ca/ca-config.json -hostname=10.32.0.1,192.168.122.154,192.168.122.155,192.168.122.156,127.0.0.1,kubernetes,kubernetes.default,kubernetes.default.svc,kubernetes.default.svc.cluster,kubernetes.svc.cluster.local -profile=kubernetes kubernetes-csr.json | cfssljson -bare kubernetes

结果如下:

  1. kubernetes.csr kubernetes-key.pem kubernetes.pem

说明:10.32.0.1 是内部 services 使用的 IP 地址区间,可以设置为其他值,后面启动 apiserver 服务时,会设置该参数。

生成 etcd 证书(可选)

部署 etcd 有两种方式:

  • 在每个 api-server 对应的机器都启动一个 etcd 服务
  • 独立部署一个 etcd 集群服务

如果是和 api-server 一起部署,那么直接使用上面生成的 kubernetes-key.pemkubernetes.pem 证书即可。

如果是独立的etcd集群,那么需要创建证书如下:

编写 etcd 的 CSR 配置:

  1. cat etcd-csr.json | jq
  2. {
  3. "CN": "ETCD",
  4. "key": {
  5. "algo": "rsa",
  6. "size": 2048
  7. },
  8. "names": [
  9. {
  10. "C": "CN",
  11. "L": "HangZhou",
  12. "O": "ETCD",
  13. "OU": "openEuler k8s etcd",
  14. "ST": "BinJiang"
  15. }
  16. ]
  17. }

生成证书:

  1. $ cfssl gencert -ca=../ca/ca.pem -ca-key=../ca/ca-key.pem -config=../ca/ca-config.json -hostname=192.168.122.154,192.168.122.155,192.168.122.156,127.0.0.1 -profile=kubernetes etcd-csr.json | cfssljson -bare etcd

说明:假设 etcd 集群的 IP地址是 192.168.122.154,192.168.122.155,192.168.122.156

结果如下:

  1. etcd.csr etcd-key.pem etcd.pem