Kubernetes Nacos

This project contains a Nacos Docker image meant to facilitate the deployment of Nacos on Kubernetes via StatefulSets.

Quick Start

  • Clone Project
  1. git clone https://github.com/nacos-group/nacos-k8s.git
  • Simple Start

If you want to start Nacos without NFS, but emptyDirs will possibly result in a loss of data. as follows:

  1. cd nacos-k8s
  2. chmod +x quick-startup.sh
  3. ./quick-startup.sh
  • Testing
    • Service registration
    1. curl -X PUT 'http://cluster-ip:8848/nacos/v1/ns/instance?serviceName=nacos.naming.serviceName&ip=20.18.7.10&port=8080'
    • Service discovery
    1. curl -X GET 'http://cluster-ip:8848/nacos/v1/ns/instances?serviceName=nacos.naming.serviceName'
    • Publish config
    1. curl -X POST "http://cluster-ip:8848/nacos/v1/cs/configs?dataId=nacos.cfg.dataId&group=test&content=helloWorld"
    • Get config
    1. curl -X GET "http://cluster-ip:8848/nacos/v1/cs/configs?dataId=nacos.cfg.dataId&group=test"

Advanced

In advanced use, the cluster is automatically scaled and data is persisted, but PersistentVolumeClaims must be deployed. In this example, NFS is used.

Deploy NFS

  • Create Role
  1. kubectl create -f deploy/nfs/rbac.yaml

If your K8S namespace is not default, execute the following script before creating RBAC

  1. # Set the subject of the RBAC objects to the current namespace where the provisioner is being deployed
  2. $ NS=$(kubectl config get-contexts|grep -e "^\*" |awk '{print $5}')
  3. $ NAMESPACE=${NS:-default}
  4. $ sed -i'' "s/namespace:.*/namespace: $NAMESPACE/g" ./deploy/nfs/rbac.yaml
  • Create ServiceAccount And deploy NFS-Client Provisioner
  1. kubectl create -f deploy/nfs/deployment.yaml
  • Create NFS StorageClass
  1. kubectl create -f deploy/nfs/class.yaml
  • Verify that NFS is working
  1. kubectl get pod -l app=nfs-client-provisioner

Deploy database

  • Deploy master
  1. cd nacos-k8s
  2. kubectl create -f deploy/mysql/mysql-master-nfs.yaml
  • Deploy slave
  1. cd nacos-k8s
  2. kubectl create -f deploy/mysql/mysql-slave-nfs.yaml
  • Verify that Database is working
  1. # master
  2. kubectl get pod
  3. NAME READY STATUS RESTARTS AGE
  4. mysql-master-gf2vd 1/1 Running 0 111m
  5. # slave
  6. kubectl get pod
  7. mysql-slave-kf9cb 1/1 Running 0 110m

Deploy Nacos

  • Modify deploy/nacos/nacos-pvc-nfs.yaml
  1. data:
  2. mysql.master.db.name: "db name"
  3. mysql.master.port: "master db port"
  4. mysql.slave.port: "slave db port"
  5. mysql.master.user: "master db username"
  6. mysql.master.password: "master db password"
  • Create Nacos
  1. kubectl create -f nacos-k8s/deploy/nacos/nacos-pvc-nfs.yaml
  • Verify that Nacos is working
  1. kubectl get pod -l app=nacos
  2. NAME READY STATUS RESTARTS AGE
  3. nacos-0 1/1 Running 0 19h
  4. nacos-1 1/1 Running 0 19h
  5. nacos-2 1/1 Running 0 19h

Scale Testing

  • Use kubectl exec to get the cluster config of the Pods in the nacos StatefulSet.
  1. for i in 0 1; do echo nacos-$i; kubectl exec nacos-$i cat conf/cluster.conf; done

The StatefulSet controller provides each Pod with a unique hostname based on its ordinal index. The hostnames take the form of <statefulset name>-<ordinal index>. Because the replicas field of the nacos StatefulSet is set to 2, In the cluster file only two nacos address

k8s

  • Use kubectl to scale StatefulSets
  1. kubectl scale sts nacos --replicas=3

scale

  • Use kubectl exec to get the cluster config of the Pods in the nacos StatefulSet after scale StatefulSets
  1. for i in 0 1 2; do echo nacos-$i; kubectl exec nacos-$i cat conf/cluster.conf; done

get_cluster_after

  • Use kubectl exec to get the state of the Pods in the nacos StatefulSet after scale StatefulSets
  1. for i in 0 1 2; do echo nacos-$i; kubectl exec nacos-$i curl -X GET "http://localhost:8848/nacos/v1/ns/raft/state"; done

You can find that the new node has joined the cluster

Prerequisites

  • Kubernetes Node configuration(for reference only)
Network IPHostnameConfiguration
172.17.79.3k8s-masterCentOS Linux release 7.4.1708 (Core) Single-core processor Mem 4G Cloud disk 40G
172.17.79.4node01CentOS Linux release 7.4.1708 (Core) Single-core processor Mem 4G Cloud disk 40G
172.17.79.5node02CentOS Linux release 7.4.1708 (Core) Single-core processor Mem 4G Cloud disk 40G
  • Kubernetes version:1.12.2+
  • NFS version:4.1+

Limitations

  • Persistent Volumes must be used. emptyDirs will possibly result in a loss of data

Project directory

Directory NameDescription
pluginHelp Nacos cluster achieve automatic scaling in K8s
deployDeploy the required files

Configuration properties

  • nacos-pvc-nfs.yaml or nacos-quick-start.yaml
NameRequiredDescription
mysql.master.db.nameYMaster database name
mysql.master.portNMaster database port
mysql.slave.portNSlave database port
mysql.master.userYMaster database username
mysql.master.passwordYMaster database password
NACOS_REPLICASYThe number of clusters must be consistent with the value of the replicas attribute
NACOS_SERVER_PORTNNacos port,default:8848
PREFER_HOST_MODEYEnable Nacos cluster node domain name support
  • nfs deployment.yaml
NameRequiredDescription
NFS_SERVERYNFS server address
NFS_PATHYNFS server shared directory
serverYNFS server address
pathYNFS server shared directory
  • mysql yaml
NameRequiredDescription
MYSQL_ROOT_PASSWORDNRoot password
MYSQL_DATABASEYDatabase Name
MYSQL_USERYDatabase Username
MYSQL_PASSWORDYDatabase Password
MYSQL_REPLICATION_USERYMaster-slave replication username
MYSQL_REPLICATION_PASSWORDYMaster-slave replication password
Nfs:serverYNFS server address
Nfs:pathYNFS server shared path