Deployment readiness (environment)

1. Overview

When the project is developed, we need to deploy, and we need to describe three deployment modes, server deployments, docker deployments, k8s deployment

Before deploying, we have installed relevant intermediaries. Our cicd is based primarily on gitlab and jenkins (three ways will be used), mirror storage using harbor(docker, k8s deployment needs to be used), a k8s cluster environment (k8s deployment).

Middleware for service use (mysql, redis, es etc.) will be deployed in srv-data.com, if you are online using cloud service and if you build it better than k8s.

So we need to configure the following：

Server NameRole
deploy-server.comDeployment of gitlab, jenkins, harbor (prepackaged docker, docker-compose)
srv-data.comDeployment of mysql, redis, es et al., simulate an independent environment, k8s internal connection to this server
nginx-gateway.comGateway, independent from k8s cluster
k8s clusterK8s Cluster

2. Build gitlab

2.1. Build gitlab

Creating folders

  1. $ mkdir gitlab && cd gitlab
  2. $ vim docker-compose.yml

docker-compose.yml

  1. version: "3"
  3. services:
  4.   gitlab:
  5.     image: "twang2218/gitlab-ce-zh"
  6.     container_name: "gitlab"
  7.     restart: always
  8.     hostname: "192.168.1.180" #部署机器的ip,非容器ip(因为是本地不是线上所以用ip，线上的话可以用域名)
  9.     environment:
  10.       TZ: "Asia/Shanghai"
  11.       GITLAB_OMNIBUS_CONFIG: |
  12.         external_url 'http://192.168.1.180'  #使用这个地址访问gitlab web ui(因为是本地不是线上所以用ip，线上的话可以用域名)
  13.         gitlab_rails['gitlab_shell_ssh_port'] = 2222 #ssh clone代码地址
  14.         unicorn['port'] = 8888 #gitlab一个内部端口
  15.     ports:
  16.       - "80:80" #web 80 端口
  17.       #- '443:443'      #web 443 端口,本次未使用就不开放了
  18.       - "2222:22" #ssh 检出代码 端口
  19.     volumes:
  20.       - ./etc:/etc/gitlab #Gitlab配置文件目录
  21.       - ./data:/var/opt/gitlab #Gitlab数据目录
  22.       - ./logs:/var/log/gitlab #Gitlab日志目录

Run

  1. $  docker-compose up -d

This execution may take a little long! Let’s go to the cup to rest again!

2.2 Visit gitlab

Visit http://192.168.1.103 (i.e. http://"docker-compose“)

Account default is root

2.3 Create project k8scode

2.4 Configure ssh public key

Click on the arrow under the avatar position, “Settings”

Configure your own public key, click “Add key” (your own search will not be generated by the public key, is not detailed here)

2.5 Upload Project

Click on the project, go back to the project just created and upload the k8scode project to this repository ssh:/git@192.168.180:2222/root/k8scode.git is enough to finish our gitlab build.

3、harbor

3.1 Deployment of harbor

Download harbo harbo https://github.com/goharbor/harbor/releases/download/v2.2.0/harbor-offline-installer-v2.2.0.tgz

Enter the harbor folder after downloading

  1. $ cd harbor && cp harbor.yml.tmpl harbor.yml

We open harbor.yml, modify the following

  1. hostname: 192.168.1.180 #修改为本机ip，不能使用localhost、127.0.0.1
  3. http:
  4.   port: 8077 #改一下http端口8077
  6. #https: #暂时将https注释掉，我们先不通过https只铜鼓http
  7. #  port: 443
  8. #  certificate: /your/certificate/path
  9. #  private_key: /your/private/key/path
  11. data_volume: /root/harbor/data #修改一下数据目录位置
  13. log:
  14.   level: info
  15.   local:
  16.     rotate_count: 50
  17.     rotate_size: 200M
  18.     location: /root/harbor/log #修改一下日志目录位置

Run “sudo ./install.sh” directly to wait a little longer.

3.2 Visit harbor

Browser input http://192.168.1.1180:8077

Account: admin

Password: Harbor12345 (recorded in harbor.yml, default is Harbor12345)

Login successful

This is our halbor build.

4、jenkins

4.1 Deployment of jenkins

Creating folders

  1. $ mkdir jenkins && cd jenkins
  2. $ vim docker-compose.yml

docker-compose.yml

  1. version: "3"
  2. services:
  3.   jenkins:
  4.     image: "jenkins/jenkins:lts"
  5.     container_name: jenkins
  6.     restart: always
  7.     environment:
  8.       - TZ=Asia/Shanghai
  9.     user: root
  10.     ports:
  11.       - "8989:8080"
  12.       - "50000:50000"
  13.     volumes:
  14.       - "./jenkins_home:/var/jenkins_home"
  15.       - "/var/run/docker.sock:/var/run/docker.sock"
  16.       - "/usr/bin/docker:/usr/bin/docker"
  17.       - "/root/port.sh:/root/port.sh"

[Note]: /root/port.sh is for subsequent k8s deployment

  1. #!/bin/sh
  3. case $1 in
  4. "identity-api") echo 1001
  5. ;;
  6. "identity-rpc") echo 1101
  7. ;;
  8. "usercenter-api") echo 1002
  9. ;;
  10. "usercenter-rpc") echo 1102
  11. ;;
  12. "message-mq") echo 1207
  13. ;;
  14. "mqueue-rpc") echo 1106
  15. ;;
  16. "order-api") echo 1004
  17. ;;
  18. "order-mq") echo 1204
  19. ;;
  20. "order-rpc") echo 1104
  21. ;;
  22. "payment-api") echo 1005
  23. ;;
  24. "payment-rpc") echo 1105
  25. ;;
  26. "travel-api") echo 1003
  27. ;;
  28. "travel-rpc") echo 1103
  29. esac

Run

  1. $ docker-compose up -d

This time is not slow. Can drink a cup of coffee

4.2 Mount Tools

1）将 goctl 复制到 jenkins 容器中

  1. $ docker cp $GOPATH/bin/goctl jenkins:/usr/local/bin
  2. $ docker exec -it jenkins /bin/sh #进入jenkins 容器
  3. $ goctl -v   #验证成功
  4. goctl version 1.3.0-20220201 linux/amd64

2）将 kubectl 文件复制到 jenkins 容器中

  1. $ curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
  2. $ sudo chmod a+x kubectl
  3. $ docker cp kubectl jenkins:/usr/local/bin
  4. $ docker exec -it jenkins /bin/sh #进入jenkins 容器
  5. $ kubectl version
  6. Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.3" .....

3）将 k8s 的配置.kube/config 复制到 jenkins 容器

  1. $ docker cp ~/.kube jenkins:/root/ #前提是家目录下的.kube文件夹中存在k8s的config配置
  2. $ docker exec -it jenkins /bin/sh #进入jenkins 容器
  3. $ kubectl ge ns
  4. default              Active   43m
  5. kube-node-lease      Active   43m
  6. kube-public          Active   43m
  7. kube-system          Active   43m
  8. local-path-storage   Active   43m

These 4 parts above can also be hit directly in the mirror, and I leave it to you yourself.

4.3 Visit to jenkins

http://192.168.1.180:8989

The first visit does not panic. Let you wait a little while it is ready to jump to the landing page when it is ready.

The following interface is ready because our directory is mounted. We see your native jenkins_home/secrets/initialAdminPassword password, enter the next step

Select “Install Recommended Plugins”

Then wait until the plugin is installed

4.4 Create User

root

root

4.5 Deployment Complete

Deployed to this jenkins

4.6 Add credentials

Click on the left menu “Manage Jenkins”

Click “Manage Credentials”

Click on the triangle after “Global” and then click “Add Credits”

Enter the “Add Credits” page, type we choose “SSH Username with private key”,Usernameis a gitlab identifier, after which you add pipeline you know this logo is self-defined on behalf of gitlab credentials, rivate Key`, a private key configured in gitlab (previously we have a public key corresponding to gitlab, here is our own private key), our voucher is for jenkins to go to gitlab free of cryptography.

Suffice it.

4.7 Add harbor repository configuration

Go to homepage, click on the menu on the left Manage Jenkins -> Configure System

Swipe down toGlobal Propertiesentry, add docker private repository information such as graphdockerUsername,dockeruser password,dockerprivate repository address

Click to save

4.8 Configure git

EnterManage Jenkins->Global Tool Configuration, find Git entry, fill jenkins in the machine git executable path; if not, download Git plugin in jenkins plugin management and don’t need to be taken into custody (graph below)

Git Parameter plugin to configure pipline

Tap “System Configuration” -> “Plugin Management”

Then click “Optional Plugins”, type “Git Parameter” in the search, like the one

Finished installing and restarting to complete this jenkin.

5、k8s

The deployment of k8s is not described. Use kubeadm, rancher, kind to install it, or buy cloud container services, all with a k8s cluster.

References