负载均衡

负载均衡

环境说明

除 JumpServer 自身组件外, 其他组件的高可用请参考对应的官方文档进行部署
按照此方式部署后, 后续只需要根据需要扩容 core web 节点然后添加节点到 tengine 即可
如果已经有 HLB 或者 SLB 可以跳过 Tengine 部署, 第三方 LB 要注意 session 和 websocket 问题
如果已经有云存储(* S3/Ceph/Swift/OSS/Azure) 可以跳过 MinIO 部署, MySQL Redis 也一样

DB	Version		Cache	Version
MySQL	>= 5.7		Redis	>= 5.0
MariaDB	>= 10.2

Server Name	IP	Port	Use	Minimize Hardware	Standard Hardware
MySQL	192.168.100.11	3306	Core	2Core/4GB RAM/1T HDD	4Core/16GB RAM/1T SSD
Redis	192.168.100.11	6379	Core, Koko	2Core/4GB RAM/60G HDD	2Core/8GB RAM/60G SSD
Tengine	192.168.100.100	80,443	All	2Core/4GB RAM/60G HDD	4Core/8GB RAM/60G SSD
Core Web 01	192.168.100.21	8080	Tengine	2Core/8GB RAM/60G HDD	4Core/8GB RAM/90G SSD
Core Web 02	192.168.100.22	8080	Tengine	2Core/8GB RAM/60G HDD	4Core/8GB RAM/90G SSD
Core Task	192.168.100.31		Tengine	4Core/8GB RAM/60G HDD	4Core/16GB RAM/90G SSD
MinIO	192.168.100.41	9000	KoKo, Guacamole	2Core/4GB RAM/1T HDD	4Core/8GB RAM/1T SSD

Core Task 目前仅支持单节点运行, 后续会优化

部署 MySQL 服务

服务器: 192.168.100.11

设置 Repo

yum -y localinstall http://mirrors.ustc.edu.cn/mysql-repo/mysql57-community-release-el7.rpm

安装 MySQL

yum install -y mysql-community-server

配置 MySQL

if [ ! "$(cat /usr/bin/mysqld_pre_systemd | grep -v ^\# | grep initialize-insecure )" ]; then
    sed -i "s@--initialize @--initialize-insecure @g" /usr/bin/mysqld_pre_systemd
fi

启动 MySQL

systemctl enable mysqld
systemctl start mysqld

数据库授权

mysql -uroot

Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.7.32 MySQL Community Server (GPL)
Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> create database jumpserver default charset 'utf8';
Query OK, 1 row affected (0.00 sec)
mysql> set global validate_password_policy=LOW;
Query OK, 0 rows affected (0.00 sec)
mysql> create user 'jumpserver'@'%' identified by 'weakPassword';
Query OK, 0 rows affected (0.00 sec)
mysql> grant all on jumpserver.* to 'jumpserver'@'%';
Query OK, 0 rows affected, 1 warning (0.00 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)
mysql> exit
Bye

配置防火墙

firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="192.168.100.0/24" port protocol="tcp" port="3306" accept"
firewall-cmd --reload

部署 Redis 服务

服务器: 192.168.100.11

设置 Repo

yum -y install epel-release https://repo.ius.io/ius-release-el7.rpm

安装 Redis

yum install -y redis5

配置 Redis

sed -i "s/bind 127.0.0.1/bind 0.0.0.0/g" /etc/redis.conf
sed -i "561i maxmemory-policy allkeys-lru" /etc/redis.conf
sed -i "481i requirepass weakPassword" /etc/redis.conf

启动 Redis

systemctl enable redis
systemctl start redis

配置防火墙

firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="192.168.100.0/24" port protocol="tcp" port="6379" accept"
firewall-cmd --reload

部署 Core Web 01

服务器: 192.168.100.21

下载 jumpserver-install

cd /opt
yum -y install wget
wget https://github.com/jumpserver/installer/releases/download/v2.9.2/jumpserver-installer-v2.9.2.tar.gz
tar -xf jumpserver-installer-v2.9.2.tar.gz
cd jumpserver-installer-v2.9.2

修改配置文件

vi config-example.txt

# 修改下面选项, 其他保持默认
### 注意: SECRET_KEY 和要其他 JumpServer 服务器一致, 加密的数据将无法解密
## Task 配置
USE_TASK=0                                                     # 不启动 jms_celery
# Core 配置
### 启动后不能再修改，否则密码等等信息无法解密
SECRET_KEY=kWQdmdCQKjaWlHYpPhkNQDkfaRulM6YnHctsHLlSPs8287o2kW  # 要其他 JumpServer 服务器一致 (*)
BOOTSTRAP_TOKEN=KXOeyNgDeTdpeu9q                               # 要其他 JumpServer 服务器一致 (*)
LOG_LEVEL=ERROR
# SESSION_COOKIE_AGE=86400
SESSION_EXPIRE_AT_BROWSER_CLOSE=true
# KoKo 配置
SHARE_ROOM_TYPE=redis                                          # KoKo 使用 redis 共享

./jmsctl.sh install

       ██╗██╗   ██╗███╗   ███╗██████╗ ███████╗███████╗██████╗ ██╗   ██╗███████╗██████╗
       ██║██║   ██║████╗ ████║██╔══██╗██╔════╝██╔════╝██╔══██╗██║   ██║██╔════╝██╔══██╗
       ██║██║   ██║██╔████╔██║██████╔╝███████╗█████╗  ██████╔╝██║   ██║█████╗  ██████╔╝
  ██   ██║██║   ██║██║╚██╔╝██║██╔═══╝ ╚════██║██╔══╝  ██╔══██╗╚██╗ ██╔╝██╔══╝  ██╔══██╗
  ╚█████╔╝╚██████╔╝██║ ╚═╝ ██║██║     ███████║███████╗██║  ██║ ╚████╔╝ ███████╗██║  ██║
   ╚════╝  ╚═════╝ ╚═╝    ╚═╝╚═╝     ╚══════╝╚══════╝╚═╝  ╚═╝  ╚═══╝  ╚══════╝╚═╝  ╚═╝
                                                             Version:  v2.9.2
>>> 安装配置 Docker
1. 安装 Docker
开始下载 Docker 程序 ...
完成
开始下载 Docker Compose 程序 ...
完成
2. 配置 Docker
是否需要自定义 Docker 数据目录, 默认将使用 /var/lib/docker 目录? (y/n)  (默认为 n): n
完成
3. 启动 Docker
Docker 版本发生改变 或 Docker 配置文件发生变化，是否要重启? (y/n)  (默认为 y): y
完成
>>> 加载 Docker 镜像
Docker: Pulling from jumpserver/core:v2.9.2         [ OK ]
Docker: Pulling from jumpserver/koko:v2.9.2         [ OK ]
Docker: Pulling from jumpserver/luna:v2.9.2         [ OK ]
Docker: Pulling from jumpserver/nginx:alpine2       [ OK ]
Docker: Pulling from jumpserver/redis:6-alpine      [ OK ]
Docker: Pulling from jumpserver/lina:v2.9.2         [ OK ]
Docker: Pulling from jumpserver/mysql:5             [ OK ]
Docker: Pulling from jumpserver/guacamole:v2.9.2    [ OK ]
>>> 安装配置 JumpServer
1. 检查配置文件
配置文件位置: /opt/jumpserver/config
/opt/jumpserver/config/config.txt                 [ √ ]
/opt/jumpserver/config/nginx/lb_http_server.conf  [ √ ]
/opt/jumpserver/config/nginx/lb_ssh_server.conf   [ √ ]
/opt/jumpserver/config/core/config.yml   [ √ ]
/opt/jumpserver/config/koko/config.yml   [ √ ]
/opt/jumpserver/config/mysql/my.cnf      [ √ ]
/opt/jumpserver/config/redis/redis.conf  [ √ ]
完成
2. 配置 Nginx
配置文件位置: /opt/jumpserver/config/nginx/cert
/opt/jumpserver/config/nginx/cert/server.crt  [ √ ]
/opt/jumpserver/config/nginx/cert/server.key  [ √ ]
完成
3. 备份配置文件
备份至 /opt/jumpserver/config/backup/config.txt.2021-03-19_08-01-51
完成
4. 配置网络
是否需要支持 IPv6? (y/n)  (默认为 n): n
完成
5. 配置加密密钥
SECRETE_KEY:     ICAgIGluZXQ2IDI0MDk6OGE0ZDpjMjg6ZjkwMTo6ZDRjLzEyO
BOOTSTRAP_TOKEN: ICAgIGluZXQ2IDI0
完成
6. 配置持久化目录
是否需要自定义持久化存储, 默认将使用目录 /opt/jumpserver? (y/n)  (默认为 n): n
完成
7. 配置 MySQL
是否使用外部mysql (y/n)  (默认为n): y
请输入mysql的主机地址 (无默认值): 192.168.100.11
请输入mysql的端口 (默认为3306): 3306
请输入mysql的数据库(事先做好授权) (默认为jumpserver): jumpserver
请输入mysql的用户名 (无默认值): jumpserver
请输入mysql的密码 (无默认值): weakPassword
完成
8. 配置 Redis
是否使用外部redis  (y/n)  (默认为n): y
请输入redis的主机地址 (无默认值): 192.168.100.11
请输入redis的端口 (默认为6379): 6379
请输入redis的密码 (无默认值): weakPassword
完成
>>> 安装完成了
1. 可以使用如下命令启动, 然后访问
./jmsctl.sh start
2. 其它一些管理命令
./jmsctl.sh stop
./jmsctl.sh restart
./jmsctl.sh backup
./jmsctl.sh upgrade
更多还有一些命令, 你可以 ./jmsctl.sh --help 来了解
3. Web 访问
http://192.168.100.248:8080
https://192.168.100.248:8443
默认用户: admin  默认密码: admin
4. SSH/SFTP 访问
ssh admin@192.168.100.248 -p2222
sftp -P2222 admin@192.168.100.248
5. 更多信息
我们的官网: https://www.jumpserver.org/
我们的文档: https://docs.jumpserver.org/

启动 JumpServer

./jmsctl.sh start

Creating network "jms_net" with driver "bridge"
Creating jms_mysql     ... done
Creating jms_redis     ... done
Creating jms_core      ... done
Creating jms_luna      ... done
Creating jms_lina      ... done
Creating jms_guacamole ... done
Creating jms_koko      ... done
Creating jms_nginx     ... done

部署 Core Web 02

服务器: 192.168.100.22

下载 jumpserver-install

cd /opt
yum -y install wget
wget https://github.com/jumpserver/installer/releases/download/v2.9.2/jumpserver-installer-v2.9.2.tar.gz
tar -xf jumpserver-installer-v2.9.2.tar.gz
cd jumpserver-installer-v2.9.2

修改配置文件

vi config-example.txt

# 修改下面选项, 其他保持默认
### 注意: SECRET_KEY 和要其他 JumpServer 服务器一致, 加密的数据将无法解密
## Task 配置
USE_TASK=0                                                     # 不启动 jms_celery
# Core 配置
### 启动后不能再修改，否则密码等等信息无法解密
SECRET_KEY=kWQdmdCQKjaWlHYpPhkNQDkfaRulM6YnHctsHLlSPs8287o2kW  # 要其他 JumpServer 服务器一致 (*)
BOOTSTRAP_TOKEN=KXOeyNgDeTdpeu9q                               # 要其他 JumpServer 服务器一致 (*)
LOG_LEVEL=ERROR
# SESSION_COOKIE_AGE=86400
SESSION_EXPIRE_AT_BROWSER_CLOSE=true
# KoKo 配置
SHARE_ROOM_TYPE=redis                                          # KoKo 使用 redis 共享

./jmsctl.sh install

启动 JumpServer

./jmsctl.sh start

Creating network "jms_net" with driver "bridge"
Creating jms_mysql     ... done
Creating jms_redis     ... done
Creating jms_core      ... done
Creating jms_luna      ... done
Creating jms_lina      ... done
Creating jms_guacamole ... done
Creating jms_koko      ... done
Creating jms_nginx     ... done

部署 Core Task

服务器: 192.168.100.31

下载 jumpserver-install

cd /opt
yum -y install wget
wget https://github.com/jumpserver/installer/releases/download/v2.9.2/jumpserver-installer-v2.9.2.tar.gz
tar -xf jumpserver-installer-v2.9.2.tar.gz
cd jumpserver-installer-v2.9.2

修改配置文件

vi config-example.txt

# 修改下面选项, 其他保持默认
### 注意: SECRET_KEY 和要其他 JumpServer 服务器一致, 加密的数据将无法解密
## Task 配置
USE_TASK=1                                                     # 启动 jms_celery
# Core 配置
### 启动后不能再修改，否则密码等等信息无法解密
SECRET_KEY=kWQdmdCQKjaWlHYpPhkNQDkfaRulM6YnHctsHLlSPs8287o2kW  # 要其他 JumpServer 服务器一致 (*)
BOOTSTRAP_TOKEN=KXOeyNgDeTdpeu9q                               # 要其他 JumpServer 服务器一致 (*)
LOG_LEVEL=ERROR
# SESSION_COOKIE_AGE=86400
SESSION_EXPIRE_AT_BROWSER_CLOSE=true
# KoKo 配置
SHARE_ROOM_TYPE=redis                                          # KoKo 使用 redis 共享

./jmsctl.sh install

启动 JumpServer

./jmsctl.sh start

Creating network "jms_net" with driver "bridge"
Creating jms_mysql     ... done
Creating jms_redis     ... done
Creating jms_core      ... done
Creating jms_celery    ... done
Creating jms_luna      ... done
Creating jms_lina      ... done
Creating jms_guacamole ... done
Creating jms_koko      ... done
Creating jms_nginx     ... done

部署 Tengine 服务

服务器: 192.168.100.100

配置 Repo

vi /etc/yum.repos.d/nginx.repo

[nginx-stable]
name=nginx stable repo
baseurl=http://nginx.org/packages/centos/$releasever/$basearch/
gpgcheck=1
enabled=1
gpgkey=https://nginx.org/keys/nginx_signing.key
module_hotfixes=true

安装 Tengine

yum install -y https://github.com/wojiushixiaobai/tengine-rpm/releases/download/2.3.2/tengine-2.3.2-1.el7.ngx.x86_64.rpm

配置 Nginx

vi /etc/nginx/nginx.conf

user  nginx;
worker_processes  auto;
error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;
events {
    worker_connections  1024;
}
stream {
    log_format  proxy  '$remote_addr [$time_local] '
                       '$protocol $status $bytes_sent $bytes_received '
                       '$session_time "$upstream_addr" '
                       '"$upstream_bytes_sent" "$upstream_bytes_received" "$upstream_connect_time"';
    access_log /var/log/nginx/tcp-access.log  proxy;
    open_log_file_cache off;
    upstream kokossh {
        # core web 节点
        server 192.168.100.21:2222;
        server 192.168.100.22:2222;
        least_conn;
    }
    server {
        # 对外 ssh 端口
        listen 2222;
        proxy_pass kokossh;
        proxy_protocol on;
        proxy_connect_timeout 1s;
    }
}
http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';
    access_log  /var/log/nginx/access.log  main;
    sendfile        on;
    #tcp_nopush     on;
    keepalive_timeout  65;
    #gzip  on;
    include /etc/nginx/conf.d/*.conf;
}

echo > /etc/nginx/conf.d/default.conf
vi /etc/nginx/conf.d/jumpserver.conf

upstream core_web {
    # 用户连接时使用 ip_hash 负载
    server 192.168.100.21:8080;
    server 192.168.100.22:8080;
    session_sticky;
}
upstream core_task {
    # use_task = 1 的任务服务器, 目前只能单任务运行
    server 192.168.100.31:8080;
}
server {
    listen 80;
    server_name demo.jumpserver.org;  # 自行修改成你的域名
    return 301 https://$server_name$request_uri;
}
server {
    listen 443 ssl;
    server_name          demo.jumpserver.org;  # 自行修改成你的域名
    ssl_certificate      /etc/nginx/sslkey/1_jumpserver.org.crt;  # 自行设置证书
    ssl_certificate_key  /etc/nginx/sslkey/2_jumpserver.org.key;  # 自行设置证书
    ssl_session_timeout  5m;
    ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE:ECDH:AES:HIGH:!NULL:!aNULL:!MD5:!ADH:!RC4;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_prefer_server_ciphers on;
    client_max_body_size 4096m;  # 录像上传大小限制
    location ~ /replay/ {
        proxy_pass http://core_web;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
    location ~ /(ops|task|tasks|flower)/ {
        proxy_pass http://core_task;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
    location /ws/ {
        proxy_pass http://core_task/ws/;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
    location / {
        proxy_pass http://core_web;
        proxy_buffering  off;
        proxy_request_buffering off;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

nginx -t

启动 Tengine

systemctl enable nginx
systemctl start nginx

配置防火墙

firewall-cmd --permanent --zone=public --add-port=80/tcp
firewall-cmd --permanent --zone=public --add-port=443/tcp
firewall-cmd --permanent --zone=public --add-port=2222/tcp
firewall-cmd --reload

部署 MinIO 服务

服务器: 192.168.100.41

安装 Docker

yum install -y yum-utils device-mapper-persistent-data lvm2
yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
sed -i 's+download.docker.com+mirrors.aliyun.com/docker-ce+' /etc/yum.repos.d/docker-ce.repo
yum makecache fast
yum -y install docker-ce

配置 Docker

mkdir /etc/docker/
vi /etc/docker/daemon.json

{
  "live-restore": true,
  "registry-mirrors": ["https://hub-mirror.c.163.com", "https://bmtrgdvx.mirror.aliyuncs.com", "http://f1361db2.m.daocloud.io"],
  "log-driver": "json-file",
  "log-opts": {"max-file": "3", "max-size": "10m"}
}

启动 Docker

systemctl enable docker
systemctl start docker

下载 MinIO 镜像

docker pull minio/minio:latest

latest: Pulling from minio/minio
a591faa84ab0: Pull complete
76b9354adec6: Pull complete
f9d8746550a4: Pull complete
890b1dd95baa: Pull complete
3a8518c890dc: Pull complete
8053f0501aed: Pull complete
506c41cb8532: Pull complete
Digest: sha256:e7a725edb521dd2af07879dad88ee1dfebd359e57ad8d98104359ccfbdb92024
Status: Downloaded newer image for minio/minio:latest
docker.io/minio/minio:latest

持久化数据目录

mkdir -p /opt/jumpserver/minio/data /opt/jumpserver/minio/config

启动 MinIO

## 请自行修改账号密码并牢记, 丢失后可以删掉容器后重新用新密码创建, 数据不会丢失
# 9000                                  # 访问端口
# MINIO_ROOT_USER=minio                 # minip 账号
# MINIO_ROOT_PASSWORD=KXOeyNgDeTdpeu9q  # minio 密码

docker run --name jms_minio -d -p 9000:9000 -e MINIO_ROOT_USER=minio -e MINIO_ROOT_PASSWORD=KXOeyNgDeTdpeu9q -v /opt/jumpserver/minio/data:/data -v /opt/jumpserver/minio/config:/root/.minio --restart=always minio/minio:latest server /data

设置 MinIO

访问 http://192.168.100.41:9000, 输入刚才设置的 MinIO 账号密码登录
点击右下角的 + 号, 选择 Create bucket 创建桶, Bucket Name 输入 jumpserver 回车确认

设置 JumpServer

访问 JumpServer Web 页面并使用管理员账号进行登录
点击左侧菜单栏的 [终端管理], 在页面的上方选择 [存储配置], 在 [录像存储] 下方选择 [创建] 选择 [Ceph]
根据下方的说明进行填写, 保存后在 [终端管理] 页面对所有组件进行 [更新], 录像存储选择 [jms-mino], 提交

选项	参考值	说明
名称 (Name)	jms-minio	标识, 不可重复
类型 (Type)	Ceph	固定, 不可更改
桶名称 (Bucket)	jumpserver	Bucket Name
Access key	minio	MINIO_ROOT_USER
Secret key	KXOeyNgDeTdpeu9q	MINIO_ROOT_PASSWORD
端点 (Endpoint)	http://192.168.184.41:9000	minio 服务访问地址