Kubernetes Deployment

This article describes how to deploy a Linkis service in a Kind Kubernetes environment for easier learning and debugging.

kind github：https://github.com/kubernetes-sigs/kind
kind web site：kind.sigs.k8s.io/

  • kind 0.14.0
  • docker 20.10.17
  • centos 7.6

  • helm 3.x

    1. Ensure that the component depends on the version
    1. kind means that the machine simulating node with docker container is restarted and the container has changed and the scheduler is not working anymore. This is a kind limitation and there is a detailed explanation in the official document.
  1. sudo yum install -y yum-utils device-mapper-persistent-data lvm2
  2. sudo yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
  3. sudo sed -i 's+download.docker.com+mirrors.aliyun.com/docker-ce+' /etc/yum.repos.d/docker-ce.repo
  4. sudo yum makecache fast
  5. sudo yum -y install docker-ce
  6. systemctl start docker
  7. systemctl enable docker
  1. vi /etc/docker/daemon.json
  2. {
  3. "registry-mirrors": ["http://hub-mirror.c.163.com"],
  4. "insecure-registries": ["https://registry.mydomain.com","http://hub-mirror.c.163.com"]
  5. }
  1. https://github.com/kubernetes-sigs/kind/releases
  1. chmod +x ./kind
  2. mv kind-linux-amd64 /usr/bin/kind
  1. apache-linkis-1.3.0-incubating-bin.tar.gz
  1. mkdir -p /opt/data/common/extendlib
  1. curl https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.28/mysql-connector-java-8.0.28.jar -o /opt/data/common/extendlib/[mysql-connector-java-8.0.28.jar](https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.28/mysql-connector-java-8.0.28.jar)
  1. ./bin/install-linkis-to-kubernetes.sh reset
  1. ./bin/install-linkis-to-kubernetes.sh pull -mghcr.dockerproxy.com
  1. ./bin/install-linkis-to-kubernetes.sh install -l -mghcr.dockerproxy.com
  1. kubectl get pods -A

Kubernetes Deployment - 图1

  1. ./helm/scripts/prepare-for-spark.sh
  1. ./helm/scripts/remote-proxy.sh start
  1. linkis-web: http://10.0.2.101:8088/
  2. eureka: http://10.0.2.101:20303/
  1. ./helm/scripts/login-pod.sh cg-engineconnmanager
  2. Executing shell tests
  3. sh ./bin/linkis-cli -engineType shell-1 -codeType shell -code "echo "hello" "  -submitUser hadoop -proxyUser hadoop
  4. Executing hive Tests
  5. sh ./bin/linkis-cli -engineType hive-2.3.3 -codeType hql -code "show tables"  -submitUser hadoop -proxyUser hadoop
  6. Executing spark Tests
  7. sh ./bin/linkis-cli -engineType spark-2.4.3 -codeType sql -code "show tables"  -submitUser hadoop -proxyUser hadoop
  8. Executing python Tests
  9. sh ./bin/linkis-cli -engineType python-python2 -codeType python -code "print(\"hello\")"  -submitUser hadoop -proxyUser hadoop  -confMap  python.version=python

Kubernetes Deployment - 图2