2.4.1 基于Kubernetes原生服务发现

Apollo 1.7.0版本增加了基于Kubernetes原生服务发现的部署模式，由于不再使用内置的Eureka，所以在整体部署上有很大简化，同时也提供了Helm Charts，便于部署。

更多设计说明可以参考#3054。

2.4.1.1 环境要求

  • Kubernetes 1.10+
  • Helm 3

2.4.1.2 添加Apollo Helm Chart仓库

  1. $ helm repo add apollo http://ctripcorp.github.io/apollo/charts
  2. $ helm search repo apollo

2.4.1.3 部署apollo-configservice和apollo-adminservice

2.4.1.3.1 安装apollo-configservice和apollo-adminservice

需要在每个环境中安装apollo-configservice和apollo-adminservice，所以建议在release名称中加入环境信息，例如：apollo-service-dev

  1. $ helm install apollo-service-dev \
  2.     --set configdb.host=1.2.3.4 \
  3.     --set configdb.userName=apollo \
  4.     --set configdb.password=apollo \
  5.     --set configdb.service.enabled=true \
  6.     --set configService.replicaCount=1 \
  7.     --set adminService.replicaCount=1 \
  8.     -n your-namespace \
  9.     apollo/apollo-service

一般部署建议通过 values.yaml 来配置：

  1. $ helm install apollo-service-dev -f values.yaml -n your-namespace apollo/apollo-service

安装完成后会提示对应环境的Meta Server地址，需要记录下来，apollo-portal安装时需要用到：

  1. Get meta service url for current release by running these commands:
  2.   echo http://apollo-service-dev-apollo-configservice:8080

更多配置项说明可以参考2.4.1.3.3 配置项说明

2.4.1.3.2 卸载apollo-configservice和apollo-adminservice

例如要卸载apollo-service-dev的部署：

  1. $ helm uninstall -n your-namespace apollo-service-dev
2.4.1.3.3 配置项说明

下表列出了apollo-service chart的可配置参数及其默认值：

ParameterDescriptionDefault
configdb.hostThe host for apollo config dbnil
configdb.portThe port for apollo config db3306
configdb.dbNameThe database name for apollo config dbApolloConfigDB
configdb.userNameThe user name for apollo config dbnil
configdb.passwordThe password for apollo config dbnil
configdb.connectionStringPropertiesThe connection string properties for apollo config dbcharacterEncoding=utf8
configdb.service.enabledWhether to create a Kubernetes Service for configdb.host or not. Set it to true if configdb.host is an endpoint outside of the kubernetes clusterfalse
configdb.service.fullNameOverrideOverride the service name for apollo config dbnil
configdb.service.portThe port for the service of apollo config db3306
configdb.service.typeThe service type of apollo config db: ClusterIP or ExternalName. If the host is a DNS name, please specify ExternalName as the service type, e.g. xxx.mysql.rds.aliyuncs.comClusterIP
configService.fullNameOverrideOverride the deployment name for apollo-configservicenil
configService.replicaCountReplica count of apollo-configservice2
configService.containerPortContainer port of apollo-configservice8080
configService.image.repositoryImage repository of apollo-configserviceapolloconfig/apollo-configservice
configService.image.tagImage tag of apollo-configservice, e.g. 1.8.0, leave it to nil to use the default version. (chart version >= 0.2.0)nil
configService.image.pullPolicyImage pull policy of apollo-configserviceIfNotPresent
configService.imagePullSecretsImage pull secrets of apollo-configservice[]
configService.service.fullNameOverrideOverride the service name for apollo-configservicenil
configService.service.portThe port for the service of apollo-configservice8080
configService.service.targetPortThe target port for the service of apollo-configservice8080
configService.service.typeThe service type of apollo-configserviceClusterIP
configService.ingress.enabledWhether to enable the ingress for config-service or not. (chart version >= 0.2.0)false
configService.ingress.annotationsThe annotations of the ingress for config-service. (chart version >= 0.2.0){}
configService.ingress.hosts.hostThe host of the ingress for config-service. (chart version >= 0.2.0)nil
configService.ingress.hosts.pathsThe paths of the ingress for config-service. (chart version >= 0.2.0)[]
configService.ingress.tlsThe tls definition of the ingress for config-service. (chart version >= 0.2.0)[]
configService.liveness.initialDelaySecondsThe initial delay seconds of liveness probe100
configService.liveness.periodSecondsThe period seconds of liveness probe10
configService.readiness.initialDelaySecondsThe initial delay seconds of readiness probe30
configService.readiness.periodSecondsThe period seconds of readiness probe5
configService.config.profilesspecify the spring profiles to activategithub,kubernetes
configService.config.configServiceUrlOverrideOverride apollo.config-service.url: config service url to be accessed by apollo-clientnil
configService.config.adminServiceUrlOverrideOverride apollo.admin-service.url: admin service url to be accessed by apollo-portalnil
configService.config.contextPathspecify the context path, e.g. /apollo, then users could access config service via http://{config_service_address}/apollo. (chart version >= 0.2.0)nil
configService.envEnvironment variables passed to the container, e.g.
JAVA_OPTS: -Xss256k		{}
configService.strategyThe deployment strategy of apollo-configservice{}
configService.resourcesThe resources definition of apollo-configservice{}
configService.nodeSelectorThe node selector definition of apollo-configservice{}
configService.tolerationsThe tolerations definition of apollo-configservice[]
configService.affinityThe affinity definition of apollo-configservice{}
adminService.fullNameOverrideOverride the deployment name for apollo-adminservicenil
adminService.replicaCountReplica count of apollo-adminservice2
adminService.containerPortContainer port of apollo-adminservice8090
adminService.image.repositoryImage repository of apollo-adminserviceapolloconfig/apollo-adminservice
adminService.image.tagImage tag of apollo-adminservice, e.g. 1.8.0, leave it to nil to use the default version. (chart version >= 0.2.0)nil
adminService.image.pullPolicyImage pull policy of apollo-adminserviceIfNotPresent
adminService.imagePullSecretsImage pull secrets of apollo-adminservice[]
adminService.service.fullNameOverrideOverride the service name for apollo-adminservicenil
adminService.service.portThe port for the service of apollo-adminservice8090
adminService.service.targetPortThe target port for the service of apollo-adminservice8090
adminService.service.typeThe service type of apollo-adminserviceClusterIP
adminService.ingress.enabledWhether to enable the ingress for admin-service or not. (chart version >= 0.2.0)false
adminService.ingress.annotationsThe annotations of the ingress for admin-service. (chart version >= 0.2.0){}
adminService.ingress.hosts.hostThe host of the ingress for admin-service. (chart version >= 0.2.0)nil
adminService.ingress.hosts.pathsThe paths of the ingress for admin-service. (chart version >= 0.2.0)[]
adminService.ingress.tlsThe tls definition of the ingress for admin-service. (chart version >= 0.2.0)[]
adminService.liveness.initialDelaySecondsThe initial delay seconds of liveness probe100
adminService.liveness.periodSecondsThe period seconds of liveness probe10
adminService.readiness.initialDelaySecondsThe initial delay seconds of readiness probe30
adminService.readiness.periodSecondsThe period seconds of readiness probe5
adminService.config.profilesspecify the spring profiles to activategithub,kubernetes
adminService.config.contextPathspecify the context path, e.g. /apollo, then users could access admin service via http://{admin_service_address}/apollo. (chart version >= 0.2.0)nil
adminService.envEnvironment variables passed to the container, e.g.
JAVA_OPTS: -Xss256k		{}
adminService.strategyThe deployment strategy of apollo-adminservice{}
adminService.resourcesThe resources definition of apollo-adminservice{}
adminService.nodeSelectorThe node selector definition of apollo-adminservice{}
adminService.tolerationsThe tolerations definition of apollo-adminservice[]
adminService.affinityThe affinity definition of apollo-adminservice{}
2.4.1.3.4 配置样例
2.4.1.3.4.1 ConfigDB的host是k8s集群外的IP
  1. configdb:
  2.   host: 1.2.3.4
  3.   dbName: ApolloConfigDBName
  4.   userName: someUserName
  5.   password: somePassword
  6.   connectionStringProperties: characterEncoding=utf8&useSSL=false
  7.   service:
  8.     enabled: true
2.4.1.3.4.2 ConfigDB的host是k8s集群外的域名
  1. configdb:
  2.   host: xxx.mysql.rds.aliyuncs.com
  3.   dbName: ApolloConfigDBName
  4.   userName: someUserName
  5.   password: somePassword
  6.   connectionStringProperties: characterEncoding=utf8&useSSL=false
  7.   service:
  8.     enabled: true
  9.     type: ExternalName
2.4.1.3.4.3 ConfigDB的host是k8s集群内的一个服务
  1. configdb:
  2.   host: apollodb-mysql.mysql
  3.   dbName: ApolloConfigDBName
  4.   userName: someUserName
  5.   password: somePassword
  6.   connectionStringProperties: characterEncoding=utf8&useSSL=false
2.4.1.3.4.4 指定Meta Server返回的apollo-configservice地址

如果apollo-client无法直接访问apollo-configservice的Service（比如不在同一个k8s集群），那么可以参照下面的示例指定Meta Server返回给apollo-client的地址（比如可以通过nodeport访问）

  1. configService:
  2.   config:
  3.     configServiceUrlOverride: http://1.2.3.4:12345
2.4.1.3.4.5 指定Meta Server返回的apollo-adminservice地址

如果apollo-portal无法直接访问apollo-adminservice的Service（比如不在同一个k8s集群），那么可以参照下面的示例指定Meta Server返回给apollo-portal的地址（比如可以通过nodeport访问）

  1. configService:
  2.   config:
  3.     adminServiceUrlOverride: http://1.2.3.4:23456
2.4.1.3.4.6 以Ingress配置自定义路径/config形式暴露apollo-configservice服务
  1. # use /config as root, should specify configService.config.contextPath as /config
  2. configService:
  3.   config:
  4.     contextPath: /config
  5.   ingress:
  6.     enabled: true
  7.     hosts:
  8.       - paths:
  9.           - /config
2.4.1.3.4.7 以Ingress配置自定义路径/admin形式暴露apollo-adminservice服务
  1. # use /admin as root, should specify adminService.config.contextPath as /admin
  2. adminService:
  3.   config:
  4.     contextPath: /admin
  5.   ingress:
  6.     enabled: true
  7.     hosts:
  8.       - paths:
  9.           - /admin

2.4.1.4 部署apollo-portal

2.4.1.4.1 安装apollo-portal

假设有dev, pro两个环境，且meta server地址分别为http://apollo-service-dev-apollo-configservice:8080和http://apollo-service-pro-apollo-configservice:8080：

  1. $ helm install apollo-portal \
  2.     --set portaldb.host=1.2.3.4 \
  3.     --set portaldb.userName=apollo \
  4.     --set portaldb.password=apollo \
  5.     --set portaldb.service.enabled=true \
  6.     --set config.envs="dev\,pro" \
  7.     --set config.metaServers.dev=http://apollo-service-dev-apollo-configservice:8080 \
  8.     --set config.metaServers.pro=http://apollo-service-pro-apollo-configservice:8080 \
  9.     --set replicaCount=1 \
  10.     -n your-namespace \
  11.     apollo/apollo-portal

一般部署建议通过 values.yaml 来配置：

  1. $ helm install apollo-portal -f values.yaml -n your-namespace apollo/apollo-portal

更多配置项说明可以参考2.4.1.4.3 配置项说明

2.4.1.4.2 卸载apollo-portal

例如要卸载apollo-portal的部署：

  1. $ helm uninstall -n your-namespace apollo-portal
2.4.1.4.3 配置项说明

下表列出了apollo-portal chart的可配置参数及其默认值：

ParameterDescriptionDefault
fullNameOverrideOverride the deployment name for apollo-portalnil
replicaCountReplica count of apollo-portal2
containerPortContainer port of apollo-portal8070
image.repositoryImage repository of apollo-portalapolloconfig/apollo-portal
image.tagImage tag of apollo-portal, e.g. 1.8.0, leave it to nil to use the default version. (chart version >= 0.2.0)nil
image.pullPolicyImage pull policy of apollo-portalIfNotPresent
imagePullSecretsImage pull secrets of apollo-portal[]
service.fullNameOverrideOverride the service name for apollo-portalnil
service.portThe port for the service of apollo-portal8070
service.targetPortThe target port for the service of apollo-portal8070
service.typeThe service type of apollo-portalClusterIP
service.sessionAffinityThe session affinity for the service of apollo-portalClientIP
ingress.enabledWhether to enable the ingress or notfalse
ingress.annotationsThe annotations of the ingress{}
ingress.hosts.hostThe host of the ingressnil
ingress.hosts.pathsThe paths of the ingress[]
ingress.tlsThe tls definition of the ingress[]
liveness.initialDelaySecondsThe initial delay seconds of liveness probe100
liveness.periodSecondsThe period seconds of liveness probe10
readiness.initialDelaySecondsThe initial delay seconds of readiness probe30
readiness.periodSecondsThe period seconds of readiness probe5
envEnvironment variables passed to the container, e.g.
JAVA_OPTS: -Xss256k		{}
strategyThe deployment strategy of apollo-portal{}
resourcesThe resources definition of apollo-portal{}
nodeSelectorThe node selector definition of apollo-portal{}
tolerationsThe tolerations definition of apollo-portal[]
affinityThe affinity definition of apollo-portal{}
config.profilesspecify the spring profiles to activategithub,auth
config.envsspecify the env names, e.g. dev,pronil
config.contextPathspecify the context path, e.g. /apollo, then users could access portal via http://{portal_address}/apollonil
config.metaServersspecify the meta servers, e.g.
dev: http://apollo-configservice-dev:8080
pro: http://apollo-configservice-pro:8080		{}
config.filesspecify the extra config files for apollo-portal, e.g. application-ldap.yml{}
portaldb.hostThe host for apollo portal dbnil
portaldb.portThe port for apollo portal db3306
portaldb.dbNameThe database name for apollo portal dbApolloPortalDB
portaldb.userNameThe user name for apollo portal dbnil
portaldb.passwordThe password for apollo portal dbnil
portaldb.connectionStringPropertiesThe connection string properties for apollo portal dbcharacterEncoding=utf8
portaldb.service.enabledWhether to create a Kubernetes Service for portaldb.host or not. Set it to true if portaldb.host is an endpoint outside of the kubernetes clusterfalse
portaldb.service.fullNameOverrideOverride the service name for apollo portal dbnil
portaldb.service.portThe port for the service of apollo portal db3306
portaldb.service.typeThe service type of apollo portal db: ClusterIP or ExternalName. If the host is a DNS name, please specify ExternalName as the service type, e.g. xxx.mysql.rds.aliyuncs.comClusterIP
2.4.1.4.4 配置样例
2.4.1.4.4.1 PortalDB的host是k8s集群外的IP
  1. portaldb:
  2.   host: 1.2.3.4
  3.   dbName: ApolloPortalDBName
  4.   userName: someUserName
  5.   password: somePassword
  6.   connectionStringProperties: characterEncoding=utf8&useSSL=false
  7.   service:
  8.     enabled: true
2.4.1.4.4.2 PortalDB的host是k8s集群外的域名
  1. portaldb:
  2.   host: xxx.mysql.rds.aliyuncs.com
  3.   dbName: ApolloPortalDBName
  4.   userName: someUserName
  5.   password: somePassword
  6.   connectionStringProperties: characterEncoding=utf8&useSSL=false
  7.   service:
  8.     enabled: true
  9.     type: ExternalName
2.4.1.4.4.3 PortalDB的host是k8s集群内的一个服务
  1. portaldb:
  2.   host: apollodb-mysql.mysql
  3.   dbName: ApolloPortalDBName
  4.   userName: someUserName
  5.   password: somePassword
  6.   connectionStringProperties: characterEncoding=utf8&useSSL=false
2.4.1.4.4.4 配置环境信息
  1. config:
  2.   envs: dev,pro
  3.   metaServers:
  4.     dev: http://apollo-service-dev-apollo-configservice:8080
  5.     pro: http://apollo-service-pro-apollo-configservice:8080
2.4.1.4.4.5 以Load Balancer形式暴露服务
  1. service:
  2.   type: LoadBalancer
2.4.1.4.4.6 以Ingress形式暴露服务
  1. ingress:
  2.   enabled: true
  3.   hosts:
  4.     - paths:
  5.         - /
2.4.1.4.4.7 以Ingress配置自定义路径/apollo形式暴露服务
  1. # use /apollo as root, should specify config.contextPath as /apollo
  2. ingress:
  3.   enabled: true
  4.   hosts:
  5.     - paths:
  6.         - /apollo
  8. config:
  9.   ...
  10.   contextPath: /apollo
  11.   ...
2.4.1.4.4.8 以Ingress配置session affinity形式暴露服务
  1. ingress:
  2.   enabled: true
  3.   annotations:
  4.     kubernetes.io/ingress.class: nginx
  5.     nginx.ingress.kubernetes.io/affinity: "cookie"
  6.     nginx.ingress.kubernetes.io/affinity-mode: "persistent"
  7.     nginx.ingress.kubernetes.io/session-cookie-conditional-samesite-none: "true"
  8.     nginx.ingress.kubernetes.io/session-cookie-expires: "172800"
  9.     nginx.ingress.kubernetes.io/session-cookie-max-age: "172800"
  10.   hosts:
  11.     - host: xxx.somedomain.com # host is required to make session affinity work
  12.       paths:
  13.         - /
2.4.1.4.4.9 启用 LDAP 支持
  1. config:
  2.   ...
  3.   profiles: github,ldap
  4.   ...
  5.   files:
  6.     application-ldap.yml: |
  7.       spring:
  8.         ldap:
  9.           base: "dc=example,dc=org"
  10.           username: "cn=admin,dc=example,dc=org"
  11.           password: "password"
  12.           searchFilter: "(uid={0})"
  13.           urls:
  14.           - "ldap://xxx.somedomain.com:389"
  15.       ldap:
  16.         mapping:
  17.           objectClass: "inetOrgPerson"
  18.           loginId: "uid"
  19.           userDisplayName: "cn"
  20.           email: "mail"