微服务租户隔离架构设计

1. 总述

需求背景

  1. 现在LinkisGateway进行服务转发时现在是基于ribbon进行负载均衡的,但是有些情况下存在一些重要业务的任务希望做到服务级别的隔离,如果基于ribbon进行服务在均衡就会存在问题。比如租户A希望他的任务都路由到特定的Linkis-CG-Entrance服务,这样当其他的实例出现异常时可以不会影响到A服务的Entrance 另外支持服务的租户及隔离也可以做到快速隔离某个异常服务,支持灰度升级等场景。

目标

  1. 支持通过解析请求的标签按照路由标签对服务进行转发
  2. 支持服务的标签注册和修改

2. 总体设计

  1. 此次特性新增主要修改点位linkis-mg-gatewayinstance-label两个模块,设计到新增Gateway的转发逻辑,以及instance-label支持服务和标签的注册。

2.1 技术架构

  1. 整体技术架构主要修改点位RestFul请求需要带上路由标签等标签参数信息,然后在Gateway进行转发时会解析对应的标签完成接口的路由转发。整体如下图所示 ![arc](/projects/Linkis-1.2.0-zh/e544d8a47121a9151a4eb92ddf22b7d3.png)

几点说明:

  1. 如果存在多个对应的服务打上了同一个roteLabel则随机转发
  2. 如果对应的routeLabel没有对应的服务,则接口直接失败
  3. 如果接口没有routeLabel则基于原有的转发逻辑,不会路由到打上了特定标签的服务

2.2 业务架构

  1. 此次的特性主要是为了完成Restful租户隔离转发功能。功能点设计的模块如下:
组件名一级模块二级模块功能点
LinkisMGGateway解析restful请求参数中的路由标签,完成接口按照路由标签的转发功能
LinkisPSInstanceLabelInstanceLabel服务,完成服务和标签的关联

3. 模块设计

核心执行流程

[输入端] 输入端为请求Gatway的restful请求,且是参数中待用roure label的请求才会进行处理 [处理流程] Gateway会判断请求是否带有对应的RouteLabel,如果存在则基于RouteLabel来进行转发。 调用时序图如下:

Time

4. 数据结构:

  1. DROP TABLE IF EXISTS `linkis_ps_instance_label`;
  2. CREATE TABLE `linkis_ps_instance_label` (
  3. `id` int(20) NOT NULL AUTO_INCREMENT,
  4. `label_key` varchar(32) COLLATE utf8_bin NOT NULL COMMENT 'string key',
  5. `label_value` varchar(255) COLLATE utf8_bin NOT NULL COMMENT 'string value',
  6. `label_feature` varchar(16) COLLATE utf8_bin NOT NULL COMMENT 'store the feature of label, but it may be redundant',
  7. `label_value_size` int(20) NOT NULL COMMENT 'size of key -> value map',
  8. `update_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT 'update unix timestamp',
  9. `create_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT 'update unix timestamp',
  10. PRIMARY KEY (`id`),
  11. UNIQUE KEY `label_key_value` (`label_key`,`label_value`)
  12. ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin;
  13. DROP TABLE IF EXISTS `linkis_ps_instance_info`;
  14. CREATE TABLE `linkis_ps_instance_info` (
  15. `id` int(11) NOT NULL AUTO_INCREMENT,
  16. `instance` varchar(128) COLLATE utf8_bin DEFAULT NULL COMMENT 'structure like ${host|machine}:${port}',
  17. `name` varchar(128) COLLATE utf8_bin DEFAULT NULL COMMENT 'equal application name in registry',
  18. `update_time` datetime DEFAULT CURRENT_TIMESTAMP COMMENT 'update unix timestamp',
  19. `create_time` datetime DEFAULT CURRENT_TIMESTAMP COMMENT 'create unix timestamp',
  20. PRIMARY KEY (`id`),
  21. UNIQUE KEY `instance` (`instance`)
  22. ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin;
  23. DROP TABLE IF EXISTS `linkis_ps_instance_label_relation`;
  24. CREATE TABLE `linkis_ps_instance_label_relation` (
  25. `id` int(20) NOT NULL AUTO_INCREMENT,
  26. `label_id` int(20) DEFAULT NULL COMMENT 'id reference linkis_ps_instance_label -> id',
  27. `service_instance` varchar(128) NOT NULL COLLATE utf8_bin COMMENT 'structure like ${host|machine}:${port}',
  28. `update_time` datetime DEFAULT CURRENT_TIMESTAMP COMMENT 'update unix timestamp',
  29. `create_time` datetime DEFAULT CURRENT_TIMESTAMP COMMENT 'create unix timestamp',
  30. PRIMARY KEY (`id`)
  31. ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin;

5. 如何使用:

add route label for entrance

  1. echo "spring.eureka.instance.metadata-map.route=et1" >> $LINKIS_CONF_DIR/linkis-cg-entrance.properties
  2. sh $LINKIS_HOME/sbin/linkis-damemon.sh restart cg-entrance

Time

Use route label

submit task:

  1. url:/api/v1/entrance/submit
  2. {
  3. "executionContent": {"code": "echo 1", "runType": "shell"},
  4. "params": {"variable": {}, "configuration": {}},
  5. "source": {"scriptPath": "ip"},
  6. "labels": {
  7. "engineType": "shell-1",
  8. "userCreator": "peacewong-IDE",
  9. "route": "et1"
  10. }
  11. }

will be routed to a fixed service:

  1. {
  2. "method": "/api/entrance/submit",
  3. "status": 0,
  4. "message": "OK",
  5. "data": {
  6. "taskID": 45158,
  7. "execID": "exec_id018030linkis-cg-entrancelocalhost:9205IDE_peacewong_shell_0"
  8. }
  9. }

or linkis-cli:

  1. sh bin/linkis-cli -submitUser hadoop -engineType shell-1 -codeType shell -code "whoami" -labelMap route=et1 --gatewayUrl http://127.0.0.1:9101

Use non-existing label

submit task:

  1. url:/api/v1/entrance/submit
  2. {
  3. "executionContent": {"code": "echo 1", "runType": "shell"},
  4. "params": {"variable": {}, "configuration": {}},
  5. "source": {"scriptPath": "ip"},
  6. "labels": {
  7. "engineType": "shell-1",
  8. "userCreator": "peacewong-IDE",
  9. "route": "et1"
  10. }
  11. }

will get the error

  1. {
  2. "method": "/api/rest_j/v1/entrance/submit",
  3. "status": 1,
  4. "message": "GatewayErrorException: errCode: 11011 ,desc: Cannot route to the corresponding service, URL: /api/rest_j/v1/entrance/submit RouteLabel: [{\"stringValue\":\"et2\",\"labelKey\":\"route\",\"feature\":null,\"modifiable\":true,\"featureKey\":\"feature\",\"empty\":false}] ,ip: localhost ,port: 9101 ,serviceKind: linkis-mg-gateway",
  5. "data": {
  6. "data": "{\r\n \"executionContent\": {\"code\": \"echo 1\", \"runType\": \"shell\"},\r\n \"params\": {\"variable\": {}, \"configuration\": {}},\r\n \"source\": {\"scriptPath\": \"ip\"},\r\n \"labels\": {\r\n \"engineType\": \"shell-1\",\r\n \"userCreator\": \"peacewong-IDE\",\r\n \"route\": \"et2\"\r\n }\r\n}"
  7. }
  8. }

without label

submit task:

  1. url:/api/v1/entrance/submit
  2. {
  3. "executionContent": {"code": "echo 1", "runType": "shell"},
  4. "params": {"variable": {}, "configuration": {}},
  5. "source": {"scriptPath": "ip"},
  6. "labels": {
  7. "engineType": "shell-1",
  8. "userCreator": "peacewong-IDE"
  9. }
  10. }
  1. will route to untagged entranceservices
  2. {
  3. "method": "/api/entrance/submit",
  4. "status": 0,
  5. "message": "OK",
  6. "data": {
  7. "taskID": 45159,
  8. "execID": "exec_id018018linkis-cg-entrancelocalhost2:9205IDE_peacewong_shell_0"
  9. }
  10. }

6. 非功能性设计:

6.1 安全

不涉及安全问题,restful需要登录认证

6.2 性能

对Gateway转发性能影响较小,有缓存相应的label和instance的数据

6.3 容量

不涉及

6.4 高可用

不涉及