Sharding-Scaling(Alpha)

快速开始

部署启动

1. 执行以下命令，编译生成sharding-scaling二进制包：

  2. git clone https://github.com/apache/shardingsphere.git；
  3. cd shardingsphere;
  4. mvn clean install -Prelease;

发布包所在目录为：/sharding-distribution/sharding-scaling-distribution/target/apache-shardingsphere-${latest.release.version}-sharding-scaling-bin.tar.gz。

2. 解压缩发布包，修改配置文件conf/server.yaml，这里主要修改启动端口，保证不与本机其他端口冲突，其他值保持默认即可：

  1. port: 8888
  2. blockQueueSize: 10000
  3. pushTimeout: 1000
  4. workerThread: 30

3. 启动sharding-scaling：

  1. sh bin/start.sh

注意： 如果后端连接MySQL数据库，需要下载MySQL Connector/J， 解压缩后，将mysql-connector-java-5.1.47.jar拷贝到${sharding-scaling}\lib目录。

4. 查看日志logs/stdout.log，确保启动成功。

创建迁移任务

Sharding-Scaling提供相应的HTTP接口来管理迁移任务，部署启动成功后，我们可以调用相应的接口来启动迁移任务。

创建迁移任务：

  1. curl -X POST \
  2.   http://localhost:8888/shardingscaling/job/start \
  3.   -H 'content-type: application/json' \
  4.   -d '{
  5.    "ruleConfiguration": {
  6.       "sourceDatasource": "ds_0: !!org.apache.shardingsphere.orchestration.core.configuration.YamlDataSourceConfiguration\n  dataSourceClassName: com.zaxxer.hikari.HikariDataSource\n  properties:\n    jdbcUrl: jdbc:mysql://127.0.0.1:3306/test?serverTimezone=UTC&useSSL=false\n    username: root\n    password: '\''123456'\''\n    connectionTimeout: 30000\n    idleTimeout: 60000\n    maxLifetime: 1800000\n    maxPoolSize: 50\n    minPoolSize: 1\n    maintenanceIntervalMilliseconds: 30000\n    readOnly: false\n",
  7.       "sourceRule": "defaultDatabaseStrategy:\n  inline:\n    algorithmExpression: ds_${user_id % 2}\n    shardingColumn: user_id\ntables:\n  t1:\n    actualDataNodes: ds_0.t1\n    keyGenerator:\n      column: order_id\n      type: SNOWFLAKE\n    logicTable: t1\n    tableStrategy:\n      inline:\n        algorithmExpression: t1\n        shardingColumn: order_id\n  t2:\n    actualDataNodes: ds_0.t2\n    keyGenerator:\n      column: order_item_id\n      type: SNOWFLAKE\n    logicTable: t2\n    tableStrategy:\n      inline:\n        algorithmExpression: t2\n        shardingColumn: order_id\n",
  8.       "destinationDataSources": {
  9.          "name": "dt_0",
  10.          "password": "123456",
  11.          "url": "jdbc:mysql://127.0.0.1:3306/test2?serverTimezone=UTC&useSSL=false",
  12.          "username": "root"
  13.       }
  14.    },
  15.    "jobConfiguration": {
  16.       "concurrency": 3
  17.    }
  18. }'

注意：上述需要修改ruleConfiguration.sourceDatasource和ruleConfiguration.sourceRule，分别为源端ShardingSphere数据源和数据表规则相关配置；

以及ruleConfiguration.destinationDataSources中目标端sharding-proxy的相关信息。

返回如下信息，表示任务创建成功：

  1. {
  2.    "success": true,
  3.    "errorCode": 0,
  4.    "errorMsg": null,
  5.    "model": null
  6. }

需要注意的是，目前Sharding-Scaling任务创建成功后，便会自动运行，进行数据的迁移。

查询任务进度

执行如下命令获取当前所有迁移任务：

  1. curl -X GET \
  2.   http://localhost:8888/shardingscaling/job/list

返回示例如下：

  1. {
  2.   "success": true,
  3.   "errorCode": 0,
  4.   "model": [
  5.     {
  6.       "jobId": 1,
  7.       "jobName": "Local Sharding Scaling Job",
  8.       "status": "RUNNING"
  9.     }
  10.   ]
  11. }

进一步查询任务具体迁移状态：

  1. curl -X GET \
  2.   http://localhost:8888/shardingscaling/job/progress/1

返回任务详细信息如下：

  1. {
  2.    "success": true,
  3.    "errorCode": 0,
  4.    "errorMsg": null,
  5.    "model": {
  6.         "id": 1,
  7.         "jobName": "Local Sharding Scaling Job",
  8.         "status": "RUNNING"
  9.         "syncTaskProgress": [{
  10.             "id": "127.0.0.1-3306-test",
  11.             "status": "SYNCHRONIZE_REALTIME_DATA",
  12.             "historySyncTaskProgress": [{
  13.                 "id": "history-test-t1#0",
  14.                 "estimatedRows": 41147,
  15.                 "syncedRows": 41147
  16.             }, {
  17.                 "id": "history-test-t1#1",
  18.                 "estimatedRows": 42917,
  19.                 "syncedRows": 42917
  20.             }, {
  21.                 "id": "history-test-t1#2",
  22.                 "estimatedRows": 43543,
  23.                 "syncedRows": 43543
  24.             }, {
  25.                 "id": "history-test-t2#0",
  26.                 "estimatedRows": 39679,
  27.                 "syncedRows": 39679
  28.             }, {
  29.                 "id": "history-test-t2#1",
  30.                 "estimatedRows": 41483,
  31.                 "syncedRows": 41483
  32.             }, {
  33.                 "id": "history-test-t2#2",
  34.                 "estimatedRows": 42107,
  35.                 "syncedRows": 42107
  36.             }],
  37.             "realTimeSyncTaskProgress": {
  38.                 "id": "realtime-test",
  39.                 "delayMillisecond": 1576563771372,
  40.                 "logPosition": {
  41.                     "filename": "ON.000007",
  42.                     "position": 177532875,
  43.                     "serverId": 0
  44.                 }
  45.             }
  46.         }]
  47.    }
  48. }

结束任务

数据迁移完成后，我们可以调用接口结束任务：

  1. curl -X POST \
  2.   http://localhost:8888/shardingscaling/job/stop \
  3.   -H 'content-type: application/json' \
  4.   -d '{
  5.    "jobId":1
  6. }'

返回如下信息表示任务成功结束：

  1. {
  2.    "success": true,
  3.    "errorCode": 0,
  4.    "errorMsg": null,
  5.    "model": null
  6. }

结束Sharding-Scaling

  1. sh bin/stop.sh