使用 OpenAPI 运维 TiDB Data Migration 集群

TiDB Data Migration (DM) 提供 OpenAPI 功能,你可以通过 OpenAPI 方便地对 DM 集群进行查询和运维操作。OpenAPI 的功能范围和 dmctl 工具相当。

如需开启 OpenAPI,可通过以下方法:

  • 如果你的 DM 集群是通过二进制直接部署的,则在 DM-master 的配置文件中添加如下配置:

    1. openapi = true
  • 如果你的 DM 集群是通过 TiUP 部署的,则在拓扑文件中添加如下配置:

    1. server_configs:
    2. master:
    3. openapi: true

OpenAPI - 图1

注意

  • DM 提供符合 OpenAPI 3.0.0 标准的 Spec 文档,其中包含了所有 API 的请求参数和返回体,你可自行复制到如 Swagger Editor 等工具中在线预览文档。

  • 部署 DM-master 后,你可访问 http://{master-addr}/api/v1/docs 在线预览文档。

你可以通过 OpenAPI 完成 DM 集群的如下运维操作:

集群相关 API

数据源相关 API

同步任务相关 API

本文档以下部分描述当前提供的 API 的具体使用方法。

API 统一错误格式

对 API 发起的请求后,如发生错误,返回错误信息的格式如下所示:

  1. {
  2. "error_msg": "",
  3. "error_code": ""
  4. }

如上所示,error_msg 描述错误信息,error_code 则是对应的错误码。

获取 DM-master 节点信息

该接口是一个同步接口,请求成功会返回对应节点的状态信息。

请求 URI

GET /api/v1/cluster/masters

使用样例

  1. curl -X 'GET' \
  2. 'http://127.0.0.1:8261/api/v1/cluster/masters' \
  3. -H 'accept: application/json'
  1. {
  2. "total": 1,
  3. "data": [
  4. {
  5. "name": "master1",
  6. "alive": true,
  7. "leader": true,
  8. "addr": "127.0.0.1:8261"
  9. }
  10. ]
  11. }

下线 DM-master 节点

该接口是一个同步接口,请求成功后返回体的 Status Code 是 204。

请求 URI

DELETE /api/v1/cluster/masters/{master-name}

使用样例

  1. curl -X 'DELETE' \
  2. 'http://127.0.0.1:8261/api/v1/cluster/masters/master1' \
  3. -H 'accept: */*'

获取 DM-worker 节点信息

该接口是一个同步接口,请求成功会返回对应节点的状态信息。

请求 URI

GET /api/v1/cluster/workers

使用样例

  1. curl -X 'GET' \
  2. 'http://127.0.0.1:8261/api/v1/cluster/workers' \
  3. -H 'accept: application/json'
  1. {
  2. "total": 1,
  3. "data": [
  4. {
  5. "name": "worker1",
  6. "addr": "127.0.0.1:8261",
  7. "bound_stage": "bound",
  8. "bound_source_name": "mysql-01"
  9. }
  10. ]
  11. }

下线 DM-worker 节点

该接口是一个同步接口,请求成功后返回体的 Status Code 是 204。

请求 URI

DELETE /api/v1/cluster/workers/{worker-name}

使用样例

  1. curl -X 'DELETE' \
  2. 'http://127.0.0.1:8261/api/v1/cluster/workers/worker1' \
  3. -H 'accept: */*'

创建数据源

该接口是一个同步接口,请求成功会返回对应数据源信息。

请求 URI

POST /api/v1/sources

使用样例

  1. curl -X 'POST' \
  2. 'http://127.0.0.1:8261/api/v1/sources' \
  3. -H 'accept: application/json' \
  4. -H 'Content-Type: application/json' \
  5. -d '{
  6. "source_name": "mysql-01",
  7. "host": "127.0.0.1",
  8. "port": 3306,
  9. "user": "root",
  10. "password": "123456",
  11. "enable": true,
  12. "enable_gtid": false,
  13. "security": {
  14. "ssl_ca_content": "",
  15. "ssl_cert_content": "",
  16. "ssl_key_content": "",
  17. "cert_allowed_cn": [
  18. "string"
  19. ]
  20. },
  21. "purge": {
  22. "interval": 3600,
  23. "expires": 0,
  24. "remain_space": 15
  25. }
  26. }'
  1. {
  2. "source_name": "mysql-01",
  3. "host": "127.0.0.1",
  4. "port": 3306,
  5. "user": "root",
  6. "password": "123456",
  7. "enable": true,
  8. "enable_gtid": false,
  9. "security": {
  10. "ssl_ca_content": "",
  11. "ssl_cert_content": "",
  12. "ssl_key_content": "",
  13. "cert_allowed_cn": [
  14. "string"
  15. ]
  16. },
  17. "purge": {
  18. "interval": 3600,
  19. "expires": 0,
  20. "remain_space": 15
  21. },
  22. "status_list": [
  23. {
  24. "source_name": "mysql-replica-01",
  25. "worker_name": "worker-1",
  26. "relay_status": {
  27. "master_binlog": "(mysql-bin.000001, 1979)",
  28. "master_binlog_gtid": "e9a1fc22-ec08-11e9-b2ac-0242ac110003:1-7849",
  29. "relay_dir": "./sub_dir",
  30. "relay_binlog_gtid": "e9a1fc22-ec08-11e9-b2ac-0242ac110003:1-7849",
  31. "relay_catch_up_master": true,
  32. "stage": "Running"
  33. },
  34. "error_msg": "string"
  35. }
  36. ]
  37. }

获取数据源

该接口是一个同步接口,请求成功会返回数据源列表信息。

请求 URI

GET /api/v1/sources/{source-name}

使用样例

  1. curl -X 'GET' \
  2. 'http://127.0.0.1:8261/api/v1/sources/source-1?with_status=true' \
  3. -H 'accept: application/json'
  1. {
  2. "source_name": "mysql-01",
  3. "host": "127.0.0.1",
  4. "port": 3306,
  5. "user": "root",
  6. "password": "123456",
  7. "enable_gtid": false,
  8. "enable": false,
  9. "flavor": "mysql",
  10. "task_name_list": [
  11. "task1"
  12. ],
  13. "security": {
  14. "ssl_ca_content": "",
  15. "ssl_cert_content": "",
  16. "ssl_key_content": "",
  17. "cert_allowed_cn": [
  18. "string"
  19. ]
  20. },
  21. "purge": {
  22. "interval": 3600,
  23. "expires": 0,
  24. "remain_space": 15
  25. },
  26. "status_list": [
  27. {
  28. "source_name": "mysql-replica-01",
  29. "worker_name": "worker-1",
  30. "relay_status": {
  31. "master_binlog": "(mysql-bin.000001, 1979)",
  32. "master_binlog_gtid": "e9a1fc22-ec08-11e9-b2ac-0242ac110003:1-7849",
  33. "relay_dir": "./sub_dir",
  34. "relay_binlog_gtid": "e9a1fc22-ec08-11e9-b2ac-0242ac110003:1-7849",
  35. "relay_catch_up_master": true,
  36. "stage": "Running"
  37. },
  38. "error_msg": "string"
  39. }
  40. ],
  41. "relay_config": {
  42. "enable_relay": true,
  43. "relay_binlog_name": "mysql-bin.000002",
  44. "relay_binlog_gtid": "e9a1fc22-ec08-11e9-b2ac-0242ac110003:1-7849",
  45. "relay_dir": "./relay_log"
  46. }
  47. }

删除数据源

该接口是一个同步接口,请求成功后返回的 Status Code 是 204。

请求 URI

DELETE /api/v1/sources/{source-name}

使用样例

  1. curl -X 'DELETE' \
  2. 'http://127.0.0.1:8261/api/v1/sources/mysql-01?force=true' \
  3. -H 'accept: application/json'

更新数据源

该接口是一个同步接口,请求成功会返回对应的数据源信息。

OpenAPI - 图2

注意

更新数据源配置时,须确保当前数据源下没有任何正在运行的同步任务。

请求 URI

PUT /api/v1/sources/{source-name}

使用样例

  1. curl -X 'PUT' \
  2. 'http://127.0.0.1:8261/api/v1/sources/mysql-01' \
  3. -H 'accept: application/json' \
  4. -H 'Content-Type: application/json' \
  5. -d '{
  6. "source": {
  7. "source_name": "mysql-01",
  8. "host": "127.0.0.1",
  9. "port": 3306,
  10. "user": "root",
  11. "password": "123456",
  12. "enable_gtid": false,
  13. "enable": false,
  14. "flavor": "mysql",
  15. "task_name_list": [
  16. "task1"
  17. ],
  18. "security": {
  19. "ssl_ca_content": "",
  20. "ssl_cert_content": "",
  21. "ssl_key_content": "",
  22. "cert_allowed_cn": [
  23. "string"
  24. ]
  25. },
  26. "purge": {
  27. "interval": 3600,
  28. "expires": 0,
  29. "remain_space": 15
  30. },
  31. "relay_config": {
  32. "enable_relay": true,
  33. "relay_binlog_name": "mysql-bin.000002",
  34. "relay_binlog_gtid": "e9a1fc22-ec08-11e9-b2ac-0242ac110003:1-7849",
  35. "relay_dir": "./relay_log"
  36. }
  37. }
  38. }'
  1. {
  2. "source_name": "mysql-01",
  3. "host": "127.0.0.1",
  4. "port": 3306,
  5. "user": "root",
  6. "password": "123456",
  7. "enable": true,
  8. "enable_gtid": false,
  9. "security": {
  10. "ssl_ca_content": "",
  11. "ssl_cert_content": "",
  12. "ssl_key_content": "",
  13. "cert_allowed_cn": [
  14. "string"
  15. ]
  16. },
  17. "purge": {
  18. "interval": 3600,
  19. "expires": 0,
  20. "remain_space": 15
  21. }
  22. }

启用数据源

这是一个同步接口,请求成功后会启用此数据源,并批量开始数据迁移任务中依赖该数据源的所有子任务。

请求 URI

POST /api/v1/sources/{source-name}/enable

使用样例

  1. curl -X 'POST' \
  2. 'http://127.0.0.1:8261/api/v1/sources/mysql-01/enable' \
  3. -H 'accept: */*' \
  4. -H 'Content-Type: application/json'

停用数据源

这是一个同步接口,请求成功后会停用此数据源,并批量停止数据迁移任务中依赖该数据源的所有子任务。

请求 URI

POST /api/v1/sources/{source-name}/disable

使用样例

  1. curl -X 'POST' \
  2. 'http://127.0.0.1:8261/api/v1/sources/mysql-01/disable' \
  3. -H 'accept: */*' \
  4. -H 'Content-Type: application/json'

获取数据源列表

该接口是一个同步接口,请求成功会返回数据源列表信息。

请求 URI

GET /api/v1/sources

使用样例

  1. curl -X 'GET' \
  2. 'http://127.0.0.1:8261/api/v1/sources?with_status=true' \
  3. -H 'accept: application/json'
  1. {
  2. "data": [
  3. {
  4. "enable_gtid": false,
  5. "host": "127.0.0.1",
  6. "password": "******",
  7. "port": 3306,
  8. "purge": {
  9. "expires": 0,
  10. "interval": 3600,
  11. "remain_space": 15
  12. },
  13. "security": null,
  14. "source_name": "mysql-01",
  15. "user": "root"
  16. },
  17. {
  18. "enable_gtid": false,
  19. "host": "127.0.0.1",
  20. "password": "******",
  21. "port": 3307,
  22. "purge": {
  23. "expires": 0,
  24. "interval": 3600,
  25. "remain_space": 15
  26. },
  27. "security": null,
  28. "source_name": "mysql-02",
  29. "user": "root"
  30. }
  31. ],
  32. "total": 2
  33. }

获取数据源状态

该接口是一个同步接口,请求成功会返回对应节点的状态信息。

请求 URI

GET /api/v1/sources/{source-name}/status

使用样例

  1. curl -X 'GET' \
  2. 'http://127.0.0.1:8261/api/v1/sources/mysql-replica-01/status' \
  3. -H 'accept: application/json'
  1. {
  2. "total": 1,
  3. "data": [
  4. {
  5. "source_name": "mysql-replica-01",
  6. "worker_name": "worker-1",
  7. "relay_status": {
  8. "master_binlog": "(mysql-bin.000001, 1979)",
  9. "master_binlog_gtid": "e9a1fc22-ec08-11e9-b2ac-0242ac110003:1-7849",
  10. "relay_dir": "./sub_dir",
  11. "relay_binlog_gtid": "e9a1fc22-ec08-11e9-b2ac-0242ac110003:1-7849",
  12. "relay_catch_up_master": true,
  13. "stage": "Running"
  14. },
  15. "error_msg": "string"
  16. }
  17. ]
  18. }

对数据源开启 relay-log 功能

这是一个异步接口,请求成功的 Status Code 是 200,可通过获取数据源状态接口获取最新的状态。

请求 URI

POST /api/v1/sources/{source-name}/relay/enable

使用样例

  1. curl -X 'POST' \
  2. 'http://127.0.0.1:8261/api/v1/sources/mysql-01/relay/enable' \
  3. -H 'accept: */*' \
  4. -H 'Content-Type: application/json' \
  5. -d '{
  6. "worker_name_list": [
  7. "worker-1"
  8. ],
  9. "relay_binlog_name": "mysql-bin.000002",
  10. "relay_binlog_gtid": "e9a1fc22-ec08-11e9-b2ac-0242ac110003:1-7849",
  11. "relay_dir": "./relay_log"
  12. }'

对数据源停止 relay-log 功能

这是一个异步接口,请求成功的 Status Code 是 200,可通过获取数据源状态接口获取最新的状态。

请求 URI

POST /api/v1/sources/{source-name}/relay/disable

使用样例

  1. curl -X 'POST' \
  2. 'http://127.0.0.1:8261/api/v1/sources/mysql-01/relay/disable' \
  3. -H 'accept: */*' \
  4. -H 'Content-Type: application/json' \
  5. -d '{
  6. "worker_name_list": [
  7. "worker-1"
  8. ]
  9. }'

清除数据源不需要的 relay-log 文件

这是一个异步接口,请求成功的 Status Code 是 200,可通过获取数据源状态接口获取最新的状态。

请求 URI

POST /api/v1/sources/{source-name}/relay/purge

使用样例

  1. curl -X 'POST' \
  2. 'http://127.0.0.1:8261/api/v1/sources/mysql-01/relay/purge' \
  3. -H 'accept: */*' \
  4. -H 'Content-Type: application/json' \
  5. -d '{
  6. "relay_binlog_name": "mysql-bin.000002",
  7. "relay_dir": "string"
  8. }'

更改数据源和 DM-worker 的绑定关系

这是一个异步接口,请求成功的 Status Code 是 200,可通过获取 DM-worker 节点信息接口获取最新的状态。

请求 URI

POST /api/v1/sources/{source-name}/transfer

使用样例

  1. curl -X 'POST' \
  2. 'http://127.0.0.1:8261/api/v1/sources/mysql-01/transfer' \
  3. -H 'accept: */*' \
  4. -H 'Content-Type: application/json' \
  5. -d '{
  6. "worker_name": "worker-1"
  7. }'

获取数据源的数据库名列表

该接口是一个同步接口,请求成功会返回对应的列表。

请求 URI

GET /api/v1/sources/{source-name}/schemas

使用样例

  1. curl -X 'GET' \
  2. 'http://127.0.0.1:8261/api/v1/sources/source-1/schemas' \
  3. -H 'accept: application/json'
  1. [
  2. "db1"
  3. ]

获取数据源的指定数据库的表名列表

该接口是一个同步接口,请求成功会返回对应的列表。

请求 URI

GET /api/v1/sources/{source-name}/schemas/{schema-name}

使用样例

  1. curl -X 'GET' \
  2. 'http://127.0.0.1:8261/api/v1/sources/source-1/schemas/db1' \
  3. -H 'accept: application/json'
  1. [
  2. "table1"
  3. ]

创建同步任务

这是一个同步接口,请求成功的 Status Code 是 200,请求成功会返回对应的同步任务信息。

请求 URI

POST /api/v1/tasks

使用样例

  1. curl -X 'POST' \
  2. 'http://127.0.0.1:8261/api/v1/tasks' \
  3. -H 'accept: application/json' \
  4. -H 'Content-Type: application/json' \
  5. -d '{
  6. "task": {
  7. "name": "task-1",
  8. "task_mode": "all",
  9. "shard_mode": "pessimistic",
  10. "meta_schema": "dm-meta",
  11. "enhance_online_schema_change": true,
  12. "on_duplicate": "overwrite",
  13. "target_config": {
  14. "host": "127.0.0.1",
  15. "port": 3306,
  16. "user": "root",
  17. "password": "123456",
  18. "security": {
  19. "ssl_ca_content": "",
  20. "ssl_cert_content": "",
  21. "ssl_key_content": "",
  22. "cert_allowed_cn": [
  23. "string"
  24. ]
  25. }
  26. },
  27. "binlog_filter_rule": {
  28. "rule-1": {
  29. "ignore_event": [
  30. "all dml"
  31. ],
  32. "ignore_sql": [
  33. "^Drop"
  34. ]
  35. },
  36. "rule-2": {
  37. "ignore_event": [
  38. "all dml"
  39. ],
  40. "ignore_sql": [
  41. "^Drop"
  42. ]
  43. },
  44. "rule-3": {
  45. "ignore_event": [
  46. "all dml"
  47. ],
  48. "ignore_sql": [
  49. "^Drop"
  50. ]
  51. }
  52. },
  53. "table_migrate_rule": [
  54. {
  55. "source": {
  56. "source_name": "source-name",
  57. "schema": "db-*",
  58. "table": "tb-*"
  59. },
  60. "target": {
  61. "schema": "db1",
  62. "table": "tb1"
  63. },
  64. "binlog_filter_rule": [
  65. "rule-1",
  66. "rule-2",
  67. "rule-3",
  68. ]
  69. }
  70. ],
  71. "source_config": {
  72. "full_migrate_conf": {
  73. "export_threads": 4,
  74. "import_threads": 16,
  75. "data_dir": "./exported_data",
  76. "consistency": "auto"
  77. },
  78. "incr_migrate_conf": {
  79. "repl_threads": 16,
  80. "repl_batch": 100
  81. },
  82. "source_conf": [
  83. {
  84. "source_name": "mysql-replica-01",
  85. "binlog_name": "binlog.000001",
  86. "binlog_pos": 4,
  87. "binlog_gtid": "03fc0263-28c7-11e7-a653-6c0b84d59f30:1-7041423,05474d3c-28c7-11e7-8352-203db246dd3d:1-170"
  88. }
  89. ]
  90. }
  91. }
  92. }'
  1. {
  2. "name": "task-1",
  3. "task_mode": "all",
  4. "shard_mode": "pessimistic",
  5. "meta_schema": "dm-meta",
  6. "enhance_online_schema_change": true,
  7. "on_duplicate": "overwrite",
  8. "target_config": {
  9. "host": "127.0.0.1",
  10. "port": 3306,
  11. "user": "root",
  12. "password": "123456",
  13. "security": {
  14. "ssl_ca_content": "",
  15. "ssl_cert_content": "",
  16. "ssl_key_content": "",
  17. "cert_allowed_cn": [
  18. "string"
  19. ]
  20. }
  21. },
  22. "binlog_filter_rule": {
  23. "rule-1": {
  24. "ignore_event": [
  25. "all dml"
  26. ],
  27. "ignore_sql": [
  28. "^Drop"
  29. ]
  30. },
  31. "rule-2": {
  32. "ignore_event": [
  33. "all dml"
  34. ],
  35. "ignore_sql": [
  36. "^Drop"
  37. ]
  38. },
  39. "rule-3": {
  40. "ignore_event": [
  41. "all dml"
  42. ],
  43. "ignore_sql": [
  44. "^Drop"
  45. ]
  46. }
  47. },
  48. "table_migrate_rule": [
  49. {
  50. "source": {
  51. "source_name": "source-name",
  52. "schema": "db-*",
  53. "table": "tb-*"
  54. },
  55. "target": {
  56. "schema": "db1",
  57. "table": "tb1"
  58. },
  59. "binlog_filter_rule": [
  60. "rule-1",
  61. "rule-2",
  62. "rule-3",
  63. ]
  64. }
  65. ],
  66. "source_config": {
  67. "full_migrate_conf": {
  68. "export_threads": 4,
  69. "import_threads": 16,
  70. "data_dir": "./exported_data",
  71. "consistency": "auto"
  72. },
  73. "incr_migrate_conf": {
  74. "repl_threads": 16,
  75. "repl_batch": 100
  76. },
  77. "source_conf": [
  78. {
  79. "source_name": "mysql-replica-01",
  80. "binlog_name": "binlog.000001",
  81. "binlog_pos": 4,
  82. "binlog_gtid": "03fc0263-28c7-11e7-a653-6c0b84d59f30:1-7041423,05474d3c-28c7-11e7-8352-203db246dd3d:1-170"
  83. }
  84. ]
  85. }
  86. }

获取同步任务

这是一个同步接口,请求成功的 Status Code 是 200。

请求 URI

GET /api/v1/tasks/{task-name}?with_status=true

使用样例

  1. curl -X 'GET' \
  2. 'http://127.0.0.1:8261/api/v1/tasks/task-1?with_status=true' \
  3. -H 'accept: application/json'
  1. {
  2. "name": "task-1",
  3. "task_mode": "all",
  4. "shard_mode": "pessimistic",
  5. "meta_schema": "dm-meta",
  6. "enhance_online_schema_change": true,
  7. "on_duplicate": "overwrite",
  8. "target_config": {
  9. "host": "127.0.0.1",
  10. "port": 3306,
  11. "user": "root",
  12. "password": "123456",
  13. "security": {
  14. "ssl_ca_content": "",
  15. "ssl_cert_content": "",
  16. "ssl_key_content": "",
  17. "cert_allowed_cn": [
  18. "string"
  19. ]
  20. }
  21. },
  22. "binlog_filter_rule": {
  23. "rule-1": {
  24. "ignore_event": [
  25. "all dml"
  26. ],
  27. "ignore_sql": [
  28. "^Drop"
  29. ]
  30. },
  31. "rule-2": {
  32. "ignore_event": [
  33. "all dml"
  34. ],
  35. "ignore_sql": [
  36. "^Drop"
  37. ]
  38. },
  39. "rule-3": {
  40. "ignore_event": [
  41. "all dml"
  42. ],
  43. "ignore_sql": [
  44. "^Drop"
  45. ]
  46. }
  47. },
  48. "table_migrate_rule": [
  49. {
  50. "source": {
  51. "source_name": "source-name",
  52. "schema": "db-*",
  53. "table": "tb-*"
  54. },
  55. "target": {
  56. "schema": "db1",
  57. "table": "tb1"
  58. },
  59. "binlog_filter_rule": [
  60. "rule-1",
  61. "rule-2",
  62. "rule-3",
  63. ]
  64. }
  65. ],
  66. "source_config": {
  67. "full_migrate_conf": {
  68. "export_threads": 4,
  69. "import_threads": 16,
  70. "data_dir": "./exported_data",
  71. "consistency": "auto"
  72. },
  73. "incr_migrate_conf": {
  74. "repl_threads": 16,
  75. "repl_batch": 100
  76. },
  77. "source_conf": [
  78. {
  79. "source_name": "mysql-replica-01",
  80. "binlog_name": "binlog.000001",
  81. "binlog_pos": 4,
  82. "binlog_gtid": "03fc0263-28c7-11e7-a653-6c0b84d59f30:1-7041423,05474d3c-28c7-11e7-8352-203db246dd3d:1-170"
  83. }
  84. ]
  85. }
  86. }

删除同步任务

该接口是一个同步接口,请求成功后返回的 Status Code 是 204。

请求 URI

DELETE /api/v1/tasks/{task-name}

使用样例

  1. curl -X 'DELETE' \
  2. 'http://127.0.0.1:8261/api/v1/tasks/task-1' \
  3. -H 'accept: application/json'

更新同步任务

该接口是一个同步接口,请求成功会返回对应同步任务的信息。

OpenAPI - 图3

注意

更新同步任务配置时,须确保该任务处于暂停状态,并已经运行到增量同步的阶段,且仅有部分字段可以更新。

请求 URI

PUT /api/v1/tasks/{task-name}

使用样例

  1. curl -X 'PUT' \
  2. 'http://127.0.0.1:8261/api/v1/tasks/task-1' \
  3. -H 'accept: application/json' \
  4. -H 'Content-Type: application/json' \
  5. -d '{
  6. "task": {
  7. "name": "task-1",
  8. "task_mode": "all",
  9. "shard_mode": "pessimistic",
  10. "meta_schema": "dm-meta",
  11. "enhance_online_schema_change": true,
  12. "on_duplicate": "overwrite",
  13. "target_config": {
  14. "host": "127.0.0.1",
  15. "port": 3306,
  16. "user": "root",
  17. "password": "123456",
  18. "security": {
  19. "ssl_ca_content": "",
  20. "ssl_cert_content": "",
  21. "ssl_key_content": "",
  22. "cert_allowed_cn": [
  23. "string"
  24. ]
  25. }
  26. },
  27. "binlog_filter_rule": {
  28. "rule-1": {
  29. "ignore_event": [
  30. "all dml"
  31. ],
  32. "ignore_sql": [
  33. "^Drop"
  34. ]
  35. },
  36. "rule-2": {
  37. "ignore_event": [
  38. "all dml"
  39. ],
  40. "ignore_sql": [
  41. "^Drop"
  42. ]
  43. },
  44. "rule-3": {
  45. "ignore_event": [
  46. "all dml"
  47. ],
  48. "ignore_sql": [
  49. "^Drop"
  50. ]
  51. }
  52. },
  53. "table_migrate_rule": [
  54. {
  55. "source": {
  56. "source_name": "source-name",
  57. "schema": "db-*",
  58. "table": "tb-*"
  59. },
  60. "target": {
  61. "schema": "db1",
  62. "table": "tb1"
  63. },
  64. "binlog_filter_rule": [
  65. "rule-1",
  66. "rule-2",
  67. "rule-3",
  68. ]
  69. }
  70. ],
  71. "source_config": {
  72. "full_migrate_conf": {
  73. "export_threads": 4,
  74. "import_threads": 16,
  75. "data_dir": "./exported_data",
  76. "consistency": "auto"
  77. },
  78. "incr_migrate_conf": {
  79. "repl_threads": 16,
  80. "repl_batch": 100
  81. },
  82. "source_conf": [
  83. {
  84. "source_name": "mysql-replica-01",
  85. "binlog_name": "binlog.000001",
  86. "binlog_pos": 4,
  87. "binlog_gtid": "03fc0263-28c7-11e7-a653-6c0b84d59f30:1-7041423,05474d3c-28c7-11e7-8352-203db246dd3d:1-170"
  88. }
  89. ]
  90. }
  91. }
  92. }'
  1. {
  2. "name": "task-1",
  3. "task_mode": "all",
  4. "shard_mode": "pessimistic",
  5. "meta_schema": "dm-meta",
  6. "enhance_online_schema_change": true,
  7. "on_duplicate": "overwrite",
  8. "target_config": {
  9. "host": "127.0.0.1",
  10. "port": 3306,
  11. "user": "root",
  12. "password": "123456",
  13. "security": {
  14. "ssl_ca_content": "",
  15. "ssl_cert_content": "",
  16. "ssl_key_content": "",
  17. "cert_allowed_cn": [
  18. "string"
  19. ]
  20. }
  21. },
  22. "binlog_filter_rule": {
  23. "rule-1": {
  24. "ignore_event": [
  25. "all dml"
  26. ],
  27. "ignore_sql": [
  28. "^Drop"
  29. ]
  30. },
  31. "rule-2": {
  32. "ignore_event": [
  33. "all dml"
  34. ],
  35. "ignore_sql": [
  36. "^Drop"
  37. ]
  38. },
  39. "rule-3": {
  40. "ignore_event": [
  41. "all dml"
  42. ],
  43. "ignore_sql": [
  44. "^Drop"
  45. ]
  46. }
  47. },
  48. "table_migrate_rule": [
  49. {
  50. "source": {
  51. "source_name": "source-name",
  52. "schema": "db-*",
  53. "table": "tb-*"
  54. },
  55. "target": {
  56. "schema": "db1",
  57. "table": "tb1"
  58. },
  59. "binlog_filter_rule": [
  60. "rule-1",
  61. "rule-2",
  62. "rule-3",
  63. ]
  64. }
  65. ],
  66. "source_config": {
  67. "full_migrate_conf": {
  68. "export_threads": 4,
  69. "import_threads": 16,
  70. "data_dir": "./exported_data",
  71. "consistency": "auto"
  72. },
  73. "incr_migrate_conf": {
  74. "repl_threads": 16,
  75. "repl_batch": 100
  76. },
  77. "source_conf": [
  78. {
  79. "source_name": "mysql-replica-01",
  80. "binlog_name": "binlog.000001",
  81. "binlog_pos": 4,
  82. "binlog_gtid": "03fc0263-28c7-11e7-a653-6c0b84d59f30:1-7041423,05474d3c-28c7-11e7-8352-203db246dd3d:1-170"
  83. }
  84. ]
  85. }
  86. }

开始同步任务

这是一个异步接口,请求成功的 Status Code 是 200。可通过获取同步任务状态接口获取最新的任务状态。

请求 URI

POST /api/v1/tasks/{task-name}/start

使用样例

  1. curl -X 'POST' \
  2. 'http://127.0.0.1:8261/api/v1/tasks/task-1/start' \
  3. -H 'accept: */*'

停止同步任务

这是一个异步接口,请求成功的 Status Code 是 200,可通过获取同步任务状态接口获取最新的任务状态。

请求 URI

POST /api/v1/tasks/{task-name}/stop

使用样例

  1. curl -X 'POST' \
  2. 'http://127.0.0.1:8261/api/v1/tasks/task-1/stop' \
  3. -H 'accept: */*'

获取同步任务状态

该接口是一个同步接口,请求成功会返回对应节点的状态信息。

请求 URI

GET /api/v1/tasks/task-1/status

使用样例

  1. curl -X 'GET' \
  2. 'http://127.0.0.1:8261/api/v1/tasks/task-1/status?stage=running' \
  3. -H 'accept: application/json'
  1. {
  2. "total": 1,
  3. "data": [
  4. {
  5. "name": "string",
  6. "source_name": "string",
  7. "worker_name": "string",
  8. "stage": "runing",
  9. "unit": "sync",
  10. "unresolved_ddl_lock_id": "string",
  11. "load_status": {
  12. "finished_bytes": 0,
  13. "total_bytes": 0,
  14. "progress": "string",
  15. "meta_binlog": "string",
  16. "meta_binlog_gtid": "string"
  17. },
  18. "sync_status": {
  19. "total_events": 0,
  20. "total_tps": 0,
  21. "recent_tps": 0,
  22. "master_binlog": "string",
  23. "master_binlog_gtid": "string",
  24. "syncer_binlog": "string",
  25. "syncer_binlog_gtid": "string",
  26. "blocking_ddls": [
  27. "string"
  28. ],
  29. "unresolved_groups": [
  30. {
  31. "target": "string",
  32. "ddl_list": [
  33. "string"
  34. ],
  35. "first_location": "string",
  36. "synced": [
  37. "string"
  38. ],
  39. "unsynced": [
  40. "string"
  41. ]
  42. }
  43. ],
  44. "synced": true,
  45. "binlog_type": "string",
  46. "seconds_behind_master": 0
  47. }
  48. }
  49. ]
  50. }

获取同步任务列表

该接口是一个同步接口,请求成功会返回对应的同步任务列表。

请求 URI

GET /api/v1/tasks

使用样例

  1. curl -X 'GET' \
  2. 'http://127.0.0.1:8261/api/v1/tasks' \
  3. -H 'accept: application/json'
  1. {
  2. "total": 2,
  3. "data": [
  4. {
  5. "name": "task-1",
  6. "task_mode": "all",
  7. "shard_mode": "pessimistic",
  8. "meta_schema": "dm-meta",
  9. "enhance_online_schema_change": true,
  10. "on_duplicate": "overwrite",
  11. "target_config": {
  12. "host": "127.0.0.1",
  13. "port": 3306,
  14. "user": "root",
  15. "password": "123456",
  16. "security": {
  17. "ssl_ca_content": "",
  18. "ssl_cert_content": "",
  19. "ssl_key_content": "",
  20. "cert_allowed_cn": [
  21. "string"
  22. ]
  23. }
  24. },
  25. "binlog_filter_rule": {
  26. "rule-1": {
  27. "ignore_event": [
  28. "all dml"
  29. ],
  30. "ignore_sql": [
  31. "^Drop"
  32. ]
  33. },
  34. "rule-2": {
  35. "ignore_event": [
  36. "all dml"
  37. ],
  38. "ignore_sql": [
  39. "^Drop"
  40. ]
  41. },
  42. "rule-3": {
  43. "ignore_event": [
  44. "all dml"
  45. ],
  46. "ignore_sql": [
  47. "^Drop"
  48. ]
  49. }
  50. },
  51. "table_migrate_rule": [
  52. {
  53. "source": {
  54. "source_name": "source-name",
  55. "schema": "db-*",
  56. "table": "tb-*"
  57. },
  58. "target": {
  59. "schema": "db1",
  60. "table": "tb1"
  61. },
  62. "binlog_filter_rule": [
  63. "rule-1",
  64. "rule-2",
  65. "rule-3",
  66. ]
  67. }
  68. ],
  69. "source_config": {
  70. "full_migrate_conf": {
  71. "export_threads": 4,
  72. "import_threads": 16,
  73. "data_dir": "./exported_data",
  74. "consistency": "auto"
  75. },
  76. "incr_migrate_conf": {
  77. "repl_threads": 16,
  78. "repl_batch": 100
  79. },
  80. "source_conf": [
  81. {
  82. "source_name": "mysql-replica-01",
  83. "binlog_name": "binlog.000001",
  84. "binlog_pos": 4,
  85. "binlog_gtid": "03fc0263-28c7-11e7-a653-6c0b84d59f30:1-7041423,05474d3c-28c7-11e7-8352-203db246dd3d:1-170"
  86. }
  87. ]
  88. }
  89. }
  90. ]
  91. }

获取同步任务的同步规则列表

该接口是一个同步接口,请求成功会返回对应同步任务的同步规则列表。

请求 URI

GET /api/v1/tasks/{task-name}/sources/{source-name}/migrate_targets

使用样例

  1. curl -X 'GET' \
  2. 'http://127.0.0.1:8261/api/v1/tasks/task-1/sources/source-1/migrate_targets' \
  3. -H 'accept: application/json'
  1. {
  2. "total": 0,
  3. "data": [
  4. {
  5. "source_schema": "db1",
  6. "source_table": "tb1",
  7. "target_schema": "db1",
  8. "target_table": "tb1"
  9. }
  10. ]
  11. }

获取同步任务关联数据源的数据库名列表

该接口是一个同步接口,请求成功会返回对应的列表。

请求 URI

GET /api/v1/tasks/{task-name}/sources/{source-name}/schemas

使用样例

  1. curl -X 'GET' \
  2. 'http://127.0.0.1:8261/api/v1/tasks/task-1/sources/source-1/schemas' \
  3. -H 'accept: application/json'
  1. [
  2. "db1"
  3. ]

获取同步任务关联数据源的数据表名列表

该接口是一个同步接口,请求成功会返回对应的列表。

请求 URI

GET /api/v1/tasks/{task-name}/sources/{source-name}/schemas/{schema-name}

使用样例

  1. curl -X 'GET' \
  2. 'http://127.0.0.1:8261/api/v1/tasks/task-1/sources/source-1/schemas/db1' \
  3. -H 'accept: application/json'
  1. [
  2. "table1"
  3. ]

获取同步任务关联数据源的数据表的创建语句

该接口是一个同步接口,请求成功会返回对应的创建语句。

请求 URI

GET /api/v1/tasks/{task-name}/sources/{source-name}/schemas/{schema-name}/{table-name}

使用样例

  1. curl -X 'GET' \
  2. 'http://127.0.0.1:8261/api/v1/tasks/task-1/sources/source-1/schemas/db1/table1' \
  3. -H 'accept: application/json'
  1. {
  2. "schema_name": "db1",
  3. "table_name": "table1",
  4. "schema_create_sql": "CREATE TABLE `t1` (`id` int(11) NOT NULL AUTO_INCREMENT,PRIMARY KEY (`id`) /*T![clustered_index] CLUSTERED */) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin"
  5. }

更新同步任务关联数据源的数据表的创建语句

该接口是一个同步接口,返回体的 Status Code 是 200。

请求 URI

POST /api/v1/tasks/{task-name}/sources/{source-name}/schemas/{schema-name}/{table-name}

使用样例

  1. curl -X 'PUT' \
  2. 'http://127.0.0.1:8261/api/v1/tasks/task-1/sources/task-1/schemas/db1/table1' \
  3. -H 'accept: */*' \
  4. -H 'Content-Type: application/json' \
  5. -d '{
  6. "sql_content": "CREATE TABLE `t1` ( `c1` int(11) DEFAULT NULL, `c2` int(11) DEFAULT NULL, `c3` int(11) DEFAULT NULL) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin;",
  7. "flush": true,
  8. "sync": true
  9. }'

删除同步任务关联数据源的数据表

该接口是一个同步接口,返回体的 Status Code 是 200。

请求 URI

DELETE /api/v1/tasks/{task-name}/sources/{source-name}/schemas/{schema-name}/{table-name}

使用样例

  1. curl -X 'DELETE' \
  2. 'http://127.0.0.1:8261/api/v1/tasks/task-1/sources/source-1/schemas/db1/table1' \
  3. -H 'accept: */*'