Adding and removing a table from a replication cluster

ALTER CLUSTER <cluster_name> ADD <table_name> adds an existing local table to the cluster. The node which receives the ALTER query sends the table to the other nodes in the cluster. All the local tables with the same name on the other nodes of the cluster get replaced with the new table.

After the table is replicated, write statements can be performed on any node but table name must be prefixed with the cluster name like INSERT INTO <clusterName>:<table_name>.

  • SQL
  • JSON
  • PHP
  • Python
  • javascript
  • Java

SQL JSON PHP Python javascript Java

  1. ALTER CLUSTER click_query ADD clicks_daily_index
  1. POST /cli -d "
  2. ALTER CLUSTER click_query ADD clicks_daily_index
  3. "
  1. $params = [
  2.   'cluster' => 'click_query',
  3.   'body' => [
  4.      'operation' => 'add',
  5.      'index' => 'clicks_daily_index'
  7.   ]
  8. ];
  9. $response = $client->cluster()->alter($params);
  1. utilsApi.sql('ALTER CLUSTER click_query ADD clicks_daily_index')
  1. res = await utilsApi.sql('ALTER CLUSTER click_query ADD clicks_daily_index');
  1. utilsApi.sql("ALTER CLUSTER click_query ADD clicks_daily_index");

Response

  1. {u'error': u'', u'total': 0, u'warning': u''}
  1. {"total":0,"error":"","warning":""}

ALTER CLUSTER <cluster_name> DROP <table_name> forgets about a local table, i.e., it doesn’t remove the table files on the nodes but just makes it an active non-replicated table.

After a table is removed from a cluster, it becomes a ‘local’ table and write statements must use just the table name as INSERT INTO <table_name>, without the cluster prefix.

  • SQL
  • JSON
  • PHP
  • Python
  • javascript
  • Java

SQL JSON PHP Python javascript Java

  1. ALTER CLUSTER posts DROP weekly_index
  1. POST /cli -d "
  2. ALTER CLUSTER posts DROP weekly_index
  3. "
  1. $params = [
  2.   'cluster' => 'posts',
  3.   'body' => [
  4.      'operation' => 'drop',
  5.      'index' => 'weekly_index'
  7.   ]
  8. ];
  9. $response = $client->cluster->alter($params);
  1. utilsApi.sql('ALTER CLUSTER posts DROP weekly_index')
  1. res = await utilsApi.sql('ALTER CLUSTER posts DROP weekly_index');
  1. utilsApi.sql("ALTER CLUSTER posts DROP weekly_index");

Response

  1. {u'error': u'', u'total': 0, u'warning': u''}
  1. {"total":0,"error":"","warning":""}

Managing replication nodes

ALTER CLUSTER <cluster_name> UPDATE nodes statement updates node lists on each node of the cluster to include every active node in the cluster. See Joining a cluster for more info on node lists.

  • SQL
  • JSON
  • PHP
  • Python
  • javascript
  • Java

SQL JSON PHP Python javascript Java

  1. ALTER CLUSTER posts UPDATE nodes
  1. POST /cli -d "
  2. ALTER CLUSTER posts UPDATE nodes
  3. "
  1. $params = [
  2.   'cluster' => 'posts',
  3.   'body' => [
  4.      'operation' => 'update',
  6.   ]
  7. ];
  8. $response = $client->cluster()->alter($params);
  1. utilsApi.sql('ALTER CLUSTER posts UPDATE nodes')
  1. res = await utilsApi.sql('ALTER CLUSTER posts UPDATE nodes');
  1. utilsApi.sql("ALTER CLUSTER posts UPDATE nodes");

Response

  1. {u'error': u'', u'total': 0, u'warning': u''}
  1. {"total":0,"error":"","warning":""}

For example, when the cluster was initially created, the list of nodes used for rejoining the cluster was 10.10.0.1:9312,10.10.1.1:9312. Since then other nodes joined the cluster and now we have the following active nodes: 10.10.0.1:9312,10.10.1.1:9312,10.15.0.1:9312,10.15.0.3:9312.

But the list of nodes used for rejoining the cluster is still the same. Running the ALTER CLUSTER ... UPDATE nodes copies the list of active nodes to the list of nodes used to rejoin on restart. After this, the list of nodes used on restart includes all the active nodes in the cluster.

Both lists of nodes can be viewed using Cluster status statement (cluster_post_nodes_set and cluster_post_nodes_view).

Removing node from cluster

To remove a node from a replication cluster you need to:

  1. stop the node
  2. remove info about the cluster in <data_dir>/manticore.json (/var/lib/manticore/manticore.json in most cases) on the node you’ve stopped
  3. run ALTER CLUSTER cluster_name UPDATE nodes on any other node

After this the other nodes will forget about the detached node and the node will forget about the cluster. It won’t impact tables neither in the cluster nor on the detached node.

Replication cluster status

Node status outputs, among other information, cluster status variables.

The output format is cluster_name_variable_name variable_value. Most of them are described in Galera Documentation Status Variables. Additionally we display:

  • cluster_name - name of the cluster
  • node_state - current state of the node: closed, destroyed, joining, donor, synced
  • indexes_count - number of tables managed by the cluster
  • indexes - list of table names managed by the cluster
  • nodes_set - list of nodes in the cluster defined with cluster CREATE, JOIN or ALTER UPDATE commands
  • nodes_view - actual list of nodes in cluster which this node sees
  • SQL
  • JSON
  • PHP
  • Python
  • javascript
  • Java

SQL JSON PHP Python javascript Java

  1. SHOW STATUS
  1. POST /cli -d "
  2. SHOW STATUS
  3. "
  1. $params = [
  2.     'body' => []
  3. ];
  4. $response = $client->nodes()->status($params);
  1. utilsApi.sql('SHOW STATUS')
  1. res = await utilsApi.sql('SHOW STATUS');
  1. utilsApi.sql("SHOW STATUS");

Response

  1. +----------------------------+-------------------------------------------------------------------------------------+
  2. | Counter                    | Value                                                                               |
  3. +----------------------------+-------------------------------------------------------------------------------------+
  4. | cluster_name               | post                                                                                |
  5. | cluster_post_state_uuid    | fba97c45-36df-11e9-a84e-eb09d14b8ea7                                                |
  6. | cluster_post_conf_id       | 1                                                                                   |
  7. | cluster_post_status        | primary                                                                             |
  8. | cluster_post_size          | 5                                                                                   |
  9. | cluster_post_local_index   | 0                                                                                   |
  10. | cluster_post_node_state    | synced                                                                              |
  11. | cluster_post_indexes_count | 2                                                                                   |
  12. | cluster_post_indexes       | pq1,pq_posts                                                                        |
  13. | cluster_post_nodes_set     | 10.10.0.1:9312                                                                      |
  14. | cluster_post_nodes_view    | 10.10.0.1:9312,10.10.0.1:9320:replication,10.10.1.1:9312,10.10.1.1:9320:replication |
  1. "
  2. {"columns":[{"Counter":{"type":"string"}},{"Value":{"type":"string"}}],
  3. "data":[
  4. {"Counter":"cluster_name", "Value":"post"},
  5. {"Counter":"cluster_post_state_uuid", "Value":"fba97c45-36df-11e9-a84e-eb09d14b8ea7"},
  6. {"Counter":"cluster_post_conf_id", "Value":"1"},
  7. {"Counter":"cluster_post_status", "Value":"primary"},
  8. {"Counter":"cluster_post_size", "Value":"5"},
  9. {"Counter":"cluster_post_local_index", "Value":"0"},
  10. {"Counter":"cluster_post_node_state", "Value":"synced"},
  11. {"Counter":"cluster_post_indexes_count", "Value":"2"},
  12. {"Counter":"cluster_post_indexes", "Value":"pq1,pq_posts"},
  13. {"Counter":"cluster_post_nodes_set", "Value":"10.10.0.1:9312"},
  14. {"Counter":"cluster_post_nodes_view", "Value":"10.10.0.1:9312,10.10.0.1:9320:replication,10.10.1.1:9312,10.10.1.1:9320:replication"}
  15. ],
  16. "total":0,
  17. "error":"",
  18. "warning":""
  19. }
  1. (
  2. "cluster_name" => "post",
  3. "cluster_post_state_uuid" => "fba97c45-36df-11e9-a84e-eb09d14b8ea7",
  4. "cluster_post_conf_id" => 1,
  5. "cluster_post_status" => "primary",
  6. "cluster_post_size" => 5,
  7. "cluster_post_local_index" => 0,
  8. "cluster_post_node_state" => "synced",
  9. "cluster_post_indexes_count" => 2,
  10. "cluster_post_indexes" => "pq1,pq_posts",
  11. "cluster_post_nodes_set" => "10.10.0.1:9312",
  12. "cluster_post_nodes_view" => "10.10.0.1:9312,10.10.0.1:9320:replication,10.10.1.1:9312,10.10.1.1:9320:replication"
  13. )
  1. {u'columns': [{u'Key': {u'type': u'string'}},
  2.               {u'Value': {u'type': u'string'}}],
  3.  u'data': [
  4.     {u'Key': u'cluster_name', u'Value': u'post'},
  5.     {u'Key': u'cluster_post_state_uuid', u'Value': u'fba97c45-36df-11e9-a84e-eb09d14b8ea7'},
  6.     {u'Key': u'cluster_post_conf_id', u'Value': u'1'},
  7.     {u'Key': u'cluster_post_status', u'Value': u'primary'},
  8.     {u'Key': u'cluster_post_size', u'Value': u'5'},
  9.     {u'Key': u'cluster_post_local_index', u'Value': u'0'},
  10.     {u'Key': u'cluster_post_node_state', u'Value': u'synced'},
  11.     {u'Key': u'cluster_post_indexes_count', u'Value': u'2'},
  12.     {u'Key': u'cluster_post_indexes', u'Value': u'pq1,pq_posts'},
  13.     {u'Key': u'cluster_post_nodes_set', u'Value': u'10.10.0.1:9312'},
  14.     {u'Key': u'cluster_post_nodes_view', u'Value': u'10.10.0.1:9312,10.10.0.1:9320:replication,10.10.1.1:9312,10.10.1.1:9320:replication'}],
  15.  u'error': u'',
  16.  u'total': 0,
  17.  u'warning': u''}
  1. {"columns": [{"Key": {"type": "string"}},
  2.               {"Value": {"type": "string"}}],
  3.  "data": [
  4.     {"Key": "cluster_name", "Value": "post"},
  5.     {"Key": "cluster_post_state_uuid", "Value": "fba97c45-36df-11e9-a84e-eb09d14b8ea7"},
  6.     {"Key": "cluster_post_conf_id", "Value": "1"},
  7.     {"Key": "cluster_post_status", "Value": "primary"},
  8.     {"Key": "cluster_post_size", "Value": "5"},
  9.     {"Key": "cluster_post_local_index", "Value": "0"},
  10.     {"Key": "cluster_post_node_state", "Value": "synced"},
  11.     {"Key": "cluster_post_indexes_count", "Value": "2"},
  12.     {"Key": "cluster_post_indexes", "Value": "pq1,pq_posts"},
  13.     {"Key": "cluster_post_nodes_set", "Value": "10.10.0.1:9312"},
  14.     {"Key": "cluster_post_nodes_view", "Value": "10.10.0.1:9312,10.10.0.1:9320:replication,10.10.1.1:9312,10.10.1.1:9320:replication"}],
  15.  "error": "",
  16.  "total": 0,
  17.  "warning": ""}
  1. {columns=[{ Key : { type=string }},
  2.               { Value : { type=string }}],
  3.   data : [
  4.     { Key=cluster_name, Value=post},
  5.     { Key=cluster_post_state_uuid, Value=fba97c45-36df-11e9-a84e-eb09d14b8ea7},
  6.     { Key=cluster_post_conf_id, Value=1},
  7.     { Key=cluster_post_status, Value=primary},
  8.     { Key=cluster_post_size, Value=5},
  9.     { Key=cluster_post_local_index, Value=0},
  10.     { Key=cluster_post_node_state, Value=synced},
  11.     { Key=cluster_post_indexes_count, Value=2},
  12.     { Key=cluster_post_indexes, Value=pq1,pq_posts},
  13.     { Key=cluster_post_nodes_set, Value=10.10.0.1:9312},
  14.     { Key=cluster_post_nodes_view, Value=10.10.0.1:9312,10.10.0.1:9320:replication,10.10.1.1:9312,10.10.1.1:9320:replication}],
  15.   error= ,
  16.   total=0,
  17.   warning= }