Remove Shards from an Existing Sharded Cluster

To remove a shard you must ensure the shard’s data is migratedto the remaining shards in the cluster. This procedure describes how tosafely migrate data and how to remove a shard.

When you remove a shard in a cluster with an uneven chunkdistribution, the balancer first removes the chunks from the drainingshard and then balances the remaining uneven chunk distribution.

This procedure describes how to safely remove a single shard. _Do not_use this procedure to migrate an entire cluster to new hardware. Tomigrate an entire shard to new hardware, migrate individual shards as ifthey were independent replica sets.

To remove a shard, first connect to one of the cluster’smongos instances using mongo shell. Then use thesequence of tasks in this document to remove a shard from the cluster.

Considerations

A shard removal may cause an open change stream cursor to close, and the closed change stream cursor maynot be fully resumable.

Ensure the Balancer Process is Enabled

To successfully migrate data from a shard, the balancer processmust be enabled. Check the balancer state using thesh.getBalancerState() helper in the mongo shell.For more information, see the section on balancer operations.

Determine the Name of the Shard to Remove

To determine the name of the shard, connect to a mongosinstance with the mongo shell and either:

  1. db.adminCommand( { listShards: 1 } )

The shards._id field lists the name of each shard.

Remove Chunks from the Shard

From the admin database, run the removeShard command.This begins “draining” chunksfrom the shard you are removing to other shards in the cluster. Forexample, for a shard named mongodb0, run:

  1. db.adminCommand( { removeShard: "mongodb0" } )

mongos converts thewrite concern of theremoveShard command to "majority".

This operation returns with the following response:

  1. {
  2.    "msg" : "draining started successfully",
  3.    "state" : "started",
  4.    "shard" : "mongodb0",
  5.    "note" : "you need to drop or movePrimary these databases",
  6.    "dbsToMove" : [
  7.       "fiz",
  8.       "buzz"
  9.    ],
  10.    "ok" : 1,
  11.    "$clusterTime" : {
  12.       "clusterTime" : Timestamp(1510716515, 1),
  13.       "signature" : {
  14.          "hash" : BinData(0,"B2ViX7XLzFLS5Fl9XEuFXbwKIM4="),
  15.          "keyId" : NumberLong("6488045157173166092")
  16.       }
  17.    },
  18.    "operationTime" : Timestamp(1510716515, 1)
  19. }

The balancer begins migrating chunks from the shard named bristol01to other shards in the cluster. These migrations happens slowly toavoid placing undue load on the overall cluster.Depending on your network capacity and the amount of data, thisoperation can take from a few minutes to several days to complete.

Note

Each database in a sharded cluster has a primary shard. If the shard youwant to remove is also the primary of one of the cluster’s databases,removeShard lists the database in the dbsToMove field.To finish removing the shard, you must either move the databaseto a new shard after migrating all data from the shard or drop thedatabase, deleting the associated data files.

Check the Status of the Migration

To check the progress of the migration at any stage in the process, runremoveShard from the admin database again. For example,for a shard named mongodb0, run:

  1. db.adminCommand( { removeShard: "mongodb0" } )

mongos converts thewrite concern of theremoveShard command to "majority".

The command returns output similar to the following:

  1. {
  2.    "msg" : "draining ongoing",
  3.    "state" : "ongoing",
  4.    "remaining" : {
  5.       "chunks" : NumberLong(2),
  6.       "dbs" : NumberLong(2)
  7.    },
  8.    "note" : "you need to drop or movePrimary these databases",
  9.    "dbsToMove" : [
  10.       "fizz",
  11.       "buzz"
  12.    ],
  13.    "ok" : 1,
  14.    "$clusterTime" : {
  15.       "clusterTime" : Timestamp(1510716515, 1),
  16.       "signature" : {
  17.          "hash" : BinData(0,"B2ViX7XLzFLS5Fl9XEuFXbwKIM4="),
  18.          "keyId" : NumberLong("6488045157173166092")
  19.       }
  20.    },
  21.    "operationTime" : Timestamp(1510716515, 1)
  22. }

In the output, the remaining document displays the remaining numberof chunks that MongoDB must migrate to other shards and the number ofMongoDB databases that have “primary” status on this shard.

Continue checking the status of the removeShard command until thenumber of chunks remaining is 0. Always run the command on theadmin database. If you are on a database other than admin, you canuse sh._adminCommand to run the command on admin.

Move Databases to Another Primary Shard

If the shard is the primary shard for one or more databases inthe cluster, then you must make that database use a different shardas its primary shard. removeShard lists any databasesthat you need to move in the dbsToMove field in the command output.If the shard isnot the primary shard for any databases, skip to the next task,Finalize the Migration.

Warning

Do not perform this procedure until you have finished draining theshard.

To move a database to another shard, use the movePrimarycommand.

Important

To ensure a smooth migration, refer to the considerations in the movePrimary commanddocumentation before running movePrimary.

To migrate the fizz database frommongodb0 to mongodb1, issue the following command:

  1. db.adminCommand( { movePrimary: "fizz", to: "mongodb1" })

mongos uses "majority" formovePrimary.

This command does not return until MongoDB completes moving all data.The response from this command willresemble the following:

  1. {
  2.    "primary" : "mongodb1",
  3.    "ok" : 1,
  4.    "$clusterTime" : {
  5.       "clusterTime" : Timestamp(1510767932, 10),
  6.       "signature" : {
  7.          "hash" : BinData(0,"OJyZ0B4/Cp9z+mdrXLbJtNC7iuo="),
  8.          "keyId" : NumberLong("6488693018630029321")
  9.       }
  10.    },
  11.    "operationTime" : Timestamp(1510767932, 10)
  12. }

Warning

  • Starting in MongoDB 4.2:

  • If you use the movePrimary command to move _unsharded_collections, you must either:

    • Restart all mongos instances and allmongod shard members (including the secondarymembers);
    • Use the flushRouterConfig command on allmongos instances and all mongodshard members (including the secondary members) before readingor writing any data to any unsharded collections that weremoved.Otherwise, the you may miss data on reads, and may not write data tothe correct shard. To recover, you must manually intervene.

  • In MongoDB 4.0 and earlier:

  • If you use the movePrimary command to move un-shardedcollections, you must either restart all mongos instances,or use the flushRouterConfig command on allmongos instances before reading or writing any data to anyunsharded collections that were moved. This action ensures that themongos is aware of the new shard for these collections.

If you do not update the mongos instances’ metadata cacheafter using movePrimary, the mongos may miss dataon reads, and may not write data to the correct shard. To recover, you mustmanually intervene.

Otherwise, the you may miss data on reads, and may not write data tothe correct shard. To recover, you must manually intervene.

Finalize the Migration

To clean up all metadata information and finalize the removal, runremoveShard again. For example, for a shard namedmongodb0, run:

  1. db.adminCommand( { removeShard: "mongodb0" } )

mongos converts thewrite concern of theremoveShard command to "majority".

A success message appears at completion:

  1. {
  2.     "msg" : "removeshard completed successfully",
  3.     "state" : "completed",
  4.     "shard" : "mongodb0",
  5.     "ok" : 1,
  6.     "$clusterTime" : {
  7.        "clusterTime" : Timestamp(1510767995, 1),
  8.       "signature" : {
  9.          "hash" : BinData(0,"Tk0evhgFHA8HgerY5HmKXhIuw3g="),
  10.          "keyId" : NumberLong("6488693018630029321")
  11.       }
  12.     },
  13.     "operationTime" : Timestamp(1510767995, 1)
  14.  
  15. }

Once the value of the state field is “completed”, you may safelystop the instances comprising the mongodb0 shard.