Resync a Member of a Replica Set

A replica set member becomes “stale” when its replicationprocess falls so far behind that the primary overwrites oplogentries the member has not yet replicated. The member cannot catch upand becomes “stale.” When this occurs, you must completelyresynchronize the member by removing its data and performing aninitial sync.

This tutorial addresses both resyncing a stale member and creating anew member using seed data from another member, both of which can beused to restore a replica set member. When syncing a member, choose atime when the system has the bandwidth to move a large amount of data.Schedule the synchronization during a time of low usage or during amaintenance window.

MongoDB provides two options for performing an initial sync:

• Restart the mongod with an empty data directory and letMongoDB’s normal initial syncing feature restore the data. Thisis the more simple option but may take longer to replace the data.

See Automatically Sync a Member.

• Restart the machine with a copy of a recent data directory fromanother member in the replica set. This procedure can replacethe data more quickly but requires more manual steps.

See Sync by Copying Data Files from Another Member.

Procedures

Automatically Sync a Member

Warning

During initial sync, mongod removes the contents of thedbPath directory.

This procedure relies on MongoDB’s regular process forReplica Set Syncing. This stores the currentdata on the member. For an overview of MongoDB initial sync process, see theReplica Set Syncing section.

Initial sync operations can impact the other members of the set andcreate additional traffic to the primary. The syncing member requires anothermember of the set that is accessible and up to date.

If the instance has no data, you can follow theAdd Members to a Replica Set orReplace a Replica Set Member procedure to add a new member to areplica set.

You can also force a mongod that is already a member of the set toperform an initial sync by restarting the instance without the contents of thedbPath directory:

• Stop the member’s mongod instance.To ensure a clean shutdown, use the db.shutdownServer()method from the mongo shell or on Linux systems, themongod —shutdown option.
• Delete all data and sub-directories from the member’sdbPath directory. Consider making a backup first.
• Start the mongod process,specifying Replication Options as appropriate.At this point, the mongod performs an initial sync. The length ofthe initial sync process depends on the size of the database and the networklatency between members of the replica set.

Sync by Copying Data Files from Another Member

This approach “seeds” a new or stale member using the data files froman existing member of the replica set. The data files must besufficiently recent to allow the new member to catch up with theoplog. Otherwise the member would need to perform an initialsync.

Copy the Data Files

You can capture the data files as either a snapshot or a direct copy.However, in most cases you cannot copy data files from a runningmongod instance to another because the data files will changeduring the file copy operation.

Important

If copying data files, ensure that your copy includesthe content of the local database.

You cannot use a mongodump backup for the data files:only a snapshot backup. For approaches to capturing a consistentsnapshot of a running mongod instance, see theMongoDB Backup Methods documentation.

Sync the Member

After you have copied the data files from the “seed” source, start themongod instance and allow it to apply all operations fromthe oplog until it reflects the current state of the replica set.