Recover a Standalone after an Unexpected Shutdown

Warning

The following procedure applies to standalone mongodinstance version 4.2. For other MongoDB versions, refer tothe corresponding version of the manual.

Do not use this tutorial to recover a member of a replicaset. Instead, you should either restore from a backup or resync from another member of the set, asdescribed in Resync a Member of a Replica Set.

Tip

If you are running with journaling enabled, there isalmost never any need to run repair since the server can use thejournal files to restore the data files to a clean state automatically.However, you may need to run repair in cases where you need to recoverfrom a disk-level data corruption.

Disk-level data corruption or missing data files can preventmongod instance from starting, and journal files may beinsufficient to recover automatically:

  1. 2018-10-24T18:05:18.248-0400 W STORAGE [initandlisten] Detected unclean shutdown - mongod.lock is not empty.
  2.  
  3. ...
  4.  
  5. 2018-10-24T17:24:53.122-0400 E STORAGE [initandlisten] Failed to get the cursor for uri: table:collection-2-6854866147293273505
  6. 2018-10-24T17:24:53.122-0400 E STORAGE [initandlisten] This may be due to missing data files. ...
  7.  
  8. ...
  9.  
  10. ***aborting after fassert() failure

In such cases, your dbPath contains a non-emptymongod.lock file.

The following procedure uses mongod —repair to recover fromthese cases:

Warning

Only use mongod —repair if you have no other options.The operation removes and does not save any corrupt data during therepair process.

Starting in MongoDB 4.0.3, for the WiredTiger storage engine,mongod —repair:

  • Rebuilds all indexes.
  • Discards corrupt data.
  • Creates empty/stub files for missing data/metadata files.

Procedure

Important

Run the repair operation as the same user that normally runs themongod process to avoid changing the permissions of theMongoDB data files.

Create a backup of the data files.

Create a backup copy of the data files in the —dbpath.

Start mongod with —repair.

To repair the data files, start the mongod instance withthe —repair option.

Issue a command similar to the following for your standalone:

  1. mongod --dbpath /data/db --repair

Upon completion, the dbpath should contain the repaired data files and an empty mongod.lock file. [1]

Note

If the repair fails to complete for any reason, youmust restart the instance with the —repair option to complete the repair.

[1](1, 2) Generally, you should not manually remove the mongod.lock file.Instead, use the above procedure to recover the database. In diresituations, you can remove the file, start the database using thepossibly corrupt files, and attempt to recover data from thedatabase. However, it is impossible to predict the state of thedatabase in these situations.