cephfs-journal-tool

Purpose

If a CephFS journal has become damaged, expert intervention may be requiredto restore the file system to a working state.

The cephfs-journal-tool utility provides functionality to aid experts inexamining, modifying, and extracting data from journals.

Warning

This tool is dangerous because it directly modifies internaldata structures of the file system. Make backups, be careful, andseek expert advice. If you are unsure, do not run this tool.

Syntax

  1. cephfs-journal-tool journal <inspect|import|export|reset>
  2. cephfs-journal-tool header <get|set>
  3. cephfs-journal-tool event <get|splice|apply> [filter] <list|json|summary|binary>

The tool operates in three modes: journal, header and event,meaning the whole journal, the header, and the events within the journalrespectively.

Journal mode

This should be your starting point to assess the state of a journal.

  • inspect reports on the health of the journal. This will identify anymissing objects or corruption in the stored journal. Note that this doesnot identify inconsistencies in the events themselves, just that events arepresent and can be decoded.

  • import and export read and write binary dumps of the journalin a sparse file format. Pass the filename as the last argument. Theexport operation may not work reliably for journals which are damaged (missingobjects).

  • reset truncates a journal, discarding any information within it.

Example: journal inspect

  1. # cephfs-journal-tool journal inspect
  2. Overall journal integrity: DAMAGED
  3. Objects missing:
  4.   0x1
  5. Corrupt regions:
  6.   0x400000-ffffffffffffffff

Example: Journal import/export

  1. # cephfs-journal-tool journal export myjournal.bin
  2. journal is 4194304~80643
  3. read 80643 bytes at offset 4194304
  4. wrote 80643 bytes at offset 4194304 to myjournal.bin
  5. NOTE: this is a _sparse_ file; you can
  6.     $ tar cSzf myjournal.bin.tgz myjournal.bin
  7.       to efficiently compress it while preserving sparseness.
  8.  
  9. # cephfs-journal-tool journal import myjournal.bin
  10. undump myjournal.bin
  11. start 4194304 len 80643
  12. writing header 200.00000000
  13.  writing 4194304~80643
  14. done.

Note

It is wise to use the journal export <backup file> command to make a journal backupbefore any further manipulation.

Header mode

  • get outputs the current content of the journal header

  • set modifies an attribute of the header. Allowed attributes aretrimmed_pos, expire_pos and write_pos.

Example: header get/set

  1. # cephfs-journal-tool header get
  2. { "magic": "ceph fs volume v011",
  3.   "write_pos": 4274947,
  4.   "expire_pos": 4194304,
  5.   "trimmed_pos": 4194303,
  6.   "layout": { "stripe_unit": 4194304,
  7.       "stripe_count": 4194304,
  8.       "object_size": 4194304,
  9.       "cas_hash": 4194304,
  10.       "object_stripe_unit": 4194304,
  11.       "pg_pool": 4194304}}
  12.  
  13. # cephfs-journal-tool header set trimmed_pos 4194303
  14. Updating trimmed_pos 0x400000 -> 0x3fffff
  15. Successfully updated header.

Event mode

Event mode allows detailed examination and manipulation of the contents of the journal. Eventmode can operate on all events in the journal, or filters may be applied.

The arguments following cephfs-journal-tool event consist of an action, optional filterparameters, and an output mode:

  1. cephfs-journal-tool event <action> [filter] <output>

Actions:

  • get read the events from the log

  • splice erase events or regions in the journal

  • apply extract file system metadata from events and attempt to apply it to the metadata store.

Filtering:

  • —range <int begin>..[int end] only include events within the range begin (inclusive) to end (exclusive)

  • —path <path substring> only include events referring to metadata containing the specified string

  • —inode <int> only include events referring to metadata containing the specified inode

  • —type <type string> only include events of this type

  • —frag <ino>[.frag id] only include events referring to this directory fragment

  • —dname <string> only include events referring to this named dentry within a directoryfragment (may only be used in conjunction with —frag

  • —client <int> only include events from this client session ID

Filters may be combined on an AND basis (i.e. only the intersection of events from each filter).

Output modes:

  • binary: write each event as a binary file, within a folder whose name is controlled by —path

  • json: write all events to a single file, as a JSON serialized list of objects

  • summary: write a human readable summary of the events read to standard out

  • list: write a human readable terse listing of the type of each event, andwhich file paths the event affects.

Example: event mode

  1. # cephfs-journal-tool event get json --path output.json
  2. Wrote output to JSON file 'output.json'
  3.  
  4. # cephfs-journal-tool event get summary
  5. Events by type:
  6.   NOOP: 2
  7.   OPEN: 2
  8.   SESSION: 2
  9.   SUBTREEMAP: 1
  10.   UPDATE: 43
  11.  
  12. # cephfs-journal-tool event get list
  13. 0x400000 SUBTREEMAP:  ()
  14. 0x400308 SESSION:  ()
  15. 0x4003de UPDATE:  (setattr)
  16.   /
  17. 0x40068b UPDATE:  (mkdir)
  18.   diralpha
  19. 0x400d1b UPDATE:  (mkdir)
  20.   diralpha/filealpha1
  21. 0x401666 UPDATE:  (unlink_local)
  22.   stray0/10000000001
  23.   diralpha/filealpha1
  24. 0x40228d UPDATE:  (unlink_local)
  25.   diralpha
  26.   stray0/10000000000
  27. 0x402bf9 UPDATE:  (scatter_writebehind)
  28.   stray0
  29. 0x403150 UPDATE:  (mkdir)
  30.   dirbravo
  31. 0x4037e0 UPDATE:  (openc)
  32.   dirbravo/.filebravo1.swp
  33. 0x404032 UPDATE:  (openc)
  34.   dirbravo/.filebravo1.swpx
  35.  
  36. # cephfs-journal-tool event get --path filebravo1 list
  37. 0x40785a UPDATE:  (openc)
  38.   dirbravo/filebravo1
  39. 0x4103ee UPDATE:  (cap update)
  40.   dirbravo/filebravo1
  41.  
  42. # cephfs-journal-tool event splice --range 0x40f754..0x410bf1 summary
  43. Events by type:
  44.   OPEN: 1
  45.   UPDATE: 2
  46.  
  47. # cephfs-journal-tool event apply --range 0x410bf1.. summary
  48. Events by type:
  49.   NOOP: 1
  50.   SESSION: 1
  51.   UPDATE: 9
  52.  
  53. # cephfs-journal-tool event get --inode=1099511627776 list
  54. 0x40068b UPDATE:  (mkdir)
  55.   diralpha
  56. 0x400d1b UPDATE:  (mkdir)
  57.   diralpha/filealpha1
  58. 0x401666 UPDATE:  (unlink_local)
  59.   stray0/10000000001
  60.   diralpha/filealpha1
  61. 0x40228d UPDATE:  (unlink_local)
  62.   diralpha
  63.   stray0/10000000000
  64.  
  65. # cephfs-journal-tool event get --frag=1099511627776 --dname=filealpha1 list
  66. 0x400d1b UPDATE:  (mkdir)
  67.   diralpha/filealpha1
  68. 0x401666 UPDATE:  (unlink_local)
  69.   stray0/10000000001
  70.   diralpha/filealpha1
  71.  
  72. # cephfs-journal-tool event get binary --path bin_events
  73. Wrote output to binary files in directory 'bin_events'