Operating a Cluster

Running Ceph with systemd

For all distributions that support systemd (CentOS 7, Fedora, DebianJessie 8 and later, SUSE), ceph daemons are now managed using nativesystemd files instead of the legacy sysvinit scripts. For example:

  1. sudo systemctl start ceph.target # start all daemons
  2. sudo systemctl status ceph-osd@12 # check status of osd.12

To list the Ceph systemd units on a node, execute:

  1. sudo systemctl status ceph\*.service ceph\*.target

Starting all Daemons

To start all daemons on a Ceph Node (irrespective of type), execute thefollowing:

  1. sudo systemctl start ceph.target

Stopping all Daemons

To stop all daemons on a Ceph Node (irrespective of type), execute thefollowing:

  1. sudo systemctl stop ceph\*.service ceph\*.target

Starting all Daemons by Type

To start all daemons of a particular type on a Ceph Node, execute one of thefollowing:

  1. sudo systemctl start ceph-osd.target
  2. sudo systemctl start ceph-mon.target
  3. sudo systemctl start ceph-mds.target

Stopping all Daemons by Type

To stop all daemons of a particular type on a Ceph Node, execute one of thefollowing:

  1. sudo systemctl stop ceph-mon\*.service ceph-mon.target
  2. sudo systemctl stop ceph-osd\*.service ceph-osd.target
  3. sudo systemctl stop ceph-mds\*.service ceph-mds.target

Starting a Daemon

To start a specific daemon instance on a Ceph Node, execute one of thefollowing:

  1. sudo systemctl start ceph-osd@{id}
  2. sudo systemctl start ceph-mon@{hostname}
  3. sudo systemctl start ceph-mds@{hostname}

For example:

  1. sudo systemctl start ceph-osd@1
  2. sudo systemctl start ceph-mon@ceph-server
  3. sudo systemctl start ceph-mds@ceph-server

Stopping a Daemon

To stop a specific daemon instance on a Ceph Node, execute one of thefollowing:

  1. sudo systemctl stop ceph-osd@{id}
  2. sudo systemctl stop ceph-mon@{hostname}
  3. sudo systemctl stop ceph-mds@{hostname}

For example:

  1. sudo systemctl stop ceph-osd@1
  2. sudo systemctl stop ceph-mon@ceph-server
  3. sudo systemctl stop ceph-mds@ceph-server

Starting all Daemons

To start all daemons on a Ceph Node (irrespective of type), execute thefollowing:

  1. sudo start ceph-all

Stopping all Daemons

To stop all daemons on a Ceph Node (irrespective of type), execute thefollowing:

  1. sudo stop ceph-all

Starting all Daemons by Type

To start all daemons of a particular type on a Ceph Node, execute one of thefollowing:

  1. sudo start ceph-osd-all
  2. sudo start ceph-mon-all
  3. sudo start ceph-mds-all

Stopping all Daemons by Type

To stop all daemons of a particular type on a Ceph Node, execute one of thefollowing:

  1. sudo stop ceph-osd-all
  2. sudo stop ceph-mon-all
  3. sudo stop ceph-mds-all

Starting a Daemon

To start a specific daemon instance on a Ceph Node, execute one of thefollowing:

  1. sudo start ceph-osd id={id}
  2. sudo start ceph-mon id={hostname}
  3. sudo start ceph-mds id={hostname}

For example:

  1. sudo start ceph-osd id=1
  2. sudo start ceph-mon id=ceph-server
  3. sudo start ceph-mds id=ceph-server

Stopping a Daemon

To stop a specific daemon instance on a Ceph Node, execute one of thefollowing:

  1. sudo stop ceph-osd id={id}
  2. sudo stop ceph-mon id={hostname}
  3. sudo stop ceph-mds id={hostname}

For example:

  1. sudo stop ceph-osd id=1
  2. sudo start ceph-mon id=ceph-server
  3. sudo start ceph-mds id=ceph-server

Running Ceph

Each time you to start, restart, and stop Ceph daemons (or yourentire cluster) you must specify at least one option and one command. You mayalso specify a daemon type or a daemon instance.

  1. {commandline} [options] [commands] [daemons]

The ceph options include:

OptionShortcutDescription
—verbose-vUse verbose logging.
—valgrindN/A(Dev and QA only) Use Valgrind debugging.
—allhosts-aExecute on all nodes in ceph.conf.Otherwise, it only executes on localhost.
—restartN/AAutomatically restart daemon if it core dumps.
—norestartN/ADon’t restart a daemon if it core dumps.
—conf-cUse an alternate configuration file.

The ceph commands include:

CommandDescription
startStart the daemon(s).
stopStop the daemon(s).
forcestopForce the daemon(s) to stop. Same as kill -9
killallKill all daemons of a particular type.
cleanlogsCleans out the log directory.
cleanalllogsCleans out everything in the log directory.

For subsystem operations, the ceph service can target specific daemon typesby adding a particular daemon type for the [daemons] option. Daemon typesinclude:

  • mon

  • osd

  • mds