ceph-mgr administrator’s guide

Manual setup

Usually, you would set up a ceph-mgr daemon using a tool suchas ceph-ansible. These instructions describe how to set upa ceph-mgr daemon manually.

First, create an authentication key for your daemon:

  1. ceph auth get-or-create mgr.$name mon 'allow profile mgr' osd 'allow *' mds 'allow *'

Place that key into mgr data path, which for a cluster “ceph”and mgr $name “foo” would be /var/lib/ceph/mgr/ceph-foo.

Start the ceph-mgr daemon:

  1. ceph-mgr -i $name

Check that the mgr has come up by looking at the outputof ceph status, which should now include a mgr status line:

  1. mgr active: $name

Client authentication

The manager is a new daemon which requires new CephX capabilities. If you upgradea cluster from an old version of Ceph, or use the default install/deploy tools,your admin client should get this capability automatically. If you use tooling fromelsewhere, you may get EACCES errors when invoking certain ceph cluster commands.To fix that, add a “mgr allow *” stanza to your client’s cephx capabilities byModifying User Capabilities.

High availability

In general, you should set up a ceph-mgr on each of the hostsrunning a ceph-mon daemon to achieve the same level of availability.

By default, whichever ceph-mgr instance comes up first will be madeactive by the monitors, and the others will be standbys. There isno requirement for quorum among the ceph-mgr daemons.

If the active daemon fails to send a beacon to the monitors formore than mon mgr beacon grace (default 30s), then it will be replacedby a standby.

If you want to pre-empt failover, you can explicitly mark a ceph-mgrdaemon as failed using ceph mgr fail <mgr name>.

Using modules

Use the command ceph mgr module ls to see which modules areavailable, and which are currently enabled. Enable or disable modulesusing the commands ceph mgr module enable <module> andceph mgr module disable <module> respectively.

If a module is enabled then the active ceph-mgr daemon will loadand execute it. In the case of modules that provide a service,such as an HTTP server, the module may publish its address when itis loaded. To see the addresses of such modules, use the commandceph mgr services.

Some modules may also implement a special standby mode which runs onstandby ceph-mgr daemons as well as the active daemon. This enablesmodules that provide services to redirect their clients to the activedaemon, if the client tries to connect to a standby.

Consult the documentation pages for individual manager modules for moreinformation about what functionality each module provides.

Here is an example of enabling the Dashboard module:

  1. $ ceph mgr module ls
  2. {
  3.         "enabled_modules": [
  4.                 "restful",
  5.                 "status"
  6.         ],
  7.         "disabled_modules": [
  8.                 "dashboard"
  9.         ]
  10. }
  11.  
  12. $ ceph mgr module enable dashboard
  13. $ ceph mgr module ls
  14. {
  15.         "enabled_modules": [
  16.                 "restful",
  17.                 "status",
  18.                 "dashboard"
  19.         ],
  20.         "disabled_modules": [
  21.         ]
  22. }
  23.  
  24. $ ceph mgr services
  25. {
  26.         "dashboard": "http://myserver.com:7789/",
  27.         "restful": "https://myserver.com:8789/"
  28. }

The first time the cluster starts, it uses the mgr_initial_modulessetting to override which modules to enable. However, this settingis ignored through the rest of the lifetime of the cluster: onlyuse it for bootstrapping. For example, before starting yourmonitor daemons for the first time, you might add a section likethis to your ceph.conf:

  1. [mon]
  2.     mgr initial modules = dashboard balancer

Calling module commands

Where a module implements command line hooks, the commands willbe accessible as ordinary Ceph commands. Ceph will automatically incorporatemodule commands into the standard CLI interface and route them appropriately tothe module.:

  1. ceph <command | help>

Configuration

mgr module path

  • Description

  • Path to load modules from

  • Type

  • String

  • Default

  • "<library dir>/mgr"

mgr data

  • Description

  • Path to load daemon data (such as keyring)

  • Type

  • String

  • Default

  • "/var/lib/ceph/mgr/$cluster-$id"

mgr tick period

  • Description

  • How many seconds between mgr beacons to monitors, and otherperiodic checks.

  • Type

  • Integer

  • Default

  • 5

mon mgr beacon grace

  • Description

  • How long after last beacon should a mgr be considered failed

  • Type

  • Integer

  • Default

  • 30