Cephx Config Reference

The cephx protocol is enabled by default. Cryptographic authentication hassome computational costs, though they should generally be quite low. If thenetwork environment connecting your client and server hosts is very safe andyou cannot afford authentication, you can turn it off. This is not generallyrecommended.

Note

If you disable authentication, you are at risk of a man-in-the-middleattack altering your client/server messages, which could lead to disastroussecurity effects.

For creating users, see User Management. For details on the architectureof Cephx, see Architecture - High Availability Authentication.

Deployment Scenarios

There are two main scenarios for deploying a Ceph cluster, which impacthow you initially configure Cephx. Most first time Ceph users useceph-deploy to create a cluster (easiest). For clusters usingother deployment tools (e.g., Chef, Juju, Puppet, etc.), you will needto use the manual procedures or configure your deployment tool tobootstrap your monitor(s).

ceph-deploy

When you deploy a cluster with ceph-deploy, you do not have to bootstrap themonitor manually or create the client.admin user or keyring. The steps youexecute in the Storage Cluster Quick Start will invoke ceph-deploy to dothat for you.

When you execute ceph-deploy new {initial-monitor(s)}, Ceph will create amonitor keyring for you (only used to bootstrap monitors), and it will generatean initial Ceph configuration file for you, which contains the followingauthentication settings, indicating that Ceph enables authentication bydefault:

  1. auth_cluster_required = cephx
  2. auth_service_required = cephx
  3. auth_client_required = cephx

When you execute ceph-deploy mon create-initial, Ceph will bootstrap theinitial monitor(s), retrieve a ceph.client.admin.keyring file containing thekey for the client.admin user. Additionally, it will also retrieve keyringsthat give ceph-deploy and ceph-volume utilities the ability to prepare andactivate OSDs and metadata servers.

When you execute ceph-deploy admin {node-name} (note: Ceph must beinstalled first), you are pushing a Ceph configuration file and theceph.client.admin.keyring to the /etc/ceph directory of the node. Youwill be able to execute Ceph administrative functions as root on the commandline of that node.

Manual Deployment

When you deploy a cluster manually, you have to bootstrap the monitor manuallyand create the client.admin user and keyring. To bootstrap monitors, followthe steps in Monitor Bootstrapping. The steps for monitor bootstrapping arethe logical steps you must perform when using third party deployment tools likeChef, Puppet, Juju, etc.

Enabling/Disabling Cephx

Enabling Cephx requires that you have deployed keys for your monitors,OSDs and metadata servers. If you are simply toggling Cephx on / off,you do not have to repeat the bootstrapping procedures.

Enabling Cephx

When cephx is enabled, Ceph will look for the keyring in the default searchpath, which includes /etc/ceph/$cluster.$name.keyring. You can overridethis location by adding a keyring option in the [global] section ofyour Ceph configuration file, but this is not recommended.

Execute the following procedures to enable cephx on a cluster withauthentication disabled. If you (or your deployment utility) have alreadygenerated the keys, you may skip the steps related to generating keys.

  • Create a client.admin key, and save a copy of the key for your clienthost:
  1. ceph auth get-or-create client.admin mon 'allow *' mds 'allow *' mgr 'allow *' osd 'allow *' -o /etc/ceph/ceph.client.admin.keyring

Warning: This will clobber any existing/etc/ceph/client.admin.keyring file. Do not perform this step if adeployment tool has already done it for you. Be careful!

  • Create a keyring for your monitor cluster and generate a monitorsecret key.
  1. ceph-authtool --create-keyring /tmp/ceph.mon.keyring --gen-key -n mon. --cap mon 'allow *'
  • Copy the monitor keyring into a ceph.mon.keyring file in every monitor’smon data directory. For example, to copy it to mon.a in cluster ceph,use the following:
  1. cp /tmp/ceph.mon.keyring /var/lib/ceph/mon/ceph-a/keyring
  • Generate a secret key for every MGR, where {$id} is the MGR letter:
  1. ceph auth get-or-create mgr.{$id} mon 'allow profile mgr' mds 'allow *' osd 'allow *' -o /var/lib/ceph/mgr/ceph-{$id}/keyring
  • Generate a secret key for every OSD, where {$id} is the OSD number:
  1. ceph auth get-or-create osd.{$id} mon 'allow rwx' osd 'allow *' -o /var/lib/ceph/osd/ceph-{$id}/keyring
  • Generate a secret key for every MDS, where {$id} is the MDS letter:
  1. ceph auth get-or-create mds.{$id} mon 'allow rwx' osd 'allow *' mds 'allow *' mgr 'allow profile mds' -o /var/lib/ceph/mds/ceph-{$id}/keyring
  • Enable cephx authentication by setting the following options in the[global] section of your Ceph configuration file:
  1. auth cluster required = cephx
  2. auth service required = cephx
  3. auth client required = cephx

For details on bootstrapping a monitor manually, see Manual Deployment.

Disabling Cephx

The following procedure describes how to disable Cephx. If your clusterenvironment is relatively safe, you can offset the computation expense ofrunning authentication. We do not recommend it. However, it may be easierduring setup and/or troubleshooting to temporarily disable authentication.

  • Disable cephx authentication by setting the following options in the[global] section of your Ceph configuration file:
  1. auth cluster required = none
  2. auth service required = none
  3. auth client required = none

Configuration Settings

Enablement

auth cluster required

  • Description

  • If enabled, the Ceph Storage Cluster daemons (i.e., ceph-mon,ceph-osd, ceph-mds and ceph-mgr) must authenticate witheach other. Valid settings are cephx or none.

  • Type

  • String

  • Required

  • No

  • Default

  • cephx.

auth service required

  • Description

  • If enabled, the Ceph Storage Cluster daemons require Ceph Clientsto authenticate with the Ceph Storage Cluster in order to accessCeph services. Valid settings are cephx or none.

  • Type

  • String

  • Required

  • No

  • Default

  • cephx.

auth client required

  • Description

  • If enabled, the Ceph Client requires the Ceph Storage Cluster toauthenticate with the Ceph Client. Valid settings are cephxor none.

  • Type

  • String

  • Required

  • No

  • Default

  • cephx.

Keys

When you run Ceph with authentication enabled, ceph administrative commandsand Ceph Clients require authentication keys to access the Ceph Storage Cluster.

The most common way to provide these keys to the ceph administrativecommands and clients is to include a Ceph keyring under the /etc/cephdirectory. For Cuttlefish and later releases using ceph-deploy, the filenameis usually ceph.client.admin.keyring (or $cluster.client.admin.keyring).If you include the keyring under the /etc/ceph directory, you don’t need tospecify a keyring entry in your Ceph configuration file.

We recommend copying the Ceph Storage Cluster’s keyring file to nodes where youwill run administrative commands, because it contains the client.admin key.

You may use ceph-deploy admin to perform this task. See Create an AdminHost for details. To perform this step manually, execute the following:

  1. sudo scp {user}@{ceph-cluster-host}:/etc/ceph/ceph.client.admin.keyring /etc/ceph/ceph.client.admin.keyring

Tip

Ensure the ceph.keyring file has appropriate permissions set(e.g., chmod 644) on your client machine.

You may specify the key itself in the Ceph configuration file using the keysetting (not recommended), or a path to a keyfile using the keyfile setting.

keyring

  • Description

  • The path to the keyring file.

  • Type

  • String

  • Required

  • No

  • Default

  • /etc/ceph/$cluster.$name.keyring,/etc/ceph/$cluster.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin

keyfile

  • Description

  • The path to a key file (i.e,. a file containing only the key).

  • Type

  • String

  • Required

  • No

  • Default

  • None

key

  • Description

  • The key (i.e., the text string of the key itself). Not recommended.

  • Type

  • String

  • Required

  • No

  • Default

  • None

Daemon Keyrings

Administrative users or deployment tools (e.g., ceph-deploy) may generatedaemon keyrings in the same way as generating user keyrings. By default, Cephstores daemons keyrings inside their data directory. The default keyringlocations, and the capabilities necessary for the daemon to function, are shownbelow.

ceph-mon

  • Location

  • $mon_data/keyring

  • Capabilities

  • mon 'allow *'

ceph-osd

  • Location

  • $osd_data/keyring

  • Capabilities

  • mgr 'allow profile osd' mon 'allow profile osd' osd 'allow *'

ceph-mds

  • Location

  • $mds_data/keyring

  • Capabilities

  • mds 'allow' mgr 'allow profile mds' mon 'allow profile mds' osd 'allow rwx'

ceph-mgr

  • Location

  • $mgr_data/keyring

  • Capabilities

  • mon 'allow profile mgr' mds 'allow ' osd 'allow '

radosgw

  • Location

  • $rgw_data/keyring

  • Capabilities

  • mon 'allow rwx' osd 'allow rwx'

Note

The monitor keyring (i.e., mon.) contains a key but nocapabilities, and is not part of the cluster auth database.

The daemon data directory locations default to directories of the form:

  1. /var/lib/ceph/$type/$cluster-$id

For example, osd.12 would be:

  1. /var/lib/ceph/osd/ceph-12

You can override these locations, but it is not recommended.

Signatures

Ceph performs a signature check that provides some limited protectionagainst messages being tampered with in flight (e.g., by a “man in themiddle” attack).

Like other parts of Ceph authentication, Ceph provides fine-grained control soyou can enable/disable signatures for service messages between the client andCeph, and you can enable/disable signatures for messages between Ceph daemons.

Note that even with signatures enabled data is not encrypted inflight.

cephx require signatures

  • Description
  • If set to true, Ceph requires signatures on all messagetraffic between the Ceph Client and the Ceph Storage Cluster, andbetween daemons comprising the Ceph Storage Cluster.

Ceph Argonaut and Linux kernel versions prior to 3.19 donot support signatures; if such clients are in use thisoption can be turned off to allow them to connect.

  • Type

  • Boolean

  • Required

  • No

  • Default

  • false

cephx cluster require signatures

  • Description

  • If set to true, Ceph requires signatures on all messagetraffic between Ceph daemons comprising the Ceph Storage Cluster.

  • Type

  • Boolean

  • Required

  • No

  • Default

  • false

cephx service require signatures

  • Description

  • If set to true, Ceph requires signatures on all messagetraffic between Ceph Clients and the Ceph Storage Cluster.

  • Type

  • Boolean

  • Required

  • No

  • Default

  • false

cephx sign messages

  • Description

  • If the Ceph version supports message signing, Ceph will signall messages so they are more difficult to spoof.

  • Type

  • Boolean

  • Default

  • true

Time to Live

auth service ticket ttl

  • Description

  • When the Ceph Storage Cluster sends a Ceph Client a ticket forauthentication, the Ceph Storage Cluster assigns the ticket atime to live.

  • Type

  • Double

  • Default

  • 60*60