User Management
This document describes Ceph Client users, and their authentication andauthorization with the Ceph Storage Cluster. Users are eitherindividuals or system actors such as applications, which use Ceph clients tointeract with the Ceph Storage Cluster daemons.
When Ceph runs with authentication and authorization enabled (enabled bydefault), you must specify a user name and a keyring containing the secret keyof the specified user (usually via the command line). If you do not specify auser name, Ceph will use client.admin
as the default user name. If you donot specify a keyring, Ceph will look for a keyring via the keyring
settingin the Ceph configuration. For example, if you execute the ceph health
command without specifying a user or keyring:
- ceph health
Ceph interprets the command like this:
- ceph -n client.admin --keyring=/etc/ceph/ceph.client.admin.keyring health
Alternatively, you may use the CEPH_ARGS
environment variable to avoidre-entry of the user name and secret.
For details on configuring the Ceph Storage Cluster to use authentication,see Cephx Config Reference. For details on the architecture of Cephx, seeArchitecture - High Availability Authentication.
Background
Irrespective of the type of Ceph client (e.g., Block Device, Object Storage,Filesystem, native API, etc.), Ceph stores all data as objects within pools.Ceph users must have access to pools in order to read and write data.Additionally, Ceph users must have execute permissions to use Ceph’sadministrative commands. The following concepts will help you understand Cephuser management.
User
A user is either an individual or a system actor such as an application.Creating users allows you to control who (or what) can access your Ceph StorageCluster, its pools, and the data within pools.
Ceph has the notion of a type
of user. For the purposes of user management,the type will always be client
. Ceph identifies users in period (.)delimited form consisting of the user type and the user ID: for example,TYPE.ID
, client.admin
, or client.user1
. The reason for user typingis that Ceph Monitors, OSDs, and Metadata Servers also use the Cephx protocol,but they are not clients. Distinguishing the user type helps to distinguishbetween client users and other users–streamlining access control, usermonitoring and traceability.
Sometimes Ceph’s user type may seem confusing, because the Ceph command lineallows you to specify a user with or without the type, depending upon yourcommand line usage. If you specify —user
or —id
, you can omit thetype. So client.user1
can be entered simply as user1
. If you specify—name
or -n
, you must specify the type and name, such asclient.user1
. We recommend using the type and name as a best practicewherever possible.
Note
A Ceph Storage Cluster user is not the same as a Ceph Object Storageuser or a Ceph File System user. The Ceph Object Gateway uses a Ceph StorageCluster user to communicate between the gateway daemon and the storagecluster, but the gateway has its own user management functionality for endusers. The Ceph File System uses POSIX semantics. The user space associatedwith the Ceph File System is not the same as a Ceph Storage Cluster user.
Authorization (Capabilities)
Ceph uses the term “capabilities” (caps) to describe authorizing anauthenticated user to exercise the functionality of the monitors, OSDs andmetadata servers. Capabilities can also restrict access to data within a pool,a namespace within a pool, or a set of pools based on their application tags.A Ceph administrative user sets a user’s capabilities when creating or updatinga user.
Capability syntax follows the form:
- {daemon-type} '{cap-spec}[, {cap-spec} ...]'
- Monitor Caps: Monitor capabilities include
r
,w
,x
accesssettings orprofile {name}
. For example:
- mon 'allow {access-spec} [network {network/prefix}]'
- mon 'profile {name}'
The {access-spec}
syntax is as follows:
- * | all | [r][w][x]
The optional {network/prefix}
is a standard network name andprefix length in CIDR notation (e.g., 10.3.0.0/16
). If present,the use of this capability is restricted to clients connecting fromthis network.
- OSD Caps: OSD capabilities include
r
,w
,x
,class-read
,class-write
access settings orprofile {name}
. Additionally, OSDcapabilities also allow for pool and namespace settings.
- osd 'allow {access-spec} [{match-spec}] [network {network/prefix}]'
- osd 'profile {name} [pool={pool-name} [namespace={namespace-name}]] [network {network/prefix}]'
The {access-spec}
syntax is either of the following:
- * | all | [r][w][x] [class-read] [class-write]
- class {class name} [{method name}]
The optional {match-spec}
syntax is either of the following:
- pool={pool-name} [namespace={namespace-name}] [object_prefix {prefix}]
- [namespace={namespace-name}] tag {application} {key}={value}
The optional {network/prefix}
is a standard network name andprefix length in CIDR notation (e.g., 10.3.0.0/16
). If present,the use of this capability is restricted to clients connecting fromthis network.
- Manager Caps: Manager (
ceph-mgr
) capabilities includer
,w
,x
access settings orprofile {name}
. For example:
- mgr 'allow {access-spec} [network {network/prefix}]'
- mgr 'profile {name} [{key1} {match-type} {value1} ...] [network {network/prefix}]'
Manager capabilities can also be specified for specific commands,all commands exported by a built-in manager service, or all commandsexported by a specific add-on module. For example:
- mgr 'allow command "{command-prefix}" [with {key1} {match-type} {value1} ...] [network {network/prefix}]'
- mgr 'allow service {service-name} {access-spec} [network {network/prefix}]'
- mgr 'allow module {module-name} [with {key1} {match-type} {value1} ...] {access-spec} [network {network/prefix}]'
The {access-spec}
syntax is as follows:
- * | all | [r][w][x]
The {service-name}
is one of the following:
- mgr | osd | pg | py
The {match-type}
is one of the following:
- = | prefix | regex
- Metadata Server Caps: For administrators, use
allow *
. For allother users, such as CephFS clients, consult CephFS Client Capabilities
Note
The Ceph Object Gateway daemon (radosgw
) is a client of theCeph Storage Cluster, so it is not represented as a Ceph StorageCluster daemon type.
The following entries describe each access capability.
allow
- Description
- Precedes access settings for a daemon. Implies
rw
for MDS only.
r
- Description
- Gives the user read access. Required with monitors to retrievethe CRUSH map.
w
- Description
- Gives the user write access to objects.
x
- Description
- Gives the user the capability to call class methods(i.e., both read and write) and to conduct
auth
operations on monitors.
class-read
- Descriptions
- Gives the user the capability to call class read methods.Subset of
x
.
class-write
- Description
- Gives the user the capability to call class write methods.Subset of
x
.
*
, all
- Description
- Gives the user read, write and execute permissions for aparticular daemon/pool, and the ability to executeadmin commands.
The following entries describe valid capability profiles:
profile osd
(Monitor only)
- Description
- Gives a user permissions to connect as an OSD to other OSDs ormonitors. Conferred on OSDs to enable OSDs to handle replicationheartbeat traffic and status reporting.
profile mds
(Monitor only)
- Description
- Gives a user permissions to connect as a MDS to other MDSs ormonitors.
profile bootstrap-osd
(Monitor only)
- Description
- Gives a user permissions to bootstrap an OSD. Conferred ondeployment tools such as
ceph-volume
,ceph-deploy
, etc.so that they have permissions to add keys, etc. whenbootstrapping an OSD.
profile bootstrap-mds
(Monitor only)
- Description
- Gives a user permissions to bootstrap a metadata server.Conferred on deployment tools such as
ceph-deploy
, etc.so they have permissions to add keys, etc. when bootstrappinga metadata server.
profile bootstrap-rbd
(Monitor only)
- Description
- Gives a user permissions to bootstrap an RBD user.Conferred on deployment tools such as
ceph-deploy
, etc.so they have permissions to add keys, etc. when bootstrappingan RBD user.
profile bootstrap-rbd-mirror
(Monitor only)
- Description
- Gives a user permissions to bootstrap an
rbd-mirror
daemonuser. Conferred on deployment tools such asceph-deploy
, etc.so they have permissions to add keys, etc. when bootstrappinganrbd-mirror
daemon.
profile rbd
(Manager, Monitor, and OSD)
- Description
- Gives a user permissions to manipulate RBD images. When usedas a Monitor cap, it provides the minimal privileges requiredby an RBD client application; this includes the abilityto blacklist other client users. When used as an OSD cap, itprovides read-write access to the specified pool to anRBD client application. The Manager cap supports optional
pool
andnamespace
keyword arguments.
profile rbd-mirror
(Monitor only)
- Description
- Gives a user permissions to manipulate RBD images and retrieveRBD mirroring config-key secrets. It provides the minimalprivileges required for the
rbd-mirror
daemon.
profile rbd-read-only
(Manager and OSD)
- Description
- Gives a user read-only permissions to RBD images. The Managercap supports optional
pool
andnamespace
keywordarguments.
Pool
A pool is a logical partition where users store data.In Ceph deployments, it is common to create a pool as a logical partition forsimilar types of data. For example, when deploying Ceph as a backend forOpenStack, a typical deployment would have pools for volumes, images, backupsand virtual machines, and users such as client.glance
, client.cinder
,etc.
Application Tags
Access may be restricted to specific pools as defined by their applicationmetadata. The wildcard may be used for the
key
argument, thevalue
argument, or both. all
is a synony for .
Namespace
Objects within a pool can be associated to a namespace–a logical group ofobjects within the pool. A user’s access to a pool can be associated with anamespace such that reads and writes by the user take place only within thenamespace. Objects written to a namespace within the pool can only be accessedby users who have access to the namespace.
Note
Namespaces are primarily useful for applications written on top oflibrados
where the logical grouping can alleviate the need to createdifferent pools. Ceph Object Gateway (from luminous
) uses namespaces for variousmetadata objects.
The rationale for namespaces is that pools can be a computationally expensivemethod of segregating data sets for the purposes of authorizing separate setsof users. For example, a pool should have ~100 placement groups per OSD. So anexemplary cluster with 1000 OSDs would have 100,000 placement groups for onepool. Each pool would create another 100,000 placement groups in the exemplarycluster. By contrast, writing an object to a namespace simply associates thenamespace to the object name with out the computational overhead of a separatepool. Rather than creating a separate pool for a user or set of users, you mayuse a namespace. Note: Only available using librados
at this time.
Access may be restricted to specific RADOS namespaces using the namespace
capability. Limited globbing of namespaces is supported; if the last characterof the specified namespace is *
, then access is granted to any namespacestarting with the provided argument.
Managing Users
User management functionality provides Ceph Storage Cluster administrators withthe ability to create, update and delete users directly in the Ceph StorageCluster.
When you create or delete users in the Ceph Storage Cluster, you may need todistribute keys to clients so that they can be added to keyrings. See KeyringManagement for details.
List Users
To list the users in your cluster, execute the following:
- ceph auth ls
Ceph will list out all users in your cluster. For example, in a two-nodeexemplary cluster, ceph auth ls
will output something that looks likethis:
- installed auth entries:
- osd.0
- key: AQCvCbtToC6MDhAATtuT70Sl+DymPCfDSsyV4w==
- caps: [mon] allow profile osd
- caps: [osd] allow *
- osd.1
- key: AQC4CbtTCFJBChAAVq5spj0ff4eHZICxIOVZeA==
- caps: [mon] allow profile osd
- caps: [osd] allow *
- client.admin
- key: AQBHCbtT6APDHhAA5W00cBchwkQjh3dkKsyPjw==
- caps: [mds] allow
- caps: [mon] allow *
- caps: [osd] allow *
- client.bootstrap-mds
- key: AQBICbtTOK9uGBAAdbe5zcIGHZL3T/u2g6EBww==
- caps: [mon] allow profile bootstrap-mds
- client.bootstrap-osd
- key: AQBHCbtT4GxqORAADE5u7RkpCN/oo4e5W0uBtw==
- caps: [mon] allow profile bootstrap-osd
Note that the TYPE.ID
notation for users applies such that osd.0
is auser of type osd
and its ID is 0
, client.admin
is a user of typeclient
and its ID is admin
(i.e., the default client.admin
user).Note also that each entry has a key: <value>
entry, and one or morecaps:
entries.
You may use the -o {filename}
option with ceph auth ls
tosave the output to a file.
Get a User
To retrieve a specific user, key and capabilities, execute thefollowing:
- ceph auth get {TYPE.ID}
For example:
- ceph auth get client.admin
You may also use the -o {filename}
option with ceph auth get
tosave the output to a file. Developers may also execute the following:
- ceph auth export {TYPE.ID}
The auth export
command is identical to auth get
.
Add a User
Adding a user creates a username (i.e., TYPE.ID
), a secret key andany capabilities included in the command you use to create the user.
A user’s key enables the user to authenticate with the Ceph Storage Cluster.The user’s capabilities authorize the user to read, write, or execute on Cephmonitors (mon
), Ceph OSDs (osd
) or Ceph Metadata Servers (mds
).
There are a few ways to add a user:
ceph auth add
: This command is the canonical way to add a user. Itwill create the user, generate a key and add any specified capabilities.ceph auth get-or-create
: This command is often the most convenient wayto create a user, because it returns a keyfile format with the user name(in brackets) and the key. If the user already exists, this commandsimply returns the user name and key in the keyfile format. You may use the-o {filename}
option to save the output to a file.ceph auth get-or-create-key
: This command is a convenient way to createa user and return the user’s key (only). This is useful for clients thatneed the key only (e.g., libvirt). If the user already exists, this commandsimply returns the key. You may use the-o {filename}
option to save theoutput to a file.
When creating client users, you may create a user with no capabilities. A userwith no capabilities is useless beyond mere authentication, because the clientcannot retrieve the cluster map from the monitor. However, you can create auser with no capabilities if you wish to defer adding capabilities later usingthe ceph auth caps
command.
A typical user has at least read capabilities on the Ceph monitor andread and write capability on Ceph OSDs. Additionally, a user’s OSD permissionsare often restricted to accessing a particular pool.
- ceph auth add client.john mon 'allow r' osd 'allow rw pool=liverpool'
- ceph auth get-or-create client.paul mon 'allow r' osd 'allow rw pool=liverpool'
- ceph auth get-or-create client.george mon 'allow r' osd 'allow rw pool=liverpool' -o george.keyring
- ceph auth get-or-create-key client.ringo mon 'allow r' osd 'allow rw pool=liverpool' -o ringo.key
Important
If you provide a user with capabilities to OSDs, but you DO NOTrestrict access to particular pools, the user will have access to ALLpools in the cluster!
Modify User Capabilities
The ceph auth caps
command allows you to specify a user and change theuser’s capabilities. Setting new capabilities will overwrite current capabilities.To view current capabilities run ceph auth get USERTYPE.USERID
. To addcapabilities, you should also specify the existing capabilities when using the form:
- ceph auth caps USERTYPE.USERID {daemon} 'allow [r|w|x|*|...] [pool={pool-name}] [namespace={namespace-name}]' [{daemon} 'allow [r|w|x|*|...] [pool={pool-name}] [namespace={namespace-name}]']
For example:
- ceph auth get client.john
- ceph auth caps client.john mon 'allow r' osd 'allow rw pool=liverpool'
- ceph auth caps client.paul mon 'allow rw' osd 'allow rwx pool=liverpool'
- ceph auth caps client.brian-manager mon 'allow *' osd 'allow *'
See Authorization (Capabilities) for additional details on capabilities.
Delete a User
To delete a user, use ceph auth del
:
- ceph auth del {TYPE}.{ID}
Where {TYPE}
is one of client
, osd
, mon
, or mds
,and {ID}
is the user name or ID of the daemon.
Print a User’s Key
To print a user’s authentication key to standard output, execute the following:
- ceph auth print-key {TYPE}.{ID}
Where {TYPE}
is one of client
, osd
, mon
, or mds
,and {ID}
is the user name or ID of the daemon.
Printing a user’s key is useful when you need to populate clientsoftware with a user’s key (e.g., libvirt).
- mount -t ceph serverhost:/ mountpoint -o name=client.user,secret=`ceph auth print-key client.user`
Import a User(s)
To import one or more users, use ceph auth import
andspecify a keyring:
- ceph auth import -i /path/to/keyring
For example:
- sudo ceph auth import -i /etc/ceph/ceph.keyring
Note
The ceph storage cluster will add new users, their keys and theircapabilities and will update existing users, their keys and theircapabilities.
Keyring Management
When you access Ceph via a Ceph client, the Ceph client will look for a localkeyring. Ceph presets the keyring
setting with the following four keyringnames by default so you don’t have to set them in your Ceph configuration fileunless you want to override the defaults (not recommended):
/etc/ceph/$cluster.$name.keyring
/etc/ceph/$cluster.keyring
/etc/ceph/keyring
/etc/ceph/keyring.bin
The $cluster
metavariable is your Ceph cluster name as defined by thename of the Ceph configuration file (i.e., ceph.conf
means the cluster nameis ceph
; thus, ceph.keyring
). The $name
metavariable is the usertype and user ID (e.g., client.admin
; thus, ceph.client.admin.keyring
).
Note
When executing commands that read or write to /etc/ceph
, you mayneed to use sudo
to execute the command as root
.
After you create a user (e.g., client.ringo
), you must get the key and addit to a keyring on a Ceph client so that the user can access the Ceph StorageCluster.
The User Management section details how to list, get, add, modify and deleteusers directly in the Ceph Storage Cluster. However, Ceph also provides theceph-authtool
utility to allow you to manage keyrings from a Ceph client.
Create a Keyring
When you use the procedures in the Managing Users section to create users,you need to provide user keys to the Ceph client(s) so that the Ceph clientcan retrieve the key for the specified user and authenticate with the CephStorage Cluster. Ceph Clients access keyrings to lookup a user name andretrieve the user’s key.
The ceph-authtool
utility allows you to create a keyring. To create anempty keyring, use —create-keyring
or -C
. For example:
- ceph-authtool --create-keyring /path/to/keyring
When creating a keyring with multiple users, we recommend using the cluster name(e.g., $cluster.keyring
) for the keyring filename and saving it in the/etc/ceph
directory so that the keyring
configuration default settingwill pick up the filename without requiring you to specify it in the local copyof your Ceph configuration file. For example, create ceph.keyring
byexecuting the following:
- sudo ceph-authtool -C /etc/ceph/ceph.keyring
When creating a keyring with a single user, we recommend using the cluster name,the user type and the user name and saving it in the /etc/ceph
directory.For example, ceph.client.admin.keyring
for the client.admin
user.
To create a keyring in /etc/ceph
, you must do so as root
. This meansthe file will have rw
permissions for the root
user only, which isappropriate when the keyring contains administrator keys. However, if youintend to use the keyring for a particular user or group of users, ensurethat you execute chown
or chmod
to establish appropriate keyringownership and access.
Add a User to a Keyring
When you Add a User to the Ceph Storage Cluster, you can use the Get aUser procedure to retrieve a user, key and capabilities and save the user to akeyring.
When you only want to use one user per keyring, the Get a User procedure withthe -o
option will save the output in the keyring file format. For example,to create a keyring for the client.admin
user, execute the following:
- sudo ceph auth get client.admin -o /etc/ceph/ceph.client.admin.keyring
Notice that we use the recommended file format for an individual user.
When you want to import users to a keyring, you can use ceph-authtool
to specify the destination keyring and the source keyring.For example:
- sudo ceph-authtool /etc/ceph/ceph.keyring --import-keyring /etc/ceph/ceph.client.admin.keyring
Create a User
Ceph provides the Add a User function to create a user directly in the CephStorage Cluster. However, you can also create a user, keys and capabilitiesdirectly on a Ceph client keyring. Then, you can import the user to the CephStorage Cluster. For example:
- sudo ceph-authtool -n client.ringo --cap osd 'allow rwx' --cap mon 'allow rwx' /etc/ceph/ceph.keyring
See Authorization (Capabilities) for additional details on capabilities.
You can also create a keyring and add a new user to the keyring simultaneously.For example:
- sudo ceph-authtool -C /etc/ceph/ceph.keyring -n client.ringo --cap osd 'allow rwx' --cap mon 'allow rwx' --gen-key
In the foregoing scenarios, the new user client.ringo
is only in thekeyring. To add the new user to the Ceph Storage Cluster, you must still addthe new user to the Ceph Storage Cluster.
- sudo ceph auth add client.ringo -i /etc/ceph/ceph.keyring
Modify a User
To modify the capabilities of a user record in a keyring, specify the keyring,and the user followed by the capabilities. For example:
- sudo ceph-authtool /etc/ceph/ceph.keyring -n client.ringo --cap osd 'allow rwx' --cap mon 'allow rwx'
To update the user to the Ceph Storage Cluster, you must update the userin the keyring to the user entry in the the Ceph Storage Cluster.
- sudo ceph auth import -i /etc/ceph/ceph.keyring
See Import a User(s) for details on updating a Ceph Storage Cluster userfrom a keyring.
You may also Modify User Capabilities directly in the cluster, store theresults to a keyring file; then, import the keyring into your mainceph.keyring
file.
Command Line Usage
Ceph supports the following usage for user name and secret:
—id
| —user
- Description
- Ceph identifies users with a type and an ID (e.g.,
TYPE.ID
orclient.admin
,client.user1
). Theid
,name
and-n
options enable you to specify the ID portion of the username (e.g.,admin
,user1
,foo
, etc.). You can specifythe user with the—id
and omit the type. For example,to specify userclient.foo
enter the following:
- ceph --id foo --keyring /path/to/keyring health
- ceph --user foo --keyring /path/to/keyring health
—name
| -n
- Description
- Ceph identifies users with a type and an ID (e.g.,
TYPE.ID
orclient.admin
,client.user1
). The—name
and-n
options enables you to specify the fully qualified user name.You must specify the user type (typicallyclient
) with theuser ID. For example:
- ceph --name client.foo --keyring /path/to/keyring health
- ceph -n client.foo --keyring /path/to/keyring health
—keyring
- Description
- The path to the keyring containing one or more user name andsecret. The
—secret
option provides the same functionality,but it does not work with Ceph RADOS Gateway, which uses—secret
for another purpose. You may retrieve a keyring withceph auth get-or-create
and store it locally. This is apreferred approach, because you can switch user names withoutswitching the keyring path. For example:
- sudo rbd map --id foo --keyring /path/to/keyring mypool/myimage
Limitations
The cephx
protocol authenticates Ceph clients and servers to each other. Itis not intended to handle authentication of human users or application programsrun on their behalf. If that effect is required to handle your access controlneeds, you must have another mechanism, which is likely to be specific to thefront end used to access the Ceph object store. This other mechanism has therole of ensuring that only acceptable users and programs are able to run on themachine that Ceph will permit to access its object store.
The keys used to authenticate Ceph clients and servers are typically stored ina plain text file with appropriate permissions in a trusted host.
Important
Storing keys in plaintext files has security shortcomings, butthey are difficult to avoid, given the basic authentication methods Cephuses in the background. Those setting up Ceph systems should be aware ofthese shortcomings.
In particular, arbitrary user machines, especially portable machines, should notbe configured to interact directly with Ceph, since that mode of use wouldrequire the storage of a plaintext authentication key on an insecure machine.Anyone who stole that machine or obtained surreptitious access to it couldobtain the key that will allow them to authenticate their own machines to Ceph.
Rather than permitting potentially insecure machines to access a Ceph objectstore directly, users should be required to sign in to a trusted machine inyour environment using a method that provides sufficient security for yourpurposes. That trusted machine will store the plaintext Ceph keys for thehuman users. A future version of Ceph may address these particularauthentication issues more fully.
At the moment, none of the Ceph authentication protocols provide secrecy formessages in transit. Thus, an eavesdropper on the wire can hear and understandall data sent between clients and servers in Ceph, even if it cannot create oralter them. Further, Ceph does not include options to encrypt user data in theobject store. Users can hand-encrypt and store their own data in the Cephobject store, of course, but Ceph provides no features to perform objectencryption itself. Those storing sensitive data in Ceph should considerencrypting their data before providing it to the Ceph system.