CephFS Quick Start

To use the CephFS Quick Start guide, you must have executed theprocedures in the Storage Cluster Quick Start guide first. Execute thisquick start on the admin host.

Prerequisites

  • Verify that you have an appropriate version of the Linux kernel.See OS Recommendations for details.
  1. lsb_release -a
  2. uname -r
  • On the admin node, use ceph-deploy to install Ceph on yourceph-client node.
  1. ceph-deploy install ceph-client

  • Optionally, if you want a FUSE-mounted file system, you would need toinstall ceph-fuse package as well.

  • Ensure that the Ceph Storage Cluster is running and in an active +clean state.

  1. ceph -s [-m {monitor-ip-address}] [-k {path/to/ceph.client.admin.keyring}]

Deploy Metadata Server

All metadata operations in CephFS happen via a metadata server, so you need atleast one metadata server. Execute the following to create a metadata server:

  1. ceph-deploy mds create {ceph-node}

For example:

  1. ceph-deploy mds create node1

Now, your Ceph cluster would look like this:

Filesystem Quick Start - 图1

Create a File System

You have already created an MDS (Storage Cluster Quick Start) but it will notbecome active until you create some pools and a file system. SeeCreate a Ceph file system.

  1. ceph osd pool create cephfs_data 32
  2. ceph osd pool create cephfs_meta 32
  3. ceph fs new mycephfs cephfs_meta cephfs_data

Note

In case you have multiple Ceph applications and/or have multipleCephFSs on the same cluster, it would be easier to name your pools as<application>.<fs-name>.<pool-name>. In that case, the above pools wouldbe named as cephfs.mycehfs.data and cephfs.mycehfs.meta.

Quick word about Pools and PGs

Replication Number/Pool Size

Since the default replication number/size is 3, you’d need 3 OSDs to getactive+clean for all PGs. Alternatively, you may change the replicationnumber for the pool to match the number of OSDs:

  1. ceph osd pool set cephfs_data size {number-of-osds}
  2. ceph osd pool set cephfs_meta size {number-of-osds}

Usually, setting pg_num to 32 gives a perfectly healthy cluster. To pickappropriate value for pg_num, refer Placement Group. You can also usepg_autoscaler plugin instead. Introduced by Nautilus release, it canautomatically increase/decrease value of pg_num; refer thePlacement Group to find out more about it.

When all OSDs are on the same node…

And, in case you have deployed all of the OSDs on the same node, you would needto create a new CRUSH rule to replicate data across OSDs and set the rule on theCephFS pools, since the default CRUSH rule is to replicate data acrossdifferent nodes:

  1. ceph osd crush rule create-replicated rule_foo default osd
  2. ceph osd pool set cephfs_data crush_rule rule_foo
  3. ceph osd pool set cephfs_meta crush_rule rule_foo

Using Erasure Coded pools

You may also use Erasure Coded pools which can be more effecient andcost-saving since they allow stripping object data across OSDs andreplicating these stripes with encoded redundancy information. The numberof OSDs across which the data is stripped is k and number of replica is m.You’ll need to pick up these values before creating CephFS pools. Thefollowing commands create a erasure code profile, creates a pool that’lluse it and then enables it on the pool:

  1. ceph osd erasure-code-profile set ec-42-profile k=4 m=2 crush-failure-domain=host crush-device-class=ssd
  2. ceph osd pool create cephfs_data_ec42 64 erasure ec-42-profile
  3. ceph osd pool set cephfs_data_ec42 allow_ec_overwrites true
  4. ceph fs add_data_pool mycephfs cephfs_data_ec42

You can also mark directories so that they are only stored on certain pools:

  1. setfattr -n ceph.dir.layout -v pool=cephfs_data_ec42 /mnt/mycephfs/logs

This way you can choose the replication strategy for each directory on yourCeph file system.

Note

Erasure Coded pools can not be used for CephFS metadata pools.

Erasure coded pool were introduced in Firefly and could be used directly byCephFS Luminous onwards. Refer this articleby Sage Weil to understand EC, it’s background, limitations and other detailsin Ceph’s context. Read more about Erasure Code here.

Mounting the File System

Using Kernel Driver

The command to mount CephFS using kernel driver looks like this:

  1. sudo mount -t ceph :{path-to-mounted} {mount-point} -o name={user-name}
  2. sudo mount -t ceph :/ /mnt/mycephfs -o name=admin   # usable version

{path-to-be-mounted} is the path within CephFS that will be mounted,{mount-point} is the point in your file system upon which CephFS will bemounted and {user-name} is the name of CephX user that has theauthorization to mount CephFS on the machine. Following command is theextended form, however these extra details are automatically figured out byby the mount.ceph helper program:

  1. sudo mount -t ceph {ip-address-of-MON}:{port-number-of-MON}:{path-to-be-mounted} -o name={user-name},secret={secret-key} {mount-point}

If you have multiple file systems on your cluster you would need to passmds_namespace={fs-name} option to -o option to the mount command:

  1. sudo mount -t ceph :/ /mnt/kcephfs2 -o name=admin,mds_namespace=mycephfs2

Refer mount.ceph man page and Mount CephFS using Kernel Driver to readmore about this.

Using FUSE

To mount CephFS using FUSE (Filesystem in User Space) run:

  1. sudo ceph-fuse /mnt/mycephfs

To mount a particular directory within CephFS you can use -r:

  1. sudo ceph-fuse -r {path-to-be-mounted} /mnt/mycephfs

If you have multiple file systems on your cluster you would need to pass—client_mds_namespace {fs-name} to the ceph-fuse command:

  1. sudo ceph-fuse /mnt/mycephfs2 --client_mds_namespace mycephfs2

Refer ceph-fuse man page and Mount CephFS using FUSE to read more aboutthis.

Note

Mount the CephFS file system on the admin node, not the server node.

Additional Information

See CephFS for additional information. See Troubleshooting if youencounter trouble.