Using Storage Classes for Dynamic Provisioning

Overview

In these examples we will walk through a few scenarios of various configuratons of StorageClasses and Dynamic Provisioning using Google Cloud Platform Compute Engine (GCE). These examples assume some familiarity with Kubernetes, GCE and Persistent Disks and OKD is installed and properly configured to use GCE.

Scenario 1: Basic Dynamic Provisioning with Two Types of StorageClasses

StorageClasses can be used to differentiate and delineate storage levels and usages. In this case, the cluster-admin or storage-admin sets up two distinct classes of storage in GCE.

  • slow: Cheap, efficient, and optimized for sequential data operations (slower reading and writing)

  • fast: Optimized for higher rates of random IOPS and sustained throughput (faster reading and writing)

By creating these StorageClasses, the cluster-admin or storage-admin allows users to create claims requesting a particular level or service of StorageClass.

Example 1. StorageClass Slow Object Definitions

  1. kind: StorageClass
  2. apiVersion: storage.k8s.io/v1
  3. metadata:
  4.   name: slow (1)
  5. provisioner: kubernetes.io/gce-pd (2)
  6. parameters:
  7.   type: pd-standard (3)
  8.   zone: us-east1-d  (4)
1Name of the StorageClass.
2The provisioner plug-in to be used. This is a required field for StorageClasses.
3PD type. This example uses pd-standard, which has a slightly lower cost, rate of sustained IOPS, and throughput versus pd-ssd, which carries more sustained IOPS and throughput.
4The zone is required.

Example 2. StorageClass Fast Object Definition

  1. kind: StorageClass
  2. apiVersion: storage.k8s.io/v1
  3. metadata:
  4.   name: fast
  5. provisioner: kubernetes.io/gce-pd
  6. parameters:
  7.   type: pd-ssd
  8.   zone: us-east1-d

As a cluster-admin or storage-admin, save both definitions as YAML files. For example, slow-gce.yaml and fast-gce.yaml. Then create the StorageClasses.

  1. # oc create -f slow-gce.yaml
  2. storageclass "slow" created
  4. # oc create -f fast-gce.yaml
  5. storageclass "fast" created
  7. # oc get storageclass
  8. NAME       TYPE
  9. fast       kubernetes.io/gce-pd
  10. slow       kubernetes.io/gce-pd

cluster-admin or storage-admin users are responsible for relaying the correct StorageClass name to the correct users, groups, and projects.

As a regular user, create a new project:

  1. # oc new-project rh-eng

Create the claim YAML definition, save it to a file (pvc-fast.yaml):

  1. apiVersion: v1
  2. kind: PersistentVolumeClaim
  3. metadata:
  4.  name: pvc-engineering
  5. spec:
  6.  accessModes:
  7.   - ReadWriteMany
  8.  resources:
  9.    requests:
  10.      storage: 10Gi
  11.  storageClassName: fast

Add the claim with the oc create command:

  1. # oc create -f pvc-fast.yaml
  2. persistentvolumeclaim "pvc-engineering" created

Check to see if your claim is bound:

  1. # oc get pvc
  2. NAME              STATUS    VOLUME                                     CAPACITY   ACCESSMODES   AGE
  3. pvc-engineering   Bound     pvc-e9b4fef7-8bf7-11e6-9962-42010af00004   10Gi       RWX           2m

Since this claim was created and bound in the rh-eng project, it can be shared by any user in the same project.

As a cluster-admin or storage-admin user, view the recent dynamically provisioned Persistent Volume (PV).

  1. # oc get pv
  2. NAME                                       CAPACITY   ACCESSMODES   RECLAIMPOLICY   STATUS    CLAIM                     REASON    AGE
  3. pvc-e9b4fef7-8bf7-11e6-9962-42010af00004   10Gi       RWX           Delete          Bound     rh-eng/pvc-engineering              5m

Notice the RECLAIMPOLICY is Delete by default for all dynamically provisioned volumes. This means the volume only lasts as long as the claim still exists in the system. If you delete the claim, the volume is also deleted and all data on the volume is lost.

Finally, check the GCE console. The new disk has been created and is ready for use.

  1. kubernetes-dynamic-pvc-e9b4fef7-8bf7-11e6-9962-42010af00004     SSD persistent disk     10 GB     us-east1-d

Pods can now reference the persistent volume claim and start using the volume.

Scenario 2: How to enable Default StorageClass behavior for a Cluster

In this example, a cluster-admin or storage-admin enables a default storage class for all other users and projects that do not implicitly specify a StorageClass in their claim. This is useful for a cluster-admin or storage-admin to provide easy management of a storage volume without having to set up or communicate specialized StorageClasses across the cluster.

This example builds upon Scenario 1: Basic Dynamic Provisioning with Two Types of StorageClasses. The cluster-admin or storage-admin will create another StorageClass for designation as the default StorageClass.

Example 3. Default StorageClass Object Definition

  1. kind: StorageClass
  2. apiVersion: storage.k8s.io/v1
  3. metadata:
  4.   name: generic (1)
  5.   annotations:
  6.     storageclass.kubernetes.io/is-default-class: "true" (2)
  7. provisioner: kubernetes.io/gce-pd
  8. parameters:
  9.   type: pd-standard
  10.   zone: us-east1-d
1Name of the StorageClass, which needs to be unique in the cluster.
2Annotation that marks this StorageClass as the default class. You must use “true” quoted in this version of the API. Without this annotation, OKD considers this not the default StorageClass.

As a cluster-admin or storage-admin save the definition to a YAML file (generic-gce.yaml), then create the StorageClasses:

  1. # oc create -f generic-gce.yaml
  2. storageclass "generic" created
  4. # oc get storageclass
  5. NAME       TYPE
  6. generic    kubernetes.io/gce-pd
  7. fast       kubernetes.io/gce-pd
  8. slow       kubernetes.io/gce-pd

As a regular user, create a new claim definition without any StorageClass requirement and save it to a file (generic-pvc.yaml).

Example 4. default Storage Claim Object Definition

  1. apiVersion: v1
  2. kind: PersistentVolumeClaim
  3. metadata:
  4.  name: pvc-engineering2
  5. spec:
  6.  accessModes:
  7.   - ReadWriteMany
  8.  resources:
  9.    requests:
  10.      storage: 5Gi

Execute it and check the claim is bound:

  1. # oc create -f generic-pvc.yaml
  2. persistentvolumeclaim "pvc-engineering2" created
  3.                                                                    3s
  4. # oc get pvc
  5. NAME               STATUS    VOLUME                                     CAPACITY   ACCESSMODES   AGE
  6. pvc-engineering    Bound     pvc-e9b4fef7-8bf7-11e6-9962-42010af00004   10Gi       RWX           41m
  7. pvc-engineering2   Bound     pvc-a9f70544-8bfd-11e6-9962-42010af00004   5Gi        RWX           7s  (1)
1pvc-engineering2 is bound to a dynamically provisioned Volume by default.

As a cluster-admin or storage-admin, view the Persistent Volumes defined so far:

  1. # oc get pv
  2. NAME                                       CAPACITY   ACCESSMODES   RECLAIMPOLICY   STATUS    CLAIM                     REASON    AGE
  3. pvc-a9f70544-8bfd-11e6-9962-42010af00004   5Gi        RWX           Delete          Bound     rh-eng/pvc-engineering2             5m (1)
  4. pvc-ba4612ce-8b4d-11e6-9962-42010af00004   5Gi        RWO           Delete          Bound     mytest/gce-dyn-claim1               21h
  5. pvc-e9b4fef7-8bf7-11e6-9962-42010af00004   10Gi       RWX           Delete          Bound     rh-eng/pvc-engineering              46m (2)
1This PV was bound to our default dynamic volume from the default StorageClass.
2This PV was bound to our first PVC from Scenario 1: Basic Dynamic Provisioning with Two Types of StorageClasses with our fast StorageClass.

Create a manually provisioned disk using GCE (not dynamically provisioned). Then create a Persistent Volume that connects to the new GCE disk (pv-manual-gce.yaml).

Example 5. Manual PV Object Defition

  1. apiVersion: v1
  2. kind: PersistentVolume
  3. metadata:
  4.  name: pv-manual-gce
  5. spec:
  6.  capacity:
  7.    storage: 35Gi
  8.  accessModes:
  9.    - ReadWriteMany
  10.  gcePersistentDisk:
  11.    readOnly: false
  12.    pdName: the-newly-created-gce-PD
  13.    fsType: ext4

Execute the object definition file:

  1. # oc create -f pv-manual-gce.yaml

Now view the PVs again. Notice that a pv-manual-gce volume is Available.

  1. # oc get pv
  2. NAME                                       CAPACITY   ACCESSMODES   RECLAIMPOLICY   STATUS      CLAIM                     REASON    AGE
  3. pv-manual-gce                              35Gi       RWX           Retain          Available                                       4s
  4. pvc-a9f70544-8bfd-11e6-9962-42010af00004   5Gi        RWX           Delete          Bound       rh-eng/pvc-engineering2             12m
  5. pvc-ba4612ce-8b4d-11e6-9962-42010af00004   5Gi        RWO           Delete          Bound       mytest/gce-dyn-claim1               21h
  6. pvc-e9b4fef7-8bf7-11e6-9962-42010af00004   10Gi       RWX           Delete          Bound       rh-eng/pvc-engineering              53m

Now create another claim identical to the generic-pvc.yaml PVC definition but change the name and do not set a storage class name.

Example 6. Claim Object Definition

  1. apiVersion: v1
  2. kind: PersistentVolumeClaim
  3. metadata:
  4.  name: pvc-engineering3
  5. spec:
  6.  accessModes:
  7.   - ReadWriteMany
  8.  resources:
  9.    requests:
  10.      storage: 15Gi

Because default StorageClass is enabled in this instance, the manually created PV does not satisfy the claim request. The user receives a new dynamically provisioned Persistent Volume.

  1. # oc get pvc
  2. NAME               STATUS    VOLUME                                     CAPACITY   ACCESSMODES   AGE
  3. pvc-engineering    Bound     pvc-e9b4fef7-8bf7-11e6-9962-42010af00004   10Gi       RWX           1h
  4. pvc-engineering2   Bound     pvc-a9f70544-8bfd-11e6-9962-42010af00004   5Gi        RWX           19m
  5. pvc-engineering3   Bound     pvc-6fa8e73b-8c00-11e6-9962-42010af00004   15Gi       RWX           6s

Since the default StorageClass is enabled on this system, for the manually created Persistent Volume to get bound by the above claim and not have a new dynamic provisioned volume be bound, the PV would need to have been created in the default StorageClass.

Since the default StorageClass is enabled on this system, you would need to create the PV in the default StorageClass for the manually created Persistent Volume to get bound to the above claim and not have a new dynamic provisioned volume bound to the claim.

To fix this, the cluster-admin or storage-admin user simply needs to create another GCE disk or delete the first manual PV and use a PV object definition that assigns a StorageClass name (pv-manual-gce2.yaml) if necessary:

Example 7. Manual PV Spec with default StorageClass name

  1. apiVersion: v1
  2. kind: PersistentVolume
  3. metadata:
  4.  name: pv-manual-gce2
  5. spec:
  6.  capacity:
  7.    storage: 35Gi
  8.  accessModes:
  9.    - ReadWriteMany
  10.  gcePersistentDisk:
  11.    readOnly: false
  12.    pdName: the-newly-created-gce-PD
  13.    fsType: ext4
  14.  storageClassName: generic (1)
1The name for previously created generic StorageClass.

Execute the object definition file:

  1. # oc create -f pv-manual-gce2.yaml

List the PVs:

  1. # oc get pv
  2. NAME                                       CAPACITY   ACCESSMODES   RECLAIMPOLICY   STATUS      CLAIM                     REASON    AGE
  3. pv-manual-gce                              35Gi       RWX           Retain          Available                                       4s (1)
  4. pv-manual-gce2                             35Gi       RWX           Retain          Bound       rh-eng/pvc-engineering3             4s (2)
  5. pvc-a9f70544-8bfd-11e6-9962-42010af00004   5Gi        RWX           Delete          Bound       rh-eng/pvc-engineering2             12m
  6. pvc-ba4612ce-8b4d-11e6-9962-42010af00004   5Gi        RWO           Delete          Bound       mytest/gce-dyn-claim1               21h
  7. pvc-e9b4fef7-8bf7-11e6-9962-42010af00004   10Gi       RWX           Delete          Bound       rh-eng/pvc-engineering              53m
1The original manual PV, still unbound and Available. This is because it was not created in the default StorageClass.
2The second PVC (other than the name) is bound to the Available manually created PV pv-manual-gce2.

Notice that all dynamically provisioned volumes by default have a RECLAIMPOLICY of Delete. Once the PVC dynamically bound to the PV is deleted, the GCE volume is deleted and all data is lost. However, the manually created PV has a default RECLAIMPOLICY of Retain.