Using Data Mover for CSI snapshots

The OADP Data Mover enables customers to back up Container Storage Interface (CSI) volume snapshots to a remote object store. When Data Mover is enabled, you can restore stateful applications, using CSI volume snapshots pulled from the object store if a failure, accidental deletion, or corruption of the cluster occurs.

The Data Mover solution uses the Restic option of VolSync.

Data Mover supports backup and restore of CSI volume snapshots only.

In OADP 1.2 Data Mover VolumeSnapshotBackups (VSBs) and VolumeSnapshotRestores (VSRs) are queued using the VolumeSnapshotMover (VSM). The VSM’s performance is improved by specifying a concurrent number of VSBs and VSRs simultaneously InProgress. After all async plugin operations are complete, the backup is marked as complete.

The OADP 1.1 Data Mover is a Technology Preview feature.

The OADP 1.2 Data Mover has significantly improved features and performances, but is still a Technology Preview feature.

The OADP Data Mover is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.

Red Hat recommends that customers who use OADP 1.2 Data Mover in order to back up and restore ODF CephFS volumes, upgrade or install OKD version 4.12 or later for improved performance. OADP Data Mover can leverage CephFS shallow volumes in OKD version 4.12 or later, which based on our testing, can improve the performance of backup times.

Prerequisites

  • You have verified that the StorageClass and VolumeSnapshotClass custom resources (CRs) support CSI.

  • You have verified that only one VolumeSnapshotClass CR has the annotation snapshot.storage.kubernetes.io/is-default-class: "true".

    In OKD version 4.12 or later, verify that this is the only default VolumeSnapshotClass.

  • You have verified that deletionPolicy of the VolumeSnapshotClass CR is set to Retain.

  • You have verified that only one StorageClass CR has the annotation storageclass.kubernetes.io/is-default-class: "true".

  • You have included the label velero.io/csi-volumesnapshot-class: "true" in your VolumeSnapshotClass CR.

  • You have verified that the OADP namespace has the annotation oc annotate --overwrite namespace/openshift-adp volsync.backube/privileged-movers="true".

    In OADP 1.1 the above setting is mandatory.

    In OADP 1.2 the privileged-movers setting is not required in most scenarios. The restoring container permissions should be adequate for the Volsync copy. In some user scenarios, there may be permission errors that the privileged-mover= true setting should resolve.

  • You have installed the VolSync Operator by using the Operator Lifecycle Manager (OLM).

    The VolSync Operator is required for using OADP Data Mover.

  • You have installed the OADP operator by using OLM.

Procedure

  1. Configure a Restic secret by creating a .yaml file as following:

    1. apiVersion: v1
    2. kind: Secret
    3. metadata:
    4. name: <secret_name>
    5. namespace: openshift-adp
    6. type: Opaque
    7. stringData:
    8. RESTIC_PASSWORD: <secure_restic_password>

    By default, the Operator looks for a secret named dm-credential. If you are using a different name, you need to specify the name through a Data Protection Application (DPA) CR using dpa.spec.features.dataMover.credentialName.

  2. Create a DPA CR similar to the following example. The default plugins include CSI.

    Example Data Protection Application (DPA) CR

    1. apiVersion: oadp.openshift.io/v1alpha1
    2. kind: DataProtectionApplication
    3. metadata:
    4. name: velero-sample
    5. namespace: openshift-adp
    6. spec:
    7. backupLocations:
    8. - velero:
    9. config:
    10. profile: default
    11. region: us-east-1
    12. credential:
    13. key: cloud
    14. name: cloud-credentials
    15. default: true
    16. objectStorage:
    17. bucket: <bucket_name>
    18. prefix: <bucket-prefix>
    19. provider: aws
    20. configuration:
    21. restic:
    22. enable: <true_or_false>
    23. velero:
    24. itemOperationSyncFrequency: "10s"
    25. defaultPlugins:
    26. - openshift
    27. - aws
    28. - csi
    29. - vsm (1)
    30. features:
    31. dataMover:
    32. credentialName: restic-secret
    33. enable: true
    34. maxConcurrentBackupVolumes: "3" (2)
    35. maxConcurrentRestoreVolumes: "3" (3)
    36. pruneInterval: "14" (4)
    37. volumeOptions: (5)
    38. sourceVolumeOptions:
    39. accessMode: ReadOnlyMany
    40. cacheAccessMode: ReadWriteOnce
    41. cacheCapacity: 2Gi
    42. destinationVolumeOptions:
    43. storageClass: other-storageclass-name
    44. cacheAccessMode: ReadWriteMany
    45. snapshotLocations:
    46. - velero:
    47. config:
    48. profile: default
    49. region: us-west-2
    50. provider: aws
    1OADP 1.2 only.
    2OADP 1.2 only. Optional: Specify the upper limit of the number of snapshots allowed to be queued for backup. The default value is 10.
    3OADP 1.2 only. Optional: Specify the upper limit of the number of snapshots allowed to be queued for restore. The default value is 10.
    4OADP 1.2 only. Optional: Specify the number of days, between running Restic pruning on the repository. The prune operation repacks the data to free space, but it can also generate significant I/O traffic as a part of the process. Setting this option allows a trade-off between storage consumption, from no longer referenced data, and access costs.
    5OADP 1.2 only. Optional: Specify VolumeSync volume options for backup and restore.

    The OADP Operator installs two custom resource definitions (CRDs), VolumeSnapshotBackup and VolumeSnapshotRestore.

    Example VolumeSnapshotBackup CRD

    1. apiVersion: datamover.oadp.openshift.io/v1alpha1
    2. kind: VolumeSnapshotBackup
    3. metadata:
    4. name: <vsb_name>
    5. namespace: <namespace_name> (1)
    6. spec:
    7. volumeSnapshotContent:
    8. name: <snapcontent_name>
    9. protectedNamespace: <adp_namespace> (2)
    10. resticSecretRef:
    11. name: <restic_secret_name>
    1Specify the namespace where the volume snapshot exists.
    2Specify the namespace where the OADP Operator is installed. The default is openshift-adp.

    Example VolumeSnapshotRestore CRD

    1. apiVersion: datamover.oadp.openshift.io/v1alpha1
    2. kind: VolumeSnapshotRestore
    3. metadata:
    4. name: <vsr_name>
    5. namespace: <namespace_name> (1)
    6. spec:
    7. protectedNamespace: <protected_ns> (2)
    8. resticSecretRef:
    9. name: <restic_secret_name>
    10. volumeSnapshotMoverBackupRef:
    11. sourcePVCData:
    12. name: <source_pvc_name>
    13. size: <source_pvc_size>
    14. resticrepository: <your_restic_repo>
    15. volumeSnapshotClassName: <vsclass_name>
    1Specify the namespace where the volume snapshot exists.
    2Specify the namespace where the OADP Operator is installed. The default is openshift-adp.
  3. You can back up a volume snapshot by performing the following steps:

    1. Create a backup CR:

      1. apiVersion: velero.io/v1
      2. kind: Backup
      3. metadata:
      4. name: <backup_name>
      5. namespace: <protected_ns> (1)
      6. spec:
      7. includedNamespaces:
      8. - <app_ns> (2)
      9. storageLocation: velero-sample-1
      1Specify the namespace where the Operator is installed. The default namespace is openshift-adp.
      2Specify the application namespace or namespaces to be backed up.
    2. Wait up to 10 minutes and check whether the VolumeSnapshotBackup CR status is Completed by entering the following commands:

      1. $ oc get vsb -n <app_ns>
      1. $ oc get vsb <vsb_name> -n <app_ns> -o jsonpath="{.status.phase}"

      A snapshot is created in the object store was configured in the DPA.

      If the status of the VolumeSnapshotBackup CR becomes Failed, refer to the Velero logs for troubleshooting.

  4. You can restore a volume snapshot by performing the following steps:

    1. Delete the application namespace and the VolumeSnapshotContent that was created by the Velero CSI plugin.

    2. Create a Restore CR and set restorePVs to true.

      Example Restore CR

      1. apiVersion: velero.io/v1
      2. kind: Restore
      3. metadata:
      4. name: <restore_name>
      5. namespace: <protected_ns>
      6. spec:
      7. backupName: <previous_backup_name>
      8. restorePVs: true
    3. Wait up to 10 minutes and check whether the VolumeSnapshotRestore CR status is Completed by entering the following command:

      1. $ oc get vsr -n <app_ns>
      1. $ oc get vsr <vsr_name> -n <app_ns> -o jsonpath="{.status.phase}"
    4. Check whether your application data and resources have been restored.

      If the status of the VolumeSnapshotRestore CR becomes ‘Failed’, refer to the Velero logs for troubleshooting.