Backing up applications

You back up applications by creating a Backup custom resource (CR).

The Backup CR creates backup files for Kubernetes resources and internal images, on S3 object storage, and snapshots for persistent volumes (PVs), if the cloud provider uses a native snapshot API or the Container Storage Interface (CSI) to create snapshots, such as OpenShift Data Foundation 4. For more information, see CSI volume snapshots.

The CloudStorage API for S3 storage is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information about the support scope of Red Hat Technology Preview features, see https://access.redhat.com/support/offerings/techpreview/.

If your cloud provider has a native snapshot API or supports Container Storage Interface (CSI) snapshots, the Backup CR backs up persistent volumes by creating snapshots. For more information, see the Overview of CSI volume snapshots in the OKD documentation.

If your cloud provider does not support snapshots or if your applications are on NFS data volumes, you can create backups by using Restic.

You can create backup hooks to run commands before or after the backup operation.

You can schedule backups by creating a Schedule CR instead of a Backup CR.

Creating a Backup CR

You back up Kubernetes images, internal images, and persistent volumes (PVs) by creating a Backup custom resource (CR).

Prerequisites

  • You must install the OpenShift API for Data Protection (OADP) Operator.

  • The DataProtectionApplication CR must be in a Ready state.

  • Backup location prerequisites:

    • You must have S3 object storage configured for Velero.

    • You must have a backup location configured in the DataProtectionApplication CR.

  • Snapshot location prerequisites:

    • Your cloud provider must have a native snapshot API or support Container Storage Interface (CSI) snapshots.

    • For CSI snapshots, you must create a VolumeSnapshotClass CR to register the CSI driver.

    • You must have a volume location configured in the DataProtectionApplication CR.

Procedure

  1. Retrieve the backupStorageLocations CRs by entering the following command:

    1. $ oc get backupStorageLocations

    Example output

    1. NAME PHASE LAST VALIDATED AGE DEFAULT
    2. velero-sample-1 Available 11s 31m
  2. Create a Backup CR, as in the following example:

    1. apiVersion: velero.io/v1
    2. kind: Backup
    3. metadata:
    4. name: <backup>
    5. labels:
    6. velero.io/storage-location: default
    7. namespace: openshift-adp
    8. spec:
    9. hooks: {}
    10. includedNamespaces:
    11. - <namespace> (1)
    12. includedResources: [] (2)
    13. excludedResources: [] (3)
    14. storageLocation: <velero-sample-1> (4)
    15. ttl: 720h0m0s
    16. labelSelector: (3)
    17. - matchLabels:
    18. app=<label_1>
    19. - matchLabels:
    20. app=<label_2>
    21. - matchLabels:
    22. app=<label_3>
    23. orlabelSelectors: (4)
    24. - matchLabels:
    25. app=<label_1>
    26. - matchLabels:
    27. app=<label_2>
    28. - matchLabels:
    29. app=<label_3>
    1Specify an array of namespaces to back up.
    2Optional: Specify an array of resources to include in the backup. Resources might be shortcuts (for example, ‘po’ for ‘pods’) or fully-qualified. If unspecified, all resources are included.
    3Optional: Specify an array of resources to exclude from the backup. Resources might be shortcuts (for example, ‘po’ for ‘pods’) or fully-qualified.
    4Specify the name of the backupStorageLocations CR.
  3. Verify that the status of the Backup CR is Completed:

    1. $ oc get backup -n openshift-adp <backup> -o jsonpath='{.status.phase}'

Backing up persistent volumes with CSI snapshots

You back up persistent volumes with Container Storage Interface (CSI) snapshots by editing the VolumeSnapshotClass custom resource (CR) of the cloud storage before you create the Backup CR.

Prerequisites

  • The cloud provider must support CSI snapshots.

  • You must enable CSI in the DataProtectionApplication CR.

Procedure

  • Add the metadata.labels.velero.io/csi-volumesnapshot-class: "true" key-value pair to the VolumeSnapshotClass CR:

    1. apiVersion: snapshot.storage.k8s.io/v1
    2. kind: VolumeSnapshotClass
    3. metadata:
    4. name: <volume_snapshot_class_name>
    5. labels:
    6. velero.io/csi-volumesnapshot-class: "true"
    7. driver: <csi_driver>
    8. deletionPolicy: Retain

You can now create a Backup CR.

Backing up applications with Restic

You back up Kubernetes resources, internal images, and persistent volumes with Restic by editing the Backup custom resource (CR).

You do not need to specify a snapshot location in the DataProtectionApplication CR.

Restic does not support backing up hostPath volumes. For more information, see additional Rustic limitations.

Prerequisites

  • You must install the OpenShift API for Data Protection (OADP) Operator.

  • You must not disable the default Restic installation by setting spec.configuration.restic.enable to false in the DataProtectionApplication CR.

  • The DataProtectionApplication CR must be in a Ready state.

Procedure

  • Edit the Backup CR, as in the following example:

    1. apiVersion: velero.io/v1
    2. kind: Backup
    3. metadata:
    4. name: <backup>
    5. labels:
    6. velero.io/storage-location: default
    7. namespace: openshift-adp
    8. spec:
    9. defaultVolumesToRestic: true (1)
    10. ...
    1Add defaultVolumesToRestic: true to the spec block.

Using Data Mover for CSI snapshots

Data Mover for CSI snapshots is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information about the support scope of Red Hat Technology Preview features, see https://access.redhat.com/support/offerings/techpreview/.

The OADP 1.1.0 Data Mover enables customers to back up container storage interface (CSI) volume snapshots to a remote object store. When Data Mover is enabled, you can restore stateful applications from the store if a failure, accidental deletion, or corruption of the cluster occurs. The OADP 1.1.0 Data Mover solution uses the Restic option of VolSync.

Data Mover supports backup and restore of CSI volume snapshots only.

Currently, Data Mover does not support Google Cloud Storage (GCS) buckets.

Prerequisites

  • You have verified that the StorageClass and VolumeSnapshotClass custom resources (CRs) support CSI.

  • You have verified that only one volumeSnapshotClass CR has the annotation snapshot.storage.kubernetes.io/is-default-class: true.

  • You have verified that only one storageClass CR has the annotation storageclass.kubernetes.io/is-default-class: true.

  • You have included the label velero.io/csi-volumesnapshot-class: 'true' in your VolumeSnapshotClass CR.

  • You have installed the VolSync Operator by using the Operator Lifecycle Manager (OLM).

    The VolSync Operator is required only for use with the Technology Preview Data Mover. The Operator is not required for using OADP production features.

  • You have installed the OADP operator by using OLM.

Procedure

  1. Configure a Restic secret by creating a .yaml file as following:

    1. apiVersion: v1
    2. kind: Secret
    3. metadata:
    4. name: <secret_name>
    5. namespace: openshift-adp
    6. type: Opaque
    7. stringData:
    8. RESTIC_PASSWORD: <secure_restic_password>

    By default, the Operator looks for a secret named dm-credential. If you are using a different name, you need to specify the name through a Data Protection Application (DPA) CR using dpa.spec.features.dataMover.credentialName.

  2. Create a DPA CR similar to the following example. The default plugins include CSI.

    Example Data Protection Application (DPA) CR

    1. apiVersion: oadp.openshift.io/v1alpha1
    2. kind: DataProtectionApplication
    3. metadata:
    4. name: velero-sample
    5. namespace: openshift-adp
    6. spec:
    7. features:
    8. dataMover:
    9. enable: true
    10. credentialName: <secret_name> (1)
    11. backupLocations:
    12. - velero:
    13. config:
    14. profile: default
    15. region: us-east-1
    16. credential:
    17. key: cloud
    18. name: cloud-credentials
    19. default: true
    20. objectStorage:
    21. bucket: <bucket_name>
    22. prefix: <bucket_prefix>
    23. provider: aws
    24. configuration:
    25. restic:
    26. enable: <true_or_false>
    27. velero:
    28. defaultPlugins:
    29. - openshift
    30. - aws
    31. - csi
    1Add the Restic secret name from the previous step. If this is not done, the default secret name dm-credential is used.

    The OADP Operator installs two custom resource definitions (CRDs), VolumeSnapshotBackup and VolumeSnapshotRestore.

    Example VolumeSnapshotBackup CRD

    1. apiVersion: datamover.oadp.openshift.io/v1alpha1
    2. kind: VolumeSnapshotBackup
    3. metadata:
    4. name: <vsb_name>
    5. namespace: <namespace_name> (1)
    6. spec:
    7. volumeSnapshotContent:
    8. name: <snapcontent_name>
    9. protectedNamespace: <adp_namespace>
    10. resticSecretRef:
    11. name: <restic_secret_name>
    1Specify the namespace where the volume snapshot exists.

    Example VolumeSnapshotRestore CRD

    1. apiVersion: datamover.oadp.openshift.io/v1alpha1
    2. kind: VolumeSnapshotRestore
    3. metadata:
    4. name: <vsr_name>
    5. namespace: <namespace_name> (1)
    6. spec:
    7. protectedNamespace: <protected_ns> (2)
    8. resticSecretRef:
    9. name: <restic_secret_name>
    10. volumeSnapshotMoverBackupRef:
    11. sourcePVCData:
    12. name: <source_pvc_name>
    13. size: <source_pvc_size>
    14. resticrepository: <your_restic_repo>
    15. volumeSnapshotClassName: <vsclass_name>
    1Specify the namespace where the volume snapshot exists.
    2Specify the namespace where the Operator is installed. The default is openshift-adp.
  3. You can back up a volume snapshot by performing the following steps:

    1. Create a backup CR:

      1. apiVersion: velero.io/v1
      2. kind: Backup
      3. metadata:
      4. name: <backup_name>
      5. namespace: <protected_ns> (1)
      6. spec:
      7. includedNamespaces:
      8. - <app_ns>
      9. storageLocation: velero-sample-1
      1Specify the namespace where the Operator is installed. The default namespace is openshift-adp.
    2. Wait up to 10 minutes and check whether the VolumeSnapshotBackup CR status is Completed by entering the following commands:

      1. $ oc get vsb -n <app_ns>
      1. $ oc get vsb <vsb_name> -n <app_ns> -o jsonpath="{.status.phase}"

      A snapshot is created in the object store was configured in the DPA.

      If the status of the VolumeSnapshotBackup CR becomes Failed, refer to the Velero logs for troubleshooting.

  4. You can restore a volume snapshot by performing the following steps:

    1. Delete the application namespace and the volumeSnapshotContent that was created by the Velero CSI plugin.

    2. Create a Restore CR and set restorePVs to true.

      Example Restore CR

      1. apiVersion: velero.io/v1
      2. kind: Restore
      3. metadata:
      4. name: <restore_name>
      5. namespace: <protected_ns>
      6. spec:
      7. backupName: <previous_backup_name>
      8. restorePVs: true
    3. Wait up to 10 minutes and check whether the VolumeSnapshotRestore CR status is Completed by entering the following command:

      1. $ oc get vsr -n <app_ns>
      1. $ oc get vsr <vsr_name> -n <app_ns> -o jsonpath="{.status.phase}"
    4. Check whether your application data and resources have been restored.

      If the status of the VolumeSnapshotRestore CR becomes ‘Failed’, refer to the Velero logs for troubleshooting.

Additional resources

Creating backup hooks

You create backup hooks to run commands in a container in a pod by editing the Backup custom resource (CR).

Pre hooks run before the pod is backed up. Post hooks run after the backup.

Procedure

  • Add a hook to the spec.hooks block of the Backup CR, as in the following example:

    1. apiVersion: velero.io/v1
    2. kind: Backup
    3. metadata:
    4. name: <backup>
    5. namespace: openshift-adp
    6. spec:
    7. hooks:
    8. resources:
    9. - name: <hook_name>
    10. includedNamespaces:
    11. - <namespace> (1)
    12. excludedNamespaces: (2)
    13. - <namespace>
    14. includedResources: []
    15. - pods (3)
    16. excludedResources: [] (4)
    17. labelSelector: (5)
    18. matchLabels:
    19. app: velero
    20. component: server
    21. pre: (6)
    22. - exec:
    23. container: <container> (7)
    24. command:
    25. - /bin/uname (8)
    26. - -a
    27. onError: Fail (9)
    28. timeout: 30s (10)
    29. post: (11)
    30. ...
    1Optional: You can specify namespaces to which the hook applies. If this value is not specified, the hook applies to all namespaces.
    2Optional: You can specify namespaces to which the hook does not apply.
    3Currently, pods are the only supported resource that hooks can apply to.
    4Optional: You can specify resources to which the hook does not apply.
    5Optional: This hook only applies to objects matching the label. If this value is not specified, the hook applies to all namespaces.
    6Array of hooks to run before the backup.
    7Optional: If the container is not specified, the command runs in the first container in the pod.
    8This is the entrypoint for the init container being added.
    9Allowed values for error handling are Fail and Continue. The default is Fail.
    10Optional: How long to wait for the commands to run. The default is 30s.
    11This block defines an array of hooks to run after the backup, with the same parameters as the pre-backup hooks.

Scheduling backups

You schedule backups by creating a Schedule custom resource (CR) instead of a Backup CR.

Leave enough time in your backup schedule for a backup to finish before another backup is created.

For example, if a backup of a namespace typically takes 10 minutes, do not schedule backups more frequently than every 15 minutes.

Prerequisites

  • You must install the OpenShift API for Data Protection (OADP) Operator.

  • The DataProtectionApplication CR must be in a Ready state.

Procedure

  1. Retrieve the backupStorageLocations CRs:

    1. $ oc get backupStorageLocations

    Example output

    1. NAME PHASE LAST VALIDATED AGE DEFAULT
    2. velero-sample-1 Available 11s 31m
  2. Create a Schedule CR, as in the following example:

    1. $ cat << EOF | oc apply -f -
    2. apiVersion: velero.io/v1
    3. kind: Schedule
    4. metadata:
    5. name: <schedule>
    6. namespace: openshift-adp
    7. spec:
    8. schedule: 0 7 * * * (1)
    9. template:
    10. hooks: {}
    11. includedNamespaces:
    12. - <namespace> (2)
    13. storageLocation: <velero-sample-1> (3)
    14. defaultVolumesToRestic: true (4)
    15. ttl: 720h0m0s
    16. EOF
    1cron expression to schedule the backup, for example, 0 7 * to perform a backup every day at 7:00.
    2Array of namespaces to back up.
    3Name of the backupStorageLocations CR.
    4Optional: Add the defaultVolumesToRestic: true key-value pair if you are backing up volumes with Restic.
  3. Verify that the status of the Schedule CR is Completed after the scheduled backup runs:

    1. $ oc get schedule -n openshift-adp <schedule> -o jsonpath='{.status.phase}'