A backup in Longhorn represents a volume state (a Snapshot) at a given time, stored in the secondary storage (backupstore in the Longhorn world) which is outside of the Longhorn system. Backup creation will involve copying the data through the network, so it will take time.

These operations happen automatically, but can also be done as needed.

A corresponding snapshot is needed for creating a backup. And user can choose to backup any snapshot previous created.

Set BackupTarget

The user can setup a S3 or NFS type backupstore to store the backups of Longhorn volumes.

If the user doesn’t have access to AWS S3 or want to give a try first, we’ve also provided a way to setup a local S3 testing backupstore using Minio.

Setup AWS S3 backupstore

  1. Create a new bucket in AWS S3.

  2. Follow the guide to create a new AWS IAM user, with the following permissions set:

  1. {
  2.     "Version": "2012-10-17",
  3.     "Statement": [
  4.         {
  5.             "Sid": "GrantLonghornBackupstoreAccess0",
  6.             "Effect": "Allow",
  7.             "Action": [
  8.                 "s3:PutObject",
  9.                 "s3:GetObject",
  10.                 "s3:ListBucket",
  11.                 "s3:DeleteObject"
  12.             ],
  13.             "Resource": [
  14.                 "arn:aws:s3:::<your-bucket-name>",
  15.                 "arn:aws:s3:::<your-bucket-name>/*"
  16.             ]
  17.         }
  18.     ]
  19. }
  1. Create a Kubernetes secret with a name such as aws-secret in the namespace where longhorn is placed(longhorn-system by default). Put the following keys in the secret:
  1. AWS_ACCESS_KEY_ID: <your_aws_access_key_id>
  2. AWS_SECRET_ACCESS_KEY: <your_aws_secret_access_key>
  1. Go to the Longhorn UI and set Settings/General/BackupTarget to
  1. s3://<your-bucket-name>@<your-aws-region>/

Pay attention that you should have / at the end, otherwise you will get an error.

Also please make sure you’ve set <your-aws-region> in the URL.

For example, for Google Cloud Storage, you can find the region code here: https://cloud.google.com/storage/docs/locations

  1. Set Settings/General/BackupTargetSecret to
  1. aws-secret

Your secret name with AWS keys from 3rd point.

Setup a local testing backupstore

We provides two testing purpose backupstore based on NFS server and Minio S3 server for testing, in ./deploy/backupstores.

Use following command to setup a Minio S3 server for BackupStore after longhorn-system was created.

  1. kubectl create -f https://raw.githubusercontent.com/longhorn/longhorn/master/deploy/backupstores/minio-backupstore.yaml

Now set Settings/General/BackupTarget to

  1. s3://backupbucket@us-east-1/

And Setttings/General/BackupTargetSecret to

  1. minio-secret

Click the Backup tab in the UI, it should report an empty list without error out.

The minio-secret yaml looks like this:

  1. apiVersion: v1
  2. kind: Secret
  3. metadata:
  4.   name: minio-secret
  5.   namespace: longhorn-system
  6. type: Opaque
  7. data:
  8.   AWS_ACCESS_KEY_ID: bG9uZ2hvcm4tdGVzdC1hY2Nlc3Mta2V5 # longhorn-test-access-key
  9.   AWS_SECRET_ACCESS_KEY: bG9uZ2hvcm4tdGVzdC1zZWNyZXQta2V5 # longhorn-test-secret-key
  10.   AWS_ENDPOINTS: aHR0cDovL21pbmlvLXNlcnZpY2UuZGVmYXVsdDo5MDAw # http://minio-service.default:9000

Please follow the Kubernetes document to create the secret.

  • Make sure to use echo -n when generating the base64 encoding, otherwise an new line will be added at the end of the string and it will cause error when accessing the S3.

Notice the secret must be created in the longhorn-system namespace for Longhorn to access.

NFS backupstore

For using NFS server as backupstore, NFS server must support NFSv4.

The target URL would looks like:

  1. nfs://longhorn-test-nfs-svc.default:/opt/backupstore

You can find an example containerized NFS server for testing purposes here.

Setup recurring snapshot/backup

Longhorn supports recurring snapshot and backup for volumes. User only need to set when they wish to take the snapshot and/or backup, and how many snapshots/backups needs to be retains, then Longhorn will automatically create snapshot/backup for the user at that time, as long as the volume is attached to a node.

Users can setup recurring snapshot/backup via Longhorn UI, or Kubernetes StorageClass.

Set up recurring jobs using Longhorn UI

User can find the settings for recurring snapshots and backup in the Volume Detail page.

Set up recurring jobs using StorageClass

Users can set field recurringJobs in StorageClass as parameters. Any future volumes created using this StorageClass will have those recurring jobs automatically set up.

Field recurringJobs should follow JSON format. e.g.

  1. kind: StorageClass
  2. apiVersion: storage.k8s.io/v1
  3. metadata:
  4.   name: longhorn
  5. provisioner: driver.longhorn.io
  6. parameters:
  7.   numberOfReplicas: "3"
  8.   staleReplicaTimeout: "30"
  9.   fromBackup: ""
  10.   recurringJobs: '[{"name":"snap", "task":"snapshot", "cron":"*/1 * * * *", "retain":1},
  11.                    {"name":"backup", "task":"backup", "cron":"*/2 * * * *", "retain":1}]'

Explanation:

  1. name: Name of one job. Do not use duplicate name in one recurringJobs. And the length of name should be no more than 8 characters.

  2. task: Type of one job. It supports snapshot (periodically create snapshot) or backup (periodically create snapshot then do backup) only.

  3. cron: Cron expression. It tells execution time of one job.

  4. retain: How many snapshots/backups Longhorn will retain for one job. It should be no less than 1.