Upgrading

Slack Docker Pulls GitHub edit source

In normal cases, users can directly shutdown the current Alluxio processes, change the Alluxio binaries to newer version, configure Alluxio clusters similar to before, and start Alluxio processes with the existing journal folder/address for upgrading. Alluxio is able to read the previous journal files and recover the Alluxio metadata.

There are two cases where master journals are not backward compatible and additional steps are required to upgrade Alluxio cluster:

  • Upgrading from Alluxio 1.x to Alluxio 2.x
  • Upgrading from Alluxio 2.3.x and below to Alluxio 2.4.0 and above, and the embedded journal is used.

This document goes over how to upgrade Alluxio to a non-backward compatible version. Even when upgrading to a backward compatible version, it is still recommended to follow the steps below to create a backup before upgrading.

Create a backup on the current version

alluxio-1.8.1 introduced a journal backup feature. Please note that the Alluxio binaries should not be changed before taking the backup. With versions older than 1.8.1 master(s) running, create a journal backup by running

  1. $ ./bin/alluxio fsadmin backup

This will print something like

  1. Successfully backed up journal to ${BACKUP_PATH}

${BACKUP_PATH} will be determined by the date, and the configuration of your journal. By default, the backup file will be saved to the alluxio.master.backup.directory of your cluster root UFS. One can also backup to the local filesystem of the current leading master node with backup [local_address] --local command.

Upgrade and start from backup

Next, download and untar newer version Alluxio. First format the cluster with

  1. $ ./bin/alluxio format

Then start the cluster with the -i ${BACKUP_PATH} argument, replacing ${BACKUP_PATH} with your specific backup path.

  1. $ ./bin/alluxio-start.sh -i ${BACKUP_PATH} all

Note that the ${BACKUP_PATH} should be a full path like HDFS address that can be accessed by all the Alluxio masters. If backing up to a local filesystem path, remember to copy the backup file to the same location of all masters nodes, and then start all the masters with the local backup file path.

Upgrade Clients and Servers

Alluxio 2.x makes significant changes in the RPC layer, so pre-2.0.0 clients do not work with post-2.0.0 servers, and vice-versa. Upgrade all applications to use the alluxio-2.x client.