The following instructions will guide you through upgrading a Rancher server that was installed on a Kubernetes cluster with Helm. These steps also apply to air gap installs with Helm.

For the instructions to upgrade Rancher installed on Kubernetes with RancherD, refer to this page.

For the instructions to upgrade Rancher installed with Docker, refer to this page.

To upgrade the components in your Kubernetes cluster, or the definition of the Kubernetes services or add-ons, refer to the upgrade documentation for RKE, the Rancher Kubernetes Engine.

If you installed Rancher using the RKE Add-on yaml, follow the directions to migrate or upgrade.

Prerequisites

Access to kubeconfig

Helm should be run from the same location as your kubeconfig file, or the same location where you run your kubectl commands from.

If you installed Kubernetes with RKE, the config will have been created in the directory you ran rke up in.

The kubeconfig can also be manually targeted for the intended cluster with the --kubeconfig tag (see: https://helm.sh/docs/helm/helm/)

Review Known Issues

Review the known upgrade issues in the Rancher documentation for the most noteworthy issues to consider when upgrading Rancher.

A more complete list of known issues for each Rancher version can be found in the release notes on GitHub and on the Rancher forums.

Note that upgrades to or from any chart in the rancher-alpha repository aren’t supported.

Helm Version

The upgrade instructions assume you are using Helm 3.

For migration of installs started with Helm 2, refer to the official Helm 2 to 3 migration docs. The Helm 2 upgrade page hereprovides a copy of the older upgrade instructions that used Helm 2, and it is intended to be used if upgrading to Helm 3 is not feasible.

For air gap installs: Populate private registry

-For air gap installs only, collect and populate images for the new Rancher server version. Follow the guide to populate your private registry with the images for the Rancher version that you want to upgrade to.

For upgrades from a Rancher server with a hidden local cluster

If you are upgrading to Rancher v2.5 from a Rancher server that was started with the Helm chart option --add-local=false, you will need to drop that flag when upgrading. Otherwise, the Rancher server will not start. The restricted-admin role can be used to continue restricting access to the local cluster. For more information, see this section.

For upgrades from v2.0-v2.2 with external TLS termination

If you are upgrading Rancher from v2.x to v2.3+, and you are using external TLS termination, you will need to edit the cluster.yml to enable using forwarded host headers.

For upgrades with cert-manager older than 0.8.0

Let’s Encrypt will be blocking cert-manager instances older than 0.8.0 starting November 1st 2019. Upgrade cert-manager to the latest version by following these instructions.

Upgrade Outline

Follow the steps to upgrade Rancher server:

1. Back up Your Kubernetes Cluster that is Running Rancher Server

For Rancher v2.5+, use the backup application to back up Rancher.

For Rancher v2.0-v2.4, take a one-time snapshot of your Kubernetes cluster running Rancher server.

You’ll use the backup as a restoration point if something goes wrong during upgrade.

2. Update the Helm chart repository

  1. Update your local helm repo cache.

    1. helm repo update
  2. Get the repository name that you used to install Rancher.

    For information about the repos and their differences, see Helm Chart Repositories.

    Latest: Recommended for trying out the newest features

    Stable: Recommended for production environments

    Alpha: Experimental preview of upcoming releases.
    Note: Upgrades are not supported to, from, or between Alphas.

    1. helm repo list
    2. NAME URL
    3. stable https://charts.helm.sh/stable
    4. rancher-<CHART_REPO> https://releases.rancher.com/server-charts/<CHART_REPO>

    Note: If you want to switch to a different Helm chart repository, please follow the steps on how to switch repositories. If you switch repositories, make sure to list the repositories again before continuing onto Step 3 to ensure you have the correct one added.

  3. Fetch the latest chart to install Rancher from the Helm chart repository.

    This command will pull down the latest charts and save it in the current directory as a .tgz file.

    1. helm fetch rancher-<CHART_REPO>/rancher

    You can fetch the chart for the specific version you are upgrading to by adding in the --version= tag. For example:

    1. helm fetch rancher-<CHART_REPO>/rancher --version=v2.4.11

3. Upgrade Rancher

This section describes how to upgrade normal (Internet-connected) or air gap installations of Rancher with Helm.

Get the values, which were passed with --set, from the current Rancher Helm chart that is installed.

  1. helm get values rancher -n cattle-system
  2. hostname: rancher.my.org

Note: There will be more values that are listed with this command. This is just an example of one of the values.

If you are also upgrading cert-manager to the latest version from a version older than 0.11.0, follow Option B: Reinstalling Rancher and cert-manager.

Otherwise, follow Option A: Upgrading Rancher.

Option A: Upgrading Rancher

Upgrade Rancher to the latest version with all your settings.

Take all the values from the previous step and append them to the command using --set key=value:

  1. helm upgrade rancher rancher-<CHART_REPO>/rancher \
  2. --namespace cattle-system \
  3. --set hostname=rancher.my.org

Note: The above is an example, there may be more values from the previous step that need to be appended.

Alternatively, it’s possible to export the current values to a file and reference that file during upgrade. For example, to only change the Rancher version:

  1. helm get values rancher -n cattle-system -o yaml > values.yaml
  2. helm upgrade rancher rancher-<CHART_REPO>/rancher \
  3. --namespace cattle-system \
  4. -f values.yaml \
  5. --version=2.4.5

Option B: Reinstalling Rancher and cert-manager

If you are currently running the cert-manger whose version is older than v0.11, and want to upgrade both Rancher and cert-manager to a newer version, then you need to reinstall both Rancher and cert-manger due to the API change in cert-manger v0.11.

  1. Uninstall Rancher

    1. helm delete rancher -n cattle-system
  2. Uninstall and reinstall cert-manager according to the instructions on the Upgrading Cert-Manager page.

  3. Reinstall Rancher to the latest version with all your settings. Take all the values from the step 1 and append them to the command using --set key=value. Note: There will be many more options from the step 1 that need to be appended.

    1. helm install rancher rancher-<CHART_REPO>/rancher \
    2. --namespace cattle-system \
    3. --set hostname=rancher.my.org

Render the Rancher template using the same chosen options that were used when installing Rancher. Use the reference table below to replace each placeholder. Rancher needs to be configured to use the private registry in order to provision any Rancher launched Kubernetes clusters or Rancher tools.

Based on the choice you made during installation, complete one of the procedures below.

PlaceholderDescription
<VERSION>The version number of the output tarball.
<RANCHER.YOURDOMAIN.COM>The DNS name you pointed at your load balancer.
<REGISTRY.YOURDOMAIN.COM:PORT>The DNS name for your private registry.
<CERTMANAGER_VERSION>Cert-manager version running on k8s cluster.

Option A: Default Self-signed Certificate

  1. helm template ./rancher-<VERSION>.tgz --output-dir . \
  2. --name rancher \
  3. --namespace cattle-system \
  4. --set hostname=<RANCHER.YOURDOMAIN.COM> \
  5. --set certmanager.version=<CERTMANAGER_VERSION> \
  6. --set rancherImage=<REGISTRY.YOURDOMAIN.COM:PORT>/rancher/rancher \
  7. --set systemDefaultRegistry=<REGISTRY.YOURDOMAIN.COM:PORT> \ # Available as of v2.2.0, set a default private registry to be used in Rancher
  8. --set useBundledSystemChart=true # Available as of v2.3.0, use the packaged Rancher system charts

Option B: Certificates from Files using Kubernetes Secrets

  1. helm template ./rancher-<VERSION>.tgz --output-dir . \
  2. --name rancher \
  3. --namespace cattle-system \
  4. --set hostname=<RANCHER.YOURDOMAIN.COM> \
  5. --set rancherImage=<REGISTRY.YOURDOMAIN.COM:PORT>/rancher/rancher \
  6. --set ingress.tls.source=secret \
  7. --set systemDefaultRegistry=<REGISTRY.YOURDOMAIN.COM:PORT> \ # Available as of v2.2.0, set a default private registry to be used in Rancher
  8. --set useBundledSystemChart=true # Available as of v2.3.0, use the packaged Rancher system charts

If you are using a Private CA signed cert, add --set privateCA=true following --set ingress.tls.source=secret:

  1. helm template ./rancher-<VERSION>.tgz --output-dir . \
  2. --name rancher \
  3. --namespace cattle-system \
  4. --set hostname=<RANCHER.YOURDOMAIN.COM> \
  5. --set rancherImage=<REGISTRY.YOURDOMAIN.COM:PORT>/rancher/rancher \
  6. --set ingress.tls.source=secret \
  7. --set privateCA=true \
  8. --set systemDefaultRegistry=<REGISTRY.YOURDOMAIN.COM:PORT> \ # Available as of v2.2.0, set a default private registry to be used in Rancher
  9. --set useBundledSystemChart=true # Available as of v2.3.0, use the packaged Rancher system charts

Apply the Rendered Templates

Copy the rendered manifest directories to a system with access to the Rancher server cluster and apply the rendered templates.

Use kubectl to apply the rendered manifests.

  1. kubectl -n cattle-system apply -R -f ./rancher

4. Verify the Upgrade

Log into Rancher to confirm that the upgrade succeeded.

Having network issues following upgrade?

See Restoring Cluster Networking.

Known Upgrade Issues

The following table lists some of the most noteworthy issues to be considered when upgrading Rancher. A more complete list of known issues for each Rancher version can be found in the release notes on GitHub and on the Rancher forums.

Upgrade ScenarioIssue
Upgrading to v2.4.6 or v2.4.7These Rancher versions had an issue where the kms:ListKeys permission was required to create, edit, or clone Amazon EC2 node templates. This requirement was removed in v2.4.8.
Upgrading to v2.3.0+Any user provisioned cluster will be automatically updated upon any edit as tolerations were added to the images used for Kubernetes provisioning.
Upgrading to v2.2.0-v2.2.xRancher introduced the system charts repository which contains all the catalog items required for features such as monitoring, logging, alerting and global DNS. To be able to use these features in an air gap install, you will need to mirror the system-charts repository locally and configure Rancher to use that repository. Please follow the instructions to configure Rancher system charts.
Upgrading from v2.0.13 or earlierIf your cluster’s certificates have expired, you will need to perform additional steps to rotate the certificates.
Upgrading from v2.0.7 or earlierRancher introduced the system project, which is a project that’s automatically created to store important namespaces that Kubernetes needs to operate. During upgrade to v2.0.7+, Rancher expects these namespaces to be unassigned from all projects. Before beginning upgrade, check your system namespaces to make sure that they’re unassigned to prevent cluster networking issues.

RKE Add-on Installs

Important: RKE add-on install is only supported up to Rancher v2.0.8

Please use the Rancher helm chart to install Rancher on a Kubernetes cluster. For details, see the Kubernetes Install.

If you are currently using the RKE add-on install method, see Migrating from a RKE add-on install for details on how to move to using the helm chart.