Air Gap Installation

Longhorn can be installed in an air gapped environment by using a manifest file, a Helm chart, or the Rancher UI.

Requirements

  • Deploy Longhorn Components images to your own registry.
  • Deploy Kubernetes CSI driver components images to your own registry.

Note:

  • A full list of all needed images is in longhorn-images.txt. First, download the images list by running:

    1. wget https://raw.githubusercontent.com/longhorn/longhorn/v1.6.1/deploy/longhorn-images.txt
  • We provide a script, save-images.sh, to quickly pull the above longhorn-images.txt list. If you specify a tar.gz file name for flag --images, the script will save all images to the provided filename. In the example below, the script pulls and saves Longhorn images to the file longhorn-images.tar.gz. You then can copy the file to your air-gap environment. On the other hand, if you don’t specify the file name, the script just pulls the list of images to your computer.

    1. wget https://raw.githubusercontent.com/longhorn/longhorn/v1.6.1/scripts/save-images.sh
    2. chmod +x save-images.sh
    3. ./save-images.sh --image-list longhorn-images.txt --images longhorn-images.tar.gz
  • We provide another script, load-images.sh, to push Longhorn images to your private registry. If you specify a tar.gz file name for flag --images, the script loads images from the tar file and pushes them. Otherwise, it will find images in your local Docker and push them. In the example below, the script loads images from the file longhorn-images.tar.gz and pushes them to <YOUR-PRIVATE-REGISTRY>

    1. wget https://raw.githubusercontent.com/longhorn/longhorn/v1.6.1/scripts/load-images.sh
    2. chmod +x load-images.sh
    3. ./load-images.sh --image-list longhorn-images.txt --images longhorn-images.tar.gz --registry <YOUR-PRIVATE-REGISTRY>
  • For more options with using the scripts, see flag --help:

    1. ./save-images.sh --help
    2. ./load-images.sh --help

Using a Manifest File

  1. Get Longhorn Deployment manifest file

    wget https://raw.githubusercontent.com/longhorn/longhorn/v1.6.1/deploy/longhorn.yaml

  2. Create Longhorn namespace

    kubectl create namespace longhorn-system

  3. If private registry require authentication, Create docker-registry secret in longhorn-system namespace:

    kubectl -n longhorn-system create secret docker-registry <SECRET_NAME> --docker-server=<REGISTRY_URL> --docker-username=<REGISTRY_USER> --docker-password=<REGISTRY_PASSWORD>

    • Add your secret name SECRET_NAME to imagePullSecrets.name in the following resources

      • longhorn-driver-deployer Deployment
      • longhorn-manager DaemonSet
      • longhorn-ui Deployment

      Example:

      1. apiVersion: apps/v1
      2. kind: Deployment
      3. metadata:
      4. labels:
      5. app: longhorn-ui
      6. name: longhorn-ui
      7. namespace: longhorn-system
      8. spec:
      9. replicas: 1
      10. selector:
      11. matchLabels:
      12. app: longhorn-ui
      13. template:
      14. metadata:
      15. labels:
      16. app: longhorn-ui
      17. spec:
      18. containers:
      19. - name: longhorn-ui
      20. image: longhornio/longhorn-ui:v0.8.0
      21. ports:
      22. - containerPort: 8000
      23. env:
      24. - name: LONGHORN_MANAGER_IP
      25. value: "http://longhorn-backend:9500"
      26. imagePullSecrets:
      27. - name: <SECRET_NAME> ## Add SECRET_NAME here
      28. serviceAccountName: longhorn-service-account
  4. Apply the following modifications to the manifest file

    • Modify Kubernetes CSI driver components environment variables in longhorn-driver-deployer Deployment point to your private registry images

      • CSI_ATTACHER_IMAGE
      • CSI_PROVISIONER_IMAGE
      • CSI_NODE_DRIVER_REGISTRAR_IMAGE
      • CSI_RESIZER_IMAGE
      • CSI_SNAPSHOTTER_IMAGE
      1. - name: CSI_ATTACHER_IMAGE
      2. value: <REGISTRY_URL>/csi-attacher:<CSI_ATTACHER_IMAGE_TAG>
      3. - name: CSI_PROVISIONER_IMAGE
      4. value: <REGISTRY_URL>/csi-provisioner:<CSI_PROVISIONER_IMAGE_TAG>
      5. - name: CSI_NODE_DRIVER_REGISTRAR_IMAGE
      6. value: <REGISTRY_URL>/csi-node-driver-registrar:<CSI_NODE_DRIVER_REGISTRAR_IMAGE_TAG>
      7. - name: CSI_RESIZER_IMAGE
      8. value: <REGISTRY_URL>/csi-resizer:<CSI_RESIZER_IMAGE_TAG>
      9. - name: CSI_SNAPSHOTTER_IMAGE
      10. value: <REGISTRY_URL>/csi-snapshotter:<CSI_SNAPSHOTTER_IMAGE_TAG>
    • Modify Longhorn images to point to your private registry images

      • longhornio/longhorn-manager

        image: <REGISTRY_URL>/longhorn-manager:<LONGHORN_MANAGER_IMAGE_TAG>

      • longhornio/longhorn-engine

        image: <REGISTRY_URL>/longhorn-engine:<LONGHORN_ENGINE_IMAGE_TAG>

      • longhornio/longhorn-instance-manager

        image: <REGISTRY_URL>/longhorn-instance-manager:<LONGHORN_INSTANCE_MANAGER_IMAGE_TAG>

      • longhornio/longhorn-share-manager

        image: <REGISTRY_URL>/longhorn-share-manager:<LONGHORN_SHARE_MANAGER_IMAGE_TAG>

      • longhornio/longhorn-ui

        image: <REGISTRY_URL>/longhorn-ui:<LONGHORN_UI_IMAGE_TAG>

      Example:

      1. apiVersion: apps/v1
      2. kind: Deployment
      3. metadata:
      4. labels:
      5. app: longhorn-ui
      6. name: longhorn-ui
      7. namespace: longhorn-system
      8. spec:
      9. replicas: 1
      10. selector:
      11. matchLabels:
      12. app: longhorn-ui
      13. template:
      14. metadata:
      15. labels:
      16. app: longhorn-ui
      17. spec:
      18. containers:
      19. - name: longhorn-ui
      20. image: <REGISTRY_URL>/longhorn-ui:<LONGHORN_UI_IMAGE_TAG> ## Add image name and tag here
      21. ports:
      22. - containerPort: 8000
      23. env:
      24. - name: LONGHORN_MANAGER_IP
      25. value: "http://longhorn-backend:9500"
      26. imagePullSecrets:
      27. - name: <SECRET_NAME>
      28. serviceAccountName: longhorn-service-account
  5. Deploy Longhorn using modified manifest file kubectl apply -f longhorn.yaml

Using a Helm Chart

In v1.6.1, Longhorn automatically adds <REGISTRY_URL> prefix to images. You simply need to set the registryUrl parameters to pull images from your private registry.

Note: Once you set registryUrl to your private registry, Longhorn tries to pull images from the registry exclusively. Make sure all Longhorn components’ images are in the registry otherwise Longhorn will fail to pull images.

Use default image name

If you keep the images’ names as recommended here, you only need to do the following steps:

  1. Clone the Longhorn repo:

    git clone https://github.com/longhorn/longhorn.git

  2. In chart/values.yaml

    • Specify Private registry URL. If the registry requires authentication, specify Private registry user, Private registry password, and Private registry secret. Longhorn will automatically generate a secret with the those information and use it to pull images from your private registry.

      1. defaultSettings:
      2. registrySecret: <SECRET_NAME>
      3. privateRegistry:
      4. registryUrl: <REGISTRY_URL>
      5. registryUser: <REGISTRY_USER>
      6. registryPasswd: <REGISTRY_PASSWORD>
      7. registrySecret: <REGISTRY_SECRET_NAME>

Use custom image name

If you want to use custom images’ names, you can use the following steps:

  1. Clone longhorn repo

    git clone https://github.com/longhorn/longhorn.git

  2. In chart/values.yaml

    Note: Do not include the private registry prefix, it will be added automatically. e.g: if your image is example.com/username/longhorn-manager, use username/longhorn-manager in the following charts.

    • Specify Longhorn images and tag:

      1. image:
      2. longhorn:
      3. engine:
      4. repository: longhornio/longhorn-engine
      5. tag: <LONGHORN_ENGINE_IMAGE_TAG>
      6. manager:
      7. repository: longhornio/longhorn-manager
      8. tag: <LONGHORN_MANAGER_IMAGE_TAG>
      9. ui:
      10. repository: longhornio/longhorn-ui
      11. tag: <LONGHORN_UI_IMAGE_TAG>
      12. instanceManager:
      13. repository: longhornio/longhorn-instance-manager
      14. tag: <LONGHORN_INSTANCE_MANAGER_IMAGE_TAG>
      15. shareManager:
      16. repository: longhornio/longhorn-share-manager
      17. tag: <LONGHORN_SHARE_MANAGER_IMAGE_TAG>
    • Specify CSI Driver components images and tag:

      1. csi:
      2. attacher:
      3. repository: longhornio/csi-attacher
      4. tag: <CSI_ATTACHER_IMAGE_TAG>
      5. provisioner:
      6. repository: longhornio/csi-provisioner
      7. tag: <CSI_PROVISIONER_IMAGE_TAG>
      8. nodeDriverRegistrar:
      9. repository: longhornio/csi-node-driver-registrar
      10. tag: <CSI_NODE_DRIVER_REGISTRAR_IMAGE_TAG>
      11. resizer:
      12. repository: longhornio/csi-resizer
      13. tag: <CSI_RESIZER_IMAGE_TAG>
      14. snapshotter:
      15. repository: longhornio/csi-snapshotter
      16. tag: <CSI_SNAPSHOTTER_IMAGE_TAG>
    • Specify Private registry URL. If the registry requires authentication, specify Private registry user, Private registry password, and Private registry secret. Longhorn will automatically generate a secret with the those information and use it to pull images from your private registry.

      1. defaultSettings:
      2. registrySecret: <SECRET_NAME>
      3. privateRegistry:
      4. registryUrl: <REGISTRY_URL>
      5. registryUser: <REGISTRY_USER>
      6. registryPasswd: <REGISTRY_PASSWORD>
  3. Install Longhorn

  1. helm install longhorn ./chart --namespace longhorn-system --create-namespace

Using a Rancher App

Use default image name

If you keep the images’ names as recommended here, you only need to do the following steps:

  • In the Private Registry Settings section specify:

    • Private registry URL
    • Private registry user
    • Private registry password
    • Private registry secret name

    Longhorn will automatically generate a secret with the those information and use it to pull images from your private registry.

    images

Use custom image name

  • If you want to use custom images’ names, you can set Use Default Images to False and specify images’ names.

    Note: Do not include the private registry prefix, it will be added automatically. e.g: if your image is example.com/username/longhorn-manager, use username/longhorn-manager in the following charts.

    images

  • Specify Private registry URL. If the registry requires authentication, specify Private registry user, Private registry password, and Private registry secret name. Longhorn will automatically generate a secret with the those information and use it to pull images from your private registry.

    images

Troubleshooting

For Helm/Rancher installation, if user forgot to submit a secret to authenticate to private registry, longhorn-manager DaemonSet will fail to create.

  1. Create the Kubernetes secret

    kubectl -n longhorn-system create secret docker-registry <SECRET_NAME> --docker-server=<REGISTRY_URL> --docker-username=<REGISTRY_USER> --docker-password=<REGISTRY_PASSWORD>

  2. Create registry-secret setting object manually.

    1. apiVersion: longhorn.io/v1beta2
    2. kind: Setting
    3. metadata:
    4. name: registry-secret
    5. namespace: longhorn-system
    6. value: <SECRET_NAME>

    kubectl apply -f registry-secret.yml

  3. Delete Longhorn and re-install it again.

    • Helm2

      helm uninstall ./chart --name longhorn --namespace longhorn-system

      helm install ./chart --name longhorn --namespace longhorn-system

    • Helm3

      helm uninstall longhorn ./chart --namespace longhorn-system

      helm install longhorn ./chart --namespace longhorn-system

Recommendation:

It’s highly recommended not to manipulate image tags, especially instance manager image tags such as v1_20200301, because we intentionally use the date to avoid associating it with a Longhorn version.

The images of Longhorn’s components are hosted in Dockerhub under the longhornio account. For example, longhornio/longhorn-manager:v1.6.1. It’s recommended to keep the account name, longhornio, the same when you push the images to your private registry. This helps avoid unnecessary configuration issues.