Upgrade Promscale

Promscale consists of the Promscale Connector, and the Promscale Database. The Promscale Database is PostgreSQL with the TimescaleDB and the Promscale extensions. When you upgrade your Promscale installation, you need to check both the Connector and the Database.

The process for upgrading your Promscale installation is:

  1. Check that the most recent required versions of PostgreSQL, TimescaleDB and the Promscale extension are available.
  2. Stop all Promscale Connector instances connected to the database.
  3. Start one instance with the new version of the Promscale Connector.
  4. Start all other Promscale Connector instances.

You can upgrade your existing Promscale to Promscale 0.11.0 and from the previous Alpine docker image.

Upgrade to Promscale 0.11.0

Promscale 0.11.0 contains significant changes, and the upgrade drops any previous tracing data you have stored. Make a backup of your installation, and test the new version, before you go ahead with this upgrade.

Upgrading to Promscale 0.11.0 could take up to 5-10 minutes on very large databases. During the upgrade, Promscale does not ingest or query data.

The upgrade process modifies a lot of database objects. On large deployments, you might need to increase the PostgreSQL max_locks_per_transaction parameter before you start the upgrade. For more information, see Transaction locks.

warning

When you upgrade to Promscale 0.11.0, all previous tracing data is dropped. Make a backup of your installation, and test the new version before you upgrade.

Upgrading to Promscale 0.11.0

To upgrade to Promscale 0.11.0 you must have installed PostgreSQL 12 or later, TimescaleDB 2.6.1 or later, and Promscale extension version 0.5.0 or later.

  1. Check the version of PostgreSQL that is installed:

    1. SHOW server_version;

  2. Check the version of TimescaleDB and Promscale that is installed:

    1. SELECT extname, extversion FROM pg_extension WHERE extname IN ('timescaledb', 'promscale');

    For information about upgrading TimescaleDB and PostgreSQL, see Update TimescaleDB and Upgrade PostgreSQL.

  3. Stop all Promscale Connector instances connected to the database.

  4. Update the Promscale extension on one instance using the Promscale Connector. For more information, see the Promscale installation instructions for various installation method.

    note

    Do not use the ALTER EXTENSION promscale UPDATE; command. to update the extension.

  5. Start one instance with the latest version of the Promscale Connector. The migration happens automatically. After the migration is completed, upgrade the remaining Promscale Connector instances.

  6. Start the other instances of Promscale Connector.

Upgrading to Promscale 0.11.0 creates a lock on your tables. If you have been using Promscale for a while, and you have a large number of metrics, this could result in locking a large number of tables. If PostgreSQL has not been correctly configured for this, the process could fail with an error similar to:

  2. ERROR: out of shared memory (SQLSTATE 53200)`.

This error occurs because PostgreSQL has a limited amount of shared memory available to store locks on objects. You can increase the amount of shared memory available by increasing the max_locks_per_transaction parameter. When you have completed the upgrade, you can set the value back to what it was before then upgrade. For more information about tuning this parameter, see troubleshooting Promscale.

Upgrade from the previous alpine image

You can upgrade from the previous alpine image on Docker or Kubernetes.

Upgrading from the previous alpine image on Docker

Previously, the recommended image was located at timescaledev/promscale-extension. It was based on the Alpine docker image for PostgreSQL.

The previous Alpine-based image are updated and supported until the end of 2022 but you are encouraged to migrate to the timescale/timescaledb-ha. All new installations should switch to the timescale/timescaledb-haimage.

note

Migrating to Debian version can be a lengthy process and involves downtime.

  1. Use docker inspect to determine the data volumes used by your database for the data directory.

  2. Shut down all Promscale Connectors.

  3. Shut down the original database Docker image, but make sure you preserve the volume mount for the data directory. You need to mount this same directory in the new image.

  4. Change ownership of the data directory to the postgres user and group in the new image. For example:

    1. docker run -v <data_dir_volume_mount>:/var/lib/postgresql/data timescale/timescaledb-ha:pg14-latest chown -R postgres:postgres /var/lib/postgresql/data

  5. Start the new Docker container with the same volume mounts that the original container used.

  6. Connect to the new database using psql and reindex all the collatable data. Use this query to reindex all the necessary indexes:

    1. DO $$DECLARE r record;
    3.   BEGIN
    5.     FOR r IN
    7.       SELECT DISTINCT indclass
    9.           FROM (SELECT indexrelid::regclass indclass, unnest(string_to_array(indcollation::text, ' ')) coll FROM pg_catalog.pg_index) sub
    11.           INNER JOIN pg_catalog.pg_class c ON (c.oid = sub.indclass)
    13.           WHERE coll !='0' AND c.relkind != 'I'
    15.     LOOP
    17.      EXECUTE 'REINDEX INDEX ' || r.indclass;
    19.   END LOOP;
    21. END$$;

    This is necessary because the collation in the Alpine image is broken and so BTREE-based indexes remain incorrect until they are reindexed. It is extremely important to execute this step before ingesting new data to avoid data corruption. This process can take a long time depending on the indexed textual data in the database.

  7. Restart the Promscale Connector

Upgrading from the previous alpine image on Kubernetes

If you are using Kubernetes instead of plain Docker:

  1. Shutdown the Promscale Connector pods

  2. Change the database pod to use the Debian Docker image.

  3. Restart the pod.

  4. Change ownership of the data directory to the postgres user and group in the new image by ssh into the database container. For example:

    1. chown -R postgres:postgres /var/lib/postgresql/data

  5. Connect to the new database using psql and reindex all the collatable data. Use this query to reindex all the necessary indexes:

    1. DO $$DECLARE r record;
    3.   BEGIN
    5.     FOR r IN
    7.       SELECT DISTINCT indclass
    9.           FROM (SELECT indexrelid::regclass indclass, unnest(string_to_array(indcollation::text, ' ')) coll FROM pg_catalog.pg_index) sub
    11.           INNER JOIN pg_catalog.pg_class c ON (c.oid = sub.indclass)
    13.           WHERE coll !='0' AND c.relkind != 'I'
    15.     LOOP
    17.      EXECUTE 'REINDEX INDEX ' || r.indclass;
    19.   END LOOP;
    21. END$$;

  6. Restart the Promscale Connector pods.