Loki Storage Retention

Retention in Loki is achieved through the Table Manager.In order to enable the retention support, the Table Manager needs to beconfigured to enable deletions and a retention period. Please refer to thetable_manager_configsection of the Loki configuration reference for all available options.Alternatively, the table-manager.retention-period andtable-manager.retention-deletes-enabled command line flags can be used. Theprovided retention period needs to be a duration represented as a string thatcan be parsed using Go’s time.Duration.

WARNING: The retention period must be a multiple of the index and chunks tableperiod, configured in the period_configblock. See the Table Manager documentation formore information.

When using S3 or GCS, the bucket storing the chunks needs to have the expirypolicy set correctly. For more details checkS3’s documentationorGCS’s documentation.

Currently, the retention policy can only be set globally. A per-tenant retentionpolicy with an API to delete ingested logs is still under development.

Since a design goal of Loki is to make storing logs cheap, a volume-baseddeletion API is deprioritized. Until this feature is released, if you suddenlymust delete ingested logs, you can delete old chunks in your object store. Note,however, that this only deletes the log content and keeps the label indexintact; you will still be able to see related labels but will be unable toretrieve the deleted log content.

For further details on the Table Manager internals, refer to theTable Manager documentation.

Example Configuration

Example configuration with GCS with a 30 day retention:

schema_config:
  configs:
  - from: 2018-04-15
    store: bigtable
    object_store: gcs
    schema: v9
    index:
      prefix: loki_index_
      period: 168h
storage_config:
  bigtable:
    instance: BIGTABLE_INSTANCE
    project: BIGTABLE_PROJECT
  gcs:
    bucket_name: GCS_BUCKET_NAME
table_manager:
  retention_deletes_enabled: true
  retention_period: 720h