Overview

Configuration

The configuration of Loggie is mainly divided into two categories:

System Configuration

The global system configuration, which can be assigned by -config.system in CMD arguments, includes the following:

  • monitor: monitor related configuration
  • discovery: service discovery and configuration delivery
  • reload: hot reload for dynamic configuration
  • defaults: global default configuration
  • http: http port used for management and monitoring

loggie.yml

  1. # loggie.yml
  2. loggie:
  3.   monitor:
  4.     logger:
  5.       period: 30s
  6.       enabled: true
  7.     listeners:
  8.       filesource: ~
  9.       filewatcher: ~
  10.       reload: ~
  11.       sink: ~
  13.   discovery:
  14.     enabled: false
  16.   reload:
  17.     enabled: true
  18.     period: 10s
  20.   defaults:
  21.     sink:
  22.       type: dev
  23.     sources:
  24.       - type: file
  25.         watcher:
  26.           cleanFiles:
  27.             maxHistory: 1
  28.   http:
  29.     enabled: true
  30.     port: 9196

Pipeline Configuration

Pipeline configuration, specified by -config.pipeline in CMD arguments, indicates the Source, Sink, Queue and Interceptor used by the pipeline.

  • Source: Multiple Sources can be configured for each Pipeline.
  • Interceptor: Multiple Interceptors can be configured for each Pipeline.
  • Sink: One Sink can be configured for each Pipeline.
  • Queue: The default is the channel queue. Generally no configuration is required

pipeline.yml

  1. pipelines:
  2. - name: demo # pipeline name, required
  3.   sources:
  4.     - type: ${sourceType}
  5.       name: access # source name, required
  6.       ...
  7.   interceptors:
  8.   - type: ${interceptorType}
  9.     ...
  10.   sink:
  11.     type: ${sinkType}
  12.     ...

Kubernetes CRD

Loggie defines the following CRDs for issuing configurations in Kubernetes:

  • LogConfig: namespace level, representing a Pipeline configuration that can be used to collect container logs of Pods.

  • ClusterLogConfig: cluster level, indicating a pipeline configuration, used for cluster-level cross-Namespace collection of Pod container logs, collection of logs on Node nodes, and distribution of general pipeline configuration for a Loggie cluster.

  • Sink: cluster level, representing a sink configuration, which can be referenced in LogConfig/ClusterLogConfig.

  • Interceptors: cluster level, representing an interceptors group, which can be referenced in LogConfig.

Note

The pipeline in ClusterLogConfig/LogConfig can define sink and interceptor, which are used for the sink/interceptor of this pipeline. If you want to reuse sinks or interceptors in multiple ClusterLogConfig/LogConfig, you can create a Sink/Interceptor CR and use sinkRef/interceptorRef in ClusterLogConfig/LogConfig for reference.