TimeChaos Experiment

This document describe how to add TimeChaos experiments in Chaos Mesh.

TimeChaos is used to modify the return value of clock_gettime, which causes time offset on Go’s time.Now() and Rust std’s std::time::Instant::now() etc.

Configuration file

Below is a sample TimeChaos configuration file:

  1. apiVersion: chaos-mesh.org/v1alpha1
  2. kind: TimeChaos
  3. metadata:
  4.   name: time-shift-example
  5.   namespace: chaos-testing
  6. spec:
  7.   mode: one
  8.   selector:
  9.     labelSelectors:
  10.       'app.kubernetes.io/component': 'pd'
  11.   timeOffset: '-10m100ns'
  12.   clockIds:
  13.     - CLOCK_REALTIME
  14.   containerNames:
  15.     - pd
  16.   duration: '10s'
  17.   scheduler:
  18.     cron: '@every 15s'

For more sample files, see examples. You can edit them as needed.

Description:

  • mode defines the mode to select pods.
  • selector specifies the target pods for chaos injection. For more details, see Define the Scope of Chaos Experiment.
  • timeOffset specifies the time offset. It is a duration string with specified unit, such as 300ms, -1.5h. Valid time units are “ns”, “us” (or “µs”), “ms”, “s”, “m”, “h”.
  • clockIds defines all affected clk_id. clk_id refers to the first argument of clock_gettime call. For most application, CLOCK_REALTIME is enough.
  • containerNames selects affected containers’ names. If not set, all containers will be injected.
  • duration defines the duration for each chaos experiment. In the sample file above, the time chaos lasts for 10 seconds.
  • scheduler defines the scheduler rules for the running time of the chaos experiment. For more rule information, see robfig/cron.

Limitation

  • Time modification can only be injected into the main process of container.
  • Time chaos has no effect on pure system call clock_gettime.
  • All injected vDSO calls use pure system calls to get the real time, so clock-related function calls can be much slower.