Traffic Trace

This policy enables tracing logging to a third party tracing solution.

Tracing is supported over HTTP, HTTP2, and gRPC protocols in a Mesh. You must explicitly specify the protocol for each service and data plane proxy you want to enable tracing for.

You must also:

  1. Add a tracing backend. You specify a tracing backend as a Mesh resource property.
  2. Add a TrafficTrace resource. You pass the backend to the TrafficTrace resource.

Kuma currently supports the following backends:

  • zipkin
    • JaegerTraffic Trace - 图1 (opens new window) as the Zipkin collector. The Zipkin examples specify Jaeger, but you can modify for a Zipkin-only deployment.
  • datadog

While most commonly we want all the traces to be sent to the same tracing backend, we can optionally create multiple tracing backends in a Mesh resource and store traces for different paths of our service traffic in different backends by leveraging Kuma tags. This is especially useful when we want traces to never leave a world region, or a cloud, for example.

Add Jaeger backend

On Kubernetes you can deploy Jaeger automatically in a kuma-tracing namespace with kumactl install tracing | kubectl apply -f -.

  1. apiVersion: kuma.io/v1alpha1
  2. kind: Mesh
  3. metadata:
  4. name: default
  5. spec:
  6. tracing:
  7. defaultBackend: jaeger-collector
  8. backends:
  9. - name: jaeger-collector
  10. type: zipkin
  11. sampling: 100.0
  12. conf:
  13. url: http://jaeger-collector.kuma-tracing:9411/api/v2/spans

Apply the configuration with kubectl apply -f [..]. :::

  1. type: Mesh
  2. name: default
  3. tracing:
  4. defaultBackend: jaeger-collector
  5. backends:
  6. - name: jaeger-collector
  7. type: zipkin
  8. sampling: 100.0
  9. conf:
  10. url: http://jaeger-collector.kuma-tracing:9411/api/v2/spans

Apply the configuration with kumactl apply -f [..] or with the HTTP API.

Add Datadog backend

Prerequisites

  1. Set up the DatadogTraffic Trace - 图2 (opens new window) agent.
  2. Set up APMTraffic Trace - 图3 (opens new window).

If Datadog is running within Kubernetes, you can expose the APM agent port to Kuma via Kubernetes service.

  1. apiVersion: v1
  2. kind: Service
  3. metadata:
  4. name: trace-svc
  5. spec:
  6. selector:
  7. app: datadog
  8. ports:
  9. - protocol: TCP
  10. port: 8126
  11. targetPort: 8126

Apply the configuration with kubectl apply -f [..].

Set up in Kuma

  1. apiVersion: kuma.io/v1alpha1
  2. kind: Mesh
  3. metadata:
  4. name: default
  5. spec:
  6. tracing:
  7. defaultBackend: datadog-collector
  8. backends:
  9. - name: datadog-collector
  10. type: datadog
  11. sampling: 100.0
  12. conf:
  13. address: trace-svc.datadog.svc.cluster.local
  14. port: 8126

where trace-svc is the name of the Kubernetes Service you specified when you configured the Datadog APM agent.

Apply the configuration with kubectl apply -f [..].

  1. type: Mesh
  2. name: default
  3. tracing:
  4. defaultBackend: datadog-collector
  5. backends:
  6. - name: datadog-collector
  7. type: datadog
  8. sampling: 100.0
  9. conf:
  10. address: 127.0.0.1
  11. port: 8126

Apply the configuration with kumactl apply -f [..] or with the HTTP API.

The defaultBackend property specifies the tracing backend to use if it’s not explicitly specified in the TrafficTrace resource.

Add TrafficTrace resource

Next, create TrafficTrace resources that specify how to collect traces, and which backend to store them in.

  1. apiVersion: kuma.io/v1alpha1
  2. kind: TrafficTrace
  3. mesh: default
  4. metadata:
  5. name: trace-all-traffic
  6. spec:
  7. selectors:
  8. - match:
  9. kuma.io/service: '*'
  10. conf:
  11. backend: jaeger-collector # or the name of any backend defined for the mesh

Apply the configuration with kubectl apply -f [..].

  1. type: TrafficTrace
  2. name: trace-all-traffic
  3. mesh: default
  4. selectors:
  5. - match:
  6. kuma.io/service: '*'
  7. conf:
  8. backend: jaeger-collector # or the name of any backend defined for the mesh

Apply the configuration with kumactl apply -f [..] or with the HTTP API.

You can also add tags to apply the TrafficTrace resource only a subset of data plane proxies. TrafficTrace is a Dataplane policy, so you can specify any of the selectors tags.

Services should also be instrumented to preserve the trace chain across requests made across different services. You can instrument with a language library of your choice, or you can manually pass the following headers:

  • x-request-id
  • x-b3-traceid
  • x-b3-parentspanid
  • x-b3-spanid
  • x-b3-sampled
  • x-b3-flags