Zipkin data source

Grafana ships with built-in support for Zipkin, an open source, distributed tracing system. Just add it as a data source and you are ready to query your traces in Explore.

Adding the data source

To access Zipkin settings, click the Configuration (gear) icon, then click Data Sources > Zipkin.

NameDescription
NameThe data source name in panels, queries, and Explore.
DefaultThe pre-selected data source for a new panel.
URLThe URL of the Zipkin instance. For example, http://localhost:9411.
Basic AuthEnable basic authentication for the Zipkin data source.
UserSpecify a user name for basic authentication.
PasswordSpecify a password for basic authentication.

Trace to logs

Note: This feature is available in Grafana 7.4+.

This is a configuration for the trace to logs feature. Select target data source (at this moment limited to Loki data sources) and select which tags will be used in the logs query.

  • Data source - Target data source.
  • Tags - The tags that will be used in the Loki query. Default is 'cluster', 'hostname', 'namespace', 'pod'.
  • Map tag names - When enabled, allows configuring how Zipkin tag names map to Loki label names. For example, map service.name to service.
  • Span start time shift - Shift in the start time for the Loki query based on the span start time. In order to extend to the past, you need to use a negative value. Use time interval units like 5s, 1m, 3h. The default is 0.
  • Span end time shift - Shift in the end time for the Loki query based on the span end time. Time units can be used here, for example, 5s, 1m, 3h. The default is 0.
  • Filter by Trace ID - Toggle to append the trace ID to the Loki query.
  • Filter by Span ID - Toggle to append the span ID to the Loki query.

Node Graph

This is a configuration for the beta Node Graph visualization. The Node Graph is shown after the trace view is loaded and is disabled by default.

Enable Node Graph - Enables the Node Graph visualization.

Query traces

Querying and displaying traces from Zipkin is available via Explore.

Screenshot of the Zipkin query editor Screenshot of the Zipkin query editor Zipkin - 图2 The Zipkin query editor allows you to query by trace ID directly or selecting a trace from trace selector. To query by trace ID, insert the ID into the text input.

Screenshot of the Zipkin query editor with trace selector expanded Screenshot of the Zipkin query editor with trace selector expanded Zipkin - 图4 Use the trace selector to pick particular trace from all traces logged in the time range you have selected in Explore. The trace selector has three levels of nesting:

  1. The service you are interested in.
  2. Particular operation is part of the selected service
  3. Specific trace in which the selected operation occurred, represented by the root operation name and trace duration.

Data mapping in the trace UI

Zipkin annotations are shown in the trace view as logs with annotation value shown under annotation key.

Upload JSON trace file

You can upload a JSON file that contains a single trace to visualize it.

Screenshot of the Zipkin data source in explore with upload selected Screenshot of the Zipkin data source in explore with upload selected Zipkin - 图6 Here is an example JSON:

  1. [
  2. {
  3. "traceId": "efe9cb8857f68c8f",
  4. "parentId": "efe9cb8857f68c8f",
  5. "id": "8608dc6ce5cafe8e",
  6. "kind": "SERVER",
  7. "name": "get /api",
  8. "timestamp": 1627975249601797,
  9. "duration": 23457,
  10. "localEndpoint": { "serviceName": "backend", "ipv4": "127.0.0.1", "port": 9000 },
  11. "tags": {
  12. "http.method": "GET",
  13. "http.path": "/api",
  14. "jaxrs.resource.class": "Resource",
  15. "jaxrs.resource.method": "printDate"
  16. },
  17. "shared": true
  18. }
  19. ]

Linking Trace ID from logs

You can link to Zipkin trace from logs in Loki by configuring a derived field with internal link. See Loki documentation for details.