Visualize Results in the Pipelines UI

Visualizing the results of your pipelines component

Out of date

This guide contains outdated information pertaining to Kubeflow 1.0. This guide needs to be updated for Kubeflow 1.1.

This page shows you how to use the Kubeflow Pipelines UI to visualize output from a Kubeflow Pipelines component. For details about how to build a component, see the guide to building your own component.

Kubeflow Pipelines provides a new method of generating visualizations. See the guide to Python Based Visualizations.

Introduction

The Kubeflow Pipelines UI offers built-in support for several types of visualizations, which you can use to provide rich performance evaluation and comparison data. To make use of this programmable UI, your pipeline component must write a JSON file to the component’s local filesystem. You can do this at any point during the pipeline execution.

You can view the output visualizations in the following places on the Kubeflow Pipelines UI:

  • The Run output tab shows the visualizations for all pipeline steps in the selected run. To open the tab in the Kubeflow Pipelines UI:

    1. Click Experiments to see your current pipeline experiments.
    2. Click the experiment name of the experiment that you want to view.
    3. Click the run name of the run that you want to view.
    4. Click the Run output tab.

    Output visualization from a pipeline run

  • The Artifacts tab shows the visualization for the selected pipeline step. To open the tab in the Kubeflow Pipelines UI:

    1. Click Experiments to see your current pipeline experiments.
    2. Click the experiment name of the experiment that you want to view.
    3. Click the run name of the run that you want to view.
    4. On the Graph tab, click the step representing the pipeline component that you want to view. The step details slide into view, showing the Artifacts tab.

    Table-based visualization from a pipeline component

All screenshots and code snippets on this page come from a sample pipeline that you can run directly from the Kubeflow Pipelines UI. See the sample description and links below.

Writing out metadata for the output viewers

The pipeline component must write a JSON file specifying metadata for the output viewer(s) that you want to use for visualizing the results. The component must also export a file output artifact with an artifact name of mlpipeline-ui-metadata, or else the Kubeflow Pipelines UI will not render the visualization. In other words, the .outputs.artifacts setting for the generated pipeline component should show: - {name: mlpipeline-ui-metadata, path: /mlpipeline-ui-metadata.json}. The JSON filepath does not matter, although /mlpipeline-ui-metadata.json is used for consistency in the examples below.

The JSON specifies an array of outputs. Each outputs entry describes the metadata for an output viewer. The JSON structure looks like this:

  1. {
  2. "version": 1,
  3. "outputs": [
  4. {
  5. "type": "confusion_matrix",
  6. "format": "csv",
  7. "source": "my-dir/my-matrix.csv",
  8. "schema": [
  9. {"name": "target", "type": "CATEGORY"},
  10. {"name": "predicted", "type": "CATEGORY"},
  11. {"name": "count", "type": "NUMBER"},
  12. ],
  13. "labels": "vocab"
  14. },
  15. {
  16. ...
  17. }
  18. ]
  19. }

If the component writes such a file to its container filesystem, the Kubeflow Pipelines system extracts the file, and the Kubeflow Pipelines UI uses the file to generate the specified viewer(s). The metadata specifies where to load the artifact data from. The Kubeflow Pipelines UI loads the data into memory and renders it. Note: You should keep this data at a volume that’s manageable by the UI, for example by running a sampling step before exporting the file as an artifact.

The table below shows the available metadata fields that you can specify in the outputs array. Each outputs entry must have a type. Depending on value of type, other fields may also be required as described in the list of output viewers later on the page.

Field nameDescription
formatThe format of the artifact data. The default is csv. Note: The only format currently available is csv.
headerA list of strings to be used as headers for the artifact data. For example, in a table these strings are used in the first row.
labelsA list of strings to be used as labels for artifact columns or rows.
predicted_colName of the predicted column.
schemaA list of {type, name} objects that specify the schema of the artifact data.
source

The full path to the data. The available locations include http, https, Amazon S3, Minio, and Google Cloud Storage.

The path can contain wildcards ‘*’, in which case the Kubeflow Pipelines UI concatenates the data from the matching source files.

source can also contain inlined string data instead of a path when storage=’inline’.

storage

(Optional) When storage is inline, the value of source is parsed as inline data instead of a path. This applies to all types of outputs except tensorboard. See Markdown or Web app below as examples.

Be aware, support for inline visualizations, other than markdown, was introduced in Kubeflow Pipelines 0.2.5. Before using these visualizations, upgrade your Kubeflow Pipelines cluster to version 0.2.5 or higher.

target_colName of the target column.
typeName of the viewer to be used to visualize the data. The list below shows the available types.

Available output viewers

The sections below describe the available viewer types and the required metadata fields for each type.

Confusion matrix

Type: confusion_matrix

Required metadata fields:

  • format
  • labels
  • schema
  • source

Optional metadata fields:

  • storage

The confusion_matrix viewer plots a confusion matrix visualization of the data from the given source path, using the schema to parse the data. The labels provide the names of the classes to be plotted on the x and y axes.

Specify 'storage': 'inline' to embed raw content of the confusion matrix CSV file as a string in source field directly.

Example:

  1. metadata = {
  2. 'outputs' : [{
  3. 'type': 'confusion_matrix',
  4. 'format': 'csv',
  5. 'schema': [
  6. {'name': 'target', 'type': 'CATEGORY'},
  7. {'name': 'predicted', 'type': 'CATEGORY'},
  8. {'name': 'count', 'type': 'NUMBER'},
  9. ],
  10. 'source': <CONFUSION_MATRIX_CSV_FILE>,
  11. # Convert vocab to string because for bealean values we want "True|False" to match csv data.
  12. 'labels': list(map(str, vocab)),
  13. }]
  14. }
  15. with file_io.FileIO('/mlpipeline-ui-metadata.json', 'w') as f:
  16. json.dump(metadata, f)

Visualization on the Kubeflow Pipelines UI:

Confusion matrix visualization from a pipeline component

Markdown

Type: markdown

Required metadata fields:

  • source

Optional metadata fields:

  • storage

The markdown viewer renders Markdown strings on the Kubeflow Pipelines UI. The viewer can read the Markdown data from the following locations:

  • A Markdown-formatted string embedded in the source field. The value of the storage field must be inline.
  • Markdown code in a remote file, at a path specified in the source field. The storage field can be empty or contain any value except inline.

Example:

  1. metadata = {
  2. 'outputs' : [
  3. # Markdown that is hardcoded inline
  4. {
  5. 'storage': 'inline',
  6. 'source': '# Inline Markdown\n[A link](https://www.kubeflow.org/)',
  7. 'type': 'markdown',
  8. },
  9. # Markdown that is read from a file
  10. {
  11. 'source': 'gs://your_project/your_bucket/your_markdown_file',
  12. 'type': 'markdown',
  13. }]
  14. }
  15. with file_io.FileIO('/mlpipeline-ui-metadata.json', 'w') as f:
  16. json.dump(metadata, f)

Visualization on the Kubeflow Pipelines UI:

Markdown visualization from a pipeline component

ROC curve

Type: roc

Required metadata fields:

  • format
  • schema
  • source

The roc viewer plots a receiver operating characteristic (ROC) curve using the data from the given source path. The Kubeflow Pipelines UI assumes that the schema includes three columns with the following names:

  • fpr (false positive rate)
  • tpr (true positive rate)
  • thresholds

Optional metadata fields:

  • storage

When viewing the ROC curve, you can hover your cursor over the ROC curve to see the threshold value used for the cursor’s closest fpr and tpr values.

Specify 'storage': 'inline' to embed raw content of the ROC curve CSV file as a string in source field directly.

Example:

  1. df_roc = pd.DataFrame({'fpr': fpr, 'tpr': tpr, 'thresholds': thresholds})
  2. roc_file = os.path.join(args.output, 'roc.csv')
  3. with file_io.FileIO(roc_file, 'w') as f:
  4. df_roc.to_csv(f, columns=['fpr', 'tpr', 'thresholds'], header=False, index=False)
  5. metadata = {
  6. 'outputs': [{
  7. 'type': 'roc',
  8. 'format': 'csv',
  9. 'schema': [
  10. {'name': 'fpr', 'type': 'NUMBER'},
  11. {'name': 'tpr', 'type': 'NUMBER'},
  12. {'name': 'thresholds', 'type': 'NUMBER'},
  13. ],
  14. 'source': roc_file
  15. }]
  16. }
  17. with file_io.FileIO('/mlpipeline-ui-metadata.json', 'w') as f:
  18. json.dump(metadata, f)

Visualization on the Kubeflow Pipelines UI:

ROC curve visualization from a pipeline component

Table

Type: table

Required metadata fields:

  • format
  • header
  • source

Optional metadata fields:

  • storage

The table viewer builds an HTML table out of the data at the given source path, where the header field specifies the values to be shown in the first row of the table. The table supports pagination.

Specify 'storage': 'inline' to embed CSV table content string in source field directly.

Example:

  1. metadata = {
  2. 'outputs' : [{
  3. 'type': 'table',
  4. 'storage': 'gcs',
  5. 'format': 'csv',
  6. 'header': [x['name'] for x in schema],
  7. 'source': prediction_results
  8. }]
  9. }
  10. with open('/mlpipeline-ui-metadata.json', 'w') as f:
  11. json.dump(metadata, f)

Visualization on the Kubeflow Pipelines UI:

Table-based visualization from a pipeline component

TensorBoard

Type: tensorboard

Required metadata Fields:

  • source

The tensorboard viewer adds a Start Tensorboard button to the output page.

When viewing the output page, you can:

  • Click Start Tensorboard to start a TensorBoard Pod in your Kubeflow cluster. The button text switches to Open Tensorboard.
  • Click Open Tensorboard to open the TensorBoard interface in a new tab, pointing to the logdir data specified in the source field.
  • Click Delete Tensorboard to shutdown the Tensorboard instance.

Note: The Kubeflow Pipelines UI doesn’t fully manage your TensorBoard instances. The “Start Tensorboard” button is a convenience feature so that you don’t have to interrupt your workflow when looking at pipeline runs. You’re responsible for recycling or deleting the TensorBoard Pods using your Kubernetes management tools.

Example:

  1. metadata = {
  2. 'outputs' : [{
  3. 'type': 'tensorboard',
  4. 'source': args.job_dir,
  5. }]
  6. }
  7. with open('/mlpipeline-ui-metadata.json', 'w') as f:
  8. json.dump(metadata, f)

Visualization on the Kubeflow Pipelines UI:

TensorBoard option output from a pipeline component

Web app

Type: web-app

Required metadata fields:

  • source

Optional metadata fields:

  • storage

The web-app viewer provides flexibility for rendering custom output. You can specify an HTML file that your component creates, and the Kubeflow Pipelines UI renders that HTML in the output page. The HTML file must be self-contained, with no references to other files in the filesystem. The HTML file can contain absolute references to files on the web. Content running inside the web app is sandboxed in an iframe and cannot communicate with the Kubeflow Pipelines UI.

Specify 'storage': 'inline' to embed raw html in source field directly.

Example:

  1. static_html_path = os.path.join(output_dir, _OUTPUT_HTML_FILE)
  2. file_io.write_string_to_file(static_html_path, rendered_template)
  3. metadata = {
  4. 'outputs' : [{
  5. 'type': 'web-app',
  6. 'storage': 'gcs',
  7. 'source': static_html_path,
  8. }, {
  9. 'type': 'web-app',
  10. 'storage': 'inline',
  11. 'source': '<h1>Hello, World!</h1>',
  12. }]
  13. }
  14. with file_io.FileIO('/mlpipeline-ui-metadata.json', 'w') as f:
  15. json.dump(metadata, f)

Visualization on the Kubeflow Pipelines UI:

Web app output from a pipeline component

Source of examples on this page

The above examples come from the tax tip prediction sample that is pre-installed when you deploy Kubeflow.

You can run the sample by selecting [Sample] ML - TFX - Taxi Tip Prediction Model Trainer from the Kubeflow Pipelines UI. For help getting started with the UI, follow the Kubeflow Pipelines quickstart.

The pipeline uses a number of prebuilt, reusable components, including:

Usage in lightweight python components

For lightweight components, the syntax is slightly different. You can refer to the lightweight python component notebook example to learn more about declaring output visualizations.

Next step

See how to export metrics from your pipeline.

Last modified 28.09.2020: Correct metadata inaccuracy in the output viewer docs (#2214) (ea56c112)