Logging

Every workflow or pipeline you execute through Hop Gui or Hop Run and a lot of the other tools generate logging as logging text in the most obvious form.

Hop allows data developers to influence how, where and at which level of detail logging information is generated and where it is written to.

Levels

Descriptions

The amount of text logging generated depends on the logging level which is used when executing a workflow or pipeline:

LevelDescription

NONE

No logging is generated

ERROR

Only errors are reported

MINIMAL

The bare essential, nothing more

BASIC

The standard logging which tries to be concise and informative

DETAILED

More logging information about what happens behind the scenes is reported

DEBUG

A lot of information is produced, often with specific values being reported

ROWLEVEL

Values are reported on an individual row level

Runtime levels

The logging level can be specified when executing using hop-run or in the Hop GUI. It is also possible to use the debug plugin to set custom logging levels on a transform:

Transform action - show logging options

You can set a custom logging level for this particular transform and only for certain rows of data:

Setting a custom transform logging level

You can do likewise for a workflow:

Setting a custom action logging level

Hop GUI logging

The logging file of the Hop GUI is saved in the audit/ folder (or in the folder set by HOP_AUDIT_FOLDER) in the hopui.log file.

Tools logging

If you want to send logging of tools like hop-run or hop-conf to a logging file you need to do this simply by piping the text stream to a file.

Logging plugins

Actions

Write to log

The Write to log acction writes a specific string to the Hop logging system.

Check the Write To Log page for more details.

Metadata Types

Pipeline Log

A pipeline log allows to log the activity of a pipeline with another pipeline.

Check the pipeline log and logging reflection pages for more details.

Workflow Log

Allows to log the activity of a workflow with a pipeline.

Check the workflow log and logging reflection pages for more details.

Perspectives

Neo4j

Hop can write execution logs for workflows and pipelines to a Neo4j database.

Check Neo4j Perspective page for more details.

Transforms

Write to Log

This transform writes information to the Hop logging system.

Check the Write To Log page for more details.

External plugins

The external hop-plugins repository contains the Send information using syslog action and Send message to syslog transforms.