Configuration Examples

This section provides examples for a wide range of logging scenarios. While not exhaustive of all possibilities (Traffic Server logging is quite flexible), these entries should hopefully get most administrators headed onto the right path.

Unless otherwise noted, the example configurations here are to be applied in logging.config.

Online Event Log Builder

If you need any assistance building your event log, you can try out our online log builder. This is a work in progress, so any comments, critique or suggestions are most welcome.

Emulating Other HTTP Server Formats

Netscape Common

The following figure shows a sample log entry in a Netscape Common log file.

The numbered sections correspond to the following log fields in Traffic Server:

To recreate this as a log format in logging.config you would define the following format object:

common = format {
  Format = '%<chi> - %<caun> [%<cqtn>] "%<cqtx>" %<pssc> %<pscl>'
}

Netscape Extended

The following figure shows a sample log entry in a Netscape Extended log file.

In addition to fields 1-7 from the Netscape Common log format, the Extended format adds the following fields:

To recreate this as a log format in logging.config you would define the following format object:

extended = format {
  Format = '%<chi> - %<caun> [%<cqtn>] "%<cqtx>" %<pssc> %<pscl> %<sssc> %<sscl> %<cqcl> %<pqcl> %<cqhl> %<pshl> %<pqhl> %<sshl> %<tts>'
}

Netscape Extended-2

The following figure shows a sample log entry in a Netscape Extended2 log file.

In addition to fields 1-16 from the Netscape Common and Netscape Extended log formats above, the Extended-2 format adds the following fields:

To recreate this as a log format in logging.config you would define the following format object:

extended2 = format {
  Format = '%<chi> - %<caun> [%<cqtn>] "%<cqtx>" %<pssc> %<pscl> %<sssc> %<sscl> %<cqcl> %<pqcl> %<cqhl> %<pshl> %<pqhl> %<sshl> %<tts> %<phr> %<cfsc> %<pfsc> %<crc>'
}

Squid

The following figure shows a sample log entry in a Squid log file.

The numbered sections correspond to the following log fields in Traffic Server:

To recreate this as a log format in logging.config you would define the following format object:

squid = format {
  Format = '%<cqtq> %<ttms> %<chi> %<crc>/%<pssc> %<psql> %<cqhm> %<cquc> %<caun> %<phr>/%<pqsn> %<psct>'
}

Separating Logs by Origin

The default log_hosts.config file is located in the Traffic Server config directory. To record HTTP transactions for different origin servers in separate log files, you must specify the hostname of each origin server on a separate line in log_hosts.config. For example, if you specify the keyword sports, then Traffic Server records all HTTP transactions from sports.yahoo.com and www.foxsports.com in a log file called squid-sports.log (if the Squid format is enabled).

Important

If Traffic Server is clustered and you enable log file collation, then you should use the same log_hosts.config file on every Traffic Server node in the cluster.

To edit the log hosts list:

1. Enter the hostname of each origin server on a separate line in log_hosts.config.

   webserver1
   webserver2
   webserver3

2. Run the command traffic_ctl config reload to apply the changes.

Hourly Rotated Squid Proxy Logs

The following example demonstrates the creation of a Squid-compatible log format, which is then applied to a log object containing an hourly rotation policy.

squid = format {
  Format = '%<cqtq> %<ttms> %<chi> %<crc>/%<pssc> %<psql> %<cqhm> %<cquc> %<caun> %<phr>/%<pqsn> %<psct>'
}
log.ascii {
  Format = squid,
  Filename = 'squid',
  RollingEnabled = 1,
  RollingIntervalSec = 3600,
  RollingOffsetHr = 0
}

Summarizing Number of Requests and Total Bytes Sent Every 10 Seconds

The following example format generates one entry every 10 seconds. Each entry contains the timestamp of the last entry of the interval, a count of the number of entries seen within that 10-second interval, and the sum of all bytes sent to clients:

mysummary = format {
   Format = "%<LAST(cqts)> : %<COUNT(*)> : %<SUM(psql)>",
   Interval = "10"
}

Dual Output to Compact Binary Logs and ASCII Pipes

This example demonstrates logging the same event data to multiple locations, in a hypothetical scenario where we may wish to keep a compact form of our logs available for archival purposes, while performing live log analysis on a stream of the event data.

ourformat = format {
  Format = '%<chi> - %<caun> [%<cqtn>] "%<cqtx>" %<pssc> %<pscl>'
}
log.binary {
  Format = ourformat,
  Filename = 'archived_events'
}
log.pipe {
  Format = ourformat,
  Filename = 'streaming_log'
}

Filtering Events to ASCII Pipe for Alerting

This example illustrates a situation in which our Traffic Server cache contains canary objects, which upon their access we want an external alerting system to fire off all sorts of alarms. To accomplish this, we demonstrate the use of a filter object that matches events against these particular canaries and emits log data for them to a UNIX pipe that the alerting software can constantly read from.

canaryformat = format {
  Format = '%<chi> - %<caun> [%<cqtn>] "%<cqtx>" %<pssc> %<pscl>'
}
canaryfilter = filter.accept('cqup MATCH "/nightmare/scenario/dont/touch"')
log.pipe {
  Format = canaryformat,
  Filters = [ canaryfilter ],
  Filename = 'alerting_canaries'
}

Summarizing Origin Responses by Hour

This example demonstrates a simple use of aggregation operators to produce an hourly event line reporting on the total number of requests made to origin servers (where we assume that any cache result code without the string HIT in it signals origin access), as well as the average time it took to fulfill the request to clients during that hour.

originrepformat = format {
  Format = '%<FIRST(cqtq)> %<COUNT(*)> %<AVERAGE(ttms)>',
  Interval = 3600
}
originfilter = filter.reject('crc CONTAINS "HIT"')
log.ascii {
  Format = originrepformat,
  Filters = [ originfilter ],
  Filename = 'origin_access_summary'
}