Hyrax Request Log Content

[Nathan Potter]

Greetings Hyrax Users,

I am writing seeking your thoughts and concerns regarding some proposed changes to the Hyrax log output.

I am hoping to make some changes to the log format. The background is that some of our users are unhappy with the free-form log content, and they have asked for more structure. Currently the log receives "request" log lines. It also receives a number of additional log lines that I describe as "informational", "timing", and "errors". The current state of affairs seems to be that people are determining the type of log line by counting the fields and inspecting some of the field content.

My goal is to add a new field to the log output to differentiate these message types.

Example of Current Log Lines:

1601667554|&|79813|&|OLFS|&|0:0:0:0:0:0:0:1|&|Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0 Safari/605.1.15|&|EE9466EBC693103543DB2E78B4D27CCE|&|-|&|1601667554458|&|14|&|HTTP-GET|&|/opendap/hyrax/data/nc/fnoc1.nc.dmr.xml|&|latitude%5B0:1:719%5D|&|BES|&|get.dmr|&|dap2|&|/opendap/hyrax/data/nc/fnoc1.nc

1601667554|&|79813|&|elapsed_us|&|1061|&|start_us|&|1601667554471419|&|stop_us|&|1601667554472480|&|-|&|DMR++ retrieval.

1601667599|&|79734|&|Master listener caught SIGTERM, exiting with SERVER_NORMAL_SHUTDOWN

1601667606|&|79813|&|PPTConnection::receive() - Failed to read chunk header


I propose to add a new field, in the third position, that defines the type of log line that follows:

1601667554|&|79813|&|request|&|OLFS|&|0:0:0:0:0:0:0:1|&|Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0 Safari/605.1.15|&|EE9466EBC693103543DB2E78B4D27CCE|&|-|&|1601667554458|&|14|&|HTTP-GET|&|/opendap/hyrax/ngap/providers/GHRC_CLOUD/collections/RSS%20SSMIS%20OCEAN%20PRODUCT%20GRIDS%20DAILY%20FROM%20DMSP%20F16%20NETCDF%20V7/granules/f16_ssmis_20200512v7.nc.dmr.xml|&|latitude%5B0:1:719%5D|&|BES|&|get.dmr|&|dap2|&|https://d1jecqxxv88lkr.cloudfront.net/ghrcwuat-protected/rss/rssmif16d__7/f16_ssmis_20200512v7.nc

1601667554|&|79813|&|timing|&|elapsed_us|&|1061|&|start_us|&|1601667554471419|&|stop_us|&|1601667554472480|&|-|&|DMR++ retrieval.

1601667599|&|79734|&|info|&|Master listener caught SIGTERM, exiting with SERVER_NORMAL_SHUTDOWN

1601667606|&|79813|&|error|&|PPTConnection::receive() - Failed to read chunk header

The intent of making these changes is to enable downstream log processing to better categorize and ingest the log content.


Thoughts?


Sincerely,


Nathan



= = =
Nathan Potter ndp at opendap.org
OPeNDAP, Inc. +1.541.231.3317