How to extract values from log line for log based metric?

[Attila-Mihaly Balazs]

Hello,

I'm trying to set up a log based metric based on some values extracted from the log lines. However I can't seem to find the field from which to extract the log line(s). I tried "textPayload" and "protoPayload.line[0]", however neither seem to work. Also, I couldn't find an example for this case in the documentation (which seems to show log based metrics based on other fields like "status code" or "user agent").

Did anyone set up a log based metric based on values extracted from the logged line(s)? If so, which field did you use?

Thanks,
Attila Balazs

[Yannick (Cloud Platform Support)]

Hello Attila, I believe you want to be looking into Logs-based Metric Labels which allow you to specify a regular expression to run on your logs when defining them. To make you you understand the entire picture, you should read the entire Logs-based Metrics section.

[Attila-Mihaly Balazs]

Hello Yannick,

Thank you for your reply. Unfortunately it doesn't help me move forward. I think I read all the relevant documentation, searched around on the interwebs and played around with different settings. I *did* mange to set up log based metrics using other information (not the log lines).

However my use-case is the following: let's say that I have the following logging statement in my code:

  logging.info("Frobbed %d widgets", widget_count)

And I would like to set up a log based metric to see how many widgets I frob over time. So I go to log based metrics and try to create a new one:

- in the search box I put "protoPayload.resource:/my/url/that/logs"
- Name: frobber
- Add a new label
- Name: frobber-count
- Label type: Integer
- Extraction regular expression: Frobbed (\d+) widgets
- Field name: <- what to put here?

Again, I tried textPayload and protoPayload.line[0], but neither seem to work.

[Yannick (Cloud Platform Support)]

I see. I'm not sure a metric is the right tool for adding up int values extracted from your logs. If you always have 1 log entry per frobbing you will be able to count the volume, but otherwise you would most likely going to need a different system.

[Attila-Mihaly Balazs]

Hello Yannick.

Fortunately Kirk provided an answer on stackoverflow: https://stackoverflow.com/a/47597013/1265

Apparently the field name I'm looking for is "protoPayload.line.logMessage" which - unfortunately - isn't shown anywhere in the UI. I tried it out and it works as expected.

I also suspect (though I didn't try this) that using the google-cloud-logging library (so not the built-in logging adapter) I could emit log messages with special fields that could be aggregated in a more straight-forward manner (without having to rely on the format of the log lines / regexes to parse them): https://googlecloudplatform.github.io/google-cloud-python/latest/logging/logger.html

It would be really nice if this feature ("structured logging" integrated with log based metrics / stackdriver) would be offered out of the box, since monitoring is a big part of running an app in production.

Attila