mtail + graphite

[Сева]

Hello, thank you in advance for responding.

I want to send nginx logs to graphite using mtail.

Nginx logs format:

log_format dev_format '$time_iso8601 $server_name $http_x_request_id $status $request_method $request_uri '

                          '$request_length/$bytes_sent $request_time';

Example logs:

2019-04-04T19:38:28+03:00 seva.localhost b2acd367-4cf6-4ef1-9d6c-261a358a4899 200 POST /frontback/conferences/getRoot?=fetch-root-conference 196/231 1.153

2019-04-04T19:38:28+03:00 seva.localhost - 204 OPTIONS /frontback/conferences/getAll?=fetch-conferences 66/251 0.015

2019-04-04T19:38:30+03:00 seva.localhost 48551679-c861-4ca1-adf3-1ed6147924d7 200 POST /frontback/conferences/getAll?=fetch-conferences 269/32402 2.027

My nginx.mtail file:

counter http_requests_total by server_name, request_method, request_uri, status

counter http_requests_length_total by server_name, status, request_method, request_uri

counter http_requests_bytes_sent_total by server_name, status, request_method, request_uri

gauge http_requests_performance_time by server_name, status, request_method, request_uri

#log_format dev_format '$time_iso8601 $server_name $http_x_request_id $status $request_method $request_uri '

#                          '$request_length/$bytes_sent $request_time';

/^/ +

/(?P<time_iso8601>\d{4}-\d{2}-\d{2}T\d{1,2}:\d{1,2}:\d{1,2}\+\d{2}:\d{2}) / +

/(?P<server_name>[0-9A-Za-z\.\-:]+) / +

/(?P<http_x_request_id>[0-9A-Za-z\-:]+) / +

/(?P<status>\d{3}) / +

/(?P<request_method>\S+) / +

/(?P<request_uri>\S+) / +

/(?P<request_length>\d+)\/(?P<bytes_sent>\d+) / +

/(?P<request_time_seconds>\d+)\.(?P<request_time_milliseconds>\d+)/ +

/$/ {

  strptime($time_iso8601, "2006-01-02T15:04:05Z07:00")

  

  tolower($request_method) != "options" {

http_requests_total[$server_name, $request_method, $request_uri, $status]++

  http_requests_length_total[$server_name, $status, $request_method, $request_uri] += $request_length

  http_requests_bytes_sent_total[$server_name, $status, $request_method, $request_uri] += $bytes_sent

  http_requests_performance_time[$server_name, $status, $request_method, $request_uri] = $request_time_seconds * 1000 + $request_time_milliseconds

  }

}

I have 2 main questions.

1) Why I can use only format: 

http_requests_total[$server_name][$request_method][$request_uri][$status]++

this create a lot of useless folder at graphite I wanna use this format a[1, 2, 3]++ by doc https://github.com/google/mtail/blob/master/docs/Metrics.md#labelling

but I have

F0404 17:35:17.745671     241 main.go:68] couldn't start: Compile encountered errors:

compile failed for nginx.mtail:

nginx.mtail:23:34: syntax error

2) How can I transorm symbol "." to "_" ? Graphite think that I wanna create new folder and show me this hell:

 Consequently graphite create dir nginx/mtail (because I must call the file with file extension ".mtail") instead nginx

 Also with "/"

[Jamie Wilkinson]

You're writing the values from mtail directly to graphite I take it?  (Can you share how you run mtail on the commandline?)

I'm afraid to say that you're the first person to tell me that you're doing it, and that means you're going to uncover these sorts of problems.

What did you expect to happen with (1) ?  You should be able to write a[1,2,3]++ or a[1][2][3]++ for a counter called 'a' with three dimensions on it.  Is that the compile error you mean?

In 2, do you mean that the name of the vairable being sent to graphite has too many dots in it, and you want some of them to be _ instead so graphite doesn't create a deep name hierarchy?  Excuse my ignorance, because I haven't used it directly.

--
You received this message because you are subscribed to the Google Groups "mtail-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mtail-users...@googlegroups.com.
To post to this group, send email to mtail...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mtail-users/cca1994c-8a7c-4492-9867-2bfc34d68d3f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Сева]

You're writing the values from mtail directly to graphite I take it?

Yes. how else? I thought in this and sense of mtail. And mtail have --graphite_host_port key

mtail -logs /var/log/nginx/seva_access.log -progs /etc/mtailProgs/nginx.mtail -port 3903 -logtostderr --graphite_host_port=127.0.0.1:2003 --metric_push_interval_seconds=5

 u should be able to write a[1,2,3]++ or a[1][2][3]++ for a counter called 'a' with three dimensions on it.  Is that the compile error you mean?

I wanna use a[1,2,3]++ but take compile error, So in screenshot you have seen   a[1][2][3]++ , but it not what I want

Very happy for your prompt reply, thanks

In 2, do you mean that the name of the vairable being sent to graphite has too many dots in it, and you want some of them to be _ instead so graphite doesn't create a deep name hierarchy?  Excuse my ignorance, because I haven't used it directly.

Yes, exactly !