Disable Jet diagnostics module

[Lukáš Herman]

Hi Jet Team,

is there any command line switch to disable Jet diagnostics module when -Dhazelcast.diagnostics.enabled=true? 

With regards

Lukas Herman

[c...@hazelcast.com]

Hi Lukas,

I'm quite not sure I understand what you mean by "Jet diagnostics module"? Could you clarify?

Regards,

Can

[Lukáš Herman]

Hi Can,

i mean  "..module=jet.." entries in diagnostics.log:

02-03-2019 08:47:27 1551516447241 BuildInfo[

                          Build=20180913

                          BuildNumber=20180913

                          Revision=6ffa2ee

                          Version=3.10.5

                          SerialVersion=1

                          Enterprise=false

                          JetVersion=0.7.2

                          JetBuild=20181219

                          JetRevision=ba00a0e]

...

02-03-2019 08:58:27 1551517107241 Metric[[module=jet,cooperativeWorker=0,unit=count,metric=iterationCount]=604609]

02-03-2019 08:58:27 1551517107241 Metric[[module=jet,cooperativeWorker=0,unit=count,metric=taskletCount]=122]

02-03-2019 08:58:27 1551517107241 Metric[[module=jet,cooperativeWorker=1,unit=count,metric=iterationCount]=561345]

02-03-2019 08:58:27 1551517107241 Metric[[module=jet,cooperativeWorker=1,unit=count,metric=taskletCount]=121]

02-03-2019 08:58:27 1551517107241 Metric[[module=jet,cooperativeWorker=2,unit=count,metric=iterationCount]=603201]

...

The issue is that each round (in our fairly complex case) generates > 4000 lines into diagnostics.log.

With regards

Lukas

Dne čtvrtek 14. března 2019 14:24:44 UTC+1 c...@hazelcast.com napsal(a):

[Can Gencer]

Hi Lukáš,

Those are metrics and they are written as part of the diagnostics. I don't think there's any way to only exclude Jet metrics from that plugin, the writing is controlled by setting METRIC_LEVEL (jet metrics have level INFO) and also by the period of seconds. If you set hazelcast.diagnostics.metrics.period.seconds to 0 then it should not print any metrics, but this will include all metrics, not just Jet ones.

Jet also exposes metrics through JMX, so perhaps that might be another way to monitor them.

--
You received this message because you are subscribed to the Google Groups "hazelcast-jet" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hazelcast-je...@googlegroups.com.
To post to this group, send email to hazelc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/hazelcast-jet/4c84d5c9-e05b-4326-84b6-3785a3b3e4cd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Lukáš Herman]

Hi Can,

thanks for explanation. I was somehow expecting this functionality can be enabled/disabled on demand, similar to SystemLog plugin control via hazelcast.diagnostics.systemlog.enabled.

Is there any reason why some diagnostics plugins output can be controlled and some are mandatory? 

With regards

Lukas

Dne pátek 15. března 2019 12:03:46 UTC+1 Can Gencer napsal(a):

[Can Gencer]

Each diagnostic plugin is doing different things, and in this case the metrics diagnostic plugin (which dumps all known metrics to file) isn't very flexible except for changing  ProbeLevel and the timeout. 

I think it would be fairly easy to add another property which can have a regexp or similar to filter in/out certain metrics? The implementation of the plugin is trivial and is available here:

https://github.com/hazelcast/hazelcast/blob/master/hazelcast/src/main/java/com/hazelcast/internal/diagnostics/MetricsPlugin.java

To view this discussion on the web visit https://groups.google.com/d/msgid/hazelcast-jet/23a585d0-616c-4587-9911-02616ef91888%40googlegroups.com.

[Lukáš Herman]

Hi Can,

now I can see the metrics are dumped by single plugin, I was somewhat confused by "module=jet" metric name, introduced by probe tagging (which is by itself a great feature). It would be excellent to see all probe categories consistently tagged across the whole metrics scope, so that metrics plugin parameter can just contain .tags=abcd,efgh to select only interesting tags for output for the start. What is your opinion on that?

regards

Lukas

 

Dne pondělí 18. března 2019 10:07:14 UTC+1 Can Gencer napsal(a):

[Can Gencer]

Hi Lukas,

Yes this is in progress, currently only Jet is using this convention but the plan is to migrate all HZ metrics to this convention as well. I'm quite not sure what version this will be implemented in though.

To view this discussion on the web visit https://groups.google.com/d/msgid/hazelcast-jet/ada9ae11-8c26-4af4-bf55-1bc161b98338%40googlegroups.com.