Updates about switching from Dropwizard to Micrometer

[Clément Dessoude]

Hi everyone !

I think I should give you some updates more regularly about what I am doing on Micrometer and ask you for feedback and indications ! So there it is.

General Information:

• I put all of the code in a github repo (https://github.com/clement26695/JHipsterMicrometer).
• In the repo, there is:
  • a file called TODO.md where I put some modifications that I need to do or some interrogations I have.
  • a file called all-metrics-output.json (also attached to the mail), which represent all the metrics I managed to retrieve from Micrometer
    

What did I do ?

1. Create a sample app (Monolithic, in angular)
   
2. Remove Dropwizard and add Micrometer
3. See if I could get the same metrics page as it was before

The issue is that Micrometer does not give exactly the same metrics as Dropwizard, and it gives it in a different format.

• In a first place, I created a custom endpoint called all-metrics to see all metrics given by Micrometer. (I saved the result of one call to this endpoint in the all-metrics-output.json file)
• After trying to parse the result of a request to this endpoint in the frontend, I realised it was easier to create another endpoint which could give exactly what I want. This endpoint, that I called "metricsPageEndpoint" for the moment, is a work in progress.
• I changed the typescript and html page to reflect the modifications (screenshots attached to the mail)

Some questions I have right now:

• There are some new pieces of information that Micrometer provide, but I don't have enough experience to know which ones are important or not. So if there are some piece of information that need to be added, let me know.
• I managed to re-create a big part of the metrics page. But there are some parts on which I am not so sure. For instance about the JVM Memory Metrics (do we want to display all of the jvm spaces ? Is it the ratio jvm.used/jvm.max which is interesting or jvm.used/jvm.commited...)
• What information about the garbage collector do we need to display ?

---

• About Service Statistics (and percentiles and histograms more generally):
  • The service appears in the list of metrics only if it had been called once.
  • It disappear if it has not been called during the reporting period (see management.metrics.export.prometheus.step in application.yml)
    • I set the reporting period at 9223372036854775807 seconds so that the service does not disappear but I guess it broke some other tools
      
    • Do we have to (and is it even possible) to have 2 different meterRegistry for having 2 different reporting step (one for metrics page and one for exports) ? Do we want to do that ?
      

Thank you in advance for your help and the advice you can give me :)

Clément

PS: sorry for not keeping you posted about that, I'll try to give you some updates more regularly in the future...

[Julien Dubois]

Thanks Clément!

Micrometer is very important to me, and a necessary migration. But unfortunately it doesn't seem to attract a lot of interest -> could you copy/paste your email as a ticket on https://github.com/jhipster/generator-jhipster/issues ? I'll tweet it to attract more people.

Julien

--
You received this message because you are subscribed to the Google Groups "JHipster dev team" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jhipster-dev...@googlegroups.com.
To post to this group, send email to jhipst...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jhipster-dev/CAPHmpfNjuje0Erq4CCU7CAO2ip7n%2BeXE-g-ME3g-ssZRsirzoQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

Julien Dubois

Twitter: @juliendubois

[Frederik Hahne]

Hi Clément!

Looks cool so far. We have also migrated some parts of dropwizard to micrometer (basically when switched to SB 2.x), but not jhipster apps. I personally think we should not add multiple registries. If an ednpoint doesn't appear on the metrics page thats at least fine for me, as I can see it is not used very much.

Regarding JVM memory metrics and gc, we often ask that ourselves. I would say it should somehow indicated if there might be a memory leak. E.g. gc runs, but some parts are not collected. i think a growing old gen space can be an indicator for a leak, but you can only tell over time. Maybe someone has more deep knowledge of the jvm internals

To view this discussion on the web visit https://groups.google.com/d/msgid/jhipster-dev/CADNXADGieR8HAW-06NAVe7nX58yRzgcneo%2B0OF6dqB7bMDXOKQ%40mail.gmail.com.

[Christophe Bornet]

Looking forward for the Micrometer integration as Dropwizard doesn't work with Webflux. Gambatte !

To view this discussion on the web visit https://groups.google.com/d/msgid/jhipster-dev/CACh2FPTA1hvPkuc8swtKJC%2BVkFduci%3DWaXEVsx6q_XQ8TAa6JA%40mail.gmail.com.

[Pierre BESSON]

Hi guys,

Sorry for the late reply.

First great work Clément !

I will explain a bit more the issue Clement ran into with the histograms and percentiles. Actually micrometer works very differently from dropwizard in that the value it reports in the percentiles is the value only for the current reporting time window (what is called "step" in the properties). This means that if the app is reporting metrics every minute and there were no activity in the last minute then the percentiles are reset to 0.

The solution is to set a very high number for the step but this is not optimal when analysing the metrics in prometheus for example. So a solution could be to setup two metrics registries one with a very big step duration and the other with a short step duration but this would cause twice the overhead for the instrumentation code.

The alternative would be to consider that this is fine and we are OK with seing the metrics percentile only for the current reporting time frame. For me this is the right choice even if it's different from what we provide right now as the metrics screen is much less important than being able to collect and report those metrics over time.

Also Clément, there is a good news. Jon Schneider, micrometer's creator is coming to Paris on December 4th and 5th https://twitter.com/jon_k_schneider/status/1057076222520369152?s=19

So you will have the occasion to meet him in person and ask him all the questions you want.

Best regards,

Pierre

[Julien Dubois]

Indeed, we should definitely block our "JHipster Day" at Ippon on that date.

I have no idea where this conference will take place, but the whole Spring team will be in Paris.

To view this discussion on the web visit https://groups.google.com/d/msgid/jhipster-dev/CAL_Tkmmp4BkC26AV0%2BuYDUACpUu0YjW49fLPAFG5q4UtFWK1-A%40mail.gmail.com.

[Deepu K Sasidharan]

Guys, please also keep in mind that we don't have to exactly recreate what we have now with DW, if micrometer provides better or different stats, IMO its ok to visualize them better. We could even show charts/graph using a simple lib like angular-charts/react-charts if needed.

Thanks & Regards,

Deepu

To view this discussion on the web visit https://groups.google.com/d/msgid/jhipster-dev/CADNXADFt60T3AeeaQtKzSSH0nY1yt8bPgL3s%2BxN6%3DkcFsc6SqA%40mail.gmail.com.

[Sendil Kumar N]

Great work Clément !

Yeah, I do agree with Deepu here, instead of retrofitting we can use it as such.

Multiple registries sound complicated and I am curious to know what Micrometer's team thinks about this.

> do we want to display all of the jvm spaces ?

I think this will be cool to display this and the GC. Especially for people using JVMs like Open J9 with different Gcs. It does give a clearer picture.

Cheers

Sendil 

To view this discussion on the web visit https://groups.google.com/d/msgid/jhipster-dev/CAFXry57ZFSsVNnhK_Mx9fHhweNwDtvfu4cC7retbhgen%2Bzrm-A%40mail.gmail.com.

[Clément Dessoude]

Hi all !

I made a few PR so that you can check an review what I'm doing on this. There are 3 PR:

• One in ng-jhispter : the idea is to move the different elements of the metrics page into the libs so that it can be easily maintained and upgraded.
• One other in the jhipster framework
• Creation of an endpoint we can request to have the list of the metrics we want to display in a format we can easily use in the front.
• We also configured the app to enable (optionnaly) metrics export to the jhipster-console. 
  
• A last PR in the generator-jhipster, to apply the modification in a new generated app.

For the moment, it is working for me with the generation of monolith, with front in angular. It looks like this :

It should not be very difficult to do the same in react.

What needs to be done :

• Some metrics can be added, according to the app generated, inlcuding
  
• Hazelcache and EhCache
• Hystrix
• Hibernate
• Kafka
• More metrics on databases (particularly postgres)
• Tomcat
• Jetty
• Logback (number of logs with level debug, info, warn...)
• Add some tests
• See how to do the same with microservices

Best regards,

Clément

[Pierre BESSON]

Very impressive work Clément. What is incredible is that the Console is working out of the box, Micrometer being able to export metrics using the Dropwizard Slf4jReporter so the export format is the exact same. We really ought to thank Jon Schneider for this.

To view this discussion on the web visit https://groups.google.com/d/msgid/jhipster-dev/CAPHmpfNxxo0_%3DZqVCg5JksxgQ5eRfs3ncfc%3DDkGfVVoZYMNhBw%40mail.gmail.com.