The future of the JHipster Registry

[Pierre BESSON]

Hi guys,

I am a great believer of the potential of our registry and how it can be the way for us to differentiate our stack and provide value on top of spring boot and spring cloud.

To integrate the registry with external discovery and configuration systems (consul and kubernetes) we can base the work on the existing abstractions provided by Spring Cloud then after this point, we can imagine going further and integrating with the service meshes that build on top of those two systems (connect for consul and istio for kubernetes).

As a side project, I have been testing out some ideas on the registry to be able to connect it to alternative discovery and config systems and the great news is that it works (mostly), see https://github.com/jhipster/jhipster-registry/pull/318

Basically what this means is that you can use the JHipster Registry without the embedded Eureka and server and to make it connect to either Consul or Kubernetes to locate the available JHipster apps. This makes our custom UI reusable to view the available instances, metrics screens and swagger API.

Another interesting direction for the future of the registry would be to integrate it with external monitoring systems so that it could function as a kind of ops center. This direction could be explored in the future after the work on micrometer is complete.

Best regards,

Pierre Besson

[Deepu K Sasidharan]

That sounds really great Pierre. +1 from me

Thanks & Regards,

Deepu

--
You received this message because you are subscribed to the Google Groups "JHipster dev team" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jhipster-dev...@googlegroups.com.
To post to this group, send email to jhipst...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jhipster-dev/CAL_TkmkCJ06RqQpKT2bys4wmUJbWCss6BJZ0%3DRkebai17U%3DdCg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Daniel Franco]

Hello,

Use Istio for Service mesh, Consul for kV and Vault for secrets would be awesome.

This is what I am planning at Office.

To view this discussion on the web visit https://groups.google.com/d/msgid/jhipster-dev/CAFXry54TwO4SopykdbnkR4eEyVtP-THBXPZCJoHkiEzkSwayNQ%40mail.gmail.com.

[Julien Dubois]

At my client they have an interesting setup in mind, as they also want to use a Kubernetes sidecar for OAuth2 authentication (it seems Keycloak has one).

So if you use all the Kubernetes features for Ingres, config, etc, you basically don't need the gateway and the registry. At the cost of totally losing control of many things in your app, which can also lead to missed business requirements. Also, that's not very devops friendly as you can't have your prod system on your laptop easily anymore.

But there's an interesting trend here, that is worth studying. 

To view this discussion on the web visit https://groups.google.com/d/msgid/jhipster-dev/CADVdDoYLpc_d1C%2BRTnJTXPQAdWCThFNmi-6wp9vhicNdyhxBbg%40mail.gmail.com.

[Daniel Franco]

Basically it is the same trend, just that for kV and secrets we use hashicorp tools for multi-cloud sync.

[Gaël Marziou]

Use Istio for Service mesh, Consul for kV and Vault for secrets would be awesome.

I also love HashiCorp tools but they are really expensive if you want to use them at their full potential, either in terms of licence fees or infrastructure costs. So I think that free tools like Eureka should be default choice.

Gael

[Julien Dubois]

Oh you are pointing out something very important for the future, as I see our industry is moving in strange ways at the moment on the OSS side:

- Yes we should always be "OSS first". As we are an OSS project, and as we believe in the power of OSS. Also, I want everybody to benefit from JHipster without needing to pay for commercial software: many of our users are in South-East Asia, Africa, South America... We also have lots of students and small start-ups. I really don't want to ask them to pay for anything when using JHipster.

- This is why I have started to switch from Intellij IDEA to VS Studio Code when I do my demos.

For me we do support 4 kinds of non "fully OSS" products:

- "True" commercial software like Oracle or MS SQL Server

- Commercial SASS like Okta or Heroku

- Software with an OSS development model but which is sold commercially (Red Hat software like Keycloak, maybe the Oracle JDK also?)

. Software with an OSS core but which is not OSS in the end (Kafka, DataStax, Couchbase)

None of those are currently our default options, as we are OSS-first, but still they are available and I'm very happy about them as they are great products and solve real problems for our users. For exemple, I tell daily to my customers to buy either Keycloak or Okta for their security needs - at some point, if you want something really good, and that includes support, you need to pay for it.

-> for me Hashicorp falls in either the 3rd or 4th category, so there's no issue at all in using them, but it just shouldn't be our default solution.

Julien

--

You received this message because you are subscribed to the Google Groups "JHipster dev team" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jhipster-dev...@googlegroups.com.
To post to this group, send email to jhipst...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jhipster-dev/3522eaed-1219-400e-bb8e-bf7b295fffd2%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Julien Dubois

Twitter: @juliendubois

[Julien Dubois]

Hi Pierre,

I had a look at your work on Consul & Kubernetes (personal message: for Kubernetes this is exactly what we need for my current client, we'll need to discuss if you can come with me a few days to work on this).

I'm not sure of the approach: shouldn't we better include those configurations in all of the applications directly? That would get rid of the JHipster Registry. I understand we disagree here, so we need to discuss this.

Julien

[Pierre BESSON]

Hi Julien,

I'm still thinking on how to approach this but I really want to keep the Registry. As for using spring-cloud-kubernetes in all applications, this was my idea initially but now I don't think it's such a good idea as it bypasses the Kubernetes or Istio load-balancers (https://github.com/IBM/spring-cloud-kubernetes-with-istio). As for managing configuration with spring-cloud-kubernetes by reading the configmaps directly from the apps, it looks like a pretty bad approach and is much less usable than using a spring cloud config server.

I would much rather have apps not trying to be smart and rely on kubernetes API, however for the JHipster Registry it would be interesting as it can provide some kind of dashboard.

--

You received this message because you are subscribed to the Google Groups "JHipster dev team" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jhipster-dev...@googlegroups.com.
To post to this group, send email to jhipst...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jhipster-dev/be69eae7-d2b1-461e-a3a9-c232696c05d4%40googlegroups.com.

[Julien Dubois]

Hi Pierre,

For Istio support: I'm pretty sure we can make it work, and for me the goal would be to just use the Spring REST template without any Netflix lib (Ribbon/Hystrix). So you do a direct HTTP request, and the magic of Istio will take care of load balancer and service breaking. That would remove a lot of code and configuration, at the expense of being tied to Kubernetes/Istio.

For the config server: I don't get why Spring Cloud Config is more usable than a Kubernetes Config Map. You get the same configuration, per application name (I didn't see the profile support in Kubernetes but I guess there's one), and also context refreshing works the same. In the end, the JHipster Registry would just be a "proxy" hiding Kubernetes config maps, and I don't understand what you gain from it.

Of course, you would then be totally tied to Kubernetes, and probably even to your Kubernetes provider, and that's what the big cloud vendors want you to do. It would also be a poor devops experience, except if JHipster does the work of generating a Docker Compose config for dev which is similar to the Kubernetes config in prod. So you have quite a bit of vendor lock-in, and also a bad dev experience as you rely on some very heavy machinery, but you would replace many parts which are actually handled by JHipster, hopefully by something which does it a better.

Julien

[Pierre BESSON]

You could reproduce something like Spring Cloud Config with only Kubernetes config maps but it would not be able to simply connect to a git repository like we do now. There are also many interesting integrations in spring cloud config (profiles support, multi source support, vault). Profile support would also be manual rather than automatic as you would have to manually set configmap to files mapping for each application.

Anyway, let's discuss it face to face and come up with a plan.

[Julien Dubois]

Sorry: I'm just starting to learn this part on Kubernetes

Now I realize that Kubernetes configurations can also be exposed as environment properties (probably not very secure: if you connect to your app with JMX, you will see them), but also as files mounted on the application's host. This is even the recommended way to share secrets (which is something we will all want to do).

So with this approach:

- no need to use the JHipster Registry at all

- we have all the power we need, as we use files like the application-*.yml we already have

- no need to use spring-cloud-kubernetes at all for configurations (it's probably still needed for service discovery), and it's even the recommended way to do it (from Kubernetes)

- it's easy to do "devops": you just have files in a directory that come from different sources.

Pierre, indeed we need some big brainstorming here: maybe we can have time this week, otherwise we'll find a way next month.

Julien

[David Steiman]

Just my few cents on spring cloud config vs kubernetes config map is that with spring cloud config services can reload configurations during runtime, while with kubernetes you need to restart the app before config changes taking effect.

cheers

To view this discussion on the web visit https://groups.google.com/d/msgid/jhipster-dev/CADNXADH3Bnz1632dDHsw4AQqx2-U%3Dw7A6bd0WQ-Sr1g-HwjFJw%40mail.gmail.com.

[Julien Dubois]

Thanks a lot David, then there are two options for reloading the config with Kubernetes:

- Configure it to restart the instance when a config is changed -> restart the Docker container instead of the Spring Container, which is slower but does basically the same thing (and if you do blue/green deployment this can be better)

- Call the Spring Actuator endpoint which can force the reload of the Spring configuration -> that's probably just a good curl request to run, that would do the same thing as Spring Cloud Config

The more I think of this:

- Kubernetes could replace service discovery and configuration as we do it today (and remove most of the need for the JHipster Registry)

- Istio could replace Ribbon/Hystrix (Istio is much more costly in terms of resources, and cannot handle advanced business case, but it's good enough for most people and also have many advantages over the Netflix stack)

So we could have a Kubernetes/Istio setup that makes a microservice look a lot like a monolith, which is really good for our users. And for us, we would have a great job of configuring everything, as always, as the Kubernetes dev experience is horrible - so there's something important to do to improve the dev and devops experiences.

Julien

[Daniel Franco]

I agree with Julien, using Kubernetes/Istio architecture we do not need anymore Registry app.

Even Consul seems not needed anymore, I would just use it to update configuration on the fly.

For me we should focus on Kubernetes + Istio + Prometheus + Grafana, everything using helm template.

Adding Vault for me is a must.

Daniel

To view this discussion on the web visit https://groups.google.com/d/msgid/jhipster-dev/CADNXADE_n02M1OC-w6uh7aGN4Zgojd-%2BqBO9n4YyaqXnyA_uhQ%40mail.gmail.com.

[Pierre BESSON]

Hi David,

I'm curious to know your experience with live spring configuration refreshes. For me, I'm always afraid that it won't work completely especially because I know of some properties that would require a full restart to be taken into account. So in the end I think it's much cleaner to always restart the app cleanly. Personally I view a config update to be similar to releasing a new version because it can potentially break things so I'm wary of dynamic config.

To answer Julien:

> Call the Spring Actuator endpoint which can force the reload of the Spring configuration -> that's probably just a good curl request to run, that would do the same thing as Spring Cloud Config

Yes you could do this, you would need depend on "Spring Cloud Context" (setting up a bootstrap application context) but without Spring Cloud Config (https://cloud.spring.io/spring-cloud-commons/2.0.x/single/spring-cloud-commons.html#_spring_cloud_context_application_context_services). We could actually enable this for monoliths as well.

Le mar. 18 déc. 2018 à 09:12, David Steiman <adina...@gmail.com> a écrit :

[Julien Dubois]

Oh yes totally agree with you Pierre -> I was just adding this to have the same capability as Spring Cloud Config, but I wouldn't recommend this. It's much easier/cleaner to restart the container, and if you do blue/green deployment it's also better for end-users.

Julien