JHipster v8 ideas

[Pascal GRIMAUD]

Hello the team,

I started to think about the v8, and I'm still thinking... :)

The idea would be to split each option into a sub generator / module.

I'm sharing to you these slides with all my ideas:

https://docs.google.com/presentation/d/1OSXKODTCgKgIwtz_TC1ZobPK-syZOeruxiXMNTKn1t0

These slides are public, you should be able to comment.

Don't hesitate to ask questions here and tell me what you think ?

If you want permission to edit, just ask. I'm too lazy to add all of you :D

If we agree, I'll open an official ticket and start my 3 first sub generators (almost done):

- init

- maven

- spring-boot

Then, we could add bounties for each generator, so it will motivate people to code.

Pascal

[Deepu K Sasidharan]

Hi Pascal,

Good work with slides, it highlights the pain points clearly. But there are a few things that we need to keep in mind before we start a big migration

• What kind of developers are the majority of our users (AFAIK these are beginner to medior devs who rely on our expertise and mostly choose the defaults we provide) would the suggested workflow be useable for them and not just power users
• One of the most important promise we make is that everything works together well (we are the glue) thats what differentiates us from things like Atomist which had the a similar workflow you suggested but they were never as successful as JHipster even with millions of VC money poured in 

I think its worthwhile to discuss and brainstorm this proposal over a team meeting. Please don't get me wrong, I'm not opposed to the idea I just want to make sure we still provide the same ease and developer experience to the majority of our users. If we can find a way to do this without breaking current workflow while giving power users more flexibility that would be perfect

Thanks & Regards,

Deepu

--
You received this message because you are subscribed to the Google Groups "JHipster dev team" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jhipster-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jhipster-dev/CALUG8Vsxx3cHMrEac02Xp4B4JsizzGubFvOqxioUs-K8J%3DA%3DRA%40mail.gmail.com.

[Matt Raible]

I love it! 😍

When a co-worker (Micah Silverman) started using JHipster a few years ago, he was puzzled at first. He expected to be able to “add OIDC” after he generated the app. This seems to be along those lines. 

Speaking of JHipster 7 and continued improvements, there’s a couple of things I’d like to see in the near future. 

- Micro Frontends support for React and Vue

- Spring Native support

Josh Long and I are hoping to do a talk on Spring Native + JHipster at the San Francisco JUG on September 29.

Cheers,

Matt

[Pascal GRIMAUD]

Oh, I forgot to mention that all the current workflow won't change ! CLI with questions, or JDL won't change.

You will be able to launch:

- jhipster

- answer to all questions

- and the engine will generate the same project than today

It's just internally, instead of reading template and check all the conditions, it will launch all the following sub generators, by respecting this order:

- init

- maven

- spring-boot

- spring-boot-mvc

- security-jwt

- database-postgresql

- etc.

The pain point will be the "common" files and duplications.

For example, "SecurityConfiguration", there are some common parts between JWT and OAuth2.

With this approach, you'll have 2 different templates for the SecurityConfiguration file. One for JWT, one for OAuth2.

But, it will be easier to maintain, as if you change SecurityConfiguration for OAuth2, you can't break the one in JWT.

Today, I'd prefer to have duplications, rather than all these if else, in templates.

To answer to you, Deepu: "If we can find a way to do this without breaking current workflow while giving power users more flexibility that would be perfect" -> it's exactly my idea :-)

Pascal

[Frederik Hahne]

Hi,

Did not think about it in detail but it looks very good. With these clear separation even in code the term of a stream lead makes much more sense. Looking forward to whatever ist coming!

To view this discussion on the web visit https://groups.google.com/d/msgid/jhipster-dev/CALUG8VsnLm4j-SOrt3m4%3Dfv%3DyponwGWXQR1Cyx-ipPh9KM8CDw%40mail.gmail.com.

[Deepu K Sasidharan]

Yes, I presumed that, and good to know we are on the same page. Yes indeed internally its not as easy as it sounds to do this. We would still have to do a lot of templating to make sure all options work together and we would have to pass data around to all sub-gens and there will be lot of duplications and fragmentation (which is my biggest concern, you know certain flows might be maintained better then others and some flows might be forgotten since they now live in separate templates)

But ya in the end we are actually already pretty diversified in terms of sub-gens. Remember when I split the entire generator into client/server sub gens :P so lets try to split into more subs while trying to keep the final output the same. There are gonna be some some challenges but we have enough smart people in this group to figure it out.

Thanks & Regards,

Deepu

To view this discussion on the web visit https://groups.google.com/d/msgid/jhipster-dev/CALUG8VsnLm4j-SOrt3m4%3Dfv%3DyponwGWXQR1Cyx-ipPh9KM8CDw%40mail.gmail.com.

[Daniel Franco]

I totally agree with this new approach on the generator, this should simplify a lot changes and addition of new features.

And if would also probably simplify integration with blueprints too if we use same approach.

++1

To view this discussion on the web visit https://groups.google.com/d/msgid/jhipster-dev/CAFXry55uS%2B8zu1FMvCP4Sew_i8Mxm7r7snUO_e3YfGtqdAiUfg%40mail.gmail.com.

[Julien Dubois]

Hi everyone,

First of all Pascal, thanks a lot for the vision and the roadmap! You're the one truly leading JHipster today, and not me. And I'm very happy about it :-)

I have a lot of similar ideas, and initially this is why I wanted to try something different with NubesGen. We need to set up some time, so I can share with the core team what I have found out in detail, but here are the main points:

- I believe we should compose stuff, and even try to work with other speciliazed generators. That's the idea with NubesGen: it only does one thing, and work with other tools (like JHipster).

- I used Mustache and not EJS: it seems too simple, but in the end it's a great thing, as it avoids all those impossible to read templates that Pascal shows.

- Performance and testing should be great: on NubesGen, I do all my tests in less than 1 second, and they are way more powerful than what we have on JHipster. Java is just so much better, faster, type-safe....

- Having a REST API is more important than a CLI. I know CLI look cool, but you need to install them, and they can cause security concerns... This is why JHipster Online is so popular, as well as start.spring.io

All those were basically experiments in NubesGen, and I'm surprised by how it is easier to work on this codebase than on JHipster.

Then, there is one thing I hate about what we generate: I believe it's too complicated. I would like far more simple front-end code, and use more Spring auto-configuration in the back-end. Ideally, JHipster would be mostly simple CRUD templates that people can tune to their needs (with a library that people can override for advanced needs).

Julien

To view this discussion on the web visit https://groups.google.com/d/msgid/jhipster-dev/CADVdDoZT_dENF7RLAoWUJov9yDS%2B7YC1sJ6TJjMTvqSKd%2BgVtA%40mail.gmail.com.

--

Julien Dubois

Twitter: @juliendubois

[Deepu K Sasidharan]

Well the problem is we have way too many options and features which would be impossible to do with Mustache and PITA with Java (Imagining the boilerplate already makes me dizzy 😜) or any other static language. Maybe TypeScript could be an option if we want static types as its more flexible but that again would just make stuff complex. Our scale is our main enemy now

Thanks & Regards,

Deepu

To view this discussion on the web visit https://groups.google.com/d/msgid/jhipster-dev/CADNXADG_MeST_gm4HFdwHK%3DtSCNLzsVOVckinmDtU_9dSu%2BScg%40mail.gmail.com.

[Sendil Kumar N]

Awesome stuff Pascal! Loved the slides! 

I do agree we should move towards micro-blueprints and compose them. 

Our templates are getting unimaginably complicated, we should find an intelligent way to generate the template rather than writing them by hand ( but that is way tooo  complex 🤪). 

Sendil

To view this discussion on the web visit https://groups.google.com/d/msgid/jhipster-dev/CAFXry54%3DVTafzyAzHTo%3DPLxbcU%2BEhwqDfi%3DFqT-fPDe%3DuFqGvg%40mail.gmail.com.

[Pierre Besson]

Lots of great ideas here. Glad to see some excitements for the future of project after a long quiet period.

Big +1 for @Pascal idea of modularity and the plan to implement this progressively makes it much more likely to succeed !

+1 for Deepu's idea of Typescript over Java. The npm ecosystem has been JHipster's greatest assets and if we lack types then I believe Typescript would be the solution not Java.

-1 for the REST API. The consequence would be to make it appear less open and free (more like close source SaaS products) and also it would be hard to access existing user's code to modify it. For the security issues, I feel they are more concerns with the npm ecosystem than the fact that it's a CLI. We could also package the CLI as a single binary (https://github.com/vercel/pkg). However the best solution would be to architect the code so that it can be used both as CLI and as API. Right now, the way it's done in Jhipster-online is a huge hack!

I also feel that we didn't talk about the JDL yet. It should be an important part of this "redesign". IMO, the ideal way to work with JHipster would be by editing the JDL file little by little and then successively "applying" the generator on it like you can do with Terraform.

Cheers,

Pierre

To view this discussion on the web visit https://groups.google.com/d/msgid/jhipster-dev/CAFXry54%3DVTafzyAzHTo%3DPLxbcU%2BEhwqDfi%3DFqT-fPDe%3DuFqGvg%40mail.gmail.com.

[Charlie Mordant]

Doesn't look really easy to achieve, but for sure it can be done incrementally, so let's try and see where we find the right balance.

Like Deepu, I'm really concerned about code duplication: it often (frequently) means that one of the option is well maintained, and the other not that well (see blueprints)...

What about composition/partials Mshima pocced ?

Finally, I do not find templates that hard to understand (some are, but ok with a little bit of effort), particularly since derived constants (and it will be better once the whole templates won't have any logic included).

The part I'm more worried about being the js one which is even 'hard for teachers (Van hallen TM)': no options model, no/few object model (no concept of primary key, no inheritance of concepts), no/few design patterns to ease adaptability (like strategy, chain of responsibility, factory...), classes mixing files generation operations, context enrichment and object creation with more than 3k lines...

I would prefer to think about a solid and scalable class model like they have done for spring framework and Jenkins codebase, then see the opportunity we get from it afterwards.

Anyway, brainstorming about improving the code looks to be the must have of the moment (I don't know if I would be able to add a new option in the current situation), because the pain points are real!

Did someone took a look at the approach taken by the other generators? Xtext/xpand/xtend, egf, rails, ... I'm sure that there are some really cool concepts to borrow from the others a trainee or a bounty to benchmark the generation approach?

Charlie

To view this discussion on the web visit https://groups.google.com/d/msgid/jhipster-dev/CAFXry54%3DVTafzyAzHTo%3DPLxbcU%2BEhwqDfi%3DFqT-fPDe%3DuFqGvg%40mail.gmail.com.

[Deepu K Sasidharan]

Excellent points everyone. 

One way to make maintenance easier indeed would be to come up with a base extensible Object model with Typescript which is reflective of the JDL. Then move all logic from templates into these objects with some sort of chain of responsibility or just simple OOP this way templates will become simpler and achieving what Pascal suggested would be less painful and we will have a strong base API with static types that others can easily extend. And yes, Rails should be a great place to look for some inspiration.

To view this discussion on the web visit https://groups.google.com/d/msgid/jhipster-dev/CANHkoGxRkg0ywS2g4%2BFdoKFPaC4x8OqYdk%3DL0MPJLEvYKPe2-A%40mail.gmail.com.

[Julien Dubois]

Concerning Rails: initially I spent some time getting ideas from it, for example our Liquibase change logs are close to Rails database migrations.

They are also a great source of ideas on what we can’t do (like the upgrade process: it’s as bad as ours) or shouldn’t do too much (their CRUD templates are much simpler).

There is a lot to learn from other tech stacks.

[Pascal GRIMAUD]

Thanks all for your feedback.

So I created this ticket https://github.com/jhipster/generator-jhipster/issues/15490 

Now, we can discuss it there too :)

Pascal

[Anthony Viard]

Hi Pascal,

as we discussed before, I was a bit annoyed by the multiplication of the new steps a user should do to achieve a complete generation and also by the duplication of the code between all these sub generators.

1. It seems we can keep the initial behavior aka "Quickstart" so it's fine and provide a more complex usage of JHipster for advanced users.

2. The duplication of code can be avoided by using partials with EJS and gather some code under injected files. This is what we did in the Quarkus Blueprint. Not sure if it's the ultimate solution but It could be something we can test over duplicate the code everywhere. (there are some drawbacks too).

Anyway, I like the idea we can still provide a ton of different ways to use JHipster and keep it simple. If it can be coupled with the possibility to have domain objects uncouple with the data layer it will be awesome.

Thanks for the sharing and the work ;)

To view this discussion on the web visit https://groups.google.com/d/msgid/jhipster-dev/CALUG8VuT_mgpngzeBYGBy8jGnHOzy1BqKvDEe%3DBgDw%2BLdx1qUw%40mail.gmail.com.

[Marcelo Shima]

Hello team and Pascal.

+1.
I completely agree with this move.

I come back to this subject in my mind from time to time.

> What about composition/partials Mshima pocced ?

It’s interesting, but not powerful enough.

As Anthony said, it has some drawbacks and overall IMO it isn't worthy a full migration.

It could be useful to small blueprints. Just put the file in same path and the blueprint version will have priority.

Another task for JHipster 8 is to move every file that contains entity needles to entities-* generators, a `jhipster` regeneration (without —with-entities) should not conflict on those files.

My real concern about modularization is that generators (eg: init/maven) should not need to know logic from others generators.
Just moving a complex template from one generator to another, but keep the inter-dependency is not ideal and makes the migration less worthy.

I think I just have found the solution for inter-dependency.
IMO it can be revolutionary for jhipster.
The implementation consists in two steps ejs templating, some tags will be parsed normally and others will be parsed and merged/injected inside a transform.

Basically the ejs template will create another ejs template and render the final file in a transform.

May sound complicated, but I don’t think the end result is (too much at least, and much better than the current situation).

I won’t go into too much details here, since I will create a PR and it will be better to explain.

Regards.

Marcelo.

To view this discussion on the web visit https://groups.google.com/d/msgid/jhipster-dev/CALUG8VuT_mgpngzeBYGBy8jGnHOzy1BqKvDEe%3DBgDw%2BLdx1qUw%40mail.gmail.com.

[Pierre Besson]

@Marcelo you are right to point out that th crux of the issue is dependency between modules. In some cases it might straightforward for example if I want to add a java dependency in a maven or gradle project we call call the same add-dependency hook. 

But for entities where the generated code depends on the chosen database, we would need to have really strong modularization of the code or have a module's code exposed to other module's code through a standardized interface (maybe by forcing the use of DTOs, enforcing a DDD approach,...). 

To view this discussion on the web visit https://groups.google.com/d/msgid/jhipster-dev/7E628899-5B71-414C-8481-E2F4FD357DB7%40gmail.com.

[Pascal GRIMAUD]

Hello team,

Following our discussion, and as I didn't manage to achieve what I wanted in generator-jhipster, I tried something new:

- a new generator (from scratch), using Java 17: the goal is to generate projects with Java 17 with Hexagonal Architecture

- the generator project is here: https://github.com/jhipster/jhipster-lite

Special thanks to Hippolyte who helped me a lot

You can find more details here: https://docs.google.com/presentation/d/1i0LOJ0GSWNG2-x0zY220IbQc0PVQ2pndQWEuQKGu8n0

This project is a "try". After some discussion with Deepu, I don't think it will replace generator-jhipster as the approach is completely different.

So, nothing will change in generator-jhipster / jhipster-bom, you can continue to contribute and improve it :-)

But, if you want to have a look at this approach, don't hesitate to ping and discuss with me.

Pascal

[Christophe Bornet]

Hi Pascal, I like the idea a lot.

Just want to warn on the choice of Java+mustache for the generator based on experience developing on OpenAPI-generator. I find that it is hard to handle logicless templates with a strongly typed language. Because I believe you need flexibility in your presentation layer where you'll add the presentation logic and POJOs are not great for that. In OpenAPI-generator we made the switch to handlebars to have a bit more functionality in the templates but even that is still too limited IMO.

Christophe

To view this discussion on the web visit https://groups.google.com/d/msgid/jhipster-dev/CALUG8VvDT21zxvG3WGF0R44uw4bnEn2w1wLfwE%3DGRF%2BzewtV%3DA%40mail.gmail.com.

[Vishal Mahajan]

Looks good to me. Do you plan to move towards jhipster specific spring boot starters to reduce generated code?

Cheers,

Vishal

To view this discussion on the web visit https://groups.google.com/d/msgid/jhipster-dev/CAE8RvoVRsaXEso7-qz9m4xRkUd4eY%3D8mmv9HtYt5wZnyCguF6Q%40mail.gmail.com.

[Marcelo Shima]

Since https://github.com/jhipster/generator-jhipster/issues/15490 was closed as won’t fix, I am wondering what is the plan for v8 if there is one.

Since we are splitting efforts, it may not happen.

My initial plan was:

- Implement https://github.com/jhipster/generator-jhipster/pull/17131 for better understanding the workflow and api documentation (jsdoc)

- Implement https://github.com/jhipster/generator-jhipster/issues/15490#issuecomment-907788727 so we don’t need a entity-foo for each foo generator.

- Implement https://github.com/jhipster/generator-jhipster/issues/15869, the generated blueprint would be esm and would have the specification of the first 2 steps.

- Start v8 cycle with breaking changes (splitting generators (modularization), reworking file tree, esm migration, properties reworking, and so one).

- JDL add support to modular and customizations.

Regards.

Marcelo.

To view this discussion on the web visit https://groups.google.com/d/msgid/jhipster-dev/CALUG8VvDT21zxvG3WGF0R44uw4bnEn2w1wLfwE%3DGRF%2BzewtV%3DA%40mail.gmail.com.

[Pascal GRIMAUD]

Hi Vishal,

People complain there are too many generated files in their JHipster project.

By using this approach, our user will add step by step only what they need.

So in the end (the final project with full options), it will be the same, only the approach is different.

I prefer keeping all generated code, instead of moving to some lib, as it's a pain to maintain:

- release react-jhipster

- sync react-jhipster with generator-jhipster, wait...

- release jhipster-registry, wait...

- release jhipster-bom, wait...

- sync bom with generator-jhipster, wait...

etc...

Then, I need to think about jhipster-online, jhipster-control-center, other blueprints/modules, etc.

In my opinion, it's better to have 1 mono repo.

But, it's just my opinion :)

Pascal

[Pascal GRIMAUD]

I closed this ticket because I was the initial author and wanted to lead this part, but I failed, sorry.

Last summer, I tried to code the sub generators Init + Maven, but I spent days, nights and weeks doing simple code... And finally, the result is not what I expected.

I asked dozens of developers (real JHipster users) why they don't contribute to JHipster. The answer is the same: it's too complex.

And today, I feel the same thing. Our generator is becoming too complex. I don't think I'm the only one in the core team.

Feel free to reopen the ticket and take the lead on this, Marcelo.

I don't know what is the plan for v8, I'm waiting for the result of the survey.

Pascal

[Deepu K Sasidharan]

Hi team, 

I think we should have a core team meeting after the survey is done to decide and finalize a plan for v8. While anyone interested in JHipster lite can work on that, we also need to move current generator forward. I like the ideas proposed by Marcelo for the current generator. Along with those I would also propose another drastic change 

- drop a lot of options with very little usage and large maintainace cost (Cassandra, couchbase, Aws related stuff, neo4j, and so on, ofcourse after looking at stats) 

- try and remove all logic from template files into js files and simplify and modularize templates. If possible split different options into different files

- consider dropping yeoman and using maybe Ocliff (if the effort is justifyable) 

- drop entity generator and use only JDL

- drop sub generators that are not widely used

- simplify composition without relying on Yeoman if feasible

To view this discussion on the web visit https://groups.google.com/d/msgid/jhipster-dev/CALUG8VttOGT3ngPwHMrWpashi4R%3DROEyF22oZkYwiZA%2BW27z7A%40mail.gmail.com.

[Julien Dubois]

Concerning entities:

- the JDL is clearly one of our top features 

- a lot of people use JHipster Online just for this

I would like to create this in a separate project (a bit like NubesGen.com, but under the JHipster umbrella). This would depend on JPA/Hibernate:

- No caching support (a higher level cache like Spring Cache works better, and solutions like Ehcache add a lot of limitations when scaling out)

- It would work for Spring and Spring Data, with some options for Quarkus and Micronaut which look quite similar

- It would work for "pure Spring" (no JHipster) projects!! Like NubesGen, we could have a tool generic enough, so non-JHipster users could also use it.

This would still be a Web app like we have today with JHipster Online, and something I’ve been willing to do for years is to add social/sharing capabilities. It would be awesome to be able to publish and share JDLs across projects.

To view this discussion on the web visit https://groups.google.com/d/msgid/jhipster-dev/CAFXry56ZAAZWLkLgqNWps4H3biHVbsFfGmvFVhYo9xj6_Gowrg%40mail.gmail.com.