One workspace, independent projects?

[Fr Jeremy Krieg (Home)]

Hi all,

I pray that you are all well.

I'm trying to solve a source management issue for my Bndtools projects, which are plugins for iDempiere.

I have a series of N projects that are all using the Bnd workspace model. They all use the same cnf/repositories, and they all have more-or-less the same project "idempiere-server" which starts up the server for manual testing. For this reason, it is convenient to have them all in the same workspace.

But apart from this, they don't have a lot of direct dependency on each other, and it feels wrong for them all to belong to the same git repository as they should be able to be built, tested, run and deployed independently of each other. On the other hand, it also seems wrong to have N separate git repositories with substantially the same cnf and idempiere-server project, because trying to maintain and deploy the tweaks across all of the workspaces will become unmaintainable.

One thought I had was to create a git uber-repo for the dev environment with cnf and idempiere-server projects in it, then create git submodules for all of the individual plugin projects. I've not used submodules before and I've heard mixed reviews... 

Does anyone have any similar experience or thoughts on how best to structure this?

Blessings,

Fr Jeremy Krieg

[Peter Kriens]

All these projects share a lot of dependencies since they are all _your_ iDempiere plugins. I would put them together. I find that with such a shared goal, over time I tend to share more and more projects for testing, utilities, and other reasons.  although each plugin likely has its own goal, I tend to see that this own goal is usually some external library. Very large projects (IBM has a thousand) for Bndtools are no problem as long as you do not have a wide dependency graph. I.e. API based projects. And it is fantastic to do system wide refactoring. 

However, if you are like most developers then it will feel more neat to separate things! :-) This aggregation topic has been a continuous struggle between BJ and me for 20 years :-) Finding the balance between aggregation and separation is very personal.

I would not go into the git model. One thing you can try out is to have a bnd file on GitHub with your setup. The bnd `-include` instruction can take URLs. 

Make sure you use Git tags in these URLs otherwise you likely break your build when in the future you checkout an old build. You really do not want to automatically pick the 'latest'. However attractive this is in the short time. This still means that if you do update, you need to make a change in all your workspaces. However, this is similar to Maven with the parent poms. (

  cnf/build.bnd:

    -include \

      https://github.com/bndtools/bnd/blob/5.1.2.REL/cnf/build.bnd

Currently the includes are not cached so it will make your build dependent on connectivity. It would not work in a plane, but hey, who is flying nowadays! :-) Adding a cache would not be too hard I think.

Kind regards,

Peter Kriens

--
You received this message because you are subscribed to the Google Groups "bndtools-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bndtools-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bndtools-users/CAO6F8YyuSQM0szJdCVusuQ%2BCQ0yPGJruKsM7d09iOcJG3dD3sg%40mail.gmail.com.

[Fr Jeremy Krieg (Home)]

Thanks Peter,

Actually,  there are two levels of abstraction sharing:

The highest level is not because they are specifically my iDempiere plugins, but simply because they are iDempiere plugins. I am trying to abstract this out so that other iDempiere developers can use it for developing their own plugins.

I'm also thinking for the future about how to use the dev model in a way that facilitates open source,  allowing others to use and possibly contribute to only one of the projects, without necessarily having to check out and build the 999 other projects that they're not interested in. 

Interested if this alters your thoughts?

Blessings, 

Fr Jeremy 

To view this discussion on the web visit https://groups.google.com/d/msgid/bndtools-users/933348D6-37A4-44CD-8AE1-746E3790C653%40gmail.com.

[Peter Kriens]

Interested if this alters your thoughts?

Nope :-)

I had the same problem eons ago with OpenHAB. I put their code base and plugins on Bndtools but they decided on the maven model, partly because of your argument to decompose. I think the bndtools workspace worked much better than what they have today.

Although I can see this minimizes the shock for a newcomer it is not as if these other plugins go away. With the working sets (you can define your own working set in the bnd.bnd file) you can easily create a view on the Bndtools explorer that limits the projects in view. And in a good API driven plugin model, you normally only compile your own code. So I think the overhead is minimal.

However, event though it is a plugin model, you are actually really collaborating with other people. Using the same workspace makes you find bugs very early in other components and see unexpected interactions that you'd only find out much later in runtime. The issue is that this (imho) small overhead for developers is hated with a vengeance by them. However, also imho, it is making the macro level **significantly** more efficient. The tragedy of the commons.

One caveat. I am thinking plugins with generic utility. If the code base is very much driven by a particular business this might not apply. However, in that case you've got security concerns anyway.

But I am used to losing this argument :-)

Kind regards,

Peter Kriens

To view this discussion on the web visit https://groups.google.com/d/msgid/bndtools-users/CAO6F8Ywgz_VBC75t6Z94Yg0zduYXNsGAaU8fGOSCNzKcvxM9dQ%40mail.gmail.com.

[Fr Jeremy Krieg (Home)]

Thank you Peter.

I think that you've hit the point of concern:

> One caveat. I am thinking plugins with generic utility. If the code base is very much driven by a particular

> business this might not apply. However, in that case you've got security concerns anyway.

I think the all-in-one approach could work, as you say, for plugins with generic utility. I see many advantages - eg, if I'm writing a test for my plugin that uses idempiere-test, and I find out there is a feature in idempiere-test missing that I need, I can quickly add it if idempiere-test is part of the same workspace. If I'm relying upon it as an external dependency, then I have to switch Eclipse workspaces, add the feature, deploy, switch back to my other Eclipse workspace, then try and use it, then find out that I didn't quite get the feature right for what I needed, wash, rinse, repeat. As I understand it, it is precisely this kind of "wash cycle" that the all-in-one-workspace approach really helps with, and I completely agree. At the moment I'm trying to work with idempiere-test as an external dependency, and I've ended up with snippets in the individual test plugins that are intended (in that mythical future time known as "one day" when everything is supposed to happen...) to become part of idempiere-test, but for short-term convenience have been put where they are. Worse, there is then a temptation to copy them from one test plugin to another when I quickly need them in the other test... then I end up with divergence, and a lack of regression tests for the newly-added features, etc.

However, for more specific in-house plugins, I'm not so sure how useful this will be. I have a mix of plugins - some will be of generic utility that I might light to share via open-source, and others are in-house plugins that I wouldn't like to share. In fact, there is probably a spectrum from generic->specific. However, it is likely that in all cases their workspace setups will have a significant overlap. It's how to best manage that overlap that I'm wrestling with.

 

Thanks for your insights, Peter - much appreciated.

Blessings,

Fr Jeremy

To view this discussion on the web visit https://groups.google.com/d/msgid/bndtools-users/235AB535-68C0-499B-8530-B28729E3E4FF%40aqute.biz.