integrating bndtools gradle with existing gradle multi-project build

[Bill Phillips]

All,

We have an existing gradle script, which is a multi-project build containing number of libraries, two wars, and many OSGI bundles (uses gradle version 1.12 and bndtools version 2.2.2).

When this script was development bndtools was not integrated with gradle at the time, so the portion of the script which builds the bundles was custom script based on an open source project.

I was delighted to find that the recent versions of bndtools contain gradle scripts by default. Thanks to all those who made this happen.

My current task is update the multi-project build to use gradle 2.3, bndtools 2.4.1, and (hopefully) to use the bndtools default gradle scripts.

This has not been straight forward because a gradle multi-project build only supports one settings.gradle file, which we already have.

Bndtools provides a settings.gradle, but the contents of this file cannot be copied into our multi-project build without modification.

The modifications are required because the bndtools settings.gradle assumes:

• that the root directory of the build is where all of the bndtools projects live (not true in our case)
  
• that everything in the root directory is a bndtools project  (not true in our case)
  
• that subprojects to non-bndtools projects are bndtools subprojects (not true in our case)
  

I am also confused about the role of the defaultProjectName in the settings.gradle:

1. It is set to to bnd_build property (blank by default)
2. It is then set to be the root-most directory which is not the gradle root directory
3. If the start parameter task name is not splitable with ":", the default project is added to the dependency graph (via "include")
4. If the current dir is a subdir and project name is empty, the default project is added to the dependency graph (via "include")

This has the effect of changing the dependency graph depending on which directory you compile from.

I am no gradle expert, but I was under the impression that the best practice in the settings.gradle was to include all gradle project and subprojects unconditionally. This allows gradle to calculate the necessary dependency graph.

Thanks,

Bill

[BJ Hargrave]

On Apr 17, 2015, at 16:02 , Bill Phillips <william.r....@gmail.com> wrote:

All,

We have an existing gradle script, which is a multi-project build containing number of libraries, two wars, and many OSGI bundles (uses gradle version 1.12 and bndtools version 2.2.2).

When this script was development bndtools was not integrated with gradle at the time, so the portion of the script which builds the bundles was custom script based on an open source project.

I was delighted to find that the recent versions of bndtools contain gradle scripts by default. Thanks to all those who made this happen.

:-)

My current task is update the multi-project build to use gradle 2.3, bndtools 2.4.1, and (hopefully) to use the bndtools default gradle scripts.

This has not been straight forward because a gradle multi-project build only supports one settings.gradle file, which we already have.

Bndtools provides a settings.gradle, but the contents of this file cannot be copied into our multi-project build without modification.

The modifications are required because the bndtools settings.gradle assumes:

• that the root directory of the build is where all of the bndtools projects live (not true in our case)
  

The default bnd gradle supports assumes the bnd workspace model. That is, all projects are peers in the “root” folder and that there is a cnf folder for workspace wide things including build.bnd, the central bnd instructions.

• that everything in the root directory is a bndtools project  (not true in our case)
  

The scripts don’t require this. See https://github.com/bndtools/bnd/blob/master/build.gradle#L28, for example, which only applies the bnd gradle plugin of the project is a bnd project (has a bnd.bnd file).

• that subprojects to non-bndtools projects are bndtools subprojects (not true in our case)
  

The build scripts don’t really care or support about subproject since they assume the bnd workspace model.

You may also want to look at the new bnd gradle plugin, see https://github.com/bndtools/bnd/wiki/Gradle-Plugin-for-non-Workspace-builds, for user’s who do not use the bnd workspace model. This may be more suitable for you. 

I am also confused about the role of the defaultProjectName in the settings.gradle:

1. It is set to to bnd_build property (blank by default)

The purpose of this property is for when one of the peer projects is a “master” project which starts the dependency chain for building the projects in the proper order. See https://github.com/bndtools/bnd/blob/master/gradle.properties#L14 and https://github.com/bndtools/bnd/blob/master/dist/bnd.bnd#L4. For the bnd repo, the dist project is the master project which packages the output of the other projects. If this bnd_build property is set, then the gradle scripts will use that project for the default target. 

2. It is then set to be the root-most directory which is not the gradle root directory
3. If the start parameter task name is not splitable with ":", the default project is added to the dependency graph (via "include")
4. If the current dir is a subdir and project name is empty, the default project is added to the dependency graph (via "include")

This has the effect of changing the dependency graph depending on which directory you compile from.

Yes, but the above is all based upon the bnd workspace model. Since your build does not appear to be structured that way, you probably cannot use the script as is.

I am no gradle expert, but I was under the impression that the best practice in the settings.gradle was to include all gradle project and subprojects unconditionally.

By default nothing is included. Most projects have to code the necessary includes to get the projects added to the dependency graph.

This allows gradle to calculate the necessary dependency graph.

Thanks,

Bill

--
You received this message because you are subscribed to the Google Groups "bndtools-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bndtools-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

-- 

BJ

[jan.winte...@gmail.com]

gradle.build (line 33,34): 

Where is the function/methode 'bnd( bnd-id-string )' declared?

line 33:      group bnd('groupid')

line 34:      version bnd('groupid')

And 

Which statement make this call possible?

-     'line 32:  plugins.apply 'biz.aQuote.bnd' ' ?

-     If Yes: Where is the corresponding class?

[BJ Hargrave]

On Apr 20, 2015, at 05:15 , jan.winte...@googlemail.com <jan.winte...@gmail.com> wrote:

gradle.build (line 33,34): 

Where is the function/methode 'bnd( bnd-id-string )' declared?

That is part of the gradle plugin. See https://github.com/bndtools/bnd/blob/master/biz.aQute.bnd.gradle/src/aQute/bnd/gradle/BndPluginConvention.groovy#L18.

line 33:      group bnd('groupid')

line 34:      version bnd('groupid')

And 

Which statement make this call possible?

-     'line 32:  plugins.apply 'biz.aQuote.bnd' ‘ ?

‘biz.aQute.bnd’ is the name of the bnd gradle plugin. See https://github.com/bndtools/bnd/blob/master/biz.aQute.bnd.gradle/resources/META-INF/gradle-plugins/biz.aQute.bnd.properties. This properties file names the plugin and also identifies the implementation class for the plugin.

-     If Yes: Where is the corresponding class?

-- 

BJ

[jan.winte...@gmail.com]

Thanks BJ.

I found both:) 

One question regarding 'BndPluginConvention.groovy'

     11 import org.gradle.api.Project

     13 class BndPluginConvention {

     14   private final Project project

     ...

     18   String bnd(String name) {

     19     return project.bnd.get(name)

     20   }

Line 19: is hard to understand: 

-    Why has 'org.gradle.api.Project project' a methode or field 'bnd' ?

[BJ Hargrave]

On Apr 20, 2015, at 09:00 , jan.winte...@googlemail.com <jan.winte...@gmail.com> wrote:

Thanks BJ.

I found both:) 

One question regarding 'BndPluginConvention.groovy'

     11 import org.gradle.api.Project

     13 class BndPluginConvention {

     14   private final Project project

     ...

     18   String bnd(String name) {

     19     return project.bnd.get(name)

     20   }

Line 19: is hard to understand: 

-    Why has 'org.gradle.api.Project project' a methode or field 'bnd' ?

bnd is an extension assigned to the Gradle Project object. See https://github.com/bndtools/bnd/blob/master/biz.aQute.bnd.gradle/src/aQute/bnd/gradle/BndPlugin.groovy#L70.

Is there some deeper issue you are working on here? Or are you just trying to understand how the bnd gradle plugin works? This is pretty normal gradle plugin stuff. You may want to read up on creating custom gradle plugins to learn more.

-- 

BJ

[jan.winte...@gmail.com]

I try to understand the bnd-graddle build to move some small projects from me to bnd-graddle.

-   mostly single-projects (rootDir == projectDir)

But I dont want to share graddle-cofiguration in so many files like the bnd-build do that.

-   settings.gradle, build.gradle (for each project + root), cnf/eclipse/jrt.properties.

Exist maybe a minimized bnd-gradle example anywhere? 

-   additional to 'https://github.com/bndtools/bnd/wiki/Gradle-Plugin-for-non-Workspace-builds' 

-   because this was not working proper for me

[BJ Hargrave]

On Apr 20, 2015, at 09:47 , jan.winte...@googlemail.com <jan.winte...@gmail.com> wrote:

I try to understand the bnd-graddle build to move some small projects from me to bnd-graddle.

-   mostly single-projects (rootDir == projectDir)

For single projects, (not using the bnd workspace model), then you do want to use the gradle plugin for non-workspace builds: biz.aQute.bnd.builder. See https://github.com/rotty3000/blade/tree/master/gradle for an example of using the biz.aQute.bnd.builder gradle plugin.

But I dont want to share graddle-cofiguration in so many files like the bnd-build do that.

-   settings.gradle, build.gradle (for each project + root), cnf/eclipse/jrt.properties.

The biz.aQute.bnd gradle plugin is for bnd workspace builds with shared information in the cnf project.

Exist maybe a minimized bnd-gradle example anywhere? 

-   additional to 'https://github.com/bndtools/bnd/wiki/Gradle-Plugin-for-non-Workspace-builds' 

-   because this was not working proper for me

You should probably focus here then rather than trying to change the bnd workspace model build for use in a non-bnd workspace model build. See the example I mentioned above as a guide.

-- 

BJ

[Bill Phillips]

Thanks for your reply and taking the time to help me out.

The default bnd gradle supports assumes the bnd workspace model. That is, all projects are peers in the “root” folder and that there is a cnf folder for workspace wide things including build.bnd, the central bnd instructions.

Given your description of the workspace model (cnf and all bundle projects are peer directories), I believe that our build script mostly uses this model, with the exception being that the workspace isn't in the root of the gradle build.

Gradle root
|-- settings.gradle
|-- gradle.properties
|-- build.gradle
|
|-- bundles
|             |-- build.gradle
|             |-- cnf
|             |-- bundle1
|             |-- bundle2
|             |-- bundle3
|
|--- lib1
|--- lib2
|--- lib3
|--- war1
|--- war2

When I move the contents of the bnd settings.gradle into the root dir settings.gradle, it attempts to load in lib1, lib2, etc. as bnd workspace projects.

I can work around this, of course, but this was the source of the observations I made in my initial questions.

Given that this is our structure, do you still recommend using the new bnd gradle plugin?

 

See https://github.com/bndtools/bnd/blob/master/build.gradle#L28, 

I am using the 2.4.1 version of the plugin (which I understand is the latest). However, the gradle files generated by the plugin do not match what is in the github.

Is it safe to use what is in the github with plugin version 2.4.1?

Thanks again,

Bill

[Ferry Huberts]

You can also try my  Gradle plugin, it's much more flexible and will include your lib projects automatically if they have an appropriate gradle build file.
For your bundle projects, there is a build.gradle file so that will get included automatically as well.

The problem, however, is that you have cnf one level too deep.

https://marketplace.eclipse.org/content/bndtools-plugins-pelagic-24xrel
https://marketplace.eclipse.org/content/bndtools-plugins-pelagic

--
You received this message because you are subscribed to the Google Groups "bndtools-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bndtools-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

-- 
Ferry Huberts

[BJ Hargrave]

On Apr 21, 2015, at 11:24 , Bill Phillips <william.r....@gmail.com> wrote:

Thanks for your reply and taking the time to help me out.

The default bnd gradle supports assumes the bnd workspace model. That is, all projects are peers in the “root” folder and that there is a cnf folder for workspace wide things including build.bnd, the central bnd instructions.

Given your description of the workspace model (cnf and all bundle projects are peer directories), I believe that our build script mostly uses this model, with the exception being that the workspace isn't in the root of the gradle build.

Gradle root
|-- settings.gradle
|-- gradle.properties
|-- build.gradle
|
|-- bundles
|             |-- build.gradle
|             |-- cnf
|             |-- bundle1
|             |-- bundle2
|             |-- bundle3
|
|--- lib1
|--- lib2
|--- lib3
|--- war1
|--- war2

When I move the contents of the bnd settings.gradle into the root dir settings.gradle, it attempts to load in lib1, lib2, etc. as bnd workspace projects.

I can work around this, of course, but this was the source of the observations I made in my initial questions.

You could move cnf and the bundles into the root. Then your settings.gradle will need to make sure to include all the projects including the lib and war projects. You may also need to make sure that the lib and war project are build in the proper order relative to the bundle projects given whatever dependencies between them you may have. The settings.gradle file used in bnd/bndtools builds sets all the project interdepedencies based upon the information from the bnd.bnd files in each project. But for non-bnd projects, you will need to establish the ordering.

Given that this is our structure, do you still recommend using the new bnd gradle plugin?

Assuming you can move cnf and the bundle projects to the root, you should be able to use the ‘biz.aQute.bnd’ plugin.

 

See https://github.com/bndtools/bnd/blob/master/build.gradle#L28, 

I am using the 2.4.1 version of the plugin (which I understand is the latest). However, the gradle files generated by the plugin do not match what is in the github.

Is it safe to use what is in the github with plugin version 2.4.1?

Yes. That should be no problem. 

Thanks again,

Bill

--
You received this message because you are subscribed to the Google Groups "bndtools-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bndtools-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

-- 

BJ

[Bill Phillips]

Assuming you can move cnf and the bundle projects to the root, you should be able to use the ‘biz.aQute.bnd’ plugin.

Moving the cnf and bundle projects into the root isn't a feasible option (many reasons for this).

Is the fact that the cnf must been in the root directory a limitation of the script implementation, or is it related to the bndtools implementation?

I was hoping that by using relative paths in the various scripts that I could work around this.

Bill

[BJ Hargrave]

On Apr 21, 2015, at 12:59 , Bill Phillips <william.r....@gmail.com> wrote:

Assuming you can move cnf and the bundle projects to the root, you should be able to use the ‘biz.aQute.bnd’ plugin.

Moving the cnf and bundle projects into the root isn't a feasible option (many reasons for this).

Is the fact that the cnf must been in the root directory a limitation of the script implementation, or is it related to the bndtools implementation?

It is an assumption about the bnd workspace model. Now you could treat the whole folder with cnf and the bundles as a “sub-build” and built it that way since it seems to follow the bnd workspace model. It would need its own settings.gradle and build.gradle I would think. But I have not tried this myself, so good luck :-)

I was hoping that by using relative paths in the various scripts that I could work around this.

This won’t work since bnd is “in-charge” not the build scripts which use bnd to setup the gradle build.

[Bill Phillips]

I was hoping that by using relative paths in the various scripts that I could work around this.

This won’t work since bnd is “in-charge” not the build scripts which use bnd to setup the gradle build

So with a few tweaks to the settings gradle and some relative path kung fu, I was able to get it working until to the point where bundles depend on each other.

At this point, I get the following exception: 

A problem occurred evaluating project ':bundles'.
> Failed to apply plugin [id 'biz.aQute.bnd']
> Project with path ':projectA' could not be found in project ':bundles:projectB'.

I would need the plugin to recognize projectA as ":bundles:projectA" instead of just ":projectA"

Bummer.

I can get the bundles building pretty easily if I split that build into its own script, but the problem there is that I have a war project in the main script which depends on some of the apis projects in the bundles build.

The only way to get that dependency to resolve is to put those bundle projects in the root directory setttings.gradle and I am back to square one.

Moving the cnf and all the bundle projects into the root isn't going to fly because we have far too many bundles project to do this.

The only options that I see are:

1. Have the bundles build dump the API projects into a directory and have the war depend on all jars in that directory.

2. Have the bunldes build pushlish the apis to a nexus and depend on it like any other third party dependency.

Neither one of these are particularly appetizing.

Bill