New to BndTools, some questions
Dave
We've been using the PDE for developing our OSGi bundle projects, and
we're trying to figure out if we can move to BndTools instead. We've
also never used Bnd before, so things are a bit confusing.
In general, we're finding it hard to understand what our workflow
should be like with BndTools.
We're not doing Declarative Services/Component development, so the
tutorial doesn't help that much. (We happen to be using Spring DM /
Blueprint)
It's not clear to me what the (oddly-named) "cnf" project is- is it a
workspace-wide "target" environment? Should any and all bundle
dependencies for projects in my workspace go there? Is there a way to
create more than one of these, or change it's name? (it appears that
the name "cnf" is hard-coded into the build file, so probably not?).
Is the repository format it uses described anywhere?
Another big question we have after reading through all the
documentation is what is the deployment process? Is there a way to
build a deployable set of bundles along with a launch config and
deploy it as a zip file, or do we have to write our own scripts for
assembling this?
When I try to create a bdntools project for one of our existing bundle
projects, things seem to work, but the generated jar has none of the
built class files in it. I've selected all the packages for export,
and the compilation seems to work fine, but they're not there in the
final jar. I suspect this may be because we're using different source
and build folder names than are chosen by default- we're using "source/
main/java" and "source/test/java" instead of "src", and "build"
instead of "bin". These are easy to configure using eclipse's java
project build path, etc, but maybe bndtools expects the default names?
It's also not clear where to put things like customizing the Bundle-
Name, setting the Provider, etc- should we put these things in the
bnd.bnd file?
I realize that there are a lot of questions here, and I appreciate in
advance any answers.
Thanks,
Dave Fogel
Peter Kriens
your build.xml
copyright, licenses
vendor headers
repos for dependencies
plugins
etc, etc.
One of the golden rules in build is that you never repeat the same information and that all information is in the source control management system and NEVER in your IDE's preferences or home directory. Builds must be repeatable. So the cnf directory is an important cornerstone of bndtools.
That said, cnd is about the shared stuff which is NOT like the target environment. Bundles should imho not share a target environment because that forces you to update the build path for all projects whenever one changes. In bnd, the versions of the package you depend on are learned from the buildpath so this would automatically make all your projects depend on later versions for no reason whatsoever. Having each project its own buildpath creates a lot of stability and makes deployment so much easier because not every project always depends on the latest and greatest. See Bnd/Versioning
However, if you have certain dependencies you always want to have in each project (Junit for example), you can easily define macros in cnf that are used in your project. These macros must be defined in build.bnd but you can also put include files in cnf/ext that are automatically included.
You can only create one cnf directory because it represents the workspace and it must reside at the same level as the projects in the workspace. This is for simplicity, it allows everything to be adressed relatively so you do not start using absolute path names that are incredibly hard to maintain between different systems. It is not restrictive because you can easily use include files to include information from any place on your hard disk. You can include in the cnf/build.bnd files, the cnf/ext directory and in the project bnd.bnd files. If you really dislike the cnf directory you can actually replace it with a text file that contains the (relative) path to the shared directory. Leaving it out will also work I think but that is not recommended because it requires to enter the information in each project directory, which imho is very bad practice.
The deployment process depends very much on how you want to work. The bnd program can actually run the project or the bndrun files. In progress is currently a packager that creates a runnable JAR containing all your dependencies. The best part imho is the release bundle function. This will allow you to release the bundle to a repository; it will first diff the new bundle against the old bundle and verify versions of the packages and bundles. The context menu is on the bnd.bnd file.
About source and build paths, if you deviate from the defaults you can set the macros src and bin in the cnf/build.bnd directory:
src = source/main/java
bin = build
It would be nice if there was a GUI for this but for now the GUI is the text editor. However, I would suggest that you start using the defaults when you try a new tool. It is a lot easier to get things working with all defaults and then change to your settings from a working example than the other way around.
In bnd, all upper cased names in the cnf/build.bnd or bnd.bnd file are copied to the manifest. Macros can be used to remove redundancy. So where you put a header depends how you want to share. Imho good bundles specify Bundle-License, Bundle-Copyright, Bundle-DocURL, Bundle-Vendor in the cnf/build.bnd (they can always be overridden in bnd.bnd or a sub bundle descriptor.) Bundle-Name and other headers that require a specific project should be set in bnd.bnd file. Notice that bnd has some very reasonable defaults as its goal is to specify the absolute minimum. For example, the Bundle-SymbolicName is derived from the project name and if used, the sub bundle descriptor. Starting with defaults is usually a good way to start and then when something is really annoying find out how to change it. Though bndools is quite a new project it is based on bnd that is extensively used by some large builds so it usually can handle most needs. The biggest problem is the lack of documentation, it is running quite behind the functions that are available :-( The disadvantage of people not willing to pay for tools is that it is hard to invest in it ...
That said, don't fight it. bnd(tools) definitely has a bit of an attitude because we try to find a better way, which is not always the same as "mainstream". We try to make it easy to migrate to the model and offer quite a lot of help when features are needed in this model but compatibility with existing work flows is not that high on the list for me personally if it conflicts with the goal we're pursuing. Just to be honest :-)
Kind regards,
Peter Kriens
Dave
Thanks very much for taking the time to respond to my post!
A couple small things first:
Before posting here, I actually had already tried making the change
you describe to the "src" and "bin" properties of the cnf/build.bnd
file, during the hours I spent trying to get it to work. It didn't
work for me at the time, but today, based on a hunch, I restarted
eclipse after making the change, and now my bundle project seems to be
building correctly. So I think there's a bug in bndtools in which
changes to the .bnd file aren't taken into account- does it get re-
parsed for every build, or maybe it's cached somewhere? I was able to
reproduce this by using a fresh workspace- create my "first" OSGi
project, get prompted to create the cnf project, make a change to
build.bnd, etc.
But since a project's source folders and output folder are project-
scoped metadata in eclipse, couldn't the build.bnd settings for src
and bin be calculated for whatever project is running? Seems much
easier than having to manually maintain duplicate settings.
In general I'm fine with sticking with a new tool's default settings.
But in this case I chose a couple of our existing bundle projects to
try bndtools on, ones that had few dependencies, and they're in source
control with the source layout I mentioned. And anyhow, of all the
things that should be flexible in a build tool, probably source layout
should be high on the list.
If, as you say, the cnf project represents the workspace (and is 1-
to-1 with a workspace), does that mean that I should have many
different workspaces, one for each group of related plugins that I
work on? I guess that I can imagine doing this, but it does seem a
bit inconvenient, especially if that means that I end up with a bunch
of single-bundle-project workspaces. Maybe I'm missing something
here.
Regarding the bndtools workflow:
We're enthusiastic about adopting a better workflow, that's what's
motivating this round of looking for an alternative to the PDE. So my
questions are really aimed at understanding how you intend bnd and
bndtools to be used (and _not_ trying to force our existing workflow
on to them).
At my company, we have products/projects that have on the order of
50-100 bundles. For a particular project, these bundles fall into
several rough categories, and I'm interested in where these categories
fit into the bndtools workflow:
1) Perhaps 10 bundles specific to an application, and which share a
common package prefix such as com.example.product.*
2) Another 6 -10 tool and utility bundles developed in-house that are
of a cross-cutting nature, not specific to a particular product, and
not necessarily related to each other except perhaps a few are
interdependent.
3) Maybe a dozen or so 3rd-party bundle projects that we are building
ourselves in our workspace from source for various reasons: some
require source changes in order to be good OSGi citizens, or we wish
to track the "trunk" changes of a project instead of downloading
official artifacts.
4) the rest, maybe 30 or so bundles are more "standard" sets of 3rd-
party released bundles, such as Spring DM, various Apache libs, etc.
For a particular application we typically have an additional non-
bundle "project" that isn't an eclipse project, but is rather the
deployment shell for running the app, which contains a set of
directories with configuration (config.ini, etc), shell scripts, and a
directory into which we export the built plugins. This folder, when
deployed, is where we run the application.
So, what we're trying to understand about BND and BndTools is how to
best apply them, in eclipse and in our build scripts, given the above
categories of bundles we work with, and given that we wish to not only
improve our IDE experience but also improve our ability to write
build, deploy, and other general automation around all this bundle
development.
From what I understand so far, it seems like we should be putting the
4th category of pre-built bundles into the "repo" folder of a cfg
project. I'm less certain about the other categories of bundles- do
they go in separate workspaces? Do some get built in separate
workspaces and then exported to application-specific cnf/repo folders?
I guess what my comments/questions boil down to is this: I think it's
great if Bnd and BndTools have, as you put it, "attitude", but it
would help to have that attitude spelled out in details that go beyond
the "my first bundle" sort of tutorial, to describe more fully what
your vision is for a full workflow for real-world multi-bundle
application development. And if there are parts of that workflow that
aren't covered by the features in Bnd or BndTools, perhaps suggest
additional tools/approaches that are compatible with bndtools. For
instance, are you assuming that most people would use bndtools in
conjunction with other tools like Ant + Ivy? or Maven? or neither?
(Incidentally, we'd prefer to steer clear of Maven, and prefer to
avoid more than the minimum of Ant. We're currently investigating
Gradle as a build and automation tool, but it's early in the process
yet.)
Thanks, and sorry for the lengthy message :-)
Dave Fogel
Peter Kriens
- The problem is that bnd is focused on the offline build so ALL information must be stored in the bnd files. Parameterizing this
from Eclipse would make the ant scripts considerably harder, especially because there is no way to parse this information
outside eclipse in a consistent way due the use of variables that are inaccessible outside eclipse.
- About the workspaces. It all depends how you want to work. In OSGi we have 130 or so projects in a single workspace. However, there is no problem to have a cnf directory per project. From the cnf directory you can easily include files from other places. Please note that though you share things in the workspace, it is not like the target. Each project defines its own dependencies. So the affinity between bundles does not have to be overly high. But this is a very personal choice. I know one person that is having a cnf directory per project.
- Notice that you can create as many repos as you want. In the OSGi we have a Licensed repo for external bundles. You can
also make a special repo for published artifacts that is shared between workspaces.
- Also notice that bndtools can have as many bundles per project as you like. You seem to wrap existing bundles, a good way imho is to do all your wrapping in a single project.
About the attitude. This is a path or journey that can be influenced by the people that use the tools. I think many existing tools have taken architectural short cuts to make the implementation simpler. In bnd(tools) we try to do it right even though that is harder. In this journey, we're completely open to different ideas. If there are really good practices out there that we need to adopt then we will adapt as long as it is right and does not violate the core concepts.
I agree that it is necessary to have more documentation but unfortunately there is no funding to this project. So it has to be done at odd hours and as part of other projects. This unfortunately means documentation takes the back burner. I hope we can scale this up in the near future but in a world where developers refuse to pay for tools it will be hard to do it the way I would like to do this :-(
Thanks for outlying your experiences, this is very valuable to me.
Kind regards,
Peter Kriens