Incremental building not working?

[Fr Jeremy Krieg (Home)]

Hi team,

I pray that you're all well.

Quick question: Is anyone else having issues with incremental building not working?

It seems to me that Bndtools is rebuilding too frequently, and from the progress meter it seems it is rebuilding certain libraries several times (eg, biz.aQute.bndlib). It also embarks on massive rebuilds even with seemingly innocuous changes - for example, just now all I did was add two dependencies to the -testpath of my new tester bundle project (ie, a project which shouldn't have any downstream dependencies in the workspace) and it seemed to rebuild pretty much the entire workspace (including bndlib several times and all of the bndtools bundles).

Anyone else having this issue? If not, any tips on debugging?

Blessings,

Fr Jeremy

[Peter Kriens]

You can put on full logging in Preferences/Bndtools. The log go to the 'error' view.

Kind regards,

Peter Kriens

--
You received this message because you are subscribed to the Google Groups "bndtools-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bndtools-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bndtools-dev/CAO6F8Yx1PG8owJ5dXSteb61NY0RKaD6_evxv0T1mL25X4SUe0g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Fr Jeremy Krieg (Home)]

Another data point: I just ran the test through Gradle from the command line after just completing the same test in Eclipse, and it looks like Gradle decided to rebuild the dependencies too (although it at least looks like it was more sensible about restricting the scope of the rebuild to the actual dependencies).

Blessings,

Fr Jeremy

[BJ Hargrave]

Sounds like the dependencies are real. Both Bndtools and Gradle see
the dependency and rebuild when there is a change.

In gradle, you can use the `echo` task of each project to see the
dependencies as Bnd sees them. You can also use Bndtools preferences
to set Full Build logging on and then see information in the Error Log
view on why Bndtools builder is running.

On Wed, Jun 26, 2019 at 5:21 AM Fr Jeremy Krieg (Home)

<jezzy...@gmail.com> wrote:
>
> Another data point: I just ran the test through Gradle from the command line after just completing the same test in Eclipse, and it looks like Gradle decided to rebuild the dependencies too (although it at least looks like it was more sensible about restricting the scope of the rebuild to the actual dependencies).
>
> Blessings,
> Fr Jeremy
>
> On Wed, Jun 26, 2019 at 6:48 PM Fr Jeremy Krieg (Home) <jezzy...@gmail.com> wrote:
>>
>> Hi team,
>>
>> I pray that you're all well.
>>
>> Quick question: Is anyone else having issues with incremental building not working?
>>
>> It seems to me that Bndtools is rebuilding too frequently, and from the progress meter it seems it is rebuilding certain libraries several times (eg, biz.aQute.bndlib). It also embarks on massive rebuilds even with seemingly innocuous changes - for example, just now all I did was add two dependencies to the -testpath of my new tester bundle project (ie, a project which shouldn't have any downstream dependencies in the workspace) and it seemed to rebuild pretty much the entire workspace (including bndlib several times and all of the bndtools bundles).
>>
>> Anyone else having this issue? If not, any tips on debugging?
>>
>> Blessings,
>> Fr Jeremy
>

> --
> You received this message because you are subscribed to the Google Groups "bndtools-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to bndtools-dev...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/bndtools-dev/CAO6F8YzMV%3DJfB3acF2ssuLszbZn%3DAjhVAGR%3DmnzNPHEZ2TyHRw%40mail.gmail.com.

> For more options, visit https://groups.google.com/d/optout.

--

BJ

[Fr Jeremy Krieg (Home)]

But why would Gradle need to rebuild immediately after Eclipse had just done it? It feels like Gradle is doing the right thing here but Bndtools is messing up. I'll try and use the logging/echo when I next get a chance and see what's going on. I suspect it's a file locking issue again at its core - it can't get the dependency because it's locked and so it tried to rebuild it instead. When the rebuild finishes, it bumps the timestamp on the artifact which causes the dependent projects to rebuild again.

Blessings,

Fr Jeremy

To view this discussion on the web visit https://groups.google.com/d/msgid/bndtools-dev/CABr4vaxT-e%2BCJNO8Q2PbHC5eML180iLmJGX2u39D%2BttkqTjJSQ%40mail.gmail.com.

[BJ Hargrave]

Gradle maintains its own out-of-date information as does Eclipse.
Also, the gradle build compiles with Javac while Eclipse uses ECJ. So
I do not think it is a file locking issue causing rebuilds.

So when you are building with gradle, you MUST NOT have Build
Automatically on in Eclipse. Otherwise they will fight each other to
build the same output files and chaos reigns.

On Wed, Jun 26, 2019 at 7:54 AM Fr Jeremy Krieg (Home)

> To view this discussion on the web visit https://groups.google.com/d/msgid/bndtools-dev/CAO6F8YyxdD9MA6HDwNDkT%3DWa9bcc0_W00NW1-peRXpA9j41FWQ%40mail.gmail.com.

[Fr Jeremy Krieg (Home)]

On Wed, 26 Jun 2019 9:27 pm BJ Hargrave <b...@bjhargrave.com wrote:

Gradle maintains its own out-of-date information as does Eclipse.

Ok, that's one mystery solved. 

Also, the gradle build compiles with Javac while Eclipse uses ECJ. So
I do not think it is a file locking issue causing rebuilds.

Sorry, I mightn't have explained that properly. I don't suspect a locking battle between Gradle and Eclipse, but within different parts of the Eclipse build process. As I noted, Gradle seems to be doing the right thing.

So when you are building with gradle, you MUST NOT have Build
Automatically on in Eclipse. Otherwise they will fight each other to
build the same output files and chaos reigns.

Makes sense, thanks! I don't think that's what's happening here though, I haven't had them both going at the same time.

Blessings,

Fr Jeremy

To view this discussion on the web visit https://groups.google.com/d/msgid/bndtools-dev/CABr4vazqNe0Weaadpmmat9ehH7D1h%2BBA2L0p4979%2BLhbwKVuoA%40mail.gmail.com.

[Fr Jeremy Krieg (Home)]

Ok, I've managed to capture some logs of this happening.

The attached logs are the build that happened after I had had automatic building off for a while (while I was working on some Gradle stuff). As you can see, bndlib was rebuilt a total of around 15 times during the whole thing. Again, it feels like there was a log more rebuilding going on than there should have been.

Unfortunately, I didn't think to do any "open files" dumps while it was running. If this would be useful, let me know and I'll try and do it next time this happens.

Any info/feedback/tips for more data collection would be greatly appreciated. As you can see, that entire rebuild session took nearly 10 minutes, during which time I couldn't run the test that I wanted to run.

Blessings,

Fr Jeremy

[BJ Hargrave]

Looking at the logs, it appears to be rebuilding bndlib because ECJ
kept changing the bin folder:

!ENTRY bndtools.builder 1 0 2019-06-30 00:51:00.722
!MESSAGE BUILD AUTO biz.aQute.bndlib 1 file was built
F/biz.aQute.bndlib/bin changed
project had changed files
Duration 21.33 sec

Why it takes 21 seconds (!), I can only blame windows. All the other
bndlib builds only took 8-9 seconds.

On my systems (mac), I did a full clean build and bndlib only took < 3
secs (whole clean build of workspace was < 2 min and only built bndlib
once).

BUILD FULL biz.aQute.bndlib 1 file was built
project was dirty from changed bnd files postponed = false
Calculated dependsOn list: [P/cnf, P/aQute.libg, P/biz.aQute.bnd.annotation]
Duration 2.71 sec

I then went and rebuilt via gradle, came back to Eclipse, refreshed
all projects and then turned on Build Automatically. It rebuilt the
workspace in < 2 min. bndlib was built only once.

BUILD AUTO biz.aQute.bndlib 1 file was built
F/biz.aQute.bndlib/bin changed
project had changed files
Duration 2.84 sec

I also see you are building your branch (since it has projects not in
master such as biz.aQute.tester.test). It is possible your new
projects have added some issue to the project dependency graph? Does
this same building problem also occur on master branch?

Do you have some windows antivirus scanning your git working
directory? That can really kill build performance on Windows which
already has a slowish file system. Is the file system rotating media
or SSD?

I have seen sometimes where the Eclipse Incremental builder can take
several iterations to settle down and complete building the Eclipse
workspace. But looking at the logs you provided, bndlib is being built
because Eclipse reported that the bin folder changed and based upon
that information, the Bndtools builder will rebuild the bundle.

On Sat, Jun 29, 2019 at 11:51 AM Fr Jeremy Krieg (Home)

> To view this discussion on the web visit https://groups.google.com/d/msgid/bndtools-dev/CAO6F8YyXHK5dto%2BOnrTtxUzKnoTpwYRGkAF-jNnFyfj_0%2B2_eg%40mail.gmail.com.