Proposal - Improve incremental bake performance

[Mike McGarr]

I would like to propose a change to JBake that allows a much more incremental approach to baking changes, both with the existence of a cache, as well in watch mode. For instance, my local bakes for my personal blog/website take 39.6 seconds without a cache, 28.8 with a cache, and in watch mode, it takes 23 seconds to rebake, even if I changed a single css file. I see huge opportunities to improve this performance.

My proposal is to make two changes to how JBake builds: improve bake performance with a cache and single file bakes.

When looking at my bake performance numbers above, a third of that time is spent copying all of the assets to the output directory, everytime. I would like to identify a method of only copying assets that have changed. My initial look through the code shows me that that only changed content is re-rendered if a cached version exists, so I would like to apply this same approach to assets. Another third of the bake time is spent re-rendering content (like tags) that hasn't changed and doesn't need to be re-rendered. I would like to tag, archive, sitemap and other site-wide content to be generated only as needed. This change feels a bit more invasive, so I wanted to get thoughts before digging deep. This change will improve JBake's performance when running the bake command from a warm cache. 

For the second change would be to enable the baking of a single file at a time. This change would be to improve the performance of watch mode. I went ahead and started making changes in this directory on a fork of JBake and also see huge performance improvements. Changing a single css file when running jbake in server mode goes from 23 second wait times to a bake time of 33 milliseconds! This is huge. Assets are the easiest to address since they only involve a copy. I will be looking into the content changes next, as this will drive me into more invasive changes. For template changes, we could also be smarter about only rebaking content that uses that template, but I will likely re-bake everything in my first sweep of this.

For both of these changes, I would like to implement them in a manner that would also enable the Gradle plugin to take advantage of the performance improvements as well. 

I recommend this be addressed in https://github.com/jbake-org/jbake/issues/391, since this is an active and open request.

[max.an...@gmail.com]

just want to say +1000 for me - i've moved from awestruct to hugo to jbake and jbake's incremental performance is definitely subpar compared to the others here.

do you have a link to your fork to try it out ?

[Jonathan Bullock]

Hi Mike,

I've commented on the issue referenced, but wanted to comment here too.

First off, thanks for taking the time to raise this topic and for for the initial investigation into it.

As Frank mentioned on the issue, if we could break these performance improvements up into backwards compatible changes then I'm happy to get them into a release as soon as possible. And I agree that these improvements should be made in a way so that the build tool plugins can take advantage of.

If you need anything else from me please let me know.

Thanks,

Jon

On Fri, 9 Nov 2018 at 07:24, <max.an...@gmail.com> wrote:

just want to say +1000 for me - i've moved from awestruct to hugo to jbake and jbake's incremental performance is definitely subpar compared to the others here.

do you have a link to your fork to try it out ?

--
You received this message because you are subscribed to the Google Groups "JBake Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jbake-user+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.