build scripts

[denstar]

Howdy folks,

I've made some pretty major modifications to the build scripts. The
biggest is the move to using dependencies for the libs instead of
bundling them in the source, and doing the same for the server and JRE
bundles.

The second biggest is creating the lucee-context.lar file on the fly,
instead of manually building it and adding it back to source control.

With those changes, the repo goes from 750+ megs to 12 megs, and
updating the dependencies is a matter of changing version numbers
(including tomcat and whatnot).

The lucee-context.lar "cycle" was adding about 2mb per change. Since it
changes semi-infrequently, it would probably be better to give it it's
own version number, and build it separately, but at least for now the
whole shebang can be built from scratch, and we're not repeatedly
putting binary resources under version control. (FWIW, bitbucket
recommends <100mb repos.)

There's the other nice stuff I've talked about too, like checking that
you're compiling with the right version of java, etc., and a target for
running tests-- including setting up a the test environment, with the
caches and data sources and whatnot set up to run the jira tests, vs.
having to manually set that stuff up to run them.

We're going to have to do a "synchronize watches" type deal at some
point, as to get rid of the binary artifacts I had to rewrite the
history for the repo, but I wanted to especially toss out the bit about
the lucee-context.lar, and see what the thoughts are about giving it
it's own version (maybe do the same thing for lucee-inst.jar too, since
it changes infrequently as well).

-Den

[Chris Blackwell]

Hi Den,

I think separating the lucee-context and lucee-inst (not required in 5.0?) would be a fine idea.  If the intention is to move the admin to an extension then having a separate artifact might be useful.

I'm not sure how those version numbers should roll, if they change infrequently then theres no point keeping them in lockstep with lucee core.

Are these changes in bitbucket yet ?

--
You received this message because you are subscribed to the Google Groups "Lucee" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lucee+un...@googlegroups.com.
To post to this group, send email to lu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lucee/55014206.8060809%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Jochem van Dieten]

On Thu, Mar 12, 2015 at 8:36 AM, denstar wrote:

I've made some pretty major modifications to the build scripts.  The
biggest is the move to using dependencies for the libs instead of
bundling them in the source, and doing the same for the server and JRE
bundles.

Sounds good. Does this use Maven to manage the dependencies? Can we preview it soewhere?

Jochem

--

Jochem van Dieten
http://jochem.vandieten.net/

[denstar]

On 3/12/15 3:46 AM, Chris Blackwell wrote:
> Hi Den,
>
> I think separating the lucee-context and lucee-inst (not required in
> 5.0?) would be a fine idea. If the intention is to move the admin to an
> extension then having a separate artifact might be useful.

Good point! It would also make the core build faster, as right now it
has to assemble a temporary war, start it, and then call the lucee
archive generation script to create the lucee-context.lar file.

> I'm not sure how those version numbers should roll, if they change
> infrequently then theres no point keeping them in lockstep with lucee core.

Using the same version number as "core" would actually be confusing,
we'd want to start at 1.0.0, and go from there (using semver). They
would just be dependencies for core, same as the other dependencies, so
we'd build say lucee-admin 1.0.1, and then bump the dependency number in
core to "1.0.1", for example.

> Are these changes in bitbucket yet ?

Yep, they're in there, but in a different repo for now as there's still
some flux, and keep in mind that I'm blowing the history away regularly
(push --all --force), to keep them in sync with the blessed repo.

https://bitbucket.org/valliant/lucee-small

Something like "./lucee-releng core" should build the core artifacts, or
"./lucee-releng war" to build a war, etc.. I'm thinking about renaming
the "lucee-releng" to "run-build" or maybe just "releng", which is short
for "release engineering", but run-build is probably the most obvious.

There are also some options like passing "java.home=/path/to/jre" to set
the java used to compile, which might be handy to know about. :)

I'll add a build/README.md or BUILDING.md file, and mirror the content
to here: https://bitbucket.org/lucee/lucee/wiki/Build_from_source to
help us discuss what makes the most sense given what's available.

It should be noted that these scripts are designed to be runnable on
every supported OS-- the idea is that at some point we'll have a
dedicated build server with VMs of the various operating systems, all
running CI.

And even setting up CI is scripted -- there are no "pets" here, we're
pure beef -- which helps a lot for "cloud" stuff, and provisioning in
general. (I freaking love automation. Side note: My brother is getting
me into building robots, so look out world! We be get'n physical now. =])

-Den

[denstar]

On 3/12/15 6:22 AM, Jochem van Dieten wrote:
...

>
> Sounds good. Does this use Maven to manage the dependencies? Can we
> preview it soewhere?

I posted the link in my last post, just keep in mind the caveats. :)

It "sort of" uses the maven-ant-task for dependencies and artifact
deployment, so the artifacts themselves are consumable from Maven
(though at some point it will probably be switched to aether, which is
compatible). There are pom parents for doing super easy maven
deployments of lucee itself in various configurations (vs. building lucee).

I say "sort of" because it's wrapped in a macrodef for handling things
related specifically to provisioning CFML applications (extensions and
what have you, setting up test harnesses), as well as geared for
handling binary artifacts in general, which is kind of a weak point for
Maven (you can do it, but it's /so/ verbose, and not super intuitive).

One of the results of the build is a lucee-lib.pom, which could be used
for maven builds in the future (defines optional artifacts like the DB
drivers, etc.), and is where you modify dependencies for lucee, though
we're in a transitory state right now-- I added the existing libs as
artifacts under a lucee groupId, whereas we really want to use existing
groupIds/artifactIds.

There's still some stuff which compares the hashes of the jars in the
lib directory with the dependencies, as I've been doing dependency based
builds for several years now, and that was a safeguard for working with
the jars being added to source control, vs. being dependencies-- but
once we've switched the scripts we can get rid of that hash check, and
testing newer versions of say httpclient will be a matter of bumping the
version and running a build.

-Den