[izpack-dev] Experimenting with Maven build

[Dan Tran]

I'd like to continue with Dennis' work to port IZPack Ant build to
Maven build at a sandbox place like
https://svn.codehaus.org/izpack/izpack-maven/trunk/sandbox

Any objection?

-Dan

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

[Julien Ponge]

> Any objection?

No objection... encouragements to be true :-)

I think one main issue you will encounter is splitting the code into
modules, as:
* the build is monolithic, and
* the packages layout is not clean.

Anyway this is a work that would be extremely useful, so thanks a lot Dan!

Cheers

--
http://izpack.org/
http://jpz-log.info/
http://julien.ponge.info/

[Julien Ponge]

BTW did we get news from Dennis? I'm sure it would be great to get his
feedback / experiences on the matter.

Cheers

[Dan Tran]

Hi Julien

I have not heard from him, how ever, base on his current work, and my
experience on the last few day, we are see the same issue you have
suggesting.

I am thinking of divide izpack into:

izpack-util
izpack-api
izpack-compiler
izpack-installer
izpack-uninstaller
izpack-event
izpack-panel
izpack-listener
izpack-native
izpack-dist

out side of those modules,

we can throw in izpack-site, and izpack-docs ( using standard docbook
or DITA :-) )

It just a thought at this moment.

-Dan

[Julien Ponge]

Hi Dan,

> I am thinking of divide izpack into:
>
> izpack-util
> izpack-api
> izpack-compiler
> izpack-installer
> izpack-uninstaller
> izpack-event
> izpack-panel

+ 1 submodule per panel?
> izpack-listener
+ (same idea)?
> izpack-native
> izpack-dist

*yes*, this sounds good :-)

Along the way you might need to do some refactorings, like renaming /
splitting packages, but it will do the code a lot of good!

As I said when Dennis launched the idea a few month back: green flag
to do any necessary change :-)

Cheers

--
http://izpack.org/
http://jpz-log.info/
http://julien.ponge.info/

---------------------------------------------------------------------

[Dan Tran]

all izpack panels can stay in one module, we will use split the
package up into multiple jar with classifier at package time. Same is
for izpack listeners

Also, please note that current user can the specify the panel and
listener using full package name, and therefor we may want to
deprecate individual jars.
This means they can just use the standalone compiler without the need
of full izpack installation. Your latest changes to standalone
packaging open make this
possible for Ant user as well.

-Dan

[Julien Ponge]

Very nice!

[René Krell]

It would be very nice to have also a build system for native JNI code, which
automatically builds at least Linux + Windows 32 + 64 bit shared libraries.
This might not be easy, may be in an emulator or virtual machine...

Or is there already a solution I haven't seen yet?

René

[Dan Tran]

native-maven-plugin will do the job

-Dan

[Dan Tran]

I spoke too soon,

We will have a separate project just to build the native code have its
own release cycle.

[Dan Tran]

Hi IZPack team

After a new night pulling my hair out, and attempt to split the
current IZPack Ant build
into multiple module with Maven as build infrastructure, I have
accepted the defeat due
to many cyclic dependencies among izpack java packages. Changing the
code, moving
class between packages is suicidal.

However, one good thing come out of this experiment is that all IZPack
java source can still build
with Maven nicely using single module. Integrating with Eclipse is a
breeze and pleasant to work with

Here a few recommendations:

- Move on with single maven build and create multiple jars out of
maven. However
I also strongly suggest that we should soon deprecate the
requirement to have multiple
panels and events jars to reduce the complexity and usage since
user can specify
the full package name to achieve the same result.

- Due to its Eclipse friendliness, we can begin slowly refactor IZPack
see this http://www.infoq.com/news/2009/11/legacy-code and add tests.

Thoughts?

-Dan

[Julien Ponge]

Hi Dan,

First of all, thanks for your efforts! I hope you found some time for
a little nap though :-)

> I also strongly suggest that we should soon deprecate the
> requirement to have multiple
> panels and events jars to reduce the complexity and usage since
> user can specify
> the full package name to achieve the same result.

Well a panel jar contains several classes (not just the one from the
panel class), so I am not sure how you would fix the problem. Also,
the thing which is important is not to include all panel classes in
every installer, but rather just the specified ones.

Could you please clarify this?

> - Due to its Eclipse friendliness, we can begin slowly refactor
> IZPack
> see this http://www.infoq.com/news/2009/11/legacy-code and add
> tests.

Nice. Refactoring "old" code is always an art :-)

That's a point I mentioned in my talk this week: code goes through
lots of hype waves over the years, and eventually looks bad... yet
works.

BTW did you have a look at IntelliJ IDEA? (now even opensource) The
refactorings it offers are superior to those of Eclipse.

[Dan Tran]

>>    I also strongly suggest that we should soon deprecate the
>> requirement to have multiple
>>    panels and events jars to reduce the complexity and usage since
>> user can specify
>>    the full package name to achieve the same result.
>
> Well a panel jar contains several classes (not just the one from the panel
> class), so I am not sure how you would fix the problem. Also, the thing
> which is important is not to include all panel classes in every installer,
> but rather just the specified ones.
>
> Could you please clarify this?

For user with required small foot print, so only include necessary
panel jar file is a good thing to have.

For my case, the installer always big so adding an extra 700K of full
izpack jar is not that bad

So my take here, how often we see user with small footprint
requirement? and extra 500K is not acceptable?

Can we poll the user?

>
>>  - Due to its Eclipse friendliness, we can begin slowly refactor IZPack
>>    see this http://www.infoq.com/news/2009/11/legacy-code and add tests.
>
>
> Nice. Refactoring "old" code is always an art :-)
>
> That's a point I mentioned in my talk this week: code goes through lots of
> hype waves over the years, and eventually looks bad... yet works.
>
> BTW did you have a look at IntelliJ IDEA? (now even opensource) The
> refactorings it offers are superior to those of Eclipse.
>

IntelliJ IDEA is also a good one to use, there is maven plugin to
generate project file for you on the fly.

The key here, we dont have to bind IZPack build to any specific IDE

> Cheers

Dan

[Julien Ponge]

> Can we poll the user?

Yes, please send an email to the user list.

> The key here, we dont have to bind IZPack build to any specific IDE

Of course :-)

The other thing I like with the recent IntelliJ IDEA builds is that it
can directly load a POM as a project, which is a super-fast way to
work with Maven-enabled projects.

--
http://izpack.org/
http://jpz-log.info/
http://julien.ponge.info/

[Dan Tran]

Let's me do some more experiments to understand more about IZPack
internals before polling IZPack user

Thanks for the feedback.

-Dan

[Anthonin Bonnefoy]

I'm currently trying to separate IZpack in maven modules but package dependencies and singletons make the work really difficult. I'm gonna try a bit more but it is necessary to do a lot of heavy refactoring.

It would be nice to have dependency injection. Is a footprint of ~250K (pico container) too much for this feature?

Anthonin

[Julien Ponge]

Hi Anthonin,

> I'm currently trying to separate IZpack in maven modules but package
> dependencies and singletons make the work really difficult. I'm gonna try a
> bit more but it is necessary to do a lot of heavy refactoring.

I fully agree.

> It would be nice to have dependency injection. Is a footprint of ~250K (pico
> container) too much for this feature?

I would not mind.

I would however be more annoyed to have all panel classes in every
installer though, as it increases the footprint by a lot more than
250k. Anyway on this particular point I think we need Dan to ask the
users and see wether they mind or not, as putting them all together is
simpler (no need for a lot of submodules).

Cheers

[Dan Tran]

Hi Anthonin

Glad to see someone on the same boat with me.

I think it is best to do straight port of the ant build using the
following module

izpack-core
izpack-compiler
izpack-installer,
....
izpack-standalone-compiler
izpack-dist

izpack-core has every thing, then extract subsets of izpack-core into
other modules according the ant build.

This structure paves the way for us to write unit tests, integration
tests, and start other refractorings
like adding dependency injection like you have suggested. The key
here is we need tests in order to
start refractoring.

I just checked in the initial izpack-core and izpack-compiler build at
izpack-maven/sandbox. Please take a look

-Dan

[Julien Ponge]

Thanks Dan,

I'll carefuly follow the activity there as well :-)

Cheers

Le 22 nov. 09 à 05:44, Dan Tran a écrit :

[Dan Tran]

btw, I wont be able to continue any more work on this experiment, will
try to get back after that

-D

[Dan Tran]

i meant this week, will get back after that :-)

-Dan

[Anthonin Bonnefoy]

Hello,

I confirm what you said Dan, moving classes in different modules or packages is quite difficult, there are too much cyclic dependencies between classes and packages. 

Nevertheless, I want to try refactoring and eliminates dependencies between packages step by step, first trying to get a working version, writting tests and refactoring in order to decompose Izpack in correct maven modules.

I'm working on a github fork (http://github.com/bonnefoa/izpack-refactoring/tree/tests) for those who are interested. 

Anthonin

[Anthonin Bonnefoy]

Dan, I looked what you have checked in. It's faster to have a working maven project with your method but I wouldn't have done it like this.
I will continue to work on my ideas so we may compare both solutions :-)

Cheers,
Anthonin

[Dan Tran]

Hi Anthonin,

By no mean, please proceed with your work.... let me know your progress.

Thanks

-Dan

On Tue, Nov 24, 2009 at 5:23 AM, Anthonin Bonnefoy

[Anthonin Bonnefoy]

Hi Dan,

Like I said on twitter (http://twitter.com/abonnefoy/), I managed to separate  compiler and installer packages.
For now, i'm still trying to get a working version by making some integration tests but, in order to do integration tests, I have to do some refactoring (quite the vicious circle...).
I'm using PicoContainer to get rid of those (evil) singletons and to be able to write tests.
The first tests are on a "Working on my computer" state. I need to figure out why they are failing on bamboo... (http://bamboo.ci.codehaus.org/browse/IZPACK-GITMVNREFACTOR-9).
I will keep you in touch as soon as I make significant progress.

Anthonin

[Julien Ponge]

Fantastic!

Le 29 nov. 09 à 14:47, Anthonin Bonnefoy a écrit :

[Anthonin Bonnefoy]

Hello IzPack team,

A point on our advancement (I say our since David came in reinforcement), it's not a little maven revamp anymore but more a (big) refactoring of IzPack.

Currently, the maven version works only with the sample installer. There is still a lot of work to do.

We keep a tracker of our current work here : http://dduponchel.dyndns.org:8080/youtrack (only avaible during day)
A CI server (some problems with codehaus bamboo...) : http://dduponchel.dyndns.org:8080/TeamCity/
We use github to work/merge/keeping synchronised with IzPack svn :
 mine : http://github.com/bonnefoa/izpack-refactoring/
 david : http://github.com/dduponchel/izpack

We made a lot of changes, especially in the architecture with an important use of PicoContainer (dependency injection) and we try to eradicate every singletons.

You may ask, why trying to erase the existence of singletons?
 - First, because they are evil.
 - Second, you can't mock them, testing the application is a nightmare.
 - Third, you can't extract interface from a singleton so it's quite difficult to separate classes in modules.

The use of PicoContainer gives a lot of advantages.
 - Pico can inject any component in your constructor. For example, you can get the ResourceManager simply by adding it as a constructor parameter.
 - Easy creation of reusable manager. There is a some duplicated code (writeUninstall) which can be factorised.
 - Easy integration testing and unit testing. You can mock every dependencies
 - And more to come :D

There is an automatic integration test (using fest) of the sample installer. I'm currently working on making an automated test of the IzPack installer; Glassfish installer coming next.

I will keep you inform when we will reach the next step, which is to make IzPack installer working.

Anthonin Bonnefoy

[Dan Tran]

I am very impressed with the commitment.

I can see bright future with IZPack, let's beat the heck out of
InstallAnywhere :-)

-Dan

[Julien Ponge]

I must say that I am very pleased to see the hard work Anthonin and
David are putting in, not only because it will eventually lead us to a
nice modular Maven setup, but because they are making some well
deserved cleanups on the code.

As a GitHub user I have been following the work and it is looking very
promising,

In case of success (which I am pretty sure of) I propose that this
work becomes our trunk, and that we rebadge it as IzPack 5.

[Anthonin Bonnefoy]

Some news on the refactoring.
We still alive and working on it. I began to refactor the compiler and try to reduce some of the big classes and remove duplication.
No real big progress this time, we progress small steps by small steps.

Two particular things: I began to write some unit tests in Scala because it is simpler when the methods require an XmlElement (see CompilerConfigScalaTest). The use of Scala is limited to testing.

Another thing is a modularization problem. Currently, a jar is produced with ant for each panels/listener/...
It is possible to reproduce this comportment with Maven by
 - creating a module for each panel/listener, which is horrible.
 - creating an assembly for each panel/listener, which is a bit better but I think it will be hard to maintain.
The aim is to merge class files and resources in the skeleton installer so it might be possible to use a custom system.
I thought of giving the possibilities to filter packages when merging a jar.
We will have a jar with all panels in which we will pick packages corresponding to the desired panels. We won't need to care anymore about separating elements in different jars and it might be simpler to use.
It's only a proposition and I am still unsure on how to do it. Nevertheless, there is a lot of refactoring to do before really worrying about this :-).

Cheers,
Anthonin

[Julien Ponge]

Hi Anthonin,

Thanks for your feedback and efforts!

>  - creating a module for each panel/listener, which is horrible.

Is that really a big deal? (apart from the long initial work to split
into modules)

>  - creating an assembly for each panel/listener, which is a bit better but I
> think it will be hard to maintain.

Exactly

> I thought of giving the possibilities to filter packages when merging a jar.
> We will have a jar with all panels in which we will pick packages
> corresponding to the desired panels. We won't need to care anymore about
> separating elements in different jars and it might be simpler to use.
> It's only a proposition and I am still unsure on how to do it. Nevertheless,
> there is a lot of refactoring to do before really worrying about this :-).

That could be a nice solution indeed.

[Anthonin Bonnefoy]

Hi IzPack team,

Small update on the revamp status.
First, still alive and still working on it.

The module and package layout should now be stable. I stopped moving batch of classes from package to another and there are no more cyclic dependency.

I am changing the way the installer is merged. You can now directly specify a package or a class inside the classPath to be merged in the installer.
There is no need to create intermediate jar anymore thus all panels can be in a single module (like this http://github.com/bonnefoa/izpack-refactoring/tree/develop/izpack-panel/src/main/java/com/izforge/izpack/panels/).

Lately, I have been working on integration testing which now automatically load a created installation jar and test it.

I have broken some features which are on my todo list.
 - Console installation
 - Pack compression
 - Uninstaller
 - MultiPackager

I am currently restoring the uninstaller feature. Once it is done, I will try to create an assembly before moving to another feature (Pack compression I think).

Cheers,
Anthonin

[Dan Tran]

You are the master of refactoring. :-)

-D

[Julien Ponge]

He's actually proving that you *can* introduce tests and refactor an
already big project ;-)

[René Krell]

Hi Anthonin,

just got to add, that I'm watching your work, too.
I'm doing implementations in the current trunk and hoping for a more
sophisticated structure, although I'm a little bit afraid of re-merging
current changes at the end :-) Nevertheless, go go go, we need this!

René

On Thursday 11 February 2010 13:20:08 Anthonin Bonnefoy wrote:
> Hi IzPack team,
>
> Small update on the revamp status.
> First, still alive and still working on it.
>
> The module and package layout should now be stable. I stopped moving batch
> of classes from package to another and there are no more cyclic dependency.
>
> I am changing the way the installer is merged. You can now directly specify
> a package or a class inside the classPath to be merged in the installer.
> There is no need to create intermediate jar anymore thus all panels can be
> in a single module (like this
> http://github.com/bonnefoa/izpack-refactoring/tree/develop/izpack-panel/src
> /main/java/com/izforge/izpack/panels/ ).
>
> Lately, I have been working on integration testing which now automatically
> load a created installation jar and test it.
>
> I have broken some features which are on my todo list.
> - Console installation
> - Pack compression
> - Uninstaller
> - MultiPackager
>
> I am currently restoring the uninstaller feature. Once it is done, I will
> try to create an assembly before moving to another feature (Pack
> compression I think).
>
> Cheers,
> Anthonin

---------------------------------------------------------------------

[Julien Ponge]

This makes me think that you may try to fork Anthonin's branch and
develop your work there.

--

http://izpack.org/
http://jpz-log.info/
http://julien.ponge.info/

---------------------------------------------------------------------

[René Krell]

It depends, whether Anthonin declares his structure now as stable ;-)

On Thursday 11 February 2010 16:52:36 Julien Ponge wrote:
> This makes me think that you may try to fork Anthonin's branch and
> develop your work there.
>
> On Thu, Feb 11, 2010 at 4:51 PM, René Krell <rkr...@gmx.net> wrote:
> > Hi Anthonin,
> >
> > just got to add, that I'm watching your work, too.
> > I'm doing implementations in the current trunk and hoping for a more
> > sophisticated structure, although I'm a little bit afraid of re-merging
> > current changes at the end :-) Nevertheless, go go go, we need this!
> >
> > René
> >
> > On Thursday 11 February 2010 13:20:08 Anthonin Bonnefoy wrote:
> >> Hi IzPack team,
> >>
> >> Small update on the revamp status.
> >> First, still alive and still working on it.
> >>
> >> The module and package layout should now be stable. I stopped moving
> >> batch of classes from package to another and there are no more cyclic
> >> dependency.
> >>
> >> I am changing the way the installer is merged. You can now directly
> >> specify a package or a class inside the classPath to be merged in the
> >> installer. There is no need to create intermediate jar anymore thus all
> >> panels can be in a single module (like this
> >> http://github.com/bonnefoa/izpack-refactoring/tree/develop/izpack-panel/

> >> src /main/java/com/izforge/izpack/panels/ ).

[Tim Farrell]

I like the direction of all of this. Great work.

These changes won't impose a Maven requirement in the build process for those users that don't use Maven to build their installers will it? I mean to say, I'm not using Maven today and I'd like to not have to start using it due to these changes. If i have to I can, but it will delay how soon I can step up to this version when its complete.

At 07:20 AM 2/11/2010, you wrote:

Hi IzPack team,

Small update on the revamp status.
First, still alive and still working on it.

The module and package layout should now be stable. I stopped moving batch of classes from package to another and there are no more cyclic dependency.

I am changing the way the installer is merged. You can now directly specify a package or a class inside the classPath to be merged in the installer.
There is no need to create intermediate jar anymore thus all panels can be in a single module (like this http://github.com/bonnefoa/izpack-refactoring/tree/develop/izpack-panel/src/main/java/com/izforge/izpack/panels/ ).

Lately, I have been working on integration testing which now automatically load a created installation jar and test it.

I have broken some features which are on my todo list.

 - Console installation
 - Pack compression
 - Uninstaller
 - MultiPackager

I am currently restoring the uninstaller feature. Once it is done, I will try to create an assembly before moving to another feature (Pack compression I think).

Cheers,
Anthonin

On Fri, Jan 8, 2010 at 9:46 AM, Julien Ponge <julien...@gmail.com > wrote:

Hi Anthonin,

Thanks for your feedback and efforts!

> Â - creating a module for each panel/listener, which is horrible.

Is that really a big deal? (apart from the long initial work to split

into modules)

> Â - creating an assembly for each panel/listener, which is a bit better but I

> think it will be hard to maintain.

Exactly

> I thought of giving the possibilities to filter packages when merging a jar.

> We will have a jar with all panels in which we will pick packages

> corresponding to the desired panels. We won't need to care anymore about

> separating elements in different jars and it might be simpler to use.

> It's only a proposition and I am still unsure on how to do it. Nevertheless,

> there is a lot of refactoring to do before really worrying about this :-).

That could be a nice solution indeed.

Cheers

--

http://izpack.org/

http://jpz-log.info/

http://julien.ponge.info/

---------------------------------------------------------------------

To unsubscribe from this list, please visit:

   http://xircles.codehaus.org/manage_email

[Dan Tran]

Hi Tim,

No, user is not imposed to use maven since the installer distribution
and ant task are still available.

But it is much more friendlier for Maven user since all they need is
izpack-maven-plugin which is automatically pulling down by maven.

-D

http://xircles.codehaus.org/manage_email

[Anthonin Bonnefoy]

@Dan

You are the master of refactoring.  :-)

I really learned a lot from this. Like how singletons are a real nightmare and how cyclic dependencies can be evil :D

@Julien

He's actually proving that you *can* introduce tests and refactor an
already big project ;-)

Thanks to you, I am git, IntelliJ and automatic-test addicted. One of the reason of my motivation is : "It might be cool (and useful) to have automated IzPack tests" ^^

@Rene

I'm doing implementations in the current trunk and hoping for a more
sophisticated structure, although I'm a little bit afraid of re-merging
current changes at the end :-) Nevertheless, go go go, we need this!

The current structure has emerged from all the classes move I made. This is not a fixed structure but more like a new base for further evolution. Sophisticated often means complicated and I am more focused of getting IzPack working with the current structure.

I try to keep synchronised with the subversion trunk but there will some regression hunting to do at some point.

If you are not afraid of not being able to build an installer (not yet working) or currently not working tests (my develop branch is really unstable), I don't see a problem, there normally won't be any major changes from now.
However, I think that there is several problems with windows that I couldn't see (I only have linux at home...). If you wait some days, I will soon have a more stable version.

@Tim
The changes are centered on IzPack development and build process and there will be no change on user side. I (try) to maintain the same features while changing the structure.

[Julien Ponge]

Just in case you don't have those errors, I can't `mvn install` from
your develop branch on Mac OS X.

Here is a log file. I ran

mvn -e -U --up clean install -DskipTests=true

Hope it helps ;-)

[Anthonin Bonnefoy]

I have the same error and it's on my pipe.

I just tend to forget it since I locally deactivated the dist module...