Sculptor: Improvement of Maven tool chain

[Torsten Juergeleit]

I'm not so fond of Sculptors current Maven tool chain:

* The Maven clean plugin is used to deleted the generated stuff from "src/generated/".
* The checksum plugin is used to deleted the unmodified one-shot generated stuff.
* The OAW workflow plugin generates stuff.
* The Graphviz plugin generates additional stuff.

These plugins have to be configured (every plugin has its own type of configuration) within every Sculptor project. To propagate state between plugin executions various files are used, e.g. ".beforeInstallFlag", ".checksum.txt", ".ignore-checksum.txt" and "oaw-generation-lastrun.timestamp".

Due to its loose-coupled nature this tool chain is complex (e.g. internal synchronization via marker files) and brittle (e.g. the plugins lifecycle phase or ordering in POM are important).


To get rid of this mix of different plugins I'm thinking about tweaking the new Sculptor Maven plugin [1] to support clean-up, checksum-generation and Graphviz dot execution:

1. Retrieve a list of all files generated / switched while running Sculptors MWE2 workflow. This can be done via a combination of a Logback logger / appender and a custom commons-exec LogOutputStream for parsing the debug log output of the class "org.eclipse.xpand2.output.FileHandleImpl".

2. After successful execution of Sculptors MWE2 workflow write the list of all generated files (together with a MD5 checksum of the file content -> via commons-codec DigestUtils.md5Hex()) to a marker file, e.g. "${basedir}/.sculptor-status".

3. Before starting Sculptors MWE2 workflow retrieve the list of files generated during the last workflow execution, compare the checksum of the file contents and delete all unmodified files. The files within "src/generated/" are always deleted regardless of their checksum.

4. Provide a configuration option to turn-off the deletion of generated file if the partial generation mode [2] of Sculptors MWE2 workflow is used.

5. Implement an additional Mojo which is bound to Mavens clean lifecycle phase to delete all files generated during the last workflow execution as described in topic 3.

6. Integrate the Graphiz plugins mojo by extending from the base class "AbstractSculptorMojo" shared by all mojos of the Sculptor Maven plugin. This base class provides the common stuff like properties (Maven project, outlet directories or maker file) and code like marker file handling.


WDYT?
/Torsten

[1] https://groups.google.com/forum/#!msg/fornax-platform/FiWmpEgbEv0/z_Q2c3Mi2WoJ
[2] http://fornax.itemis.de/confluence/display/fornax/7.+Developer%27s+Guide+%28CSC%29#7.Developer%27sGuide%28CSC%29-PerformanceTuningofGenerator

[Patrik Nordwall]

I like anything that will make this simpler and more robust, but still flexible to tune for different usage. See comments inline...

On Thu, Jan 19, 2012 at 12:14 AM, Torsten Juergeleit <torsten.j...@gmail.com> wrote:

I'm not so fond of Sculptors current Maven tool chain:

* The Maven clean plugin is used to deleted the generated stuff from "src/generated/".
* The checksum plugin is used to deleted the unmodified one-shot generated stuff.
* The OAW workflow plugin generates stuff.
* The Graphviz plugin generates additional stuff.

These plugins have to be configured (every plugin has its own type of configuration) within every Sculptor project. To propagate state between plugin executions various files are used, e.g. ".beforeInstallFlag", ".checksum.txt", ".ignore-checksum.txt" and "oaw-generation-lastrun.timestamp".

Due to its loose-coupled nature this tool chain is complex (e.g. internal synchronization via marker files) and brittle (e.g. the plugins lifecycle phase or ordering in POM are important).


To get rid of this mix of different plugins I'm thinking about tweaking the new Sculptor Maven plugin [1] to support clean-up, checksum-generation and Graphviz dot execution:

1. Retrieve a list of all files generated / switched while running Sculptors MWE2 workflow. This can be done via a combination of a Logback logger / appender and a custom commons-exec LogOutputStream for parsing the debug log output of the class "org.eclipse.xpand2.output.FileHandleImpl".

2. After successful execution of Sculptors MWE2 workflow write the list of all generated files (together with a MD5 checksum of the file content -> via commons-codec DigestUtils.md5Hex()) to a marker file, e.g. "${basedir}/.sculptor-status".

3. Before starting Sculptors MWE2 workflow retrieve the list of files generated during the last workflow execution, compare the checksum of the file contents and delete all unmodified files. The files within "src/generated/" are always deleted regardless of their checksum.

4. Provide a configuration option to turn-off the deletion of generated file if the partial generation mode [2] of Sculptors MWE2 workflow is used.

It's a big performance difference to run

mvn clean generate-sources

compared

to mvn generate-sources

I tend to use mvn -o generate-sources most of the time, and only do the cleaning when bigger changes.

Cleaning is also not very nice for Eclipse because normally the compiler gets out of sync and a full Project>Clean is needed in Eclipse also.