3.1.0-SNAPSHOT build errors

[El Deffo]

Hi, 

  I'm trying to build Sculptor 3.1.0-SNAPSHOT, but end up with build failures like these:

[ERROR]

ERROR:  SculptordslScopeProvider.xtend -

27: org.sculptor.dsl.sculptordsl.DslDomainObject cannot be resolved to a type.

[ERROR]

ERROR:  SculptordslScopeProvider.xtend -

28: org.sculptor.dsl.sculptordsl.DslOppositeHolder cannot be resolved to a type.

...

[INFO] Reactor Summary:

[INFO]

[INFO] Sculptor ........................................... SUCCESS [  1.409 s]

[INFO] Sculptor :: Eclipse ................................ SUCCESS [  0.228 s]

[INFO] Sculptor :: Eclipse :: Textual DSL ................. FAILURE [02:41 min]

...

[ERROR] Failed to execute goal org.eclipse.xtend:xtend-maven-plugin:2.7.3:compile (default) on project org.sculptor.dsl: Error compiling xtend sources in 'P:\GitHome\sculptor\sculptor-eclipse\org.sculptor.dsl\src;P:\GitHome\sculptor\sculptor-eclipse\org.sculptor.dsl\src-gen'. -> [Help 1]

org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.eclipse.xtend:xtend-maven-plugin:2.7.3:compile (default) on project org.sculptor.dsl: Error compiling xtend sources in 'P:\GitHome\sculptor\sculptor-eclipse\org.sculptor.dsl\src;P:\GitHome\sculptor\sculptor-eclipse\org.sculptor.dsl\src-gen'.

What could be the issue here?

Also, development snapshots at https://oss.sonatype.org/content/repositories/snapshots/ are at least a year old. Would it be possible to update them?

Thanks.

[Torsten Juergeleit]

These compile errors are hinting that the execution of Xtexts code generator (via Maven Exec Plugin) failed and the source code of the corresponding Java classes wasn't generated (in the directory "src-gen").

Hhm, I didn't try to build Sculptor on windows recently =8-o
So I'm not sure if there's any windows-related glitch in the paths (this slash vs. backslash thing) used in the projects POM and the MWE2 workflow.

Please run you build with verbose logging "mvn clean install -X" and check the filesystem path logged in the configuration of the "exec-maven-plugin" for the following line from the POM:

<argument>file://${project.basedir}/src/org/sculptor/dsl/GenerateSculptorDsl.mwe2</argument>

And check the log output of the MWE2 workflow launcher (the Java class executed by the Maven Exec Plugin) for any errors.

/Torsten

[Torsten Juergeleit]

Also, development snapshots at https://oss.sonatype.org/content/repositories/snapshots/ are at least a year old. Would it be possible to update them?

Done

/Torsten

On Thursday, January 15, 2015 at 2:48:59 PM UTC+1, El Deffo wrote:

[El Deffo]

Thanks, I'm able to continue with the snapshot for now.

I tried to debug that mwe2 workflow for a while, but as I don't know anything about how xtext works and have no idea what the correct outputs should be, I won't get anywhere.

Seems that when I run mwe2 from eclipse, different files are producent than when I run mvn clean install in sculptor\sculptor-eclipse\org.sculptor.dsl .

I attached the output of sculptor\sculptor-eclipse\org.sculptor.dsl>mvn clean install -e -X -l=mvn.log, but I don't think it's going to be of much use without proper debugging by someone who knows his way around, hm.

[Torsten Juergeleit]

The attached log file looks fine: The MWE2 workflow execution took ~39seconds and some info logging from the Junit4Fragment is included (unluckily the other Xtext code generator fragments don't do any logging:-( ). here's the corresponding output from the build on my Mac:

[INFO] --- exec-maven-plugin:1.2.1:java (default) @ org.sculptor.dsl ---

0    [org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher.main()] INFO  lipse.emf.mwe.utils.StandaloneSetup  - Using resourceSet registry. The registered Packages will not be registered in the global EPackage.Registry.INSTANCE!

7    [org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher.main()] INFO  lipse.emf.mwe.utils.StandaloneSetup  - Adding URI mapping from 'platform:/resource/org.eclipse.xtext.xbase/' to 'classpath:/'

9    [org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher.main()] INFO  lipse.emf.mwe.utils.StandaloneSetup  - Adding URI mapping from 'platform:/resource/org.eclipse.xtext.common.types/' to 'classpath:/'

9    [org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher.main()] INFO  lipse.emf.mwe.utils.StandaloneSetup  - Registering platform uri '/Users/torsten/Develop/git/sculptor/sculptor-eclipse'

39   [org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher.main()] WARN  lipse.emf.mwe.utils.StandaloneSetup  - Skipping conflicting project org.sculptor.dsl.ui at 'archive:file:/Users/torsten/Develop/git/sculptor/sculptor-eclipse/eclipse-repository/target/repository/plugins/org.sculptor.dsl.ui_3.0.6.jar!/' and using 'archive:file:/Users/torsten/Develop/git/sculptor/sculptor-eclipse/org.sculptor.dsl.ui/target/org.sculptor.dsl.ui-3.1.0-SNAPSHOT.jar!/' instead.

1135 [org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher.main()] INFO  ipse.emf.mwe.utils.DirectoryCleaner  - Cleaning /Users/torsten/Develop/git/sculptor/sculptor-eclipse/org.sculptor.dsl/model

5134 [org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher.main()] INFO  clipse.emf.mwe.utils.GenModelHelper  - Registered GenModel 'http://sculptor.org/dsl/sculptordsl' from 'platform:/resource/org.sculptor.dsl/model/generated/Sculptordsl.genmodel'

31405 [org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher.main()] INFO  text.generator.junit.Junit4Fragment  - generating Junit4 Test support classes

31414 [org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher.main()] INFO  text.generator.junit.Junit4Fragment  - generating Compare Framework infrastructure

31475 [org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher.main()] INFO  .emf.mwe2.runtime.workflow.Workflow  - Done.

[INFO]

[INFO] --- xtend-maven-plugin:2.7.3:compile (default) @ org.sculptor.dsl ---

Please check the output folder "src-gen/" for the generated Java source code of Sculptors DSL and "target/xtend/" for the corresponding Java classes created by Xtends Java compiler. Here's a screenshot of my folders after executing "mvn clean generate-sources" in the DSL Eclipse plugin project (no folder "target/classes/" [created by Tychos Java compiler] yet):

At first glance the only difference (comparing to my local build and Sculptors CI build on Travis) is that you're using Java 8. Any chance to check with Java 7?

/Torsten

[El Deffo]

Tried it with Java 7, didn't help.

src-gen\org\sculptor\dsl\sculptordsl directory is missing, everything else is the same as on your screenshot.

my log output is missing the line:

39   [org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher.main()] WARN  lipse.emf.mwe.utils.StandaloneSetup  - Skipping conflicting project org.sculptor.dsl.ui at 'archive:file:/Users/torsten/Develop/git/sculptor/sculptor-eclipse/eclipse-repository/target/repository/plugins/org.sculptor.dsl.ui_3.0.6.jar!/' and using 'archive:file:/Users/torsten/Develop/git/sculptor/sculptor-eclipse/org.sculptor.dsl.ui/target/org.sculptor.dsl.ui-3.1.0-SNAPSHOT.jar!/' instead.

and 

contains Registered GenModel 'http://sculptor.org/dsl/sculptordsl' from 'file:/P:/DevAppHome/GitHome/sculptor/sculptor-eclipse/org.sculptor.dsl/model/generated/Sculptordsl.genmodel'
instead of Registered GenModel 'http://sculptor.org/dsl/sculptordsl' from 'platform:/resource/org.sculptor.dsl/model/generated/Sculptordsl.genmodel'

Maybe these could be related?

[Torsten Juergeleit]

The warning "Skipping conflicting..." is not important for your issue. In my local build the Eclipse plugins of the last release build of Sculptor 3.0.6 are still present. Here a Maven profile is used the during release build.

But the difference in the GenModel URIs (file: vs. platform:) is interesting. In your case the Sculptors Eclipse plugin is referenced as file URI instead of a its platform URI referenced. That's invalid.

Please check if the Eclipse workspace metadata file ".project" is present in all Sculptor Eclipse projects. This metadata is needed because Sculptor uses Eclipse Tycho to build the Eclipse plugins and Tycho requires Eclipse metadata (MANIFEST.MFand .project) to handle Eclipse plugin projects.

If this metadata is not present then create it by importing Sculptors Eclipse plugin projects in Eclipse via Eclipse M2E "Import existing Maven project".

/Torsten

[El Deffo]

.project files are present, I already opened the project in eclipse.

As to why the paths are different, maybe it has something to do with the segment of org.sculptor.dsl\pom.xml you mentioned? 

<mainClass>org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher</mainClass>

<arguments>

<argument>file://${project.basedir}/src/org/sculptor/dsl/GenerateSculptorDsl.mwe2</argument>

<argument>-p</argument>

<argument>runtimeProject=/${project.basedir}</argument>

</arguments>

As paths on Windows don't start with /, maybe something other than <argument>runtimeProject=/${project.basedir}</argument> should be used?

[Torsten Juergeleit]

.project files are present, I already opened the project in eclipse.

You're right. Our Travis CI build is working without Eclipse meta data files as well.

As to why the paths are different, maybe it has something to do with the segment of org.sculptor.dsl\pom.xml you mentioned?

Sculptors Maven / Tycho / M2E / Xtext / Xtend configuration is copied from the GitHub example project provided by the Xtext team.

Maybe you can check if this Xtext project builds in your environment. If this isn't working as well then the Xtext team will come up with a solution.

/Torsten

[El Deffo]

So I just tried to build the xtext example project on both windows and ubuntu14 (I'm trying to do a fix for issue #168, @ForeignKey, as it's blocking my project), they work without a problem. Maybe you could fire up some VM and test it?

[El Deffo]

Sculptor doesn't build on linux either, same error as on windows.

[Torsten Juergeleit]

Maybe you could fire up some VM and test it?

Done.

I did a checkout on the local drive c: of a Win7 box and built Sculptor without a Maven repo manager and without a local Eclipse P2 mirror of the Luna distort (which is usually created via "mvn -Pmirror.initialize").

Here the expected behaviour of resolving the Eclipse platform URI (platform:/resource/) of the Sculptor DSL project was shown in the same way as on my Mac and Sculptors Travis CI Linux box:

    Registering platform uri 'C:\sculptor\git\sculptor-eclipse'

    Registered GenModel 'http://sculptor.org/dsl/sculptordsl' from 'platform:/resource/org.sculptor.dsl/model/generated/Sculptordsl.genmodel'

The differences in my windows test setup in comparison to yours are as follows (don't know if these matter):

1. Local drive c: instead of network share
2. No Eclipse P2 mirror -> latest version of Eclipse plugin dependencies from Luna distro
3. No Maven repo manager

Topic 2 and 3 are the same with Travis CI. But for efficient development you need both of them.

So how about updating your local Eclipse P2 mirror via "mvn -Pmirror initialise" and building Sculptor on a local drive instead of a network share?

And update to the latest dev version of Sculptor - I've updated the version of the Maven exec plugin.

/Torsten

[El Deffo]

Hm, my P: drive is actually a truecrypt drive, so local, not network. My p2 mirror was initialized maybe two weeks ago. And the build failed on almost vanilla Ubuntu14, without p2 mirror and maven repo manager, as well.
Anyway, I'll try to do some more tests, update the p2 mirror and what not.

[El Deffo]

So, did a fresh checkout, building now works on Ubuntu, still fails on Windows, but later in the process.

[WARNING] Don't override file P:\DevAppHome\GitHome\sculptor\sculptor-maven\sculptor-maven-archetype-parent\target\test-classes\projects\spring-with-ear\project\basic-parent\pom.xml

[INFO] project created from Archetype in dir: P:\DevAppHome\GitHome\sculptor\sculptor-maven\sculptor-maven-archetype-parent\target\test-classes\projects\spring-with-ear\project\basic-parent

[INFO] Comparing generated project with reference content: P:\DevAppHome\GitHome\sculptor\sculptor-maven\sculptor-maven-archetype-parent\target\test-classes\projects\spring-with-ear\reference

[WARNING] Contents of file pom.xml are not equal

[INFO] ------------------------------------------------------------------------

[INFO] Reactor Summary:

[INFO]

[INFO] Sculptor :: Maven :: Archetype for parent .......... FAILURE [  2.803 s]
...

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-archetype-plugin:2.2:integration-test (default-integration-test) on project sculptor-maven-archetype-parent:

[ERROR] Archetype IT 'ejb-with-ear' failed: Some content are not equals

[ERROR] Archetype IT 'plain' failed: Some content are not equals

[ERROR] Archetype IT 'spring-with-ear' failed: Some content are not equals

I think you are comparing sculptor-maven\sculptor-maven-archetype-parent\target\test-classes\projects\spring-with-ear\project\basic-parent\pom.xml with reference pom.xml, the content of both files is the same, but the line endings are different (windows), so the test fails. You probably should convert the generated file's line endings to unix format before the comparison, or vice versa?

By the way, is the local p2 mirror really neccessary, wouldn't it be possible to use just nexus and its p2 plugin instead? The complete mirror takes forever, nexus downloads just what is really needed, should be a lot faster.

[Torsten Juergeleit]

You probably should convert the generated file's line endings to unix format before the comparison, or vice versa?

Or how about changing your git clients configuration to do this conversion? :-P

This is a known is known issue ARCHETYPE-410 (e.g other OSS project).

By the way, is the local p2 mirror really neccessary, wouldn't it be possible to use just nexus and its p2 plugin instead? 

Nexus (with P2 support) or Artifactory are  fine. I didn't manage to get Nexus p2 support running with remote repositories in a stable way (without freezing if a remote p2 repo wasn't responding, e.g. this one).

So if you're able to get Nexus up-and-running with remote p2 repos then go for it.

The complete mirror takes forever, nexus downloads just what is really needed, should be a lot faster.

You're right - creating a local mirror of all p2 repos needed for Sculptor development takes hours and eats up a few GB of disk space.

But at the end of the day the performance of the remote p2 repos are critical - Nexus does only proxying the remote repo instead of creating a full mirror. So Nexus has to check for new artefacts on a regular basis.

And if you prefer having a specific version of a remote p2 repo (to omit side-effects due to newly published artefacts) you have no other chance then creating a local mirror and re-sync on demand.

It's your choice....

/Torsten

[El Deffo]

On Monday, February 16, 2015 at 9:22:53 PM UTC+1, Torsten Juergeleit wrote:

You probably should convert the generated file's line endings to unix format before the comparison, or vice versa?

Or how about changing your git clients configuration to do this conversion? :-P

This is a known is known issue ARCHETYPE-410 (e.g other OSS project).

 

You're right, I forgot I was mucking around with the setting some time ago :). My bad.

 

By the way, is the local p2 mirror really neccessary, wouldn't it be possible to use just nexus and its p2 plugin instead? 

Nexus (with P2 support) or Artifactory are  fine. I didn't manage to get Nexus p2 support running with remote repositories in a stable way (without freezing if a remote p2 repo wasn't responding, e.g. this one).

So if you're able to get Nexus up-and-running with remote p2 repos then go for it.

Hm, I didn't notice any issues so far, but I'll keep the p2 mirror for now.

Thanks.

I'm now stuck at a different phase on Windows
Building Sculptor :: Maven :: Plugin 3.1.0-SNAPSHOT:

Running org.sculptor.maven.plugin.CleanMojoTest

[INFO] Skipping deletion of previously generated files

Tests run: 5, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 1.518 sec <<< FAILURE! - in org.sculptor.maven.plugin.CleanMojoTest

testExecute(org.sculptor.maven.plugin.CleanMojoTest)  Time elapsed: 1.058 sec  <<< FAILURE!

junit.framework.AssertionFailedError: null

        at junit.framework.Assert.fail(Assert.java:55)

        at junit.framework.Assert.assertTrue(Assert.java:22)

        at junit.framework.Assert.assertFalse(Assert.java:39)

        at junit.framework.Assert.assertFalse(Assert.java:47)

        at junit.framework.TestCase.assertFalse(TestCase.java:219)

        at org.sculptor.maven.plugin.CleanMojoTest.testExecute(CleanMojoTest.java:80)

testDeleteGeneratedFilesAll(org.sculptor.maven.plugin.CleanMojoTest)  Time elapsed: 0.119 sec  <<< FAILURE!

junit.framework.AssertionFailedError: null

        at junit.framework.Assert.fail(Assert.java:55)

        at junit.framework.Assert.assertTrue(Assert.java:22)

        at junit.framework.Assert.assertFalse(Assert.java:39)

        at junit.framework.Assert.assertFalse(Assert.java:47)

        at junit.framework.TestCase.assertFalse(TestCase.java:219)

        at org.sculptor.maven.plugin.CleanMojoTest.testDeleteGeneratedFilesAll(CleanMojoTest.java:46)

... 

[INFO] Everything is up-to-date - no generator run is needed

Tests run: 11, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 1.278 sec <<< FAILURE! - in org.sculptor.maven.plugin.GeneratorMojoTest

testUpdateStatusFile(org.sculptor.maven.plugin.GeneratorMojoTest)  Time elapsed: 0.141 sec  <<< FAILURE!

junit.framework.ComparisonFailure: expected:<[e747f800870423a6c554ae2ec80aeeb6]> but was:<[bd74239dca680cfe53013915533a184d]>

        at junit.framework.Assert.assertEquals(Assert.java:100)

        at junit.framework.Assert.assertEquals(Assert.java:107)

        at junit.framework.TestCase.assertEquals(TestCase.java:269)

        at org.sculptor.maven.plugin.GeneratorMojoTest.testUpdateStatusFile(GeneratorMojoTest.java:105)

testExecuteWithClean(org.sculptor.maven.plugin.GeneratorMojoTest)  Time elapsed: 0.108 sec  <<< FAILURE!

junit.framework.AssertionFailedError: null

        at junit.framework.Assert.fail(Assert.java:55)

        at junit.framework.Assert.assertTrue(Assert.java:22)

        at junit.framework.Assert.assertFalse(Assert.java:39)

        at junit.framework.Assert.assertFalse(Assert.java:47)

        at junit.framework.TestCase.assertFalse(TestCase.java:219)

        at org.sculptor.maven.plugin.GeneratorMojoTest.testExecuteWithClean(GeneratorMojoTest.java:144)

Any idea what could be the issue there?

[El Deffo]

Also, the build was successful on Linux, so I updated the Eclipse plugin as well, but there is an issue with it:
import "classpath:/model-profile.btdesign"

throws errors about not being able to find the imported file (same source works fine in 3.0.6 version of the plugin)

Is there a p2 update site of the snapshot of the plugin somewhere? I couldn't find it on sonatype.

[Torsten Juergeleit]

You probably should convert the generated file's line endings to unix format before the comparison, or vice versa?

Or how about changing your git clients configuration to do this conversion? :-P

This is a known is known issue ARCHETYPE-410 (e.g other OSS project).

 

You're right, I forgot I was mucking around with the setting some time ago :). My bad.

Glad this is fixed.
Btw. don't put autcrlf in .gitconfig because it's handled by the default values of the platform-specific git client (e.g. core.autocrlf=true on Windows).

I'm now stuck at a different phase on Windows

   :

Any idea what could be the issue there?

The CleanMojoTest ones are due to the dreaded "java cannot delete file on windows" issue (google it). It seems that Plexus FileUtils.forceDelete() - which does System.gc() + Thread.sleep(10) - isn't enough here.

And the GeneratorMojoTest ones are due to different line endings on *nix vs. Windows (which leads to different hash codes calculated from file content).

Damn...  I hate Windows...

So meanwhile skip the unit tests of the Maven plugin...

/Torsten

[Torsten Juergeleit]

What do you mean by "throws errors about not being able to find the imported file"?
Are there any exceptions thrown (if yes then having the stacktrace would be great) or are there error markers shown (related to Xtexts global index)?
 

Is there a p2 update site of the snapshot of the plugin somewhere? I couldn't find it on sonatype.

Nope, there's no Eclipse p2 repo with snapshot releases of the Eclipse plugin available. You have to build it locally via "mvn install" in "sculptor-eclipse/eclipse-repository/".

/Torsten