new build process merged in trunk

[Luc Bourlier]

Hi all,

As you may have noticed, I spent some time cleaning up, updating and refactoring the build. And it was merged in trunk today.

There are 2 main changes visible from the outside. First, the build relies only on maven and maven profile, no bash scripts are needed anymore. And the sdt part of the build is using the Scala bundles (library, compiler, sbt) create locally (it was using the ones created during the nightly build and deployed on download.scala-ide.org).

I will update the doc on the website, but here's a quick run down of the new structure so you can continue to play with Scala IDE during the weekend.

There are 4 parts:

- the root pom is the parent pom. It contains version numbers, default configurations and the profiles

- org.scala-ide.build-toolchain creates the osgi bundles for the Scala library, compiler and sbt

- org.scala-ide.toolchain.update-site creates an update site containing the bundles created by the build-toolchain (needed for the nightly builds)

- org.scala-ide.sdt.build compiles, tests and creates the update site for Scala IDE

build-all.sh is simply a helper script that calls maven on the different parts.

Run ./build-all.sh clean install once to get every initialized, then you can run maven from any subproject as needed.

Add -P scala-2.10.x to build Scala IDE for Scala 2.10. Iulian did the work so it is compatible with the latest 2.10.x code.

The generated update site contains a new section called 'dev support'. Right now it contains org.scala-ide.sdt.core.test so other Scala Eclipse plugins can use the testing framework it contains.

I hope you'll like it,

Luc

[Eugene Burmako]

Great news, I just wanted to do some experiments.

The changes that Iulian made, do they make Scala IDE work with the new reflection?

[Luc Bourlier]

It compiles and the tests run fine with the latest 2.10.0-SNAPSHOT. We didn't try it more than that.

One thing missing is to automatically add the new reflect jar to a project with the compiler jar. But if you add it manually (like from the m2 repo), it should work fine.

Luc

[Mirko Stocker]

Hi,

On Fri, Jun 8, 2012 at 10:52 PM, Luc Bourlier <luc.bo...@typesafe.com> wrote:
> Run ./build-all.sh clean install once to get every initialized

That doesn't work for me; something seems to be wrong with a
"${osgi.version.number}":

[INFO] Resolving dependencies of MavenProject:
org.scala-ide:org.scala-ide.sdt.aspects:2.1.0-SNAPSHOT @
/home/misto/scala/2.9/scala-ide/org.scala-ide.sdt.aspects/pom.xml
[INFO] Resolving class path of MavenProject:
org.scala-ide:org.scala-ide.sdt.aspects:2.1.0-SNAPSHOT @
/home/misto/scala/2.9/scala-ide/org.scala-ide.sdt.aspects/pom.xml
[INFO] Computing target platform for MavenProject:
org.scala-ide:org.scala-ide.sdt.core:2.1.0-SNAPSHOT @
/home/misto/scala/2.9/scala-ide/org.scala-ide.sdt.core/pom.xml
[WARNING] invalid version "${osgi.version.number}": non-numeric "${osgi"
java.lang.IllegalArgumentException: invalid version
"${osgi.version.number}": non-numeric "${osgi"
at org.osgi.framework.Version.parseInt(Version.java:170)
at org.osgi.framework.Version.<init>(Version.java:126)
at org.osgi.framework.Version.parseVersion(Version.java:233)
at org.eclipse.osgi.internal.resolver.StateBuilder.addExportPackages(StateBuilder.java:342)
at org.eclipse.osgi.internal.resolver.StateBuilder.createExportPackages(StateBuilder.java:324)
at org.eclipse.osgi.internal.resolver.StateBuilder.createBundleDescription(StateBuilder.java:114)
at org.eclipse.osgi.internal.resolver.StateObjectFactoryImpl.createBundleDescription(StateObjectFactoryImpl.java:33)
at org.eclipse.equinox.p2.publisher.eclipse.BundlesAction.createBundleDescription(BundlesAction.java:543)
at org.eclipse.equinox.p2.publisher.eclipse.BundlesAction.createBundleDescription(BundlesAction.java:558)
at org.eclipse.equinox.p2.publisher.eclipse.BundlesAction.getBundleDescriptions(BundlesAction.java:858)
at org.eclipse.equinox.p2.publisher.eclipse.BundlesAction.perform(BundlesAction.java:669)
at org.eclipse.equinox.p2.publisher.Publisher$ArtifactProcess.run(Publisher.java:207)
at org.eclipse.equinox.p2.repository.artifact.spi.AbstractArtifactRepository.executeBatch(AbstractArtifactRepository.java:187)
at org.eclipse.equinox.p2.publisher.Publisher.publish(Publisher.java:231)
at org.eclipse.tycho.p2.target.TargetPlatformBundlePublisher$PublisherRun.executePublisherAction(TargetPlatformBundlePublisher.java:165)
at org.eclipse.tycho.p2.target.TargetPlatformBundlePublisher$PublisherRun.execute(TargetPlatformBundlePublisher.java:146)
at org.eclipse.tycho.p2.target.TargetPlatformBundlePublisher.attemptToPublishBundle(TargetPlatformBundlePublisher.java:95)
at org.eclipse.tycho.p2.target.TargetPlatformBuilderImpl.publishAndAddArtifactIfBundleArtifact(TargetPlatformBuilderImpl.java:197)
at org.eclipse.tycho.p2.resolver.PomDependencyProcessor.addPomDependenciesToResolutionContext(PomDependencyProcessor.java:84)
at org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.addPomDependenciesToTargetPlatform(P2TargetPlatformResolver.java:321)
at org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.computeTargetPlatform(P2TargetPlatformResolver.java:214)
at org.eclipse.tycho.core.resolver.DefaultTychoDependencyResolver.resolveProject(DefaultTychoDependencyResolver.java:89)
at org.eclipse.tycho.core.maven.TychoMavenLifecycleParticipant.afterProjectsRead(TychoMavenLifecycleParticipant.java:64)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:274)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:537)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:196)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:141)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:290)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:230)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:352)
Caused by: java.lang.NumberFormatException: For input string: "${osgi"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
at java.lang.Integer.parseInt(Integer.java:449)
at java.lang.Integer.parseInt(Integer.java:499)
at org.osgi.framework.Version.parseInt(Version.java:168)
... 35 more

I also deleted my .m2/repository, but that didn't help either. Any ideas?

Cheers,

Mirko

--
Mirko Stocker | m...@misto.ch
Work: http://ifs.hsr.ch | http://infoq.com
Personal: http://misto.ch | http://twitter.com/m_st

[Luc Bourlier]

It is not an error I encountered before.

It looks like it is doing dependency resolution for sdt.core, which shouldn't not see any '${osgi.version.number}'.

This property is used to build the bundles for the library and the compiler. Can you check if you see it in org.scala-ide.scala.library/target/META-INF/MANIFEST.MF or org.scala-ide.scala.compiler/target/META-INF/MANIFEST.MF, in the Bundle-Version section?

Also check if you have org.scala-ide.scala.library/target/library.properties and org.scala-ide.scala.library/target/compiler.properties files. They should contain a osgi.version.number entry.

Luc

[Mirko Stocker]

Hi Luc,

On Wed, Jun 13, 2012 at 2:28 PM, Luc Bourlier <luc.bo...@typesafe.com> wrote:
> This property is used to build the bundles for the library and the compiler.
> Can you check if you see it in
> org.scala-ide.scala.library/target/META-INF/MANIFEST.MF or
> org.scala-ide.scala.compiler/target/META-INF/MANIFEST.MF, in the
> Bundle-Version section?

There it says Bundle-Version: 0

> Also check if you have org.scala-ide.scala.library/target/library.properties
> and org.scala-ide.scala.library/target/compiler.properties files. They
> should contain a osgi.version.number entry.

Hm, nope, they only contain these two lines:

version.number=2.9.1.final
copyright.string=Copyright 2002-2011, LAMP/EPFL

I wonder what I'm doing wrong, my system:

Apache Maven 3.0.4 (r1232337; 2012-01-17 09:44:56+0100)
Maven home: /usr/share/maven-bin-3.0
Java version: 1.6.0_32, vendor: Sun Microsystems Inc.
Java home: /opt/sun-jdk-1.6.0.32/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.4.0-gentoo", arch: "amd64", family: "unix"

[Luc Bourlier]

You appear to be pulling the 2.9.1.final version of Scala. You should be getting the 2.9.3-SNAPSHOT version, which contains the right property.

You may have a profile defining a different version of Scala, activated from one of your settings file: ~/.m2/settings.xml or $MAVEN_HOME/conf/settings.xml. Look for an activeProfiles section.

You can try using ./build-all.sh -P scala-2.9.x clean install to force to use only the profile define in the Scala IDE build.

[Mirko Stocker]

On Wednesday 13 June 2012 06:09:52 Luc Bourlier wrote:
> You appear to be pulling the 2.9.1.final version of Scala. You should be
> getting the 2.9.3-SNAPSHOT version, which contains the right property.
>
> You may have a profile defining a different version of Scala, activated from
> one of your settings file: ~/.m2/settings.xml or
> $MAVEN_HOME/conf/settings.xml. Look for an activeProfiles section.

Ah indeed, that solved it :-)

Thanks!

[Eric Molitor]

Finally had a break from other work and merged the new build logic in. Brilliant stuff and much better than my half assed attempt!

- Eric

[Eric Molitor]

... and quickly followed by a new question/request. The nightly is now failing for platform/juno (makes sense https://jenkins.scala-ide.org:8496/jenkins/view/Scala%20IDE%202.1-29%20Juno/job/scala-ide-juno-nightly-master-29/127/console) but how/who can update the Jenkins config?

- Eric

On Friday, 8 June 2012 21:52:28 UTC+1, Luc Bourlier wrote:

[Mirco Dotta]

Should be fixed now.

http://j.mp/Mcy57x 

-- Mirco

[Antoras]

I got some errors while executing the new build process:

$ ./build-all.sh -e -P scala-2.10.x clean install

In org.scala-ide.sdt.core.tests/target/surefire-reports/ there are some exceptions logged. Most are of the form

java.lang.NoSuchMethodError: scala.collection.Seq.toArray(Lscala/reflect/ArrayTag;)Ljava/lang/Object;

It seems the new TypeTags can't be found, why not?

Full stack-trace:

java.lang.NoSuchMethodError: scala.collection.Seq.toArray(Lscala/reflect/ArrayTag;)Ljava/lang/Object;
    at sbinary.Generic$$anon$17.<init>(generic.scala:371)
    at sbinary.Generic$class.asUnion(generic.scala:370)
    at sbinary.DefaultProtocol$.asUnion(defaultprotocol.scala:4)
    at sbt.inc.AnalysisFormats$.stampFormat(AnalysisFormats.scala:53)
    at scala.tools.eclipse.buildmanager.sbtintegration.EclipseSbtBuildManager.<init>(EclipseSbtBuildManager.scala:204)
    at scala.tools.eclipse.ScalaProject.buildManager(ScalaProject.scala:535)
    at scala.tools.eclipse.ScalaProject.prepareBuild(ScalaProject.scala:547)
    at scala.tools.eclipse.ScalaBuilder.build(ScalaBuilder.scala:54)
    at org.eclipse.core.internal.events.BuildManager$2.run(BuildManager.java:728)
    at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
    at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:199)
    at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:239)
    at org.eclipse.core.internal.events.BuildManager$1.run(BuildManager.java:292)
    at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
    at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:295)
    at org.eclipse.core.internal.events.BuildManager.basicBuildLoop(BuildManager.java:351)
    at org.eclipse.core.internal.events.BuildManager.build(BuildManager.java:374)
    at org.eclipse.core.internal.resources.Workspace.buildInternal(Workspace.java:513)
    at org.eclipse.core.internal.resources.Workspace.build(Workspace.java:422)
    at scala.tools.eclipse.sbtbuilder.ProjectDependenciesTest.transitive_dep_with_error_stops_build(ProjectDependenciesTest.scala:67)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
    at org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored(BlockJUnit4ClassRunner.java:79)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:71)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:49)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
    at org.junit.runners.Suite.runChild(Suite.java:128)
    at org.junit.runners.Suite.runChild(Suite.java:24)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
    at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53)
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123)
    at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:104)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164)
    at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110)
    at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:175)
    at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcess(SurefireStarter.java:123)
    at org.eclipse.tycho.surefire.osgibooter.OsgiSurefireBooter.run(OsgiSurefireBooter.java:84)
    at org.eclipse.tycho.surefire.osgibooter.HeadlessTestApplication.run(HeadlessTestApplication.java:21)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at org.eclipse.equinox.internal.app.EclipseAppContainer.callMethodWithException(EclipseAppContainer.java:587)
    at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:198)
    at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
    at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
    at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:344)
    at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:622)
    at org.eclipse.equinox.launcher.Main.basicRun(Main.java:577)
    at org.eclipse.equinox.launcher.Main.run(Main.java:1410)
    at org.eclipse.equinox.launcher.Main.main(Main.java:1386)

[Mirco Dotta]

I think I've seen this as well, and my first attempt has been to wipe out my local maven repo (well, not all of it, just ~/.m2/repository/org/scala-*)

I just compiled master against 2.10, and it's working fine for me.

-- Mirco

---------------

Mirco Dotta

Typesafe - The software stack for applications that scale

PSE-D, 1015 Lausanne, Switzerland

Work: +41 (0)21 691 49 65

Twitter: @mircodotta

[iulian dragos]

On Wed, Jun 20, 2012 at 3:05 PM, Mirco Dotta <mirco...@typesafe.com> wrote:
> I think I've seen this as well, and my first attempt has been to wipe out my
> local maven repo (well, not all of it, just ~/.m2/repository/org/scala-*)

Indeed, there's a stale sbinary_2.10.0-SNAPSHOT in your .m2. You can
remove just .m2/repository/org/scala-tools/sbinary/sbinary_2.10.0-SNAPSHOT/

(we need to republish sbinary whenever the 2.10 compiler has a
binary-incompatible update)

iulian

--
« Je déteste la montagne, ça cache le paysage »
Alphonse Allais

[Antoras]

Ok, thanks. After deleting the file all works fine.