sbt and eclipse

[francisco treacy]

Hi,

I have occasionally used sbt for the past few months - and I love its
speed and doing my configuration in Scala. Today I wanted to
completely migrate a Maven build to sbt, but I had some trouble (had a
quick exchange @frank06 with @n8han on twitter).

Basically... can I use sbt like I do with the interpreter and -at the
same time- keep my Maven classpath container within Eclipse?
(Automatically keeping dependencies in the classpath is arguably the
greatest advantage of Maven plugins for Eclipse)

Obviously I don't want to maintain an sbt project *and* a pom.xml just
for the plugin to work. I tried out the make-pom action but I guess
this is not intended for this purpose (builds jar first, then creates
a very basic pom).

How would you go about this in a simple fashion? Thanks for any
pointers.

Cheers,

Francisco

[Nathan Hamblen]

How would you go about this in a simple fashion? Thanks for any

pointers.

My first thought is to export .classpath as Buildr does,  and I just started to hack this out:

http://technically.us/git?p=dispatch.git;a=commitdiff;h=840861f48198dd7309f5a67ef1e4840bfbd536ed

But as sbt uses Ivy there are more options:

1) You write an ivy.xml for your project, to be read by both sbt and IvyDE. You aren't duplicating anything; sbt does not even require a project spec to run.

2) Use a pom.xml for your project, but only to define the dependencies. sbt can understand it as could your current Eclipse plugins for Maven.

3) Define your dependencies from sbt, but export a pom.xml or ivy.xml to be used by an eclipse plugin. You would probably need to write or own action to get these in the right place for the plug-ins, unless support for that scenario is added to sbt.

4) Have native eclipse metadata exported from sbt, through your own project spec or eventually by a built-in sbt action.

I like 4 the best because I try to limit my exposure to Eclipse plugins; I stuck to that workflow with Maven, Buildr, and would do the same with sbt if I used Scala IDE (itself... an unstable Eclipse plugin!). I'm glad to help write an eclipse action that generates the metadata, maybe I will use it someday myself. But if you're happy with the Maven plugin, Frank, you may want to pursue one of the other options.

Also wondering what Mark's thoughts are on IDE integration.

Nathan

[Josh Suereth]

It really sounds like you need some kind of eclipse plugin that
updates itself from sbt. Not an easy thing to do. I could offer some
help here, but not a lot of time currently.

Sent from my iPhone

On Jun 14, 2009, at 6:14 PM, francisco treacy <francisc...@gmail.com

[Josh Suereth]

I must recommend against 4.  The maven-eclipse-plugin has caused no end of grief for chosing this option!  The big issue here is that you don't give eclipse plugins a hook to respond to changes, and therefore take the brunt of maintenance of every plugin's meta-data on your shoulders. 

Sent from my iPhone

[Mark Harrah]

Not knowing anything about native eclipse metadata other than what
I've seen in your action, I would recommend 4 as well. It seems like
it should be the best way to be consistent between sbt and eclipse.
Except for the Scala jars, compileClasspath is exactly what does on
the project's classpath at compile time. This isn't necessarily the
case for the other options (manually managed dependencies, for
example).

> Also wondering what Mark's thoughts are on IDE integration.

Since I don't use an IDE, I would defer anything IDE-specific to
others. I would want to include in sbt the basic support like the
eclipseClasspath action you wrote. Anything much further would be a
separate project.

-Mark

[Mark Harrah]

Oh, bummer.

[Josh Suereth]

Actually, If you supported a full OSGi build environment for Java + Scala, I would actually think about improving SBT to the point of using it to develop Eclipse Plugins.

Needed functionality before that would happen:

1) OSGi support for scalac.  Realistically I doubt my plugin will get finished before someone else solves that problem
2) OSGi aware artifact resolution in SBT.  I doubt Ivy does this.  Maven 3.0 is frantically working towards it (Tycho is a neato prerelease that works for Java Eclipse Plugins).
3) p2 metadata creation inside SBT.  This might be easier, as you can defer to jars in an eclipse directory.


Anyway, my take on writing eclipse plugins is that they're fairly easy to develop and distribute.  The real pain comes in integrating diverse plugins together.  It's almost as if people don't expect their code to be used in situations they didn't anticipate... (what kind of developers are they!!!)


Anyway, eclipse metadata as an interim workaround is probably fine, just expect lots of requests/complaints around that feature.  The eclipse ecosystem is big enough, it's tough to appease all users, and updating metadata can require delving into many different plugins.

- Josh

[Josh Suereth]

If you guys really do want to go with number 4, this is what I recommend:

1) Stick to just generating .classpath.  In fact, modifying .classpath to add missing entries would be better. (I define "missing" loosely).
2) Make sure you do not add the scala-library to the classpath.  You should use the scala classpath container contributed by the Scala IDE Tools for Eclipse.
3) You should issue a warning/failure if the SBT project uses a different version of scala from the eclipse plugin
4) Ordering (used to?) matter for where the Scala Classpath container shows up in the .classpath file.  I believe it needs to be the last thing on the list before the JDT.
5) Try to figure out which other plugins provide "Classpath Containers" and make sure you use these where appropriate (i.e. if the plugin is installed, and the versions line up, etc.)
6) Prevent yourself from walking down the path of generating ..project and .metadata/* files.  Only pain can come from that much reverse engineering.


- Josh

P.S. I recommend a simple "integration of integrations" SBT plugin that lets people who care about Plugin X (e.g. AspectJ) wire in knowledge to the SBT eclipse plugin.   This seems to be the eclipse way in a very large ecosystem.

[Nathan Hamblen]

If you guys really do want to go with number 4, this is what I recommend:

You changed my mind already. As an Eclipse plugin abstainer I haven't hit the problems you describe, but I see that any builder would be on the hook for generating the .classpath that causes problems for some random plugin, and that would be no fun. Seems more prudent to generate a pom.xml or ivy.xml for the mature Maven/Ivy plugins that already exist, with a marker so that sbt itself does not try to process it. If IvyDE works well enough and doesn't require a build.xml I think ivy.xml is a better fit, otherwise a pom.xml should work.

Nathan

[francisco treacy]

Thanks all for your valuable input.

> Seems more prudent to generate a pom.xml or
> ivy.xml for the mature Maven/Ivy plugins that already exist, with a marker so that sbt itself does not try to process it.

I think so. This is what I was wondering about the "make-pom" action.
As you say it should generate a marker - and replace whatever pom
there is in the root of the project (I don't want to dig now into the
multi-project scenario).

> If IvyDE works well enough
> and doesn't require a build.xml I think ivy.xml is a better fit, otherwise a
> pom.xml should work.

Although I am not a big fan, I would go the Maven way. (I'm aware of
Ivy's advantages but never used IvyDE - just guessing now with Eclipse
IAM Maven support is getting better and better - let alone the fact
that it's more popular).

Francisco


Sent from Amsterdam, NH, Netherlands


> >
>

[Nathan Hamblen]

Although I am not a big fan, I would go the Maven way.  (I'm aware of
Ivy's advantages but never used IvyDE - just guessing now with Eclipse
IAM Maven support is getting better and better - let alone the fact
that it's more popular).

Francisco

I did some testing of both IAM and IvyDE on Eclipse 3.5, using pom.xml and ivy.xml files published by sbt (copied them from ~/.ivy/local/...). I was unable to get Maven Dependency Management fully enabled for a Scala project with a pom.xml. It did download dependencies but it didn't add them to the classpath in the Package Explorer or teh actual classpath. With IvyDE I was able to right-click on my ivy.xml, Add Ivy Library, see all the jars under a new ivy.xml library, and they were made available to the Scala compiler.

I do think this Ivy approach is simpler, and appropriately focused on dependency management. The extra things that IAM does (running tests, publishing) would not be possible anyway with a pom.xml generated by sbt; instead you're going to be running those through an adjacent sbt console. And with IvyDE you have the considerable benefit of sharing a dependency cache with sbt. Finally, it can bind to any xml file so we can just output ivy-mirror.xml or something and not have to worry about sbt trying to read it in as a set of external dependencies.

So IvyDE is my recommendation for an sbt dependency bridge into Eclipse. But, the solution is pretty much the same in either case so maybe it's easiest to support both and let people decide for themselves? As for what that solution is exactly, I can't yet say. I went looking into the sbt source but it seems to be one of those things where Ivy does the heavy lifting, so, it maybe we ask Ivy to publish into target/ and then copy the desired xml file into the project base.

Nathan

[francisco treacy]

Thanks for trying it out. You're right, at the end of the day the only
interesting thing is dependency management. I personally don't use the
rest (running tests, install, etc) in the context menu but on an
external console.
So if IvyDE integrates better with sbt then we should stick to it for
the moment.

Does sbt provide any hooks to generate this new file to feed IvyDE?
Perhaps with the newly introduced plugin architecture?

Francisco



2009/6/30 Nathan Hamblen <nat...@technically.us>:

[Mark Harrah]

Nathan,

Thanks for looking at this...

Is 'deliver-local' what you are looking for? It should generate an
Ivy file in the output directory with name ivy-<project-version>.xml.
It is the step prior to publishing.

-Mark

[Nathan Hamblen]

Is 'deliver-local' what you are looking for?  It should generate an
Ivy file in the output directory with name ivy-<project-version>.xml.
It is the step prior to publishing.

Aw, man. That makes it too easy. Then you just have to copy it somewhere fairly permanent for IvyDE:

--

  val ivyOutput = outputPath / ("ivy-" + version + ".xml")

  val ivyExport = info.projectPath / "ivy-export.xml"

  

  lazy val exportIvy = fileTask(ivyExport from ivyOutput) {

    FileUtilities.copyFile(ivyOutput.asFile, ivyExport.asFile, log)

  } dependsOn deliverLocal

--

(This goes in a project definition, then run `sbt export-ivy`.)

I'm sure that can be improved. The ivyOutput value must be defined somewhere already but I can't seem to find it. And maybe there's a shortcut for a single-file copy task, although, I don't mind writing file tasks at all!

Nathan

[Mark Harrah]

There is no ivyOutput because Ivy works off of patterns, so there is a
deliveredIvyPattern that tells Ivy to write the Ivy file to outputPath
/ "[artifact]-[version].[ext]" and for the Ivy file, [artifact]=ivy
and so forth.

There is a copyTask(sources, targetDirectory), but it always performs
the copy (it is not a fileTask) and you don't specify the output file.
Also, you can just do path("ivy-export.xml") if you like.

-Mark

[francisco treacy]

Beautiful!

I just tried it out and works great (so much smoother without all the
Maven machinery).

To see it by example, follow these steps:

1. Install IvyDE - see http://ant.apache.org/ivy/ivyde/

2. Make your sbt project look like {

//val httpclient = "org.apache.httpcomponents" % "httpclient" % "4.0-beta2"
val wicket = "org.apache.wicket" % "wicket" % "1.4-rc6"
val wicketExt = "org.apache.wicket" % "wicket-extensions" % "1.4-rc6"

(...)

lazy val exportIvy = copyTask(path("ivy-export.xml"), path("."))
dependsOn deliverLocal

}

3. Comment/ uncomment dependencies in the project definition; after each change:
a) Issue 'reload' and 'export-ivy' in the sbt console
b) Update Eclipse classpath container:
i) To setup IvyDE for the first time: Go to project properties ->
Java Build Path. Make sure you use the same version of Scala in both
Eclipse and sbt. Then Add Library... -> IvyDE managed dependencies ->
Ivy file: target/ivy-1.0.xml. Select appropriate configurations.
ii) To update once it's set up: right-click on the Ivy container
in your project (should look like target/ivy-1.0.xml [compile]) and
click on the bottommost Refresh (ie not the one with F5 but just below
Resolve)


I am going to carry on playing with it - to prove it right.

Thanks,

Francisco



2009/7/2 Mark Harrah <dmha...@gmail.com>:

[francisco treacy]

Hmmm... I got a bit confused - it's actually even simpler.

You can drop the lazy val exportIvy and instead of export-ivy just
call deliver-local.

Francisco


2009/7/5 francisco treacy <francisc...@gmail.com>:

[Mark Harrah]

I'm glad to hear this is working for you. If you repost your steps using
deliver-local, I'll add it to the wiki. Or, I can give you permissions to
add it yourself if you'd like.

Thanks,
Mark

[francisco treacy]

Hi Mark,

I am still testing (now with a webapp) and trying to sort out problems
/ make sure it works fine. We should definitely document it.

Let me finish and I'll post a complete version so you can upload it to
the wiki. Don't hesitate to bug me if I ever don't get back to you.

Thanks,

Francisco


2009/7/5 Mark Harrah <dmha...@gmail.com>:

[francisco treacy]

This should be good enough to get started:

1. Install IvyDE - see http://ant.apache.org/ivy/ivyde/

2. Setup your sbt project (dependencies, repos, settings, ...)

3. Comment/ uncomment dependencies in your project definition; after
each change:
a) Issue 'reload' and 'deliver-local' in the sbt console
b) Update Eclipse classpath container like so:

i) To setup IvyDE for the first time: Go to project properties ->
Java Build Path. Make sure you use the same version of Scala in both
Eclipse and sbt. Then Add Library... -> IvyDE managed dependencies ->
Ivy file: target/ivy-1.0.xml. Select appropriate configurations.
ii) To update once it's set up: right-click on the Ivy container
in your project (should look like target/ivy-1.0.xml [compile]) and
click on the bottommost Refresh (ie not the one with F5 but just below
Resolve)

Note: A glitch is that the "1.0" part of the file is your project
version - so of course YMMV, especially when incrementing your minor
version (i.e. after 'release' or any other call to
'increment-version'). What basically Nathan's approach aimed to solve.

Question: is there any way of getting the resolved ivy.xml sometime
before than "deliver-local"? Like, why do I need to pass through the
'package' phase?

Btw, I've been playing with a webapp in Eclipse and sbt ... I realize
this is an awesome piece of software. Thanks.
Hopefully at some point I will elaborate on these experiences.

Cheers,

[Mark Harrah]

Thanks! Google wiki syntax is somewhat limiting, so I reformatted it
a bit, but your writeup is now at [1]. Further comments inline...

On 7/6/09, francisco treacy <francisc...@gmail.com> wrote:
>
> This should be good enough to get started:
>
> 1. Install IvyDE - see http://ant.apache.org/ivy/ivyde/
>
> 2. Setup your sbt project (dependencies, repos, settings, ...)
>
> 3. Comment/ uncomment dependencies in your project definition; after
> each change:
> a) Issue 'reload' and 'deliver-local' in the sbt console
> b) Update Eclipse classpath container like so:
> i) To setup IvyDE for the first time: Go to project properties ->
> Java Build Path. Make sure you use the same version of Scala in both
> Eclipse and sbt. Then Add Library... -> IvyDE managed dependencies ->
> Ivy file: target/ivy-1.0.xml. Select appropriate configurations.
> ii) To update once it's set up: right-click on the Ivy container
> in your project (should look like target/ivy-1.0.xml [compile]) and
> click on the bottommost Refresh (ie not the one with F5 but just below
> Resolve)
>
> Note: A glitch is that the "1.0" part of the file is your project
> version - so of course YMMV, especially when incrementing your minor
> version (i.e. after 'release' or any other call to
> 'increment-version'). What basically Nathan's approach aimed to solve.
>
> Question: is there any way of getting the resolved ivy.xml sometime
> before than "deliver-local"? Like, why do I need to pass through the
> 'package' phase?

Yes, there should really be something specific to your requirements.
'package' is needed for the usual case, but not for synchronizing with
an IDE. Try the following:

def idePublishConfiguration = new DefaultPublishConfiguration("local",
"release") {
override protected def deliveredPathPattern = outputPath / "[artifact].[ext]"
}
lazy val deliverIde = deliverTask(idePublishConfiguration, deliverOptions)

The first part defines a configuration that delivers the Ivy file
without a version. The second part uses that configuration for a
custom deliver action. Let me know how this works out.

> Btw, I've been playing with a webapp in Eclipse and sbt ... I realize
> this is an awesome piece of software. Thanks.
> Hopefully at some point I will elaborate on these experiences.

Great, thanks for what you have already contributed!

-Mark

[1] http://code.google.com/p/simple-build-tool/wiki/IntegrationSupport