Using SBT inside an IDE

[Sandro Gržičić]

Hello!

So, in short: my GSoC project this summer is the integration of SBT inside ScalaIDE (Eclipse). I'm going to create a custom console for SBT, which will have all sorts of nice features.

Anyway, right now I'm wondering what would be the best way to actually integrate SBT. The only other SBT integrated console that I know of - the SBT Console for IDEA - seems to be using the java.lang.ProcessBuilder class in order to host the SBT process inside the IDEA SBT console. This is obviously doable as well inside Eclipse, but I was looking for a better option. I want to be able to somehow interact with SBT itself, not just attach to its input/output streams.

So, is there any sort of support for doing this - running SBT and programatically hooking into the spawned SBT instance - or should I forget about it and just use it as a dumb process? I'm really hoping against doing the latter. :)

If some minimal support for this exists or is in the making, but requires someone to write proper support and generate a pull request, I'm definitely the person for that as well. I've already gone through SBT's source (it has a very .. interesting .. structure :D) but I haven't been able to find anything useful (besides API calls for instantiating a new Scala REPL, or evaling some code). Maybe there's already mostly complete support for this kind of thing, but I haven't been able to find it yet.

What I'd like to achieve with this is at least some proper output syntax highlighting, preferably with clickable links in stacktraces - most of the hard work will be done by myself I believe, but it would be nice if I could query SBT for the currently loaded classes, etc. Also, as I understand, SBT has a graph-like representation of the currently loaded classes when a project is loaded, so I think that this could be used to generate a visual graph using (for example) JUNG. There are definitely more things that could be accomplished by directly using SBT API calls.

Thanks!

[James Earl Douglas]

This would be very useful - I look forward to it!

[Mark Harrah]

On Thu, 21 Jun 2012 15:06:33 -0700 (PDT)
Sandro Gržičić <serio...@gmail.com> wrote:

> Hello!
>
> So, in short: my GSoC project this summer is the integration of SBT inside
> ScalaIDE (Eclipse). I'm going to create a custom console for SBT, which
> will have all sorts of nice features.
>
> Anyway, right now I'm wondering what would be the best way to actually
> integrate SBT. The only other SBT integrated console that I know of - the
> SBT Console for IDEA - seems to be using the java.lang.ProcessBuilder class
> in order to host the SBT process inside the IDEA SBT console. This is
> obviously doable as well inside Eclipse, but I was looking for a better
> option. I want to be able to somehow interact with SBT itself, not just
> attach to its input/output streams.
>
> So, is there any sort of support for doing this - running SBT and
> programatically hooking into the spawned SBT instance - or should I forget
> about it and just use it as a dumb process? I'm really hoping against doing
> the latter. :)
>
> If some minimal support for this exists or is in the making, but requires
> someone to write proper support and generate a pull request, I'm definitely
> the person for that as well. I've already gone through SBT's source (it has
> a very .. interesting .. structure :D) but I haven't been able to find
> anything useful (besides API calls for instantiating a new Scala REPL, or
> evaling some code). Maybe there's already mostly complete support for this
> kind of thing, but I haven't been able to find it yet.

You will probably want to fork sbt as a separate process to isolate eclipse from sbt, but it doesn't need to be a dumb process. I would write an sbt Command that listens on a local socket for instructions from the main Eclipse code, schedules commands, and provides information back to the Eclipse code. You would start sbt like 'sbt custom-command'. For more on Commands, see:

https://github.com/harrah/xsbt/wiki/Commands

See also the implementation of the interactive prompt (called 'shell') and the '<' command, which can read commands over the loopback interface.

> What I'd like to achieve with this is at least some proper output syntax
> highlighting, preferably with clickable links in stacktraces - most of the
> hard work will be done by myself I believe, but it would be nice if I could
> query SBT for the currently loaded classes, etc. Also, as I understand, SBT
> has a graph-like representation of the currently loaded classes when a
> project is loaded, so I think that this could be used to generate a visual
> graph using (for example) JUNG. There are definitely more things that could
> be accomplished by directly using SBT API calls.

Can you clarify what you mean by currently loaded classes? Do you meant the classes used to define the build or application classes or what?

-Mark

> Thanks!
>
> --
> You received this message because you are subscribed to the Google Groups "simple-build-tool" group.
> To view this discussion on the web visit https://groups.google.com/d/msg/simple-build-tool/-/CAPhQu84ZeQJ.
> To post to this group, send email to simple-b...@googlegroups.com.
> To unsubscribe from this group, send email to simple-build-t...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/simple-build-tool?hl=en.
>

[Sandro Gržičić]

Great, I'll definitely try doing that.

One question though: do you suggest doing this for 0.12 and up, ie. did the API change much between 0.11.3 and 0.12? I mean, can I just code to 0.12's API and expect my code to work with 0.11.3 as well, or did you do major changes to Command and related inner APIs which means such code would be mostly incompatible?

Also, do you agree with me, in coding against 0.12, since it's going to be a released soon and is presumably better than 0.11.x? :)

 

See also the implementation of the interactive prompt (called 'shell') and the '<' command, which can read commands over the loopback interface. 

Ok, I'm going to have to search a bit for that because I have no idea where in the API or in the wiki docs this is explained. Yet. :)

I mean I know about this https://github.com/harrah/xsbt/blob/0.13/main/command/BasicCommands.scala but there's no < there. Anyway.

 

> What I'd like to achieve with this is at least some proper output syntax

 

> highlighting, preferably with clickable links in stacktraces - most of the

 

> hard work will be done by myself I believe, but it would be nice if I could
> query SBT for the currently loaded classes, etc. Also, as I understand, SBT
> has a graph-like representation of the currently loaded classes when a
> project is loaded, so I think that this could be used to generate a visual
> graph using (for example) JUNG. There are definitely more things that could
> be accomplished by directly using SBT API calls.

Can you clarify what you mean by currently loaded classes?  Do you meant the classes used to define the build or application classes or what?

I mean user's application classes. This shouldn't be too difficult to retreive from SBT? I mean, it tracks compiled classes because of incremental compilation, so it should know about all classes inside a project, right? This can be used for ie. highlighting class names in stacktraces and building visual class dependency graphs, I think..? 

Thanks.

[Mark Harrah]

On Fri, 22 Jun 2012 03:10:51 -0700 (PDT)

I don't think Command changed much. Some APIs did change, although we try to deprecate ones used by plugins and builds.

> Also, do you agree with me, in coding against 0.12, since it's going to be
> a released soon and is presumably better than 0.11.x? :)

I think 0.12 is a good target if you have to pick one, yes.

> > See also the implementation of the interactive prompt (called 'shell') and
> > the '<' command, which can read commands over the loopback interface.
> >
>
> Ok, I'm going to have to search a bit for that because I have no idea where
> in the API or in the wiki docs this is explained. Yet. :)
> I mean I know about this
> https://github.com/harrah/xsbt/blob/0.13/main/command/BasicCommands.scala
> but there's no < there. Anyway.

Commands are split between the implementation and the name/description/help. Names and help are in BasicCommandStrings.scala. The < command implementation is here:

https://github.com/harrah/xsbt/blob/0.13/main/command/BasicCommands.scala#L127

> > > What I'd like to achieve with this is at least some proper output syntax
> > > highlighting, preferably with clickable links in stacktraces - most of
> > the
> > > hard work will be done by myself I believe, but it would be nice if I
> > could
> > > query SBT for the currently loaded classes, etc. Also, as I understand,
> > SBT
> > > has a graph-like representation of the currently loaded classes when a
> > > project is loaded, so I think that this could be used to generate a
> > visual
> > > graph using (for example) JUNG. There are definitely more things that
> > could
> > > be accomplished by directly using SBT API calls.
> >
> > Can you clarify what you mean by currently loaded classes? Do you meant
> > the classes used to define the build or application classes or what?
>
> I mean user's application classes. This shouldn't be too difficult to
> retreive from SBT? I mean, it tracks compiled classes because of
> incremental compilation, so it should know about all classes inside a
> project, right? This can be used for ie. highlighting class names in
> stacktraces and building visual class dependency graphs, I think..?

Yes, this structure is provided as the result of 'compile'. See:

https://github.com/harrah/xsbt/blob/0.13/compile/inc/Analysis.scala

-Mark

> Thanks.
>

[Lukas Rytz]

Hi Sandro,

this programmatic interface to a running sbt instance sounds like exactly what i'm looking

for in a different context - is there already an implementation for it by now?

Or did you chose an alternative solution?

Thanks,

Lukas

On Friday, June 22, 2012 3:32:26 AM UTC+2, Mark Harrah wrote:

[Sandro Gržičić]

Hey Lukas,

for now I've decided to go with the "dumb" approach, filtering SBT's output and sending it input in response to the user. It works well for now, for example if I want SBT to exit I just send it "exit". :)

An "API" plugin for SBT would be great, but great effort would probably be expended in making it compatible across all the different versions users would use SBT with (probably from 0.11.2 to the latest nightly). I'm not dismissing this out of hand, but it probably won't come for some time (unless somebody has a great and specific idea on how to do this - then I'm up for it :)).

BTW, SBTConsole is in pretty much feature-complete and hopefully bug-ridden state right now, so feel free to check it out: https://github.com/SandroGrzicic/sbtconsole

[Sandro Gržičić]

Hey Iulian,

I hope it's OK that I used your code as a base for SBT Console, I hope you like how it evolved. :)

Sandro

On Monday, June 25, 2012 6:02:58 PM UTC+2, Iulian Dragoș wrote:

On Friday, June 22, 2012 12:06:33 AM UTC+2, Sandro Gržičić wrote:

Hello!

So, in short: my GSoC project this summer is the integration of SBT inside ScalaIDE (Eclipse). I'm going to create a custom console for SBT, which will have all sorts of nice features.

This will be extremely useful. I finally spent a bit of time bringing up to date an old branch I have described here:

https://groups.google.com/forum/?fromgroups#!searchin/scala-ide-dev/from:me$20sbt$20console/scala-ide-dev/PyjRg4wPjLs/INNbHaKUNV4J

The code is here: https://github.com/dragos/scala-ide/tree/feature/sbt-console

It runs fine with the current master, and it does the 'dumb I/O' redirection. It's pretty limited, but still has hyperlinking for errors and simple color highlighting. May be a source of inspiration.

 

Anyway, right now I'm wondering what would be the best way to actually integrate SBT. The only other SBT integrated console that I know of - the SBT Console for IDEA - seems to be using the java.lang.ProcessBuilder class in order to host the SBT process inside the IDEA SBT console. This is obviously doable as well inside Eclipse, but I was looking for a better option. I want to be able to somehow interact with SBT itself, not just attach to its input/output streams.

So, is there any sort of support for doing this - running SBT and programatically hooking into the spawned SBT instance - or should I forget about it and just use it as a dumb process? I'm really hoping against doing the latter. :)

A dumb process has the advantage that it will work with any version of Sbt that the user may have. Otherwise, your plugin (or command) needs to be compatible with the version of Sbt that you're running. Is that going to be configurable, or shipped with the IDE? If shipped, the projects need to adhere to that version as well. If not, your command needs to be compatible. If the API is stable enough, this may be a non-issue.

cheers,

iulian