Progress

[Alan & Kim Zimmerman]

Hi all

After lots of discussion around layers and licensing, it looks like the way forward is as follows

a. BIOS layer

ghc-mod stays an AGPL project, and is used for its "awesome sauce" in terms of the BIOS functions that it does so well. This interface is straightforward to use[1], and if a license-constrained user wants to do something else it is also easy to replace, if there is strong control of the operating environment.

b. Plugin layer

A layer providing a point to integrate tools and existing functions, probably including ghci.

c. IDE interfacing layer

This provides a set of logical channels that can be integrated into standard IDEs. The details still need to be worked out, but I would imagine something like a channel for querying information about a project, one for a ghci session, and whatever others are needed.  These logical channels can then run over whatever transport is appropriate to the specific IDE being integrated.

Layers b and c are very fuzzy at this point, and I am pleased to see there has been some discussion on IRC around it. These layers may well live in a single repository, and should probably be hosted in the haskell github organisation, against a license compatible with GHC, such as BSD3.

The most important thing is that the people I have spoken to who have strong views wrt licensing are comfortable with this breakdown.

So, is there support for this approach, and if so can we start making it happen? Probably by bikeshedding a name for the project, unless it retro-fits into ghc-ng or some other.

I am very impressed with the amount of support and goodwill this initiative seems to have, and I hope we can put the initial teething troubles behind us and get down to something that can make all Haskellers proud.

Regards

  Alan

[1] http://alanz.github.io/haskell%20refactorer/2015/10/02/ghc-mod-for-tooling
 

[Michael Sloan]

Hi Alan!

That's a good plan!  I was just writing a rather in-depth email response, when I found out that the haskell-ide repo got created.  Cool!  So, I split out parts of my email into some issues:

https://github.com/haskell/haskell-ide/issues/1 "Do plugins the same way as GHC"
https://github.com/haskell/haskell-ide/issues/2 "Decide on matching an existing IDE protocol or designing a new one"

https://github.com/haskell/haskell-ide/issues/3 "Use Idris's protocol?"

Considering we're still at the drawing board on this, I think it makes sense to carefully consider the approach.  In particular, the approach that ghci-ng and Idris take seem quite nice - both of these tools add IDE integration directly to the REPL.

It seems to me like we'd have most of what we want by supercharging GHCI, ghci-ng style, with:

* Multiple connections sharing the same session.  For example, enable querying for info on a connection, while simultaneously using the repl

* Plugin support, for tooling that uses the GHC API

* IDE protocol in a nice machine-readable format

I'm not hugely attached to this approach, but I think it's highly worth consideration at this stage.  It has all of the following positive points in its favor:

* It seems easier to me to add these things directly to GHCI, rather than getting GHCI working atop ghc-mod.  I am not very familiar with the details, but from the surface level it seems like adapting GHCI to run in someone else's ghc session would be nontrivial.

* GHCI is maintained as part of GHC.  This means it tends to do everything correctly, and we can benefit from applying changes from it and pushing changes to it.

* We can use stack and cabal's ability to properly run ghci to get an IDE session.

I don't see many downsides to this approach.  But then again, maybe I don't know the full degree of awesome-sauce that ghc-mod provides.  Or perhaps it's easier than I think to do ghci atop ghc-mod than I'm thinking.

Thoughts?

-Michael

[Alan & Kim Zimmerman]

I think I am in broad agreement with what you are proposing.

However, at the risk of waving my hands and looking foolish, I think I disagree with you around the ghci/ghc-mod interaction.

My understanding is that everything needs some sort of context. For GHC this has traditionally been provided by cabal, more recently by stack, and in simple cases GHC can work it out directly. In advanced cases specific flags can be passed to GHC to set up a specific environment.

So the awesome sauce that ghc-mod provides is to work out what this context should be for a particular project. This results in knowing the correct flags to pass on the GHC to set up the GHC session, which as I understand it can be a ghci session.

Alan

--
You received this message because you are subscribed to the Google Groups "Haskell IDE implementation discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to haskell-ide...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Bardur Arantsson]

On 10/22/2015 03:02 PM, Alan & Kim Zimmerman wrote:
> I think I am in broad agreement with what you are proposing.
>
> However, at the risk of waving my hands and looking foolish, I think I
> disagree with you around the ghci/ghc-mod interaction.
>
> My understanding is that everything needs some sort of context. For GHC
> this has traditionally been provided by cabal, more recently by stack, and
> in simple cases GHC can work it out directly. In advanced cases specific
> flags can be passed to GHC to set up a specific environment.
>
> So the awesome sauce that ghc-mod provides is to work out what this context
> should be for a particular project. This results in knowing the correct
> flags to pass on the GHC to set up the GHC session, which as I understand
> it can be a ghci session.
>

There's also an issue with ghci being inherently stateful -- this can
lead to hard-to-diagnose bugs where the IDE state gets out of sync with
ghci's state. IMO, ideally, all state (that's not simple caching or
front-loading of cache) really needs to reside in the IDE itself -- the
layer below it should be as stateless as possible. (I realize that e.g.
cabal-install[1] are stateful, but ideally it *shouldn't* be and
hopefully there'll be some upcoming work to move it in that direction.)

Regards,

[1] cabal-install being an example of something that would be
"underneath" the IDE integration layer.

[Thomas Schilling]

> There's also an issue with ghci being inherently stateful -- this can
> lead to hard-to-diagnose bugs where the IDE state gets out of sync with
> ghci's state. IMO, ideally, all state (that's not simple caching or
> front-loading of cache) really needs to reside in the IDE itself -- the
> layer below it should be as stateless as possible. (I realize that e.g.
> cabal-install[1] are stateful, but ideally it *shouldn't* be and
> hopefully there'll be some upcoming work to move it in that direction.)

Yes, that is the concept for Scion (v2). I stopped working on it at
some point because it was so messy to get the necessary information
out of GHC. I did a little GHC API proof of concept with GHC 7.10
recently. That looks more promising. Also, Stack could take care of a
lot of build issues.

The basic idea is:
- Building is not managed by GHC's --make mode. Instead an external
build manager (e.g., based on Shake) invokes compiler in one-shot
mode.
- The compiler is basically GHC + hooks.
- The compiler generates .hi files, optionally .o files, and .meta
files. This contains all meta data which may grow over time: error
messages, parse tree, renamed/typechecked code

A (logically) separate service manages and caches additional
information derived from the meta data. For example,
- all error messages in the whole project (or which ones might be
outdated because a dependency has changed)
- definition sites of all identifiers in the project
- use sites of things
- etc.

The IDE layer would sit on top of this separate service.

Open issues:
- How are projects specified? Ideally stack based.
- How are custom build steps integrated?
- Will this work for GHCi?

I talked about this with Duncan Coutts (who was involved in the
backend that FP Complete IDE work) and they came to basically the same
conclusion. This work has now been open sourced, but I only took a
brief look at it a while ago. Probably worth another closer look.

/ Thomas

[Alan & Kim Zimmerman]

Yes, that is the concept for Scion (v2). I stopped working on it at
some point because it was so messy to get the necessary information
out of GHC. I did a little GHC API proof of concept with GHC 7.10
recently. That looks more promising. Also, Stack could take care of a
lot of build issues.

The project you refer to is https://github.com/nominolo/scion-tools?

Alan

[Thomas Schilling]

> The project you refer to is https://github.com/nominolo/scion-tools?

Yup, that one. :)

[Alan & Kim Zimmerman]

Great, added it to https://github.com/haskell/haskell-ide/blob/master/docs/Inspirations.md

Maybe its time to dust it off again?

On Thu, Oct 22, 2015 at 10:47 PM, 'Thomas Schilling' via Haskell IDE implementation discussion <haske...@googlegroups.com> wrote:

> The project you refer to is https://github.com/nominolo/scion-tools?

Yup, that one. :)