[Haskell-cafe] haskell-src-exts maintenance

[Matthew Pickering]

Dear List,

There are a large number of untriaged pull requests and issues
currently open on the issue tracker for haskell-src-exts. The tool is
widely used by tool writers but the last commit to the project is now
over 6 months ago.

I think it is important to get this work merged and update the parser
to support modern language features introduced in GHC 7.8 and 7.10. I
would personally be capable of helping with this effort but before
doing so I would appreciate a comment from the current maintainers as
to the current status of the project.

So,

1. Does anyone know if Niklas or Peter plan to return to the project?
2. Is there anyone else who is interested in updating the HSE parser?

Matt
_______________________________________________
Haskell-Cafe mailing list
Haskel...@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe

[Christian Marie]

On Mon, Jun 29, 2015 at 01:13:54PM +0100, Matthew Pickering wrote:
> There are a large number of untriaged pull requests and issues
> currently open on the issue tracker for haskell-src-exts. The tool is
> widely used by tool writers but the last commit to the project is now
> over 6 months ago.

I was about to post this exact message after searching around for a good hour.

> 1. Does anyone know if Niklas or Peter plan to return to the project?

I didn't contact them directly. I'm not so sure if Nicklas is active at all.

> 2. Is there anyone else who is interested in updating the HSE parser?

Yes, I've got at least one fix I'd like to make myself and I'd be happy to do
some triaging and releasing too. Most of the PRs look pretty good.

Roman Cheplyaka is a maintainer on hackage, hopefully he sees and knows
something aobut this.

I'll email some people in the haskell suite group directly if this thread
doesn't go anywhere.

[Mike Izbicki]

I've always been curious why we need a stand alone package for parsing
haskell code? Last time I tried to use haskell-src-exts (admittedly
several years ago), it was out of date and didn't support the
extensions I wanted.

It seems like the ideal solution would be to have GHC's parsing code
exposed as a library. And this library would be compatible with
template haskell. Is there some technical reason why this is
inherently difficult? Or is it just an accident of history that
things didn't develop that way?

[Roman Cheplyaka]

I no longer maintain haskell-src-exts[1]. That said, I am happy to give
maintainer rights on github and hackage to a new volunteer(s) unless Niklas or
Peter (cc'd) objects. Just let me know your github and hackage usernames.

[1]: https://ro-che.info/articles/2014-11-01-rebalancing-open-source-portfolio

Roman

[Alan & Kim Zimmerman]

I think the main argument against using GHC is that the library is intimately tied to the current compiler, so writing a tool to process haskell source which is compiler version independent it tricky, requiring lots of CPP.

I am starting to wonder if it does not make sense to make a companion library for each GHC release with is just the parser/AST, and so can be used with any other compiler, even a non-ghc one.

Alan

[Erik Hesselink]

I also have commit rights on github, and although I don't have enough
free time to be official maintainer, I could merge the occasional pull
request when I have some free time. I have not done so since I'm not
the maintainer and I was added to the haskell-suite project for other
packages, but if people would like me to be slightly more involved in
haskell-src-exts, I could do that.

Regards,

Erik

[Roman Cheplyaka]

Thanks for the update, Peter. Forwarding it to cafe.

On 01/07/15 10:36, Peter A Jonsson wrote:
> Hi Roman,
>
> I’ve inserted what I replied to Matthew a couple of days ago below.
> Unfortunately my reply bounced from both libraries and haskell-cafe
> since I’m not a subscriber. Matthew should have gotten the reply
> though.
>
> Kind Regards,
>
> Peter
>
>
> Dear Matthew,

>
>> I think it is important to get this work merged and update the
>> parser to support modern language features introduced in GHC 7.8
>> and 7.10.
>

> We all agree about the importance of HSE supporting modern language
> features that people actually use. The simple reason for my
> inactivity lately is that I’ve been swamped at work so HSE has been
> on the backburner. I just started my vacation though so I’m slowly
> working through my backlog of things that I haven’t had time to do
> previously. Without having looked at any patches in the bug tracker I
> can say that the current state of HSE is stable so we should be able
> to merge the simple things quickly and the more involved things
> should be doable. Once that is done it’s probably time to make a new
> yearly major release.

[Christian Marie]

On Wed, Jul 01, 2015 at 10:31:08AM +0300, Roman Cheplyaka wrote:
> I no longer maintain haskell-src-exts[1]. That said, I am happy to give
> maintainer rights on github and hackage to a new volunteer(s) unless Niklas or
> Peter (cc'd) objects. Just let me know your github and hackage usernames.
>
> [1]: https://ro-che.info/articles/2014-11-01-rebalancing-open-source-portfolio

Your link helped with context, thanks.

If Niklas or Peter does not object:

My hackage username: ChristianMarie http://hackage.haskell.org/user/ChristianMarie
Github username: christian-marie https://github.com/christian-marie/

[Roman Cheplyaka]

Since Peter has responded and remains an active maintainer, I'll leave
it up to him.

[Christian Marie]

On Wed, Jul 01, 2015 at 09:33:26AM +0200, Alan & Kim Zimmerman wrote:
> I am starting to wonder if it does not make sense to make a companion
> library for each GHC release with is just the parser/AST, and so can be
> used with any other compiler, even a non-ghc one.

This would have been my intuition, I suspect it isn't that simple though.

The duplication of effort does seem odd, and this would solve the problem of
linters and the like always playing catch up.

This is affecting many users. I, personally, have avoided extensions in the
past when haskell-src-exts threw its toys out of the cot (specifically
LambdaCase, then DataKinds).

[Matthew Pickering]

Separating the parser from GHC is not trivial. Firstly the parser
takes DynFlags as an argument which is used in all stages of the GHC
pipeline to keep track of which flags are enabled. Second - this would
also require all syntactic datatypes to be extracted into a separate
module as well as all the various datatypes which they depend on as
well, for example RdrName depends on Name which is only introduced
after the parsing stage but would also need to be extracted.

HSE is also desirable because it already has momentum, a not
insignificant number of programs use tooling based on HSE. Secondly,
it is much easier to manipulate in general than the GHC AST,
especially the difference between the annotated and unannotated
source.

All this being said, I would much prefer a world where GHC was easier
to use as a library for this kind of thing as it is difficult to
remain consistent between the two but keeping HSE updated should
remain important in the near future.

Matt

[Matthew Pickering]

Hi Peter,

Do you know when you have some time to go through the recently
submitted patches?

If it's not soon then please could you add myself and Christian to the
maintainers so that we can perform some short term maintenance. My
immediate plan of action would be

1. Immediately cut a 1.17 release from the master branch as it
contains much better parsing of lifted constructors which are becoming
more prevalent.
2. Review and merge recent pull requests.
3. Update the parser as far as possible with missing features.
4. Release 1.18 with these improvements within 1 month.

Matt

[Neil Mitchell]

As a very heavy HSE user (HLint and Hoogle), who has had a reasonably
close working relationship with all of Niklas (my SoC mentor and my
SoC mentee), Roman and Peter (a fellow supercompiler developer) and
Matt (currently hacking on HLint), I thought I should weigh in.

The current state of HSE is a little forlorn - there are 22 open pull
requests, everything from reducing algorithm complexity (an accidental
n^3), fixing the default encoding (should be UTF8) and lots of actual
parser fixes. A lot of this is good stuff, and the high level of
external contributions is a really good sign. But they need to be
merged and reviewed, and it seems the bandwidth just isn't there at
the moment. Given Matt's comments on the existing tickets, and the
triage he's already performing on the repo, I have high hopes that
Matt could provide a bit more of the time and attention. Peter,
perhaps a co-maintainer would be good for a little while?

> If it's not soon then please could you add myself and Christian to the
> maintainers so that we can perform some short term maintenance. My
> immediate plan of action would be
>
> 1. Immediately cut a 1.17 release from the master branch as it
> contains much better parsing of lifted constructors which are becoming
> more prevalent.
> 2. Review and merge recent pull requests.
> 3. Update the parser as far as possible with missing features.
> 4. Release 1.18 with these improvements within 1 month.

Awesome! Although as a preference I'd go for as much of 2 as you can
in a couple of days before doing 1 - a lot of the patches are simple
and valuable, and a 2 day wait isn't going to be too problematic.

Thanks, Neil

[Matthew Pickering]

Peter,

Please can you do the release. I completed the work updating HSE
several months ago, a much better version of HSE has been waiting on
the master branch since then. I thank you for providing helpful review
for the patches I submitted but since then I have sent three emails
asking about the release. After the second you promised to do a
release at the end of October but that never happened.

It is a shame that the user's of HSE have had to live with many long
standing bugs when they have been fixed for months.

If you can not do it for whatever reason then please can you add me as
a maintainer on hackage and the github repo so that users can get an
updated version of HSE.

Best wishes,

Matt

On Tue, Jul 28, 2015 at 11:35 PM, Peter A Jonsson
<jonsson...@gmail.com> wrote:
> I’m not even going to bother to CC haskell-cafe since that will just bounce or go into a black hole.
>
> I exchanged a couple of mails with David (Lemmih) and my impression was that he was going to act upon some things but I didn’t follow up on that so there was probably some miscommunication between the two of us.
>
> I’ve merged some things and reviewed some patches. I agree with Neil that stable things should be merged and the yearly release of HSE should be made. I don’t want to rush things out through the door at any cost though—I believe HSE’s users are better served by something that is a strict improvement and contains more features than the last release compared to something that has even more features but is a bit shaky and requires a new major version of HSE in just a month or two which would force the client code to be re-tested and re-released.
>
> Best Wishes,
>
> Peter

[malcolm.wallace]

We too have been waiting a long time for a refreshed release of HSE.  If Matthew has already made the edits, and we are simply awaiting the package's release to Hackage, perhaps Matthew should go ahead make that release himself, and petition the Hackage admins to accept it, in view of the official maintainers' long silence?

Regards,
    Malcolm

[Roman Cheplyaka]

I just added Matthew as a maintainer on hackage and github (with Peter's
approval), so he is now able to make the release.

>> Haskel...@haskell.org <mailto:Haskel...@haskell.org>
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe