Improving the "Get Haskell Experience"

[Mark Lentczner]

tl;dr: We'd like to incorporate stack into Haskell Platform, and stop shipping pre-built packages, so we banish cabal hell, and have a single common way to 'get Haskell' that just works.

At ICFP '14, there were several long group discussions of the state of "getting Haskell", including Haskell Platform, Stackage, and other approaches. Over the last year, there have been a few more public discussions and evolution on the ideas, and other installer developments. In particular, Stackage's LTS package sets are a direct development from this work, and the release of stack has offered new options.

Today, drawing from all this good work and ideas from so many, we'd would like to propose a concrete plan:

• Haskell Platform becomes the standard way to get GHC and related tools: alex, cabal, happy, hscolour, and stack. It's a user-friendly, cross-platform installer that gives a standard way to "get Haskell" for most users.
• Use the stack model for package installation:
  
• The global db has only the GHC packages
• There is a package db for each curated set, Haskell Platform becomes one such set
• Projects each have their own package db, much like sandboxes.
• Haskell Platform's installer will no longer include pre-built, pre-installed packages other than GHC's set. Instead, it is configured so that stack can be used to build and install, on as needed, the corresponding Haskell Platform release packages.

We think this plan solves many different community needs:

• We have a clear way to "get Haskell" that works for a wide variety of use cases.
• HP installer gets much smaller, and about as minimal as a working installation can get.
• By leaving most packages out of the global database, users of cabal-install, will now have far fewer problems. Sandbox builds should now never give users "cabal hell" like warnings.
• By building and installing the Platform packages into it's own package db, users get the benefit of building and installing these common packages only once per system, yet can easily bypass them for any given project if desired.
• Since the Platform packages are now built and installed as needed, installing on smaller systems or servers without OpenGL will work.

To do this, we have a bit of work ahead of us: We need to settle on installation layout for each OS (including getting msys into the Windows installer); decide on the naming and versioning of the Platform package set (is it just LTS? does it have a different life cycle? etc...); getting stack ready such a distribution; and configuring (or updating) cabal-install to support the three-layer package db scheme. We think we can do this in short order.

We recognize this represents a significant change for the Platform, and will require buy-in from the various parts, especially ghc, cabal, and stack. We'd like to get your input.

- Michael & Mark

[Mark Lentczner]

No - we are not talking about the upcoming, 7.10.2 HP release. It would for the next major release after that.

Yes, stack has only been out ~1 month, but it has already shown traction in the community, and it has a clear working solution for managing "curated" pacakge sets, like Haskell Platform.

​

[Michael Snoyman]

I'm +1 on nailing down an LTS release cycle, I've been pushing for it, and planning that it would happen after the first few releases. I'm fine with doing it starting with the next release if that's desired.

The cygwin/msys conflict is a problematic one. stack has more flexibility addressing this than MinGHC did, since stack can control the PATH more directly. What we do now is, by default, add msys to the beginning of the PATH, and provide a command line option to not use msys at all. I believe this combination has addressed the bug reports we've received in the past.

That's not to say I'm opposed to generating binary databases; I'm actually in favor of having that option for all platforms at some point (there's an open stack issue about it[1]). But it's a significant amount of work, and I think if possible we should defer it until we've figured out initial integration points.

[1] https://github.com/commercialhaskell/stack/issues/117

On Sun, Jul 12, 2015 at 9:20 AM Gershom B <gers...@gmail.com> wrote:

I’m ccing cabal-devel, as that list seems pertinent to this discussion too. In general I’m in favor of unifying efforts such as this.

For windows, I’d still like the _option_ to get prebuilt library binaries for the full platform db. The modularity means that they can be just downloaded in a separate folder now, rather than installed into e.g. the global db? If so, that’s a nice win. Here is my motivation — I tried the minghc installer and it mainly worked, but it played terribly with cygwin, which I use pervasively. So I found myself in a situation where one set of paths worked with cygwin, and the other set of paths let me build the network library. I still have some anxieties about those sorts of issues, and being able to “just download the binaries” would make me feel a bit safer about at least one workaround should things get too muddled up.

If we synchronize LTS and “platform libs,” then I would like A) some way of distinguishing “blessed platform libs” from “other libs in LTS” (to solve the “curation problem” which has always been a goal of the platform) and B) a standardized release schedule for LTS.

(And, of course, I assume this will _not_ be for the platform release upcoming, which is due very shortly, but we’ll have a longer lead time to integrate and shake out all the various issues and test, etc.)

—Gershom


On July 12, 2015 at 12:04:22 PM, Mark Lentczner (mark.le...@gmail.com) wrote:
> *tl;dr: We'd like to incorporate stack into Haskell Platform, and stop

> shipping pre-built packages, so we banish cabal hell, and have a single

> common way to 'get Haskell' that just works.*

>
> At ICFP '14, there were several long group discussions of the state of
> "getting Haskell", including Haskell Platform, Stackage, and other
> approaches. Over the last year, there have been a few more public
> discussions and evolution on the ideas, and other installer developments.
> In particular, Stackage's LTS package sets are a direct development from
> this work, and the release of stack has offered new options.
>
> Today, drawing from all this good work and ideas from so many, we'd would
> like to propose a concrete plan:
>

> - Haskell Platform becomes the standard way to get *GHC* and related
> tools: *alex*, *cabal*, *happy*, *hscolour*, and *stack*. It's a

> user-friendly, cross-platform installer that gives a standard way to "get
> Haskell" for most users.

> - Use the *stack* model for package installation:
> - The global db has only the GHC packages
> - There is a package db for each curated set, Haskell Platform
> becomes one such set
> - Projects each have their own package db, much like sandboxes.
> - Haskell Platform's installer will no longer include pre-built,

> pre-installed packages other than GHC's set. Instead, it is configured so

> that *stack* can be used to build and install, on as needed, the

> corresponding Haskell Platform release packages.
>
> We think this plan solves many different community needs:
>

> - We have a clear way to "get Haskell" that works for a wide variety of
> use cases.
> - HP installer gets much smaller, and about as minimal as a working
> installation can get.
> - By leaving most packages out of the global database, users of

> cabal-install, will now have far fewer problems. Sandbox builds should now
> never give users "cabal hell" like warnings.

> - By building and installing the Platform packages into it's own package

> db, users get the benefit of building and installing these common packages
> only once per system, yet can easily bypass them for any given project if
> desired.

> - Since the Platform packages are now built and installed as needed,

> installing on smaller systems or servers without OpenGL will work.
>
> To do this, we have a bit of work ahead of us: We need to settle on
> installation layout for each OS (including getting msys into the Windows
> installer); decide on the naming and versioning of the Platform package set
> (is it just LTS? does it have a different life cycle? etc...); getting

> *stack* ready such a distribution; and configuring (or updating)
> *cabal-install* to support the three-layer package db scheme. We think we

> can do this in short order.
>
> We recognize this represents a significant change for the Platform, and

> will require buy-in from the various parts, especially *ghc*, *cabal*, and
> *stack*. We'd like to get your input.
>
> - Michael & Mark
> _______________________________________________
> Libraries mailing list
> Libr...@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries
>

_______________________________________________
ghc-devs mailing list
ghc-...@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

[Dan Burton]

Stack has the capability of downloading and installing GHC on demand, as well as auto-updating itself. Both of these features, by default, occur in subdirectories of the user's home directory. (Slightly different on Windows but same idea.) Would the Platform install GHC to the stack location directly, or install it system-wide as previously? (Stack can still make use of GHC anywhere on your path.)

--
You received this message because you are subscribed to the Google Groups "haskell-stack" group.
To unsubscribe from this group and stop receiving emails from it, send an email to haskell-stac...@googlegroups.com.
To post to this group, send email to haskel...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/haskell-stack/CAKA2JgKYuBMU5wtbjGV72dDKvSJePWbhoZeRCuJYR9SFoV8EDw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
-- Dan Burton

[Michael Snoyman]

I think it depends somewhat on operating system, since there are permissions issues to be dealt with regarding user vs global installation. In principle though: I think the HP installers would install to a non-stack-specific location, and then stack would simply use that GHC based on its inclusion in the PATH. (I'm also guessing this will tie in nicely with Linux distro packages, which would likely want ghc to live in /usr/bin).

To view this discussion on the web visit https://groups.google.com/d/msgid/haskell-stack/CALSygwfNjyxWFL0y4-_5aSyZZVQSeb5705MzZnaM4Q7GA2MLBg%40mail.gmail.com.

[Greg Weber]

The main reason I am using stack is for its support for building a project containing multiple packages. There aren't any other tools that make this a first-class concept that is easy to use and not buggy.

In addition, stack has a first-class concept of curated package sets. All of these are required to have smooth installs in real world projects, and they aren't likely to appear in cabal-install in a short time frame.

On the other hand, stack already has good package distribution.
But providing both cabal-install and stack is the right choice that helps keep the Haskell platform relevant and avoid new users having to go through multiple installation steps.

On Sun, Jul 12, 2015 at 12:07 PM, <ami...@gmail.com> wrote:

The HP is a place for tools which have stabilized. A few reasons why I don't think stack is ready:

- stack has initial interest from the community, but so did ghc-dev and hsenv. It's still (imo) in proof of concept stage, and we don't know where its lessons will ultimately end up. In those other two cases, they ended up being folded into cabal, which is one likely scenario with stack.

- There are other solutions on the horizon! There's a GSOC project that will add nix-style package management to cabal. This should eliminate most (all?) the cabal hell we know and love, and that's why most stack users are trying it, no?

- The HP is for stable apis for users, too -- we provide tools that users won't have to unlearn or re-learn. Most languages don't have two package managers! We're clearly in an unstable state, which isn't when we should add to the HP.

If 9 months from now stack is clearly the right tool for the job (i.e. we're considering deprecating cabal), then I'd definitely consider supporting adding it to the HP but otherwise i'm strongly -1. (If it needs saying, I don't think stack is a bad tool).

tl;dr: The dust hasn't settled. Please don't add tools to the HP which we don't know will be in common use (in their current form) a year from now.

Tom

El Jul 12, 2015, a las 12:25, Mark Lentczner <mark.le...@gmail.com> escribió:

> No - we are not talking about the upcoming, 7.10.2 HP release. It would for the next major release after that.
>
> Yes, stack has only been out ~1 month, but it has already shown traction in the community, and it has a clear working solution for managing "curated" pacakge sets, like Haskell Platform.
> ​

[Greg Weber]

On Mon, Jul 13, 2015 at 12:34 AM, Kosyrev Serge <_dee...@feelingofgreen.ru> wrote:

Greg Weber <gr...@gregweber.info> writes:
> The main reason I am using stack is for its support for building a project
> containing multiple packages. There aren't any other tools that make this a
> first-class concept that is easy to use and not buggy.
> In addition, stack has a first-class concept of curated package sets. All of these
> are required to have smooth installs in real world projects, and they aren't
> likely to appear in cabal-install in a short time frame.

The problem that I'm personally facing all too often, is exploratory
development, where the modus operandi is to try using versions/branches
that are not yet released on Hackage.

Things like this happen especially during GHC version transitions, where
ghc-new buildability of libraries/tools is often in flux, and so you
/have/ to chase the tail (or is it HEAD?).

As an example, the version of ghc-mod that works with 7.10 is still
unreleased on Cabal -- months passed, the prospects still indefinite.

But it also happens out of plain curiosity and willingness to try out
how new things could affect the way of problem expression.

For an example of that, let's take the 'nokinds' branch of GHC, where
Richard Eisenberg does research on formulating GHC with dependent types.

Another situation where these things matter especially is collaborative
development -- trying to help with bugs, for example.

All these things ring of the same, actually..

..where Hackage puts a mild barrier to sharing small contributions with
the world, Stack puts a wall.

I think you are using the wrong terminology. You probably mean to compare "stackage" with "Hackage".

But your replly is supposed to be about "stack", which has first-class support for building packages that are not on Hackage as part of your project, including fetching the package from github for you.
https://github.com/commercialhaskell/stack/wiki/stack.yaml#project-config

Which is a good thing.

But also a bad one.

..and unless I'm wrong, and we're indeed moving to a world where people would
increasingly use Stack, this dichotomy is /bound/ to become more pressing.

Curiously, there's a technology to solve to both sides of the story -- Nix,
which was mentioned before, but I think its salient point applies especially
well to this dichotomy:

 1. use the existing curated releases, but also can
 2. override packages with a Github fork URL, commit id and the physical
    checkout hash -- and have everything pick it up in a transparent, fixpoint way.

Again, there is already support for this in stack. Give it a try.

Admittedly it's not been made as trivial to use -- there's more moving
parts to factor, and up until now there just wasn't any party seriously
interested in doing the user-facing parts.

..And so, I can't help but wonder.. what if the Stack authors would have
applied their expertise to provide the same user experience they
achieved..

..but with Nix as an underlying technology?

Then stack would be forcing Windows users to use cygwin
 

--
respectfully,
Косырев Серёга

[Mark Lentczner]

I'm glad to read the variety of response, and want to summarize and respond to the most common points:

stack is too new & having two package managers will confuse

— It is indeed new, though it has arrived very well formed, and has gained a lot of users in short order.  There are two reasons why I think we should be including it (or rather, including the version a few months down the road when we do the next major HP):

1) It's approach to package db organization is very good,

2) I would be better for all that stack were part of the suite of tools, rather than an alternative suite.

Too be clear: I don't anticipate stack replacing cabal - different tasks will need one or the other. But I would like to see them unified in package db management.

nix-like package management

—  I'll be honest: I don't think nix-style package management is all that useful. Enabling the package machinery to be able to support all those different package builds is fine. But it seems the devil is in the user experience of the tools that manage it. Until we see what use cases those tools support, I really have no way to evaluate what extending package dbs this way will add.

possible change to cabal sandbox handling of the global db

— Of course this will improve more complex builds for people who download the HP - but does so by reducing the HP install to a minimal install. This is fine, but I think what we proposed goes further.

pre-built package binaries are good

— Yes, especially on Windows, but I'd like to see a way that users can get those binaries, easily, and securely, without bloating the HP download. At present, users have to download 150MB ~ 230MB of installer - it is heavy! If we can make it so that users only have to compile a package the first time they use it in any project, for many this is ideal: you start minimal, incur cost is only once on first use, and build exactly in your system environment. If we augment this with a "binary repository" then we can let users decide to trade the compile cost for the download cost.

- Mark

Sorry for slow, group response, I'm still on vacation!