Reproducible Stackage Builds

[Matthew Pickering]

Hello,

I am trying to build stackage but am finding it hard to find a reproducible set of instructions. 

1. On the github page it says that the build "should be done only by the Stackage build machine by the Stackage curation team", does anyone know why this is?

2. Beneath this there are some instructions for a docker image which looks promising but it says is deprecated, why is this no longer maintained? Could it be resurrected?

3. I then look at the curators guide to try and do it myself. There aren't any instructions there about how to actually run the build but it seems build.sh is relevant. There are also some instructions on another email thread that I have used in the past.

stackage-curator create-plan --target nightly-2015-05-19 --plan-file current-plan.yaml
stackage-curator check --plan-file someplan.yaml
stackage-curator fetch --plan-file someplan.yaml
stackage-curator make-bundle --plan-file someplan.yaml --docmap-file somedocmap.yaml --bundle-file --somebundle.yaml --target nightly-2015-05-19

However, running these commands, the check command fails with the following errors: http://lpaste.net/177324 I thought that the latest nightlies were for GHC 8.0.1? Is there a reason for this failure?

After this, the build starts but many packages fail due to unspecified C dependencies. It seems that debian-bootstrap is meant to be used to setup the machine. Is it kept up to date? it seems that this information could be kept in the build-constraints file so that if a package which requires a dependency is removed then the package is no longer installed. 

Then I look at `build.sh` in more detail and it seems to presuppose a specific environment is already setup and quite specific to the maintainers using it to generate builds.

I was naive to think that it would be possible to do the build by just 'cabal install package1 package2 package3...' with a frozen cabal.config file listing the specific versions of each package. Would this work properly? It would also be much easier to specify additional compiler flags and different versions of GHC if it were installable in this way.

Lots of questions! I appreciate any help in getting this sorted, I want to use stackage to generate some statistics about GHC.

Matt

[Matthew Pickering]

Even better, if the C dependencies were listed in build-constraints then I could just choose to skip building these packages (and their parents).

[Adam Bergmark]

Hi Matt,

I can answer some of this.

> 1. On the github page it says that the build "should be done only by the Stackage build machine by the Stackage curation team", does anyone know why this is?

I think the point is that a stackage maintainer doesn't need to build all of stackage, they can use more efficient tools to verify that their package can be included (packdeps, travis, manually w/ stack/cabal-install, stackage-curator check)

> 2. Beneath this there are some instructions for a docker image which looks promising but it says is deprecated, why is this no longer maintained? Could it be resurrected?

We currently use the lts6 and nightly images on https://hub.docker.com/r/snoyberg/stackage/tags/

> However, running these commands, the check command fails with the following errors: http://lpaste.net/177324 I thought that the latest nightlies were for GHC 8.0.1? Is there a reason for this failure?

Do you have another version of ghc in your env? I'm not sure if there are other ways this can fail.

> After this, the build starts but many packages fail due to unspecified C dependencies. It seems that debian-bootstrap is meant to be used to setup the machine. Is it kept up to date? 

Yes it's up to date and is used for the docker images. 

> It seems that this information could be kept in the build-constraints file so that if a package which requires a dependency is removed then the package is no longer installed.

It's not always as simple as just adding a dependency, in the past we've had to install them other ways so we need a script. It would also mean that build-constraints.yaml would list instructions specific to Ubuntu 14.04. Finally we stackage curators don't have enough resources to run builds on multiple platforms. I think it would be better to distribute this work, e.g. how Peter Simons is basing the NixOS builds on stackage. This information could still be incorporated into stack when using a snapshot, but there's nothing like that implemented yet AFAIK.

Then I look at `build.sh` in more detail and it seems to presuppose a specific environment is already setup and quite specific to the maintainers using it to generate builds.

HTH,

Adam

--
You received this message because you are subscribed to the Google Groups "Stackage" group.
To unsubscribe from this group and stop receiving emails from it, send an email to stackage+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.