On Mon, Oct 7, 2013 at 11:12 AM, Pjotr Prins <
pjotr...@gmail.com> wrote:
> Hi Brad,
>
> On Mon, Oct 07, 2013 at 09:59:50AM -0400, Brad Chapman wrote:
>> > I am not saying homebrew is a bad idea - these Ruby things tend to
>> > gain momentum, and I am a Ruby guy - but still they haven't gotten all
>> > things right. What you trade is convenience for correctness.
>>
>> What features do you feel like Homebrew is missing that Nix provides?
>
> Two things, really. Multi-system binary support with guaranteed
> correctness, and transactional installs.
>
> It boils down that you can trust a Nixpkg to be what it presents -
> with ALL its dependencies. In a non-correct system, there is no way
> you can guarantee that a package has not been improperly compiled,
> that libraries have not been overwritten during upgrades (say a CBL
> compile after a Debian base install). Nix does away with all those
> worries. Better even, you can bundle a software with its dependencies
> and deploy it on another system, and you are guaranteed it is the same
> software running. This is what we want in Science (reproducibility),
> this is what we require in medicine (certified diagnostic tools).
> People just don't realise what this really means - to have a correct
> system.
>
> Transactional installs means that the software will complain if the
> install was not complete. Think SQL transactions. Nix does that.
>
>> > It just hurts me that no one here even spends a single day seriously
>> > trying nixpkgs - a system which I have been deploying for the last 5
>> > years with gratifying results.
>>
>> I spent quite a bit of time with Nix based on your recommendation. The
>> major issues I had were:
>>
>> - It was difficult for me to setup for a single user non-root account.
>> What I'd like to be able to do is have CloudBioLinux install nix in a
>> non-privileged directory and do all the setup/configuration inside of
>> there. I ran into issues where nix wanted to put things into other
>> directories by default and required root. There might be workarounds
>> here, but I generally found the installation heavy-weight.
>
> That is different than using Nix for CBL, right? You have root
> privilige on a VM. Meanwhile, I actually use Nixpkgs on PBS clusters,
> where there is no root privilige. It works really well. You can copy
> binary files with dependencies across nodes, as long as the HOME dir
> has the same name (a different HOME dir every time does away with
> correctness testing). For me userland Nix is very useful. I can
> compile on Debian (my desktop) and run on CentOS (our PBS).
>
> For CBL with root you can just install in the default /nix/store. No
> HOME dir in sight.
> That dir is mine. And, no, I am the only bioinformatician that I am
> aware of who is deploying Nix. I stopped sharing packages on github some
> years back. If CBL gets serious about Nixpkgs I will start
> contributing again.
>
> Writing packages in Nix is actually gratifying. Nix has great
> isolation support during builds - i.e. you figure out all dependencies
> (guaranteed) and once a package installs and runs, it will do so for
> years to come. That is very attractive for bioinformatics. Running
> several versions of software next to each other is attractive too. So
> you can some old version, with its dependencies, and the latests
> version, with dependencies. And they are both guaranteed to work.
>
> I LOVE THAT. I want my systems to be predictable. That is one reason I
> use Linux over Windows. That is one reason I use Nix over Homebrew
> when I create a new deployment protocol. Especially the Galaxy crowd
> should take note of that.
>
>> - It wasn't clear how to setup an experimental/CloudBioLinux-specific
>> channel for pushing packages. One of the nice things about the
>> current `custom` framework is that we can push fixes live immediately
>> for rapidly changing development work. It would be great to have a
>> non-stable channel able to do this.
>>
>> I could have spend more time on the install and channel issues but the
>> lack of existing packages or a nix biology community didn't give me a
>> push to do so. Am I missing repositories of existing packages we could
>> benefit from?
>
> Setting up your own channel can be done - and with enough packages -
> we can even get a channel on the central system of Nix with automated
> build testing provided.
>
> In fact, Debian, homebrew and Nix packages can co-exist happily. The
> one policy we should have, is to get rid of the hard-wired build
> scripts in CBL. They don't scale, for sure, and it is the road of
> least correctness (that software just overwrites stuff - at least a
> well-behaved Debian package won't). When a homebrew package works -
> great. When we want multiple versions and/or correctness, use Nix.
I look forward to investigating way to integrate Brew and Nix into the
Galaxy Tool Shed ecosystem, but support for the custom install stuff
has been implemented and will be integrated soon and it is a step
forward relative to the state of things in Galaxy. I hope support for
these is not dropped.
It seems there is a continuum of easy to correct. The custom installs
represent one end of that, brew somewhere in the middle, and nix the
other end. Given the history and state of this industry, it is not
that surprising that most correct is not the most popular.
There are going to be developers who want something done quickly and
can use the custom stuff, others who want to reach the widest audience
and will utilize brew, and still others that want to do things most
correctly, they should be able to utilize nix. CloudBioLinux seems to
have a history of not being very opinionated, and I don't think it
should start now. I am happy to be working in an ecosystem where
people can utilize nix support, I hope CloudBioLinux continues to be a
community where I can utilize these custom install procedures.
Thanks,
-John
>
> So, referring to Tim's cartoon, this is not about standards. This is
> about policies and useful tools with an eye on the future. I am
> talking from experience, it is not that I want to lead you into a dark
> future of hard work! In fact, Nix has saved my ass a few times :) All
> I can do is recommend it.
>
> Pj.