Notes from Ryan & I's conversation today about making ripple/nuget suck less down the line

[Jeremy D. Miller]

I popped by the Dovetail office today and had a conversation with Ryan about ripple and nuget I thought I'd share while I breathlessly wait for EC2 to decide to wake up and build FubuCore....

If you've used ripple on a project with more than one developer or branch, you've probably encountered the fact that ripple and nuget can make for some absolutely dreadful merge hell issues with the csproj and packages.config files.  You've also probably noticed that mixing in the usage of the Nuget VS addin quickly gets you into snarls where different projects depend on different versions of the same upstream nuget.

Ryan's idea (and I know the Extend Health guys do or used to do this) is to make ripple store the exploded nugets into a folder structure without the version number so that the csproj files never have to change and keeping the entire app on the same version gets a lot easier.  That might not help with the packages.config file issue, unless we can simply remove the usage of packages.config altogether.  What if we had a folder somewhere where each nuget was listed as a file like "Bottles.version" and it had only a single line of text denoting the version?  That should be about the cleanest possible merge I can think of.

Of course, all of the above pretty well dictates that we would need to completely drop the usage of "nuget.exe" and the VS nuget addin for projects using ripple.  My thoughts on that:

* Today ripple uses the "nuget.exe" codebase a couple versions back as a linked library.  The nuget.exe code is, um, challenging to work with as an API.  I wouldn't shed a tear if it completely disappeared from my life

* We *could* build our very own, limited client for the nuget package spec and server API.  We'd also need to be able to edit VS.Net project and solution files (MonoDevelop too????).  The cool thing is that we could do it intelligently and get a much more Mono-friendly approach than anything MS will ever do.

* Ryan also talked about making the ripple command line be more unix-like where you could pipe output from ripple to other CLI commands.  I'll let him worry about that one;)

* I'd like to consider having a model where we don't have to put the "ripple.exe" and the associated support into a git submodule.  I'd kind of like git submodules to disappear from my life if it's all the same to you folks.  Maybe ripple2 could get rid of that some how.

* I wouldn't mind having a small OWIN hosted fubu app in ripple to be a simple GUI to do more complicated nuget type operations.  We'd almost have to have that if we eliminate the usage of the nuget VS addin.  I really kind of like it for an example FubuMVC app anyway

* I would really like "ripple" to be able to import and update JavaScript/asset packages.  Whether that means CommonJS, AMD, NPM, or something that doesn't even exist yet, I don't know.  I really like being able to do "ripple update" on .Net code.  It'd be nice to be able to do the same thing w/ assets

I absolutely cannot tackle any of this stuff until after Pablo's, but either some one else can run with it or at least we can start talking about it.  It's a huge departure from the .Net mainstream.

 

Jeremy D. Miller
The Shade Tree Developer
jeremy...@yahoo.com

[Corey Kaylor]

That is how we use nuget. We still use a packages.config file at the root of our source and leverage *most* of what Nuget.exe already does, but with modifications. If no version attribute is found it pulls the latest, if it specifies a version that version is used. We have lost the tooling support and makes updating things a bit more manual. Ripple commands for updating nugets would make that less of an issue (for us anyways). We also bring branching into consideration so a published nuget can be specific to a branch. You *might* be able to leverage the prerelease naming conventions for this, but I haven't looked or thought about it to be honest.

--
You received this message because you are subscribed to the Google Groups "FubuMVC Development Group" group.
To post to this group, send email to fubumv...@googlegroups.com.
To unsubscribe from this group, send email to fubumvc-deve...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/fubumvc-devel?hl=en.

[Jeremy D. Miller]

Did you guys modify nuget.exe or did you have to write your own client for nuget?
 

Jeremy D. Miller
The Shade Tree Developer
jeremy...@yahoo.com

---

From: Corey Kaylor <co...@kaylors.net>
To: fubumv...@googlegroups.com
Sent: Thu, August 2, 2012 4:37:35 PM
Subject: Re: [fubumvc] Notes from Ryan & I's conversation today about making ripple/nuget suck less down the line

[Corey Kaylor]

It's a client that uses a slightly modified Nuget.Core (that gets around the requirements for a version attribute).

[Ryan Rauh]

I can expand on my idea more but

TL;DR; Bundler for nuget

I'd really like to have one file that specifies your dependencies much like the Gemfile in bundler.

Then have ripple track down the correct version of everything download them and then copy (or symlink) those dependencies into your /packages folder without version numbers and then track the exact versions you used for 

everything in a Gemfile.lock that you could commit to source control.

You would then need ripple to be able to replay that exact configuration from the .lock file so that you can support having different dependencies across branches. You could also archive the .lock file in CI so that the CI builds reproducible.

In theory I think it will work, but I will defer to Jeremy's experience with nuget.exe that its easier said than done. 

-Ryan

[Jeremy D. Miller]

This is where I think writing our own nuget client isn't as quite as crazy NIH as it sounds.  I didn't get into it, but nuget is also really stupid the first second you go off the beaten path.  A smarter nuget client could do things like re-apply or patch assembly references instead of just assuming things are ok.
 

Jeremy D. Miller
The Shade Tree Developer
jeremy...@yahoo.com

---

From: Corey Kaylor <co...@kaylors.net>
To: fubumv...@googlegroups.com

Sent: Thu, August 2, 2012 5:01:04 PM

[Matthew Smith]

+1 for mimicking Bundler

Our modified client that Corey mentioned is also multi-threaded which yields faster build times and dependency updates. (We've seen this shave *minutes* off build times)

We also setup a list of sources to hit in order, the last being Nuget.org. That order being:

1 - Branch specific (0-n branches)

2 - Internal master branch

3 - Internal Cache

4 - Nuget Official Feed

The internal cache being a fileshare where all packages from Nuget.org are saved so we aren't dependent on Nuget.org servers for speed/reliability. We overrode the nuget.exe caching that only caches to the machine that ran the command. The branch specific feed is determined by the branch you are on when you run the command, but we also left a backdoor to configure the addition of another development branch in the event that you need changes from another project's branched packages.

Matt

[Jeremy Miller]

Looking at what's going on in our TC builds.  I think your strategy of caching the nugets machine wide would be huge for fubu build times.  I'm pulling my hair out watching how slow ripple restore is.

[Rob Reynolds]

Just catching up on this. I wanted to chime in with a bundler for .net that exists now. Chewie ( https://github.com/Ang3lFir3/Chewie ) - I'm not sure how viable an option it is for you though seeing that it is written in powershell...

--

You received this message because you are subscribed to the Google Groups "FubuMVC Development Group" group.

To view this discussion on the web visit https://groups.google.com/d/msg/fubumvc-devel/-/VWR-gxtbJ24J.

[Jeremy D. Miller]

Powershell == Fail.

I looked at Chewie once upon a time before we started ripple, but I didn't think it got very far. 
 

Jeremy D. Miller
The Shade Tree Developer
jeremy...@yahoo.com

---

From: Rob Reynolds <ferven...@gmail.com>
To: fubumv...@googlegroups.com
Sent: Mon, August 6, 2012 11:12:15 AM

Subject: Re: [fubumvc] Notes from Ryan & I's conversation today about making ripple/nuget suck less down the line