Packaging on nuget

[Ryan Riley]

I am more and more convinced that we should break up the F#x project into smaller packages with more meaningful names. I'm still somewhat a fan of just using source files rather than dlls, but perhaps both could be offered? Anyone else on board with this? It offers the possibility of providing additional, beta items, though it will certainly conflate our build procedure.

 

If you are interested, we need to work out a proposed structure for nuget packages. Also, should this affect our project structure? I started pulling out new files and folders at one point and never finished. Should that continue?

 

Cheers,

Ryan

[Mauricio Scheffer]

I think it could be a good idea, but honestly, I don't have any pain with the current packaging method or have a real need for source packages.

Note that source-only, no-binary packaging would preclude using FSharpx in C#/VB.NET projects (well, you'd have to create an ad-hoc F# project just to  host the source files, essentially reinventing the binary package), so please don't remove the binary package.

About additional, beta items: we could always mark the NuGet package as pre-release ( http://nuget.codeplex.com/wikipage?title=Pre-Release%20Packages ) in the master branch and have a separate 'stable' branch (or the other way around).

I usually push for fine-grained libraries, but I think something like FSharpx is an exception. It's like System.dll or mscorlib.dll or FSharp.Core.dll: a big library of fundamental, though not always related, stuff.

Also, having more packages could be confusing for newcomers. For example take a look at the number of packages for Rx http://nuget.org/packages?q=rx , though I wouldn't worry too much about this.

But I don't mean to sound too dogmatic. What are your pain points with the current packaging? Or how would you split things concretely?

Cheers,

Mauricio

[Ryan Riley]

On Mon, Jul 30, 2012 at 3:58 PM, Mauricio Scheffer <mauricio...@gmail.com> wrote:

I think it could be a good idea, but honestly, I don't have any pain with the current packaging method or have a real need for source packages.

Note that source-only, no-binary packaging would preclude using FSharpx in C#/VB.NET projects (well, you'd have to create an ad-hoc F# project just to  host the source files, essentially reinventing the binary package), so please don't remove the binary package.

About additional, beta items: we could always mark the NuGet package as pre-release ( http://nuget.codeplex.com/wikipage?title=Pre-Release%20Packages ) in the master branch and have a separate 'stable' branch (or the other way around).

I usually push for fine-grained libraries, but I think something like FSharpx is an exception. It's like System.dll or mscorlib.dll or FSharp.Core.dll: a big library of fundamental, though not always related, stuff.

Also, having more packages could be confusing for newcomers. For example take a look at the number of packages for Rx http://nuget.org/packages?q=rx , though I wouldn't worry too much about this.

But I don't mean to sound too dogmatic. What are your pain points with the current packaging? Or how would you split things concretely?

Cheers,

Mauricio

I don't have any pain points myself. I have heard some people discuss how large F#x is and that they wished it were more broken up. I really started this thread as a way to encourage that feedback. The only other thing is that sometimes it would be nice to be able to pull in an implementation of some computation expression or agent and make a tweak here or there locally. I don't do that often, but I have from time to time.

Ryan

[Mauricio Scheffer]

About those local changes, perhaps they should be merged upstream, or there's some parameterization missing? Do you recall any particular cases?

--

Mauricio

[Ryan Riley]

On Monday, July 30, 2012 4:52:36 PM UTC-5, Mauricio Scheffer wrote:

About those local changes, perhaps they should be merged upstream, or there's some parameterization missing? Do you recall any particular cases?

 

I don't think anything was really that useful, but perhaps I was wrong. I've mostly made minor tweaks to BlockingQueueAgent or the Async.AwaitObservable (for awaiting SocketAsyncEventArgs) or other minor tweaks to some of the IObservable stuff. Perhaps those are useful, after all. You can see several in the frack project, which is now a web server that doesn't support keep alives/persistent connections. I'm trying to sort that issue out now, actually. 

[Jack Fox]

As a consumer of FSharpx I greatly appreciate the efforts of the team. My main interest is you guys stay as productive as possible in terms of keeping the project current with MS and other releases, bug fixes, new functionality, documentation, etc. So however splitting the project or keeping it monolithic affects productivity seems most important. Having a single Nuget package and Github project to deal with is convenient for me. In fact I would favor merging in the PowerPack project.

[Gustavo Pérez León]

Here are my reflexions which are based on my very particular view of the Typeclasses branch that at some point may be merged into F#x.

In files like prelude.fs the goal is to be able to define Typeclasses and include in their definition instances for predefined F# and .NET types.

Then the rest of the F#x project may define additional types and specify instances for some type classes.

 

Now let's suppose I am the guy who did FParsec and would like to create a new version defining the parser monad in such a way that can share all operators and generic monadic functions from F#x, instead of rewriting and shadowing them (*).

Right now I would have to include and distribute the whole Fsharpx library with FParsec. I'm not sure if I (as him) would agree to do that.

I think if we pack everything together may be a problem for other base-libs-developers.

 

If we have (at least) a very basic component separated, something like FsharpXCore may be a different story and then they may have no problem to include it and distribute it within their project.

 

(*) Another reason to re-use the typeclasses could be that this way it would be possible in theory to define monad transformers combining the Parser monad with any other one.

Regarding the discussion between source and DLLs, stuff like this takes time to compile and may slow down significantly compilation time on the user project.

Having them precompiled in a dll speeds up the build process and still if you don't want to distribute the dll you can use the compiler static link option.

 

Gustavo

[Stephen Swensen]

Anecdotally, I've appreciated the way Daniel has broken up the F# PowerPack packages http://bloggemdano.blogspot.com/2011/08/new-f-powerpack-packages-on-nuget.html. I think the observation about the increased likelihood of adoption by other library developers if smaller, more targeted packages were provided is true: we'd be more likely to pull in, and likely statically link, 3rd party libraries if they were useful and didn't bloat the size of our own distributions.

[Mauricio Scheffer]

That's ok, it also happened to me that I start some ad-hoc extensions or modifications or library functions, and then realize they could be useful/generic enough to be moved to FSharpx. If you think that's the case, just move them :)

--

Mauricio

[Mauricio Scheffer]

Thanks everyone for their insights! I'll play devil's advocate a bit longer :-)

About PowerPack's packages: just yesterday I wanted to pull PowerPack through NuGet. I found FSPowerPack.Community and FSPowerPack.Core.Community, and had to download both and peek inside to see which one I really wanted (I only wanted FSharp.PowerPack.dll so I went with Core).

One way to avoid this potential confusion is to explicitly enumerate in the the package descriptionwhat DLLs are included. 

Another one would be to make the catch-all package (in this case FSPowerPack.Community) an empty package with no DLLs and just dependencies to all other packages.

About the hypothetical FParsec depending on FSharpx: *why* couldn't it depend on the current FSharpx? If it's about bloat/size, it's already depending on FSharp.Core.dll which is around 1MB and has to be deployed with the application. And even the whole .NET framework is also a dependency. So there must be another reason behind that. Is it because FSharp.Core is more stable? Is it because FSharp.Core is from Microsoft? (you could also replace FSharp.Core with PowerPack in this argument) Is it because FSharpx is perceived as "too opinionated" or something like that?

Now dropping the devil's advocate, any concrete ideas about how to split things? When taking a dependency on FSharpx, what features would you rather *not* have and why?

I'm not categorically opposed to splitting things, but I'd like to hear about concrete cases so we can make an informed decision and not just split it because of fuzzy feelings. As Jack says, having a single package can be convenient.

Cheers,

Mauricio

[Tomas Petricek]

Hi all,

here are some assorted thoughts that I had while reading the conversation so far:

 

·         Why would I want to reference only some features from FSharpX? I think code size is not an issue, but complexity is. If I was a team leader, wanting to use i.e. async features or type providers, I would not want to reference entire FSharpX, because I would not want developers to use some of the Haskell-y features of FSharpX.

 

(This is an example based on my preference in F# coding style, but you can imagine similar situations in other configurations – i.e. somebody wants to use lenses & monads, but only rely on standard F# library async and not accidentally use some imported extensions that they may not trust or like)

 

·         I think many of the components can mirror standard namespace naming from .NET and F# (FSharpX.Collections, FSharpX.Async or FSharpX.Control, FSharpX.TypeProviders, FSharpX.Interoperability). I personally would really like to see more opinionated Haskell-y parts of FSharpX in a separate library which people may or may not want to reference (I think FSharpX.Functional would be a fairly adequate name).

 

·         I also prefer just copying single (or a couple of) file(s) rather than referencing a whole library, so maybe more fine-grained organization would support this scenario (you’d know what you need to copy to get working subset).

 

·         Finally, it might make contributing to FSharpX easier – i.e. different parts could have different main owners and they would not have to fully care/understand the rest of the project.

 

Though I agree there is a lot of disadvantages too...

 

Tomas

[Ryan Riley]

Thanks for the feedback everyone. Here are a few considerations:

• Nothing stops us from continuing to deploy the all-in packages we already have. A lot of projects have taken to the empty package referencing others, but it should be just as valid to include a single, all-inclusive assembly with no dependencies on other projects, just as we do now. I think this is a good approach, and it can co-exist with many, smaller packages, though that would need to be clear in the package description.
• Two ways to do this:
• One primary solution with additional solutions/projects per NuGet package
• Break up the solution into smaller project solutions and then ILMerge during the build to create the larger, single-assembly, all-inclusive packages.
• I have no idea which of those is better, though the comment about smaller projects making it easier to contribute might mean the latter is more desirable.

Break outs

• Core: prelude and other "common" extensions. We would need to define "common."
• Collections: this is currently extensions to existing collections and some additional collections unique to F#x. The extensions to existing collections might better go in Core. Not sure here. 
• Computation Expressions:
• Break each out into its own package. This helps discoverability as these are going to be the hardest to describe and the most likely candidates for love/scorn (e.g. iteratee).
• Async: This should definitely be its own package as an extensions to the existing Async features.
• Agents: This could be bundled with Async, but I think it's enough to separate on its own, if not break each out into their own packages, again for discoverability. I could see many of these as Content packages as these are the ones I have found I am most likely to just copy and modify locally.
• MiniRx: Break this back out. A lot of this is tied to Async now, so it'll need that as a dependency. This is sort of already done wrt the Observable project. There are a few other pieces in F#x.Core that could potentially move out.
• Type Providers: Break these into their own, individual packages for discoverability. Like CE's, these will be hard to describe as a collection, and some will likely have the same love/scorn issue. This should also clean out dependencies. The DSL could be an additional, helper library or even a Content package.