The equivalent of Boost for Go

[Carl Menezes]

Apart from providing functionality that is not obvious in C++, the BOOST libraries also serve as a good staging ground for new features of the language or additions to its standard library.

I was wondering if such an area might be of benefit to Go, not so much for language features, but more for tuning the standard library. 

Go already has a rich standard library, but if there was one place where you knew you could get high quality code, then this might help with the evolution of the standard library that Go ships with. I'm particularly thinking of the flag package. No, this is not another flag package rant.

When newcomers to Go want to process command line arguments, they land up at StackOverflow questions which usually direct them to the flag package.

Now, what they probably want is something more intuitive like docopt (https://github.com/docopt/docopt.go), whereas what they get with flag is something a little more complex. 

However, what I'm talking about is more general than the flag package. It's about the amount of code needed to get something in the standard library do something you want. This is essentially one of the reasons the BOOST library became so popular. 

I don't know what the solution is, but it seems to me that such an area would be useful. It would allow newbies to stand on the shoulders of the seasoned vets.

While I don't like highlighting a problem without a solution, what I propose is to go one better than BOOST and make use of Go's built in package management.

Extend "go get" to have search like functionality. 

Maybe "go search"? Actually, let's use "go find" instead. Less typing and it fits better with "go get".

So, if I needed something to process command line arguments, I would 

1. Type "go find command line"
2. It would give me a list packages from this high quality repository (this is important) that matched my keywords.
3. I would most probably install the first one.
4. I'd write my code and profit.

Just a thought from a new comer to Go. I'd like to hear what the community thinks. 

[andrey mirtchovski]

> I'd like to hear what the community thinks.

the community's first thought would be to remove the .go from 'docopt.go'.

[Carl Menezes]

Lol.

Seriously though, does this idea have merit, or is my reasoning flawed?

[Benjamin Measures]

On Sunday, 24 November 2013 19:51:53 UTC, Carl Menezes wrote:

Apart from providing functionality that is not obvious in C++, the BOOST libraries also serve as a good staging ground for new features of the language or additions to its standard library.

One of the reasons for rise of BOOST is that it takes *so* long for a new C++ standard [library] to be released.

I certainly don't think we have this problem with Go just yet: Go 1 was only released last year. Besides, in contrast to some other languages (such as Java 7) which seem compelled to include the kitchen sink, in Go "less is exponentially more".

"There may well be a Go 2 one day, but not for a few years and it will be influenced by what we learn using Go 1 as it is today."

http://golang.org/doc/faq#What_is_the_status_of_the_project

  

Extend "go get" to have search like functionality. 

Maybe "go search"? Actually, let's use "go find" instead. Less typing and it fits better with "go get".

How about Go ogle? Dot com.

Seriously though, there's godoc.org for this sort of stuff.

[andrey mirtchovski]

I don't speak for the community but here's what I think:

- if one is looking for good examples of idiomatic Go code there are
plenty of those in the source of go packages and commands. it can not
get any more idiomatic than packages in the standard library
- if one is looking for staging of new ideas, those can be found in
the go.* repositories inside the go project: things that are not in
the standard library but are considered important to Go's authors. at
one point go.exp was the experimental staging ground
- if one is looking for go projects http://godoc.org/ is the place to
find them. this includes documentation, popularity and compilation
status. this is much richer than what a command-line tool can provide

lastly, an unpopular personal opinion: we see a constant stream of
requests for improvement of the language and its ecosystem, mostly
from newcomers just getting their feet wet. why is it now démodé to
learn by experimenting? go is a simple language, possibly the simplest
new language in the past 20 years. people should just write code and
stop worrying whether other newcomers have sufficient choice in
argument parsing libraries.

[Robert Melton]

On Sun, Nov 24, 2013 at 3:32 PM, andrey mirtchovski
<mirtc...@gmail.com> wrote:
> ... people should just write code and

> stop worrying whether other newcomers have sufficient choice in
> argument parsing libraries.

Also, the idea that bundling, centralization, gatekeepers and various
other addition things "on top of" the source will lead to more choice
is infuriating. Boost is a LOT (http://www.ohloh.net/p/boost) of
code, a huge beast. It is an ecosystem that once you buy into you are
expected to stay in for most things.

Give me lots of small interesting github projects any day of the week.
Lets not put barriers in place to new great projects getting off the
ground, lets not pick winners. http://godoc.org/?q=command+line seems
like a wonderful start for a language that only hit 1.0 about 18
months ago.

--
Robert Melton

[Carl Menezes]

On Mon, Nov 25, 2013 at 9:39 AM, Robert Melton <rob...@robertmelton.com> wrote:

Also, the idea that bundling, centralization, gatekeepers and various
other addition things "on top of" the source will lead to more choice
is infuriating.  

I never said it would lead to more choice.

 

Lets not pick winners.

Agreed.

 

 http://godoc.org/?q=command+line seems like a wonderful start for a language that only hit 1.0 about 18 months ago.

 

Exactly. However, https://www.google.com/search?q=golang+command+line does not mention godoc.org on the first page. 

My point is that everyone knows how to use go get. Everyone knows to use "golang" on Google to get what they want. 

Very few beginners know to use godoc.org to get packages. Most will Google.

So let's say that godoc.org fits the bill for the purpose of the discussion. 

Why not integrate a search of godoc into Go's toolset? 

It would be a tiny tool, but would get mentioned everywhere and everyone would start using it.

It would increase the simplicity of the process of finding packages. It would increase convenience.

You could still Google around for interesting little projects if you wanted. 

[Kevin Gillette]

On Sunday, November 24, 2013 1:39:41 PM UTC-7, Robert Melton wrote:

On Sun, Nov 24, 2013 at 3:32 PM, andrey mirtchovski
<mirtc...@gmail.com> wrote:
> ... people should just write code and
> stop worrying whether other newcomers have sufficient choice in
> argument parsing libraries.

Also, the idea that bundling, centralization, gatekeepers and various
other addition things "on top of" the source will lead to more choice
is infuriating.  Boost is a LOT (http://www.ohloh.net/p/boost) of
code, a huge beast.  It is an ecosystem that once you buy into you are
expected to stay in for most things.

Speaking of the actual Boost specifically, an effect of this is that due to its immense size (and what seems to be a tendency for some C++ programmers to use it as a crutch, even for trivial tasks), the barrier to acceptance can be high. For example, I'm not a practicing C++ programmer and do not have the boost development libraries installed on any of my machines, but when I have to deal with any C++ code that uses Boost, particularly for just one thing, I either rewrite that thing without Boost (given that it's often quite easy to do so and can be done in less time than is required to download the Boost dev package), or I simply find an alternative to the whole of that C++ code.

A similar thing happens with me regarding some third-party Go packages: if a single repository contains several completely unrelated Go packages, I usually either find alternatives or fork that repository and prune my fork down only to the single package of interest; for philosophical reasons, I really dislike repository layouts which cause `go get` to pull in the kitchen sink, and the family dog, when I needed was a fork. Granted, part of this issue is that popular hosting sites like GitHub only supporting a flat namespace for repositories, but choosing to bundle unrelated packages in the same repo when those packages are intended for public use is only done at the expense of package users.

For unrelated reasons, I feel that except for the stdlib and closely related libraries (go.image, etc), centralization is often a mistake -- let Darwinian Calculus determine what thrives, and the end result will generally be much better, even if there's some intermediate turbidity to deal with along the way.

[Carl Menezes]

Ok. I think that addresses the need for something like Boost. 

However, what are your thoughts on integrating a search of godoc.org into Go's toolset. For example, a command like "go find"?

[luz...@gmail.com]

On Sunday, November 24, 2013 11:18:24 PM UTC+1, Carl Menezes wrote:

However, what are your thoughts on integrating a search of godoc.org into Go's toolset. For example, a command like "go find"?

If someone wants this tool right now:

 

#!/bin/sh

curl -sSGH 'Accept: text/plain' --data-urlencode "q=$*" http://godoc.org/ | awk '{p=$1; $1=""; print p "\n   " $0}'

[RickyS]

Hey!  It works.  Cool.  (That curl thing from luz...).

[Andrew Gerrand]

On 25 November 2013 09:18, Carl Menezes <carl...@gmail.com> wrote:

what are your thoughts on integrating a search of godoc.org into Go's toolset. For example, a command like "go find"?

To inspect the results of go find" in a useful way you'd need a web browser, so I don't think this is a particularly compelling idea. Might as well use godoc.org.

We are probably going to integrate the godoc.org search into golang.org at some point soon.

Andrew

[Carl Menezes]

Thanks. I think that would help a lot.

[Florian Weimer]

* Kevin Gillette:

> For unrelated reasons, I feel that except for the stdlib and closely
> related libraries (go.image, etc), centralization is often a mistake -- let
> Darwinian Calculus determine what thrives, and the end result will
> generally be much better, even if there's some intermediate turbidity to
> deal with along the way.

On the other hand, this leads to multiple implementations of the
exactly the same task, and sometimes they are used in the same
project. This means that your executables will become larger, and it's
also more difficult to fix bugs (because you have to understand more
code, or have to apply the same fix twice). Unlike with C (and to a
lesser extent with C++), in Go, there is little chance of breakage
just because of the duplication. But that doesn't mean that
duplication isn't bad for other reasons.

With top-level application support libraries such as command line
parsers, this is of course not much of a concern, but it's annoying
when it happens elsewhere. (I see it a lot in Fedora with C
libraries.)

[Kevin Gillette]

On Sunday, November 24, 2013 2:46:28 PM UTC-7, Florian Weimer wrote:

On the other hand, this leads to multiple implementations of the
exactly the same task

Hence the Darwinian part ;)

 

and sometimes they are used in the same project.

The trick is not to be afraid of forking/modifying someone else's library to meet your own needs. If the library in question is well written, then it's usually not too difficult or time consuming to replace on of its library dependencies with an equivalent; if it's not well written, then it's certainly less appealing to use anyway.

 

and it's also more difficult to fix bugs (because you have to understand more
code, or have to apply the same fix twice).

I suspect bug fixing is less of a problem here than you fear -- say a panic occurred, so you have a stack trace -- you don't even have to know that there's a near-duplicate library built into the library to fix the bug that caused the panic. Tracing the path of execution is core to debugging and bug fixing, and any time the path of execution can be traced, not much information needs to be known about what the application as a whole is doing. For all other cases, test with the race detector enabled.

 

Unlike with C (and to a lesser extent with C++), in Go, there is little chance of breakage just because of the duplication. But that doesn't mean that duplication isn't bad for other reasons.

Agreed, though a large part of the third party libraries available are rather thin and/or depend only on the stdlib, in which case the bloat from redundancy may be negligible (sometimes just a few kb), and the duplication-at-depth issue doesn't apply.

[Gerard]

Offtopic: Each time I hear about Boost I think of this and this. Lots of code doing essentially the same thing. Macros, function overloading and templates all used together.

I am glad there is no equivalent of Boost in Go (yet). :-)

[John Nagle]

On 11/24/2013 12:58 PM, Carl Menezes wrote:
> On Mon, Nov 25, 2013 at 9:39 AM, Robert Melton <rob...@robertmelton.com>wrote:
>
>> Also, the idea that bundling, centralization, gatekeepers and various
>> other addition things "on top of" the source will lead to more choice
>> is infuriating.
>>
>
> I never said it would lead to more choice.
>
>> Lets not pick winners.
>>
>
> Agreed.

No, let's pick winners.

One of the big problems with the Python world is that
it doesn't pick winners. So there are four ISO date parsers
on PyPi, all with different bugs. CPAN for Perl has
testing and quality control, which tends to keep the package
quality up.

What's troublesome is having multiple buggy versions of
little stuff, or even medium-sized stuff like MySQL
connectors. One that works and gets maintained is better
than multiple beta and abandonware versions.

John Nagle

[Robert Melton]

On Tue, Nov 26, 2013 at 5:49 PM, John Nagle <na...@animats.com> wrote:
> On 11/24/2013 12:58 PM, Carl Menezes wrote:
>> On Mon, Nov 25, 2013 at 9:39 AM, Robert Melton <rob...@robertmelton.com>wrote:
>>
>>> Also, the idea that bundling, centralization, gatekeepers and various
>>> other addition things "on top of" the source will lead to more choice
>>> is infuriating.
>>>
>>
>> I never said it would lead to more choice.
>>
>>> Lets not pick winners.
>>>
>>
>> Agreed.
>
> No, let's pick winners.

Who should pick? Who do we hand over that power to? Explain the
process you have in mind for picking winners.

> One of the big problems with the Python world is that
> it doesn't pick winners. So there are four ISO date parsers
> on PyPi, all with different bugs.

Do you attribute this to too much choice? Do you attribute this to
the lack of more barriers to entry for adding packages (NPM and PyPi
have very low barriers)? Do you see this as a problem in the node
community as well? Do you attribute this to lack of a single person
or committee picking the "winner" among ISO date parsers?

> CPAN for Perl has
> testing and quality control, which tends to keep the package
> quality up.

There are literally dozens of date parsers in CPAN. So this is a
problem for Python but a feature for Perl? Also, there is a good bit
of trash in CPAN... abandoned and bit-rotting packages are becoming
more and more of an issue.

> What's troublesome is having multiple buggy versions of
> little stuff, or even medium-sized stuff like MySQL
> connectors. One that works and gets maintained is better
> than multiple beta and abandonware versions.

Why set up an obvious false choice (between one "working and
maintained" version and multiple "beta and abandonware" versions)?
Why can't multiple beta versions grow into multiple working and
maintained versions that make different technical decisions and
service different user bases? Why can't these other projects act as a
hedge and check on the other ones going to hell?

--
Robert Melton

[Kevin Gillette]

On Tuesday, November 26, 2013 3:49:19 PM UTC-7, John Nagle wrote:

   One of the big problems with the Python world is that
it doesn't pick winners. So there are four ISO date parsers
on PyPi, all with different bugs.  CPAN for Perl has
testing and quality control, which tends to keep the package
quality up.

Picking winners doesn't require centralization -- the typical alternative being networks of trust; I have personally disagreed with many of the "winners" I've seen picked by central authorities, but there's always someone I trust to pick what I'd consider to be a winner within a given topic. The extreme of centralization, imposed centralization, would increase the barrier to entry/access for people who do need custom implementations/behavior.

Consider that a "rather decent" sqlite library may conform exactly to database/sql while providing exactly zero additional sqlite-specific functionality; simple, clean, general (and more likely to build due to a minimal set of sqlite build options); good for everybody most of the time, insufficient for some people some of the time -- but in any case, that is the primary implementation that you'd want people to use for common tasks. Compare it to an equivalent library that also conforms to database/sql, but provides all of the low level facilities as well -- the documentation is much, much more complex, the implementation less simple/clean (creating a larger potential bug surface), and is harder to build; largely fluff for most people, but critical for some applications.

 

   What's troublesome is having multiple buggy versions of
little stuff, or even medium-sized stuff like MySQL
connectors.  One that works and gets maintained is better
than multiple beta and abandonware versions.

That's not picking winners anymore, since as you describe it, none of them were any good to begin with. In practice, at least a few of the 'losers' will be maintained, and in practice, there's no guarantee that 'winners' will stay maintained, even on CPAN

[roger peppe]

On 27 November 2013 03:48, Kevin Gillette <extempor...@gmail.com> wrote:
> Picking winners doesn't require centralization -- the typical alternative
> being networks of trust; I have personally disagreed with many of the
> "winners" I've seen picked by central authorities, but there's always
> someone I trust to pick what I'd consider to be a winner within a given
> topic. The extreme of centralization, imposed centralization, would increase
> the barrier to entry/access for people who do need custom
> implementations/behavior.

I'd love to see some kind of network of trust thing implemented around Go
packages. There are all kinds of possibilities though - I'm not sure
what kind of model would work best.