Recommended SQLite wrapper?

[Greg Ward]

Hi all --

a quick visual scan of http://godashboard.appspot.com/ finds five
SQLite wrapper libraries. It *appears* that
https://github.com/mattn/go-sqlite3 is up-to-date; the others, not so
much.

Should I just ignore the others and use mattn's wrapper? Has anyone
tried them all?

Related: is it possible/desirable/advisable to remove projects from
the dashboard?

Greg
--
Greg Ward http://www.gerg.ca
<gr...@gerg.ca> @gergdotca

[Archos]

I choosed "https://github.com/mattn/go-sqlite3" by that reason and I've no had problems with it.

[Kamil Kisiel]

See http://code.google.com/p/go-wiki/wiki/SQLDrivers for a list of compatible SQL drivers, with the starred ones being vetted.

[Dave Collins]

Unfortunately, I don't think that dashboard is maintained anymore.  It's quite outdated and new submissions don't seem to make it on to the list.

I've found http://godoc.org/-/index to be a pretty good resource for discovering packages as well.

[Sebastien Binet]

Greg Ward <gr...@gerg.ca> writes:

> Hi all --
>
> a quick visual scan of http://godashboard.appspot.com/ finds five
> SQLite wrapper libraries. It *appears* that
> https://github.com/mattn/go-sqlite3 is up-to-date; the others, not so
> much.

I had good feedback+experience with mattn's wrapper.

-s

[John Nagle]

How a language's third party libraries are discovered, maintained,
and tested matters. It has a strong influence on language success.
Perl's CPAN, which has centralized code hosting, rules for tests,
quality control, and a procedure for dealing with packages abandoned
by the original developer, seems to have this working reasonably well.
Python's PyPi, which is just a link directory, does not.

An indicator of trouble with the process is
multiple packages for very similar functions, and it's difficult to
tell which ones, if any, are any good. This is a big problem in the
Python world. There are five packages in PyPi for parsing ISO
standard date strings. They're all defective, but in different ways.
Each is owned by some individual, and no one else can fix it.

It's partly a technical problem, and partly a social problem.
More on this in the discussion on Lambda the Ultimate:

http://lambda-the-ultimate.org/node/4675

John Nagle
Animats

[Maxim Khitrov]

I think this is a very good point. Trying to figure out which SQLite
packages are any good is what prompted this post:

https://groups.google.com/d/topic/golang-nuts/iTi3fl9Xjmg/discussion

Personally, I'm always wary of basing my software on libraries from
github, Google Code, etc. Some projects are excellent, without a
doubt. But since creating a new project and dumping some code to these
places is so easy, you're always wondering about the current status
and longevity of the code. Was it thoroughly tested, is it ready to be
used in production, is it being maintained, will it still be in a
usable form when I try to build my project a year from now?

An ideal solution would be language-specific and have a higher barrier
to entry than github and friends, but not quite as high as the
standard library. As you point out, the organization of the packages,
testing requirements, long-term storage, and the possibility of
transferring code ownership are all key points.

- Max

[Dan Kortschak]

Is this an argument for the addition of some kind of rating system into godoc.org?

[Rui Maciel]

On 01/16/2013 07:35 PM, John Nagle wrote:
> An indicator of trouble with the process is
> multiple packages for very similar functions, and it's difficult to
> tell which ones, if any, are any good.

What's wrong with downloading the packages and trying them out?

> This is a big problem in the
> Python world. There are five packages in PyPi for parsing ISO
> standard date strings. They're all defective, but in different ways.
> Each is owned by some individual, and no one else can fix it.

Why do you believe that the availability of more than one wrapper
package represents a problem? There might be more than one way to skin
a cat, and there is no way to be sure what way better fits your needs
without having someone actually put in the legwork to write these
packages, as well as testing and maintaining them.

If that was actually a problem, which isn't, then I don't know how it
would be possible to fix it. Do you intend to pick one package and then
proceed to order the people behind all the alternative packages to stop
spending their time as they see fit? And what if you (or anyone else)
picked a package that others, for any reason, might not like?

In addition, more harm has been brought to the world by forcing a one
true library onto everyone than by having a hand full of alternatives
for anyone to pick from. See, for example, the C standard library.

Finally, even if you are trying to make a case to elect a single package
as the official one, there's the maturity issue. Is there any sqlite
wrapper which can make that claim? As far as I can tell, no. At least
not yet.

Choice is a good thing. Let's not pretend it isn't.


Rui Maciel

[Maxim Khitrov]

On Wed, Jan 16, 2013 at 3:48 PM, Dan Kortschak
<dan.ko...@adelaide.edu.au> wrote:
> Is this an argument for the addition of some kind of rating system into godoc.org?

I don't think so. Maybe a rating between "works" and "doesn't work"
would be useful, but even that is insufficient. What happens when the
highest rated package is abandoned by its owner? Forking is a bad
solution when stability is the goal. This is particularly true given
that Go encourages you to import packages by their URLs.

Look at it from the perspective of a newcomer to Go. You get the
following search results:

http://godoc.org/?q=sqlite

Now what? Ten packages for working with SQLite... seriously? Of
course, half of those are forks, maybe with some modifications. Why
were those modifications made? Did the author want a feature not
present in the parent? Was there a bug that needed to be fixed, but
the owner was unresponsive? Most people do not care about any of this.
They are probably familiar with SQLite, if that's what they are
searching for, and have probably worked with it in other languages. If
they came from Python, they might even expect this functionality in
the standard library :)

It's a problem that requires only one well-designed, tested, and
maintained package that is easy to find and "just works." The
difficulty is in preserving these properties while keeping the code in
the same place and with multiple owners, as one leaves and another
takes their place. As John said, it's a social problem as well as a
technical one.

[Dan Kortschak]

'Rating system' is a very broad and careful poorly defined thing. What might be valuable is a multidimensional system which includes a boolean, 'builds', and possibly 'passes tests' (this is more difficult since some tests may rely on unsafe - I know some of mine do), a star rating may or may not be included and Gary has an open issue to report download (perhaps something like the sourceforge downloads mini-graphs would show use activity over time, which in conjunction with a last modification data for the package would give a good indication of whether the package is stale. Putting these on the doc page would be very noisy, but a link a package meta page would give this without cluttering the docs.

I agree it is primarily a social problem, but information helps resolve these kinds of issues.

Another issue is that old packages often pollute the available name spaces requiring that to maintain functional packages a new developer must often fork to a new name. A solution to this is that devs, when dropping active development of a package might like to advertise this to allow someone else to pick it up in the same location. I saw an example of this the other day, though I don't recall the package name.

A single canonical package may not be the correct approach, for reasons that have been posted here today.

Dan

[John Nagle]

On 1/16/2013 1:28 PM, Rui Maciel wrote:
> On 01/16/2013 07:35 PM, John Nagle wrote:
>> An indicator of trouble with the process is
>> multiple packages for very similar functions, and it's difficult to
>> tell which ones, if any, are any good.
>
> What's wrong with downloading the packages and trying them out?

It's a time sink for each user to redo a job that only needs
to be done once.

> Why do you believe that the availability of more than one wrapper
> package represents a problem?

Because, when this occurs, the usual problem is that most,
if not all, of the alternative packages are defective, but in
different ways.

> If that was actually a problem, which isn't, then I don't know how it
> would be possible to fix it.

OK, but that doesn't mean others might not find a solution.

The usual problem is that a package has open issues that
aren't being fixed. The goal is to come up with a way that
gets those issues fixed, if necessary without help from the original
author. A full fork of the project is one alternative, but
that's a desperation measure.

The concept here is to have a system which supports some
degree of code ownership for open source code, but not total
control by the original author. Someone other
than the original author should be able to create a new version
of an existing package. Through some approval process, the new
version could become the released version of the package.

If this were tied to the "issues" system, it might be less
controversial. If someone creates a version that fixes specific
logged issues, then there's an objective standard for deciding if
it should be approved. The original author(s) or previous maintainers
should have a voice in this, but not a veto by inaction.

The social aspects of this are complicated, and looking at how
"approved revisions" work in Wikipedia is useful.

John Nagle

[Patrick Mylund Nielsen]

Besides, this is already done in the form of GitHub Watchers, BitBucket fans, etc.

--

[Dan Kortschak]

Which 'this'? There were a few.

[Patrick Mylund Nielsen]

Voting/ratings

[Rui Maciel]

On 01/17/2013 05:46 AM, John Nagle wrote:
> On 1/16/2013 1:28 PM, Rui Maciel wrote:
>> On 01/16/2013 07:35 PM, John Nagle wrote:
>>> An indicator of trouble with the process is
>>> multiple packages for very similar functions, and it's difficult to
>>> tell which ones, if any, are any good.
>>
>> What's wrong with downloading the packages and trying them out?
>
> It's a time sink for each user to redo a job that only needs
> to be done once.

How sure are you that you won't end up wasting time having to rewrite
your code to replace a package that you've adopted without testing it first?

>> Why do you believe that the availability of more than one wrapper
>> package represents a problem?
>
> Because, when this occurs, the usual problem is that most,
> if not all, of the alternative packages are defective, but in
> different ways.

Why do you assume that only one exceptional individual (or group of
people) in the entire world is actually able to write a package that
isn't defective, and once that quota is filled then everyone else is
doomed to hack defective alternatives?

>> If that was actually a problem, which isn't, then I don't know how it
>> would be possible to fix it.
>
> OK, but that doesn't mean others might not find a solution.
>
> The usual problem is that a package has open issues that
> aren't being fixed. The goal is to come up with a way that
> gets those issues fixed, if necessary without help from the original
> author. A full fork of the project is one alternative, but
> that's a desperation measure.

If a package has an issue then you are free to help the authors by
contributing a patch. This tends to work well.

If you insist in eliminating the authors from the equation then, with
free software projects, you are free to fork them. Every single sqlite
wrapper mentioned in godashboard.appspot.com provide access to the
source code. Sites such as github even have a "fork" button on each
project, and 4 out of the 5 sqlite packages mentioned in godashboard are
hosted there.

> The concept here is to have a system which supports some
> degree of code ownership for open source code, but not total
> control by the original author. Someone other
> than the original author should be able to create a new version
> of an existing package. Through some approval process, the new
> version could become the released version of the package.

What's wrong with simply contributing patches to any of those packages?
Github even has a nifty feature called "pull request", which does
pretty much everything you've mentioned.

https://help.github.com/articles/using-pull-requests

> If this were tied to the "issues" system, it might be less
> controversial. If someone creates a version that fixes specific
> logged issues, then there's an objective standard for deciding if
> it should be approved. The original author(s) or previous maintainers
> should have a voice in this, but not a veto by inaction.

The author does have rights over his own code, which including the right
to have a say on how his work is used. If you don't like it and the
copyright owner released his work under a free software license then you
are free to fork it.

I don't know why you are so keen in taking the author out of the
equation. It's their work. Why is it such a bad idea to, in the very
least, help them take care of their own work?


Rui Maciel

[Dan Kortschak]

Yes. The points I was making was that a rating may be more complex than
a set of yellow stars in a row.

[Sebastien Binet]

On Jan 17, 2013 7:27 AM, "Dan Kortschak" <dan.ko...@adelaide.edu.au> wrote:
>
> Yes. The points I was making was that a rating may be more complex than
> a set of yellow stars in a row.

What about a somewhat automatic popularity metric ?
I suppose godoc already knows, for each package, what are the dependencies.
It's then 'just' a matter of inversing that map, count, and add a field "this package is being used by N packages (link to explicit list of packages)"

You'd get an idea of its use in the wild, how it's being used and by whom.
Community vote :)

-s
--
sent from my droid.

[Patrick Mylund Nielsen]

Yeah, that would be nice and cumberless.

It already does this, e.g. http://godoc.org/code.google.com/p/go.crypto/scrypt?view=importers

It would just need to be added to e.g. the index page. You'd still have a problem with lots of packages bitrotting, but godoc also knows when a package was last updated, so perhaps it could filter out old packages in that metric.

[Jan Mercl]

On Thu, Jan 17, 2013 at 7:43 AM, Sebastien Binet <seb....@gmail.com> wrote:
> What about a somewhat automatic popularity metric ?
> I suppose godoc already knows, for each package, what are the dependencies.
> It's then 'just' a matter of inversing that map, count, and add a field
> "this package is being used by N packages (link to explicit list of
> packages)"

Some frequently used/highly popular/very useful packages, like
<some-db-go-driver>, might get almost never imported by another
package, but only by a tool, and even that might be quite often a non
open sourced one. IOW, I don't think that _any_ bot based mechanism
can be really usable.

-j

[Patrick Mylund Nielsen]

Sure it can. Just consider the whole chain. This is how search engines treat links for page popularity. Instead now you're looking at VCS repositories instead of pages.

I don't see how a manual mechanism will provide more value. Using a library is, basically by definition, voting for it, or at least saying "this is good to use."

[Dan Kortschak]

Yes, that's very nice. As Patrick has said, some component of time in the graph is probably valuable; a weighting on incoming edges (dependent packages) that is inversely related to their modification age. Essentially this is PageRank for packages.

Dan

[Patrick Mylund Nielsen]

Jan pointed out that e.g. mattn/sqlite could be used by hundreds of packages that haven't been open sourced, and e.g. godoc wouldn't see. I think this is a good point. Will it be relatively accurate given only the open sourced packages? Maybe.

[Dan Kortschak]

And many of us use google as a search engine to find information despite it not indexing closed corporate knowledge and limiting itself to what our search history suggests we'll find interesting, because despite these limitations it is still useful - provided we are aware of the tool's limitation (just like any other tool).

So long as people are aware that the system is a guide, I think it adds value.

[Andy Balholm]

I see that godoc.org has a list of popular packages on the front page. I suppose this is based on how often their docs are viewed. This metric is useful, but probably skewed toward packages where you need to read the docs a lot.

What if it could be combined with a PageRank-like rating of how often a package is imported by other packages? I think that then it would be a pretty accurate measure of popularity.

[dcad...@gmail.com]

And the pagerank package is already done! https://github.com/dcadenas/pagerank

[Carl Johnson]

I think this is a social problem, not a technical one, so it needs a social solution. What Google should do is to provide money to set up a "Go Software Foundation" along the lines of the "Python Software Foundation." The idea would be that whether or not Google continues to be interested in it, Go will survive. (This would probably also help bring in more official corporate backers of Go besides Google.)

One of the useful functions the GSF could perform is employing people to work full time on godoc.org. Automated lists of what packages are popular are helpful, but ultimately what would be good would be a human controlled list that could say things like, "the best version of SQLite is XYZ." Doing this would take a lot of time, but if there were a well endowed foundation one or two people might be able to work on it exclusively and let people know that such-and-such package is obsolete and superseded by such-and-such fork of it.