Go PostgreSQL? pg versus pgx

[Paul Förster]

Hi,

is there any important advantage of using github.com/jackc/pgx/v4 instead of github.com/lib/pq? I ask because in my view, pgx is bloated beyond repair.

Please consider this:

$ go clean --modcache
$ du -sh ~/go/pkg; find ~/go/pkg -type f | wc -l
4.0K ~/go/pkg
1

$ go get github.com/lib/pq
go: downloading github.com/lib/pq v1.9.0
$ du -sh ~/go/pkg; find ~/go/pkg -type f | wc -l
712K ~/go/pkg
74

So, only 712K in 74 files, which is good. But:

$ go clean --modcache
$ du -sh ~/go/pkg; find ~/go/pkg -type f | wc -l
4.0K ~/go/pkg
1
$ go get github.com/jackc/pgx/v4
go: downloading github.com/jackc/pgx v3.6.2+incompatible
go: downloading github.com/jackc/pgx/v4 v4.10.1
go: downloading github.com/jackc/pgio v1.0.0
go: downloading github.com/jackc/pgconn v1.8.0
go: downloading github.com/jackc/pgproto3 v1.1.0
go: downloading github.com/jackc/pgtype v1.6.2
go: downloading golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543
go: downloading github.com/jackc/chunkreader v1.0.0
go: downloading github.com/jackc/pgservicefile v0.0.0-20200714003250-2b9c44734f2b
go: downloading github.com/jackc/pgpassfile v1.0.0
go: downloading golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9
go: downloading golang.org/x/text v0.3.3
go: downloading github.com/jackc/pgproto3/v2 v2.0.6
go: downloading github.com/jackc/chunkreader/v2 v2.0.1
$ du -sh ~/go/pkg; find ~/go/pkg -type f | wc -l
56M ~/go/pkg
2178

56M in 2178 files??? I just can't see the advantage of this, even though pgx obviously has more features. But that does not justify it being 80 times as big, does it?

Any thought beyond the usage of some data types which pgx can handle and pg can't?

Cheers,
Paul

[Matthew Holiday]

From the "status" section at the bottom of the README for pq:

"This package is effectively in maintenance mode and is not actively developed. Small patches and features are only rarely reviewed and merged. We recommend using pgx which is actively maintained."

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/F77C0E34-B761-4573-9E21-D4AE838527AC%40gmail.com.

--

Matt Holiday

Senior Gopher, Marketing Technologies

620 Eighth Avenue

New York, NY 10018

matthew...@nytimes.com

[Paul Förster]

Hi Matthew,

> On 22. Feb, 2021, at 15:28, Matthew Holiday <matthew...@nytimes.com> wrote:
>
> From the "status" section at the bottom of the README for pq:
>
> "This package is effectively in maintenance mode and is not actively developed. Small patches and features are only rarely reviewed and merged. We recommend using pgx which is actively maintained."

in this case, can I reduce the heavyweight somehow? Why would I need all this stuff? I only want to be able to connect and do a few selects, no DML, no DDL, just a few select statements. Also, I'm a fan of small footprints. :-)

I'm new to Go, so please bear with me if my question seems somewhat strange to you.

Cheers,
Paul


$ go get github.com/jackc/pgx/v4
go: downloading github.com/jackc/pgx/v4 v4.10.1
go: downloading github.com/jackc/pgx v3.6.2+incompatible

go: downloading github.com/jackc/pgio v1.0.0
go: downloading github.com/jackc/pgconn v1.8.0
go: downloading github.com/jackc/pgproto3 v1.1.0
go: downloading github.com/jackc/pgtype v1.6.2
go: downloading golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543
go: downloading github.com/jackc/chunkreader v1.0.0

go: downloading github.com/jackc/pgpassfile v1.0.0
go: downloading github.com/jackc/pgservicefile v0.0.0-20200714003250-2b9c44734f2b
go: downloading golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9
go: downloading github.com/jackc/pgproto3/v2 v2.0.6
go: downloading golang.org/x/text v0.3.3
go: downloading github.com/jackc/chunkreader/v2 v2.0.1

[Andrew Werner]

Hi Paul,

What makes you say that that is “heavyweight”? For what it’s worth, many of those modules used to just be directories inside the pgx library. Splitting them out has been good for the ecosystem. In general, I’d encourage you not to worry about a small handful of transitive dependencies, especially if they are standard “x” libraries and things from the library author as is the case here. 

Best,

Andrew

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/B0F53786-F171-42DE-9413-8B54B99C2732%40gmail.com.

[Matthew Holiday]

I don't know of a way without re-writing the library (it's not mine, btw :-)

Most of that appears to be internal stuff except xerrors and crypto; that's not a bad dependency list compared to some

Maybe pq was one very large package, less stuff broken into subpackages?

[Paul Förster]

Hi Andrew,

> On 22. Feb, 2021, at 16:02, Andrew Werner <awer...@gmail.com> wrote:
>
> Hi Paul,
>
> What makes you say that that is “heavyweight”? For what it’s worth, many of those modules used to just be directories inside the pgx library. Splitting them out has been good for the ecosystem. In general, I’d encourage you not to worry about a small handful of transitive dependencies, especially if they are standard “x” libraries and things from the library author as is the case here.
>
> Best,
> Andrew

for example go get saying:
go: downloading github.com/jackc/pgx v3.6.2+incompatible

and downloading much other stuff makes me think it's heavyweight (at least on the modcache). Why is it downloading v3 stuff when I explicitly stated v4 in my go get command?

Maybe I'm wrong, maybe I just haven't grasped the module concept fully yet, I don't know. That's why I asked. As I said, I'm new to Go and just making my first steps beyond simple things.

Cheers,
Paul

[Paul Förster]

Hi Matthew,

> On 22. Feb, 2021, at 16:02, Matthew Holiday <matthew...@nytimes.com> wrote:
>
> I don't know of a way without re-writing the library (it's not mine, btw :-)

I know. :-)

> Maybe pq was one very large package, less stuff broken into subpackages?

which means, only the packages whose functionality I use from import "github.com/jackc/pgx/v4", i.e. sql.Ping(), etc., are actually linked into the final executable?

Cheers,
Paul

[Andrew Werner]

V4 clearly depends in some form or another on the earlier version. This on its face also is not necessarily bad. At the end of the day here, we’re talking about an absolutely tiny amount of data. I hear that you’re new. I hope I’m successful in setting expectations that even 10s of small transitive dependencies does not indicate a problem and certainly is not “bloated beyond repair”. What resources are you concerned with? Was the compile time slow? Is the number of files upsetting? My best guess is absolutely not! Code depending on other code which is modularity designed to fit together is a very good thing. Just trust it for now and raise concerns in the context of problems backed by data. When you start to have a bad time for real rather than perceived reasons, come back and let us know. 

-Andrew

[Paul Förster]

Hi Andrew,

sorry, I got up on the wrong foot today. :-( I didn't mean to insult anyone, especially not jackc, by my statement. I'm sure a lot of work has gone into pgx. I'd like to apologize for that.

I try to keep in mind what you said.

P.S.: is this list generally a top-post list? Other mailing lists I subscribed to are generally inline-quoted.

Cheers,
Paul

> On 22. Feb, 2021, at 16:14, Andrew Werner <awer...@gmail.com> wrote:
>
> V4 clearly depends in some form or another on the earlier version. This on its face also is not necessarily bad. At the end of the day here, we’re talking about an absolutely tiny amount of data. I hear that you’re new. I hope I’m successful in setting expectations that even 10s of small transitive dependencies does not indicate a problem and certainly is not “bloated beyond repair”. What resources are you concerned with? Was the compile time slow? Is the number of files upsetting? My best guess is absolutely not! Code depending on other code which is modularity designed to fit together is a very good thing. Just trust it for now and raise concerns in the context of problems backed by data. When you start to have a bad time for real rather than perceived reasons, come back and let us know.
>
> -Andrew

> Paul