Current Project Visibility

[Daniel Theophanes]

database/sql tasks I'm aware of:

https://docs.google.com/document/d/1DtnyOhuHhmeuRor7bbYjGMSW4sFCplot3zqDBGKeWg4

It is read-only for now, please reply to this thread for comments.

 * Are there issues you have encountered that are not on this list?

 * Do you like or dislike particular proposals?

 * Do you have proposals for some of the tasks?

 * Do you think we should NOT do some of the tasks listed?

 * Other thoughts?

Please reply to this thread, for now I'll update the document directly.

Thanks, -Daniel

[Elumalai S]

Hi Daniel,

Thanks for the time and effort you put in database/sql package.

We particularly like to get a single connection from the pool. For multi tenant applications, you have to switch the database/schema and query the details in the same connection. As of now, we are using pgx library directly instead of std database/sql.

--Elumalai

[Derek Perkins]

These are the three we're most interested in.

• *Better support for non-standard types: #18417

• *Support timing tracing via Context: #18080

• Better prepared statement support in Tx: #15606
  

Thanks for all your work in the 1.8 cycle! Having context support will solve the biggest problems we've had with sql to date.

Derek

--
You received this message because you are subscribed to the Google Groups "golang-sql" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-sql+unsubscribe@googlegroups.com.
To post to this group, send email to golan...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-sql/40778a16-aaf3-41cc-8c37-e730abc7db55%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Steve Roth]

One issue that is troublesome for me that is not on your design list: revisions to the design to allow for easier testing of database code.  In particular I would like to see Begin, Query, and QueryRow return interfaces (as Exec does), rather than structures.

Here's why.  If I want to unit test a package that interacts with database/sql, I need to run it with stubs for the database/sql functionality.  So I would write that code to expect a DB handle that was an interface (matching that of sql.DB) rather than an actual sql.DB.  So far, so good.  But the Begin call in that interface returns an actual sql.Tx structure, which I can't easily stub for testing.  If it instead returned an interface, then my stub implementation of Begin could return a stub transaction that matched the same interface.

Once inside a transaction, the same is true.  A stub transaction can easily stub the Exec call, because the return from it is an interface (sql.Result) that can itself be a stub.  But Query and QueryRow are much harder to stub because they return actual structures.

Bottom line:  any time you return a concrete structure, defined in your package, that has methods on it, you make it dramatically harder to stub out your package for unit testing of its clients.  For most standard library packages, that doesn't matter too much, because stubbing them out for unit testing isn't really essential.  But for standard library packages that interact with the outside world, being able to stub them is extremely helpful.

net/http is an interesting study along these lines.  It takes a significantly different approach to testability, by providing its own stubs (in net/httptest).  But the important thing is that it does provide a way to stub out the regular functionality for testing.  I really think it would be better if database/sql did also.

Regards,

Steve

[jimmy frasche]

Testing database interactions are pretty hard without an actual
database, in general.

The easiest way around it is to have a separate api for your database
calls which is itself mockable. For example, call users.New which runs
the actual INSERT on the user table and replace that call with a
mockUsersInsert in your test code. This also provides better
separation in your code and makes application logic read more clearly.
You could still have tests that use a testing instance of your db or
schema.

You could implement a custom test driver that returned predetermined
data when given queries that match predetermined query strings, but
that's more brittle and much more complicated.

Regardless, the database/sql package falls under the Go 1
compatability agreement, so none of the existing signatures can change
to return interfaces instead of concrete types.

> --
> You received this message because you are subscribed to the Google Groups
> "golang-sql" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to golang-sql+...@googlegroups.com.

> To post to this group, send email to golan...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/golang-sql/288d48bb-656b-4545-bcaa-22a2a65ac0dd%40googlegroups.com.

[Steven Roth]

As it happens, I do have "a separate api for your database calls which is itself mockable".  But that doesn't get me out of needing to unit test the code for that API.  And certainly there will be integration tests against a real database, but those are not a substitute for true unit tests, since they cannot be run without a specific test environment.  One should be able to clone a repo and run "go test" without needing to set up anything fancy like an external database.

I agree with you that the existing interface to database/sql is fixed due to compatibility guarantees.  In light of that, perhaps the best answer would be for database/sql to make a test driver available, much like net/http makes net/http/httptest available.  The test driver could expose hooks for testing database interactions.

Regards,

Steve

> email to golang-sql+unsubscribe@googlegroups.com.

> To post to this group, send email to golan...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-sql/288d48bb-656b-4545-bcaa-22a2a65ac0dd%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "golang-sql" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-sql/hi1CpLsedJg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to golang-sql+unsubscribe@googlegroups.com.

To post to this group, send email to golan...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-sql/CANG3jXJ9ht70x2VEOJXeqHQT1KLq1Zfgc1iVc0DKe7WKi1zZJw%40mail.gmail.com.

[Derek Perkins]

I'm looking at implementing https://github.com/DATA-DOG/go-sqlmock for unit tests, though I haven't tried it out yet.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-sql/CAAnpqKEnXZCfotiGkuO%2BHrN83GoX9theLYfogCJ8-pYE6XDW5g%40mail.gmail.com.

[Daniel Theophanes]

Steve,

I've added a bullet point to the list regarding mock interfaces. However in this case I don't think the correct way to do such testing is by define interfaces sql package that you inject. In this case I think you would choose a testing database driver / connection and run your tests over that. If you used a basic ORM, you could use a basic SQL parser to return results from the queries. If you used custom sql with CTEs and who knows what PL-SQL or T-SQL, you'd probably need to have a function that setup your database in a particular way before acting on it, possibly using TestMain. In drivers it is common to provide an ENV var of the test database to use, if you are doing meaningful work on the system that shouldn't be too big of a requirement to do the test.

All that to say I'm open to such interfaces and use cases, but I couldn't be the one design them. What would such interfaces look like? How would you use it in a mock? If you could provide some examples that would go a long way for me. 

For instance, you could define a few interfaces:

type Querier interface {
   BeginTx(Context, TxOptions) (Querier, error)
   ExecContext(Context, sql string, args) (Result, error)
   QueryRowContext(Context, sql string, args) Row
   QueryContext(Context, sql string, args) (Rows, error)
}
type ConnectionPool interface {
   Querier
   ... other methods ...
}
   

And the implementations behind it could be minor wrappers around the sql objects. Then you could mock away, but again, I think the custom driver is a better way to mock in this case.

Let me know.