Preemptive interfaces in Go

[Tim Peoples]

For years I've read the old adage, "Accept interfaces, return structs" and have spent years working to instill this understanding among my colleagues. I gathered a great many skills while learning Go (and acquiring readability)  back in the day -- and one of the strongest of those is the idea that interfaces should be defined by their consumer instead of an API producer -- but I've now been away from Google longer than I was there and I'm beginning to suspect that the general consensus among the Go Literati may have shifted around some things -- like preemptive interfaces.

My arguments against preemptive interfaces have recently run into more and more pushback  -- especially among the influx of developers coming from the Java and/or C# world who seem to continually reject any notion that Go should be any different from the way they've always done things.

This has recently come to a head with a brand new job (I'm 3 weeks in) where virtually all of their services are built atop a dependency injection framework having a data model with dozens (if not hundreds) of preemptive interfaces and my initial, cursory review tells me the codebase is at least an order of magnitude more complex that it needs to be.  (Note, I was told that none SWEs at this company (other than myself) knew any Go before they started).

So, my questions to the group are thus, "Should I even care about this at all?  Are preemptive interfaces now considered the norm with Go? Or, should I just shut up and crawl back into my hole?

TIA,

Tim.

[burak serdar]

I believe both approaches have their uses. What you call preemptive interfaces can be effectively used to hide implementation details where multiple implementations can exist. This approach can coexist very well with interfaces defined by the consumer. For example we have services that are written to implement an interface, so it becomes a logical deployment unit. Then we have consumers of that service that define parts of the interface service implements, so the consumer is not dependent on the complete service, and we can add any interceptors/filters.

However, I agree with your assessment that especially newcomers tend to choose the traditional "interface is a contract" approach. In addition to the effect of other languages, people seem to like "clean architecture". Nevertheless, even with people dedicated to clean architecture, the idea of "interface parts" seems to resonate, especially when you show that you can define an interface on the consumer side that combines parts of multiple "contracts".

 

TIA,

Tim.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/f4777928-875d-4c0a-a4a7-9fb57bf9d51fn%40googlegroups.com.

[Tim Peoples]

I don't necessarily consider the "multiple implementations" case as being truly preemptive -- if there really are multiple implementations (e.g. the "hash" package from the standard library).

I'm much more concerned about interfaces that are defined by an API producer -- for one and only one impl -- and then adding a bunch of extra (often autogenerated) code to deal with that.

t.

[burak serdar]

On Mon, Aug 8, 2022 at 12:51 PM Tim Peoples <t...@timpeoples.com> wrote:

I don't necessarily consider the "multiple implementations" case as being truly preemptive -- if there really are multiple implementations (e.g. the "hash" package from the standard library).

I'm much more concerned about interfaces that are defined by an API producer -- for one and only one impl -- and then adding a bunch of extra (often autogenerated) code to deal with that.

Like a gRPC client/server, or auto-generated swagger client/server?

I've had many instances where such an auto-generated client had to be passed down components that have no knowledge of those services. Writing such components using interfaces declaring only parts of those service implementations have benefits. An example that I can think of is an audit-trail service that deals with recording transaction metadata, looking them up, etc. It makes sense to write components that use only the writer part of that service, instead of requiring the whole thing. It makes writing tests easier. It lets you decouple services better, add adapters/interceptors etc.

 

t.

On Monday, August 8, 2022 at 11:02:31 AM UTC-7 bse...@computer.org wrote:

On Mon, Aug 8, 2022 at 11:17 AM Tim Peoples <t...@timpeoples.com> wrote:

For years I've read the old adage, "Accept interfaces, return structs" and have spent years working to instill this understanding among my colleagues. I gathered a great many skills while learning Go (and acquiring readability)  back in the day -- and one of the strongest of those is the idea that interfaces should be defined by their consumer instead of an API producer -- but I've now been away from Google longer than I was there and I'm beginning to suspect that the general consensus among the Go Literati may have shifted around some things -- like preemptive interfaces.

My arguments against preemptive interfaces have recently run into more and more pushback  -- especially among the influx of developers coming from the Java and/or C# world who seem to continually reject any notion that Go should be any different from the way they've always done things.

This has recently come to a head with a brand new job (I'm 3 weeks in) where virtually all of their services are built atop a dependency injection framework having a data model with dozens (if not hundreds) of preemptive interfaces and my initial, cursory review tells me the codebase is at least an order of magnitude more complex that it needs to be.  (Note, I was told that none SWEs at this company (other than myself) knew any Go before they started).

So, my questions to the group are thus, "Should I even care about this at all?  Are preemptive interfaces now considered the norm with Go? Or, should I just shut up and crawl back into my hole?

I believe both approaches have their uses. What you call preemptive interfaces can be effectively used to hide implementation details where multiple implementations can exist. This approach can coexist very well with interfaces defined by the consumer. For example we have services that are written to implement an interface, so it becomes a logical deployment unit. Then we have consumers of that service that define parts of the interface service implements, so the consumer is not dependent on the complete service, and we can add any interceptors/filters.

However, I agree with your assessment that especially newcomers tend to choose the traditional "interface is a contract" approach. In addition to the effect of other languages, people seem to like "clean architecture". Nevertheless, even with people dedicated to clean architecture, the idea of "interface parts" seems to resonate, especially when you show that you can define an interface on the consumer side that combines parts of multiple "contracts".

 

TIA,

Tim.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/f4777928-875d-4c0a-a4a7-9fb57bf9d51fn%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/34469b98-c37e-4a1d-af13-729b639a71ben%40googlegroups.com.

[Tim Peoples]

I can't speak to the auto-generated swagger client case but I believe gRPC is still doing things the right way -- in that the framework defines an interface I (the framework API consumer) then implements.  IOW: I don't see that as a "java style interface" (where the interface defines the API contract).

I suspect you and I are saything the same thing.

t.

[Henry]

I am sure that many of us have been on that journey. After using Go for some time, we discover some practices that are not necessarily in agreement with the existing "adages" but effectively solve our problems.  

For me, if the data type is mutable, I prefer returning interfaces. It would be something like this:
```

type Student interface {

   //...

}

type studentImpl struct {

   //...

}

func NewStudent(id string) Student {

   return &studentImpl{

      //...

   }

}

```
There is a bit of history why I use this approach. For a struct with a mutex, I wanted to ensure that the user did not accidentally copy the struct. Nowadays we have go vet to give us a warning, but this was before go vet had this functionality. So, I return a pointer to the struct and hide it behind an interface. That way, it hides the implementation details from the user and the user can pass the object around without knowing whether it has a mutex or not.

And then I ended up with some constructors returning structs and some returning interfaces. To ensure consistency, my colleagues and I decided to return interfaces for all mutable objects. For immutable objects, we return structs.

The nice thing about this approach is that it makes the syntax a lot cleaner as you have to deal with fewer pointers. 

```

//instead of this

func Update(student *Student) {

  //...

}

func UpdateMany(students []*Student){

  //...

}

//now you have this

func Update(student Student) {

  //...

}

func UpdateMany(students []Student){

  //...

}

```

Some members in the team came from higher level languages and they found working with pointers a bit awkward, so we made some accommodation for them. 

There are times when I need to upgrade some of these mutable objects, and this approach has proven to be quite flexible. It also plays nicely with code generators.

Some people may disagree with this approach, but I have been using it ever since: return interface for mutable objects, return structs for immutable objects.

[burak serdar]

I am one of those who disagrees. I have not seen any benefit from having interfaces for data objects other than making other developers happy. In my opinion, this amounts to emulating another language in Go. There are cases where this might make sense, but as a general principle, I think it should be avoided. Data is data. There are no implementation details to hide.

 

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/3774ae88-45ea-4181-909d-f18c788c7395n%40googlegroups.com.

[roger peppe]

One significant argument against preemptive interfaces is that you can't add a method to an interface type without breaking compatibility.

Also, an interface is significantly less amenable to static analysis because it's not certain where a method call is implemented.

One concern I have about the current generics implementation is that it does not infer types for generic interfaces, which increases the likelihood that people will return preemptive interfaces rather than force people to mention the type parameters explicitly (see https://github.com/golang/go/issues/41176)

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAMV2RqrP9Jy2mYhYtxo2zjjwJyMyVb8RakKwyKDaiuKwJy%3DTpw%40mail.gmail.com.

[Tim Peoples]

Yeah, I'm with Burak on this one. The interface usage you're describing Henry is exactly the kind of thing I'm talking about.  While on the surface it may seem advantageous -- in fact, I also tried writing Go that way when I first started -- my readability reviewers at Google did well to enlighten me about the many problems this can cause with Go -- some of which rog was kind enough to enumerate.

Also, since originally posting this yesterday, I've come to learn that my new shop is not only utilizing preemptive interface definitions but also a complete dependency injection framework and rather strict adherence to Clean Architecture™ -- (likely from the scripts in this repo) which goes a long way towards explaining why so much of the code looks like Java written in Go syntax.

[burak serdar]

On Tue, Aug 9, 2022 at 1:52 PM Tim Peoples <t...@timpeoples.com> wrote:

Yeah, I'm with Burak on this one. The interface usage you're describing Henry is exactly the kind of thing I'm talking about.  While on the surface it may seem advantageous -- in fact, I also tried writing Go that way when I first started -- my readability reviewers at Google did well to enlighten me about the many problems this can cause with Go -- some of which rog was kind enough to enumerate.

I think "readability" is not the right metric to use here. "Code comprehension" (comprehensibility?) should be the right metric. Readability does not always imply it can be easily comprehended. Java is readable, but not necessarily comprehensible. I argue that Go code is more comprehensible than code written in most other languages, because you can understand all the implications of the code using mostly "local knowledge", that is, knowledge you can gain by reading pieces of code "close" to the point of interest. Wherever you have interfaces, you need non-local knowledge to understand what's going on.

 

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/e8bc57a6-bcee-4904-82f6-f52cdf371b6bn%40googlegroups.com.

[Tim Peoples]

I'm Sorry -- when I said "readability reviewer" I was referring to a very google-specific term about how they ensure the author of a change understands the language they're writing. Granted, it's been a while since I worked there but, at that time, each change request (aka MR, PR, etc...) required approval from (1) a code owner and (2) someone with readability for the language in question -- and, if the author covered both of those, an "LGTM" from someone else was still required (i.e. Minimum Two Brains). Regardless, no code change could be submitted without someone with readability being involved (either the author or a reviewer).

Each language had their own procedures about how readability would be granted. For Go, each and every CR from a non-readability SWE would get assigned to a random readability reviewer  to ensure the code meets certain standards and is idiomatically Go.  It took me ~15 months and just under 60 CRs to get Go readability (compared to 4 months and 3 CRs for Python).

[Henry]

Someone mentioned that data is data and there is no need to hide implementation details. If we are to use that reasoning, there is no need for Go to allow un-exported fields/types/functions. Why do we need them at all? The reason for hiding implementation details is to narrow down the access points to the code. By keeping the access points narrow, it reduces user's cognitive burden and leaves room for future refactoring/changes/extension to the code. It is called abstraction. 

Someone also mentioned that you cannot add methods to an interface without breaking compatibility. The only case that this is true is when you have no control over the implementation. If you have control over the implementation, I don't see why you can't add methods to an interface and change its implementation in the same way you would to a struct if you are to return a struct. If you have no access to the struct, you can't add methods to the structs either. It isn't fair to consider this argument as the "return struct" superiority over "return interface".

Readability is subjective. Haskell code is often incredibly terse and elegant. Some people even swear by its readability and comprehension. While I agree that Haskell is elegant, I don't find it to be easy to read. There have been many metrics to measure code readability/comprehension/complexity over the years, and none is quite reliable. I even wrote a linter that was used in my company back then, and the tool became the company's standard that the code must pass before it can be merged to the main branch. Thinking back, what was readable back then is no longer the most readable today. Readability varies from person to person, and it changes over time.

Anyhow, I am wary when someone tries to promote a certain principle as the one true way. There is no one true way. I have seen best practices come and go. C++ is the best documentation of how best practices change over the years. C++ is a catalogue of best practices. The bad things we see in C++ were once considered best practices at some point in time. Programming languages that promote specific concepts as the one-true-way fade away when the concepts are no longer in fads.   

Treat a programming language and its features as tools to solve your problem. I am not here to promote "returning interface". I was just sharing how "returning interface" was useful in my case. 

[Tim Peoples]

Responses inline...

On Wednesday, August 10, 2022 at 1:17:21 AM UTC-7 Henry wrote:

Someone mentioned that data is data and there is no need to hide implementation details. If we are to use that reasoning, there is no need for Go to allow un-exported fields/types/functions. Why do we need them at all? The reason for hiding implementation details is to narrow down the access points to the code. By keeping the access points narrow, it reduces user's cognitive burden and leaves room for future refactoring/changes/extension to the code. It is called abstraction. 

I can't really speak to the motivation for the above "data is data" statement but my personal interpretation of it is simply that Go provides several facilities for hiding implementation details without using an interface to define your API contract.  If that's what they meant then I agree with it; if not, then I doubt I do.  However, I'm still quite lost on how exposing my API through an interface enhances my ability to hide implementation details beyond what I can do otherwise -- other than, as you pointed out earlier, syntactically hiding a pointer value. Other than that, it's been my experience that using an interface to wrap a single implementation adds little more than an annoying level of indirection.  Of course, YMMV.

 

 Someone also mentioned that you cannot add methods to an interface without breaking compatibility. The only case that this is true is when you have no control over the implementation. If you have control over the implementation, I don't see why you can't add methods to an interface and change its implementation in the same way you would to a struct if you are to return a struct. If you have no access to the struct, you can't add methods to the structs either. It isn't fair to consider this argument as the "return struct" superiority over "return interface".

I think you nailed it when you said, "The only case that this is true is when you have no control over the implementation." -- and therein lies the beauty and power of Go's interfaces over those explicit interface languages. Since Go's interfaces are implicit, they can (and quite often should) be defined outside the control of the implementer.  A really good example of this is the Marshaler interface from "encoding/json", which is defined as: 

    type Marshaler interface {

        MarshalJSON() ([]byte, error)
    }

Any type (anywhere) can implement this interface with package "json" being none the wiser.  However, when the logic in json's  Marshal(...) function runs into one of these types, it will call its MarshalJSON method as intended so the type can customize its JSON representation. Of course, if someone were to add a method to the above interface, I cannot imagine the number of things that would break.

 

Readability is subjective. Haskell code is often incredibly terse and elegant. Some people even swear by its readability and comprehension. While I agree that Haskell is elegant, I don't find it to be easy to read. There have been many metrics to measure code readability/comprehension/complexity over the years, and none is quite reliable. I even wrote a linter that was used in my company back then, and the tool became the company's standard that the code must pass before it can be merged to the main branch. Thinking back, what was readable back then is no longer the most readable today. Readability varies from person to person, and it changes over time.

The word "readability" was introduced with a very specific context and definition. Please see my previous post for a description. 

[Mike Schinkel]

On Tuesday, August 9, 2022 at 3:52:13 PM UTC-4 t...@timpeoples.com wrote:

 I've come to learn that my new shop is ... utilizing ... a complete dependency injection framework and rather strict adherence to Clean Architecture™ -- (likely from the scripts in this repo) which goes a long way towards explaining why so much of the code looks like Java written in Go syntax.

 

Sounds like your new shop is programming in GWAIJ[1].  

Yeah, I have run into that in my current shop, too.

On Tuesday, August 9, 2022 at 3:35:48 PM UTC-4 rog wrote:

> One significant argument against preemptive interfaces is that you can't add a method to an interface type without breaking compatibility.

Exactly.  Which is why single method (or at least only a few method) interfaces are preferred in Go vs. large, monolithic interfaces.

I do see the value in having an interface with methods used as a contract for an extension — like a database driver  — but those need to be very carefully considered with all efforts made up front for them to be as complete as possible upon publishing. Otherwise when modifying those interfaces later to add requirements that were not previously understood all the client code that uses them break or you need to version your interfaces, e.g. DBDriver, DBDriver2, etc,  which I have not seen done (much?) in Go. 

But I am sure I am preaching to the choir on this broader point.

On Wednesday, August 10, 2022 at 4:17:21 AM UTC-4 Henry wrote:

> Someone also mentioned that you cannot add methods to an interface without breaking compatibility. The only case that this is true is when you have no control over the implementation. If you have control over the implementation, I don't see why you can't add methods to an interface and change its implementation in the same way you would to a struct if you are to return a struct. 

In my experience having control over the implementation is not a black-and-white thing, unless you are the sole developer on a project.

What I have found is a desire to limit changes in code reviews that could affect other parts of the code and a desire to keep commits small to make reviewing code easier.  While I don't always agree with my colleagues on that rationale in all cases, these are still organizational pressures that developers can face. Changing an interface can cause large ripples in a tested and approve codebase requiring many different files to be updated, and again in my experience those are often frowned on. 

Which, the moral of the story is: strive not to change interfaces once published and in use when and if you can avoid it.

-Mike
[1] Go Written As If Java

[Dan Kortschak]

On Wed, 2022-08-10 at 11:17 -0700, Mike Schinkel wrote:
> In my experience having control over the implementation is not a
> black-and-white thing, unless you are the sole developer on a
> project.

This also brings in the consideration of that if you have complete
control over the implementation, why do you need to enforce this kind
of encapsulation so strictly?

The other question is why does that encapsulation need to happen at the
boundary of the producer of the data rather than at the boundary of the
consumer of the data?

In my experience, functions that return interfaces egregiously (they
are doing it for this kind of encapsulation) generally make my work
harder when I am debugging, and make code comprehension significantly
harder.

Dan

[Paolo Calao]

On Mon, Aug 8, 2022 at 7:17 PM Tim Peoples <t...@timpeoples.com> wrote:

For years I've read the old adage, "Accept interfaces, return structs" and have spent years working to instill this understanding among my colleagues. I gathered a great many skills while learning Go (and acquiring readability)  back in the day -- and one of the strongest of those is the idea that interfaces should be defined by their consumer instead of an API producer -- but I've now been away from Google longer than I was there and I'm beginning to suspect that the general consensus among the Go Literati may have shifted around some things -- like preemptive interfaces.

 

 I don't think the general consensus has shifted. The problem of interface pollution has always been around. 

 

My arguments against preemptive interfaces have recently run into more and more pushback  -- especially among the influx of developers coming from the Java and/or C# world who seem to continually reject any notion that Go should be any different from the way they've always done things.

 

This looks to me like a cultural problem rather than a technical one. 

Each language has its own idiom and best practices. Any programmer can decide not to follow guidelines just as any carpenter is free to hit the nail with the handle.   

 

This has recently come to a head with a brand new job (I'm 3 weeks in) where virtually all of their services are built atop a dependency injection framework having a data model with dozens (if not hundreds) of preemptive interfaces and my initial, cursory review tells me the codebase is at least an order of magnitude more complex that it needs to be.  (Note, I was told that none SWEs at this company (other than myself) knew any Go before they started).

So, my questions to the group are thus, "Should I even care about this at all?  Are preemptive interfaces now considered the norm with Go? Or, should I just shut up and crawl back into my hole?

IMO you should care, each of us should spread good practices and idioms to increase the overall quality of Go code.

You can leverage good references to show people your're not alone:

https://github.com/golang/go/wiki/CodeReviewComments#interfaces
https://www.ardanlabs.com/blog/2016/10/avoid-interface-pollution.html

 

TIA,

Tim.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/f4777928-875d-4c0a-a4a7-9fb57bf9d51fn%40googlegroups.com.

Privacy and Confidentiality Notice: This email and all attachments are confidential and for the designated recipient(s) only. They may also be privileged. If you are not the intended recipient or among the intended recipients, you may not use, read, retransmit, disseminate, store/copy the information in any medium or take any action in reliance upon it. Please notify the sender that you have received it in error and immediately delete the entire communication, including any attachments.

[Robert Engels]

I would say to temper judgement on both sides. 

If you’ve ever written or read Go code in an IDE and used “find implementors” you’ve seen the power of interface based design. Sure, you can still do this in Go - kind of - because an implementation may not be by design nor according to the semantics of the interface. 

Personally I find large systems benefit greatly from explicit interfaces. Go’s implicit interfaces work fine for utility code, “scripts” and internal design implementations. 

On Aug 10, 2022, at 8:51 PM, 'Paolo Calao' via golang-nuts <golan...@googlegroups.com> wrote:



To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CABJBrS69ZRf_vaz-ChaDGjM5KStJ%3DcYuT5Tb6KnEK%3DyzONqBMA%40mail.gmail.com.