Interface Performance Overhead

[Henry]

Hi,

I was curious and did some benchmark. It appears that there is a significant performance overhead when dealing with interface (over 100x on my machine). On the other hand, it is interesting to note that blank interface (interface{}) has barely any performance overhead.

type MyStruct struct{
   content string
}

func (m MyStruct) String() string{
   return m.content
}

type Stringer interface {
   String() string
}

//0.32ns
func MyFunctionStruct(str MyStruct) string {
   return str.String()
}

//0.32ns
func MyFunctionPointer(str *MyStruct) string {
   return str.String()
}

//this is interesting! - 0.33ns
func MyFunctionBlankInterface(str interface{}) string{
   o:=str.(MyStruct)
   return o.String()
}

//43.3ns!
func MyFunctionInterface(str Stringer) string{
   return o.String()
}

The same goes when you are returning interface or doing type assertion involving interface. As long as there is an interface involved (with the exception of blank interface), there is a pretty significant performance overhead. I would imagine that if you use many interfaces in your project, this overhead would add up pretty quickly. Given that interface is one of the primary ways for doing abstraction in Go, I suppose this is one area worthy for improvement. I hope this would be a useful feedback to the Go Team.

Henry

[Dave Cheney]

Hi,

I’d expect there to be some overhead dispatching via an interface but not 100x. Would you be able to post your benchmarks and maybe someone else can verify your findings.

Thanks

[Reinhard Luediger]

I'm not able to reproduce it with this little test thing.

package main

import "fmt"

type Greeter interface{
   Greet(Name string) string
}


type Greet struct{

}

func (g *Greet) Greet(Name string) string {
   return fmt.Sprintf("Hello %s",Name)
}


func Structure(g Greet,Name string) string{
   return g.Greet(Name)
}

func Interface(g Greeter,Name string) string{
   return g.Greet(Name)
}


func main() {

   var G Greeter

   G = &Greet{}
   fmt.Println(G.Greet("Reinhard"))
   
}

And these Benchmarks

package main


import "testing"


func BenchmarkGreet_GreetInterface(b *testing.B) {
   var G Greeter
   G = &Greet{}

   for n :=0 ; n < b.N;n++{
      _ = G.Greet("Reinhard")
   }
}


func BenchmarkGreet_GreetStruct(b *testing.B) {
   var G Greet

   G = Greet{}

   for n :=0 ; n < b.N;n++{
      _ = G.Greet("Reinhard")
   }
}


func BenchmarkStructure(b *testing.B) {
   G := Greet{}
   for n :=0 ; n < b.N;n++{
      _ = Structure(G,"Reinhard")
   }
}

func BenchmarkInterface(b *testing.B) {
   G := &Greet{}
   for n :=0 ; n < b.N;n++{
      _ = Interface(G,"Reinhard")
   }
}

I got these results

go test -v -bench=.

goos: darwin

goarch: amd64

pkg: luediger.net/playground/interfaces/bench

BenchmarkGreet_GreetInterface-4         20000000               104 ns/op

BenchmarkGreet_GreetStruct-4            20000000               103 ns/op

BenchmarkStructure-4                    20000000               105 ns/op

BenchmarkInterface-4                    20000000               106 ns/op

PASS

ok      luediger.net/playground/interfaces/bench        8.879s

kind regards

Reinhard Luediger

[Dave Cheney]

Please use gists or the playground rather than html email. The latter cannot be easily copied and pasted.

Thank you

[Henry]

Hi Dave,

Here is the code:

import "testing"

//======================
//  Data Definition
//======================

type myStruct struct {
    content string
}

func (s myStruct) String() string {
    return s.content

}

type Stringer interface {
    String() string
}

func printStruct(str myStruct) string {
    return str.String()
}

func printPointer(str *myStruct) string {
    return str.String()
}

func printInterface(str Stringer) string {
    return str.String()
}

func printBlankInterfaceWithStructAssertion(str interface{}) string {
    o := str.(myStruct)
    return o.String()
}

func printBlankInterfaceWithInterfaceAssertion(str interface{}) string {
    o := str.(Stringer)
    return o.String()
}

//=========================
//   Benchmark
//=========================

func BenchmarkStruct(b *testing.B) {
    data := myStruct{"My data"}
    for i := 0; i < b.N; i++ {
        printStruct(data)
    }
}

func BenchmarkPointer(b *testing.B) {
    data := myStruct{"My data"}
    for i := 0; i < b.N; i++ {
        printPointer(&data)
    }
}

func BenchmarkInterface(b *testing.B) {
    data := myStruct{"My data"}
    for i := 0; i < b.N; i++ {
        printInterface(data)
    }
}

func BenchmarkBlankInterface_WithStructAssertion(b *testing.B) {
    data := myStruct{"My data"}
    for i := 0; i < b.N; i++ {
        printBlankInterfaceWithStructAssertion(data)
    }
}

func BenchmarkBlankInterface_WithInterfaceAssertion(b *testing.B) {
    data := myStruct{"My data"}
    for i := 0; i < b.N; i++ {
        printBlankInterfaceWithInterfaceAssertion(data)
    }
}

or link to the playground https://play.golang.org/p/kN5UmgX8UfR

The result of the benchmark on my machine:

BenchmarkStruct-8                                       2000000000               0.32 ns/op
BenchmarkPointer-8                                      2000000000               0.32 ns/op
BenchmarkInterface-8                                    30000000                43.9 ns/op
BenchmarkBlankInterface_WithStructAssertion-8           2000000000               0.32 ns/op
BenchmarkBlankInterface_WithInterfaceAssertion-8        30000000                55.1 ns/op

[Jan Mercl]

On Mon, Jul 23, 2018 at 11:43 AM Henry <henry.ad...@gmail.com> wrote:

> Here is the code:

>
> func BenchmarkStruct(b *testing.B) {
>         data := myStruct{"My data"}
>         for i := 0; i < b.N; i++ {
>                 printStruct(data)
>         }
> }

Those 0,32 ns probably indicate the benchmark reports just handling of the loop control (registered) variable 'i'. Please try to assign the returned value to a TLD variable. That usually forbids the compiler to optimize out the call completely.

--

-j

[Henry]

If you were to remove the fmt.Sprintf and to just return the name, you should get a similar result to mine. Change your method to this:

func (g *Greet) Greet(name string) string {
    return name
}

[Henry]

Just to add some interesting observation. If I were to use any of the functions from the fmt package, the differences are not that large. However, just for the sake of preventing compiler optimization, I use the following dummy function as a wrapper:

func do(str string){
   if str == "" {
      panic("abc")
   }
}

//so the benchmark code would now look like this
//
//  do(printBlabla(data))

 

The result is still pretty large, over 20x. Here is the benchmark with the wrapper.

Benchmark-8                                             2000000000               0.31 ns/op
BenchmarkStruct-8                                       1000000000               2.52 ns/op
BenchmarkPointer-8                                      1000000000               2.31 ns/op
BenchmarkInterface-8                                    30000000                44.5 ns/op
BenchmarkBlankInterface_WithStructAssertion-8           1000000000               2.78 ns/op
BenchmarkBlankInterface_WithInterfaceAssertion-8        30000000                55.3 ns/op

Note: Benchmark-8 is a blank benchmark with a do-nothing loop as a control.

[peterGo]

Henry,

You should benchmark both time and space.

$ go version
go version devel +48c79734ff Fri Jul 20 20:08:15 2018 +0000 linux/amd64
$ go test henry_test.go -bench=. -benchmem
goos: linux
goarch: amd64
BenchmarkStruct-4                                   2000000000     0.30 ns/op     0 B/op    0 allocs/op
BenchmarkPointer-4                                  2000000000     0.59 ns/op     0 B/op    0 allocs/op
BenchmarkInterface-4                                30000000       38.3 ns/op    16 B/op    1 allocs/op
BenchmarkBlankInterface_WithStructAssertion-4       2000000000     0.57 ns/op     0 B/op    0 allocs/op
BenchmarkBlankInterface_WithInterfaceAssertion-4    30000000       45.1 ns/op    16 B/op    1 allocs/op

Peter

[Jan Mercl]

On Mon, Jul 23, 2018 at 12:27 PM Henry <henry.ad...@gmail.com> wrote:

My machine produces:

jnml@r550:~/src/tmp> go test -v -run @ -bench . -benchmem

goos: linux

goarch: amd64

pkg: tmp

BenchmarkStruct-4                                  2000000000          0.34 ns/op        0 B/op        0 allocs/op

BenchmarkPointer-4                                  2000000000          0.34 ns/op        0 B/op        0 allocs/op

BenchmarkInterface-4                                20000000         63.5 ns/op       16 B/op        1 allocs/op

BenchmarkBlankInterface_WithStructAssertion-4      2000000000          0.34 ns/op        0 B/op        0 allocs/op

BenchmarkBlankInterface_WithInterfaceAssertion-4    20000000         74.8 ns/op       16 B/op        1 allocs/op

PASS

ok  tmp 5.064s

jnml@r550:~/src/tmp>

I suggest to try this version: https://play.golang.org/p/lxY7k_I-GSr

jnml@r550:~/src/tmp/b> go test -v -run @ -bench . -benchmem

goos: linux

goarch: amd64

pkg: tmp/b

BenchmarkStruct-4                                  2000000000          1.34 ns/op        0 B/op        0 allocs/op

BenchmarkPointer-4                                  2000000000          1.35 ns/op        0 B/op        0 allocs/op

BenchmarkInterface-4                                200000000          8.42 ns/op        0 B/op        0 allocs/op

BenchmarkBlankInterface_WithStructAssertion-4      2000000000          1.67 ns/op        0 B/op        0 allocs/op

BenchmarkBlankInterface_WithInterfaceAssertion-4    50000000         22.3 ns/op        0 B/op        0 allocs/op

PASS

ok  tmp/b 12.866s

jnml@r550:~/src/tmp/b>

As peterGo correctly hinted earlier, the overhead comes mainly from avoidable allocations.

--

-j

[Axel Wagner]

I think the effects you are seeing are in part explained by inlining and in part by escape-analysis/allocation. I ran your benchmarks with

go test -gcflags="-m -N -l" -bench=.

and also assigning the results of the print* functions to a global string variable (as Jan suggested). As a result all the interface-functions have their arguments escape and those are exactly the ones that are slower by 10x than the rest.

That's not to say the difference is unavoidable, but it's pretty much to be expected, I'd argue.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Henry]

On Monday, July 23, 2018 at 8:39:34 PM UTC+7, Axel Wagner wrote:

That's not to say the difference is unavoidable, but it's pretty much to be expected, I'd argue.

That was unexpected to me.

Many people are aware that there are some overhead associated with using interface. However, I don't think they anticipate the cost would be something like 10x, 20x, or more. I personally expected it to be about 1 to 1.5x slower, which would still be acceptable to me. But 10x slower?!  I have seen people using interface extensively and I wonder whether they are aware of the cost.

I am saying maybe we should take a look at this. I could be wrong in my benchmark too, but I think my code is pretty idiomatic - meaning that's what most people would likely write without trying to consciously optimize the allocation and stuffs. Assuming my finding is accurate, I don't think the Go Team intended the cost of the interface to be this much.

[Axel Wagner]

On Mon, Jul 23, 2018 at 6:20 PM Henry <henry.ad...@gmail.com> wrote:

On Monday, July 23, 2018 at 8:39:34 PM UTC+7, Axel Wagner wrote:

That's not to say the difference is unavoidable, but it's pretty much to be expected, I'd argue.

That was unexpected to me.

Many people are aware that there are some overhead associated with using interface. However, I don't think they anticipate the cost would be something like 10x, 20x, or more. I personally expected it to be about 1 to 1.5x slower, which would still be acceptable to me. But 10x slower?! I have seen people using interface extensively and I wonder whether they are aware of the cost.

I don't think it makes sense to talk about "10x slower" in this case. It's only 10x, if we're talking about methods that do ~nothing. In general, it makes more sense to talk about "an overhead of tens of ns" (and even then, that overhead might vanish depending on inlining decisions and escape analysis). Of course if what you do only takes single-digit ns this overhead is significant. But most methods don't. And even then, if they are called rarely enough, it doesn't matter.

But yes, that's for example the reason you for example probably don't want to use the image/ package via interfaces for colors: The methods do trivial things, get called millions of times and then the overheads add up.

I am saying maybe we should take a look at this. I could be wrong in my benchmark too, but I think my code is pretty idiomatic - meaning that's what most people would likely write without trying to consciously optimize the allocation and stuffs. Assuming my finding is accurate, I don't think the Go Team intended the cost of the interface to be this much.

The only way to reduce the costs here is to do more extensive de-virtualization work - i.e. have the compiler realize that it can elide the conversion to an interface altogether. As long as you're going through an interface, the costs are pretty much intrinsic. It's not a cheap analysis to do for non-trivial cases (no idea what the impact on compile times would be) and there are not that many trivial cases where the wins are obvious. For example, I have no idea how this is supposed to work when moving between package boundaries - and if it wouldn't, that would already completely eliminate wins for the image package.

However, I'm sure the Go team is aware of this. In fact, quick googling surfaces at least one issue related to this: https://github.com/golang/go/issues/19361

Just saying that I think the problem is less severe and the fix more complicated than you might think it is.

[Jakob Borg]

On 23 Jul 2018, at 18:20, Henry <henry.ad...@gmail.com> wrote:
>
> However, I don't think they anticipate the cost would be something like 10x, 20x, or more. I personally expected it to be about 1 to 1.5x slower, which would still be acceptable to me. But 10x slower?! I have seen people using interface extensively and I wonder whether they are aware of the cost.

Your benchmark code is doing roughly nothing, taking roughly no time at all. At that point, allocation overhead for an interface can be 10x because even a small amount becomes a large multiplier of almost nothing. If your functions did any amount of relevant work the overhead becomes less striking.

//jb

[Henry]

This isn't an attack on Go or anybody. So take it easy. :)

About my benchmark doing nothing, that is the point of a benchmark, which is to isolate the thing you want to test as much as possible from all other things. It would be difficult to draw a conclusion if you have too many things going on inside the benchmark.

It is difficult to talk about performance when it comes to real projects. In an ideal world, we would have the measurement of our project's optimal performance and we would compare our real performance against that, to see how close we are to perfection. In reality, we just don't have that optimal measurement. Most people would just try to get to an acceptable speed. There are also other things that may contribute to a greater slowdown than interface. So, when you put all that into the perspective, I agree that this 10x interface cost is probably nothing. This is also the reason why the supposedly slow languages, such as python, are still pretty much in use in the real world and why the faster languages don't always win out.

However, being engineers, it is our job to be thoroughly familiar with our tools. Knowing the cost of the things we use allows us to make better decisions. While that nanoseconds slowdown may mean nothing in an isolated test, when you have a busy loop that uses interface and calls various functions that work with interface, the potential saving could be substantial. At least now there is another area to look at when we need to squeeze more performance out of our projects.

Still, my gut tells me that using interface shouldn't have that much cost. It is up to the Go Team to decide whether this is worth their time. To me, one interesting aspect out of this is that using blank interface incurs almost no overhead.

[Ian Lance Taylor]

Your results are all about memory allocation. If you build with
-gcflags=-m, you will see that the cases that are slow are the cases
where the data variable escapes to the heap. So while I agree that
people should have some idea of the cost of operations, you should be
clear on where the cost is coming from. It's not coming from using
interfaces as such. It's coming from allocating memory on the heap.
(That said, storing non-pointer values in interfaces does increase the
likelihood of allocating memory.) For code that is highly performance
sensitive, you should minimize memory allocations. And that said, the
compiler is always getting better at eliminating memory allocations
itself, and the GC is always getting faster at handling them.

Ian

[Marvin Renich]

* Henry <henry.ad...@gmail.com> [180723 23:42]:

> Still, my gut tells me that using interface shouldn't have that much cost.

It doesn't, but it does have some cost. I still think you are not
measuring what you think you are measuring. Add this to your benchmark
and see if it makes it more clear:

func BenchmarkInterface_FromInterface(b *testing.B) {
var data Stringer = myStruct{"My data"}

for i := 0; i < b.N; i++ {
printInterface(data)
}
}

How you use interfaces is at least as important as whether or not you
use interfaces.

...Marvin