Differences between go1.16 and go1.15 ?

[TiT8]

Hello everyone,

I'm Lorenzo and I'm not a computer scientist, I have used some "Matlab" and I am new with Go (surprisingly fast, simple and elegant).

I'm trying to do some basic operation on matrix with the "Gonum" package using vectorization. I was curious about the speed of development and the performance (repeat... surprisingly for me!). So I've tried to do some measures:

package main

import (

    "fmt"

    "time"

    "gonum.org/v1/gonum/mat"

)

func main() {

    p := mat.NewDense(3, 3, nil)

    a := mat.NewDense(3, 3, []float64{

        1, 2, 3, 1, 2, 3, 1, 2, 3,

    })

    start := time.Now()

    p.Mul(a, a)  // element-wise

    fmt.Printf("%v\n", mat.Formatted(p, mat.Prefix(""), mat.Squeeze()))

    p.Add(a, a) 

    fmt.Printf("%v\n", mat.Formatted(p, mat.Prefix(""), mat.Squeeze()))

    p.MulElem(a, a)

    fmt.Printf("%v\n", mat.Formatted(p, mat.Prefix(""), mat.Squeeze()))

    fmt.Println(time.Since(start))

}

Well... when I run the "go run mat.go" command the result time is about 150 microseconds, but when I run "go build" and then execute the binary the result time is about 4 milliseconds. This happen when I use go1.16.

When I use go1.15.8, on the same code, the exe is faster (about 120 microseconds) then the "go run mat.go" (about 180 microseconds). One order of magnitude faster than go1.16 exe (and basic Matlab).

So:

• am I in error with the code (sorry, it's all new for me)? If yes, where is my fault?
• if the "benchmark" was correct, why go1.16 build executable is slower than go1.15.8 exe and "go run" command?
• Is it my machine's fault or Gonum package or what?
• Do you also have these values?  
  

My pc:  HP ELITEBOOK 8560w Intel(R) Core(TM) i7-2670QM CPU @ 2.20GHz, 2201 Mhz, 4 Core(s), 8 Logical Processor(s) x64.

I am on the Windows Subsystem Linux 2.  

Sorry for my English, I hope you understand my issue, thank you for the attention

[Volker Dobler]

1. Do not use go run main.go. Never!

2. Do not run your code with the much less problematic go run because

go run compiles and executes your code and the time needed to compile

it will ruin all benchmarks.

3. Always use the benchmarking infrastructure built into go test and 

and package testing.

[Volker Dobler]

(premature send, sorry)

4. Your matrix multiplication is just a few CPU instructions, it is hard

to measure something tiny reliable. So use larger matrixes.

5. You are benchmarking also how fast you can format and output

the results to stdout. This might or might not be intentional but

probably is just wrong.

6. There is no point in comparing Go 1.15 and Go 1.16 with microbenchmarks

which do not properly measure what you think they do. Your code spends

far too much time in a lot of things. Come up with a _proper_ benchmark

based on testing.B and go test. Make sure this benchmark is stable. Make

sure this benchmarks really benchmarks what you are trying to measure.

Then redo the benchmarks 10 times and compare them doing proper

statistics.

V.

On Saturday, 20 February 2021 at 15:48:32 UTC+1 TiT8 wrote:

[TiT8]

Thank you Volker.

I want to say that I was not trying to benchmark the two version of Go (two weeks ago I had go1.15.8, now 1.16). At the beginning I was trying to do some linear algebra with Go as a free “alternative of Matlab”, so I noted the difference between the exe coming from the two versions.

Now I have understand that this is not the way to measure the specific performance of my issue. I will learn more and also testing.

Sorry for your time, but thank you very much for your fast help.

[DrGo]

Dear Lorenzo,

Here’s a link to an old but still relevant tutorial on benchmarking Go programs

https://dave.cheney.net/2013/06/30/how-to-write-benchmarks-in-go

Happy benchmarking

[TiT8]

Awesome !! 

Thank you.

[Sebastien Binet]

also, if you find any odd behavior, or API issues, or possible improvements, etc...
feel free to reach out on the gonum mailing list:
- gonu...@googlegroups.com

(feel free to also reach out if Gonum did a good job at fulfilling your use case :P)

-s

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

On Saturday, February 20th, 2021 at 8:13 PM, TiT8 <lorenz...@gmail.com> wrote:

> Awesome !! Thank you.
>
> Il giorno sabato 20 febbraio 2021 alle 19:59:37 UTC+1 DrGo ha scritto:
>

> > Dear Lorenzo,Here’s a link to an old but still relevant tutorial on benchmarking Go programs

> >
> > https://dave.cheney.net/2013/06/30/how-to-write-benchmarks-in-go
> >
> > Happy benchmarking
> >
> > On Saturday, February 20, 2021 at 11:19:40 AM UTC-6 TiT8 wrote:
> >
> > > Thank you Volker.
> > >

> > > I want to say that I was not trying to benchmark the two version of Go (two weeks ago I had go1.15.8, now 1.16). At the beginning I was trying to do some linear algebra with Go as a free “alternative of Matlab”, so I noted the difference between the exe coming from the two versions.Now I have understand that this is not the way to measure the specific performance of my issue. I will learn more and also testing.

> > >
> > > Sorry for your time, but thank you very much for your fast help.Il giorno sabato 20 febbraio 2021 alle 17:31:24 UTC+1 Volker Dobler ha scritto:
> > >
> > > > (premature send, sorry)
> > > >

> > > > 4. Your matrix multiplication is just a few CPU instructions, it is hardto measure something tiny reliable. So use larger matrixes.5. You are benchmarking also how fast you can format and outputthe results to stdout. This might or might not be intentional butprobably is just wrong.6. There is no point in comparing Go 1.15 and Go 1.16 with microbenchmarkswhich do not properly measure what you think they do. Your code spendsfar too much time in a lot of things. Come up with a _proper_ benchmarkbased on testing.B and go test. Make sure this benchmark is stable. Makesure this benchmarks really benchmarks what you are trying to measure.Then redo the benchmarks 10 times and compare them doing properstatistics.

> > > >
> > > > V.
> > > >
> > > > On Saturday, 20 February 2021 at 15:48:32 UTC+1 TiT8 wrote:
> > > >

> > > > > Hello everyone,I'm Lorenzo and I'm not a computer scientist, I have used some "Matlab" and I am new with Go (surprisingly fast, simple and elegant).

> > > > >
> > > > > I'm trying to do some basic operation on matrix with the "Gonum" package using vectorization. I was curious about the speed of development and the performance (repeat... surprisingly for me!). So I've tried to do some measures:
> > > > >

> > > > > package mainimport (    "fmt"    "time"    "gonum.org/v1/gonum/mat")func main() {    p := mat.NewDense(3, 3, nil)    a := mat.NewDense(3, 3, []float64{        1, 2, 3, 1, 2, 3, 1, 2, 3,    })    start := time.Now()p.Mul(a, a) // element-wise    fmt.Printf("%v\n", mat.Formatted(p, mat.Prefix(""), mat.Squeeze()))    p.Add(a, a)     fmt.Printf("%v\n", mat.Formatted(p, mat.Prefix(""), mat.Squeeze()))    p.MulElem(a, a)    fmt.Printf("%v\n", mat.Formatted(p, mat.Prefix(""), mat.Squeeze()))    fmt.Println(time.Since(start))}

> > > > >
> > > > > Well... when I run the "go run mat.go" command the result time is about 150 microseconds, but when I run "go build" and then execute the binary the result time is about 4 milliseconds. This happen when I use go1.16.
> > > > >
> > > > > When I use go1.15.8, on the same code, the exe is faster (about 120 microseconds) then the "go run mat.go" (about 180 microseconds). One order of magnitude faster than go1.16 exe (and basic Matlab).
> > > > >
> > > > > So:
> > > > >

> > > > > - am I in error with the code (sorry, it's all new for me)? If yes, where is my fault?
> > > > > - if the "benchmark" was correct, why go1.16 build executable is slower than go1.15.8 exe and "go run" command?
> > > > > - Is it my machine's fault or Gonum package or what?
> > > > > - Do you also have these values?  
> > > > >
> > > > > My pc:  HP ELITEBOOK 8560w Intel(R) Core(TM) i7-2670QM CPU @ 2.20GHz, 2201 Mhz, 4 Core(s), 8 Logical Processor(s) x64.I am on the Windows Subsystem Linux 2.  

> > > > >
> > > > > Sorry for my English, I hope you understand my issue, thank you for the attention
>

> --
>
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
>
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
>
> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/b7b558d9-d14c-4ba7-bcb8-977015e01bfdn%40googlegroups.com.

[Ian Lance Taylor]

On Sat, Feb 20, 2021 at 8:26 AM Volker Dobler
<dr.volke...@gmail.com> wrote:
>
> 1. Do not use go run main.go. Never!

I want to point out that this is too strong. There are perfectly good
reasons to use "go run main.go". It's a simple convenience operation
that does what it says. If you have a single file Go main package,
"go run main.go" is fine. I do it all the time.

Ian

[Volker Dobler]

I admit that from a technical perspective of an expert

this advice is too strong. go run with filename arguments

exists for a purpose and of course I too do use go run gen.go

regularly and even go run main.go occasionally.

But unfortunately "go run main.go" seems to have become

a synonym for "run Go code", promoted and parroted on every

low barrier introduction blog post about Go resulting in a false

impression that "go run main.go" is "the way you execute Go code."

Which it is not in general, leading to a lot of confusion on questions

like:

• Why doesn't go run main.go work any longer if I move some of my code to a new file server.go?
• Why doesn't go build main.go foo.go special.go not honour build tags?
• Why doesn't go test some_test.go run the tests in file some_test.go but fails with compiler errors?
• Why does the trivial program package main; func main(){} lead to memory allocations as can be seen by GODEBUG=gctrace=1 go run trivial.go ?
• Why shouldn't I execute my Go script in my docker container via go run?
• Why does importing import "./mypkg/mycode.go" throw an error but the file ./mypkg/mycode.go does exist and contains the code?

[https://github.com/golang/go/issues/43970]

I think there is a major problem with "go run main.go": It creates

a _false mental model_ of how Go code is built and executed in the

minds of _beginners_. Experts know that Go code is not "executed"
but compiled, linked and the executable is loaded and executed.

Experts know that the Go's main building blocks are packages and

that the go tool works best on packages. Experts find it convenient to

run a file which is // +build ignore'ed. Experts find go run main.go

convenient and will switch to go build the moment it is no longer

a single main.go file without thinking about this switch.

But newcomers (especially coming from interpreted languages)

are not experts and I do believe that exposing them to

"go run main.go" does harm building a correct mental model

of how Go source code is compiled, linked and executed.

Why do people make the errors in the bullet list above?

Especially the last 4 points? My hypothesis is that they are too

much focused on _files_ than on packages. Why? Maybe because

if you are told "Use go run main.go to run your Go code." then

this settles the idea that source code files like main.go are the

relevant building blocks.

Because of that I think you "Never use go run main.go!" is a

good advice. It is an advice which is technically wrong.

But it helps building the right mental model of how Go works.

The advice/rule "You cannot divide by 0!" is a very good advice

albeit being technically wrong as there are rings with zero dividers

as we experts know. Nevertheless you always tell people to

_never_ divide by 0, a blunt lie. But you first have to understand

that "you cannot divide by 0" before you can do the expert step and

start working on abstract algebra and regularly use zero dividers.

In the same sense you have to understand first that the correct

way to build Go code is via "go build" (no arguments) which does

all the heavy lifting before you become an expert and will happily

and safely use go run with filename arguments.

V.

[Paul Förster]

Hi Volker,

> On 25. Feb, 2021, at 08:46, Volker Dobler <dr.volke...@gmail.com> wrote:
>
> I think there is a major problem with "go run main.go": It creates
> a _false mental model_ of how Go code is built and executed in the
> minds of _beginners_.

I am a Go newbie and I agree, though I immediately understood that "go run ..." should only be for the developing/testing phase. But then, I did a little C before and know about what compilers do. But I also found tons of stuff immediately that said to "go build ...".

Also, experience is good, but RTFM is also good, especially when starting with something completely new.

Maybe rename the "run" command to "test", so it would be "go test ..." instead of "go run ..."?

Just a thought.

Cheers,
Paul

[Brian Candler]

For me, it took a while to twig that the normal unit of compilation is the package, *and* that a package is normally a directory of files.  "go run main.go" breaks this by having a special case where the package is a single file (or worse, a collection of listed files).

I suggest that newcomers are told to create an empty directory, create main.go in that directory, and then use "go run ." (a literal dot), "go build .", "go fmt ." etc.  Of course, this means that if you have several programs you need to create separate directories for them - but that's the right thing to do.  Plus, these days, you're soon going to need "go mod init myprog" anyway, which will create a go.mod alongside it.

When starting I had a single directory with separate programs "prog1.go", "prog2.go", "prog3.go" and then did "go run prog1.go" or whatever - which kind-of works, but breaks in all kinds of unexpected ways later.

 

[Volker Dobler]

Well, there is already subcommand "test": go test runs the (unit) tests

defined in the *_test.go files using package testing, so this won't work.

Well there is nothing wrong with a plain "go run" (nor arguments at all)

or "go run ." (a single dot for "current directory") and this is even _less_

to type than "go run main.go" and doesn't suffer from the problematic

"file focus".

I think the problem is not "go run" per se, but "go run main.go"

as cargo cult for "execute my program written in Go".

V.

[Paul Förster]

Hi Volker,

> On 25. Feb, 2021, at 09:34, Volker Dobler <dr.volke...@gmail.com> wrote:
>
> Well, there is already subcommand "test": go test runs the (unit) tests
> defined in the *_test.go files using package testing, so this won't work.
>
> Well there is nothing wrong with a plain "go run" (nor arguments at all)
> or "go run ." (a single dot for "current directory") and this is even _less_
> to type than "go run main.go" and doesn't suffer from the problematic
> "file focus".
> I think the problem is not "go run" per se, but "go run main.go"
> as cargo cult for "execute my program written in Go".

I agree. It's technically sound and well documented. But unfortunately, people tend to try things out and read the docs only if something doesn't work. So, misuse and questions can be expected in my opinion.

Cheers,
Paul

[Kevin Chadwick]

On 2/25/21 8:31 AM, Brian Candler wrote:
> I suggest that newcomers are told to create an empty directory, create main.go
> in that directory, and then use "go run ." (a literal dot), "go build .", "go
> fmt ." etc.  Of course, this means that if you have several programs you need to
> create separate directories for them - but that's the right thing to do.  Plus,
> these days, you're soon going to need "go mod init myprog" anyway, which will
> create a go.mod alongside it.

I have found that using replace was problematic wrt vscode gopls and possibly
build messages. Apparently that should be improved however I believe users are
using replace that likely do not actually want or need to. There seems to be
more interest in improving that situation than the documentation bugs,
especially the onboarding documentation. The onboarding documentation, in my
mind, is far more important.

The wiki seems to point back to the start coding in go documentation too.

I have switched from replace (though I still have one in use currently) to
having each project in a folder with a single go.mod at it's root.

Within that folder you can have multiple packages without any other .mod files
and you do not need to worry about module versioning. All the files in each
folder will be treated as a separate build. You can just cd into each folder,
run go build and as long as the package is main then all .go files under that
directory will be built into one binary.

If you have another folder in the root directory next to the go.mod and within
that, the .go files are package lib. Then you can simply import it into all your
'main' packages using:

'import lib "example.com/modulename/libfoldername"'

In fact in vscode with gopls, typing lib.functionname will automatically import
it for you.

This way you can use any version control that you want to import files into any
project (folder/package within the module) and not worry about the go specific
tools except perhaps for publishing. Even then, you may not need to, as your end
result may be something else, such as an installer.

[Ian Lance Taylor]

That all makes sense, but in cases like this thread we are speaking
with people new to Go and trying to be helpful. I understand better
what you are saying when you say "Do not use go run main.go. Never!",
but it's not a helpful way to begin a reply to a question that has
nothing to do with "go run". The first response to a question should
not be "You are doing it wrong!" when in fact this person was not
doing it wrong. That isn't helpful.

Perhaps you should write a blog post with something like the above
information, and then you can link to it at the end of a reply, rather
than as the first thing to say.

Thanks.

Ian

[TiT8]

Sorry for my late reply.

Thanks Ian for advice on the point "1" of Volker's list (and "fiuu", in that case go run is so convenient).

I tried benchmark the program (also with the formatting, with larger matrix) as exercise and everything work well (great performance, beautiful world that of compiled languages) both for go1.15.8 and go1.16. 

So the answer to the title of this issue is NO (Volker was right, obviously, I was wrong with nonsense comparison).

[TiT8]

I deleted the previous message, it was old and useless.

Thank you all for the support.