Go vs C speed - What am I doing wrong?

[Miki Tebeka]

Hi,

I'm comparing two loops in Go and C. The Go code on my machine is about 3 times slower than C. I know C can be faster but didn't think it'll be that faster. Any ideas what's making the Go code slower?

You can see the code at https://github.com/tebeka/go-c-loop

Go Code:

package main


import (
 "fmt"
 "os"
 "strconv"
 "time"
)


func main() {
 n, _ := strconv.Atoi(os.Args[1])
 m, _ := strconv.Atoi(os.Args[2])


 sum := int(0)
 start := time.Now()
 for i := 0; i < 10000000; i++ {
 if i%n != m {
 sum += n
 }
 }


 fmt.Println(time.Now().Sub(start).Seconds(), sum)
}

C Code

#include <stdio.h>
#include <unistd.h>
#include <x86intrin.h>




int main(int argc,char** argv) {
            unsigned long long ull0,ull1;
            unsigned int sum=0,n,m;


            sscanf(argv[1],"%d",&n);
            sscanf(argv[2],"%d",&m);


            ull0 = __rdtsc();
            for(int i=0; i<10000000; i++) {
                        if(i%n != m) {
                                    sum += n;
                        }


            }


            ull1 = __rdtsc();
            printf("%f %d\n",(ull1-ull0)/2.1e9,sum);
}

[Robert Engels]

Don’t use rtdsc in the C program use gettimeofday to ensure you are comparing the same. 

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Miki Tebeka]

A bit of profiling shows that the modulo operator takes most of the time:

 1.77s      1.77s (flat, cum)   100% of Total
         .          .      2:
         .          .      3:import (
         .          .      4: "testing"
         .          .      5:)
         .          .      6:
      10ms       10ms      7:func loop(m, n int) int {
         .          .      8: sum := int(0)
      30ms       30ms      9: for i := 0; i < 10000000; i++ {
     1.73s      1.73s     10: if i%n != m {
         .          .     11: sum += n
         .          .     12: }
         .          .     13: }
         .          .     14: return sum
         .          .     15:}

[Robert Engels]

Also the go program is most likely using 64 bit math. Use int32 to compare it correctly. 

[Robert Engels]

You are using g 64 bit in Go and 32 bit in C

[Milind Thombre]

This is exactly why I asked a day or two ago if there has been a quantitative performance study/Benchmarking study done, but nobody seems to have the time to contribute to it. Are we expected to mindlessly adopt whatever "New" technology is doled out without quantitative proof of its degree of goodness? Agreed there are features that are not even available in other languages, but for common and often used features should we as engineers not ask "Exactly how fast is golang vs say Python/JS?" or even C as this case states.

If a verifiable (unbiased) study is already done and published, I'd greatly appreciate it if someone can post the link. If anyone wants to contribute to a research paper, I'd love to get started as well and can volunteer with creating the Python benchmarking routines initially in my free time. 

Contributors can recreate the same benchmarking routines in C, golang, JS <<add>>.

Do let me know if anyone has second thoughts about the study, can devote time for it or has moneybags for sponsoring such a study

Regards

Milind

[Robert Engels]

I’ll state again, it’s because these benchmarks have little to do with the success of systems. Experienced designers know this. Take, architecture, barring some large scale dynamics, everyone would build simple boxes. It doesn’t mean the project will be a commercial success, most likely not. 

Tying to distill a software design choice down to simple checkboxes is a fools errand. 

[Miki Tebeka]

Yup. Using int32 in Go reduces the difference to 1.5. Thanks

[Robert Engels]

It will also depend on which math library and compiler options you use on the C side. 

I’ll give you a bit of a warning though, if you are making decisions based solely on tests like this you you might be missing the bigger picture of the value of systems like Go. 

A simple example, even 1.5 sounds like a lot, as soon as you do any IO that will fall to about .0000001 percent difference. 

[Milind Thombre]

Right! Just Listen to what ever the Architect's (or City Planner's) opinion is, implement it, and we in all certainty have a performant system. Numerical Evidence is for dummies.... 

Whatever!

[Bakul Shah]

Suggestion: specify all relevant details as we don’t know what kind of machine you have.

I suspect your machine is a 64bit word size machine. C ints are typically 4 bytes even on 64bit word size machines. Go ints are 8 bytes on these machines.

Suggestion2: look at the generated assembly language code for each program. Making sense of compiler output is a very useful skill to learn for benchmarking (& more).

[robert engels]

I will take an experienced architect or city planner with a track record of success over any ‘isolated numerical comparison’. Comparisons need to be made in their entirety, and this is a skill set that often cannot be taught, and is learned via human experience. It is similar to how many AI systems struggle with basic operations that a human child can easily do. 

We as humans have a well honed ability to summarize, categorize and compare that is difficult to replicate.

Inputs, like discrete numerical comparisons, are only a small part of making successful design choices.

[Miki Tebeka]

Thanks. You're right - this is not the way to choose a language. I was just curious. Go has many, many more things going for it - multi core support, networking, standard library, community ...

[Amnon Baron Cohen]

If you like these kind of benchmarks https://benchmarksgame-team.pages.debian.net/benchmarksgame/faster/go-gpp.html

may be what you are looking for. In these results Go is generally half the speed of C, but 50 times faster than python.

Go does not attempt to generate the fastest code at all costs. But the generated code is usually fast enough,
and you gain much faster development and compilation times. Garbage collection, for example, does impose
some runtime overhead. But it frees the programmer from the tedious and error-prone task of manually managing 
memory allocations. If you really care about each nanosecond of execution time, and are prepared to do a lot of work
to minimise run-time, then C, C++ or Rust may be your best choice. But for most of us here, Go does hit the sweetspot.

For those interested, I gave a talk to the Go London meetup last year about the limitations of these kind of benchmarks.
https://youtu.be/YDPKUJndhw4

[Kurtis Rader]

On Sun, Feb 3, 2019 at 10:47 PM Amnon Baron Cohen <amn...@gmail.com> wrote:

If you really care about each nanosecond of execution time, and are prepared to do a lot of work

to minimise run-time, then C, C++ or Rust may be your best choice. But for most of us here, Go does hit the sweetspot.

Bingo! For that matter if shaving nanoseconds matters you should be using assembly language. You won't write a lot of useful code but what you do write might execute fast. Emphasis on "might" since, in my experience, many people who care about micro benchmarks over all other considerations seldom understand related considerations such as CPU branch prediction costs and CPU TLB and cache invalidation. Not to mention considerations that are almost always more important such as the time required to write the code, it's correctness, and maintainability. Which are almost always orders of magnitude more important than whether you have managed to shave every microsecond from its run time.

 -- 

Kurtis Rader

Caretaker of the exceptional canines Junior and Hank

[Robert Engels]

It is even more involved than that. A GC language in a highly concurrent dynamic memory environment can be faster than a hand rolled C or assembly implementation. The important consideration is memory allocation and deallocation costs in the critical path. In a GC environment they are essentially free. In most malloc based ones they are not. It can make a huge difference. Think deallocation of a large tree structure...

--

[David Chase]

A general problem with interlanguage benchmarks is that you can only compare those features that both languages have, so these always reduce to lowest common denominator and are inherently biased against new features.  So for example, go will get no credit for having a garbage collector, checked array and string indexing operations, maps, slices, type assertions, interfaces, reflection, channels, resizeable stacks, etc.

[Wojciech S. Czarnecki]

On Mon, 4 Feb 2019 00:13:39 +0530
Milind Thombre <thom...@gmail.com> wrote:

> ask "Exactly how fast is golang vs say Python/JS?" or even C

It depends on domain, metrics and task at hand, but from personal experience
(Go vs C) Go is about four times faster than C at prototyping phase, two times
faster at doing mainline and tenfold of that faster at maintenance phase.
[Domain: CAM, task: telemetry gathering, metrics: time from whiteboard to code]

> If anyone wants to contribute to a research paper, I'd love to get started as well
> and can volunteer with creating the Python benchmarking routines

If we're at CPython what a **routine** may possibly compare? Python C runtime
piece vs Go's? (So using Python you'd measure C vs Go, as Go runtime is almost
pure go)

> Contributors can recreate the same benchmarking routines in C, golang, JS

This is how "benchmark game" **entertaining** sites are architectured. Their
"comparisons" are moot for the industry. Look at the history of java samples
and benchmark earlier vs current at same task.

The real knowledge one may acquire at such benchmark game sites is how
much dark craft any given language demands from its disciples.
And in this regard, IMO, Go is the least demanding. Could be ex aequo with
BASIC and Pascal.

> Do let me know if anyone has second thoughts about the study,

The "language1 vs language2" comparison to be of real value would need
separate expert teams to write a piece of sophisticated software from the scratch
then to deploy it then measure it on the field with same data feed.
Even then the chosen domain inevitably would bring a bias were
stdlibs allowed.

Hope this helps.

--
Wojciech S. Czarnecki
<< ^oo^ >> OHIR-RIPE

[David Skinner]

For me, my C code is very micro-efficient so it generally runs faster than Go, but it takes me up to 4 to 5 times longer to write things in C than it does to write things in Go. If the resulting Go code is not fast enough, I have three solutions:

1. Run the profiler to find the bottleneck, sometimes I can fix it.
2. Complain about the performance on GoNuts and when Go updates, my code usually ends up running faster. The devs are working hard so we don't have too.
3. Give the code to my eldest son Daniel, he is a much better Go programmer than I am and he can optimize the hell out of any bottlenecks.
   
   

When I wrote my first C program in 1988 or 1989, it was a true color video card driver for IBM and micro-efficiency was very important.  Before that I used Forth, MacroASM, UniComal on Fortan77 Runtime. Today, it is rarely important and the profiler can be used as a tool for cost justification for any efforts to speed up a program, but mostly, it is not much of an issue anymore for me. My concern is time to complete the project and cost to complete the project. If it is a small poorly defined project, a talented PHP programmer can interface with the stakeholder and complete the project almost as fast as a Go programmer for much less money because here in Arkansas PHP programmers are being train in the local colleges and are relatively cheap while experience Go programmers want three times as much money for their time. For a larger well defined project Go is just much easier to partition out the work amongst a team working parallel. And for me, personally, Go is just much easier to be productive.

[Isaac Gouy]

On Sunday, February 3, 2019 at 10:44:11 AM UTC-8, Milind Thombre wrote:

… a quantitative performance study/Benchmarking study … If a verifiable (unbiased) study is already done and published, I'd greatly appreciate it if someone can post the link.

What do you not find acceptable about what Google finds?

[Isaac Gouy]

On Tuesday, February 5, 2019 at 3:03:42 AM UTC-8, ohir wrote:

> Contributors can recreate the same benchmarking routines in C, golang, JS

This is how "benchmark game" **entertaining** sites are architectured. Their
"comparisons" are moot for the industry. Look at the history of java samples
and benchmark earlier vs current at same task.

"Look at the history of java samples and benchmark earlier vs current at same task" and conclude what?

 

The "language1 vs language2" comparison to be of real value would need
separate expert teams to write a piece of sophisticated software from the scratch
then to deploy it then measure it on the field with same data feed.
Even then the chosen domain inevitably would bring a bias were
stdlibs allowed.

So, like — [pdf] "Plat_Forms: A Web Development Platform Comparison by an Exploratory Experiment Searching for Emergent Platform Properties"

[Wojciech S. Czarnecki]

On Sat, 9 Feb 2019 09:30:24 -0800 (PST)
"'Isaac Gouy' via golang-nuts" <golan...@googlegroups.com> wrote:

> > *"Look at the history of java samples and benchmark earlier vs current at
> > same task"*
> and conclude what?

That some languages demands very deep intrinsic knowledge to get code
running faster than in its most naïve — aka comprehensible — form.
And that there are languages that tends to have no or almost no tricks under
the hood that allows for shortcuts to faster code.

> > Even then the chosen domain inevitably would bring a bias

> So, like [platforms07-tse-2010.pdf]

Yes, indeed :)

Kind regards,

[Isaac Gouy]

> Nothing, so far. But I feel Independent studies are necessary to confirm or refute claims.

"If you're interested in something not shown on the benchmarks game website then please take the program source code and the measurement scripts and publish your own measurements."

[Isaac Gouy]

On Saturday, February 9, 2019 at 9:30:25 AM UTC-8, Isaac Gouy wrote:

On Tuesday, February 5, 2019 at 3:03:42 AM UTC-8, ohir wrote:

> Contributors can recreate the same benchmarking routines in C, golang, JS

This is how "benchmark game" **entertaining** sites are architectured. Their
"comparisons" are moot for the industry. Look at the history of java samples
and benchmark earlier vs current at same task.

"Look at the history of java samples and benchmark earlier vs current at same task" and conclude what?

> Robert Engels wrote:The point is that when you leverage a runtime, you often get performance improvements as the runtime improves with no work on your end. Java has seen 1000x performance improvements in many areas as compared to version 1 - mostly due to the JIT. 

Ummm yeah, until language implementations are abandoned, people will try to make improvements — https://benchmarksgame-team.pages.debian.net/benchmarksgame/faster/yarv-mri.html

[Isaac Gouy]

On Monday, February 11, 2019, 10:59:20 AM PST, Robert Engels  wrote:

Please don’t base any decisions on the benchmark games. They are seriously flawed.

Someone could write "They are seriously flawed." without ever having looked to see :-)

[Henry]

Micro optimization is only a small part of the picture. While you can shave off a few ms with micro optimization, you can save much more with macro optimization. Macro optimization deals with algorithm, program design, caching, etc. This is why some people consider language vs language micro-benchmark to be irrelevant, because it plays only a very minor part. My colleague is a Haskell guru. Despite Haskell being cast as slow and inefficient in various micro benchmarks, his real-world Haskell programs are very fast that you wouldn't believe they are written in Haskell. He often come up with ingenious solutions to problems. I think Haskell tend to attract bright individuals and programming theoreticians.

I find it funny that people now use C as the golden standard to measure performance. Back then people used to think that C was slow compared to ASM. In fact, C is still slow compared to well-written ASM code. Often, C++ compilers are able to produce much more efficient code compared to C, thanks to some C++ constructs that allow the compilers to make better optimization decision. So C isn't really the fastest programming language out there.

While micro benchmark may give valuable feedback, do not use it as the sole measure to judge a programming language's merits. At the end of the day, it is the carpenter who makes the furniture and not his fancy hammer.

[Nigel Tao]

On Mon, Feb 4, 2019 at 5:34 AM Miki Tebeka <miki....@gmail.com> wrote:
> A bit of profiling shows that the modulo operator takes most of the time:

C's modulo operator is faster, but can crash if you look at it funny.

$ cat x.c
#include <stdio.h>

int main(int argc, char** argv) {
int x = -2147483648;
int y = -1;
printf("x=%d\n", x);
printf("y=%d\n", y);
printf("m=%d\n", x % y);
return 0;
}
$ gcc x.c && ./a.out
x=-2147483648
y=-1
Floating point exception

Compare it to the output of https://play.golang.org/p/Yj2RZmB7ZRI

Yes, both the Go code and the C code will panic if y is zero. Still,
"-2147483648 % -1" has a sensible mathematical definition (zero), and
C fails to calculate it.

[Nigel Tao]

On Wed, Feb 13, 2019 at 11:23 AM Nigel Tao <nige...@golang.org> wrote:
> Yes, both the Go code and the C code will panic if y is zero. Still,
> "-2147483648 % -1" has a sensible mathematical definition (zero), and
> C fails to calculate it.

I forgot to mention that, even with a mod-by-zero, Go lets you recover
the panic, instead of just letting the hardware issue a raw SIGFPE.
Explicitly checking y==0 (in Go's generated machine code) requires
some up-front computation, compared to the C code.

https://godbolt.org/z/3EPkq1 might be instructive (haha).

[Bakul Shah]

According to the C standard:

If the quotient a/b is representable, the
expression (a/b)*b + a%b shall equal a.

In your example, dividing x by -1 would make x +2147483648,
which is not representable in int32_t. May be this is
why -2147483648/-1 also exceptions out?

Nitpicking: C's % is a remainder operator, not modulus.