In case you missed it: language benchmarks for Go 1.7, and language adoption

[Eric Johnson]

Not that I think these account for much, but sort of fun to point at:

https://benchmarksgame.alioth.debian.org/u64q/go.html

(Short summary - now with Go 1.7, Go is faster for most benchmarks.)

And then, for language adoption, the TIOBE language index for August of 2016:

http://www.tiobe.com/tiobe-index/

(Note that the above is updated every six months, and I've not been able to find a link to previous versions of the reports, however, you can look for the trend for Go itself http://www.tiobe.com/tiobe-index/go/ ).

Go finally breaks into the top twenty.

As with any metrics, these are selective measures, and not really good indicators by themselves. But at least they're positive with respect to Go.

Eric.

[Ian Lance Taylor]

On Mon, Aug 29, 2016 at 4:51 PM, 'Eric Johnson' via golang-nuts
<golan...@googlegroups.com> wrote:
>
> And then, for language adoption, the TIOBE language index for August of
> 2016:
> http://www.tiobe.com/tiobe-index/
>
> (Note that the above is updated every six months, and I've not been able to
> find a link to previous versions of the reports, however, you can look for
> the trend for Go itself http://www.tiobe.com/tiobe-index/go/ ).
> Go finally breaks into the top twenty.

It's because they changed their definition of Go. Quoting from
farther down the page: "The restriction "Google" has been removed from
the search queries for the programming language Go. Ilja Heilager
sorted out that without the search term "Google" the resulting Go hits
are still referring to the Go language. After having removed this
restriction Go jumped from position #55 to #20. Thanks Ilja!"

Ian

[Eric Johnson]

Well, sure. I read that too. I think it is still useful that my intuition that Go is gaining adoption aligns with what prominent metrics are reporting, even if it is because the reporting changed how they measured.

Eric.

[Asit Dhal]

Hi,

Go1.7  is faster in most benchmarks, but still slower than Java in some benchmarks(like Go 1.6).

GO Garbage Collector needs time to become mature like JVM.

K-nucleotide, binary tree, Regex-dna are bad for GO(lack of fast GC and good standard libraries).

But, for me, Go is an awesome substitute of Python and C++.

Warm Regards,

Asit

http://asit-dhal.github.io 

[Torsten Bronger]

Hallöchen!

'Eric Johnson' via golang-nuts writes:

> Not that I think these account for much, but sort of fun to point at:
>
> https://benchmarksgame.alioth.debian.org/u64q/go.html
> (Short summary - now with Go 1.7, Go is faster for most benchmarks.)
>
> And then, for language adoption, the TIOBE language index for August of
> 2016:
> http://www.tiobe.com/tiobe-index/

I don't think TIOBE is useful beyond position 10 or so. Anyway ...

https://www.openhub.net/languages/compare?utf8=%E2%9C%93&measure=commits&language_name%5B%5D=c&language_name%5B%5D=golang&language_name%5B%5D=java&language_name%5B%5D=rust&language_name%5B%5D=-1&commit=Update

is limited to open source, but covers a big corpus. Besides,
"monthly commits" is a very expressive, clearly defined indicator.
Obviously, C is on a steep decline in the OSS world, while Go (you
may switch off the C line to see it better) is gaining ground
steadily, and doing so faster than Rust.

Tschö,
Torsten.

--
Torsten Bronger Jabber ID: torsten...@jabber.rwth-aachen.de

[Isaac Gouy]

On Tuesday, August 30, 2016 at 1:01:51 AM UTC-7, Torsten Bronger wrote:

 

Obviously, C is on a steep decline in the OSS world, while Go (you
may switch off the C line to see it better) is gaining ground
steadily, and doing so faster than Rust.

Well, obviously, we find JavaScript on the rise,  but I admit to checking several others before finding a partial-match to that most-recent C down-tick --

https://www.openhub.net/languages/compare?utf8=%E2%9C%93&measure=commits&language_name%5B%5D=c&language_name%5B%5D=golang&language_name%5B%5D=javascript&language_name%5B%5D=python&language_name%5B%5D=-1&commit=Update

What fun!

[Scott Pakin]

On Monday, August 29, 2016 at 5:51:42 PM UTC-6, Eric Johnson wrote:

Not that I think these account for much, but sort of fun to point at:

https://benchmarksgame.alioth.debian.org/u64q/go.html

(Short summary - now with Go 1.7, Go is faster for most benchmarks.)

Go 1.7 is faster than C on the mandelbrot test and faster than C++ also on reverse-complement?  How did that happen?

— Scott

[Torsten Bronger]

Hallöchen!

> also on reverse-complement? How did *that* happen?

For simple things, Go will produce almost the same machine code than
a C/C++ compiler, at least in the inner loops.

Besides, I think in some cases, the quality of the implementation is
tested more than anything else.

[Isaac Gouy]

On Tuesday, August 30, 2016 at 10:09:39 AM UTC-7, Scott Pakin wrote:

Go 1.7 is faster than C on the mandelbrot test and faster than C++ also on reverse-complement?  How did that happen?

mandelbrot -- look at the program source code.

[Michael Jones]

I looked at that this morning and sped it up a little.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Isaac Gouy]

I looked at that this morning and sped it up a little.

fyi "Contribute your programs"

[Eric Johnson]

I looked at the k-nucleotide program, and was unable to figure out a way
to make it faster.

Most of the time appears to be spent on this one line:
pointer, ok := counts[key]

pprof reports:
19080ms 56.50% 56.50% 22100ms 65.44% runtime.mapaccess2_fast64

This is, of course, exactly what the test is suppose to be checking -
the speed of the built in map. Anyone else have any insight?

Eric.

On 8/30/16 3:46 PM, Michael Jones wrote:
>
> I looked at that this morning and sped it up a little.
>

> *From: *'Isaac Gouy' via golang-nuts <golan...@googlegroups.com>
> *Reply-To: *Isaac Gouy <igo...@yahoo.com>
> *Date: *Tuesday, August 30, 2016 at 12:15 PM
> *To: *golang-nuts <golan...@googlegroups.com>
> *Subject: *[go-nuts] Re: In case you missed it: language benchmarks

> for Go 1.7, and language adoption
>
>
>
> On Tuesday, August 30, 2016 at 10:09:39 AM UTC-7, Scott Pakin wrote:
>
> Go 1.7 is faster than C on the mandelbrot test and faster than

> C++ also on reverse-complement? How did /that/ happen?

>
>
> mandelbrot -- look at the program source code.
>
> --
> You received this message because you are subscribed to the Google
> Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to golang-nuts...@googlegroups.com

> <mailto:golang-nuts...@googlegroups.com>.

> For more options, visit https://groups.google.com/d/optout.
>

> --
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/golang-nuts/dyq3z5dWqrc/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> golang-nuts...@googlegroups.com
> <mailto:golang-nuts...@googlegroups.com>.

[Harald Weidner]

Hello,

On Tue, Aug 30, 2016 at 04:41:29PM -0700, 'Eric Johnson' via golang-nuts wrote:

> I looked at the k-nucleotide program, and was unable to figure out a way to
> make it faster.

> This is, of course, exactly what the test is suppose to be checking - the
> speed of the built in map. Anyone else have any insight?

The Java counterpart of this benchmark does not use the Java build-in
maps, but imports a map implementation for fixed data types from the
fastutil project.

http://fastutil.di.unimi.it/

Those libraries are based on generated source code. I think something
similar could be built for Go.

Harald

[Torsten Bronger]

Hallöchen!

YMMV, but if such tests are not written ideomatically, they are
useless in my opinion.

[Nicola Larosa (tekNico)]

Speaking of adoption, here's another awesome graph from OpenHub: :-)

<https://www.openhub.net/languages/compare?language_name%5B%5D=dmd&language_name%5B%5D=erlang&language_name%5B%5D=golang&language_name%5B%5D=-1&language_name%5B%5D=rust&language_name%5B%5D=scala&language_name%5B%5D=-1&measure=commits>


--
Nicola 'tekNico' Larosa <https://www.tekNico.net/>

[Peter Herth]

I am not sure I agree. Writing them in idiomatic might be good to detect whether idiomatic works for getting fast code, but on the other side, if I am writing a large application, I might use slower, idiomatic code in 90% of it, and then optimize the hell out of the bottlenecks, throwing any elegance over board :). And, as this very benchmarking site is far too often abused to talk down on languages, just because they have some bad test results, we as the Go community should try to get the fastest results there - most other programs are not very idiomatic or reasonable even. 

Peter

[Torsten Bronger]

Hallöchen!

Peter Herth writes:

> I am not sure I agree. Writing them in idiomatic might be good to
> detect whether idiomatic works for getting fast code, but on the
> other side, if I am writing a large application, I might use
> slower, idiomatic code in 90% of it, and then optimize the hell
> out of the bottlenecks, throwing any elegance over board :).

There is no clear line, but then, "ideomatic" is not clearly defined
either. Turbo Pascal used to have an assembler keyword and would
have been the fastest language of all. :-) Seriously, if you start
generating code as was suggested, the comparison is not helpful
anymore.

[Eric Johnson]

On 8/31/16 2:04 AM, Harald Weidner wrote:
> Hello,
>
> On Tue, Aug 30, 2016 at 04:41:29PM -0700, 'Eric Johnson' via golang-nuts wrote:
>
>> I looked at the k-nucleotide program, and was unable to figure out a way to
>> make it faster.
>> This is, of course, exactly what the test is suppose to be checking - the
>> speed of the built in map. Anyone else have any insight?
> The Java counterpart of this benchmark does not use the Java build-in
> maps, but imports a map implementation for fixed data types from the
> fastutil project.
>
> http://fastutil.di.unimi.it/

I hadn't noticed that. That would seem to violate the spirit of the test:
"The work is to use the built-in or library hash table implementation to
accumulate count values - lookup the count for a key and update the
count in the hash table. Don't optimize away the work."

Eric.

[Harald Weidner]

Hello,

On Wed, Aug 31, 2016 at 08:44:41AM -0700, 'Eric Johnson' via golang-nuts wrote:

> >The Java counterpart of this benchmark does not use the Java build-in
> >maps, but imports a map implementation for fixed data types from the
> >fastutil project.

> I hadn't noticed that. That would seem to violate the spirit of the test:

The regex-dna benchmark is even more strange. Neither the measured Go nor the
Java implementation use the standard regexp libs. The Go program imports
bindings to PCRE, the Java program uses TclRE.

Harald

[Tim Hawkins]

I would have expected that aside from informing about general performance, one of the purposes for benchmarks would have been to create pressure for improvement of key features of the laguage, this seems to circumvent this, im not dure what they are trying to achive. Perhaps restrictions on only using the language as distributed and its standard lib would be a healthy constraint to evangelise for.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.

[Jason E. Aten]

The description of k-nucleotide does specify that the use of third-party hash table libraries is allowed. See the http://benchmarksgame.alioth.debian.org/u64q/knucleotide-description.html#knucleotide :

Some language implementations have hash tables built-in; some provide a hash table as part of a collections library; some use a third-party hash table library. (For example, use either khash or CK_HT for C language k-nucleotide programs.) The hash table algorithm implemented is likely to be different in different libraries.
The work:
The work is to use the built-in or library hash table [emphasis mine] implementation to accumulate count values - lookup the count for a key and update the count in the hash table.

 

I tried substituting my own offheap hashtable, open source/available at https://github.com/glycerine/offheap

The version specialized for uint32->uint32 runs about 25% faster on my laptop. I submitted it here:

https://alioth.debian.org/tracker/index.php?func=detail&aid=315482&group_id=100815&atid=413122

Jason

[Isaac Gouy]

On Wednesday, August 31, 2016 at 3:53:12 PM UTC-7, Harald Weidner wrote:

The regex-dna benchmark is even more strange. Neither the measured Go nor the
Java implementation use the standard regexp libs. The Go program imports
bindings to PCRE, the Java program uses TclRE.

There's more than one Go program shown and one of them uses "regexp".

There's more than one Java program shown and some of them use java.util.regex.

[Isaac Gouy]

On Wednesday, August 31, 2016 at 6:31:05 AM UTC-7, Peter Herth wrote:

And, as this very benchmarking site is far too often abused to talk down on languages, just because they have some bad test results, we as the Go community should try to get the fastest results there - most other programs are not very idiomatic or reasonable even. 

When the benchmarks game is abused to talk down on Go, that's also an opportunity to talk up the benefits of Go.

"… the versions of the C code we've pickled in our repo are not even close to the fastest. They were chosen for clarity and simplicity for easy comparison *by us* with equivalent Go code."

https://groups.google.com/d/msg/golang-nuts/CF5wyVEjEaY/KRgzpjdH3fAJ

Recently the Chapel group did much the same.

[Isaac Gouy]

On Wednesday, August 31, 2016 at 4:47:35 PM UTC-7, Tim Hawkins wrote:

I would have expected that aside from informing about general performance, one of the purposes for benchmarks would have been to create pressure for improvement of key features of the laguage, this seems to circumvent this, im not dure what they are trying to achive. Perhaps restrictions on only using the language as distributed and its standard lib would be a healthy constraint to evangelise for.

Do you think using a replacement for Java collections library is at-all unusual in-practice?

https://projects.eclipse.org/projects/technology.collections

[sascha.l....@googlemail.com]

Have a look at my version  https://bitbucket.org/s_l_teichmann/knucleotide which is nearly on-par with the Java version.

[sascha.l....@googlemail.com]

2016-09-06 22:47 GMT+02:00, Jason E. Aten <j.e....@gmail.com>:
> nice! would you mind releasing under an MIT or BSD license? I think it
> would be

It's MIT licensed now.

> worth posting the hashmap alone as a reusable library.

Maybe. For the shootout I prefer the embedded variant to
to demonstrate it is do-able without resorting to 3rd party libs.

> I hope you submit your code... I would be great to be on par with java all
> around! here's the instructions --
>
> http://benchmarksgame.alioth.debian.org/play.html

I will try. Currently my Go version is ~10% faster than the Java one.

[sascha.l....@googlemail.com]

Am Mittwoch, 7. September 2016 11:12:21 UTC+2 schrieb sascha.l....@googlemail.com:

> I hope you submit your code... I would be great to be on par with java all
> around! here's the instructions --
>
> http://benchmarksgame.alioth.debian.org/play.html

I will try. Currently my Go version is ~10% faster than the Java one.

https://alioth.debian.org/tracker/index.php?func=detail&aid=315484&group_id=100815&atid=413122

[Isaac Gouy]

On Wednesday, September 7, 2016 at 2:12:21 AM UTC-7, sascha.l....@googlemail.com wrote:

Maybe. For the shootout I prefer the embedded variant to
to demonstrate it is do-able without resorting to 3rd party libs.

Unfortunately, k-nucleotide now explicitly requires use of a built-in / library HashMap. Programs that used a custom hash-table implementation have been replaced.

By "library" I don't mean a custom hash table implementation that you've put into a library.

[sascha.l....@googlemail.com]

Okay, I made https://bitbucket.org/s_l_teichmann/fastmap/overview which offers uint64 to int mapping.
There is no special code for k-nukleotide in this library. It is custom in the way that I've wrote it
and it is optimized, but there is no good reason to not use it in other projects, too.

It is Free Software / Open Source covered by the MIT license. So it can be used by other people, too.

If this does not count the Benchmark game follows a skewed defintion of a library.

Can you eleborate please, what a library is? And how do you think does this match to the library definition
in terms of Go?

[Isaac Gouy]

On Wednesday, September 7, 2016 at 8:15:09 AM UTC-7, sascha.l....@googlemail.com wrote:

If this does not count the Benchmark game follows a skewed defintion of a library.

 

I'm sorry that you don't seem to understand what is expected.

Can you eleborate please, what a library is?

 Examples were given in the description:

http://attractivechaos.github.io/klib/#About

http://concurrencykit.org/index.html

[sascha.l....@googlemail.com]

Am Mittwoch, 7. September 2016 18:20:28 UTC+2 schrieb Isaac Gouy:

On Wednesday, September 7, 2016 at 8:15:09 AM UTC-7, sascha.l....@googlemail.com wrote:

If this does not count the Benchmark game follows a skewed defintion of a library.

 

I'm sorry that you don't seem to understand what is expected.

I do. But you still miss the point that 3rd party libs are just software
written by someone. If I follow your logic if some else (not me)
posts my solution (what is possible ... its free software) using my library
it will be accepted. Lets do this.

Taking this benchmark would only be possible if the 3rd party exists.
So you would have forbidden Ken and Dennis to create a benchmark entry for C
in the early days?  Because there were no such libs.
Theoretically speaking .. but I hope you get the point.
 

Can you eleborate please, what a library is?

 Examples were given in the description:

http://attractivechaos.github.io/klib/#About

http://concurrencykit.org/index.html

But this are just two third party libraries, like mine. So where is the difference?
They may be larger, okay. But there is nothing in the rules which says that they
have to be complex and at least a few years old and so on and so on.
If you want to enforce this the description text has to be adjusted.
And no it's not about cheating, it's about software making.

I think an important fact of the Go ecosystem is the usage of small fitting libraries.


BTW: The godoc of my library can be found here:
https://godoc.org/bitbucket.org/s_l_teichmann/fastmap

[Clark Wierda]

Apparently, you can only use approved libraries that produce the ranking he wants.

[Jason E. Aten]

Observe that the java benchmark continues to be allowed to use an accelerated hash table library.

[Isaac Gouy]

On Wednesday, September 7, 2016 at 6:47:59 PM UTC-7, Clark Wierda wrote:

Apparently, you can only use approved libraries that produce the ranking he wants.

Which one of us is the language advocate? :-)

[Isaac Gouy]

On Wednesday, September 7, 2016 at 7:44:28 PM UTC-7, Jason E. Aten wrote:

Observe that the java benchmark continues to be allowed to use an accelerated hash table library.

"accelerated" ?

By all means, post your reasons for why that Java library should not be used in the benchmarks game discussion forum.

Let's have pity on golang-nuts.

[Dan Kortschak]

Can you explain the rationale behind the classification of libraries as
libraries or not libraries? It seems pretty arbitrary.

(I'm interested from a sociological perspective, but not enough to
bother to go to the benchmarks game discussion forum).

On Fri, 2016-09-09 at 15:08 -0700, 'Isaac Gouy' via golang-nuts wrote:
>
> On Wednesday, September 7, 2016 at 7:44:28 PM UTC-7, Jason E. Aten wrote:
> >
> > Observe that the java benchmark continues to be allowed to use an
> > accelerated hash table library.
> >
>
> "accelerated" ?
>

> By all means, post your reasons for *why that Java library should not be
> used* in the benchmarks game discussion forum.

[RogerV]

I replaced C with C++ (mind you not C/C++) and then the curve takes a dramatic upward spike in year 2016 (I'd say it was getting going well in 2015).

IOW, it looks like the efforts to modernize C++, starting with C++11, are paying off big time.

I program mostly in Java these days. But I've introduced Go for some aspects of my current project and it works very well for those situations. And there is a crucial program that I wrote in C++11 - which was a very refreshing experience compared to the C++ of yesteryear.

If I had my personal preferences I'd do the development that am currently doing in Java with Go instead. But there are still a few things that cause me to hesitate. A crucial program in my processing pipeline does a lot of meta programming and it turns out there are superb libraries, such as ByteBuddy, JavaAsssist, and even meta programming capabilities in Jackson CSV/JSON library that are proving crucial. Plus am having to mesh with code that is code generated by a high-level graphical data mapping tool and that tool generates code in Java source form. So the mainline of this project is still being well served by using Java - as much as I wouldn't mind using Go. Go meta programming - well maybe people do that, but Java does it in spades with all manner of tools, libraries, frameworks, and even compilers (I've used a little bit of AspectJ for some things).

I also wrote my own Java launcher program for the Linux platform that I use in place of the Oracle Java launcher, I call it Spartan, and it is written in C++11. It's forte is that it makes it convenient to do forked multi-processing in Java programs and makes it very civilized to write Linux daemons where can invoke commands on the daemon from a bash command line. I'm discovering lots of other interesting things that can do with Spartan to radically revitalize Java programming. Is kind of strange that nothing significant has been done with the technology of the Java launcher since its inception. But re-conceiving that for the Linux platform is a kind of game changer. I intend to release Spartan as open source in 2017.

On Tuesday, August 30, 2016 at 1:01:51 AM UTC-7, Torsten Bronger wrote:

Hallöchen!

'Eric Johnson' via golang-nuts writes:

> Not that I think these account for much, but sort of fun to point at:
>
> https://benchmarksgame.alioth.debian.org/u64q/go.html
> (Short summary - now with Go 1.7, Go is faster for most benchmarks.)
>
> And then, for language adoption, the TIOBE language index for August of
> 2016:
> http://www.tiobe.com/tiobe-index/

I don't think TIOBE is useful beyond position 10 or so.  Anyway ...

https://www.openhub.net/languages/compare?utf8=%E2%9C%93&measure=commits&language_name%5B%5D=c&language_name%5B%5D=golang&language_name%5B%5D=java&language_name%5B%5D=rust&language_name%5B%5D=-1&commit=Update

is limited to open source, but covers a big corpus.  Besides,
"monthly commits" is a very expressive, clearly defined indicator.
Obviously, C is on a steep decline in the OSS world, while Go (you
may switch off the C line to see it better) is gaining ground
steadily, and doing so faster than Rust.

[Isaac Gouy]

On Friday, September 9, 2016 at 6:25:38 PM UTC-7, kortschak wrote:

Can you explain the rationale behind the classification of libraries as
libraries or not libraries? It seems pretty arbitrary.

(I'm interested from a sociological perspective, but not enough to
bother to go to the benchmarks game discussion forum).

From a sociological perspective:

mid-May, contributors of k-nucleotide programs that used a custom written-for-k-nucleotide hash table were notified that those programs would no longer be shown, but programs that used a built-in / library hash table would be acceptable.

Within a few days, a couple of Rust contributors were discussing the concerns they had about the performance of std::collections::HashMap but reached an impasse. The lack of a Rust k-nucleotide program was mentioned a couple of times in the following months.

A week ago, someone posted their idea for a possible implementation to a Rust forum - it was discussed, improved, re-written and two different programs contributed.

One person ("not a native English speaker") thought it was odd to allow:

import it.unimi.dsi.fastutil.longs.Long2IntOpenHashMap;

No one seems to have needed to discuss what was meant by library, like sascha they seem to think they understood what was expected.

[Torsten Bronger]

Hallöchen!

RogerV writes:

> I replaced C with C++ (mind you not C/C++) and then the curve
> takes a dramatic upward spike in year 2016 (I'd say it was getting
> going well in 2015).

I think all the very recent changes in this statistics are due to a
new (set of) repos that were added to the corpus. This is just an
assumption but I would not interpret too much into the far right of
the lines.

Regards,

[Torsten Bronger]

Hallöchen!

'Isaac Gouy' via golang-nuts writes:

> [...]

>
> From a sociological perspective:
>
> mid-May, contributors of k-nucleotide programs that used a custom
> written-for-k-nucleotide hash table were notified that those
> programs would no longer be shown, but programs that used a
> built-in / library hash table would be acceptable.
>

> [...]

>
> One person ("not a native English speaker") thought it was odd to
> allow:
>
> import it.unimi.dsi.fastutil.longs.Long2IntOpenHashMap;
>
> No one seems to have needed to discuss what was meant by library,
> like sascha they seem to think they understood what was expected.

I don't really understand what you want to say with this. My two
cents are that only built-in or *standard* library should be
allowed. This is a) more clearly defined and b) encourages
languages to ship with a good standard library.

Tschö,

[Dan Kortschak]

On Sat, 2016-09-10 at 10:21 -0700, 'Isaac Gouy' via golang-nuts wrote:
> No one seems to have needed to discuss what was meant by library, like
> sascha they seem to think they understood what was expected.

Prior to your earlier post, I thought I did too, but after that many
libraries I have used or written apparently no longer qualify. It would
be nice to know what the definition is.