Go 2.0 without garbage collector?

5,586 views
Skip to first unread message

Archos

unread,
Jul 17, 2014, 4:52:03 AM7/17/14
to golan...@googlegroups.com
Does the Go team has thinked into a next version without the garbage collector?

Go is very cool to use but I need the best possible performance since I've not deep pockets like Google. When you are small, you need try to minimize the cost of the servers.
And Rust does a great balance between a language of high level without a significant loss of performance.

I've been translating some Go projects from Go to Rust but the task is very large, especially since every time there are more interesting projects for me.
So, the fastest way for me will be to build a transpiler from Go to Rust. At least there is some intention to have an optional garbage collection.

However, Go is easier to prototype.

Hasan Yasin Öztürk

unread,
Jul 17, 2014, 5:05:28 AM7/17/14
to golan...@googlegroups.com
What kind of servers/applications are you running? It is very unlikely that Go's garbage collection cost you any extra on your hosting bills whatever infrastructure you are using. Instead of porting applications from language to language, you should do serious profiling to be sure that garbage collection is such a big bottleneck for you. That would be very interesting if it reduces your performance in such levels. Maybe you share some insights about your applications and platform with us?

I am not sure about any plans related to making garbage collection optional. I don't think it will happen.

David Symonds

unread,
Jul 17, 2014, 5:06:21 AM7/17/14
to Archos, golang-nuts

On 17 Jul 2014 18:52, "Archos" <raul...@sent.com> wrote:

> Does the Go team has thinked into a next version without the garbage collector?

Nope.

andrewc...@gmail.com

unread,
Jul 17, 2014, 5:15:16 AM7/17/14
to golan...@googlegroups.com
Rust does not currently have the tooling, simplicity, or the stability of Go. I'd advise you to weigh these trade offs carefully. After rust is more stable, there will still be trade offs making neither language superior for every application.

If garbage collection really is hurting your performance, you should also consider reducing you object/heap allocation rates by re factoring your code if possible.

tomwilde

unread,
Jul 17, 2014, 5:24:52 AM7/17/14
to golan...@googlegroups.com
On Thursday, July 17, 2014 10:52:03 AM UTC+2, Archos wrote:
Does the Go team has thinked into a next version without the garbage collector?

Go is very cool to use but I need the best possible performance since I've not deep pockets like Google. When you are small, you need try to minimize the cost of the servers.
 
If performance is a critical factor in your business then I recommend programming in ANSI C.

If you don't feel like writing C then your next-best bet is Go. If you don't feel like writing Go then do whatever else feels appropriate.

But please realize that there is a tradeoff factor between development-time/simplicity and execution speed.

Russel Winder

unread,
Jul 17, 2014, 5:26:32 AM7/17/14
to Archos, golan...@googlegroups.com
On Thu, 2014-07-17 at 01:52 -0700, Archos wrote:
> Does the Go team has thinked into a next version without the garbage
> collector?
>
> Go is very cool to use but I need the best possible performance since I've
> not deep pockets like Google. When you are small, you need try to minimize
> the cost of the servers.
> And Rust does a great balance between a language of high level without a
> significant loss of performance.

You seem to be making an assumption that good performance mean no GC and
GC mean poor performance. Certainly people in the games industry make
this assumption, sometimes validly but usually because the idea has
become a meme. Without benchmark data to prove GC is the source of the
problem, I see no reason to take on all the hard work of explicit memory
management (even using the RAII approach of C++).

Interestingly, or not, similar debates about garbage collection are
being had on the D language email lists. D though has a way of switching
off garbage collection for those situation where it really is a problem.
Whic is not very often.

> I've been translating some Go projects from Go to Rust but the task is very
> large, especially since every time there are more interesting projects for
> me.
> So, the fastest way for me will be to build a transpiler from Go to Rust.
> At least there is some intention to have an optional garbage collection.
>
> However, Go is easier to prototype.
>

--
Russel.
=============================================================================
Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel...@ekiga.net
41 Buckmaster Road m: +44 7770 465 077 xmpp: rus...@winder.org.uk
London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
signature.asc

Robert Melton

unread,
Jul 17, 2014, 5:31:50 AM7/17/14
to Archos, golang-nuts
On Thu, Jul 17, 2014 at 4:52 AM, Archos <raul...@sent.com> wrote:
> I've been translating some Go projects from Go to Rust but the task is very
> large, especially since every time there are more interesting projects for
> me.
> So, the fastest way for me will be to build a transpiler from Go to Rust. At
> least there is some intention to have an optional garbage collection.

You are moving production work to Rust 0.11.0? That seems (to put it
mildly) extremely unwise... even the biggest fans of it agree it isn't
ready (http://blog.octayn.net/blog/2014/07/15/state-of-rust-0.11.0/)
or even done. Did you profile your code? Did you look at it with
gcvis (http://dave.cheney.net/2014/07/11/visualising-the-go-garbage-collector,
which is awesome!)? How exactly did you come to the conclusion that
Go GC is a bottleneck, and what did you do to try to reduce
allocations?

--
Robert Melton | http://robertmelton.com

Archos

unread,
Jul 17, 2014, 5:38:33 AM7/17/14
to golan...@googlegroups.com, raul...@sent.com

El jueves, 17 de julio de 2014 10:26:32 UTC+1, Russel Winder escribió:
You seem to be making an assumption that good performance mean no GC and
GC mean poor performance. Certainly people in the games industry make
this assumption, sometimes validly but usually because the idea has
become a meme.

Russel, video game development has several constraints that aren't commonly found in other development.

Traditional garbage collected languages, whilst achieving high throughput, often struggle with predictable latencies as when GC kicks in, particularly in Stop-The-World collectors.

Targeting 30fps, you have 33ms to render whole frame; Losing 20 ms of that to GC would make things challenging. And if you're targeting 60fps, that's 16ms; stop the world for 20ms, and you've missed 1 and 1/4 of a frame.

Archos

unread,
Jul 17, 2014, 5:42:45 AM7/17/14
to golan...@googlegroups.com, raul...@sent.com
 The first stable version (1.0) will be ready before of finish this year. At the first, I'm supposed that there will a version 0.12 and then some beta ones.

Archos

unread,
Jul 17, 2014, 6:09:20 AM7/17/14
to golan...@googlegroups.com
Well, I'll try to do profiling and I'll use gcvis (thanks Robert Melton by the link).
Thanks to all!

Archos

unread,
Jul 17, 2014, 6:10:27 AM7/17/14
to golan...@googlegroups.com
I discard development in C/C++ because they are unsafe languages and the development time is greater.

Jesper Louis Andersen

unread,
Jul 17, 2014, 6:15:13 AM7/17/14
to Archos, golang-nuts

On Thu, Jul 17, 2014 at 11:38 AM, Archos <raul...@sent.com> wrote:
Traditional garbage collected languages, whilst achieving high throughput, often struggle with predictable latencies as when GC kicks in, particularly in Stop-The-World collectors.

This is only true for naive garbage collectors. The OCaml garbage collector is usually very good at achieving low-latency operation even in the presence of large heaps. But granted, it doesn't stop the world for long and iteratively collects on minor heap collections.

Another example is Erlang, where each process has its own heap and thus the collection times are usually sub-ms. It provides very good soft realtime GC properties, which should be fine for games. OTOH, the language proper is interpreted, so reaching the revered 16ms per frame is going to be hard for compute-intensive work.

In this situation however, I would definitely measure where the trouble lies. There are numerous ways to lower allocation rates in tight loops and you only need a single bad performer. The difference is that in a GC'ed language, your program will actually work, albeit with pauses, whereas in a language like C++, your program will just exhaust memory and die.


--
J.

alexru...@gmail.com

unread,
Jul 17, 2014, 6:47:04 AM7/17/14
to golan...@googlegroups.com, raul...@sent.com
четверг, 17 июля 2014 г., 12:26:32 UTC+3 пользователь Russel Winder написал:
On Thu, 2014-07-17 at 01:52 -0700, Archos wrote:
Certainly people in the games industry make
this assumption, sometimes validly but usually because the idea has
become a meme. 
And yet almost every modern game has GC somewhere inside (many games use some scripting engine for game logic, i.e. Lua, and there are many games written in C#, Java, ActionScript), 

Archos

unread,
Jul 17, 2014, 7:11:08 AM7/17/14
to golan...@googlegroups.com, raul...@sent.com, alexru...@gmail.com


El jueves, 17 de julio de 2014 11:47:04 UTC+1, alexru...@gmail.com escribió:
четверг, 17 июля 2014 г., 12:26:32 UTC+3 пользователь Russel Winder написал:
On Thu, 2014-07-17 at 01:52 -0700, Archos wrote:
Certainly people in the games industry make
this assumption, sometimes validly but usually because the idea has
become a meme. 
And yet almost every modern game has GC somewhere inside (many games use some scripting engine for game logic, i.e. Lua,
A small embedded scripting language, mainly Lua, is used to write the game logic but it does not affect the processing of entire frames.
 
and there are many games written in C#, Java, ActionScript), 
But those games are not real-time games running at 30 or 60fps.
These are other requirements mean that most high level AAA games have to be written to be very highly performant, and in particular, to have consistent latencies.

Jesse McNelis

unread,
Jul 17, 2014, 7:11:55 AM7/17/14
to Archos, golang-nuts


On 17 Jul 2014 19:38, "Archos" <raul...@sent.com> wrote:
>
> Targeting 30fps, you have 33ms to render whole frame; Losing 20 ms of that to GC would make things challenging. And if you're targeting 60fps, that's 16ms; stop the world for 20ms, and you've missed 1 and 1/4 of a frame.
>

Go is not the language for this. High performance gaming is not the target. Go without a GC would basically be Rust and have the difficulties associated with that level of resource management. You can't actually make GC optional in a programming language because it completely changes the design of APIs.

But most gaming isn't that high performance. The best selling PC game of all time is a first person 3D game written in Java.

alexru...@gmail.com

unread,
Jul 17, 2014, 8:31:15 AM7/17/14
to golan...@googlegroups.com, raul...@sent.com, alexru...@gmail.com
четверг, 17 июля 2014 г., 14:11:08 UTC+3 пользователь Archos написал:
A small embedded scripting language, mainly Lua, is used to write the game logic but it does not affect the processing of entire frames.
Of course it does, it's not some magic.
 
 But those games are not real-time games running at 30 or 60fps.
You are seriously wrong. There are tons of "realtime" games (some are very famous and popular) written in languages with GC.

Archos

unread,
Jul 17, 2014, 8:37:37 AM7/17/14
to golan...@googlegroups.com, raul...@sent.com, alexru...@gmail.com


El jueves, 17 de julio de 2014 13:31:15 UTC+1, alexru...@gmail.com escribió:
четверг, 17 июля 2014 г., 14:11:08 UTC+3 пользователь Archos написал:
A small embedded scripting language, mainly Lua, is used to write the game logic but it does not affect the processing of entire frames.
Of course it does, it's not some magic.
Whether the usage of a embedded scripting language (garbaged) does not affect to the processing of frames is because such processing is done with a language no-garbaged (usually C++).
 
 
 But those games are not real-time games running at 30 or 60fps.
You are seriously wrong. There are tons of "realtime" games (some are very famous and popular) written in languages with GC.

 Ok, tell me three real-time games built with a language without GC, and of course without bindings to a language garbaged.

Archos

unread,
Jul 17, 2014, 8:39:05 AM7/17/14
to golan...@googlegroups.com, raul...@sent.com, alexru...@gmail.com


El jueves, 17 de julio de 2014 13:37:37 UTC+1, Archos escribió:
El jueves, 17 de julio de 2014 13:31:15 UTC+1, alexru...@gmail.com escribió:
четверг, 17 июля 2014 г., 14:11:08 UTC+3 пользователь Archos написал:
You are seriously wrong. There are tons of "realtime" games (some are very famous and popular) written in languages with GC.

 Ok, tell me three real-time games built with a language without GC, and of course without bindings to a language garbaged.

I mean:

Tell me three real-time games built with a language WITH GC, and of course without bindings to a language garbaged.

Archos

unread,
Jul 17, 2014, 8:47:26 AM7/17/14
to golan...@googlegroups.com, raul...@sent.com, alexru...@gmail.com
Sorry, I was wrong again :(
To make it clear, please

Tell me three real-time games built with a language with GC (i.e. Java), and of course, without bindings to a language without GC (i.e. C++).

Steve McCoy

unread,
Jul 17, 2014, 8:48:43 AM7/17/14
to golan...@googlegroups.com, raul...@sent.com, alexru...@gmail.com
Every game that uses the Unity game engine uses garbage-collected C#, including Gone Home, Hearthstone, and Kerbal Space Program.

Steve McCoy

unread,
Jul 17, 2014, 8:58:26 AM7/17/14
to golan...@googlegroups.com, raul...@sent.com
It's easy to make GC sound bad if you fabricate runtime numbers.

Archos

unread,
Jul 17, 2014, 9:05:57 AM7/17/14
to golan...@googlegroups.com, raul...@sent.com, alexru...@gmail.com
I'm asking about AAA real-time games
Unity is right to build indie games to be used in mobile and tablet devices, or in desktops.

As reference about AAA games, you have Tomb Raider (2013), Bioshock Infinite, and Skyrim from Steam games.

alexru...@gmail.com

unread,
Jul 17, 2014, 9:46:29 AM7/17/14
to golan...@googlegroups.com, raul...@sent.com, alexru...@gmail.com
четверг, 17 июля 2014 г., 15:37:37 UTC+3 пользователь Archos написал:
Whether the usage of a embedded scripting language (garbaged) does not affect to the processing of frames is because such processing is done with a language no-garbaged (usually C++).
Game logic is the part of "frame processing", and it's usually done with the scripting engine, which produces garbage while working, which gets collected by GC. It's as simple as that.

четверг, 17 июля 2014 г., 15:47:26 UTC+3 пользователь Archos написал:
Tell me three real-time games built with a language WITH GC, and of course without bindings to a language garbaged.
This "without bindings" requirement is nonsensical, because without bindings you usually cannot use APIs and middleware. And even if you have those bindings - GC is still there.

Just some from my Steam library:
Bastion (С#)
The Binding of Isaac (Flash)
Dust: An Elysian Tail С#)
FEZ (С#)
Magicka (С#)
Droid Assault (Java)
Ultratron (Java)
Rogue Legacy (С#)
Terraria (С#)

Just look at any realtime Unity/XNA game.

четверг, 17 июля 2014 г., 16:05:57 UTC+3 пользователь Archos написал:
I'm asking about AAA real-time games
AAA was never mentioned. And most games are not AAA. And most AAA games use scripting engines, therefore GC.




Steve McCoy

unread,
Jul 17, 2014, 10:07:51 AM7/17/14
to golan...@googlegroups.com, raul...@sent.com, alexru...@gmail.com
No, you weren't. And all three of those AAA games run on desktop computers, so I no longer have any idea what you are really asking. You keep saying "real-time", but none of those games have anything more than soft real-time requirements, just like the ones I listed. They all target 30 or 60fps, and even the ones you listed (especially Skyrim) drop frames as a matter of course.

Carlos Castillo

unread,
Jul 17, 2014, 10:33:30 AM7/17/14
to golan...@googlegroups.com, raul...@sent.com, alexru...@gmail.com
I would hardly consider Hearthstone to be "indie".

Also, any game built with XNA is using .Net, and those games run on the Xbox 360. The advice posted here about managing memory w.r.t. GC mostly apply to go as well: http://xnafinalengine.codeplex.com/wikipage?title=XBOX%20360%20Garbage%20Collections

Michael Banzon

unread,
Jul 17, 2014, 10:34:02 AM7/17/14
to golang-nuts
This "discussion" is crazy.

It should NOT be about games - the initial statement was about using server resources on GC causing the cost to move beyond acceptable - so can we _please_ stop with the games-*ock-measuring? ;-)

Please take the initial advice and profile - GC-cost should be ruining your server budget - on the services we run (transportning and calculating a few GB of data around some databases every hour) I think GC accounts for ~3% CPU time. If you find that the GC is in fact the cause of increased cost let's work on that problem.

On another note: Go is "a GC'ed language" - it is one of the reasons we chose it and if that decision is reversed (by the initial request) we would be very unhappy, and our productivity would decrease dramatically. I guess that this is also the fact for many other companies using Go.


--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--
Michael Banzon
http://michaelbanzon.com/

Alexei Sholik

unread,
Jul 17, 2014, 10:59:21 AM7/17/14
to Archos, golang-nuts
Just use Rust.

Go - GC - <x> + <y> = Rust

<x> and <y> are intentionally kept secret


On Thu, Jul 17, 2014 at 11:52 AM, Archos <raul...@sent.com> wrote:
Does the Go team has thinked into a next version without the garbage collector?

Go is very cool to use but I need the best possible performance since I've not deep pockets like Google. When you are small, you need try to minimize the cost of the servers.
And Rust does a great balance between a language of high level without a significant loss of performance.

I've been translating some Go projects from Go to Rust but the task is very large, especially since every time there are more interesting projects for me.
So, the fastest way for me will be to build a transpiler from Go to Rust. At least there is some intention to have an optional garbage collection.

However, Go is easier to prototype.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--
Best regards
Alexei Sholik

Michael Jones

unread,
Jul 17, 2014, 11:02:47 AM7/17/14
to Michael Banzon, golang-nuts
Go = GC ⇒ ¬GC = ¬Go

Valid questions are more in the realm of:

a. How to make a program more efficient under GC. (Generate less garbage being one idea)

b. How to make GC faster or have a strategy to avoid long pauses.

c. How to make GC more compatible with specific requirements (maybe arenas so a fast frame goroutine can run with data in its own arena and not be stopped during STW for the main arena...of course this would mean importing and exporting data to that arena.)
Michael T. Jones | Chief Technology Advocate  | m...@google.com |  +1 650-335-5765

Hasan Yasin Öztürk

unread,
Jul 17, 2014, 12:27:23 PM7/17/14
to golan...@googlegroups.com, mic...@banzon.dk


On Thursday, July 17, 2014 9:34:02 PM UTC+7, mbanzon wrote:
This "discussion" is crazy.

It should NOT be about games - the initial statement was about using server resources on GC causing the cost to move beyond acceptable - so can we _please_ stop with the games-*ock-measuring? ;-)


I strongly agree! At first, the original post was mentioned how his pockets were not as deep as Google so he was considering moving some of his projects to Rust to save some of the money spent on servers. Then, some other people mentioned a case where GC can be problematic: game development. Then the original poster came back and started talking about dropped frames and how many frames he has to catch in games and I was: HUH?

Seriously, I don't know what is discussed anymore. I guess this post has to be sent to oblivion and everybody should return to their real problems such as legitimately washing your mug after using it for days with a fake rinse, walking in the mornings, focusing on code, improving the ability to stand against the urge to jump on helping the solution of a problem that that doesn't exist.

My brain has started dropping frames much faster than Skyrim does after level 2013938238823. Now it is time to install some new servers with my Google credits.





DV

unread,
Jul 17, 2014, 12:37:36 PM7/17/14
to golan...@googlegroups.com, raul...@sent.com, alexru...@gmail.com
I have recently started dabbling with the Unreal Engine development kit. You might know the Unreal Engine from almost every single AAA FPS game currently on the market. The language you write your game logic in for it is C++, or Epic's "dialect" of it. You know what they implemented for their C++ framework? Garbage collection. 

Brad Fitzpatrick

unread,
Jul 17, 2014, 12:45:59 PM7/17/14
to Archos, golang-nuts
You should work on an automatic Go to Rust transpiler!



On Thu, Jul 17, 2014 at 1:52 AM, Archos <raul...@sent.com> wrote:
Does the Go team has thinked into a next version without the garbage collector?

Go is very cool to use but I need the best possible performance since I've not deep pockets like Google. When you are small, you need try to minimize the cost of the servers.
And Rust does a great balance between a language of high level without a significant loss of performance.

I've been translating some Go projects from Go to Rust but the task is very large, especially since every time there are more interesting projects for me.
So, the fastest way for me will be to build a transpiler from Go to Rust. At least there is some intention to have an optional garbage collection.

However, Go is easier to prototype.

bsr

unread,
Jul 17, 2014, 12:54:04 PM7/17/14
to golan...@googlegroups.com, raul...@sent.com

Dmitry Vyukov

unread,
Jul 17, 2014, 12:58:16 PM7/17/14
to Archos, golang-nuts
On Thu, Jul 17, 2014 at 12:52 PM, Archos <raul...@sent.com> wrote:
> Does the Go team has thinked into a next version without the garbage
> collector?
>
> Go is very cool to use but I need the best possible performance since I've
> not deep pockets like Google. When you are small, you need try to minimize
> the cost of the servers.
> And Rust does a great balance between a language of high level without a
> significant loss of performance.
>
> I've been translating some Go projects from Go to Rust but the task is very
> large, especially since every time there are more interesting projects for
> me.
> So, the fastest way for me will be to build a transpiler from Go to Rust. At
> least there is some intention to have an optional garbage collection.
>
> However, Go is easier to prototype.


It seems that what you are asking for is a concurrent/pauseless GC.
Eventually Go will have it. I mean Go1.
You also seem to assume that GC is necessary slower than manual memory
management. This is not necessary true. There are several reasons why
GC can be faster:
1. To track object lifetime in a concurrent setting you need to resort
to something like atomic reference counting, which is costly. While in
a GC environment you have it for free.
2. GC batches free operations which makes them significantly more
efficient than assorted one-by-one free operations.
3. In a GC environment you can have bump-the-pointer allocator which
is way more efficient than segregated allocators that languages with
manual object lifetime tracking have to use.
4. GC can improve locality and defragment heap by moving objects.
5. In a GC environment you can combine several objects into a single
allocation, reducing cost of allocation, free and memory consumption.
Something that languages with manual object lifetime tracking cannot
do.
So I would say that Go1 just needs a better GC/allocator.

Archos

unread,
Jul 17, 2014, 2:57:36 PM7/17/14
to golan...@googlegroups.com, raul...@sent.com
Only to have your project Calimstore built in Rust, it would be worth the effort.

Archos

unread,
Jul 17, 2014, 2:59:00 PM7/17/14
to golan...@googlegroups.com, raul...@sent.com
 Very interesting, thanks!

Archos

unread,
Jul 17, 2014, 3:03:54 PM7/17/14
to golan...@googlegroups.com, raul...@sent.com, alexru...@gmail.com
I didn't know about it, but I'm sure that it's only used for the game logic. I doubt that it been used with the processing of video frames.

akwillis

unread,
Jul 17, 2014, 8:30:16 PM7/17/14
to golan...@googlegroups.com
Most rigs built for gaming have ample memory and more than enough computational power. I'd rather waste a bit of memory/computation by using a gc than actually managing that memory myself. I'd hate to see the golang community waste resources trying to shoehorn manual memory management and gc together into one language. Right now it is an issue of a naive gc implementation. Golang has a great community and is progressing rapidly, I'm confident you won't be waiting long for what you desire(performance+garbage collection)

Andrew Gerrand

unread,
Jul 17, 2014, 8:33:09 PM7/17/14
to akwillis, golang-nuts

On 18 July 2014 10:30, akwillis <akwi...@gmail.com> wrote:
I'd hate to see the golang community waste resources trying to shoehorn manual memory management and gc together into one language

Don't worry. It's never going to happen. Go has been a garbage collected language from day one. If you want a language with optional GC then use one that was designed that way.

Andrew

Andrew Gerrand

unread,
Jul 17, 2014, 9:05:39 PM7/17/14
to Archos, golang-nuts

On 18 July 2014 04:57, Archos <raul...@sent.com> wrote:
Only to have your project Calimstore built in Rust, it would be worth the effort.

Why would you care? Camlistore works great in Go. Optional GC wouldn't help. It's a largely I/O bound system.

andrewc...@gmail.com

unread,
Jul 17, 2014, 9:44:03 PM7/17/14
to golan...@googlegroups.com, raul...@sent.com

There is a recent trend where people care what language something is written in, even when it doesn't affect the end user.
My favourite examples are when people call things like a spreadsheet program Gospreadsheet.or Jspreadsheet. end users shouldn't care what language X is written in if its a useful tool, so don't name your project like that.
In reality it only matters if its a software library, or is meant to be extensible.

I admit when I read that RabbitMQ was in erlang I had a fear of the unknown even though it doesn't make any difference whatsoever.


andrewc...@gmail.com

unread,
Jul 17, 2014, 9:50:08 PM7/17/14
to golan...@googlegroups.com, raul...@sent.com, andrewc...@gmail.com

Also, I'm announcing my new Search and Advertising business Rustogle, we are a lot faster than Google for various technical reasons..

Hasan Yasin Öztürk

unread,
Jul 17, 2014, 10:14:25 PM7/17/14
to golan...@googlegroups.com, raul...@sent.com, andrewc...@gmail.com
Rustogle is really blazing fast. No garbage collection, no frame drops. Good job there.

I have seen people starting to use Linkedin after the big hype started with Linkedin moving to Nodejs in the backend. After this, I can really understand programmers being curious about the language that their favorite tools were written in. 

andrewc...@gmail.com

unread,
Jul 17, 2014, 10:24:22 PM7/17/14
to golan...@googlegroups.com, raul...@sent.com, andrewc...@gmail.com
Go's garbage collector is a defining feature. It's critical to how go routines work and share memory. Concurrent and more incremental GC's will help Go more than removing the collector.

I think there are misconceptions about garbage collection - many have been stated by Dmitry and others regarding performance vs manual memory allocation.

One extra thing I'd like to point out is reference counting can have unpredictable delays too. One reference decrement can cause a whole chain of frees to occur, with no predictability at all. So shifting Go to reference counting wouldn't be a solution.

Bakul Shah

unread,
Jul 17, 2014, 10:43:14 PM7/17/14
to andrewc...@gmail.com, golan...@googlegroups.com, raul...@sent.com
On Thu, 17 Jul 2014 19:24:21 PDT andrewc...@gmail.com wrote:
>
> One extra thing I'd like to point out is reference counting can have
> unpredictable delays too. One reference decrement can cause a whole chain
> of frees to occur, with no predictability at all.

Not necessarily. When a object's refcount goes to zero,
rather than freeing it right away you can enqueue it for lazy
refcount decrementing of objects it points to.

andrewc...@gmail.com

unread,
Jul 17, 2014, 10:53:18 PM7/17/14
to golan...@googlegroups.com, andrewc...@gmail.com, raul...@sent.com

Interesting point, but to me that sounds like reinventing a mark and sweep garbage collector in a round about way. If you process the lazy list incrementally, well then its an incremental GC again.

I know your comment wasn't actually suggesting we add reference counting to Go, but I'd like to refute that for people who may read it that way.

jonathan....@gmail.com

unread,
Jul 18, 2014, 12:58:15 AM7/18/14
to golan...@googlegroups.com, raul...@sent.com
Is there any chance of getting Erlang-style thread-local GC in Go, or at least thread-local GC for young generations when Go gets a generational collector?

Chandru

unread,
Jul 18, 2014, 1:12:33 AM7/18/14
to Dmitry Vyukov, Archos, golang-nuts
Why isn't #5 possible in a non-GC language? Rust's TypedArea[1] seems to be doing exactly that.


--
Chandra Sekar.S


Damon Zhao

unread,
Jul 18, 2014, 1:39:19 AM7/18/14
to golan...@googlegroups.com, raul...@sent.com, alexru...@gmail.com
The WOW use erlang for backend server

在 2014年7月17日星期四UTC+8下午8时47分26秒,Archos写道:
Sorry, I was wrong again :(
To make it clear, please

Tell me three real-time games built with a language with GC (i.e. Java), and of course, without bindings to a language without GC (i.e. C++).

El jueves, 17 de julio de 2014 13:39:05 UTC+1, Archos escribió:


El jueves, 17 de julio de 2014 13:37:37 UTC+1, Archos escribió:
El jueves, 17 de julio de 2014 13:31:15 UTC+1, alexru...@gmail.com escribió:
четверг, 17 июля 2014 г., 14:11:08 UTC+3 пользователь Archos написал:
You are seriously wrong. There are tons of "realtime" games (some are very famous and popular) written in languages with GC.

 Ok, tell me three real-time games built with a language without GC, and of course without bindings to a language garbaged.

I mean:

Tell me three real-time games built with a language WITH GC, and of course without bindings to a language garbaged.

Archos

unread,
Jul 18, 2014, 2:09:38 AM7/18/14
to golan...@googlegroups.com
I didn't say by the optional GC; Rust is lower-level (than Go) so programs that are I/O bound system (file systems, databases) should have a better performance.

But until that a filesystem or database is not built in Rust, I could not do an  an objective comparison.

Gerard

unread,
Jul 18, 2014, 2:56:49 AM7/18/14
to golan...@googlegroups.com

On Friday, July 18, 2014 8:09:38 AM UTC+2, Archos wrote:
I didn't say by the optional GC; Rust is lower-level (than Go) so programs that are I/O bound system (file systems, databases) should have a better performance.

I am not really sure that is true. Rust uses different approaches, yes. Go is taking more the Wirth (Pascal/Modula/Oberon) approach of better data abstraction combined with GC.

I wouldn't consider it higher-level, just simpler.

Archos

unread,
Jul 18, 2014, 3:06:25 AM7/18/14
to golan...@googlegroups.com, raul...@sent.com, andrewc...@gmail.com

El viernes, 18 de julio de 2014 02:44:03 UTC+1, andrewc...@gmail.com escribió:
On Friday, July 18, 2014 1:05:39 PM UTC+12, Andrew Gerrand wrote:
On 18 July 2014 04:57, Archos <raul...@sent.com> wrote:
Only to have your project Calimstore built in Rust, it would be worth the effort.

Why would you care? Camlistore works great in Go. Optional GC wouldn't help. It's a largely I/O bound system.

There is a recent trend where people care what language something is written in, even when it doesn't affect the end user.
My favourite examples are when people call things like a spreadsheet program Gospreadsheet.or Jspreadsheet. end users shouldn't care what language X is written in if its a useful tool, so don't name your project like that.
In reality it only matters if its a software library, or is meant to be extensible.

 
I'm sure that the choose of a language does not affect to the end user but to your pockets.

IronWorker was from 30 servers (using Ruby) to only 2 servers (with Go):

http://blog.iron.io/2013/03/how-we-went-from-30-servers-to-2-go.html

Archos

unread,
Jul 18, 2014, 3:10:33 AM7/18/14
to golan...@googlegroups.com
A "simpler sintaxis" seems to be a euphemism for "fewer features".

David Symonds

unread,
Jul 18, 2014, 3:13:21 AM7/18/14
to Archos, golang-nuts
On 18 July 2014 17:10, Archos <raul...@sent.com> wrote:

> A "simpler sintaxis" seems to be a euphemism for "fewer features".

And sometimes a cigar is just a cigar.

Gerard

unread,
Jul 18, 2014, 3:15:32 AM7/18/14
to golan...@googlegroups.com

Yes, exactly !

"Are you quite sure that all those bells and whistles, all those wonderful facilities of your so called powerful programming languages, belong to the solution set rather than the problem set?"

        — Edsger W. Dijkstra

Sebastien Binet

unread,
Jul 18, 2014, 3:16:46 AM7/18/14
to David Symonds, Archos, golang-nuts
this, "|", is not a pipe.

-s

Dmitry Vyukov

unread,
Jul 18, 2014, 5:27:02 AM7/18/14
to Chandru, Archos, golang-nuts
On Fri, Jul 18, 2014 at 9:12 AM, Chandru <chand...@gmail.com> wrote:
> Why isn't #5 possible in a non-GC language? Rust's TypedArea[1] seems to be
> doing exactly that.
>
> [1] http://doc.rust-lang.org/arena/struct.TypedArena.html

It's possible with user assistance, in extreme case user just manually
puts several objects into a single struct. But even that is not always
possible. First, there are abstraction boundaries. Then, subobjects
may need to be deleted at different times, and you don't know which
one will be the last. And in the end, users just do not do it most of
the time.
Go does this transparently and it can combine objects of different
types and across abstraction boundaries. For example, in the following
code:

func libAfoo(...) {
var mu sync.Mutex
libBfoo(&mu)
mu.Lock()
}

func libBfoo(mu *sync.Mutex) {
s := string(byteSlice)
...
}

mu and s buffer can end up in a single memory block.

Nicolas Grilly

unread,
Jul 18, 2014, 8:35:25 AM7/18/14
to golan...@googlegroups.com, andrewc...@gmail.com, raul...@sent.com
The more you optimize, the more garbage collection and reference counting are behaving similarly.

akwillis

unread,
Jul 18, 2014, 11:07:01 AM7/18/14
to golan...@googlegroups.com, raul...@sent.com, alexru...@gmail.com
Also, Lua is used in wow as a scripting engine in the macro system.

Andy Balholm

unread,
Jul 18, 2014, 11:58:37 AM7/18/14
to jonathan....@gmail.com, golan...@googlegroups.com, raul...@sent.com

On Jul 17, 2014, at 9:58 PM, jonathan....@gmail.com wrote:

> Is there any chance of getting Erlang-style thread-local GC in Go, or at least thread-local GC for young generations when Go gets a generational collector?

Pure goroutine-local GC is not possible, since goroutines share memory. But I can imagine an enhanced escape analysis in the compiler that could find allocations that escape the current function but don’t escape the goroutine. Will someone ever do all that work? Will it actually be a big performance gain? I don’t know.

Konstantin Khomoutov

unread,
Jul 18, 2014, 12:57:55 PM7/18/14
to Andy Balholm, jonathan....@gmail.com, golan...@googlegroups.com, raul...@sent.com
On Fri, 18 Jul 2014 08:58:10 -0700
Andy Balholm <andyb...@gmail.com> wrote:

> > Is there any chance of getting Erlang-style thread-local GC in Go,
> > or at least thread-local GC for young generations when Go gets a
> > generational collector?
>
> Pure goroutine-local GC is not possible, since goroutines share
> memory. But I can imagine an enhanced escape analysis in the compiler
> that could find allocations that escape the current function but
> don't escape the goroutine.

I'm afraid you can't do that: a function running within a goroutine
might call another function using its address -- that is, divert the
control flow to a path determined at runtime.

Kevin Malachowski

unread,
Jul 18, 2014, 2:03:39 PM7/18/14
to golan...@googlegroups.com
Wouldn't the compiler detect this dynamic situation and say that the data passed would have to be shared-heap allocated? I think the idea is if it can be proved that it doesn't escape then it would be local heap allocated.

For example, a value used in a closure in a function could be locally heap allocated if the compiler notices the closure itself doesn't escape, unless I'm missing something. I have no idea how hard that would be to implement, though.

Bakul Shah

unread,
Jul 18, 2014, 3:25:27 PM7/18/14
to andrewc...@gmail.com, golan...@googlegroups.com, raul...@sent.com
On Thu, 17 Jul 2014 19:53:18 PDT andrewc...@gmail.com wrote:
>
> On Friday, July 18, 2014 2:43:14 PM UTC+12, Bakul Shah wrote:
> >
> > On Thu, 17 Jul 2014 19:24:21 PDT andrewc...@gmail.com <javascript:>
> > wrote:
> > >
> > > One extra thing I'd like to point out is reference counting can have
> > > unpredictable delays too. One reference decrement can cause a whole
> > chain
> > > of frees to occur, with no predictability at all.
> >
> > Not necessarily. When a object's refcount goes to zero,
> > rather than freeing it right away you can enqueue it for lazy
> > refcount decrementing of objects it points to.
>
> Interesting point, but to me that sounds like reinventing a mark and sweep
> garbage collector in a round about way. If you process the lazy list
> incrementally, well then its an incremental GC again.

Delayed scanning of dead nodes is much simpler since none of
the embedded ptrs are going to change. But during concurrent
scanning of live nodes, embedded ptrs *can* change & you run
into all sort of issues. In RC no pauses are required but you
need atomic updates if done concurrently. Even in "pauseless"
concurrent generational incremental continuous[1] GC you will
have pauses (but presumably tolerable) and synchronization
points and need to scan the stack and write barriers or read
barriers &/or you will need help from special hardware or
kernel (if you are playing virtual memory games). So much more
complexity. Still, RC doesn't solve the problem of garbage
containing reference loops so you still need GC and it is
probably less performant in concurrent programs than GC.

> I know your comment wasn't actually suggesting we add reference counting to
> Go

Correct. I was just commenting on your "reference counting can
have unpredictable delays".

[1] not sure if I missed any GC related buzzwords here.

Tamás Gulácsi

unread,
Jul 18, 2014, 3:45:01 PM7/18/14
to golan...@googlegroups.com
Thank you for your efforts!

Jsor

unread,
Jul 19, 2014, 10:19:14 AM7/19/14
to golan...@googlegroups.com
On Thursday, July 17, 2014 9:37:36 AM UTC-7, DV wrote:
I have recently started dabbling with the Unreal Engine development kit. You might know the Unreal Engine from almost every single AAA FPS game currently on the market. The language you write your game logic in for it is C++, or Epic's "dialect" of it. You know what they implemented for their C++ framework? Garbage collection. 



Also every Jak and Daxter game and Crash Bandicoot game use engines written in variants of Allegro Common LISP that compiled to Playstation 2 native MIPS, and that uses a generational GC. To be fair, they also used inline assembly in places, but Go can use assembly too if you want. But was the engine very good? Well, they made a seamless streaming world with it. With multiple viewports. On a Playstation 2. So yes, I'd say so.

Donovan Hide

unread,
Jul 19, 2014, 10:38:54 AM7/19/14
to Jsor, golang-nuts
Also every Jak and Daxter game and Crash Bandicoot game use engines written in variants of Allegro Common LISP that compiled to Playstation 2 native MIPS, and that uses a generational GC. To be fair, they also used inline assembly in places, but Go can use assembly too if you want. But was the engine very good? Well, they made a seamless streaming world with it. With multiple viewports. On a Playstation 2. So yes, I'd say so.

Thanks for mentioning this. Led to a very interesting read:


Sam Harwell

unread,
Jul 23, 2014, 11:37:07 PM7/23/14
to Archos, golan...@googlegroups.com, alexru...@gmail.com

All UnrealEngine 3 games use a garbage collected scripting language for a wide variety of critical tasks.

 

Sam

 

From: golan...@googlegroups.com [mailto:golan...@googlegroups.com] On Behalf Of Archos
Sent: Thursday, July 17, 2014 7:39 AM
To: golan...@googlegroups.com
Cc: raul...@sent.com; alexru...@gmail.com
Subject: Re: [go-nuts] Go 2.0 without garbage collector?

 



El jueves, 17 de julio de 2014 13:37:37 UTC+1, Archos escribió:

El jueves, 17 de julio de 2014 13:31:15 UTC+1, alexru...@gmail.com escribió:

четверг, 17 июля 2014 г., 14:11:08 UTC+3 пользователь Archos написал:

You are seriously wrong. There are tons of "realtime" games (some are very famous and popular) written in languages with GC.


 Ok, tell me three real-time games built with a language without GC, and of course without bindings to a language garbaged.


I mean:

Tell me three real-time games built with a language WITH GC, and of course without bindings to a language garbaged.

--

Sam Harwell

unread,
Jul 24, 2014, 12:12:23 AM7/24/14
to Archos, golan...@googlegroups.com

The best possible performance actually comes from using a Garbage Collector for memory management in all but one case. That specific case is writing an application which needs to run in as little memory as possible, regardless of the performance impact of that decision. As we all know, that situation virtually never arises in modern programming. Also note that this situation generally does not apply to large programs, because the coding complexity of manual memory management throughout a large codebase results in inefficient memory practices that negate the theoretical advantage offered by manual management.

 

However, the following caveats do hold:

 

1.       For a particular application (algorithm and/or memory usage characteristics), some garbage collector will outperform the available manually-managed allocators for that application. However, there is no garbage collector which performs best for all applications. The best production garbage collectors generally fall into three categories according to the anticipated majority of applications which will run within the environment:

a.       High-throughput collectors which target applications with exceptionally large memory spaces, and often allocate objects at higher rates than seen in other applications. These could be called “server” collectors (not to be confused with the -server command line option in the JVM).

b.      Concurrent, low-latency collectors which balance throughput with pauses and are widely applicable to “client” applications. By overall usage, these are currently the most commonly used garbage collectors, and are the foundations of both the current .NET Framework collector and the JVM.

c.       Real-time garbage collectors which provide hard guarantees regarding the duration of a pause. These collectors are used in specialized applications where a long GC pause poses a physical threat (e.g. system control software for aviation). IBM’s Metronome is a prime example of this type of collector.

2.       The amount of work required to implement an efficient garbage collection environment within a new language is not only enormous, but often requires the involvement of researchers who focus on this particular area of software development (and aren’t particularly common). The “easy” to implement collectors are frequently much slower than the better algorithms, resulting in the unfortunately common situation where garbage collectors get a bad name. In particular, when a garbage collector is first introduced to a new environment, users observe poor performance compared to the manually-managed environment in which they previously worked, but they do not realize that this deficit is due to the particular garbage collector which was added and not an overall characteristic of all garbage collectors.

a.       One of the best ways for young garbage-collected languages to be fairly evaluated for new language features is to write a compiler that targets the bytecode of an existing runtime system that includes a high-performance garbage collector. The most well supported options are the CLI (.NET) and the JVM (Java). For better or worse, larger projects with better funding (such as Go) tend to have an easier time avoiding this path.

 

Thanks,

Sam

 

From: golan...@googlegroups.com [mailto:golan...@googlegroups.com] On Behalf Of Archos
Sent: Thursday, July 17, 2014 3:52 AM
To: golan...@googlegroups.com
Subject: [go-nuts] Go 2.0 without garbage collector?

 

Does the Go team has thinked into a next version without the garbage collector?

Go is very cool to use but I need the best possible performance since I've not deep pockets like Google. When you are small, you need try to minimize the cost of the servers.
And Rust does a great balance between a language of high level without a significant loss of performance.

I've been translating some Go projects from Go to Rust but the task is very large, especially since every time there are more interesting projects for me.
So, the fastest way for me will be to build a transpiler from Go to Rust. At least there is some intention to have an optional garbage collection.

However, Go is easier to prototype.

--

Philip O'Toole

unread,
Jul 24, 2014, 12:44:56 AM7/24/14
to Sam Harwell, Archos, golan...@googlegroups.com
Sam -- interesting statements. In your opening paragraph, what do you mean by "performance"? You're talking about sheer througput of, say, a program moving data through RAM?

And can you give an example of what you mean by "inefficient memory practices"? I've seen plenty of inefficent memory practices in garbage-collected languagues (notably Java, running in a modern HotSpot JVM) that negate the theoretical advantage offered by automatic management. :-)

(FWIW, I don't have a strong opinion about GC, or lack thereof. I am using Go because it, so far, makes writing system software pretty nice).

Philip
 
-----------------------------------------
http://www.philipotoole.com

Sam Harwell

unread,
Jul 24, 2014, 1:27:53 AM7/24/14
to Philip O'Toole, Archos, golan...@googlegroups.com

Hi Philip,

 

For this abstract overview, performance is measured in terms of particular applications. I base this on a general observation that for any frequently used metric of performance in an application, specialized garbage collectors have been created which outperform manual allocation/releasing in the same application. Runtime environments which are known to run many different kinds of programs typically use collectors that balance the needs of a wide variety of expected use cases, and within the past few years have done an exceptional job in practice.

 

“Inefficient memory practices” can manifest in many ways, both direct and indirect. One example of a direct inefficiency involves implementing GC-like features such as reference-counting pointers in C++ to addresses certain memory management concerns, often with great success in very large codebases. The downsides of this include both the use of additional memory for the reference count and the performance impact of manipulating reference counts. While “best practices” for these pointers (e.g. passing by const reference to avoid incrementing/decrementing the count) serve to reduce the performance impact, you will never reach the performance offered by GC-backed reference counting solutions such as Ulterior Reference Counting. An indirect efficiency might be that time spent resolving memory allocation issues may leave developers with less available time during the development cycle to address higher-level concerns such as algorithms and communication layers within the software. Performance frequently suffers in these areas regardless of the memory management system in use, so keeping developer resources available to focus on them can be very valuable.

 

Thanks,

Sam

Philip O'Toole

unread,
Jul 24, 2014, 2:05:22 AM7/24/14
to Sam Harwell, Archos, golan...@googlegroups.com
Good stuff - thanks for the extra detail and the references. I have spent plenty of time coding with shared pointers in C++, so the whitepaper should be interesting.

Philip 

Nicolas Grilly

unread,
Jul 24, 2014, 9:36:20 AM7/24/14
to Sam Harwell, Philip O'Toole, Archos, golan...@googlegroups.com
Hi Sam,

Your detailed comment about garbage collection versus manual management of memory is really interesting.

I'm curious to know what you think of Rust emphasis on making garbage collection optional (FAQ says "Rust is usable on bare metal with no extra runtime"), and letting the programmer manage object ownership explicitly?

Their approach gives a lot of power to the programmer but also makes the language more complex, in my opinion. Is it worth it in terms of improved performance?

Sam Harwell

unread,
Jul 24, 2014, 9:20:18 PM7/24/14
to Nicolas Grilly, Philip O'Toole, Archos, golan...@googlegroups.com

Hi Nicolas,

 

Every “usable” language provides a standard runtime library of some form. Such a library could easily be considered an “extra runtime”, on the definition that an “extra runtime” provides necessary supporting features which were not directly implemented as part of the running application. The only difference between a garbage collector implementation and a standard runtime library is whether the features are implicitly or explicitly invoked.

 

Garbage collectors have two primary purposes:

 

1.       Accurately reclaim memory, typically measured as a bounded approximation over liveness.

2.       Meet or exceed the performance of manual memory management strategies.

 

In reality, the first item is very easy to implement using simple, proven strategies. Mark-sweep garbage collectors have existed since (at least) 1967, and copying collectors since at least 1978. The second item, on the other hand, is extraordinarily difficult, with substantial contributions being made even in the last 5 years. Addressing the second point frequently leads to questions about the ongoing correctness of the collector (i.e., attempting to meet item #2 can easily cause bugs where it no longer meets item #1). For young languages which do not execute within a well-known virtual machine and include garbage collection as part of the language specification, limiting the implementation to just the first point allows for developers to focus on other critical aspects of the language.

 

As a case-in-point, the Mono project used a relatively slow off-the-shelf garbage collector (boehm-gc) from 2002 all the way until late 2009. It wasn’t until February 2011 that the performance-enhanced collector finally shipped in Mono 2.10. For other languages which haven’t reached this point and have facilities in the language specification to support manual memory management, it could make sense to provide garbage collection as an optional feature. I don’t personally support such a decision, but I understand both the motivation that leads to it and recognize the potential for it to improve near-term benchmarks for young languages prior to the introduction of high-performance garbage collection.

 

Rather than resort to this strategy, I typically encourage language developers to initially compile to the bytecode of an existing virtual machine which already offers high-performance garbage collection. An obvious added benefit is these runtime environments typically have well-established support for features like reliable debugging, profiling, and extensive runtime memory analysis. Language developers must evaluate these advantages in the context of their expected audience and the types of applications they intend to support.

 

Thanks,

Sam

 

From: golan...@googlegroups.com [mailto:golan...@googlegroups.com] On Behalf Of Nicolas Grilly
Sent: Thursday, July 24, 2014 8:36 AM
To: Sam Harwell
Cc: Philip O'Toole; Archos; golan...@googlegroups.com
Subject: Re: [go-nuts] Go 2.0 without garbage collector?

 

Hi Sam,

--

branimir....@gmail.com

unread,
Jul 25, 2014, 1:00:47 AM7/25/14
to golan...@googlegroups.com

I love Rust but it is unstable. I am sick and tired of changing my Rust programs in order to compile after each version.
So I am waiting 1.0 .
Go is much simpler than Rust and therefore easier to program, so I am experimenting now with Go and am quite
satisfied. However performance is not strong point of Go, and it is not just because of GC. 

Dave Cheney

unread,
Jul 25, 2014, 1:10:09 AM7/25/14
to golan...@googlegroups.com, branimir....@gmail.com


On Friday, 25 July 2014 15:00:47 UTC+10, branimir....@gmail.com wrote:

I love Rust but it is unstable. I am sick and tired of changing my Rust programs in order to compile after each version.
So I am waiting 1.0 .
Go is much simpler than Rust and therefore easier to program, so I am experimenting now with Go and am quite
satisfied. However performance is not strong point of Go, and it is not just because of GC. 

Them's fighting words, Citation Needed, please.

branimir....@gmail.com

unread,
Jul 25, 2014, 1:14:41 AM7/25/14
to golan...@googlegroups.com, branimir....@gmail.com


On Friday, July 25, 2014 7:10:09 AM UTC+2, Dave Cheney wrote:


On Friday, 25 July 2014 15:00:47 UTC+10, branimir....@gmail.com wrote:

I love Rust but it is unstable. I am sick and tired of changing my Rust programs in order to compile after each version.
So I am waiting 1.0 .
Go is much simpler than Rust and therefore easier to program, so I am experimenting now with Go and am quite
satisfied. However performance is not strong point of Go, and it is not just because of GC. 

Them's fighting words, Citation Needed, please.
 

That's my opinion based on my experience.
 

Jonathan Barnard

unread,
Jul 25, 2014, 1:28:12 AM7/25/14
to branimir....@gmail.com, golang-nuts

I think most of the performance problems people post on this board seem to come from overusing hashmaps. If you treat Go as C with channels and slices, it generally performs quite well.

--
You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/rvFasvA3usk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.

branimir....@gmail.com

unread,
Jul 25, 2014, 2:01:11 AM7/25/14
to golan...@googlegroups.com, branimir....@gmail.com, jonathan....@gmail.com


On Friday, July 25, 2014 7:28:12 AM UTC+2, Jonathan Barnard wrote:

I think most of the performance problems people post on this board seem to come from overusing hashmaps. If you treat Go as C with channels and slices, it generally performs quite well.



I don't have performance problems with Go, I just notice that it is slower than C/C++. There is no auto vectorization and sometimes generated code is subpar
when compared to C/C++. But there are gccgo and 6/8g. Sometimes gccgo performs better, sometimes 6/8g . gccgo seems better at function calls
through interfaces and abstract code, but 6/8g seems better with arrays. All in all performance is not bad but is subpar in comparison to C++.
Let's compare three simple programs:

package main
import (
  "fmt"
  "time"
)

const N = 500000000
func init_cumul(a []int64) {
  for k := 0; k<len(a);k++ {
    a[k] = int64(k/3+k)
  }
}

func cumul(a []int64) {
  var sum int64
  for i := range a {
    a[i] += sum
    sum = a[i]
  }
}


func main() {
  var array [] int64 = make([]int64,N)
  for i := 0;i<10;i++ {
  start := time.Now()
  init_cumul(array)
  end := time.Since(start)
  fmt.Printf("%f\n",float64(N*1000)/float64(end.Nanoseconds()))
  start = time.Now()
  cumul(array)
  end = time.Since(start)
  fmt.Printf("%f\n",float64(N*1000)/float64(end.Nanoseconds()))
  }
  fmt.Printf("%d\n",array[N-1])
}

gccgo:
273.403325
1202.290122
880.824452
1198.635473
896.225992
1201.429220
886.751227
1189.385920
895.564627
1200.140176
897.002576
1197.966332
892.188002
1206.825807
895.226295
1196.813603
892.334490
1199.296732
895.304842
1197.125940
166666666166666667

6g:
263.543236
501.597159
804.681819
512.730177
803.912885
517.131290
798.258510
515.190764
801.201592
515.878526
803.515553
514.060331
797.765770
517.076759
797.631910
518.080521
801.722531
518.507745
796.869586
518.152138
166666666166666667

java:
public class Cumul {

public static void sum(long[] data) {
if(data.length == 0) return;
for( int i = 1; i !=  data.length; ++i)
 data[i] += data[i-1];
}



    public static long[]  givemeanarray(int N) {
    long[] bigarray = new long[N];
        for(int k = 0; k<N; ++k)
          bigarray[k] = k+k/3;
        return bigarray;
     }
     
     public static void test(int N) {
      for (int i=0;i<10;++i)
      {
      long bef = System.currentTimeMillis();
      long[] data = givemeanarray(N);
      long aft = System.currentTimeMillis();
      System.out.println(N/(1000.0 * (aft-bef)));
      bef = System.currentTimeMillis();
      sum(data);
      aft = System.currentTimeMillis();
      System.out.println(N/(1000.0 * (aft-bef)));
      System.out.println(data[N-1]);
      }
     }
     
     public static void main(String[] args) {
      test( 500 * 1000 * 1000 );
     }


}

268.3843263553409
1312.3359580052493
166666666166666667
247.52475247524754
1312.3359580052493
166666666166666667
259.0673575129534
1315.7894736842106
166666666166666667
262.7430373095113
1308.9005235602094
166666666166666667
259.0673575129534
1302.0833333333333
166666666166666667
261.3695765812859
1315.7894736842106
166666666166666667
263.43519494204423
1305.4830287206266
166666666166666667
265.3927813163482
1322.7513227513227
166666666166666667
263.5740643120717
1319.2612137203166
166666666166666667
263.99155227032736
1322.7513227513227
166666666166666667

C++:
#include <stdio.h>

const int N=500000000;
long array[N];
unsigned long elapsed=0;

void init_cumul()
{
  for(int k=0;k<N;k++)
    array[k]=k/3+k;
}

void cumul()
{
  long sum = array[0];
  for(int k=1;k<N;++k)
  {
    array[k]+=sum;
    sum=array[k];
  }
}

void timer_init()
{
  unsigned edx,eax;
  asm volatile(
  "rdtsc \n":"=d"(edx),"=a"(eax)
  );
  elapsed = ((unsigned long)edx<<32) | eax;
}
void timer()
{
  unsigned edx,eax;
  asm volatile(
  "rdtsc \n":"=d"(edx),"=a"(eax)
  );
  elapsed = (((unsigned long)edx<<32) | eax) - elapsed ;
  printf("%f\n",double(N)/(elapsed/3800.0));
}

int main()
{
  for(int i=0;i<10;++i)
  {
  timer_init();
  init_cumul();
  timer();
  timer_init();
  cumul();
  timer();
  }
  printf("%ld\n",array[N-1]);
}

350.401962
1477.418397
1515.326802
1468.208677
1515.674478
1462.651715
1495.352347
1469.423731
1519.040236
1477.555610
1507.452754
1462.954432
1521.180948
1480.971117
1517.591325
1470.561027
1491.424521
1468.494121
1513.532281
1488.670784
166666666166666667

As you can see in this example Go is subpar to C++, but better than Java on average.

Nicolas Grilly

unread,
Jul 25, 2014, 6:21:24 AM7/25/14
to golan...@googlegroups.com, nic...@vocationcity.com, philip...@yahoo.com, raul...@sent.com
Thank you Sam. That's a really interesting take on the subject. Do you have any link to some papers supporting this point of view? 

Sam Harwell

unread,
Jul 25, 2014, 7:57:16 AM7/25/14
to Nicolas Grilly, golan...@googlegroups.com, philip...@yahoo.com, raul...@sent.com

One of my favorites is Daniel Frampton’s doctoral dissertation, which is very well-written and very thorough.

Garbage Collection and the Case for High-level Low-level Programming

 

If you really have some time to spend on the subject, here’s the reading list from a course I took on Memory Management.

http://www.cs.utexas.edu/users/mckinley/395Tmm/schedule.html

 

The work on Jikes RVM eventually prompted me to build a bootstrapping all-managed virtual machine based on ECMA-335 (though it was never released). Between having to get a “real job” and shifting my research emphasis to high-performance parsing for IDEs (e.g. GoWorks), I haven’t had the time available to complete this project. I do have a public repository containing my C# port of AsmJit, which I used for the x86 and x64 code generation.

https://github.com/tunnelvisionlabs/nasmjit

 

Sam

 

From: golan...@googlegroups.com [mailto:golan...@googlegroups.com] On Behalf Of Nicolas Grilly
Sent: Friday, July 25, 2014 5:21 AM
To: golan...@googlegroups.com
Cc: nic...@vocationcity.com; philip...@yahoo.com; raul...@sent.com
Subject: Re: [go-nuts] Go 2.0 without garbage collector?

 

Thank you Sam. That's a really interesting take on the subject. Do you have any link to some papers supporting this point of view? 

--

Nicolas Grilly

unread,
Jul 25, 2014, 9:49:13 AM7/25/14
to Sam Harwell, golan...@googlegroups.com, philip...@yahoo.com, raul...@sent.com
Sam,

A big thank you for the links!

One of the papers, "Myths and Realities: The Performance Impact of Gargabe Collection", has a paragraph that offers a good summary:

Is garbage collection a good idea?

The software engineering benefits of garbage collection over explicit memory management are widely accepted, but the performance trade-off in languages designed for garbage collection is unexplored. Section 5.3 shows a clear mutator performance advantage for contiguous over free-list allocation, and the architectural comparison shows that architectural trends should make this advantage more pronounced. The traditional explicit memory management use of malloc() and free() is tightly coupled to the use of a free-list allocator—in fact the MMTk free-list allocator implemen- tation is based on Lea allocator, which is the default allocator in standard C libraries. Standard explicit memory management is thus unable to exploit the locality advantages of contiguous allocation. It is therefore possible that garbage collection presents a performance advantage over explicit memory management on current or future architectures.

Dmitry Vyukov

unread,
Jul 25, 2014, 9:54:54 AM7/25/14
to Nicolas Grilly, Sam Harwell, golan...@googlegroups.com, philip...@yahoo.com, raul...@sent.com
You can also see some other reasons why GC can be faster that I outlined above:
https://groups.google.com/d/msg/Golang-nuts/rvFasvA3usk/FvwRBeFAtl0J

Nicolas Grilly

unread,
Jul 25, 2014, 10:21:42 AM7/25/14
to golan...@googlegroups.com, nic...@vocationcity.com, s...@tunnelvisionlabs.com, philip...@yahoo.com, raul...@sent.com
On Friday, July 25, 2014 3:54:54 PM UTC+2, Dmitry Vyukov wrote:
You can also see some other reasons why GC can be faster that I outlined above:
https://groups.google.com/d/msg/Golang-nuts/rvFasvA3usk/FvwRBeFAtl0J 

Hi Dmitry,

I had already read your message a few days ago, but thanks for the link :)

Some people are spreading FUD about garbage collection, using this as an argument to promote languages like Rust, where GC is optional, and disqualify Go. This is the reason why I try to educate myself about the **real** advantages and shortcomings of GC.

Maybe the FAQ could be updated with an additional paragraph citing why GC can be **faster**, and why manual memory management can be slower.

Henrik Johansson

unread,
Jul 25, 2014, 12:23:01 PM7/25/14
to Nicolas Grilly, golang-nuts, s...@tunnelvisionlabs.com, philip...@yahoo.com, Archos
Ye GC can be faster than manual memory management there is no denying that but the Java world has been touting that as a mantra forever and now they are moving towards "off heap" solutions and similar things anyway.

GC is definitely worth it and has not necessarily any negative impact but still, as of today, it is almost always more performant without it.


--

Archos

unread,
Jul 25, 2014, 12:51:30 PM7/25/14
to golan...@googlegroups.com, nic...@vocationcity.com, s...@tunnelvisionlabs.com, philip...@yahoo.com, raul...@sent.com

El viernes, 25 de julio de 2014 15:21:42 UTC+1, Nicolas Grilly escribió:
Some people are spreading FUD about garbage collection, using this as an argument to promote languages like Rust, where GC is optional, and disqualify Go. This is the reason why I try to educate myself about the **real** advantages and shortcomings of GC.
 
Some people are fan-boys/girls of a language and they think that a language can be used for whatever task.

If Rust was designed to build a browser engine is because its original author wanted a easier and safer low level language than C/C++. If you, using Go, can build an engile browser like Firefox, or a kernel, it's good for you, but I would never use those programs in my systems. That you can use a language to build something, it does not mean that it can be usable or that the performance can be acceptable for everybody; as an example, you have the desktop programs built in Java.

Whether Rust has attracted to professional developers from gaming industry has been due to very good reasons. It is not a coincidence that there are so many projects related to games being built in Rust and any other related to kernels.

By the another side, also Rust have their fan-boys which want to use Rust to build web services. But the reality is that Go is the language that lets you build a service faster out of the box.
Now well, if any part of the infrastructure of my system needs a better performance (after of profiling) then I do not hesitate in to use Rust to get a better performance simply because I'm not a fan-boy of a language neither of a company, instead I'll use the best tool to get the task done.

andrey mirtchovski

unread,
Jul 25, 2014, 1:11:08 PM7/25/14
to Archos, golang-nuts
> Now well, if any part of the infrastructure of my system needs a better
> performance (after of profiling) then I do not hesitate in to use Rust to
> get a better performance simply because I'm not a fan-boy of a language
> neither of a company, instead I'll use the best tool to get the task done.

then why did you ask the original question in the first place?

Archos

unread,
Jul 25, 2014, 1:42:08 PM7/25/14
to golan...@googlegroups.com, raul...@sent.com

You have just the answer in the original question.

Gerard

unread,
Jul 25, 2014, 2:16:44 PM7/25/14
to golan...@googlegroups.com, nic...@vocationcity.com, s...@tunnelvisionlabs.com, philip...@yahoo.com, raul...@sent.com
On Friday, July 25, 2014 6:51:30 PM UTC+2, Archos wrote:

<snip>

 
> If you, using Go, can build an engile browser like Firefox, or a kernel, it's good for you, but I would never use those programs in my systems.

The browsers of today are written in C or C++ AFAIK. In what sense is that any "better" than Go?

alexru...@gmail.com

unread,
Jul 25, 2014, 3:12:52 PM7/25/14
to golan...@googlegroups.com, nic...@vocationcity.com, s...@tunnelvisionlabs.com, philip...@yahoo.com, raul...@sent.com
пятница, 25 июля 2014 г., 19:51:30 UTC+3 пользователь Archos написал:
Whether Rust has attracted to professional developers from gaming industry has been due to very good reasons. It is not a coincidence that there are so many projects related to games being built in Rust and any other related to kernels.
Speaking of games, I think that those currently attracted to Rust aren't professionals from the "game industry" (not even indies), but mostly hobbyists writing their "engines" for fun.
 
Reply all
Reply to author
Forward
0 new messages