From Python to Go, and Back Again…

[Nicolas Grilly]

Any thoughts about this?

https://docs.google.com/presentation/d/1LO_WI3N-3p2Wp9PDWyv5B6EGFZ8XTOTNJ7Hd40WOUHo/mobilepresent

[Gustavo Niemeyer]

It's not clear what sort of thought you're looking for. There's a lot of content in there.

The generic answer is that Go will not please everyone, and that's alright. In fact, that's great. That's what moves us forward.

On Fri, Oct 16, 2015 at 2:16 PM, Nicolas Grilly <nic...@vocationcity.com> wrote:

Any thoughts about this?

https://docs.google.com/presentation/d/1LO_WI3N-3p2Wp9PDWyv5B6EGFZ8XTOTNJ7Hd40WOUHo/mobilepresent

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

gustavo @ http://niemeyer.net

[Shawn Milochik]

It sounds like an interesting story. I'd like to see the video to get the more fleshed-out version. Are you going to record it and put it on YouTube after presenting it?

In general, while your criticisms of error handling (lack of context and being mostly just strings are technically true, code that does what you complain about is not the example set by the standard library (io.EOF, for example). If people are writing bad Go code, you can't consider that a flaw in Go. If the core team was any more restrictive in how you could use Go, the public would revolt! The same thing goes for the lack of context -- as you said in the presentation. It's the way packages use errors that make them more or less useful. Changing the error interface to require more than Error() would impose more of a burden on developers and get in their way.

As a Python veteran and a relative Go newbie (about 1.5 years of Go, 8 years of Python), I also have these comments:

The deployment of a Go application is far superior to the deployment of a Python application that has any dependencies outside the standard library. I won't go into details since I'm sure you know all about it.

All things being equal, I feel much more confident deploying Go code than Python code for the simple reason that errors must be handled in Go or your code is broken, but in Python you're free to never use try/except until you hit runtime errors in production. Even if you try to behave, you don't know every exception type you could run into, and using a bare except or "except Exception" is bad for other reasons, mostly because they mask errors.

These are my opinions, and I'm trying not to be on the defensive. No hostility is intended by any of the words above, and any hint of it is due to this being written instead of spoken.

[Egon]

As usual YMMV... Go is not perfect for everyone and everything. When prototyping algorithms I still use JS, because it lets me iterate faster than other languages. But, anyways...

Note, most of these comments are somewhat grumpy view :)

> Debugging

> Basic strings lack context on call-stack

The place where the error is created is not necessarily the place where it is returned, they can be miles apart. However there are packages that allow adding context information. I see needing such context-information a design problem, rather than a compiler problem -- if you cannot understand where the error comes from, maybe the code isn't clear enough.

> Boilerplate for error handling is tedious

Yes, it is, but debugging improperly handled errors is more..

> Live debugging option at time was gdb, not fun...

You shouldn't need it... if you need it, most likely the code isn't clear enough. (Also... https://static.lwn.net/2000/0914/a/lt-debugger.php3)

Although sometimes you really need it, but then the simplistic, line-by-line, aren't that good. You can create something nicer, as long as you have created nice boundaries in your code. e.g. 

https://github.com/egonelbre/spexs2/tree/master/debugger

In the errorPage example, I would have loaded the error pages into memory instead of reading them from disk, this would get rid one of the error handling if-s.

> Goroutine Leaks

Just started thinking, would it be possible to detect goroutines that haven't done anything in 1h? It should help to weed such goroutines out.

> Very labor intensive to achieve high degree of testing

I suspect that the Python examples lie about the 100% code coverage.. i.e. each function that can throw an exception, should be tried.

f = open("filename", "r+")

f.write("123")

f.close()

...

Would be same as:

f = open("filename", "r+")

if !thrown {

   f.write("123")

   if !thrown {

     f.close()

     if !thrown {

       ...

     }

   }

}

I.e. all of those hidden branches should be tested to get 100% coverage.

> Extensive interfaces needed all over, regardless of appropriateness

Hard to say what was the exact problem, but I suspect it might have been some design errors that caused those extra interfaces. One of those could be separating code by form rather than function.

[Nicolas Grilly]

On Friday, October 16, 2015 at 7:33:11 PM UTC+2, Shawn Milochik wrote:

It sounds like an interesting story. I'd like to see the video to get the more fleshed-out version. Are you going to record it and put it on YouTube after presenting it?

I'm not the author :-) Ben Bangert is the author (I'm CCing him). 

[Nicolas Grilly]

On Friday, October 16, 2015 at 7:27:51 PM UTC+2, Gustavo Niemeyer wrote:

It's not clear what sort of thought you're looking for. There's a lot of content in there.

What kind of thoughts? I'm a long time Python developer and a 2 year old Go developer, and this presentation intrigued me on several points on which I'm willing to hear the advice of other Gophers:

- I'm surprised about the memory usage being lower with PyPy.

- What is the experience of other developers regarding goroutine leaks?

- The argument about the SSL implementation in Go sounds weak for two reasons: 1) the implementation is still young but improves quickly for example thanks to Cloudflare (which is cited in the presentation) and 2) you can still use a reverse proxy for SSL termination if you want.

[andrey mirtchovski]

it was a similar post that was the impetus for the "Profiling Go
Programs" article [1]. without seeing the code we can only speculate
on what the issues might be, if there are any.

that said, i find the statement "[1.4] will require a large refactor
of code to go back to the now optimal reader/writer per conn"
suspicious. in my experience Go makes refactoring like this easier, at
least in programs that are designed idiomatically.

i was doing a lot of twisted before Go arrived. i'm happy i don't have
to anymore, probably for the same reasons the author is unhappy with
Go: it didn't fit my preferred mental model for writing programs of
particular type.

--
1: http://blog.golang.org/profiling-go-programs

[John Kemp]

> On Oct 16, 2015, at 1:16 PM, Nicolas Grilly <nic...@vocationcity.com> wrote:
>
> Any thoughts about this?

Some random thoughts:

1. If I had the same scalability and capacity requirements as the author, I might have chosen a language/environment that I already knew would support what I needed to do, rather than pick a new language with unknown (to me, at the time) and likely-to-change (new versions of Go seem to appear relatively frequently, which I would expect given its relative newness).

2. The main complaints are largely aimed at performance and actual bugs (leaky go-routines) - both of these will affect any new language/environment.

3. I, myself, like Go for reasons that are not explored at all in this critique - they were apparently not as important for the author:

* Type-checking before runtime. This solves almost all bugs I ever write (leaving only the really tough ones ;), before I’ve even run the program. I love that. In general, the other pre-runtime checks on imports and so on are also really helpful, and help me get to the smallest amount of necessary code to have my program working, really really quickly. In most cases, this means I have a running and correctly designed program after “mere” type-checking. I’m amazed by that, but it really seems to work that way for me.
* Simple concurrency which I largely don’t have to personally police (I hate callbacks in callbacks, for example) but which does not require entirely unfamiliar (to me) syntax (e.g. functional programming) to achieve.
* Simplest possible syntax in most places, leading to the simplest possible correct code. I contrast that to Java, say, where I have to write an awful lot of code to get what I want (contrast, say, Go package accessibility rules like upper/lower case vs. the Java private/public/package debacle). This again helps me get running code very quickly without lots of boilerplate or other extraneous stuff.
* A bonus, for me, was discovering how easy it was in Go to marshall/serialize other things like JSON, HTML forms and XML. This has shortened my development cycles with those integrations too.

In summary, the issues mentioned in the slides seem quite valid, but are only a subset of all of the reasons one might choose, or discard, a language/environment. YMMV, basically, so you should choose the right language for the job. Go shows a lot of potential to be that language in a large subset of cases, even if not all.

- johnk

[Lars Seipel]

On Fri, Oct 16, 2015 at 10:59:41AM -0700, Egon wrote:
> > Goroutine Leaks
>
> Just started thinking, would it be possible to detect goroutines that
> haven't done anything in 1h? It should help to weed such goroutines out.

The stack dumps already show you how long a goroutine has been blocked
on a channel or mutex operation. For a human familiar with the program
it's not hard to see a leak from that once it occured.

Doing it automatically is something else entirely. A goroutine blocked
on a channel for an hour is not necessarily an error. It might just mean
that no one needed its service in that time.

[Ben Bangert]

In my case, a reverse proxy for SSL termination is not an option. Every socket has a cost associated with it, especially on AWS EC2. Each tcp connection gets a 4kb kernel send/receive buffer allocated to it, if your SSL termination has to proxy to the actual app, then you've doubled the sockets and TCP memory costs. On a side-note EC2 seems to limit your sockets to around 200k, regardless of instance size.

One of the things I did point out at the talk was that Go's SSL memory use is one of the lowest, at 20kb, and the code is actually readable.

I continue to write code in Go, Python, Haskell, and now some Rust. This talk was more of a case-study on a solution that we've been more productive with, for this particular project, not as an argument about why no one should use Go, etc. There's been lots of great blog posts and talks out there on people that've done Python, then gone to Go, I figured it'd be nice if some people saw there's some good reasons that doesn't need to always be the case.

[Ben Bangert]

So, maybe someone might know if this is feasible, because I think it'd be incredibly helpful. What if Go had a runtime analyzer that would sweep goroutines blocked on a channel, and just go check that some other goroutine out there still has the channel in scope? If no other goroutine, anywhere, has that channel in scope (nor is there a reference in a struct somewhere to it), then obviously it will *never* unblock, and the goroutine is leaked.

Ideally Go could auto-close channels that had a goroutine blocking on it, and there were zero possible references to it from any other goroutine.

[Axel Wagner]

Ben Bangert <b...@groovie.org> writes:
> In my case, a reverse proxy for SSL termination is not an option. Every
> socket has a cost associated with it, especially on AWS EC2. Each tcp
> connection gets a 4kb kernel send/receive buffer allocated to it, if your
> SSL termination has to proxy to the actual app, then you've doubled the
> sockets and TCP memory costs.

The frontend doesn't have to open a new connection for every incomming
connection, has it? As HTTP is stateless there should be nothing
preventing you from using keep-alive to just have a handfull of
long-running connections to it.

At least that's what I would have expected to happen in a reverse proxy.

[Axel Wagner]

Ben Bangert <b...@groovie.org> writes:
> So, maybe someone might know if this is feasible, because I think it'd be
> incredibly helpful. What if Go had a runtime analyzer that would sweep
> goroutines blocked on a channel, and just go check that some other
> goroutine out there still has the channel in scope? If no other goroutine,
> anywhere, has that channel in scope (nor is there a reference in a struct
> somewhere to it), then obviously it will *never* unblock, and the goroutine
> is leaked.
>
> Ideally Go could auto-close channels that had a goroutine blocking on it,
> and there were zero possible references to it from any other
> goroutine.

That is impossible, because it would be backwards incompatible. It might
potentially break correct programs.

What has been talked about in the past is, that such goroutines might
instead just being removed. As they could never progress anyway, just
reaping them away shouldn't change anything in terms of correctnes. As
far as I know, the only thing missing for this is someone writing up a
decent proposal (e.g. you would need to think about what to do in
selects) and implementing it.

All of that being said: A goroutine leak is an incorrect program
(currently). The go philosophy is that incorrect programs blow up sooner
rather than later so you shouldn't mask errors. A way to do blow up the
program safely when you know there's a leak might be better, maybe in
the same way the deadlock detection currently blows you up: Dump a
"goroutine-leak detected" with a stacktrace of all goroutines or something.

Best,

Axel

[Egon]

The solution doesn't have to be that hardcore to be useful.

Two simpler packages:

* package antifreeze that routinely checks whether any goroutines, excluding things in a whitelist, has been stuck for 10min+... if yes, then panics

* package icicle that routinely checks whether any goroutines, excluding things in a whitelist, is stuck for 10min+... logs, and adds them to the whitelist (alternatively, prints some shorter log, e.g. P1 is still stuck)

+ Egon

[krolaw]

Part of me wants that feature.  A bigger part however, wants out of scope channel blocked goroutines to fail compilation...

On Saturday, 17 October 2015 20:55:22 UTC+13, Axel Wagner wrote:. 

[Egon]

On Saturday, 17 October 2015 11:05:50 UTC+3, Egon wrote:

On Saturday, 17 October 2015 04:30:37 UTC+3, Ben Bangert wrote:

On Friday, October 16, 2015 at 3:39:38 PM UTC-7, Lars Seipel wrote:

On Fri, Oct 16, 2015 at 10:59:41AM -0700, Egon wrote:
> > Goroutine Leaks
>
> Just started thinking, would it be possible to detect goroutines that
> haven't done anything in 1h? It should help to weed such goroutines out.

The stack dumps already show you how long a goroutine has been blocked
on a channel or mutex operation. For a human familiar with the program
it's not hard to see a leak from that once it occured.

Doing it automatically is something else entirely. A goroutine blocked
on a channel for an hour is not necessarily an error. It might just mean
that no one needed its service in that time.

So, maybe someone might know if this is feasible, because I think it'd be incredibly helpful. What if Go had a runtime analyzer that would sweep goroutines blocked on a channel, and just go check that some other goroutine out there still has the channel in scope? If no other goroutine, anywhere, has that channel in scope (nor is there a reference in a struct somewhere to it), then obviously it will *never* unblock, and the goroutine is leaked.

Ideally Go could auto-close channels that had a goroutine blocking on it, and there were zero possible references to it from any other goroutine.

The solution doesn't have to be that hardcore to be useful.

Two simpler packages:

* package antifreeze that routinely checks whether any goroutines, excluding things in a whitelist, has been stuck for 10min+... if yes, then panics

Aand, here's a POC: https://github.com/egonelbre/antifreeze

[Ben Bangert]

This is a long-lived connection, we hold it open for hours on end, until the client drops, using SSL websockets. Amazon didn't seem too keen on terminating 90 million SSL connections for us on their ELB's unfortunately. Though the ELB can only retain the proxy for a max of 15 mins, which would cause too high of a connection churn for the amount of connections we're talking about.

The bit in the slides about possibly using some Rust, was for making our own such front-end. Though right now, everything is working fine.

[Micky]

Ben is mostly using others' bad code to point out that a language is flawed.
I am sure you can do that to pretty much any language out there.

Also, he compared his short amount of time coding in Go with years of
experience writing the other counter part.

As for the bugs, I am sure, it's justifiable considering the time Go's
been around vs 15, 20 or 25 years of other languages.

To his defense, it was better to go back where he came from because
that's what he mostly appears to have used in his career.

[Nicolas Grilly]

On Friday, October 16, 2015 at 7:59:41 PM UTC+2, Egon wrote:

I suspect that the Python examples lie about the 100% code coverage.. i.e. each function that can throw an exception, should be tried.

f = open("filename", "r+")

f.write("123")

f.close()

...

Would be same as:

f = open("filename", "r+")

if !thrown {

   f.write("123")

   if !thrown {

     f.close()

     if !thrown {

       ...

     }

   }

}

I.e. all of those hidden branches should be tested to get 100% coverage.

 That's really a great comment. I've never thought of that before. Thanks for sharing, Egon.

[Nicolas Grilly]

On Saturday, October 17, 2015 at 1:51:27 AM UTC+2, Ben Bangert wrote:

In my case, a reverse proxy for SSL termination is not an option. Every socket has a cost associated with it, especially on AWS EC2. Each tcp connection gets a 4kb kernel send/receive buffer allocated to it, if your SSL termination has to proxy to the actual app, then you've doubled the sockets and TCP memory costs. On a side-note EC2 seems to limit your sockets to around 200k, regardless of instance size.

One of the things I did point out at the talk was that Go's SSL memory use is one of the lowest, at 20kb, and the code is actually readable.

I continue to write code in Go, Python, Haskell, and now some Rust. This talk was more of a case-study on a solution that we've been more productive with, for this particular project, not as an argument about why no one should use Go, etc. There's been lots of great blog posts and talks out there on people that've done Python, then gone to Go, I figured it'd be nice if some people saw there's some good reasons that doesn't need to always be the case.

Ben, that's exactly what I wanted to know. Thanks for replying.

[Nicolas Grilly]

On Saturday, October 17, 2015 at 9:55:22 AM UTC+2, Axel Wagner wrote:

What has been talked about in the past is, that such goroutines might
instead just being removed. As they could never progress anyway, just
reaping them away shouldn't change anything in terms of correctnes. As
far as I know, the only thing missing for this is someone writing up a
decent proposal (e.g. you would need to think about what to do in
selects) and implementing it.

It sounds very similar to a garbage collector, isn't it?

All of that being said: A goroutine leak is an incorrect program
(currently). The go philosophy is that incorrect programs blow up sooner
rather than later so you shouldn't mask errors. A way to do blow up the
program safely when you know there's a leak might be better, maybe in
the same way the deadlock detection currently blows you up: Dump a
"goroutine-leak detected" with a stacktrace of all goroutines or something. 

A goroutine leak, instead of being considered as an incorrect program, could be just considered as something that is not used anymore and must be collected by the "zombie goroutine collector"?

[Egon]

Note, I should probably clarify that Go has a similar problem with panics.

Aaaand... I'm completely disregarding the question how valuable and/or wasteful 100% code coverage is. As a simple rhetorical question, would you benefit more from making code clearer by 50% (by some reasonable metric) or making tests cover 1% more, using the same amount of time? Or when would one outweigh the other?

Note, questions are rhetorical and I'm not expecting real answers, because good answers to those questions will start with "it depends on X, Y, Z, W ..." and go on for a few pages.

+ Egon

[Axel Wagner]

Nicolas Grilly <nic...@vocationcity.com> writes:
> It sounds very similar to a garbage collector, isn't it?

Yes and probably the garbage collector would play a major role in the
implementation. However, there are more nuances to the implementation
that one has to think about.

> A goroutine leak, instead of being considered as an incorrect program,
> could be just considered as something that is not used anymore and must be
> collected by the "zombie goroutine collector"?

That would require a change to the spec and would be incompatible with
the go1 stability guarantee. Think about third-party compilers (like
gopherjs). All of them would become incorrect with this language change.

I don't think that has to necessarily mean we can't do it. But a
reasonable migration path has to be figured out at the very least to
give them time to implement this and gauge how well that works and
everything :)

Best,

Axel

[Nicolas Grilly]

On Sun, Oct 18, 2015 at 9:09 AM, Axel Wagner <axel.wa...@googlemail.com> wrote:

Yes and probably the garbage collector would play a major role in the
implementation. However, there are more nuances to the implementation
that one has to think about.

If a goroutine is blocked on a channel that is not in scope anywhere else, and the runtime reaps this blocked goroutine, what is it supposed to do regarding resource release (file descriptors, sockets, external locks, etc.)? Do we panic at the point where the goroutine is blocking on send or receive, in order to run defer statements? I'm afraid there is a lot of subtle issues here.

[Axel Wagner]

The answer is: We can do nothing. Doing anything like panicing would
potentially break currently correct programs. As a goroutine blocked
on a channel operation that is impossible to succeed can't interact in
any way with a program anyway, reaping it without taking any other
action won't influence the program. It will just reclaim some
memory. This is the only safe option.

[Nicolas Grilly]

On Sun, Oct 18, 2015 at 10:01 AM, Axel Wagner <axel.wa...@googlemail.com> wrote:

The answer is: We can do nothing. Doing anything like panicing would
potentially break currently correct programs. As a goroutine blocked
on a channel operation that is impossible to succeed can't interact in
any way with a program anyway, reaping it without taking any other
action won't influence the program. It will just reclaim some
memory. This is the only safe option.

In such a case, I'm not sure the benefit is strong enough to compensate the cost in complexity. Moreover, debugging resource leaks will be a lot harder after the "owner" goroutines were reaped by the runtime.

[Klaus Post]

On Sunday, 18 October 2015 09:42:09 UTC+2, Nicolas Grilly wrote:

If a goroutine is blocked on a channel that is not in scope anywhere else, and the runtime reaps this blocked goroutine, what is it supposed to do regarding resource release (file descriptors, sockets, external locks, etc.)? Do we panic at the point where the goroutine is blocking on send or receive, in order to run defer statements? I'm afraid there is a lot of subtle issues here.

I had exactly the same thought as you. And while this isn't reasonable in *production*, I cannot see why this cannot be implemented similar to the race detector.

I can see at least three things that could be checked:

1) A select {} that is blocked on a number of channel reads/writes, where none of the channels have any references left elsewhere.

2) A sync.Mutex what is blocked but no one else holds a reference to it.

3) A sync.Condition/sync.WaitGroup that it stuck in Wait(), but no-one is holding a reference to it.

If any of these conditions are met, it will display the "leak" similar to how "-race" displays races. Of course this will find (a lot) of bad existing code, but that would be the purpose.

Maybe good people like Dmitry Vyukov or Andrew Gerrand would consider such a tool for 1.7 :)

/Klaus