Advice on how to join golang team at Google

[⚛]

I would like to join the golang team at Google in Slovakia as a full time job related to improving the Go compiler and runtime. I do not know how to execute this and what steps to follow. I would like to ask you for an advice.

Thanks.

[minux]

On Feb 25, 2014 2:48 AM, "⚛" <0xe2.0x...@gmail.com> wrote:
> I would like to join the golang team at Google in Slovakia as a full time job related to improving the Go compiler and runtime. I do not know how to execute this and what steps to follow. I would like to ask you for an advice.

That's really great news!
PS: I don't think you need any advice besides what you've already done.

[Ian Lance Taylor]

Here are some general comments that may or may not apply to you
specifically.

Google is a very large company with an unavoidable level of
bureaucracy. The Go team at Google is a small team and like any other
small team we operate under limits. We can not choose to hire as many
people as we like or think we need. I assure you that we would like
to have more people working on Go than we actually do, and it's really
great that we have so many contributions from people outside Google.

Google in general does not hire programmers into specific positions.
While there are exceptions to this, Google normally prefers to hire
people with good general skills and then assign them to teams that
need people. It's not completely arbitrary, and people's wishes are
taken into account, but it means that most people hired at Google do
not know specifically what they will be working on.

The combination of these facts means that there is no real way to
apply for a job on the Go team, and there is no real way for the Go
team to hire somebody.

Also, I should note that Google in general expects programmers to work
in a Google engineering office. It is quite unusual for Google to
permit people to work remotely. As far as I know Google does not have
an engineering office in Slovakia (there is a small sales office in
Bratislava but I don't think there are any programmers there).

So the way to join the Go team at Google is to first get a job at
Google. See http://google.com/jobs for a lot more information about
that. As with any large company, the best way to start that process
is to ask somebody already at Google to refer you. I'm sure there are
people on this list who would be happy to do that. Note that getting
a job at Google is hard; the percentage of applicants who get hired is
small (I don't know the exact numbers). I have personally seen people
I thought were excellent programmers fail to get hired, for reasons
that are opaque to me.

Once you get hired at Google, it is possible, though quite unlikely,
that you would be placed directly onto the Go team. Otherwise, after
joining, you would have to contact the Go team (of course in your case
we know you already) and see about an internal transfer. That would
only be possible when we are permitted to add somebody to the team.
We normally have a long list of people who would like to join the
team. When and if we have an opening we are able to choose the person
we think would be the best fit.

So you asked for advice, and that's the best advice I can give you. I
should make clear that I am speaking only for myself here. Although I
work at Google I am not in any way an official representative of the
company, and although I think what I'm saying is accurate I could be
mistaken, and conditions could change over time.

Ian

[⚛]

Thank you for explaining it. The conclusion is, basically, that it is impossible.

I predict that Google's policy will in the end generate a significant delay (decades) of the next stage of compiler technology. By this I do not necessarily mean my work on the Tulgo compiler, I mean it in a more general way.

Basically the identification of the next stage in compiler technology will be easy to discern: the compiler will be able to more-or-less routinely add SIMD and multi-threading to benchmarks such as the Mandelbrot Go benchmark. That is where things are heading in the long term. Go is more appropriate than C for this task because it is safer than C.

I will try to achieve this goal on the Mandelbrot benchmark, but it will surely take many years of effort. Currently I have nothing concrete to back my claims, so you do not need to trust my prediction.

Well, I hope that the above goal does not seem too far stretched.

If Google is somehow interested in employing a person with the above research goal, please let me know. I will be heading to the goal irrespective of whether Google employs me or not, the only difference is that with Google the mentioned goal may perhaps be reached more quickly.

Thank you for your patience.

[Han-Wen Nienhuys]

On Tue, Feb 25, 2014 at 5:48 PM, ⚛ <0xe2.0x...@gmail.com> wrote:
> If Google is somehow interested in employing a person with the above
> research goal, please let me know. I will be heading to the goal
> irrespective of whether Google employs me or not, the only difference is
> that with Google the mentioned goal may perhaps be reached more quickly.

Google does have some grant programs for research, see
http://research.google.com/university/ - I'm not sure if you would be
eligible, but it might be worth a look.

--
Google Germany GmbH - ABC-Str. 19 - 20354 Hamburg
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg - Geschäftsführer: Graham Law,
Christine Elizabeth Flores

[Niklas Schnelle]

You do realize Go is one of the lesser optimizing modern compilers simply because it wasn't a primary goal in development?
Maybe have a look into llgo, the LLVM backend already does put a lot of effort into optimization, especially automatic vectorization i.e

using SIMD. I don't get why you obsess about benchmarks though, obviously benchmarks are just a tool to do a somewhat educated

guess on the optimization capabilities. Take a look at http://llvm.org/docs/Vectorizers.html

[minux]

On Feb 26, 2014 7:21 AM, "Niklas Schnelle" <niklas....@gmail.com> wrote:
> You do realize Go is one of the lesser optimizing modern compilers simply because it wasn't a primary goal in development?

that's not correct. the goal should be as fast as possible without impacting the compilation speed too much.
And I remembered that Rob once said with the new Go compiler in Go, it's possible to add vectorization.

to put in another way, low-overhead optimizing compiler is the goal.

> Maybe have a look into llgo, the LLVM backend already does put a lot of effort into optimization, especially automatic vectorization i.e
> using SIMD. I don't get why you obsess about benchmarks though, obviously benchmarks are just a tool to do a somewhat educated
> guess on the optimization capabilities. Take a look at http://llvm.org/docs/Vectorizers.html

we (the go users) certainly want the gc toolchain to be more optimizing, without using LLVM (consider how much time/memory is required to simply compile LLVM, I'm still glad that they made the decision to not use LLVM in gc).

[Niklas Schnelle]

You are correct, optimization is definitely a goal and gc already has many classical optimizations like registeriazation and even inlining but I mean't it more

in a "compared to gcc or llvm" way. For both llvm and gcc the performance of the resulting code vs compilation time goal has definitely a different leaning.
Which is fine in and of itself but makes the gc toolchain less suited when you are interested in doing bleeding edge vectorization work which sounds

like what OP is after.

[⚛]

On Wednesday, February 26, 2014 1:21:21 PM UTC+1, Niklas Schnelle wrote:

You do realize Go is one of the lesser optimizing modern compilers simply because it wasn't a primary goal in development?

That is questionable. For example, in func block() in http://golang.org/src/pkg/crypto/sha256/sha256block.go the compiler is emitting rotation instructions. However this optimization works only with the source code of func block() as it is. If the source code is slightly rearranged, the compiler can no longer recognize the pattern and will not emit the rotation instructions. In my opinion what I just described means that performance is in fact an important goal, but the compiler is making illogical shortcuts to achieve the goal (in my opinion).

 

Maybe have a look into llgo, the LLVM backend already does put a lot of effort into optimization, especially automatic vectorization i.e

using SIMD. I don't get why you obsess about benchmarks

There is a simple explanation for this: even the smaller benchmarks have sufficient complexity that enables me to test the algorithms used by the Tulgo compiler.

Secondly, no compiler that I know of is capable of autovectorizing and/or auto-multithreading the Mandelbrot benchmark.

[Gustavo Niemeyer]

On Wed, Feb 26, 2014 at 1:12 PM, ⚛ <0xe2.0x...@gmail.com> wrote:
> but the compiler is making illogical shortcuts to achieve the goal (in my opinion).

It's not a shortcut, but rather a tradeoff. All the improvements that
have shown up in Go 1, 1.1, 1.2 and that are coming in 1.3, are a lot
more interesting to some than raw speed. Being a tradeoff also means
the optimization can eventually be done. Ideally you or somebody else
will find the problem interesting enough, and have the time to contribute.


gustavo @ http://niemeyer.net

[⚛]

In my opinion a better solution for the compiler to enable the rotate instruction in a consistent way would be to add a runtime.RotateLeft32(int32, int32) function. The compiler would recognize this function and generate the rotation instruction. This would enable programmers to rearrange their codes at will and be sure that the compiler emits the instruction. If it is just a hardcoded pattern and not a special function, like in the current Go compiler, we cannot be sure that the compiler emits the instruction.

[Michael Jones]

I believe the (previously discussed?) idea of operators "<<<" amd ">>>" to mean rotate rather than shift has merit.What Atom Symbol suggests is along these lines. Personally, I see both exponentiation and the W() function as first-class intrinsics too, but that's just me. ;-)  

--
 
---
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

Michael T. Jones | Chief Technology Advocate  | m...@google.com |  +1 650-335-5765

[minux]

On Wed, Feb 26, 2014 at 12:08 PM, ⚛ <0xe2.0x...@gmail.com> wrote:

In my opinion a better solution for the compiler to enable the rotate instruction in a consistent way would be to add a runtime.RotateLeft32(int32, int32) function. The compiler would recognize this function and generate the rotation instruction. This would enable programmers to rearrange their codes at will and be sure that the compiler emits the instruction. If it is just a hardcoded pattern and not a special function, like in the current Go compiler, we cannot be sure that the compiler emits the instruction.

Speaking of this problem, I believe the more desirable way is to make the gc compiler able

to inline (simple) assembly functions (even if the user need some special annotation in the

assembly, but I don't expect the compiler to need much special annotation). Which means

if you have a very simple inner loop code segment that the compiler isn't able to optimize

correct, you can just code that segment in assembly, instead of the whole loop (which is

the case for today's assembly code in Go tree).

Then we don't need to care about rotate, add/sub which returns result and carry separately, etc.

I'm not keen on making the compiler detect specific routines and emit better code for that.

(Optimizing for a single special case is always limiting, as new special cases keep coming

up.)

[minux]

On Wed, Feb 26, 2014 at 7:40 PM, minux <minu...@gmail.com> wrote:

On Wed, Feb 26, 2014 at 12:08 PM, ⚛ <0xe2.0x...@gmail.com> wrote:

In my opinion a better solution for the compiler to enable the rotate instruction in a consistent way would be to add a runtime.RotateLeft32(int32, int32) function. The compiler would recognize this function and generate the rotation instruction. This would enable programmers to rearrange their codes at will and be sure that the compiler emits the instruction. If it is just a hardcoded pattern and not a special function, like in the current Go compiler, we cannot be sure that the compiler emits the instruction.

Speaking of this problem, I believe the more desirable way is to make the gc compiler able

to inline (simple) assembly functions (even if the user need some special annotation in the

assembly, but I don't expect the compiler to need much special annotation). Which means

if you have a very simple inner loop code segment that the compiler isn't able to optimize

correct, you can just code that segment in assembly, instead of the whole loop (which is

the case for today's assembly code in Go tree).

Even better, after the inline phase, the compiler should do register allocation with the rest of

the code. Essentially the user teaches the compiler to use a better instruction template.

Yes, this means that the compiler needs more understanding of the machine instructions.

(that's where the compiler might need extra user annotations, esp. when the user uses

BYTE/WORD instruction for unsupported instructions.)

[ron minnich]

On Wed, Feb 26, 2014 at 8:12 AM, ⚛ <0xe2.0x...@gmail.com> wrote:

Secondly, no compiler that I know of is capable of autovectorizing and/or auto-multithreading the Mandelbrot benchmark.

 

Given the number of autovectorizing compilers written in the last 30 years it's possible that it's because this particular computation is not a driver for anything in HPC today.

The last time I remember anyone talking much about Mandelbrot as a driver in parallel computation was the 90s. In fact your comment brought back old memories of mandelbrot-over-pvm on PC clusters from those days :-)

In terms of the optimizations you are talking about, did you ever look at what the ?l linkers do in the way of inlining? I don't know at the moment if that stuff is even active any more, or if ?l has any role to play in the future of Go. I just played with the innards of 6l for a while on Plan 9 for some object code transformations I wanted to do. It was really easy. And 6l in Go has its roots in the Plan 9 version. 

ron