Compile Go to another language

[Archos]

What steps there is to take to build a compiler which compiles Go to
another language?

[Archos]

The first step is to use the Go scanner to parsing a file with Go
syntax.
What's the next?

[André Moraes]

Probably the hardest part will be keeping the semantics of
panic,recover and chanels on the target language.

Like you said, the go std lib has a lexer and tools to build a AST of
you go file.

After that is just a matter of translating the concepts from Go to
your other language.
--
André Moraes
http://andredevchannel.blogspot.com/

[chris dollin]

2011/11/17 André Moraes <and...@gmail.com>:

> After that is just a matter of translating the concepts from Go to
> your other language.

"Just", as in that's the hardest part of the whole enterprise (depending
a little\\\\\ lot on which target language you choose to use and how much
performance you're prepared to trade off and how much of the
original Go structure you want to preserve in the compiled code).

(I've vaguely wondered about a Java-to-Go translator to take first
steps in converting existing Java code to Go, but it's probably more
sensible, if less meta-satisfying, to just rewrite from scratch guilded
by the available code.)

Chris

--
Chris "allusive" Dollin

[Archos]

On Nov 17, 1:28 pm, André Moraes <andr...@gmail.com> wrote:
> After that is just a matter of translating the concepts from Go to
> your other language.

But what tool should be used? a parser like bison?

[Jim Whitehead II]

No, as people have already said, the difficulty is not in parsing and
determining what a Go program is doing, but it is translating those
concepts into another language. That's not done with a parser, that's
done with a compiler that is aware of the source and target languages,
in this case Go and the other language.

I'm not sure what you're trying to accomplish, but the tools for
lexing and parsing Go code are already well established, they're in
the standard libraries. If you want to convert your Go code to another
language, you need to define a way to translate each of the different
concepts.

- Jim

[chris dollin]

Um, no. Bison and its ilk are for parsing token streams.

If you're compiling to some kind of machine code, then there are
tools to help with that, but they're probably written in C or C++.
If you're compiling into something like C, then you assemble
structures that represent C (etc) code and write those out.

As it happens, writing out C isn't particularly hard, either. What's
hard is deciding what C to write and what information is needed
to write it, and that's the hard part, and the part you have to do
with your brain and not a tool.

So if you really, really want to compile Go into some other
language, pick a language -- just one, because this job is hard
enough with one and trying to do another will just burn effort -- and
work out how to represent each of Go's constructs in that language
and what runtime support they will need.

(I have a compiler for a dynamically-typed open-stack programming
language which translates into C, so I have played this game
before. I wimped out and didn't write the garbage collector. The
generated C isn't fit for consumption by humans.)

[André Moraes]

>> After that is just a matter of translating the concepts from Go to
>> your other language.

I should have placed "just a matter of translating" under quotes.

I don't intended to mean that this is easy but after loading the source code
in the AST what is left is translate that into the target language.

So he don't need to work with tools like bison,lex,yacc, etc...
all that is already done.

Sorry for the misunderstood

[Archos]

No problem. It's just that I've no experience with lexers and parsers.
And I was watching how has been created CoffeScript so I was confused
with all that.

Yes, it's to translate from Go to JS because I'm tired of "fighting"
with JS at creating the user interface for some server.

[John Asmuth]

I've been fooling around with the idea of creating something like GWT, except for go.

The skeleton is published at github.com/skelterjohn/goui, but it doesn't do much yet.

The idea is that you'd interface only with a go library, and it'd serve the right html/js/css to have it look nice, allow refreshing from the server (ie not based on a user event on the client side), etc.

[chris dollin]

On 17 November 2011 15:53, Archos <raul...@sent.com> wrote:

> Yes, it's to translate from Go to JS because I'm tired of "fighting"
> with JS at creating the user interface for some server.

You'll probably spend just as much time fighting your go-to-js
compiler and non-idiomatic generated js code. This is not a
trivial conversion to do.

Watch out for longs!

[Brian Ketelsen]

This would be quite a worthy project. And a big one!

[John Asmuth]

I think the basics wouldn't be too hard, if I knew html/js/css worth a damn.

Proof of concept for a component that can be refreshed by the server (using websocket), sending messages back to the server (using ajax), being placed correctly (using css), and being interactive (using js/jquery?).

Building up a decent set of widgets would be a considerable task, and if I got anywhere with the basics I'd have to lean on the community for help with the rest :)

[André Moraes]

I already thought about doing the go-to-js conversion.

But after thinking and rethinking the value of such solution (and the
ammount of problemns that I would need to solve).

I decided to follow two paths:
For the "single page application" use something like GWT or a very
good js library (Ext-Js maybe). There is this project Jangaroo
(http://www.jangaroo.net/home/) which compiles from AS3 to JavaScript
code and I currently studying it.

For the "good and old multi page site" I just wrote a view engine on
top of the std library, very easy to use and I am really loving the
new template package.

That was just my personal opinion, also the Dart Language from google
tries to make some concepts like chanels to the browser, take a look
at that project too.

[Ian Lance Taylor]

Archos <raul...@sent.com> writes:

> No problem. It's just that I've no experience with lexers and parsers.
> And I was watching how has been created CoffeScript so I was confused
> with all that.
>
> Yes, it's to translate from Go to JS because I'm tired of "fighting"
> with JS at creating the user interface for some server.

I think the hardest part of translating Go to JavaScript would be the
runtime library support for go, panic, defer, recover, channels, and
select. There are other hard parts, but I would start by thinking
seriously about how those could be implemented--i.e., how to write a
JavaScript program which uses those concepts. If you can get all those
working, I think the rest can be done.

Ian

[Kyle Lemons]

I think the hardest part of translating Go to JavaScript would be the
runtime library support for go, panic, defer, recover, channels, and
select.  There are other hard parts, but I would start by thinking
seriously about how those could be implemented--i.e., how to write a
JavaScript program which uses those concepts.  If you can get all those
working, I think the rest can be done.

Indeed.  FWIW, some of this work has been done already for Dart.  It has support for message passing and events, so (to me) it seems like a better candidate for conversion of Go in a browser.  It will compile down to JavaScript for you, and will theoretically have first-class support in Chrome (and theoretically other browsers too) at some point.

[Glenn Brown]

> What steps there is to take to build a compiler which compiles Go to
> another language?

Here's an excellent post describing language transformation:
http://www.moserware.com/2008/06/ometa-who-what-when-where-why.html
It uses OMeta, an intriguing "meta-language" to simplify the process,
as an alternative to repurposing an existing scanner/parser.

The real challenge will be mimicking the features that do not map
one-to-one to the target language. Any advanced features requiring
runtime support will require you to create the equivalent runtime
library in the target language.

But if you want to play and learn, you could start small with Ometa and just
arithmetic expressions, then add 'var' and '=', and build from there.

--Glenn

[Archos]

I really dislike Dart. If you like Java, you'll like Dart... but if I
use Go is because I dislike Java

[Archos]

Very interesting!

You could be interested in this gui, and overall in its web gui


https://github.com/droundy/gui
https://github.com/droundy/gui/blob/master/web/web.go

[Archos]

But my idea is not emulate (or translate) those statements. I mean, I
want to use Go instead of JS and for it I don't need to use those Go's
features. There will be a warning or an error when some of these
statements are used at compiling at JS.

Anyway, I'll think if it's possible to emulate some of them but I'm
not going to thing about it from the beginning. When it's finished
then there will be time.

On Nov 17, 5:29 pm, Ian Lance Taylor <i...@google.com> wrote:

[j...@webmaster.ms]

1. Do the type inference. There is some (incomplete) typechecker next to Go parser. Type inference seems not to be complex, but it is quite tricky, especially with << (see shift2.go in test directory).

2. Give types to the polymorphic functions (infix +, etc). Note that some of arithmetic functions (+ for strings, <<, >>) may works not like in C or your target lang, thus it would be better to convert them to functions (__shl_32_32, __plus_str, ...) at the AST level.

3. Resolve consts. That implies infinite-precision arithmetic with complex numbers. There are also tricky issues with type casting inside the const declarations.

4. Resove the syntax which cannot be emited as-is to C or javascript, for example { x := 1; { x := x + 1; /* two different x'es in a expression */ } }

5. Remove syntax sugar from AST to make further processing simpler (It may be worth to define "Kernel Go" (same way as Kernel Mozart/Oz), which is still compilable Go but has minimal number of construct).

 - get rid of constructions with duplicated semantic (&([100]int{}) and new([100]int), make([]int, 50, 100) and new([100]int)[:50])

 - get rid of :=, "x := 1", "var x int = 1", "var x = 1" to be the same

 - desugar multielement assigment ( *a(),*b()=c(),d() ), see Go 1 spec for the evaluation order.

 - desugar swap(swap(a,b))

 - implement 'switch' and 'for' via 'goto' (and get rid of break/continue/fallthrough)

 - convert struct comparision to elementwise comparision

 - desugar named returns to local vars with uniq names

 - make zero-initialization of vars explicit

 - ...

[Archos]

If I read all that before of start with it, then I don't build it...
Just kidding! The project is being fun once that you get experience
with go/Parser.

And thanks! I just to add it to the ToDo list for don't forget it

== Status
In a little time, maybe today, I'll have finished the translating of
all declarations included structs. So I hope that I can enjoy writing
Go -> JS scripts at the beginning of the new year :)

All code transformed is being checked through JavaScript Lint

== Advantages

+ Simple and clean sintaxis (due to Go).
+ Use one only language for all development. Also, I will don't need
to contract to some JS developer.
+ The translation will be line to line so if you get an error in the
JS file you'll can match this its Go code. This is not possible with
another translating compilers and it's very important. (It is in my
ToDo list).

[Jan Mercl]

On Saturday, November 26, 2011 3:12:43 PM UTC+1, Archos wrote:

If I read all that before of start with it, then I don't build it...
Just kidding! The project is being fun once that you get experience
with go/Parser.

And thanks! I just to add it to the ToDo list for don't forget it

== Status
In a little time, maybe today, I'll have finished the translating of
all declarations included structs. So I hope that I can enjoy writing
Go -> JS scripts at the beginning of the new year :)

I'm not sure if such optimism is rational. E.g. what is/should be the JS version of the following Go declaration?

        var v int64 = 1<<63-1 // JS can't natively represent this number exactly

And that's only scratching the surface of a huge number of other problems in such translation. Don't get me wrong, in theory the translation can be done (actually into any Turing complete language by definition).

[Archos]

The mathematical expressions will be calculated during the translating

[Archos]

To stay it very clear. I'm implementing a subset of Go which could be
translated to JS. Things like channels, goroutines, numbers of 64
bits,etc., could not be used in JS so my program will show an error
about it.

The idea is to use the sintaxis of Go to write JS.

On Nov 26, 2:54 pm, Jan Mercl <jan.me...@nic.cz> wrote:

[Jan Mercl]

On Saturday, November 26, 2011 4:36:50 PM UTC+1, Archos wrote:

On Nov 26, 2:54 pm, Jan Mercl <jan....@nic.cz> wrote:
> I'm not sure if such optimism is rational. E.g. what is/should be the JS
> version of the following Go declaration?
>
>         var v int64 = 1<<63-1 // JS can't natively represent this number
> exactly
>

The mathematical expressions will be calculated during the translating

How does that solve the problem of JS having no native way to represent exactly the resulting numbers ("calculated during the translating")? Will your JSGo "runtime" evaluate all integers in a string representation or what? The translating can't in general change the semantics of the original Go program. But, preserving that semantics, what concerns just Go's integer types, *and* reaching acceptable performance - how it'll be done?

And finally, this is only one of several major problems to solve.

I'm not convinced your project is going to fly.

[Jan Mercl]

On Saturday, November 26, 2011 4:52:01 PM UTC+1, Archos wrote:

To stay it very clear. I'm implementing a subset of Go which could be
translated to JS. Things like channels, goroutines, numbers of 64
bits,etc., could not be used in JS so my program will show an error
about it.

The idea is to use the sintaxis of Go to write JS.

OK, so then it's not a Go to JS translator, but a new Go like syntax on top of the existing JS semantics. Which is a different problem and my previous concerns are no more applicable to it.

Anyway please design the syntax to be not accepted by the Go compiler to avoid confusion caused by valid GoJS programs which happen to compile but don't work in Go (due to differences in Go's and JS's semantics). Like e.g. instead of "package" define some other keyword to disambiguate it early. Though then you would not be able to use the original go/parser package anymore (but you can simply "fork" it).

[j...@webmaster.ms]

No language (except Go) has native way to represent interfaces,
channels and complex numbers, so they suppose to be represented as
structs in target lang.
Also, int64 can be represented as structs in case of JavaScript
target.

Corresponding all integers to target's native types looks promising as
it allows to skip the step of type inference for polymorphic
functions, emiting "+" for "+" in Go code, etc. But doing so would
raise other problems due to lack of native complex numbers or
different semantic of bitwise shift (when the right param is bigger
than the bitwidth of the left param), etc.
Otherwise, converting all +,-,*,/,<<,>>,... to the non-polymorphic
functions (_plus_8, _plus_16, _plus_str, ...) allows to use non-native
int64 same way as non-native channel, interface, ...

I'd say, the most difficult part of the translator is the simplifier,
which will convert "var v int64 = 1<<63-1" to "var v int64; v =
CONSTDECL_I64(0x7FFFFFFFFFFFFFFF);".
Then such a simplified code can be converted easily.
int64 would be a type in target lang (struct in case of Javascript,
native in case of C), CONSTDECL_I64 is a kind of macro (which forms
struct constsnt in case of Javascript, or append LL suffix in case of
C). "1<<63-1" has to be calculated at the translation stage (it can be
also "(1<<100)>>37", which is correct for Go, but will not work in
most targets being just passed there).