Re: [go-nuts] Is Go 1.1 going to have new dynamic features?

1,961 views
Skip to first unread message

Steven Degutis

unread,
Apr 25, 2013, 12:10:11 PM4/25/13
to Gustavo Niemeyer, golan...@googlegroups.com
Then this means we can't put Clojure (or anything like it) on top of Go.


On Thu, Apr 25, 2013 at 11:08 AM, Gustavo Niemeyer <gus...@niemeyer.net> wrote:
On Thu, Apr 25, 2013 at 1:06 PM, Steven Degutis <sbde...@gmail.com> wrote:
> For example, will it have the ability to lookup functions by name at
> runtime?

This will most likely never come, because it's also an important
misfeature. If you can access functions by name at runtime, the
compiler can't trash unused code.

As a side note, in my experience an explicit registry has always been
a better interface in any case.


gustavo @ http://niemeyer.net

Gustavo Niemeyer

unread,
Apr 25, 2013, 12:08:52 PM4/25/13
to Steven Degutis, golan...@googlegroups.com

Gustavo Niemeyer

unread,
Apr 25, 2013, 12:13:37 PM4/25/13
to Steven Degutis, golan...@googlegroups.com
On Thu, Apr 25, 2013 at 1:10 PM, Steven Degutis <sbde...@gmail.com> wrote:
> Then this means we can't put Clojure (or anything like it) on top of Go.

There are several ways to make that kind of thing work, but to be
honest I'm not personally concerned about that one limitation. If I
wanted Clojure, I'd just use it.


gustavo @ http://niemeyer.net

Steven Degutis

unread,
Apr 25, 2013, 12:18:46 PM4/25/13
to Gustavo Niemeyer, golan...@googlegroups.com
That kind of attitude never leads to useful innovation.

Imagine if Rob et al. had that kind of attitude! They'd never have written Go, they'd have said "look, if you want something low-level just use C or C++".

Henrik Johansson

unread,
Apr 25, 2013, 12:30:52 PM4/25/13
to sbde...@gmail.com, Gustavo Niemeyer, golang-nuts

b is probably true but a is only true if you like lisp... ;-)

On Apr 25, 2013 11:23 PM, <sbde...@gmail.com> wrote:
To make my point more explicit:

(1) Clojure sits on top of Java
(2) Clojure enhances Java
(3) Go is better than Java
(4) If Clojure sat on top of Go, it would (a) enhance Go, and (b) be better than Clojure-on-Java


On Thursday, April 25, 2013 11:18:46 AM UTC-5, Steven Degutis wrote:
That kind of attitude never leads to useful innovation.

Imagine if Rob et al. had that kind of attitude! They'd never have written Go, they'd have said "look, if you want something low-level just use C or C++".


On Thu, Apr 25, 2013 at 11:13 AM, Gustavo Niemeyer wrote:

On Thu, Apr 25, 2013 at 1:10 PM, Steven Degutis wrote:
> Then this means we can't put Clojure (or anything like it) on top of Go.

There are several ways to make that kind of thing work, but to be
honest I'm not personally concerned about that one limitation. If I
wanted Clojure, I'd just use it.


gustavo @ http://niemeyer.net

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

andrey mirtchovski

unread,
Apr 25, 2013, 12:30:48 PM4/25/13
to sbde...@gmail.com, golang-nuts, Gustavo Niemeyer
An elephant on top of a turtle may somewhat enhance an outdated cosmology, true [1], but an elephant on top of a gopher will definitely not enhance ours :)

--

Rob Pike

unread,
Apr 25, 2013, 12:54:09 PM4/25/13
to andrey mirtchovski, sbde...@gmail.com, golang-nuts, Gustavo Niemeyer
I'd prefer to have gophers all the way down.

-rob

Kamil Kisiel

unread,
Apr 25, 2013, 1:06:50 PM4/25/13
to golan...@googlegroups.com, Gustavo Niemeyer, sbde...@gmail.com
Really? There are a lot of dynamic languages implemented on top of C/C++ and it has similar behaviour.

Steven Degutis

unread,
Apr 25, 2013, 1:08:38 PM4/25/13
to Kamil Kisiel, golan...@googlegroups.com, Gustavo Niemeyer
Yes really. How can I dynamically call net.Dial() from inside a dynamic language implemented in Go? Apparently I can't, and from the sounds of it never will be able to.

-Steven

Steven Johnson

unread,
Apr 25, 2013, 1:11:32 PM4/25/13
to golang-nuts
On Thu, Apr 25, 2013 at 10:03 AM, <sbde...@gmail.com> wrote:
You need code to be super fast, I don't. You need it to be super efficient on memory, I don't.

I would have thought that being fast and memory-efficient was a desirable property of all languages, regardless of what sort of thing you're writing.


 

Gustavo Niemeyer

unread,
Apr 25, 2013, 1:13:49 PM4/25/13
to sbde...@gmail.com, golan...@googlegroups.com, andrey mirtchovski
On Thu, Apr 25, 2013 at 2:03 PM, <sbde...@gmail.com> wrote:
> This is why it should come as no surprise to me that Go will never be
> suitable for what I do for a living, which is writing web apps.

My comments were well-meaning, Steve. Many of us do write web apps in
Go and are very happy with it, and there are high-profile web services
at Google that are written in Go.

This is all unrelated to retrieving functions by name at runtime. You
could implement a registry to add the functions you want to access at
runtime, and you could even automate that task quite easily with Go,
if you really wanted to do that.

I am not interested on running Clojure on Go, but that doesn't mean
you or anyone else can't be. Diversity is one of the things that make
a community strong.

And on that note, I'm back to coding.


gustavo @ http://niemeyer.net

Rob Pike

unread,
Apr 25, 2013, 1:16:44 PM4/25/13
to sbde...@gmail.com, golan...@googlegroups.com, andrey mirtchovski, Gustavo Niemeyer
That is a peculiar overreacotion to a polite little response to a polite little joke.

-rob

On Thursday, April 25, 2013, wrote:
And I just realized why. You have totally different requirements than us.

You write low-level server code at Google. We wright high-level web apps.

Our requirements are fundamentally different. Go will never be suitable for this task, only for what you're using it for.

In fact, you even admitted you wrote it to replace C++, not Ruby.

So your requirements are very different. You need code to be super fast, I don't. You need it to be super efficient on memory, I don't.

For you, it's better to write a for-loop that looks for the index of a given element and breaks out of the loop when you've found it. But we prefer to have a single function like indexOf() which takes a collection and a function, because it's cleaner, easier to read, easier to write, and easier to maintain. But that's probably too inefficient for you, and it isn't type-safe enough for you either.

This is why it should come as no surprise to me that Go will never be suitable for what I do for a living, which is writing web apps.

-Steven


On Thursday, April 25, 2013 11:54:09 AM UTC-5, Rob Pike wrote:
I'd prefer to have gophers all the way down.

-rob

--

Henrik Johansson

unread,
Apr 25, 2013, 1:22:57 PM4/25/13
to sbde...@gmail.com, golang-nuts

Has these functional aspects been ruled out of future versions of Go? They are not related to dynamically looking up and invoking methods.
I wouldn't mind them really but I can do without. As for lisp on Go, cool but I would never use it.

On Apr 26, 2013 12:17 AM, <sbde...@gmail.com> wrote:
To some extend, yes.

But all features are trade-offs, so they come at a price.

In giving high-level functions like map(), reduce(), indexOf(), etc, we would be losing type-safety and some efficiency/speed.

I'm okay with this trade-off. In fact I prefer the code's clarity and maintainability over efficiency. Especially because in the apps I write, speed and memory are not the bottleneck.

-Steven
--

Brad Fitzpatrick

unread,
Apr 25, 2013, 1:25:05 PM4/25/13
to sbde...@gmail.com, golang-nuts, andrey mirtchovski, Gustavo Niemeyer
Google should write some web apps.



On Thu, Apr 25, 2013 at 10:03 AM, <sbde...@gmail.com> wrote:
And I just realized why. You have totally different requirements than us.

You write low-level server code at Google. We wright high-level web apps.

Our requirements are fundamentally different. Go will never be suitable for this task, only for what you're using it for.

In fact, you even admitted you wrote it to replace C++, not Ruby.

So your requirements are very different. You need code to be super fast, I don't. You need it to be super efficient on memory, I don't.

For you, it's better to write a for-loop that looks for the index of a given element and breaks out of the loop when you've found it. But we prefer to have a single function like indexOf() which takes a collection and a function, because it's cleaner, easier to read, easier to write, and easier to maintain. But that's probably too inefficient for you, and it isn't type-safe enough for you either.

This is why it should come as no surprise to me that Go will never be suitable for what I do for a living, which is writing web apps.

-Steven


On Thursday, April 25, 2013 11:54:09 AM UTC-5, Rob Pike wrote:
I'd prefer to have gophers all the way down.

-rob

--

Ian Lance Taylor

unread,
Apr 25, 2013, 1:30:39 PM4/25/13
to sbde...@gmail.com, golan...@googlegroups.com
On Thu, Apr 25, 2013 at 10:26 AM, <sbde...@gmail.com> wrote:
>
> The second is to "box" types into a type that these functions operate on.
> This would be quite ugly but work well. And if you strap a dynamic language
> on top, which translates into the pre-boxed code, it would actually work
> perfectly fine. That was my goal.

...and you can do that today with a bit of work. So we're all good.

If you need to look up a function dynamically by name, build a lookup
table that maps names to functions. You can generate that table
easily enough using by running the go/parser package over whatever
source code you care about.

Go is not a dynamic language but it can certainly support a dynamic
language, just as many dynamic languages are written in C/C++.

Ian

Jesse McNelis

unread,
Apr 25, 2013, 1:34:34 PM4/25/13
to Steven Degutis, Gustavo Niemeyer, golang-nuts
On Fri, Apr 26, 2013 at 2:10 AM, Steven Degutis <sbde...@gmail.com> wrote:
Then this means we can't put Clojure (or anything like it) on top of Go.
 
You can't look up functions by name in C, but somehow Python, Ruby and Javascript could all be put on top of C.
Trying to map the language your interpreting to the language you're implementing the interpreter in is always going to greatly limit your flexibility.

Brad Fitzpatrick

unread,
Apr 25, 2013, 1:35:59 PM4/25/13
to sbde...@gmail.com, golang-nuts, andrey mirtchovski, Gustavo Niemeyer
Plenty of people at Google don't like Go for webapps.
Plenty do.
Plenty of people outside Google don't like Go for webapps. (like you).
A non-zero number do.  (not you)

You like making blanket statements, but they don't help your arguments.

Yes, if Go had a richer type system and generics, much more awesome stuff like map/fold/filter could be written in the language efficiently.  But Go as-is is great for so many things and is the most pleasant language I've used for the sort of code I typically write.  Maybe it's not best for you yet, or ever, but it doesn't have to be all things.



On Thu, Apr 25, 2013 at 10:29 AM, <sbde...@gmail.com> wrote:
If you guys at Google are willing to put up with Go's set of trade-offs, then I'm glad for you. But Google is known for being.. unique. I don't know a single other person outside of this mailing list who is willing to put up with Go's trade-offs to write a web app.

-Steven

roger peppe

unread,
Apr 25, 2013, 1:41:26 PM4/25/13
to sbde...@gmail.com, golang-nuts, Gustavo Niemeyer, Jessta
It has been stated in the past that Go will support some kind
of dynamic code loading at some point. It hasn't happened
yet, but I don't see it as in any way clashing with the way
Go does thing currently, and I hope and trust that it will happen
at some point.

And when/if it does happen, I still don't believe that it would require
dynamic lookup of functions by name - interface values would be
sufficient AFAICS.

BTW you asked about new dynamic features in Go 1.1. Here's
a cool one that hasn't received much press and has some
very interesting potential applications:
http://tip.golang.org/pkg/reflect/#MakeFunc


On 25 April 2013 18:36, <sbde...@gmail.com> wrote:
> Sure you can, using dlsym(). But no such thing exists to look up Go
> functions.
>
> -Steven

minux

unread,
Apr 25, 2013, 1:41:29 PM4/25/13
to Henrik Johansson, sbde...@gmail.com, golang-nuts
On Fri, Apr 26, 2013 at 1:22 AM, Henrik Johansson <dahan...@gmail.com> wrote:

Has these functional aspects been ruled out of future versions of Go? They are not related to dynamically looking up and invoking methods.

those higher order functions could be implemented in Go with the reflect package. They just won't be efficient.

Ian Lance Taylor

unread,
Apr 25, 2013, 1:45:57 PM4/25/13
to sbde...@gmail.com, golan...@googlegroups.com
On Thu, Apr 25, 2013 at 10:34 AM, <sbde...@gmail.com> wrote:
> Sure, if you have access to all the functions at compile-time.
>
> But say I'm writing a language Gruby, and I compile the gruby executable
> with all these functions built-in. Now, someone wants to dynamically load a
> twitter lib or something that's not in Go's stdlib.
>
> They can't.

Of course they can, if you do some work to prepare the ground. There
is nothing in the language or the libraries that prevents this. It
just hasn't been done.

You are assuming that the only way to implement this is to have a
lookup function in the runtime mapping function names to functions,
but that just isn't do. No such functionality exists in C/C++. You
mentioned dlsym; dlsym works for Go too (via cgo), just as well or
just as poorly as it works for C/C++.

Ian

Ian Lance Taylor

unread,
Apr 25, 2013, 1:49:04 PM4/25/13
to sbde...@gmail.com, golan...@googlegroups.com, andrey mirtchovski, Gustavo Niemeyer
On Thu, Apr 25, 2013 at 10:43 AM, <sbde...@gmail.com> wrote:
> And that's just the realization I was explaining. Go isn't for us who prefer
> the trade-offs that Ruby/Clojure make. And from the sounds of it, it never
> will be.

If your realization is that Go is more like C/C++ than it is like
Ruby, then you are entirely correct. Nobody would object to such a
statement.

It's your other statements that sound odd.

Ian

Michael Jones

unread,
Apr 25, 2013, 2:18:53 PM4/25/13
to luz...@gmail.com, golang-nuts, Gustavo Niemeyer, sbde...@gmail.com
...as Gustavo and minux answered, you already can, indirectly. But that was the question you asked, "Will Go add X so that I can Y," rather than the deeper question you might ask, "Does Go need X so that I can Y." I'll take this one more broadly, first in detail and then by analogy.

Typical interpretive languages execute at two speeds--the slow speed of the outer language and the fast speed of the inner language, the language in which the outer language is implemented (often C or C++, spiced with assembler.) From the programmer's perspective, it is much faster if a feature is "in" the outer language as a feature and not just "on" the language using an outer-level implementation, so the goal is to anticipate the developers by including important features in the language. It is also likely prettier in syntax. On the downside, the fact of being built-in also means that the feature works just one way so you lose the option of similar but different solutions that could have been developed for any special cases.

This ecosystem of interlocking concerns is entirely different in a single-level "all fast" compiled language because the developer could--with skill--build the same implementation that would have needed to be built in to the interpreter's language so as to access the faster execution path. The issues in this case (C, C++, Go) are to make sure that the language does not prevent the developer from building important features.

Now, for the analogy: Go lets you compose new, first-class solutions; think atoms, not molecules. Argue for the simplest and most general primitive features in the type system, reflection support, someday dynamic loading, and whatever else might be needed to enable someone with your particular higher-level ambition to realize it successfully in Go. But don't argue for major language concepts if they are for features already efficiently done by combining existing atoms. If Go is good, then few new atoms will ever need to be added, even though billions of useful molecules may be composed with it.

Michael (in Oslo)
Michael T. Jones | Chief Technology Advocate  | m...@google.com |  +1 650-335-5765

Sebastien Binet

unread,
Apr 25, 2013, 2:30:56 PM4/25/13
to sbde...@gmail.com, golang-nuts, Gustavo Niemeyer
On Thu, Apr 25, 2013 at 6:22 PM, <sbde...@gmail.com> wrote:
> To make my point more explicit:
>
> (1) Clojure sits on top of Java
> (2) Clojure enhances Java
> (3) Go is better than Java
> (4) If Clojure sat on top of Go, it would (a) enhance Go, and (b) be better
> than Clojure-on-Java

there is already a lisp-in-go project:
https://github.com/bytbox/kakapo

-s

Andrew Gallant

unread,
Apr 25, 2013, 2:57:54 PM4/25/13
to golan...@googlegroups.com, sbde...@gmail.com, Gustavo Niemeyer, Jessta
> BTW you asked about new dynamic features in Go 1.1. Here's 
> a cool one that hasn't received much press and has some
> very interesting potential applications:
http://tip.golang.org/pkg/reflect/#MakeFunc

Like automatic memoization! :-)


- Andrew

Devon H. O'Dell

unread,
Apr 25, 2013, 3:04:56 PM4/25/13
to Steven Degutis, Kamil Kisiel, golang-nuts, Gustavo Niemeyer
2013/4/25 Steven Degutis <sbde...@gmail.com>:
> Yes really. How can I dynamically call net.Dial() from inside a dynamic
> language implemented in Go? Apparently I can't, and from the sounds of it
> never will be able to.

Write a language interpreter in Go. Write a new parser / lexer on top
of the Go runtime. For the level of project you suggest, these are two
reasonable solutions that are available right now.

> -Steven
>
>
> On Thu, Apr 25, 2013 at 12:06 PM, Kamil Kisiel <kamil....@gmail.com>
> wrote:
>>
>> Really? There are a lot of dynamic languages implemented on top of C/C++
>> and it has similar behaviour.
>>
>>
>> On Thursday, April 25, 2013 9:10:11 AM UTC-7, Steven Degutis wrote:
>>>
>>> Then this means we can't put Clojure (or anything like it) on top of Go.
>>>
>>>
>>> On Thu, Apr 25, 2013 at 11:08 AM, Gustavo Niemeyer <gus...@niemeyer.net>
>>> wrote:
>>>
>>>> On Thu, Apr 25, 2013 at 1:06 PM, Steven Degutis <sbde...@gmail.com>
>>>> wrote:
>>>> > For example, will it have the ability to lookup functions by name at
>>>> > runtime?
>>>>
>>>> This will most likely never come, because it's also an important
>>>> misfeature. If you can access functions by name at runtime, the
>>>> compiler can't trash unused code.
>>>>
>>>> As a side note, in my experience an explicit registry has always been
>>>> a better interface in any case.
>>>>
>>>>
>>>> gustavo @ http://niemeyer.net
>>>
>>>
>

Brad Fitzpatrick

unread,
Apr 25, 2013, 3:24:33 PM4/25/13
to Andrew Gallant, golang-nuts, sbde...@gmail.com, Gustavo Niemeyer, Jessta
Incidentally, I started bugging Russ for MakeFunc as a result of my embedding Perl inside Go: https://github.com/bradfitz/campher/ (speaking of dynamic things)

I never got back to using it, though, once it finally arrived.


Volker Dobler

unread,
Apr 25, 2013, 4:08:01 PM4/25/13
to golan...@googlegroups.com, Steven Degutis

Am Donnerstag, 25. April 2013 21:07:51 UTC+2 schrieb sbde...@gmail.com:
My current plan is just to write a Go-code-emitter. It'll emit all the boxing and unboxing that lets us do all sorts of cool FP things but is tedious to write by hand.

I think I already know how to solve most of the problems this will entail. The only problem left is type conversion.

So if you pass a MyObject type into net.Dial, it'll have to know how to convert itself to a string. And I currently have no idea how to use /pkg/go/* to figure out that net.Dial takes ['string', 'string'] and returns ['Conn', 'error'].

That's the only unsolved part I can think of.


That seems to be a clever idea as it would remove this painful static typing.
These stupid type restrictions really hinder writing code, and we all know
that writing code takes most of our valuable time, not reading or even 
debugging code.

V. 

Kyle Lemons

unread,
Apr 25, 2013, 4:53:53 PM4/25/13
to sbde...@gmail.com, golang-nuts, Kamil Kisiel, Gustavo Niemeyer
On Thu, Apr 25, 2013 at 12:07 PM, <sbde...@gmail.com> wrote:
My current plan is just to write a Go-code-emitter. It'll emit all the boxing and unboxing that lets us do all sorts of cool FP things but is tedious to write by hand.

I think I already know how to solve most of the problems this will entail. The only problem left is type conversion.

So if you pass a MyObject type into net.Dial, it'll have to know how to convert itself to a string. And I currently have no idea how to use /pkg/go/* to figure out that net.Dial takes ['string', 'string'] and returns ['Conn', 'error'].

I believe this is one of the many things that will be enabled by http://godoc.org/code.google.com/p/go.exp/go/types (note, it's still in active development and its API is probably still subject to change before it moves into the stdlib as go/types).
 
That's the only unsolved part I can think of.

-Steven

Kyle Lemons

unread,
Apr 25, 2013, 5:12:29 PM4/25/13
to sbdegutis, golang-nuts, Kamil Kisiel, Gustavo Niemeyer

On Thu, Apr 25, 2013 at 1:56 PM, <sbde...@gmail.com> wrote:
Does that mean it's not possible using what's already in /pkg/go?

It is possible but requires more work.

Rob Pike

unread,
Apr 25, 2013, 5:46:06 PM4/25/13
to Kyle Lemons, sbdegutis, golang-nuts, Kamil Kisiel, Gustavo Niemeyer
It's impossible to eat the sun. Everything else is computable in a
Turing-complete language.

Rob Pike

unread,
Apr 25, 2013, 6:05:12 PM4/25/13
to sbde...@gmail.com, golan...@googlegroups.com, Kyle Lemons, Kamil Kisiel, Gustavo Niemeyer
Programming is a game for pedants. If you disagree or find programming
boring, you're in the wrong business.

-rob

Devon H. O'Dell

unread,
Apr 25, 2013, 7:01:05 PM4/25/13
to Steven Degutis, Kyle Lemons, Kamil Kisiel, Gustavo Niemeyer, golan...@googlegroups.com

What you are asking to do is non trivial in the first place regardless of language. Please don't tell me you are suggesting that implementing clojure on top of the jvm was a trivial task. With that in mind, it is completely reasonable to consider rob's point without seeing feasibility. You are already proposing something outside the realm of "easy" feasibility in any language.

--dho

On Apr 25, 2013 6:17 PM, <sbde...@gmail.com> wrote:
I agree that our profession requires high precision with words, names, semantics, meaning, etc.

But in conversation we should only apply this to the proper degree. We should only offer correction when we discern that there's a misunderstanding. We should be unambiguous only when we perceive that our audience would otherwise lack sufficient context to understand our point.

Otherwise we're just adding noise, not signal.

-Steven
--

Devon H. O'Dell

unread,
Apr 25, 2013, 8:19:10 PM4/25/13
to Steven Degutis, Kyle Lemons, Kamil Kisiel, Gustavo Niemeyer, golan...@googlegroups.com

You said feasible in such a way that it implied it. Everything you are asking for is feasible but non trivial.

On Apr 25, 2013 8:12 PM, <sbde...@gmail.com> wrote:
Who ever said "easy" or "trivial"? :P

Kyle Lemons

unread,
Apr 25, 2013, 8:36:43 PM4/25/13
to sbdegutis, golang-nuts, Kamil Kisiel, Gustavo Niemeyer
Is there something wrong with the go.exp/go/types package?  While the API might change, you can probably still use it and the changes are likely to be straightforward, and you can make note of the repo version in case breaking changes are made while you have a chance to get the updates in.


On Thu, Apr 25, 2013 at 2:59 PM, <sbde...@gmail.com> wrote:
Yes Rob, it may be possible, but not necessarily practical. As it's a given that anything is possible in Turing-complete languages, the reader should mentally replace "is it possible" with "is it feasible without a disproportionate amount of work". This way we can all continue to use language of convenience without needing to be tiringly explicit all the time. I've found that in general this works quite well, except when the pedant or the bored are present.

-Steven

Kevin Gillette

unread,
Apr 25, 2013, 10:23:01 PM4/25/13
to golan...@googlegroups.com, Gustavo Niemeyer, sbde...@gmail.com
On Thursday, April 25, 2013 10:10:11 AM UTC-6, Steven Degutis wrote:
Then this means we can't put Clojure (or anything like it) on top of Go.

On Thu, Apr 25, 2013 at 11:08 AM, Gustavo Niemeyer <gus...@niemeyer.net> wrote:
On Thu, Apr 25, 2013 at 1:06 PM, Steven Degutis <sbde...@gmail.com> wrote:
> For example, will it have the ability to lookup functions by name at
> runtime?

To start with, it's fallacious to suggest that Clojure (or anything else) couldn't exist on top of Go. Go is Turing Complete, and so literally anything that is computable (including Clojure programs) can successfully exist atop a Go foundation. Perhaps what you meant to say was "then this means that Clojure can't subjugate my Go runtime."

Consider the implications of what you're suggesting... Can functions, types, and variables be found across package boundaries?  Can a list of function/type/variable names be retrieved at runtime? What reasonably simple mechanism can be used to control what gets exposed for runtime lookup and what does not? If what you propose answers yes to either of the first two questions, then you've just proposed nothing more useful than a security hole (for example, malicious Clojure code could 'discover' os.Open or the unsafe package, and use either of those to wreak havoc).

Also keep in mind that, for security purposes, a blacklist model will always be worse than a whitelist model, and the best way to model a whitelist is to keep a map of names to function values (that even gives you the added flexibility of being able to rename functions, e.g., to fit more in line with the guest language's semantics and style). Any kind of black-list system (which is what you seem to be favoring) would require far more code in any kind of sane program, since nearly all functions should be blacklisted (particularly anything that could result in a syscall or allocation), unless the guest code is known to need something along those lines (typical candidates would be stdin/stdout, and time handling).

Even if this functionality did exist, the builtin functions would have to be exempt, since they have generic implementations yet Go doesn't have first-class generics (essentially, the builtins can only be counted on as a compile-time phenomenon).

Regardless of any positive or negative aspects of the Lua language, the C-Lua interface is the best interpreter API I've seen. Yes, C has no reflection, so explicit registration would be the only way to provide interpreted code with native functions. That said, certainly someone could have made a SWIG-like promiscuous code generator that finds all of a C program's application and library functions and exports those into the interpreter, but for what should be obvious reasons, that would have been an obviously bad idea. The Lua interface is strictly opt-in, and even requires the host program to opt into providing Lua's own stdlib. In contrast, the Python C-API pretty much assumes you want to give your interpreter the keys to the universe, and Python has far less embedded market-share than it could have had because of that naive decision -- I have seen many C/Python programmers choose to use Lua for embedding (despite disliking the Lua language) strictly because of this reason.

Surely you want to restrict access to the hosted Clojure environment to only those tools it actually needs to do whatever application-specific embedded processing tasks you envision (for example, why would Clojure possibly need to be able to open a network connection if the application task it's to perform involves numerical calculations?). Not only should you not want the hosted language to have access to everything in Go, but a well designed application should attempt to limit the functionality that is provided by standard Clojure. If you really want to allow it to do anything, then it's not really a matter of embedding Clojure -- you're instead talking about a standalone Clojure interpreter (which already exists).

Daniel Bryan

unread,
Apr 25, 2013, 11:06:54 PM4/25/13
to golan...@googlegroups.com
Setting aside all this talk of dynamic function lookup, does Go even have the necessary primitives to implement Closure's immutable data structures, short of hiding every value behind an interface and forbidding type assertion on interface values?

Go is a good platform for implementing a language interpreter or compiler and runtime, but that doesn't mean all languages are a good fit. The strengths of the JVM that make Closure a strong language are very different to those of the Go runtime and Go's static environment.

Statically linked Go binaries are a little island of sanity in a world of DLL hell. You want that to go away because of the myth that productive web app development == dynamic runtimes?

Steven Degutis

unread,
Apr 25, 2013, 11:11:31 PM4/25/13
to Daniel Bryan, golan...@googlegroups.com
Nah.

My new plan is to write a "compiler" that just translates Clojure-ish code to Go code and compiles it statically.

Admittedly it loses a lot of Go's type-safety and such, since it will basically emit lots of translating to/from interface{} or whatever.

But that's a trade-off I'm willing to take, especially if it means I get to use these functions while still having Go's stdlib and fast start-up time.

-Steven


--
You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/BmFX-PF9bSA/unsubscribe?hl=en-US.
To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.

Nathan Youngman

unread,
Apr 25, 2013, 11:13:44 PM4/25/13
to golan...@googlegroups.com, sbde...@gmail.com, andrey mirtchovski, Gustavo Niemeyer
As a Rubyist learning Go, I've been quite amused by this conversation, especially "gophers all the way down". ^_^

While porting a composition example from Ruby, I found the need to filter a slice. I was slightly disappointed that there is no generic builtin for select/filter, even though though there is a generic built-in for append. But for is generic, range avoids off-by-one errors, and there is really nothing wrong with for, even if it's more verbose. I realize I could write my own filter -- Go has first-class functions. I could use reflection and/or specialize it by hand, but would it be worth it? IMO, not really.

What I found particularly interesting, is Alex Gaynor's talk on "Why Python, Ruby and JavaScript are Slow." The synopsis is that the VMs are getting better all the time, and can actually be quite good. But dynamic languages encourage a style of programming that is less efficient, a style that is less concerned with memory and performance. https://vimeo.com/61044810

An example relevant to this discussion, in Ruby it is typical to see a chain of functional filters, maps, etc. Prior to lazy evaluation being introduced in Ruby 2.0, this would result in looping over the dataset multiple times. If it was using a series of for loops as in Go, it would be pretty obvious that there is room for optimization if needed.

Aaron Patterson's talk, the section on "Cleaning up a MetaMess" https://vimeo.com/61059531 uses polymorphism in place of eval (granted, this example uses define_method as well). Charles Nutter's High Performance Ruby talk https://vimeo.com/61255646 is all about doing things more statically/simply, making it easier for the VM optimize. So there is quite a lot of focus in dynamic languages on getting better performance, and it usually means imposing some constraints.

Personally, I find Go strikes a great balance between dynamic and static languages. Something I intend to talk about more at the inaugural Google Developers Group here in Edmonton.

Certainly, it'll be interesting if/when Go gets generics, but a little research would reveal that such a change has been deemed to warrant a 2.0 release, and the Go 1.1 release notes aren't hard to find: http://tip.golang.org/doc/go1.1. Meanwhile, people are doing some pretty interesting things with Go as it stands:

Nathan.

Steven Degutis

unread,
Apr 25, 2013, 11:19:14 PM4/25/13
to Nathan Youngman, golan...@googlegroups.com, andrey mirtchovski, Gustavo Niemeyer
Using higher-order functions like filter/reduce/map/etc is shorter, clearer, easier to understand, easier to maintain, easier to read, easier to write, and safer than the equivalent code using a primitive for-loop.

It's not a trivial difference, these higher-order functions have become a must-have for me when writing almost anything.

If I ever need to write game software or if I ever work for Google and need to write a search engine, maybe I'll have more need for the performance increase gained by using a for-loop, and maybe then the trade-off of losing all these benefits might be worth it. Maybe.

Nathan Youngman

unread,
Apr 25, 2013, 11:32:31 PM4/25/13
to golan...@googlegroups.com
I'm no guru on data structures, much less persistent bitmap tries. So I could be wrong about this.

The tries used extensively in Clojure were implemented in Java, and are available in one form or another in Scala, Python and Ruby, as well as Objective-C. So I'm sure it is possible, just as it would be possible to implement a VM and a JIT in Go.

As to what the leaf nodes contain, I imagine generics would be helpful, but the link I mentioned from Nate Finch could do the trick: http://blog.natefinch.com/2013/04/statically-typed-generic-data.html

Nathan.

Michael Jones

unread,
Apr 25, 2013, 11:40:33 PM4/25/13
to Steven Degutis, Nathan Youngman, golang-nuts, andrey mirtchovski, Gustavo Niemeyer
It seems like we're talking past each other. 

If you want to write a program in a style where every step is a generalized abstraction (other examples are Mathematica's language, which includes a type-spanning and value driven generalized pattern matching stage, Smalltalk, FP, J, etc.) then there must be an interpreter of sorts at the outer edge of evaluation. Call this the slower bit awesomely general world. 

If you want high performance in the sense of efficient compared to peak CPU performance for the stated task, then you must use the programmer and the compiler to recognize as much structure and regularity in the computation as can be understood; this is precisely where efficiency comes from.

You can build a system for the first case that is quite good, and several exist. You can do the latter, and most of computing is there. But to see some desired features in the former and mentally "require" the delayed abstraction in the latter seems a mistake. I'm not defending Go, I'm just puzzled why this feels "I want it all in one place" feels so essential when it will (seemingly) be a much less flexible and less performant solution than a hybrid that uses two programming universes with shared data.

Nathan Youngman

unread,
Apr 25, 2013, 11:46:55 PM4/25/13
to golan...@googlegroups.com, Nathan Youngman, sbde...@gmail.com

Hi Steven,

* There are some Go libraries that you might find helpful as a starting point, eg. https://github.com/keep94/gofunctional2
* If you don't mind C code, maybe you could work towards adding more bulitins for Go 1.2? It is open source.
* If you are particularly ambitious, you could dig through any past proposals and try to come up with a proposal for generic that Ken, Robert, Rob and the rest of the community likes. It would need to take into account that methods can be defined on any user-defined type, including ints, slices, function types, etc.
* Maybe a different approach would be to help improve nailgun or some other efforts to make JVM startup time more palatable when using Clojure?

Not to dissuade you from writing a source-to-source compiler, which sounds like an interesting project as well.

All the best in your endeavours,
Nathan.

Dan Kortschak

unread,
Apr 25, 2013, 11:56:14 PM4/25/13
to sbde...@gmail.com, golan...@googlegroups.com, Nathan Youngman
On Thu, 2013-04-25 at 20:52 -0700, sbde...@gmail.com wrote:
> But who can argue with someone who doesn't see the benefit of syntax
> highlighting? Such a person lives in a different world than everyone
> else I know and cannot be reasoned with.
>
While that's probably intended facetiously, it's not all that helpful.

Everybody in the world thinks differently to how I do too, yet I manage.


Gustavo Niemeyer

unread,
Apr 26, 2013, 12:24:50 AM4/26/13
to sbde...@gmail.com, golan...@googlegroups.com, Nathan Youngman
On Fri, Apr 26, 2013 at 12:58 AM, <sbde...@gmail.com> wrote:
> It was intended as an explanation as for why I will not bother with
> submitting proposals to the Go team, because they seem to view the world
> much differently than me and everybody I've worked with.

The only person I found that could see the world entirely unlike me,
I've made to sure to have as a wife.


gustavo @ http://niemeyer.net

Rob Pike

unread,
Apr 26, 2013, 12:25:15 AM4/26/13
to sbde...@gmail.com, golan...@googlegroups.com, Nathan Youngman
I see the benefits of syntax highlighting, just as I see the benefits
of diapers.

-rob

andrey mirtchovski

unread,
Apr 26, 2013, 12:33:47 AM4/26/13
to sbde...@gmail.com, golang-nuts, Nathan Youngman
> Yep, proving my point. Not only are we living in two different worlds, but
> your condescension is pointy.

With you being so busy putting Clojure on top of absolutely
everything[1] shouldn't you be wasting less of your valuable trolling
time with us old farts here? We don't even like syntax highlighting,
for crying out loud!

--
1: http://osdir.com/ml/clojure/2013-04/msg00660.html

Nick Owens

unread,
Apr 26, 2013, 2:10:08 AM4/26/13
to golan...@googlegroups.com, sbde...@gmail.com
Rob Pike doesn't need color, just contrast. https://www.youtube.com/watch?v=qwIAjB99ucw

UNIX compilers are slow!

Andrew Gerrand

unread,
Apr 26, 2013, 3:23:25 AM4/26/13
to sbde...@gmail.com, golang-nuts
Steven:

If you want Clojure, please use Clojure. Go will never be Clojure and, critically, Clojure will always be a better Clojure than Go. (The important corollary here for those of us who love Go, is that Go will always be a better Go than Clojure.)

To better understand Go's design goals, please read: http://talks.golang.org/2012/splash.article

Can we please end this thread? It is going nowhere. Please move on.

Andrew


Steven E. Harris

unread,
Apr 26, 2013, 4:21:49 PM4/26/13
to golan...@googlegroups.com
Daniel Bryan <danb...@gmail.com> writes:

> Setting aside all this talk of dynamic function lookup, does Go even
> have the necessary primitives to implement Closure's immutable data
> structures, short of hiding every value behind an interface and
> forbidding type assertion on interface values?

Clojure's persistent data structures are implemented in Java, and don't
even use generics. They store values of type Object, which is the same
thing Go would do with interfaces. See, for instance, class
PersistentVector's nth() method¹. It retrieves an Object reference from
an array of Objects.

The primitive arrays like `int-array' do use primitive Java types.²


Footnotes:
¹ https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/PersistentVector.java#L110
² https://github.com/clojure/clojure/blob/c6756a8bab137128c8119add29a25b0a88509900/src/clj/clojure/core.clj#L4776
https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/Numbers.java#L1073

--
Steven E. Harris

Michael Harooni

unread,
Apr 26, 2013, 4:27:09 PM4/26/13
to sbde...@gmail.com, golan...@googlegroups.com
On Thu, Apr 25, 2013 at 9:22 AM, <sbde...@gmail.com> wrote:
(3) Go is better than Java


That's one heck of a blanket statement.
 

Michael Harooni

unread,
Apr 26, 2013, 4:50:05 PM4/26/13
to sbde...@gmail.com, golan...@googlegroups.com
On Thu, Apr 25, 2013 at 10:17 AM, <sbde...@gmail.com> wrote:
In giving high-level functions like map(), reduce(), indexOf(), etc, we would be losing type-safety and some efficiency/speed.


Not necessarily. Type safety and performance can still be preserved with the functional approach. 

Gustavo Niemeyer

unread,
Apr 26, 2013, 5:06:51 PM4/26/13
to Michael Harooni, sbde...@gmail.com, golan...@googlegroups.com
On Fri, Apr 26, 2013 at 5:50 PM, Michael Harooni <mh4...@gmail.com> wrote:
> Not necessarily. Type safety and performance can still be preserved with the
> functional approach.

I suggest reading the whole thread before replying, including Andrew's
message that says let's please kill this thread.


gustavo @ http://niemeyer.net

SteveD

unread,
Apr 29, 2013, 2:37:18 AM4/29/13
to golan...@googlegroups.com, sbde...@gmail.com
On Thursday, April 25, 2013 7:26:06 PM UTC+2, sbde...@gmail.com wrote:
The second is to "box" types into a type that these functions operate on. This would be quite ugly but work well. And if you strap a dynamic language on top, which translates into the pre-boxed code, it would actually work perfectly fine. That was my goal.

Yep, and that's exactly what Gustavo means by registering.  There is a fundamental difference between Go and Java; everything in the JVM is a class, and is mostly open to access with reflection.  So it's possible to put a dynamic language on top of the JVM with reflection in a straightforward way.  In Go, libraries are linked in statically, and it's not possible to use reflection to find a package - if it were so, then the linker would lose the freedom to throw away unused code.  If Go was more like Java, it would eat more memory and suffer from the same weaknesses as Java.

However, Go reflection works fine on dynamically created objects.  The luar package defines reflection-based Lua bindings on top of luago, and you will see that after the needed functions are registered, the rest works just as you would expect. E.g this is a Go program using Lua as a configuration language:

https://github.com/stevedonovan/luar/blob/master/examples/struct.go

In that directory, have a look at hello.lua, and the Go examples that load it.

steve d.

Reply all
Reply to author
Forward
0 new messages