All Forms of Wishful Generics

[dc0d]

All forms of generics that I would love to have in Go:

[prade...@gmail.com]

Have a look at Rust, Ada or even C++, they all have some form of generic programming and are fast or faster than Go. Bonus, none of these use garbage collection.

Don't hold your breath with Go getting any substantial changes in its type system. Move on. 

Developers should use tools that suit them, not tools that force developers to adapt them. 

[dc0d]

“There are only two kinds of languages: the ones people complain about and the ones nobody uses.”

― Bjarne Stroustrup,

I use other programming languages too - obviously. And I will continue to think of better ways to perform Go, if not complaining.

Meanwhile this is a tool to write generic code in Go, employing code generation (that I wrote) - not waiting for day dreams to come true and instead get done with the job.

This was just some thought sharing.

Especially the last two form - the generic Method Expression and the interface one - if they be a single-type-parameter type of generics, are interesting to investigate.

Like if we could write:

type method func (*) append(n int, m anotherConcreteType, prev *)



type comparer interface {
 (*) compare(*)
}

None of these mean that Go has to change - while it might, a bit - but brain teasers.

[matthe...@gmail.com]

Can you write some examples that use these types?

Matt

[prade...@gmail.com]

Go is hot right now just like Rails or Node.js used to be hot. So people feel compelled to use the hip thing right now even when they hate it because of fear of missing out. There are better languages out there like Ada who do everything Go does, including concurrency but better and have generics. Trying to fix a feature with code generation is a fool's errand. Competition is good, only when the Go team feels the heat of competition they will think about working on their type system seriously. 

You talked about C++ and that's exactly what happened to C++ and why it got all these features in the latest 15 years like auto and constant expressions, it is trying to stay relevant against Java, C# and co. C is what happens when there is no competition.

[Dave Cheney]

I certainly don’t want what happened to C++ to happen to Go. If it’s a choice between obsolescence or being crushed under the weight of self inflicted complexity, I’ll gladly vote with my feet for the former.

[Mandolyte]

I think this falls into a variation of package level generics.

Look over the document below and see if there is any changes you would recommend.

https://docs.google.com/document/d/1vrAy9gMpMoS3uaVphB32uVXX4pi-HnNjkMEgyAHX4N4/edit?usp=drivesdk

Thanks for sharing!

[dc0d]

Agreed.

But it's not necessarily a "obsolescence" vs "being crushed (...)" thing. With the brainpower behind Go, sure the best things will happen.

I'm not saying that Go has to add generics (or not). In fact generics is just one of the things that I would like to see in Go.

There are other things too, that I would like to have in Go; like a faster FFI/CGO, or safe packages by restricting features so that a Go package cannot harm the hosting machine or application, like Safe Tcl. A list of those things is available here.

[dc0d]

Indeed it's a form of package/block level code-specialization, so called generics.

There is a proposal for package level generics by me too.

My main concern here was to have this feature without invalidating current Go code bases, by allowing the rebinding of Type Aliases.

For block level generics - as suggested at the first post - there might be some substantial changes needed to be made to the language - except for the last form: Interface Receivers with Original Types, which allows to not touch current Go code bases.

All in all Go is a fantastic tool with it's own merits.

[matthe...@gmail.com]

Competition is good, only when the Go team feels the heat of competition they will think about working on their type system seriously.

I think everybody here is aware that some people really want generics and that generics would probably be useful for everybody. Instead of restating that why not share what you think about this version of Go generics?

If you have other ideas for generics please read the discussion thoroughly and add your thoughts: https://github.com/golang/go/issues/15292

Go is hot right now just like Rails or Node.js used to be hot.

I think everybody here likes Go and would like to keep it hot. Obviously adding a bad feature would hurt not just popular perception but also people who write a lot of Go.

Anyway, the two cases I’ve been looking at are a generic unordered set type and a collection pick function without adding a new built-in. Here’s those with that first data case.

type set []T

func (s *set) append(t T) { s = append(set, t) }

On the surface this looks straightforward here for the compiler since append is already generic, but how would you make a set?

a := set<string>{“hello,”, “world!”}

a := set{“hello,”, “world!”}

// the compiler says mismatched types?
a := set{“hello”, 1, map[string]string{}}

// can the compiler check for this kind of thing in all cases?
a := set{}
a = a.append(“hello”)
a = a.append(1)

// pseudorandomly pick n elements from collection T
pick(t T, n uint) []U

How to constrain T to collection types only?

Thanks,

Matt

[Lars Seipel]

On Sat, Feb 17, 2018 at 01:10:29AM -0800, dc0d wrote:
> There are other things too, that I would like to have in Go; like a faster
> FFI/CGO, or safe packages by restricting features so that a Go package
> cannot harm the hosting machine or application, like Safe Tcl

Go already has a NaCl backend which might fit the bill. See
misc/nacl/README for how to set it up. It links to a design document
(https://golang.org/s/go13nacl) with some background.

[dc0d]

Lars,

That's nice! Yet it's a sandboxed execution context. The Safe Packages mentioned above are just Go packages, with source code.

And when the day comes for a proper dependency manager, I would like to be able to tell the DM fail on import any unsafe packages - except for a list that I trust.

[RickyS]

Back when I first learned about the diamond problem with multiple inheritance, I've known we need someone to invent the next and better thing after inheritance. I do hope somebody smarter than me is somewhere trying. Or even has succeeded.

And back when I first learned about the code bloat that could result from C++ generics/templates, I've known we need somebody invent the next and better thing after "Generators". Generators, by the way, first appeared in the 1950's, when they were considered a failure.

It's been decades, and I'm still waiting. I would like to believe the Go Authors are waiting for better solutions. Or even inventing them.

N.B. All praise to the Go Authors for upgrading past the massive inefficiency of #include files. Also for many other things.

Well built software is easily modified. Easily-modified software is modified and modified until it is incomprehensible and no longer easily modified. Progress gets slower and slower and slower. Until it stops entirely. Having the maturity and character to refrain from second-best modifications is rare and wonderful. The ideas behind generics and inheritance are basically: "Do this thing just like that other thing, except differently here and here". Re-use code and concepts. I can feel the passion for this goal. I have felt the sand in my gears as I drive the new machine with 256 levers and switches.


On Friday, February 16, 2018 at 8:25:35 AM UTC+2, dc0d wrote:

All forms of generics that I would love to have in Go: ...

 

[Doğan Kurt]

Generics would divide the Go community.

I would probably never use it, like many people who comes to Go from C. People who are already experienced Go programmers also likely to avoid it. Java programmers on the other hand will surely overuse it.

There it is, you have two different Go code bases that looks nothing like each others. One uses Generics and the other does not. Nobody wants Go to have the same fate as C++. We love how there is only one way to accomplish something in Go.

There are billions of lines of code written in Go. Nobody would happily transform all those working codes just to use new idiomatic Go with generics. I suspect even Go team would hesitate to transform all the standard library.

If Go team add generics to Go 2, i am afraid that Go 2 will have the same fate as python 3.

Let's hope it never happens.

[dc0d]

Dogan,

Why does it have to be a breaking change? And there are other things too that are equally - if not more - important to me (the provided link to the list).

RickyS,

The success of Go in the field of application development (besides it's success in the devops and infrastructure sector) was unexpected - as Rob Pike himself stated, they expected to attract C and C++ developers and instead they've got developers from Python and .NET and other PLs that are mostly used for application development.

Having that in mind, Go must adopt. It does not have to be an adoption in terms of syntax. It might at some point. But - in my experience - the far more important factors that can affect this and push the adoption further, are 1) the expansion of tools and workflow and 2) the cultural roadmaps, which if handled in a pragmatic and realistic way, can bear fruit beyond initial any anticipation made by now.

There is something amiss in the current situation with Go. While the Go team are doing their best (they are the best after all), other things can get improved in far more effective ways by a coherent community which cares about quality and staying real to real world problems most.

[dc0d]

This is a great talk on designing PLs, by Brian Kernighan in which he describes (too) how Perl stops being relevant by providing too little, too late.

Again, improvements can be made in term of things other than just the syntax or semantics - while it is a possibility.

[RickyS]

1. Oh, I'm all for improvement. I just want somebody to invent something in the field of re-use that is as much a breakthrough as Cooperating Sequential Processes was to the field of multi-processing. Go would be a good place to install that invention, as it is still a simple language.
   
   
2. And you mean go must adapt, perhaps by adopting new features...

On Monday, February 19, 2018 at 1:17:22 PM UTC+2, dc0d wrote:

 ...Go must adopt.  ...

[dc0d]

Yeah! And since I've learnt English mostly from SciFi and Fantasy movie heroes (with a misplaced sense of self-righteousness), I make boo boos from time to time - which are not that harmful and entertain people! :)

There are other things that I might be interested to be improved about Go (MO-Oriented improvements which MO-Oriented tweaks about the syntax are just a small part of it). But as for this thread seems the discussion made it's way to a level that needs some more time to manifest itself as conclusions - if it meant to be any at all.

And I'm positive Go will continue to grow and get better at things.

[Jesper Louis Andersen]

On Sun, Feb 18, 2018 at 4:47 AM Lars Seipel <lars....@gmail.com> wrote:

Go already has a NaCl backend which might fit the bill. See
misc/nacl/README for how to set it up. It links to a design document
(https://golang.org/s/go13nacl) with some background.

The sucessor of that project is WebAssembly.

WebAssembly is a really good design, where the virtual machine is formally specified, with a testbed implementation in Ocaml. One of the defining features is that the VM is a stack-vm with a twist:

* Stack-based VMs usually have smaller opcodes (since you don't have to specify registers to operate on)

* Stack-based VMs are usually easier to target

* WebAsm's VM is built in a way such that the layout of the stack is statically known at compile time.

The latter means that you can compile it to a register-based VM, which tend to be faster at execution.

Ok, C programs can only be implemented in it if you provide a shadow stack. But given that C generally allows too much of the programmer, this is perfect :) Also note, I don't think Go would need a shadow stack, but there might be a subtlety somewhere which mandates it.

[Chris Hopkins]

I think everybody here is aware that some people really want generics and that generics would probably be useful for everybody. Instead of restating that why not share what you think about this version of Go generics?

IMO this is not a proven statement.

One of the things that for me I like about Go is that one can read code and reason about its behaviour to a reasonable extent.  In my very limited experience generics would make it considerably more difficult to reason about exactly what a piece of code will do.

While generics would allow some problems to be expressed in a simpler fashion, I'm not convinced that that would produce a more comprehensible code-base. I really worry they would just facilitate lazy programming where people save a few keystrokes at the expense of: "You too need a brain the size of a planet, like mine, to understand this."

Maybe this is unfounded, but I'm far from convinced that generics would make my experience of Go better. I'm really thinking here of, are we really sure they would make large unfamiliar codebases more comprehensible? Every feature of a language can be abused, so are they worth the abuse that they would allow?

Chris

[Ignazio Di Napoli]

The only time I "needed" generics was for an r-tree container I wrote. 

I'd been much happier if I could statically catch my errors about contained data type, instead of catching them at runtime with panics during conversion. 

For example:

    x := 0.0

    y := 0.0

    data := 1

 

    container := NewRTree()

    container.Add(x, y, data)

    found := container.FindNearest(x, y)   // found type is interface{}

    data2 := found.(float32)   // THIS PANICS AT RUNTIME, data2 is int

Maybe something like this would be better:

    container := NewRTree<float32>()

    container.Add(x, y, data)   // compile error: data is int, should be float32

If there is another solution instead generics to solve this problem, I'd be equally happy.

Ignazio

[Ignazio Di Napoli]

On Monday, February 19, 2018 at 4:04:12 PM UTC+1, Ignazio Di Napoli wrote:

    data2 := found.(float32)   // THIS PANICS AT RUNTIME, data2 is int

Sorry, found is int. 

[Henrik Johansson]

I disagree that generics would decrease readability at the call site. Perhaps within the library where it is used but maybe not even there. The only complexity is within the compiler and other internals. This is not irrelevant by far but the carte blanche "generics is bad" is most often hyperbole.

I can also "do without" generics and in fact even though Go lacks it I prefer Go to all other languages that I know. I still would appreciate a generics version in Go's spirit.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[matthe...@gmail.com]

I would probably never use it, like many people who comes to Go from C.

But if you use maps, slices, append, make, you are already using generics.

Maybe this is unfounded, but I'm far from convinced that generics would make my experience of Go better. I'm really thinking here of, are we really sure they would make large unfamiliar codebases more comprehensible? Every feature of a language can be abused, so are they worth the abuse that they would allow?

Sometimes having another built-in function would be helpful, and generics would provide a way to write those without changing the compiler. What I’ve seen is that built-in functions and new types expanding on the map/slice pattern will not make it into Go 2 without a major reason.

The other call for generics is interface{}. The Go 1 uses of interface{} can make strange code that maybe would be more comprehensible another way.

Matt

[Michael Jones]

Matthew Juran wrote: "...but if you use maps, slices, append, make, you are already using generics."

This seems deeply insightful to me. Perhaps a better question than the self-defeatingly open question of "how should Go embrace generics?" would be "what change would allow maps, slices, append, and make to be implemented as user code in Go 2?"

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Michael T. Jones
michae...@gmail.com

[roger peppe]

On 19 February 2018 at 17:36, Michael Jones <michae...@gmail.com> wrote:
> Matthew Juran wrote: "...but if you use maps, slices, append, make, you are
> already using generics."
>
> This seems deeply insightful to me. Perhaps a better question than the
> self-defeatingly open question of "how should Go embrace generics?" would be
> "what change would allow maps, slices, append, and make to be implemented as
> user code in Go 2?"

It's not just maps, slices, append, and make. Every single operator
could be considered too
to be "generic" because each operates on an indefinite number of types.

But Go gains much from the special-case syntax and semantics associated with
each of these language features, IMHO. I'm sure one could make a language where
maps, slices, pointers, etc were all just generic types and operations. I'm sure
there are many such languages already. Such a language would
make for a very different experience than Go provides, though.

If maps are just user code in Go 2, what about map literals? If slices are just
user code, do we have to be wary that everything that looks like an
index operation
might have arbitrary side-effects? If channels are just user code, can
we still have a select statement?

The difficulty of generics is not to provide some number of generic container
types, but how to provide a *general* way of specifying operations
generically over
types.

[By the way, I think any discussion on generics in Go is hampered by the
fact that the term itself is not well defined. Everyone has their own idea
what "generics" might mean, and it's easy to talk at cross purposes.]

[Rob Pike]

Jesper,

I find myself in rare but mild disagreement about your claims for stack-based virtual machines. Please have a look at this short paper about the Dis VM from Inferno: http://flint.cs.yale.edu/jvmsem/doc/inferno.ps

We found a JIT for Dis could be tiny, especially compared to Java JITs, and at the time outperformed Java.

-rob

[Jesper Louis Andersen]

On Mon, Feb 19, 2018 at 9:29 PM Rob Pike <r...@golang.org> wrote:

Jesper,

I find myself in rare but mild disagreement about your claims for stack-based virtual machines. Please have a look at this short paper about the Dis VM from Inferno: http://flint.cs.yale.edu/jvmsem/doc/inferno.ps

There is a chance I'm just wrong here.

The webassembly paper[0] mentions the size of the program as being important. They compare this size to older models such as NaCl and asm.js, and they note the stack based instruction set makes for a smaller "binary size". However, I have a feeling that a well-engineered VM, register-based or not, is likely to be able to compete in size of the produced executable.

The obvious reason for wanting that size down is bandwidth usage on the internet. While caching can play a role, transfer sizes does matter in the end, especially given mobile.

My experience, though YMMV, is that when a VM is based on registers and matches a real-world CPU more closely, it is easier to write a compiler/JIT from the bytecode to the machine instructions. On the other hand, if the VM is based on a stack, it is easier to write a compiler from a high-level language to the bytecode. In short, you pick a "split point" of abstraction and this makes one or the other easier to implement.

Efficiency was not something I touched upon in the post, but I have a hunch that the experiences of Dis carries over: a VM close to the machine is easier to get to run fast, with a fraction of the investment.

The brilliance of WebAssembly is that it is really both models in one. The "transfer format" over the wire is a stack-based instruction set. But the instructions are built in such a way that we can derive a register machine from it at any point in the program (statically!). So you obtain a relatively easy compile target, and small bytecode "binaries". But once you want to execute the program, you first convert the machine to something else, probably resembling Dis, and then you proceed to handle that new machine.

[0] https://people.mpi-sws.org/~rossberg/papers/Haas,%20Rossberg,%20Schuff,%20Titzer,%20Gohman,%20Wagner,%20Zakai,%20Bastien,%20Holman%20-%20Bringing%20the%20Web%20up%20to%20Speed%20with%20WebAssembly.pdf

[Mandolyte]

+1

Michael T. Jones
michae...@gmail.com

[Egon]

On Monday, 19 February 2018 12:06:09 UTC+2, RickyS wrote:

Back when I first learned about the diamond problem with multiple inheritance, I've known we need someone to invent the next and better thing after inheritance. I do hope somebody smarter than me is somewhere trying. Or even has succeeded.

There are many problems that are difficult in abstract, but easier in practice. As such I think that best solution to the diamond problem is not to use inheritance. The question becomes: what to do instead and which problems does the "alternative solutions" have. There are of course many known alternative solutions already: traits, Entity-Component-System, BOT architecture or DCI.

The problem is not "inheritance diamond", it's usually something else... for example, how to clearly and succinctly model complex entities in a game. When we just try to figure out the most flexible approach we may miss the mark by providing sufficient flexibility, but not the clarity in the problems.

And back when I first learned about the code bloat that could result from C++ generics/templates, I've known we need somebody invent the next and better thing after "Generators". Generators, by the way, first appeared in the 1950's, when they were considered a failure.

It's been decades, and I'm still waiting. I would like to believe the Go Authors are waiting for better solutions. Or even inventing them.

Swift for example has a solution that avoids code-bloat to some degree (https://www.youtube.com/watch?v=ctS8FzqcRug). Of course, this comes at the cost of performance.

In that sense, there is progress, although it is slow.

[matthe...@gmail.com]

This seems deeply insightful to me. 

Thanks Michael.

It's not just maps, slices, append, and make.  Every single operator could be considered too to be "generic" because each operates on an indefinite number of types. 
But Go gains much from the special-case syntax and semantics associated with each of these language features, IMHO. I'm sure one could make a language where maps, slices, pointers, etc were all just generic types and operations. I'm sure there are many such languages already. 

Focusing on the collection side of generics, other languages allow specifying a method for each operator but in Go you can’t use ==>><<+-<>!^/*!= operators on the collection types. I do agree that the collection specifics are worthwhile, and they are surprisingly orthogonal:

Array:

    - cannot be nil

Slice:

    - reslice

    - append

    - cap

    - make(T, len, optional cap)

Map:

    - “has” check

    - delete

    - map indexing

    - key, value range

    - make(T, optional cap)

Array/Slice:

    - index get

    - index set

    - index, value range

Slice/Map

    - can be nil

Array/Slice/Map:

    - len

    - new

Swift for example has a solution that avoids code-bloat to some degree (https://www.youtube.com/watch?v=ctS8FzqcRug).

I haven't watched the whole talk yet but a consideration for Go is that inheritance isn't used for most standard library types. Perhaps generic collections would implement the range interface that has a specification shortcut for use in for range, but there's no inheritable collection type.

Matt

[David Chase]

I agree that this is one of the two large risks.

The other is whether the language remains comparatively easy to learn.

The next largest problem after those two is "what exactly do you mean by generics"?

Some people want code stenciling, like C++ (it's fast, it's easy to understand, code size might explode, generic methods are harder).

Some people want dictionaries (you get type checking and generic methods, but not too likely you get speed from specialization).

Some people think erased Java-style generics are fine (these do have issues with reflection, and it might look different w/o type tags).

Note that Java-style generics were supposed to be the best choice for old/new compatibility.

As a technical problem it is *relatively* easy.