gotit generic templates

86 views
Skip to first unread message

David Roundy

unread,
Feb 26, 2010, 11:13:48 AM2/26/10
to golang-nuts
Hi all,

I've implemented a much nicer version of my gotit templating
implementation of generics for go, which you can grab at:

http://github.com/droundy/gotit

The syntax is much improved (I now use go.scanner), so a template
looks something like:

package slice(type a)

func Append(s *[]a, val a) {
l = len(*a)
if l == cap(*a) { ... }
*a = (*a)[0:l+1]
(*a)[l] = val
}

You can see that the package declaration just has a list of type
parameters added. There's a bit more complexity, in that you need to
specify a "default type" if your code relies on some particular sort
of type (e.g. an interface). So for instance, if you write a package
like

package mymath(type a int)

func max(x, y a) { if x > y { return x }; return y }

the package will only be able to be instantiated with types that are
assignment-compatible with int. If you left off the int default type,
then the template would fail to compile, because > is not defined for
the interface{} type.

Which brings us to how to actually instantiate a template. You use a
template by importing it, as in:

import int32math "mymath(int32)"

There is a helper program "gotimports", which will parse your go file,
and instantiate any templates it finds imported, provided the
templates are in your local path. Searching for templates installed
in a central location is still on the TODO list. You can also use the
"gotit" program to generate templated go files directly, which is just
a bit more work.

I'll appreciate any feedback! I think this is quite an exciting
approach for generics, and should very useful.
--
David Roundy

David Roundy

unread,
Feb 27, 2010, 10:28:20 AM2/27/10
to golang-nuts
Hi all,

Based on a couple of suggestions, I've renamed gotit to gotgo, so the new URL is

http://github.com/droundy/gotit

David

Beoran

unread,
Feb 27, 2010, 1:57:09 PM2/27/10
to golang-nuts
I think this a great idea! +Kudos to you.

Perhaps such a preprocessor would also be nice
for other language features that features that perhaps won't make it
into Go proper? Like operator overloading, perhaps? :)

Jessta

unread,
Feb 27, 2010, 7:59:00 PM2/27/10
to Beoran, golang-nuts
On 28 February 2010 05:57, Beoran <beo...@gmail.com> wrote:
> Perhaps such a preprocessor would also be nice
> for other language features that features that perhaps won't make it
> into Go proper? Like operator overloading, perhaps? :)

We'll be C++ feature complete in no time!

-jessta
--
=====================
http://jessta.id.au

Marcelo Cantos

unread,
Feb 27, 2010, 11:53:05 PM2/27/10
to golang-nuts
I prefer to see such projects as incubators for features to be
eventually added to Go. Without baked-in generics, Go cannot be taken
seriously. Anyone not convinced of this should go on a quick excursion
to http://golang.org/pkg/container/vector/. If generics remain in an
non-normative preprocessor, such embarrassments will never go away.

Ryanne Dolan

unread,
Feb 28, 2010, 12:11:04 AM2/28/10
to Marcelo Cantos, golang-nuts
Marcelo,

I thought the same thing at first, but now I'm not so sure.  It's not really embarrassing, just different from what I would expect, coming from languages with generics.  In particular, Java didn't originally have generics, and yet it was taken seriously!  I've been working on code which I _thought_ would require generics down the road to be at all useful, but now I'm seeing that interfaces do the trick.  The only place interfaces fail (for me) is when I'd like to use basic types without boxing.  Go's Vector package skirts this problem by providing specialized implementations for Int and String.  Since there are only a few basic types, I'm not sure this a horrible compromise.

Thanks.
Ryanne

--
www.ryannedolan.info

Marcelo Cantos

unread,
Feb 28, 2010, 1:23:21 AM2/28/10
to golang-nuts
On Feb 28, 4:11 pm, Ryanne Dolan <ryannedo...@gmail.com> wrote:
> I thought the same thing at first, but now I'm not so sure.  It's not really
> embarrassing, just different from what I would expect, coming from languages
> with generics.  In particular, Java didn't originally have generics, and yet
> it was taken seriously!  I've been working on code which I _thought_ would
> require generics down the road to be at all useful, but now I'm seeing that
> interfaces do the trick.  The only place interfaces fail (for me) is when
> I'd like to use basic types without boxing.  Go's Vector package skirts this
> problem by providing specialized implementations for Int and String.  Since
> there are only a few basic types, I'm not sure this a horrible compromise.

Any serious large-scale project will invariably create a rich library
of data types and related code that is fairly domain-specific (i.e.,
don't rely on someone else writing it for you). For example, a math
library will spawn a variety of types such as vector, matrix,
quaternion, etc., and the first question that will come up is how to
handle the three different float types and ideally the 11 different
integer types. Older languages are stuck with replicating every single
data structure, and you will indeed see just this kind of mess in
libraries written for C. Some of the more innovative library authors
will wrap entire headers and source files in macros that substitute
some "T" for the appropriate type, thus essentially emulating
generics. It would be disappointing to see Go end up in the same
pickle, and the Vector library hints that this is precisely where it's
headed.

Java may have achieved considerable success without generics, but
after experiencing the pain of programming with pre-generics Java, I
can well understand why they finally capitulated. Java is a perfect
example of why Go _should_ get generics. Of course, Java's generics
were done quite badly[1], so firstly, extreme caution is in order when
thinking about Go's generics and secondly, the best time to do it is
when the language design is still fluid. This is why I very much like
the idea of a preprocessor-based implementation to serve as a test-
bed; with minimal effort, it lets the community sift through a wide
range of ideas, discarding a lot of bad ones in the process.


[1]The version of the story that I am familiar with: When implementing
Java generics, the designers tried very hard to avoid changing the JVM
and bytecode format. This lead to the introduction of the now infamous
type-erasure semantics. Apparently the bytecode ended up requiring
changes anyway, but it was too late in the process to revisit the
whole design, so they were stuck with an ugly compromise that proved
unnecessary. Generics were nonetheless a big improvement to Java, in
spite of the compromises.

Ryanne Dolan

unread,
Feb 28, 2010, 3:07:09 AM2/28/10
to Marcelo Cantos, golang-nuts
Marcelo, 

Whether Java's generics were really an improvement is debatable.

I think there are two main roads a language can follow.  First: template everything.  See STL and Boost.  Second: support only a few built-in data types in your libraries.  See Go's math, Vector, etc.

I've written enough C++ to know that templating everything can be hell.  Maybe you save a few instruction cycles in the end, maybe even a few hundred milliseconds somewhere, but is it really worth writing STL-style code every time you want a new container?  Have you ever tried to extend STL or Boost?  Not a fun time.

Java, for legacy reasons, takes the middle road: a lot of packages are implemented with generics, but a lot aren't.  It is easy to find methods in common packages that accept a parameter of type, say, float[], and you must convert your generic <Float> container back into an array to use it, etc.  While generics were meant to avoid conversions like these, a half-baked approach is inefficient and a pain.

Therefore, I think Go's current approach isn't horrible.  You might need to convert a []float to a []float64 once in a while, but the same can be said of Java.  You still need conversions in Java, and Java's generics don't really improve efficiency, so what is really improved?  I'd rather deal with conversions (which are simple, well-defined) than with STL-like templates all over the place.

Thanks.
Ryanne

--
www.ryannedolan.info

chris dollin

unread,
Feb 28, 2010, 3:32:57 AM2/28/10
to Ryanne Dolan, Marcelo Cantos, golang-nuts
On 28 February 2010 08:07, Ryanne Dolan <ryann...@gmail.com> wrote:

Whether Java's generics were really an improvement is debatable.

In my (limited) Java coding experience, they are a significant
improvement. Yes, there are warts and half-bakings, but the
post-generics Java is a much nicer language to read and write
than the pre-generics language.

 You still need conversions in Java, and Java's generics don't really improve efficiency, so what is really improved?

Clarity. Anywhere one uses a container or an iterator, which is to
say, all over the place. Anywhere one can manage to use Java's
weak implied polymorphic methods, which while a parsec or ninety-three
from what ML can give you, at least let you /glimpse/ what's possible.

And Go is a language with /first-class functions/, for which polymorphic
types are a gift from heaven. I understand the Go teams desire to
keep the language simple and adhering to their go-als; but I hope that
they will grasp and pull the generics nettle.

--
Chris "allusive" Dollin

Ryanne Dolan

unread,
Feb 28, 2010, 3:44:22 AM2/28/10
to chris dollin, Marcelo Cantos, golang-nuts
Clarity. Anywhere one uses a container or an iterator, which is to say, all over the place. 

I'm not convinced generics make anything clearer.

GenericContainer <Float> myContainer = new GenericContainer <Float> ();
..
for (float v : myContainer) {
...
}

Compare to:

Container myFloatContainer = new Container ();

for (float v : myFloatContainer) {
...
}

I don't think there is much of a difference.  Notice that the iterator syntactic sugar in both examples has nothing to do with generics. Also notice that I can abuse the container in either case, and that they are no different after erasure.

Thanks.
Ryanne

--
www.ryannedolan.info

chris dollin

unread,
Feb 28, 2010, 4:02:33 AM2/28/10
to Ryanne Dolan, Marcelo Cantos, golang-nuts
On 28 February 2010 08:44, Ryanne Dolan <ryann...@gmail.com> wrote:
Clarity. Anywhere one uses a container or an iterator, which is to say, all over the place. 

I'm not convinced generics make anything clearer.

GenericContainer <Float> myContainer = new GenericContainer <Float> ();

The declaration stuttering is tedious but on a par with
non-generic declarations. 

for (float v : myContainer) {
...
}

Compare to:

Container myFloatContainer = new Container ();

for (float v : myFloatContainer) {
...
}

I don't think there is much of a difference.

If all your containers are containers of float, sure. I expect
to be using containers -- sets, maps, lists -- and iterators
over any and all of Statement, Resource, RDFNode, Literal,
OntResource, Property, OntProperty, Model, Triple, Node,
and String. The differences are important and it's really
useful that the compiler can spot places where I got my
Strings and my Resources upmixed. Similarly in a chain
like

  model
    .listStatements()
    .mapWith( getSubject )
    .filterDrop( someFilteOrOther )
    .toSet()
    ;

the checking makes it much easier to get this right. Before we
had generics, straightforward processing chains like this were
significantly harder to get right.
 
 Notice that the iterator syntactic sugar in both examples has nothing to do with generics.

Sure.

Also notice that I can abuse the container in either case, and that they are no different after erasure.

Typically I haven't had to abuse containers.

--
Chris "allusive" Dollin

Marcelo Cantos

unread,
Feb 28, 2010, 4:45:44 AM2/28/10
to Ryanne Dolan, golang-nuts
On 28 February 2010 19:07, Ryanne Dolan <ryann...@gmail.com> wrote:
Whether Java's generics were really an improvement is debatable.

I don't know how debatable. Down in the trenches, generics-based Java code is worlds apart from pre-generics code. Whatever the theoretical problems, and design faux pas (I'm no stranger to those), no sane programmer would wish to remove them, or code without them. The proof is in the pudding.

I think there are two main roads a language can follow.  First: template everything.  See STL and Boost.  Second: support only a few built-in data types in your libraries.  See Go's math, Vector, etc.

I've written enough C++ to know that templating everything can be hell.  Maybe you save a few instruction cycles in the end, maybe even a few hundred milliseconds somewhere, but is it really worth writing STL-style code every time you want a new container?  Have you ever tried to extend STL or Boost?  Not a fun time.

And I've written enough C++ to know that appropriate use of templates can vastly simplify code, making it less bug-prone and faster at the same time. I agree that Boost is a great example of how badly awry things can go when powerful language features are placed in the hands of unbridled optimists.

Java, for legacy reasons, takes the middle road: a lot of packages are implemented with generics, but a lot aren't.  It is easy to find methods in common packages that accept a parameter of type, say, float[], and you must convert your generic <Float> container back into an array to use it, etc.  While generics were meant to avoid conversions like these, a half-baked approach is inefficient and a pain.

Therefore, I think Go's current approach isn't horrible.  You might need to convert a []float to a []float64 once in a while, but the same can be said of Java.  You still need conversions in Java, and Java's generics don't really improve efficiency, so what is really improved?  I'd rather deal with conversions (which are simple, well-defined) than with STL-like templates all over the place.

Attacking C++ templates and Java generics is a strawman argument. If Go were to implement generics according to those languages, that would be plain stupid. But it hardly constitutes an argument against generics per se. We all know what went right and what went wrong in those languages, and we ought to be able to learn from them.

Generics have very little to do with performance. When they are done well, performance is a nice by-product. The real gain is in type-safe code reuse. Go's current Vector implementation allows the wrong types to be thrown into a container, and the compiler will say nothing. This will lead programmers to either: 1) write code that breaks in production; 2) bake their own vector implementations, adding bugs as they go; 3) write way more unit tests to check for things the compiler could check; 4) use built-in arrays big enough to handle maximum expected counts, and which therefore break in production; 5) use built-in arrays with hand-coded growth logic and more bugs; or 6) use all the above in any decent-size program. I know this because I've seen it in production code.

If you want to see a well-designed generics architecture, have a look at .Net generics. They are straightforward, coherent, conservative, separately compiled, visible via reflection and focused on a few core goals, rather than turing completeness. And they possess none of the awful idiosynchrasies of Java generics. They don't give you the kitchen sink like C++ templates do, but the 80/20 approach has proven wildly successful among day-to-day programmers.

Beoran

unread,
Feb 28, 2010, 6:59:14 AM2/28/10
to golang-nuts
I agree wholly with Marcello's feelings here. Generics are very
useful.
We just have to find a way to get them "right".

That's why, rather than arguing abstractly, implementations such as
David's preprocessor are very important. They let us test the feature.
So in stead of "generic criticism", if you pardon my pun, I'd rather
see
criticism on David's implementation, how it could be improved, what
it
is missing, and what you like and dislike about it. That's the way
forward.

Ryanne Dolan

unread,
Feb 28, 2010, 11:57:58 AM2/28/10
to Marcelo Cantos, golang-nuts
If you want to see a well-designed generics architecture, have a look at .Net generics.

I have used .NET extensively, but I'm not impressed.  Improving a bit over Java isn't saying much.

 But it hardly constitutes an argument against generics per se. 

Sorry if I'm pessimistic here.  .NET languages suffer from a commercial mentality wherein MS has tried to please everyone (including yourself) by throwing in every feature imaginable.  It is still easier than C++ (shocking) and feels more powerful than Java (amazing) but isn't particularly better than either, if you ask me.  .NET certainly fails by _my_ standard of excellence, which is only simplicity.

If you have a revolutionary approach to generic code, I'd be very interested to hear about it at POPL or something.  I'm just saying that I _have_ used generics in many languages, and the languages I actually use do not support them, and I don't often miss them.

I wouldn't be horribly upset if Go supported generics at some point, but my experience with Go so far has shown me that generics aren't altogether orthogonal to interfaces.
 
Thanks.
Ryanne

--
www.ryannedolan.info

Ryanne Dolan

unread,
Feb 28, 2010, 12:04:56 PM2/28/10
to chris dollin, Marcelo Cantos, golang-nuts
Typically I haven't had to abuse containers.

Ha! I'd hope not.  I meant to imply that Java's generics can still be abused in the same way that pre-generics Java code can be abused, so that generics in Java don't afford you any real security.

the checking makes it much easier to get this right

I've argued for more compiler checks previously on this list, and in general I want the compiler to find as many bugs as possible.  I don't wholly disagree with you on this point.  However, I've also said that compiler checks should be added so long as they don't require a complicated or confusing language change.  I think generics are well-understood by most coders, but I don't think this particular compiler check is worth the added complexity.

Especially when you consider what happens when a language like Java goes from no-generics to generics.  Everyone re-writes everything, including the standard library!  I'm not saying the code was better the first way, but I am saying that generics are a feature that changes how people think about and write code.  Again, I will point out that interfaces will solve most of you problems in Go, if you are willing to think about them in the right way, rather than seeing them only from a perspective heavily influenced by generics.

Thanks.
Ryanne

--
www.ryannedolan.info

Marcelo Cantos

unread,
Feb 28, 2010, 7:18:35 PM2/28/10
to Ryanne Dolan, golang-nuts
On 1 March 2010 03:57, Ryanne Dolan <ryann...@gmail.com> wrote:
If you want to see a well-designed generics architecture, have a look at .Net generics.

I have used .NET extensively, but I'm not impressed.  Improving a bit over Java isn't saying much.

I strongly disagree that the difference is small, but this isn't a C#-vs-Java debate so I will have to leave it at that.

 But it hardly constitutes an argument against generics per se. 

Sorry if I'm pessimistic here.  .NET languages suffer from a commercial mentality wherein MS has tried to please everyone (including yourself) by throwing in every feature imaginable.  It is still easier than C++ (shocking) and feels more powerful than Java (amazing) but isn't particularly better than either,

I disagree with this black-and-white characterisation of .Net. Of course commercial forces play a role in anything Microsoft builds, but everything I've read and heard from the architects and other team members has left me with the overwhelming impression that they genuinely care about building something good and useful. But again, we digress.

if you ask me.  .NET certainly fails by _my_ standard of excellence, which is only simplicity.

Simplicity of what? The untyped lambda calculus is just about the simplest programming language imaginable. I don't know anyone who uses it to build web apps, though.

A key goal of language design is that it lets you write simple (i.e., easy to read/write/maintain) programs, which tends to push programming languages towards higher complexity. If I had to choose one goal, it would be simplicity of expression, not simplicity of language. The overarching goal, though, is to minimise the cognitive burden on the programmer.

If you have a revolutionary approach to generic code, I'd be very interested to hear about it at POPL or something.  I'm just saying that I _have_ used generics in many languages, and the languages I actually use do not support them, and I don't often miss them.

I think I get what you are saying. It is very difficult to get generics right (i.e., difficult to avoid "foolishness" in the design), therefore it is better to do without than to create a white elephant. I sympathise with this viewpoint, but at the same time, I want to write real programs that solve interesting problems, and I favour anything that makes it easier to write correct code. For all their failings in mainstream languages, templates and generics have invariably made things easier, usually much easier, by increasing code reuse, largely eliminating explicit casting and catching a bunch of errors at compile-time that would otherwise have been postponed to the testing phase. Compile-time errors are vastly easier to detect, and therefore fix, than runtime errors.

Of course, you can go too far and end up with, say, Haskell, which can leave you tearing your hair out for days just trying to get your code to compile. You need to find the sweet-spot that minimises the sum: coding time + testing time. Generics are one of those rare features that deliver savings on both sides of this formula, even when done badly. This makes them a no-brainer.

I wouldn't be horribly upset if Go supported generics at some point, but my experience with Go so far has shown me that generics aren't altogether orthogonal to interfaces.

Certainly there is some overlap. But two key differences, run-time vs. compile-time and boxing vs. contiguous-and-type-safe, matter an awful lot when you get beyond toy programs.

My original claim that started this discussion was that "Go cannot be taken seriously" without generics. I'll revise that to, "Go won't be taken seriously by a very large segment of the developer community". Rightly or wrongly, there are some things that developers today simply expect of a statically-typed language, and they won't forgive you if you tell them that it's not worth the effort or too hard to get right. Their experience is that it has been done well enough in other languages to be well worth the effort.


Cheers,
Marcelo

Ryanne Dolan

unread,
Feb 28, 2010, 7:39:49 PM2/28/10
to Marcelo Cantos, golang-nuts
Marcelo,

Great summary of both viewpoints.  I agree with you for the most part.

However, I suggest you look at Go more like a compiled Python than a competitor to C++, Java, C#, D, etc.  From that perspective, generics don't seem very necessary, nor does the efficiency or type safety considerations seem nearly as important.

In particular, runtime boxing etc don't keep people away from dynamic languages.

I agree that Go will not fit some niches without generics, but those niches don't happen to overlap with mine.

Thanks.
Ryanne

--
www.ryannedolan.info

D TQ

unread,
Mar 2, 2010, 5:17:10 AM3/2/10
to golang-nuts
Hi,
I'm new go user here my 2cents about generics.
Generics is about introducing type operator so that one get a new type
by combining 2 other types, like Vector<Car> is the result of Vector
and Car. Having type operator will lead to nesting construct like
Type1<Type2<Type3...>>>. While mere mortal developers will not use
that construct that often, the language implementator will have to get
it right. That will certainly complicate the type checker and as
consequence, slowdown the compiler.
One goal of go is "it compiles fast". To achieve that, the language
has to be relative simple.

Thanks

chris dollin

unread,
Mar 2, 2010, 8:19:50 AM3/2/10
to D TQ, golang-nuts
On 2 March 2010 10:17, D TQ <d.go...@googlemail.com> wrote:
Hi,
I'm new go user here my 2cents about generics.
Generics is about introducing type operator so that one get a new type
by combining 2 other types, like Vector<Car> is the result of Vector
and Car.

Not just two types: one, or three or more.
 
Having type operator will lead to nesting construct like
Type1<Type2<Type3...>>>.

Indeed.
 
While mere mortal developers will not use
that construct that often,
I'm not so sure.
 
the language implementator will have to get

it right. That will certainly complicate the type checker

Well, yes, if there's more to do, there's more to do. But
I'm not convinced that it's /that/ much more complicated.

and a consequence, slowdown the compiler.

Certainly there will be /some/ slowdown. The question is,
how much. A factor of 100? Unacceptable. A factor of 10?
Very uncomfortable. An extra 10%? I'll have two.
 
One goal of go is "it compiles fast". To achieve that, the language has to be relative simple.

Relatively simple, but not trivial. And some of us would
accept somewhat slower compilation for greater flexibility
in creating & reading code.

--
Chris "allusive" Dollin

Ioannis Mavroukakis

unread,
Mar 2, 2010, 8:31:15 AM3/2/10
to golang-nuts
Sorry I don't understand why you are seeing this as a new type. All I
would read is that Vector<Car> is
only allowed to hold type Car, perhaps you can explain your reasoning?

Thanks!

Y.

chris dollin

unread,
Mar 2, 2010, 9:55:05 AM3/2/10
to Ioannis Mavroukakis, golang-nuts
On 2 March 2010 13:31, Ioannis Mavroukakis <i.mavr...@btinternet.com> wrote:
Sorry I don't understand why you are seeing this as a new type. All I would read is that Vector<Car> is
only allowed to hold type Car, perhaps you can explain your reasoning?

Thanks!

Vector<Car> is a type that isn't Car and isn't Vector, so
it's a new type.

(A language with generic types need not be constructed
so that Vector is a type; it can be a type constructor. Or a
procedure ...)

--
Chris "allusive" Dollin

Marcin 'Qrczak' Kowalczyk

unread,
Mar 7, 2010, 4:03:30 AM3/7/10
to Ryanne Dolan, chris dollin, Marcelo Cantos, golang-nuts
2010/2/28 Ryanne Dolan <ryann...@gmail.com>:


> Especially when you consider what happens when a language like Java goes
> from no-generics to generics.  Everyone re-writes everything, including the
> standard library!

Since they got it wrong the first time, they suffer the pain, and now they have a complicated and flawed system. They should have generics from the beginning.


> Again, I will point out that interfaces will solve most of
> you problems in Go, if you are willing to think about them in the right way,
> rather than seeing them only from a perspective heavily influenced by
> generics.

I don't believe this.

Here is a merge sort I just coded in C++. Can you express it more conveniently in Go, using interfaces?

#include <algorithm>
#include <functional>
#include <iostream>
#include <iterator>
#include <vector>

template<typename InIt, typename OutIt, typename Cmp>
static void merge(InIt begin1, InIt end1,
                  InIt begin2, InIt end2,
                  OutIt result,
                  Cmp cmp) {
  while (begin1 != end1 || begin2 != end2) {
     *result++ =
        begin1 == end1 ? *begin2++ :
        begin2 == end2 ? *begin1++ :
        cmp(*begin1, *begin2) ? *begin1++ : *begin2++;
  }
}

template<typename InIt, typename OutIt, typename Cmp>
void merge_sort(InIt begin, InIt end,
                OutIt result,
                Cmp cmp) {
  std::size_t len = end - begin;
  if (len < 2) {
     std::copy(begin, end, result);
     return;
  }
  std::vector<typename std::iterator_traits<InIt>::value_type> tmp1, tmp2;
  std::size_t half = len / 2;
  merge_sort(begin, begin + half,
             std::back_inserter(tmp1),
             cmp);
  merge_sort(begin + half, end,
             std::back_inserter(tmp2),
             cmp);
  merge(tmp1.begin(), tmp1.end(),
        tmp2.begin(), tmp2.end(),
        result,
        cmp);
}

int main() {
  int values[] = { 3, 1, 4, 1, 5, 9, 2, 6, 5, 3 };
  std::vector<int> result;
  merge_sort(values, values + sizeof values / sizeof *values,
             std::back_inserter(result),
             std::less<int>());
  for (size_t i = 0; i < result.size(); ++i) {
     if (i != 0) std::cout << ", ";
     std::cout << result[i];
  }
  std::cout << std::endl;
}

--
Marcin Kowalczyk

Eleanor McHugh

unread,
Mar 7, 2010, 2:58:38 PM3/7/10
to golang-nuts
On 1 Mar 2010, at 00:39, Ryanne Dolan wrote:
> Marcelo,
>
> Great summary of both viewpoints. I agree with you for the most part.
>
> However, I suggest you look at Go more like a compiled Python than a competitor to C++, Java, C#, D, etc. From that perspective, generics don't seem very necessary, nor does the efficiency or type safety considerations seem nearly as important.
>
> In particular, runtime boxing etc don't keep people away from dynamic languages.

That'd be because programmers don't have to write code to handle the unboxing. A large part of the appeal of dynamic languages is that they don't clutter code with declarations, type casts or pointer maths. They also don't need generics because they focus on runtime state rather than compile-time assumptions about that state.

> I agree that Go will not fit some niches without generics, but those niches don't happen to overlap with mine.

Then your niche is far removed from the mainstream. Go could choose to be a ghetto, helping a few people solve particular problems where concurrency is the overriding consideration, or it embraces the needs of mainstream developers - many of whom work on largish (100k+ line) programs and rely on tools such as generics to simplify and reduce that codebase to a more maintainable level of complexity.

Personally I'm not particularly keen on generics but they're certainly a functional tool that's helped simplify a lot of C++ and Java code, and that's a definite win for developers working in those languages.

I'd much rather have concrete interfaces capable of supporting methods. For types such as Vector this would allow a lot of the drudgery involved in implementing type-specific variants to be factored out into a single core implementation as well as reducing the amount of explicit boxing and unboxing necessary.

In either case Vector is a type in its own right regardless of its contents and having a way of concretely expressing that fact is a good thing.


Ellie

Eleanor McHugh
Games With Brains
http://feyeleanor.tel
----
raise ArgumentError unless @reality.responds_to? :reason

Ryanne Dolan

unread,
Mar 7, 2010, 10:40:21 PM3/7/10
to Marcin 'Qrczak' Kowalczyk, chris dollin, Marcelo Cantos, golang-nuts
Can you express it more conveniently in Go, using interfaces?

See package "sort".

func Sort(data Interface)

Is that a less convenient interface?
 
Thanks.
Ryanne

--
www.ryannedolan.info

Marcelo Cantos

unread,
Mar 8, 2010, 12:09:50 AM3/8/10
to Ryanne Dolan, Marcin 'Qrczak' Kowalczyk, chris dollin, golang-nuts
On 8 March 2010 14:40, Ryanne Dolan <ryann...@gmail.com> wrote:
Can you express it more conveniently in Go, using interfaces?

See package "sort".

func Sort(data Interface)

Is that a less convenient interface?

Much less. The sort package also provides SortInts, SortFloats and SortStrings, and sort.Interface doesn't offer the primitives required to implement merge-sort. At bare minimum, a mergesort package requires its own analog of sort.Interface. To be of practical use, it would need to replicate the Sort... family of functions and the ...Array family of types.

What if I want to define a merge-sort for large input sets that don't fit in RAM (externalsort)? Do I need to go through the whole four-way replication mess yet again?

Finally, none of this extra implementation effort is at all convenient if you want to sort an array containing any of the other 13 numeric types (or use your own comparator).


Cheers,
Marcelo

Ryanne Dolan

unread,
Mar 8, 2010, 1:53:38 AM3/8/10
to Marcelo Cantos, Marcin 'Qrczak' Kowalczyk, chris dollin, golang-nuts
Finally, none of this extra implementation effort is at all convenient if you want to sort an array containing any of the other 13 numeric types (or use your own comparator).

That's exactly what the interface is for.  You can sort any of your 13 numeric types, or any other type with the sort package already.

sort.Interface doesn't offer the primitives required to implement merge-sort

If you need a merge-sort specific interface, make one.  

Again, I encourage everyone to try implementing generic code in Go using interfaces before you insist that generics are necessary.  It is possible, simple, and only slightly redundant (if you want to avoid boxing), as illustrated already in the standard library.

Thanks.
Ryanne

--
www.ryannedolan.info

Russ Cox

unread,
Mar 8, 2010, 2:17:51 AM3/8/10
to Marcelo Cantos, golang-nuts
Have you looked at the uses of sort in the tree?
The amount of code you need to write to sort a new
data type is very small. Here's the code from
src/pkg/io/ioutil.go that sorts directories read from
an *os.File:

// A dirList implements sort.Interface.
type dirList []*os.Dir

func (d dirList) Len() int { return len(d) }
func (d dirList) Less(i, j int) bool { return d[i].Name < d[j].Name }
func (d dirList) Swap(i, j int) { d[i], d[j] = d[j], d[i] }

There's absolutely no requirement that the input fit in RAM here.
You can implement those three functions however you see fit.
The SortInts, SortFloats, etc., are just helpers for a couple of
common types. But it's a lot more common to have your own
type, as for *os.Dir here, or to substitute a different comparison,
and so on.

Is four lines of code too much?

Russ

Beoran

unread,
Mar 8, 2010, 2:50:08 AM3/8/10
to golang-nuts
You see, the nice thing about Go is that it has a very simple syntax,
that makes it easy to parse, and easy to write preprocessors for. Did
you check out David Roundy's implementation of templates? It's just a
few hundred lines of Go code.

This convinces me that this discussion we're having here may be more
or less academic. I think that the best way to convince people of the
usefulness of templates is not by arguing about it here, but by
actually /using/ and /improving/ tools like gotgo. Let's create a
whole bunch of wonderful Go generics packages that use gotgo to
demonstrate the usefulness of generics in GO. The proof will be in the
pudding. :)


Kind Regards,

B.

Marcin 'Qrczak' Kowalczyk

unread,
Mar 8, 2010, 2:52:26 AM3/8/10
to Ryanne Dolan, chris dollin, Marcelo Cantos, golang-nuts
2010/3/8 Ryanne Dolan <ryann...@gmail.com>

Can you express it more conveniently in Go, using interfaces?

See package "sort".

func Sort(data Interface)

Is that a less convenient interface?
 
This interface is good only for sorting based on swapping elements of the original array. It is not good to implement merge sort (which guarantees O(N*log(N)) operations in the worst case, at the cost of temporary memory, so it's not a mere implementation detail what sort algorithm is used).

You could probably implement merge sort by creating an array of indices, sorting that by looking up the original elements for comparison, and then performing a series of swaps to match the computed permutation (and to first create a copy of the array if we want to return a new sorted arrray instead of sorting in place). Quite a roundabout way, definitely not convenient to write.

Merge is was just an example that generic container manipulation functions are awful to write in a statically typed language without generics.

--
Marcin Kowalczyk

Marcelo Cantos

unread,
Mar 8, 2010, 2:55:23 AM3/8/10
to Ryanne Dolan, Marcin 'Qrczak' Kowalczyk, chris dollin, golang-nuts
On 8 March 2010 17:53, Ryanne Dolan <ryann...@gmail.com> wrote:
Finally, none of this extra implementation effort is at all convenient if you want to sort an array containing any of the other 13 numeric types (or use your own comparator).

That's exactly what the interface is for.  You can sort any of your 13 numeric types, or any other type with the sort package already.

Then I must be missing something. In C++, I might do this:

    unsigned short a[10] = { 3, 2, 4, 5, ... };
    std::sort(a, a + 10);

I can sort in one line without declaring any new functions, interfaces or types. I can also sort in one line in Python, C#, Erlang, ML, Haskell, JavaScript, Perl, and so on. How do I do this in Go?

sort.Interface doesn't offer the primitives required to implement merge-sort

If you need a merge-sort specific interface, make one.  

Again, I encourage everyone to try implementing generic code in Go using interfaces before you insist that generics are necessary.

No one is saying that generics are necessary. Very little is necessary for turing-completeness. You could get by with a single byte array representing all available RAM and a looping construct. But that wouldn't be much fun, would it? Implementing Sort, SortInts, SortFloats, SortStrings, IntArray, FloatArray and StringArray in a mergesort package (and then an externalsort package, and a parallelsort package, and ...) doesn't sound like much fun either.
 
 It is possible, simple, and only slightly redundant (if you want to avoid boxing), as illustrated already in the standard library.

 How is this slightly redundant?

Marcelo Cantos

unread,
Mar 8, 2010, 5:40:13 AM3/8/10
to Russ Cox, golang-nuts
On 8 March 2010 18:17, Russ Cox <r...@google.com> wrote:
Have you looked at the uses of sort in the tree?
The amount of code you need to write to sort a new
data type is very small.  Here's the code from
src/pkg/io/ioutil.go that sorts directories read from
an *os.File:

// A dirList implements sort.Interface.
type dirList []*os.Dir

func (d dirList) Len() int           { return len(d) }
func (d dirList) Less(i, j int) bool { return d[i].Name < d[j].Name }
func (d dirList) Swap(i, j int)      { d[i], d[j] = d[j], d[i] }

There's absolutely no requirement that the input fit in RAM here.

[Minor quibble] Many sort algorithms go glacial when the input doesn't fit in physical RAM. Does sort.Sort avoid this?

You can implement those three functions however you see fit.
The SortInts, SortFloats, etc., are just helpers for a couple of
common types.  But it's a lot more common to have your own
type, as for *os.Dir here, or to substitute a different comparison,
and so on.

Since Go's designers felt the need to create helpers, surely this present a dilemma for anyone wanting to write a different sort or any other kind of algorithm over collections, for that matter. Can they just write a base version, and require all clients to expose an interface to it, or should they follow suit and write multiple wrappers for everything?

On a related note, why are the Sort... funcs present? Couldn't the user just call, e.g., sort.Sort(sort.IntArray(...))? Or are these functions full copies of the base algorithm recompiled for performance? If so, this makes the library author's dilemma worse.

Is four lines of code too much?

Frankly, yes. In C# (among many others), it's a zero-liner, since all the code is at the call site.

    files.OrderBy(f => f.Name);

The real difference is in reading effort due to loss of locality. I will scan the C# version without blinking, whereas an invocation of sort.Sort(dirList(...)) stops me dead in my tracks. I now have to skim backwards looking for the implementation, not only scanning the Less() func to figure out what it does, but also scanning the other funcs to confirm that they behave as expected. I then have to jump back to the call site and hope that the brief excursion hasn't scrambled my short-term memory (I am a bear of very little brain...). The relative cost is at least an order of magnitude, probably closer to two.

Moreover, if the requirement changes, and I now have to also sort by size, I don't face the dilemma of duplicating dirList and tweaking the copy vs. adding an extra layer of complexity to the original. I just add:

    files.OrderBy(f => f.Size);

Four lines may not be much, but it's four lines more than most modern languages require (It's even more than C's sort requires!), and when people come from those languages to kick Go's tyres, this is the sort of thing that makes them go "Huh?"


Cheers,
Marcelo

D TQ

unread,
Mar 8, 2010, 6:24:28 AM3/8/10
to Marcelo Cantos, golang-nuts
>
>     files.OrderBy(f => f.Name);
>
>
>     files.OrderBy(f => f.Size);
>

I'm not sure, but things like that are done with reflection?

D.

Marcelo Cantos

unread,
Mar 8, 2010, 7:46:18 AM3/8/10
to D TQ, golang-nuts

No, it all happens at compile time via type inference:

1. "OrderBy" resolves to Enumerable.OrderBy<TSource, TKey>(this IEnumerable<TSource> source, Func<TSource, TKey> key).
2. TSource resolves to FileInfo, based on the "input" parameter (files), which is an IEnumerable<FileInfo>.
3. The lambda function's f parameter is in turn inferred to be of type FileInfo.
4. The lambda function's expression is in turn inferred to be the type of f.Name == FileInfo.Name == string.
5. The output type, TKey, in turn resolves to type string (the type of the lambda expression).

Semantically, it is almost identical to this:

    IEnumerable<FileInfo> files;
    ...
    string GetName(FileInfo f) { return f.Name; }
    long GetSize(FileInfo f) { return f.Size; }
    ...
        Enumerable.OrderBy<FileInfo, string>(files, GetName);
        Enumerable.OrderBy<FileInfo, long>(files, GetSize);

Of course, this explanation hides (or perhaps highlights) the fact that the actual code is at once remarkably concise and clear, and all in one place.

roger peppe

unread,
Mar 8, 2010, 7:57:07 AM3/8/10
to Marcelo Cantos, Russ Cox, golang-nuts
On 8 March 2010 10:40, Marcelo Cantos <marcelo...@gmail.com> wrote:
> Moreover, if the requirement changes, and I now have to also sort by size, I
> don't face the dilemma of duplicating dirList and tweaking the copy vs.
> adding an extra layer of complexity to the original. I just add:
>
>     files.OrderBy(f => f.Size);

in the absence of generics, the current definition of sort.Interface
supports the sorting of unboxed arrays of objects.

it's easy to implement something similar to the C# approach
if you're willing to pay the boxing/unboxing penalty.

see the attached code for an example of how this can be done.

sortfunc.go

Marcelo Cantos

unread,
Mar 8, 2010, 4:44:46 PM3/8/10
to roger peppe, Russ Cox, golang-nuts
It is of course possible to do all kinds of interesting things with lambdas. But, being a pragmatist, I only care about what is useful and what is likely to achieve widespread adoption. The code you provided is certainly clever and interesting, but I don't see anyone jumping to write this kind of code:

    sort.Sort(FuncSort(d, func(x, y interface{}) bool {return x.(os.Dir).Name < y.(os.Dir).Name}))

vs. this:

    sort.Sort(d, func(x, y os.Dir) bool {return x.Name < y.Name})

or this:

    sort.SortBy(d, func(x os.Dir) string {return x.Name})

or (after adding a dash of type-inference to Go) this:

    sort.SortBy(d, func(x) {return x.Name})

Under the current state of affairs, FuncSort isn't of practical value, because programmers will always favour the out-of-the-box solution, especially if the alternative is clumsy and slow (no disrespect intended; this is unavoidable without generics).

Brian Slesinsky

unread,
Mar 8, 2010, 11:03:02 PM3/8/10
to golang-nuts
I haven't tried this out yet, but I'm curious. What happens if you
have a .got file that depends on another .got file?

If you have:

package foo(type a);

Can you do:

package bar(type b);
import foo "./foo(b)"


On Feb 27, 7:28 am, David Roundy <roun...@physics.oregonstate.edu>
wrote:
> Hi all,
>
> Based on a couple of suggestions, I've renamed gotit to gotgo, so the new URL is
>
> http://github.com/droundy/gotit
>
> David

roger peppe

unread,
Mar 9, 2010, 3:36:23 AM3/9/10
to Marcelo Cantos, Russ Cox, golang-nuts
On 8 March 2010 21:44, Marcelo Cantos <marcelo...@gmail.com> wrote:
> It is of course possible to do all kinds of interesting things with lambdas.
> But, being a pragmatist, I only care about what is useful and what is likely
> to achieve widespread adoption. The code you provided is certainly clever
> and interesting, but I don't see anyone jumping to write this kind of code:
>
>     sort.Sort(FuncSort(d, func(x, y interface{}) bool {return
> x.(os.Dir).Name < y.(os.Dir).Name}))
>
> vs. this:
>
>     sort.Sort(d, func(x, y os.Dir) bool {return x.Name < y.Name})

actually, that was (almost) the signature that i originally gave FuncSort,
but i changed it to return a sort.Interface because that means that it's
compatible with any other function that expects a sort.Interface
(e.g. sort.IsSorted). the type casting is necessary without generics.

> this is unavoidable without generics).

yes, go does not (yet?) have generics. this has been much discussed,
and complaining about it won't make it happen any faster, i'm afraid.

Marcelo Cantos

unread,
Mar 9, 2010, 7:50:48 AM3/9/10
to roger peppe, Russ Cox, golang-nuts
On 9 March 2010 19:36, roger peppe <rogp...@gmail.com> wrote:
yes, go does not (yet?) have generics. this has been much discussed,
and complaining about it won't make it happen any faster, i'm afraid.

I didn't start out trying to complain, merely to observe that Go really needs generics (not in the absolute you-can't-do-anything-without-them sense, just in the their-absence-will-seriously-hinder-adoption sense). Then a few tried to convince me how simple and easy everything is without generics, implying that Go can do without them. I disagreed and did my best to back up my dissent with cogent and valid arguments.

So far, this discussion has seemed to me to have the quality of a healthy debate, so I'm not sure what I should be doing differently.

In any event, I've said my piece regarding generics, and maybe everyone is sick to death of the subject, so I'll say no more.


Cheers,
Marcelo

David Roundy

unread,
Mar 9, 2010, 1:12:16 PM3/9/10
to Brian Slesinsky, golang-nuts
Hi Brian,

That's a good question. That *should* work, but it isn't something
I've gotten around to testing, so it probably doesn't. I've been
quietly hacking away on gotgo, and recently stopped to spend some time
switching my other projects to use gotgo, but I haven't gotten around
to making sure that works.

It's not so very hard to do, it just means that gotgo is going to have
to parse the import statements looking for references to the type
parameters. I recently replaced gotimports with a new gotmake program
that does the same job, but generates a makefile rather than running
gotgo itself. You can then either use the makefile by hand or use it
automatically (which is what I'm doing).

In the process of this rewrite (and getting things working in
iolaus-go and goopt), I changed to actually use the go parser to parse
the import strings when looking for imports needed to define the type.
e.g. so importing "./gotgo/slice([]foo.Bar)" will ensure that package
foo is available to the generated go file. So I already have code to
parse those strings, and would just need to put it into gotgo itself,
and we'd have your feature (which really is an important one).

David

--
David Roundy

Reply all
Reply to author
Forward
0 new messages