types of slice indices and range operators

[Steve Roth]

Greetings,

I'm considering submitting a proposal for a language change, and would like some informal feedback on the idea before engaging the formal process.  The idea being proposed here is intended to improve type safety by removing the need for some error-prone type casting.  It is a backward compatible change.

Consider a body of code that works with people and events.  Both are identified with integer IDs; in order to prevent them being used in the wrong contexts, they are given specific integer types:

type EventID int

type PersonID int

Assume also that people have lists of events they attend:

type Person struct {

Events []EventID

}

The code maintains slices of each type:

var events []*Event

var people []*Person

When indexing into those slices, one can use the appropriate ID type:

var eventID EventID = 3

println(events[eventID])

However, iterating over these slices requires casting in the current Go specification:

for eventID := range events {

// eventID here has type int, not type EventID

person.Events = append(person.Events, eventID) // compile error, wrong type

person.Events = append(person.Events, EventID(eventID)) // cast required

}

In cases where the event ID needs to be used inside the loop, it has lost its type safety.  It has to be casted back to the type it is supposed to have.  And that casting is error-prone; I could easily cast it to PersonID by mistake.  This seems to be a noteworthy gap in type safety.

My proposal is to allow this construction:

var eventID EventID

for eventID = range events {

// eventID here has type EventID

person.Events = append(person.Events, eventID) // accepted by compiler

}

Phrased more formally: a slice range operator can be assigned to a variable of any integer-derived type, not just "int".  If this were done, error-prone casting could be avoided.

Admittedly, there is still room for error here, since it would be possible to write

var eventID PersonID

for eventID = range events {

// eventID here has the wrong type PersonID

person.Events = append(person.Events, eventID) // compile error, wrong type

}

However, this error seems far less likely that a mistaken cast.  And since there's no expectation that casts should be needed, the compiler error would cause greater scrutiny leading to fixing the real problem.  (In any case, I don't wish to propose the (much larger and more impactful) change of having slices with typed indices.)

I believe this proposal would significantly improve type safety in programs that work with multiple integer-derived types (such as most anything working with a relational database that prefers integer primary keys).  I also believe it is backward compatible since it does not change the semantic of any existing code; it just adds a semantic to code that currently would not compile.

I solicit the community's feedback.  Should this proposal be formally submitted?

Regards,

Steve

[Robert Engels]

You are using range incorrectly - you are using the index not the value. 

See https://tour.golang.org/moretypes/16

On Feb 2, 2020, at 7:39 PM, Steve Roth <st...@rothskeller.net> wrote:



--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAAnpqKHS%3D2z0ZbNVSrULXLLGz%3DhAqpmrsLieciFu8L6%3DAn%3DLHA%40mail.gmail.com.

[Steve Roth]

Oh, please, Robert.  No need to be condescending.  I know perfectly well how to use the value when that's what I want.  Often it is not.  The examples that I cited are such a case: we are adding the IDs of the events — the indices of the slice — to the person.Events field.  If you don't like my example, take a look through the standard library code: you'll find a great many instances of range loops using only the index.

Regards,

Steve

[Robert Engels]

It not being condescending. Your code is not correct. It is not an eventID - it is an index (an int). Slice indexes are ints. If you want a typed key (other than int) you need a map (or variant). 

On Feb 2, 2020, at 8:17 PM, Steve Roth <st...@rothskeller.net> wrote:



To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAAnpqKEEOPicjwD%3Dmy08ORMzOwFiMFy4F%3DG3L9bdLFfbbsnHWA%40mail.gmail.com.

[Robert Engels]

Also, what you are asking for I believe is covered by generics. 

On Feb 2, 2020, at 8:17 PM, Steve Roth <st...@rothskeller.net> wrote:



To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAAnpqKEEOPicjwD%3Dmy08ORMzOwFiMFy4F%3DG3L9bdLFfbbsnHWA%40mail.gmail.com.

[Robert Engels]

One last point, if you are interested in type safety, why not create simple structs like EventList to encapsulate the indexing, etc. Trivial and can be done today. 

On Feb 2, 2020, at 8:28 PM, Robert Engels <ren...@ix.netcom.com> wrote:



To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/75113376-DE42-4B0E-BF0E-456AC8D44418%40ix.netcom.com.

[Steve Roth]

Hi, Robert,

Thanks for the quick replies.  Let me address each of your points:

"Slice indexes are ints" is clearly true in the current language definition.  From a pure language standpoint, one might assert that any index type other than plain int requires a map.  However, if one has a data structure indexed by (small positive) typed integers, it is unrealistic to suggest using a map for storage of it when a slice is dramatically more efficient in both space and time.  I wrote up this potential proposal because my colleagues and I have tripped over this issue multiple times in real-world applications.

What I'm suggesting may be covered by generics if and when they ever happen, but considering that generics are a breaking change and are years away at best, they are not relevant to the evaluation of a small, backward-compatible change.

As for the simple structs like EventList: they don't solve the problem of excess type casts; they merely move it to a different part of the code.  In fact we do use encapsulating structures, but I omitted them from my writeup for clarity.

As a separate, straw-man example, consider the common case of defining an enumerated integer type:

type MyEnum int

const (

EnumVal1 MyEnum = iota

EnumVal2

EnumVal3

)

Once again, if one needs a data structure indexed by MyEnum, it makes much more sense to define it as a slice rather than a map.  The use of iota quite literally guarantees that the values are suitable for slice indexing.  I'm simply trying to alleviate the need for excess, error-prone casting when doing so.

Regards,

Steve

[Robert Engels]

I think the major difference is that by using EventID as a constrained type, you are then attempting to constrain index values - for which there are no constraints other than slice length. 

Similarly, using an enum as a slice index would NEVER be safe. Enums are sparse. Slices are contiguous. Any safety here would only be by convention - and subject to great runtime risk (invalid index). 

I really believe using EventList and the Iterable pattern will give you the greatest type safety and “idiomatic Go” simplicity. 

As you state, you are already doing this (but you omitted it). The type casts are zero cost. Can you explain why this pattern doesn’t work for you (other than the common arguments for genetics)?

On Feb 2, 2020, at 9:08 PM, Steve Roth <st...@rothskeller.net> wrote:



[Robert Engels]

*generics :)

On Feb 2, 2020, at 9:27 PM, Robert Engels <ren...@ix.netcom.com> wrote:



[Steve Roth]

Your assertion that "using an enum as a slice index would NEVER be safe" seems rather extreme to me.  It may be true in an academic sense, but in pragmatic engineering, enums defined using iota are not sparse and using them as slice indices would generally be quite safe.  Yes, this is safety by convention, but conventions are often effective.

To answer your question:  type casts are not "zero cost".  They don't carry any runtime penalty, but they carry a cost in terms of code maintenance.  Code that is sprinkled with type casts is harder to follow and reason about.  More perniciously, once a developer gets in a habit of "fixing" compiler errors by adding type casts, it's only a matter of time before incorrect ones are added.

At any rate, I think both of our perspectives on this proposal are clear.  At this point, I think we should let it sit to see what other opinions get voiced.  If there is no support for the idea by anyone else, then further discussion between the two of us is moot.

Regards,

Steve

[burak serdar]

On Sun, Feb 2, 2020 at 8:52 PM Steve Roth <st...@rothskeller.net> wrote:
>
> Your assertion that "using an enum as a slice index would NEVER be safe" seems rather extreme to me. It may be true in an academic sense, but in pragmatic engineering, enums defined using iota are not sparse and using them as slice indices would generally be quite safe. Yes, this is safety by convention, but conventions are often effective.
>
> To answer your question: type casts are not "zero cost". They don't carry any runtime penalty, but they carry a cost in terms of code maintenance. Code that is sprinkled with type casts is harder to follow and reason about. More perniciously, once a developer gets in a habit of "fixing" compiler errors by adding type casts, it's only a matter of time before incorrect ones are added.

I disagree with your assertion that type casts are hard to follow. In
my opinion, implicit type conversions are harder to follow because
reading code with implicit type conversions requires the reader to
keep going back to figure out the actual types of variables. Explicit
type casts are more work when writing, but lets one read the code with
mostly "local" knowledge.

That said, I think your proposal addresses an edge case with limited
use. The cost is that you can no longer assume the index of a range is
always an int.

> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAAnpqKFTAvvLenhBVi_mvw2tS%2B4QzAXvJE1RGHGwWy6CTbPhmg%40mail.gmail.com.

[Kurtis Rader]

On Sun, Feb 2, 2020 at 7:52 PM Steve Roth <st...@rothskeller.net> wrote:

Your assertion that "using an enum as a slice index would NEVER be safe" seems rather extreme to me.  It may be true in an academic sense, but in pragmatic engineering, enums defined using iota are not sparse and using them as slice indices would generally be quite safe.  Yes, this is safety by convention, but conventions are often effective.

Sorry, but I disagree with you. In fact, I have encountered several bugs in the past four decades precisely because the "convention" was that the range of an enum started at zero (or one) and the values were contiguous. When the definition of the enum was changed so that it violated either of those conventions the code was broken. Note that I didn't deliberately, just for the heck of it, change the definition of the enum values. I did so because the incorrect assumption masked bugs in the code. The bugs were most often in unit tests but I found at least two bugs in the code being tested when I violated those "conventions".

--

Kurtis Rader

Caretaker of the exceptional canines Junior and Hank

[Ian Lance Taylor]

On Sun, Feb 2, 2020 at 7:08 PM Steve Roth <st...@rothskeller.net> wrote:
>
> What I'm suggesting may be covered by generics if and when they ever happen, but considering that generics are a breaking change and are years away at best, they are not relevant to the evaluation of a small, backward-compatible change.

The current generics design draft
(https://go.googlesource.com/proposal/+/master/design/go2draft-contracts.md)
is not a breaking change. It may be years away, but that isn't a
strong argument for making a change that would be unnecessary if we
had generics.

As far as I can tell, you can already do what you want in Go by using
a map. And in fact a map would be even more type safe, as in your
suggestion there is nothing that forces the index of a []*Event to be
an EventID. Of course a map would have different performance
characteristics.

All in all this seems like a fairly subtle change, that introduces an
additional implicit type conversion to the language. Or, perhaps,
treats the first value in a range clause as an untyped integer,
although it is not a constant. It seems to me that this in effect
adds another inconsistency to the language, which means another
special case that people have to learn. That doesn't sound like a
great idea. It would be better if it could be a specific effect of a
general rule, but I don't see how to do that.

Ian