Go has eliminated most of my off-by-one errors except 0... I mean 1.

[Tom Limoncelli]

tl;dr: Go's "range" operator has eliminated the most common trap where
I make off-by-one errors. The next largest category of off-by-one
errors would be eliminated if there was a way to specify the last item
in an array. It would also improve a developer's ability to convey
intent.

...

I've been coding for a few years (first line of BASIC was in September
1979) and I have to admit that off-by-one errors have always been a
problem for me. I guess I'm just that kind of sloppy.

Speaking for myself, the biggest category of potential off-by-one
errors is when constructing "for" loops. Go's "range" operator
eliminates that entire category of potential mistakes. For that I'm
thankful.

So what is my next largest category?

Again, speaking only for myself, it is the fact that the last item in
an array is len(a)-1. It's easy to remember to subtract 1, but when
it's buried in the middle of a formula, it's non-obvious and obscured.
More importantly, it hides intent.

For example often the -1 is cancelled out by a +1 elsewhere in the
formula. Now that was originally len(a)-1-+1-y becomes len(a)-y.
Original intent obscured.

Or worse, a formula like len(a)-1-y becomes len(a)-z because someone
noticed that z==y-1 and made the substitution. Yes, it is a premature
optimization but I bet you’ve done a similar substitution many times.

Oh wait. Did you notice the typo in the previous paragraph? The
substitution would be valid if z==y+1, not z==y-1.

The fact that you didn’t notice proves my point. Now do you understand my pain?

Luckily I have a simple proposal that would fix this.

What if there was a built in function highest() that was like len(),
but returns the highest valid index. Sure, it is nothing more than a
substitute for len(a)-1. Not a super huge difference, but I think it
would eliminate this class of off-by-one errors.

My example from before becomes:

OLD: len(a)-1-+1-y
NEW: highest(a)+1-y`

The second example loses the temptation for premature optimization:

OLD: len(a)-1-y
NEW: highest(a)-y

Example: The classic Go “Pop from a stack” idiom is improved:

OLD: x, a = a[len(a)-1], a[:len(a)-1]
NEW: x, a = a[highest(a)], a[:len(a)-1]
Note: Intention is clearer: x is the last element, a has a reduction
in the length by 1.
NEW2: x, a = a[highest(a)], a[:highest(a)]
In this case the intention is even more clear: “x is the last element,
a is all but the last element.”

Loop through elements in reverse order:

OLD: for i := len(s)-1; i >= 0; i-- {
NEW: for i := highest(s); i >= 0; i-- {
Note: The intention is much clearer.


I realize that the go project is very reluctant to add new features
and rightfully so. That said, I think highest() (or maybe end(),
ultimate()? final(), hind()?) would be a great human-centric feature
to add.

Tom
WIth better formatting... https://www.yesthatblog.com/post/0068-go-off-by-one

--
Email: t...@whatexit.org Work: tlimo...@StackOverflow.com
Twitter: YesThatTom
Blog: https://www.yesthatblog.com

[Jan Mercl]

On Sat, Jun 13, 2020 at 4:42 PM Tom Limoncelli <t...@whatexit.org> wrote:

I'd suggest to just write the short 'end' function by yourself if you
think it's useful for you. It does not harm performance as it will get
inlined AFAICT.

But it IMO hurts readability a bit. I'd prefer to read the explicit
`len(x)-1` instead. Even within `len(x)-1-(y-1)` and similar.

And wrt to coders "optimizing" that to `len(x)-y`? I think I've sinned
that way too, but I hope I've learnt the lesson.

[Jake Montgomery]

On Saturday, June 13, 2020 at 10:55:43 AM UTC-4, Jan Mercl wrote:

On Sat, Jun 13, 2020 at 4:42 PM Tom Limoncelli <t...@whatexit.org> wrote:

I'd suggest to just write the short 'end' function by yourself if you
think it's useful for you. It does not harm performance as it will get
inlined AFAICT.

You cant really write a short 'end()' function. You have to write an end function for every type of slice. Or use reflection, which would make the function slow, and not inlined. Generics anyone?

[C Banning]

https://play.golang.org/p/EZEXmBig1y-

[Jan Mercl]

On Sat, Jun 13, 2020 at 5:37 PM Jake Montgomery <jake...@gmail.com> wrote:

> You cant really write a short 'end()' function. You have to write an end function for every type of slice. Or use reflection, which would make the function slow, and not inlined. Generics anyone?

I haven't mentioned anything about the function being universal. IMO
in the majority of cases, it does not have to be. Why is such
limitation being inserted?

It takes probably less time to write the "specialized" end function
than we have already spent discussing it. And that's assuming it's a
good idea to do so in the first place, which I don't really accept.

[Tom Limoncelli]

Thank you for the feedback!

I took Jan Mercl's suggestion and wrote the short function to see if
it is useful. It is based on C Banning's suggested code. I'm going to
start using it on my projects to see if it helps.

The module is called "hind": https://github.com/TomOnTime/hind

hind.S(x) works on strings. The compiler should inline this. It should
be no different that "len(x)-1" (other than conveying intent better).

hind.G(x) uses reflect to work on anything you pass it. It is slow
but it is a good placeholder until we have generics or whatever. It
is useful for when showing intent is more important than performance.

Feedback on the module would be greatly appreciated! If you use the
module, I'd love to know if you felt it reduced your off-by-one errors
(or the potential for them).

Best,
Tom

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAA40n-VbNBCKkbrtwG7mb4GEKmM6OBe_AetPRMG6jAsVf_ULxA%40mail.gmail.com.

--
Email: t...@whatexit.org Work: tlimo...@StackOverflow.com

Blog: http://EverythingSysadmin.com/

[Jake Montgomery]

On Saturday, June 13, 2020 at 1:09:16 PM UTC-4, Jan Mercl wrote:

On Sat, Jun 13, 2020 at 5:37 PM Jake Montgomery <jake...@gmail.com> wrote:

> You cant really write a short 'end()' function. You have to write an end function for every type of slice. Or use reflection, which would make the function slow, and not inlined. Generics anyone?

I haven't mentioned anything about the function being universal. IMO
in the majority of cases, it does not have to be. Why is such
limitation being inserted?

Yes, it was an insertion on my part. My point was that you cant just write a [single] performant end() function, you have to write lots of them. However, it was based on the original problem stated. Here was my reasoning. There is already a really short and very fast way of accessing the end of a slice: s[len(s)-1]. The OP seemed to indicate that he wanted something that would be used universally, and reflexively, instead. Something that would be more clear, and less error prone. It seems like it would be a hard sell to convince coders to stop, check if the function already existed, then probably switch files, and write a new end() function, every time they want to access the last element of a new slice type, instead of just typing a few characters.

It takes probably less time to write the "specialized" end function
than we have already spent discussing it. And that's assuming it's a
good idea to do so in the first place, which I don't really accept.

If I have to write a new one for every slice type, and then check or remember  if it exists every time I use it, then I would argue that it would add up to a great deal of time and cognitive energy over the life of a project. Certainly more than was spent, by me anyway, on this discussion.

[Jesper Louis Andersen]

On Sat, Jun 13, 2020 at 4:42 PM Tom Limoncelli <t...@whatexit.org> wrote:

I've been coding for a few years (first line of BASIC was in September
1979) and I have to admit that off-by-one errors have always been a
problem for me.  I guess I'm just that kind of sloppy.

They are surprisingly common! It has to do with recursion, but not the kind that is usually associated with the word in programming. We are not talking about calling yourself and extending the stack! A for-loop is a recursive process insofar every iteration is "the same" but an index is updated. Correctness proofs often work by showing that an "empty" structure (array) has the desired property. And that taking a step from a state with the desired property likewise exhibit it after the interaction.

The generalizations are known as "recursion schemes" which are ways to produce and consume structural data in a program. Functional languages are somewhat famous for treating programs as sheer symbolic processing on such structures. An interesting concept is the "unfold" which can be seen in Go as a function producing elements on an unbuffered channel. 

Nailing these kinds of schemes tend to eradicate a whole slew of typical processing bugs in programs.

[C Banning]

Well, I imagine there's only a few types in any code where the logic would require retrieving the "last" member of a list or array. Something like the following could address that and be more performant than using reflection - https://play.golang.org/p/XTbuf7RegF2.

> To unsubscribe from this group and stop receiving emails from it, send an email to golan...@googlegroups.com.