Potential language feature for testing value membership in if statements?

[Mike Schinkel]

Hi all,

While working on a parser, I've repeatedly encountered patterns like:

if c == ' ' || c == '\t' { /* handle whitespace */ }
if token == EOL || token == EOF { /* handle end markers */ }
if lexer.Type == TextLine || lexer.Type == WhitespaceLine { /* handle lines */ }

Go already allows multiple values in case statements:

switch c {
case ' ', '\t':
/* handle whitespace */
}

I'm wondering if there could be a more elegant way to express this pattern in Go, perhaps something conceptually similar to:

if c [some_syntax] ' ', '\t' { /* handle whitespace */ }

Do you think that the Go team would be likely to consider such a proposal, or would taking the time to prepare and submit a proposal likely just be for naught?

If this seems worthy of a proposal, what syntax do you think might be the best fit for the Go language?

Thanks,

-Mike

[robert engels]

Why not use something like

if token.in(EOL,EOF) …

with varadic args this is trivial.

And with generics you probably only need to write a single ‘in’ function (haven’t tested).

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/golang-nuts/F06145BC-B54C-4C3C-A04A-E9E6839E8169%40newclarity.net.

[Ian Lance Taylor]

We can already write

func Is[T comparable](s T, vals ...T) bool {
for _, v := range vals {
if s == v {
return true
}
}
return false
}

and then we can write

if Is(c, ' ', '\t') {
...
}

So I don't see a big need for additional syntax here.

Ian

[Mike Schinkel]

Hi Ian,

Thank you for your quick reply. I appreciate your suggestion of using a generic function approach.

After your response, I wondered about the performance difference so I wrote some performance benchmarking to see if there is a tangible difference. to better understand the implications of different approaches to membership testing.

I wrote a benchmark checking four dimensions including:

• Different techniques (OR expressions, switch statements, generic functions)
• Small vs. large sets of values (3 and 26 characters)
• Different positions in the value set (first, middle, last, no match)
• Different value sources (literals, constants, variables, method calls)

The results indicated a material difference for when performance is a top concern. While the generic function approach does work, it comes with a performance cost compared to switch statements:

• Median case: Generic function is 78% slower than switch case. I figure this is the common case.

• Average case: Generic function is 131% slower than switch case

• Worst cases: Up to 181% or more slower in scenarios with a large number of values to compare to

These performance differences are magnified in parser implementations.

The complete benchmark results are here: https://docs.google.com/spreadsheets/d/15cddzW8xG5JHYzPzu_u6XsXdoQ8Uh_6nVKslwODNSu0/edit?usp=sharing

Anyone reading this with any concerns please check the logic of my benchmarks and analysis (source found here: https://gist.github.com/mikeschinkel/2f848e25ca383d9bb76fd60e8c7b0a4f) as there was a lot of complexity, and I certainly could have made some errors in my analysis.

Given these benchmark results, would the Go team reconsider such a language feature on a basis of performance vs. a comparison of syntax to bring if statement performance closer to that of switch statements? The current disparity means developers must choose between the awkward single-case syntax of switch statements and the more natural control flow of if statements, often sacrificing either readability or performance.

I understand Go's philosophy of keeping the language small and orthogonal, but this seems like a case where a targeted language feature could improve performance for use-cases that needs it, or at least make the gaining that performance much less awkward.

Thank you again for the consideration,

-Mike

[Ian Lance Taylor]

On Wed, Mar 19, 2025 at 10:43 AM Mike Schinkel <mi...@newclarity.net> wrote:
>
> Given these benchmark results, would the Go team reconsider such a language feature on a basis of performance vs. a comparison of syntax to bring if statement performance closer to that of switch statements? The current disparity means developers must choose between the awkward single-case syntax of switch statements and the more natural control flow of if statements, often sacrificing either readability or performance.
>
> I understand Go's philosophy of keeping the language small and orthogonal, but this seems like a case where a targeted language feature could improve performance for use-cases that needs it, or at least make the gaining that performance much less awkward.

It's very unusual for Go to adopt language changes for performance
reasons. In fact, nothing comes to mind. Language changes are made to
improve expressibility and readability. Where performance is
important, we prefer to address it using tooling or library functions.

Ian

[Mike Schinkel]

Okay, thank you for considering and replying.

-Mike
P.S. The proposed change would improve expressibility and readability too, but I digress.

[Robert Engels]

I suspect if you wrote a small real world parser you would find a negligible performance difference.

I’ll look for one on GitHub.

> On Mar 19, 2025, at 3:24 PM, Mike Schinkel <mi...@newclarity.net> wrote:
>
> 

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

> To view this discussion visit https://groups.google.com/d/msgid/golang-nuts/87033509-CD21-4C98-A4F7-8C18F76ADFF5%40newclarity.net.

[Mike Schinkel]

> On Mar 19, 2025, at 5:34 PM, Robert Engels <ren...@ix.netcom.com> wrote:
>
> I suspect if you wrote a small real world parser you would find a negligible performance difference.

I am writing a real-world parser. I'll benchmark it when I am done.

-Mike

[Jason E. Aten]

Hi Mike, 

I don't know what quirk of human nature is that causes 

so many to want to change a language to fit their whims, when 

that language has as a principal virtue that it wants to be super readable

and almost never change (visit C++ for a strong argument 

as to Go's posture is so beneficial; Go was in part a reaction to C++ 

and so actual language changes are very rare), 

but I'm starting to grok why Lisps gets cult-like fervent adherents.

A Lisp bends to any whim, and that must be

a powerful human behavioral reinforcer. 

Digression aside, and back to the point at hand:

Generics are great for type safety and expressiveness. Most folks 

have figured out by staring at their profiler output that at their 

current maturity level, they can also be quite a bit slower than 

non-generic code, on a case-by-case basis.

Recall that RSC blog where he posted the 3-way speed trade-offs that generics

represent, and realize that those performance tradeoffs 

still hold, and are still real, despite the tangible gains in

usability and type safety that the current "hybrid" generics

implementation offers. (Reference: the 2009 blog,

 "The Generic Dilemma", https://research.swtch.com/generic )

So the point is: if you are optimizing for speed, you don't _typically_ 

reach for generics in Go. They've got to be compiled fast

and not bloat the binary size, so they aren't going to be

super runtime efficient.

You reach for super simple code, maybe codedgen that allows

inlining, and perhaps regular expressions for some lexing scenarios such as 

floating point numbers, because they compile to a little finite

state machine that can end up being very fast while being

relatively easy to specify and check that it is correct. 

You can always write assembly or SIMD AVX512 code if you really need it

to go faster. That's all supported today. And LLMs are not half

bad at assembly these days.

Best wishes,
Jason

[Mike Schinkel]

> On Mar 19, 2025, at 8:18 PM, Jason E. Aten <j.e....@gmail.com> wrote:
> You reach for super simple code, maybe codedgen that allows
> inlining,

There are definitely code generators that exist for lexers and parsers — Pigeon, goyacc, Gocc, Textmapper, etc. — and those are great projects in their own right.

But I am working on a fork of an existing hand-coded parser, one that was not created with such a code generator, and that type of coding is much more common in day-to-day work, at least for any projects I have been a part of.

For day-to-day coding work — vs. working on a dedicated code generator project — the idea of writing a code generator to generate a state machine for a lexer reminds me of that old meme "The developer had a problem so they thought 'Hey, I can use a regex for this!' Now the developer has two problems."

> I don't know what quirk of human nature is that causes
> so many to want to change a language to fit their whims,

While some people view other's motivations as "whims" the people being viewed pejoratively are often experiencing actually pain points. The pain point I was experiencing is having to write (and later having to read) this tortured control structure in many different places:

switch c {
case ' ', '\t', '\n':
return Token
}
}

Since I envisioned that Go could have a much simpler, clearer, and easier to read alternative, it seemed to me like it would be a great fit for the Go language especially given the more ergonomic recent improvements the Go team made to the for loop:

if c is ' ', '\t', '\n':
return Token
}

This construct could have leveraged the existing multi-matching behavior of switch-case and it felt smaller in scope but with similar utility compared to the recent for loop enhancements.

> when that language has as a principal virtue that it wants to be super readable

Given the specific pain point that was my motivation to request a language enhancement, I find that comment to be super ironic.

> You can always write assembly or SIMD AVX512 code if you really need it
> to go faster. That's all supported today.

It sounds like you are suggesting jumping out of the frying pan — e.g. having to write and later read five (5) lines with two (2) lines of indent vs. three (3) lines with one (1) level of indent — and planting one's self firmly in the fire. AFAICT, an assembly/SIMD AVX512 solution would create more pain for this use-case, not less.

BTW, I replied a second time to Ian after my initial email because — even though my primary interest was in having improved readability — I discovered that what I proposed would perform typically ~75% better than the generic Is(), assuming the current switch case is any indication. So since it seemed to me like a wonderful "kill two birds with one stone" kind of win — an easier-to-write and easier-to-read construct that performs better than the offered alternative — I made the incorrect assumption that such a feature would be a no-brainer enhancement to Go so I asked to verify if performance would provide any addition motivation to revive what I was told was a dead horse.

But given my intuition on this idea failed — and it clearly is a dead horse, with no hope for revival on the horizon — is there any upside to continue beating it?

-Mike

P.S. I was not planning to continue this thread because Ian shut down the idea so AFAICT further discussion was a moot point. OTOH, since you took the effort to write a detailed argument I felt I should at least give you the courtesy of responding.

[Wojciech S. Czarnecki]

Dnia Wed, 19 Mar 2025 23:21:56 -0400
Mike Schinkel <mi...@newclarity.net> napisał/a:

> The pain point I was experiencing is having to write (and later having to read) this tortured control structure in many different places:

> switch c {
> case ' ', '\t', '\n':
> return Token
> }

You can write this in a single line, if you wish. While you are not required to write semicolons in appropriate places, you can. You can write Go in a single line, to let feel it "smaller in scope":

" switch c { case ' ', '\t', '\n': return Token }; " will compile happily. Try:

echo "package main; func main() { x := func (c rune) int { switch c { case ' ', '\t', '\n': return 1 }; return 0 }; println(x(' ')) }" > foo.go ; go run foo.go

Hope this helps,

--

Wojciech S. Czarnecki
OHIR-RIPE

[robert engels]

For a gut check, I asked AI to generate a basic parser and tokenizer.

It wrote it pretty much the way I would, except for a few places if I was really concerned about performance I MIGHT change. For instance, the isOperator() should have a precheck that the rune is even in the range before searching the set.

In reviewing the code, I don’t even see a single place where ‘in’ would help the readability or the performance. Clearly this has very basic functionality, and a there might be more opportunities in a production version - but I think most of the techniques used here would be performant enough - and the readability seems ok to me.

The code is here https://go.dev/play/p/Zto4h7xVDE7

> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

> To view this discussion visit https://groups.google.com/d/msgid/golang-nuts/31FAFEE2-04AE-48CC-A3BB-2E5BA269D76E%40newclarity.net.