Preprocessing "go" statements with preserved semantics

[André Eriksson]

I am working on a type of Go preprocessor that rewrites source code to add additional instrumentation to certain types of statements.

One such statement is the go statement. I would like to instrument the newly created goroutine, injecting some instrumentation code at the start and finish of the goroutine.

In the simple case, the rewrite is straightforward:

go fn()

becomes

go func() {

defer instrument()()

fn()

}()

However this approach does not work when fn takes parameters.

If we were to rewrite go fn(expr) into the equivalent form above:

go func() {

defer instrument()()

fn(expr)

}()

the semantics change, since in the rewrite expr gets evaluated inside the newly created goroutine, which can change the behavior and introduce data races.

My attempts to address this have not been particularly fruitful.

One cannot pass in expr as an argument to the closure, because the type of the expression may not have a valid name in the current package (for example if expr evaluates to a private type in some other package). 

Similarly, if expr is a constant expression (like 1 or nil) the type may depend on the corresponding parameter in fn’s signature.

The only semantics-preserving rewrite I can think of revolves around using package reflect, and rewriting like so:

go func(fn reflect.Value, vals …reflect.Value) {

defer instrument()

fn.Call(vals)

}(reflect.ValueOf(fn), reflect.ValueOf(expr))

As far as I understand, this should be semantics-preserving, although with a slight performance cost. (Though I imagine the cost of a reflection-based call is dwarfed by the cost of spawning a goroutine.)

Unfortunately this also comes with a major downside: the rewritten code does not typecheck identically to the original code. Ideally I would like the rewritten form to cause identical typechecking failures to the old code, so that these errors are caught at compile time without requiring a separate typechecking pass for the original code.

Am I correct in the above reasoning? Can anyone think of a way to do this sort of rewrite in a semantics-preserving and typechecking-preserving way?  

[Michael Jones]

If expr was evaluable in the original code then why not rewrite in place after assigning temporaries?

go fn(e1,e2)

{

t1,t2 := e1,e2

go func() {

  defer instrument()

  fn(t1,t2)

}

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/a92641f3-2eda-4d4a-ab02-d2b40e3bde75%40googlegroups.com.

--

Michael T. Jones
michae...@gmail.com

[André Eriksson]

That works in simple cases, but does not work when the expression is an untyped constant, like 1 or nil. In the case of 1 the variable will get a concrete type of int, while fn may accept a float32, or even a private type that cannot be named in the current package.

To unsubscribe from this group and stop receiving emails from it, send an email to golan...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/a92641f3-2eda-4d4a-ab02-d2b40e3bde75%40googlegroups.com.

[Michael Jones]

Alas. Thus the need for and glory of macros, hold/uneval, and backtick in LISP. (Problems solved in the 1970s)

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/ec41a345-163f-4a8a-a24f-b868def081a0%40googlegroups.com.

[Ian Lance Taylor]

On Fri, Nov 8, 2019 at 9:08 AM André Eriksson <ean...@gmail.com> wrote:
>
> That works in simple cases, but does not work when the expression is an untyped constant, like 1 or nil. In the case of 1 the variable will get a concrete type of int, while fn may accept a float32, or even a private type that cannot be named in the current package.

You might want to look at the rewriting that cmd/cgo does, as it
handles this exact kind of case. Basically, for untyped constants,
you know the desired type, because it's in the function signature. So
use that.

It does get kind of complicated, though.

Ian

> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/ec41a345-163f-4a8a-a24f-b868def081a0%40googlegroups.com.

[André Eriksson]

Interesting. Do you have a reference to where that happens?

If i understand you correctly, however, it doesn't appear to solve the case where the called function fn lives in a different package, and takes an argument which is a private type. That is:

-- a/a.go --

package a

type myString string

func Fn(str myString) { }

-- b/b.go --

package b

import "a"

func rewrite() {

    go a.Fn("some string")
}

In this example knowing the desired type from the function signature does not help, I would think?

> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/ec41a345-163f-4a8a-a24f-b868def081a0%40googlegroups.com.

[Ian Lance Taylor]

On Fri, Nov 8, 2019 at 10:06 AM André Eriksson <ean...@gmail.com> wrote:
>
> Interesting. Do you have a reference to where that happens?

The method (*Package).rewriteCall in cmd/cgo/gcc.go. But more useful
might be to experiment with some cgo code and build with `go build
-work` and look in the $WORK directory to see the generated files.

> If i understand you correctly, however, it doesn't appear to solve the case where the called function fn lives in a different package, and takes an argument which is a private type. That is:
>
> -- a/a.go --
> package a
>
> type myString string
>
> func Fn(str myString) { }
>
> -- b/b.go --
> package b
>
> import "a"
>
> func rewrite() {
> go a.Fn("some string")
> }
>
> In this example knowing the desired type from the function signature does not help, I would think?

That is correct.

Fortunately for a case like that you don't need to use a temporary
variable at all, since the argument is a constant.

> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/0ff39043-4699-4f0b-a1c0-6c4ebc5eb208%40googlegroups.com.

[Bakul Shah]

There is Massimiliano Ghilardi’s gomacro.... Github.com/cosmos72/gomacro

Note that in case of untyped constants, there is no need to use temps so your idea can still work.

On Nov 8, 2019, at 9:30 AM, Michael Jones <michae...@gmail.com> wrote:



To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CALoEmQxe--frSeHfKo822bKhWStJoQdJpkNdAYM6zTPLUuZ%2BAg%40mail.gmail.com.

[André Eriksson]

On Friday, November 8, 2019 at 7:16:42 PM UTC+1, Ian Lance Taylor wrote:

On Fri, Nov 8, 2019 at 10:06 AM André Eriksson <ean...@gmail.com> wrote:
>
> Interesting. Do you have a reference to where that happens?

The method (*Package).rewriteCall in cmd/cgo/gcc.go.  But more useful
might be to experiment with some cgo code and build with `go build
-work` and look in the $WORK directory to see the generated files.


> If i understand you correctly, however, it doesn't appear to solve the case where the called function fn lives in a different package, and takes an argument which is a private type. That is:
>
> -- a/a.go --
> package a
>
> type myString string
>
> func Fn(str myString) { }
>
> -- b/b.go --
> package b
>
> import "a"
>
> func rewrite() {
>     go a.Fn("some string")
> }
>
> In this example knowing the desired type from the function signature does not help, I would think?

That is correct.

Fortunately for a case like that you don't need to use a temporary
variable at all, since the argument is a constant.

Ian

That makes sense. I did some more research and came across another case where even this might not work.

It appears that there is a distinction between an untyped value and an untyped constant. I hadn't appreciated this distinction until now.

According to the spec (https://golang.org/ref/spec#Comparison_operators), "Comparison operators compare two operands and yield an untyped boolean value."

As a result, it's possible to get untyped, non-constant values from non-constant expressions.

So we could have fn(a() == b()) passed to fn(b myBool) with no ability to store this argument in a temporary variable without coercing the type to a regular typed bool.

> To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/0ff39043-4699-4f0b-a1c0-6c4ebc5eb208%40googlegroups.com.

[Ian Lance Taylor]

Technically true but essentially never happens in practice.

Ian

[André Eriksson]

I suppose that's fair, but it's hard to come up with concrete rules for what is permissible given a particular implementation of the rewrite, which makes it hard to explain to potential users of the tool.

Switching gears a bit, I guess this might be easier to implement as a modification to the compiler? Or can you think of another way of making the general case more tractable?

Thanks,

André 

[Ian Lance Taylor]

The case where you have trouble is code that uses an unexported type
set as bool, functions that take parameters of that type that are
invoked by the go statement, and arguments passed as comparisons. I
would be mildly surprised if any such code exists anywhere. And even
then you could still make it work by using temporaries for the
operands of the comparison, and keeping the comparison unchanged in
the actual call to the function.

> Switching gears a bit, I guess this might be easier to implement as a modification to the compiler? Or can you think of another way of making the general case more tractable?

This could be done by modifying the compiler but it's unlikely that
the compiler change would be accepted in the master sources.

Ian