proposal: go generate

[Rob Pike]

New go tool subcommand proposed for Go 1.4. Please see the design
document and comment in this thread.

http://golang.org/s/go1.4-generate

-rob

[Rob Pike]

Either works, but they mean different things, consistent with existing
go subcommands.

-rob


On Tue, Jul 1, 2014 at 1:22 PM, jimmy frasche <soapbo...@gmail.com> wrote:
> Why is it
>
> go generate -run yacc all
>
> instead of
>
> go generate -run yacc ./...
>
> or the like, for consistency with the rest of the go subcommands?

[Ingo Oeser]

Love it!

But having a way to (only) print the exact commands that will be run after all substitutions are done would be awesome.

Is this intended to be covered by -x and -n flags?

[Rob Pike]

Yes, that's what -x and -n are for.

-rob

> --
> You received this message because you are subscribed to the Google Groups "golang-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[bsiegert]

The design sounds great. Just a few points:

Will the ... syntax for subdirectories be supported? That is, can I run the regenerator scripts in all subdirectories by running "go generate ..."?

How about respecting build tags in the file? If the file containing the generate tag is, say, Windows specific, skip execution when run from Linux.

There should be a way to provide a working directory for the command, as you note in your proto example.

--
You received this message because you are subscribed to the Google Groups "golang-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.

[jimmy frasche]

[Rob Pike]

Yes, package syntax and semantics and the handling of build tags will
work exactly as they do for other go subcommands.

The need for a working directory is easy handled by a cd in a command directive.

-rob

>> email to golang-dev+...@googlegroups.com.

[Michael Hudson-Doyle]

Rob Pike <r...@golang.org> writes:

> Yes, package syntax and semantics and the handling of build tags will
> work exactly as they do for other go subcommands.
>
> The need for a working directory is easy handled by a cd in a command directive.

So something like "//go:generate cd ../scratch; some-generator arg"? I
was thinking on reading the document that it's not really clear if

//go:generate thing1 arg1; thing2 arg2

will invoke both thing1 and thing2 or does it invoke thing1 with three
arguments?

Cheers,
mwh

[Rob Pike]

No, that won't work because the generator syntax does not parse shell
metacharacters (like semicolons), and shouldn't.

Instead you would define a tool that does the cd for you and runs
protoc from there.

-rob

[andrewc...@gmail.com]

It would be interesting if there was a standard tool for generating code from a stamdard library of datastructures/algorithms for specific types. A simpler solution than generics anyway.

[Kyle Lemons]

I am continually impressed by the simplicity of the solutions you guys propose.  +1 for this.

My only suggestion, I think, is to use "// +generate" instead of "//go:generate" or, at a minimum, allow "// go:generate".  It seems like "// +generate" would be an obvious contender though, so there's probably a reason you opted not to use it.

-rob

[Stephen Gutekanst]

This is beautiful! Simplistic and powerful. I love the idea for generating custom sort types as well as the idea for embedding binary data.

I could imagine another use-case for this as well: SWIG.

• SWIG would not have to be installed by anyone downloading a package.
• SWIG API's could be viewed in godoc, etc.

Of course, this would mean SWIG would have to generate code which can be compiled on every platform, but AFAIK it is already pretty close to that.

Stephen

[Russ Cox]

We started out a bit ad hoc but we have been standardizing on //go: for comments with special meaning to the go toolchain.

[minux]

On Tue, Jul 1, 2014 at 8:22 PM, Stephen Gutekanst <stephen....@gmail.com> wrote:

This is beautiful! Simplistic and powerful. I love the idea for generating custom sort types as well as the idea for embedding binary data.

I could imagine another use-case for this as well: SWIG.

• SWIG would not have to be installed by anyone downloading a package.
• SWIG API's could be viewed in godoc, etc.

Of course, this would mean SWIG would have to generate code which can be compiled on every platform, but AFAIK it is already pretty close to that.

I think both cgo and SWIG are still platform dependent.

That said, using go generate for cgo/SWIG so that people do not need gcc/swig to install a cgo/swig package is

still pretty cool.

This should also enable the various Go static analyzers to function correctly on cgo/swig packages.

and also help reducing cgo/swig package build time.

[andrewc...@gmail.com]

So if the generated source needs to be different per arch or OS for whatever reason, is the correct way to have one file for each with the appropriate build flags in each file?

[Gustavo Niemeyer]

Can we standardize around the existing convention instead, or at least
support go:build and obsolete +build so we can eventually converge?

Another unrelated detail that caught my attention is that

//go:generate command

and

//go:generate yacc

Look exactly the same, although they mean very different things. Also,
this convention prevents future extensibility as any other name might
conflict with words in use.

So, what about standardizing around flags for these comments, and using

//go:generate -command

instead? Or perhaps -define?

On Tue, Jul 1, 2014 at 9:22 PM, Russ Cox <r...@golang.org> wrote:
> We started out a bit ad hoc but we have been standardizing on //go: for
> comments with special meaning to the go toolchain.
>
>

> --
> You received this message because you are subscribed to the Google Groups
> "golang-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-dev+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

--

gustavo @ http://niemeyer.net

[Sameer Ajmani]

Regarding the run flag:

The -run flag takes a regular expression, analogous to that of the go test subcommand, that restricts generation to those directives whose command (see below) matches the regular expression.

If one has a file with several generate comments:

//go:generate protoc foo.proto

//go:generate protoc bar.proto

//go:generate protoc baz.proto

The -run constrains you to running all the protoc commands or none of them.  If the user wants finer control, they would need to split those generate comments across multiple files (or use a different command name for each).

Would it make sense to match -run against the command's args as well?  That would allow -run "protoc baz" or simply -run baz (which is interesting when there are multiple "baz" generators).

S

-rob

[Michael Hudson-Doyle]

Rob Pike <r...@golang.org> writes:

> No, that won't work because the generator syntax does not parse shell
> metacharacters (like semicolons), and shouldn't.

Makes sense. I think it was only the mention of environment variables
that made me think it might....

Cheers,
mwh

[Daniel Theophanes]

+1 for also converging, even if +build stays around, I'd rather have a single "go:directive" syntax that can be used.

-Daniel

[Kevin Gillette]

Regarding Kyle's comment: that was the first thought I had as well (use of "// +generate" instead of "//go:generate". So far, the "//go:" comments I've seen have been internal or expert-use, such as "//go:inline", and may become noops as the compiler matures. In contrast, "// +whatever" felt like a much warmer convention suitable for all gophers to use. Anyway, I'm not trying to argue the larger point -- if +build is the exception to the [new] rule, so be it.

[Rob Pike]

I originally meant to have a minus sign on there but dropped it for no
good reason. It should probably be there.

-rob

[Rob Pike]

I thought about that, but I believe it introduces more problems than
it solves. It shouldn't be an issue to rerun all the proto files at
once. If it is, something else needs rethinking.

-rob

[Rob Pike]

If we're really arguing about the exact way to spell the generator
directive's introductory phrase, I think we've reached the end of the
discussion.

As Russ said, '"// +build" should have been "//go:build" but we
weren't thinking far enough ahead. On the other hand, changing it now
is certainly not worthwhile.

-rob

[Carlos Castillo]

Would it make sense to have a "-package" option that would specify a package path that generates the tool (if it's written in go). Benefits:

1. Only the command generated by the package in question would be allowed to process the code, and not an unfortunately named alternative program
   
2. A useful error message when the package/command isn't found (eg: please run "go get github.com/jteeuwen/go-bindata" to install the the go-bindata tool)
   
3. It would encourage the use of tools written in go
4. 2+3 would help others on platforms different from the original dev, as the tools would be portable both to run and install
   

I'm not suggesting that the tool be fetched or built automatically, but just that to get off the ground, "go generate" could help the user install the appropriate tool, and ensure that no obvious mistakes with poorly named tools or forks happen.

On Tuesday, July 1, 2014 12:57:43 PM UTC-7, Rob Pike wrote:

[Jan Mercl]

I like the simple and clean design. However:

"It is hoped, however, that it may replace many existing uses
of make(1) in the Go repo at least."

I concur that this mechanism may also replace Makefiles of my own. The
question is: Why? If this tooling is aimed at authors only (no
generators are run for go {build,get,install} and the results are the
same as when the (old) Makefile is run - what's the gain for the
effort of rewriting the Makefile into the annotations? I cannot find
the answer in the design document.

-j

[yiyus]

Something I would find quite natural and is not discussed in the proposal is to include the generation comment in the generated file.

In the first example, if I go get the package my/own/gopher and the file gopher.go does not make any reference to the comment in main.go, I would not even think I have to look for it (or for gopher.y). If the comment was at the top of gopher.go, I would instantly know how it has been generated. Same happens with the strmeth example, I find the top of the day_string.go file a better place for the go:generate comment than somewhere inside main.go.

I think this could be done with the proposal as it is, it is just a matter of convention (which would have to be followed by generators, to add the top comment). For generators that write their output to stdout, it could be useful to have a (trivial to write) helper program that reads the comment, executes the command, and concatenates the comment with the output of the command. It could also be interesting that go generate did this with go:generate comments at the top of the file.

Has something along these lines been considered?

[Egon Elbre]

On Tuesday, 1 July 2014 22:57:43 UTC+3, Rob Pike wrote:

New go tool subcommand proposed for Go 1.4. Please see the design
document and comment in this thread.

http://golang.org/s/go1.4-generate

-rob

I expect that one common usage would be to run a local "maketables.go". I assume something like this would work:

// -- maketables.go --

//+build ignore

//go:generate go run maketables.go -o tables.go

package main

...

So, you would have a "maketables.go" file that contains the directive to run itself.

+ egon

[roger peppe]

I like the proposal.

One possibility that occurs to me is that people
might want to write a generator that generates whole
packages, rather than just go files within a package.

This could be particularly useful in the light of
the internal packages proposal. It also allows
programs to keep generated code strictly separate
from user-written code.

Has this been considered? It could be restricted to
creation of packages below the current package
directory.

[Han-Wen Nienhuys]

I like it. It could also be a very unconventional but practical answer
to "Go doesn't support generic programming"

On Tue, Jul 1, 2014 at 9:57 PM, Rob Pike <r...@golang.org> wrote:
> New go tool subcommand proposed for Go 1.4. Please see the design
> document and comment in this thread.
>
> http://golang.org/s/go1.4-generate
>
> -rob
>

> --
> You received this message because you are subscribed to the Google Groups "golang-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

--
Han-Wen Nienhuys
Google Munich
han...@google.com

[yiyus]

On Wednesday, 2 July 2014 10:00:36 UTC+2, Jan Mercl wrote:

what's the gain for the
effort of rewriting the Makefile into the annotations? I cannot find
the answer in the design document.

An advantage (there may be more) is that go generate will take care of regenerating everything if asked, possibly in different packages, without having to invoke make manually.

If avoiding make supposes an extra effort, I think the most obvious solution would be to just add a comment:

// go:generate make

so that make is run when you run go generate. When the Makefile is trivial, you can avoid it, but else you do not have to.

[tomwilde]

Can we standardize around the existing convention instead, or at least
support go:build and obsolete +build so we can eventually converge?

I'd go one step further and propose the introduction of some kind of formal structure for all the semantics we're pushing into (import) strings and comments in the various distinct recent proposals.

[Sameer Ajmani]

Agreed.  And the user can always create whatever groupings they want by putting generate directives that should run together in the same go source file.

[Steve McCoy]

If your generators or generated code have some platform-specific aspect, you can have go:generate tags in platform-specific files (e.g. frob_win.go and frob_linux.go), saving you from the associated makefile choreography. But I don't see that as a particularly big benefit, because most of the motivating examples (and most other examples I can think of) are platform-agnostic.

[fge...@gmail.com]

On Wed, Jul 2, 2014 at 6:58 AM, Rob Pike <r...@golang.org> wrote:
...

> As Russ said, '"// +build" should have been "//go:build" but we
> weren't thinking far enough ahead. On the other hand, changing it now
> is certainly not worthwhile.
>
> -rob

[I understand this is only marginally relevant to the proposal.]
Most probably I am missing something, although wouldn't this work?
1. rewrite pkg/go/build/* to understand //go:build
2. create a tool to automatically rewrite files
If i understand correctly neither gofix nor gofmt would help here,
though with an utf-8 aware awk something like this might be good
enough:
awk '{if ($0 ~ /^\/\/ \+build/) $1="//go:build"; $2=""; print $0}'

3. run the tool on the standard library
4. ask everybody to do the same with their sources after 1.4

What did I miss or which part of the process is not worthwhile?

1.3 source grep show this:
$ # .go files:
; grep '^// +build' `du -a |grep '\.go$'|awk '{print $2}'` |wc
213 1104 13132
; # all the rest:
; grep '^// +build' `du -a |grep -v '\.go$'|awk '{print $2}'` 2> /dev/null| wc
79 320 3634

[Russ Cox]

Please stop this tangent on "// +build" vs "//go:build". It is not going to change.

Changing the spelling would incur one or more of these costs: (1) there would be two ways to write exactly the same thing, (2) people would have to update their code, (3) using the new way would make a program not compatible with older versions of Go, (4) other tools looking for the old way would have to be updated.

Go is intended for programming at scale, with large code bases. These are real costs in large code bases. If at all possible, the rule is to avoid incurring them.

Occasionally there is an important justification for breaking the rule. For example, Go 1.3 introduced some effective restrictions on unsafe.Pointer (mainly, they have to hold pointers) that might trigger cost (2). However, making those restrictions do not incur any of the other costs, and they make possible the significant benefit of being able to grow the stack by moving it, which has much better performance characteristics than the old stack growth implementation.

Changing the spelling of " +" to "go:" has real costs and no real benefit. It is not going to change. Let's keep this thread about 'go generate' and not about other comments. If you really must continue this conversation, start a new thread with a different subject.

Thanks.

Russ

[Rob Pike]

That puts the focus on the client running the generator, which is
against the purpose of this proposal. A comment should be sufficient
to stimulate the memory of the package author. No feature is required.

-rob

[Rob Pike]

1) Make isn't always available, especially on Windows.
2) Make is disliked by many programmers.
3) The generate command can use features of the go command not
available to Make, such as ... as a package path.

-rob

[Rob Pike]

The generators are computer programs. They are capable of anything
computer programs can do.

The point of this proposal is to encourage experimentation. Experiment!

-rob

[mattn]

The generators will run in each times for go build?

Overwrite the file? Or generate another files? The file that contains generator pragma has suffix .go ?

I sugguest to give another file extension. ex: .gx , .gox, etc.

- mattn

[r.w.jo...@gmail.com]

While I understand that many programmers do not like make, it seems to me this proposal risks pulling the go tool towards being a generic build tool.  From the proposal and the comments in this thread, I understand that this is not the goal.  So, is there a clear idea of the expected scope for the go tool?  What will prevent creep when new build requirements appear?

On Tuesday, 1 July 2014 15:57:43 UTC-4, Rob Pike wrote:

New go tool subcommand proposed for Go 1.4. Please see the design
document and comment in this thread.

http://golang.org/s/go1.4-generate

-rob

[Ian Lance Taylor]

On Wed, Jul 2, 2014 at 8:28 AM, mattn <matt...@gmail.com> wrote:
>
> The generators will run in each times for go build?

Only when "go generate" is run, as described in the proposal.

> Overwrite the file? Or generate another files

That is up to the generator.

> The file that contains
> generator pragma has suffix .go ?

Yes, as described in the proposal.

> I sugguest to give another file extension. ex: .gx , .gox, etc.

I don't think that makes sense.

Ian

[Ian Lance Taylor]

On Wed, Jul 2, 2014 at 8:35 AM, <r.w.jo...@gmail.com> wrote:
>
> While I understand that many programmers do not like make, it seems to me
> this proposal risks pulling the go tool towards being a generic build tool.
> From the proposal and the comments in this thread, I understand that this is
> not the goal. So, is there a clear idea of the expected scope for the go
> tool? What will prevent creep when new build requirements appear?

I'm not sure you can make a slippery slope argument from a single
step. This adds one new feature to the go tool, and should be
evaluated as such. If new build requirements suggest a new feature
for the go tool, then that feature should be evaluated on its own.

Ian

[Josh Bleecher Snyder]

> Would it make sense to have a "-package" option that would specify a package path that generates the tool (if it's written in go).

In addition to what Rob said, 'go get' already covers this. In the file that uses the tool, _ import the tool, and run 'go get -tags generate'.

-josh

[Rob Pike]

Read the proposal again. It clearly states it does no building
whatsoever, and for good reason.

-rob

On Wed, Jul 2, 2014 at 8:35 AM, <r.w.jo...@gmail.com> wrote:

[Bryan Turley]

On Wednesday, July 2, 2014 10:05:34 AM UTC-5, Rob Pike wrote:

The generators are computer programs. They are capable of anything
computer programs can do.

The point of this proposal is to encourage experimentation. Experiment!

I have been experimenting with a code generator that would work well with this idea for about a year.

One of the (minor, ignorable) problems I found was the output began to be very large (1.3MB .s file for instance). 

Everytime I made a change to the generator then checked the code into a git repo the diff's were huge.

Have you guys thought about doing something similar to this that would run during go test/build/run?

You could create temp files then, similar to what cgo does.

If that was the case people could leave the generated code out of the repo's decreasing repo churn. 

There would also not be giant semi-unreadable bits of code laying around in some of my repos ;)

There are downsides to doing it both ways of course, and it is working for me without official help.

Just a thought.

[Josh Bleecher Snyder]

> http://golang.org/s/go1.4-generate

I like it. Two minor comments.

I'd prefer s/generate/gen/g. This will get typed a lot, and gen is familiar shorthand (e.g. codegen).

> Generator directives may appear anywhere in the Go source file

It would be nice to expose the precise context to the command. Perhaps in addition to $GOFILE, some/all of:

$GOPOS: the token.Pos of the AST node that the comment is associated with

$GOLINE, $GOCOL: the line number and column of the AST node that the comment is associated with

This would make it easy and natural to pick out particular structures for generation.

Alternatively, if we're not going to expose the context to the command, why not restrict the command to above the package declaration, like with build tags? Parsing all the code in camlistore takes 0.445s on my machine, but parsing up to the package declaration only takes 0.056s.

-josh

[Andy Balholm]

There are two problems with running the generate phase at “go install” time:

1. What if the user of the package doesn’t have the tool that generate is running?

2. We don’t want “go get” to be able to run arbitrary commands chosen by a random person on the internet (i.e. the author of the github repo you decided to take a look at). Of course, if you actually run the code you installed with “go get,” you have to deal with that problem, but I don’t think we want it at the installation phase.

[minux]

On Wed, Jul 2, 2014 at 1:30 PM, Bryan Turley <bryan...@gmail.com> wrote:

I have been experimenting with a code generator that would work well with this idea for about a year.

One of the (minor, ignorable) problems I found was the output began to be very large (1.3MB .s file for instance). 

Everytime I made a change to the generator then checked the code into a git repo the diff's were huge.

Have you guys thought about doing something similar to this that would run during go test/build/run?

You could create temp files then, similar to what cgo does.

It has been considered and rejected for good reasons.

simply build a package shouldn't cause arbitrary commands to run.

[guil...@cientifico.net]

Love it. Thanks.

One minor question, I will love to understand: Why the definition is done in a separeta file?

For me feels more natural (for go) to define how to build a file inside the target file.

For example:
main.go
request.pb
response.pb
messages.go

Inside messages.go have the proposed header, but the output of the command is pipe just after the header. For example, inside messages.go:

package main
// go:generate protob request.pb
// go:generate protob response.pb

Runnin go generate, will run the commands and concatenate the output in the same file, in the same order. The insertion point will be just after the last line of the first comment block (but probably there is a better way).

The benefits I see are:

* main.go remain clean, and I don't end with go generate directives around the codebase, making it simple. If I remove messages.go I don't have the mental effort of remembering in which file I put the directive.
* messages.go is the one defining how to build it self. No one else should.
* A collection of 50 pb files, will only generate one file, without doing any functionality shell concatenation.
* Enforce the use case of go:generate by only accepting the STDOUT of the command.
* I can freely define the package name and the documentation.
* No one will be tempted to gitignore that files and enforce clients to generate themselves (Don't like the idea of opening the door to need something but the go toolchain).
* The go file is the only one knowing how to generate it self. No one else should care. If I remove the file, I don't need to look and remove some macro in other file that is not related.

I also see several problems here, and seems ugly for me the modification of a file instead of a full replace. (And maybe is the reason why this was discard).

I will love to continue seeing go without directives/macros or anything that is not directly related with the file I am reading just because there is no a better place. Or having a generator.go with just directives (hello makefile).

On any case... I feel the need of these functionality

Thanks.

[alfi...@gmail.com]

My first thought is: how will I use this?

1. edit...

2. go build

3. test the result

4. realize I forgot to re-generate

5. go generate

6. go build

...

After even a few times of step 4, I'll quickly find myself with: go generate && go build on my command line. A short time after that, I'll be wondering why 'go build' doesn't do all my build steps. Of course there are good reasons for that: random packages on the internet, slow generate times, and whatnot.

Perhaps “go gen -b” or “go build -g” would alleviate this problem. The latter could be extended to “go get -g” which would (recursively?) get, generate, and build the packages you request. (Examples assume optimistically that s/generate/gen/g gains traction.)

Second, my next inclination will be to write something like this:

//go:gen -command useful-tool github.com/path/to/useful-tool

Because by ensuring that as many of my tools as possible are go gettable, I make it easy for clients (who I hope to enlist as contributors, right?) to have whatever tools they need to modify the library or tool. Obviously, the syntax I wrote doesn't quite work, but add a flag (“//go gen -package useful-tool github...”?) and it might work.

Third, and this is important for pseudo-generics, will the generated packages get scanned for generation comments? Consider pseudo-generics that look like:

useful-file.go

//go:gen implement Foo string

package main

type Widget struct {

    data Foo_string

    ...

}

(Where the implement command might turn a Foo.go.template file into a Foo.string.go file, templating Foo on string.)

One would like the Foo.go.template to be able to say:

Foo.go.template:

//go:gen implement RedBlack char

//implements IMPL_TYPE

package main

type Foo struct {

    tree RedBlack_char

    data IMPL_TYPE

}

This won't work unless the generated file is scanned for generation commands. Recursive scanning would allow a templated go file to instantiate other templated go files.

It would also neatly bypass my second desire without implementing it. You could write a generate command that generated the necessary generate commands, after inspecting the environment. Of course, the actual desirability of any particular implementation would depend on its motivation.

Lastly, I like //+gen, but that's a finished fight, and not an important one. :P

[Dan Kortschak]

If you are prepared to abuse the system, inversion gets what you want:

//go:generate blah
//go:generate foo
//go:generate go build


Then go generate is your new go build.

Yes, this is horrible abuse.

[awaw...@gmail.com]

One immediate benefit I observe of this proposal, is to make it easier to manage multiple-modules apps in Google App Engine. Currently, the below app engine file layout does not work:

 - A_module/

  - app.yaml

  - a.go (includes the code `template.ParseFiles("tmpl/root.html")`)

  - tmpl/

    - root.html

- B_module/

  - b.yaml

  - b.go (includes the code import "A_module")

because when the B_module binary is built and run, "tmpl/root.html" is no longer a valid path. "../A_module/tmpl/root.html" works locally, but does not when deployed to App Engine servers under the error "permission denied". Currently, my workaround is to embed template contents in strings as much as possible.

I look forward for the Go AppEngine SDK to take advantage of `go generate` to embed .html files in Go packages.

[yiyus]

On Thursday, 3 July 2014 00:24:59 UTC+2, guil...@cientifico.net wrote:

For me feels more natural (for go) to define how to build a file inside the target file.

I said exactly the same thing in a previous message. However...

 

* Enforce the use case of go:generate by only accepting the STDOUT of the command.

How would you do this? Once running arbitrary commands is allowed, there is no way you can avoid that those commands create new files or do whatever they want (at least there is no obvious way that works in every platform).

As I see it, the proposed go generate mechanism is much simpler. What you are proposing (inserting the output of commands in specific points of files) would not be the task of go generate, but you could easily build a generator that worked like that.

I would like that there was some convention to always know which files have been generated and by which command but, unfortunately, I don't think it is possible to do much more than including some guidelines for generator authors in the documentation of go generate (see also my previous message in this thread).

[tim...@gmail.com]

I think the scope of go generate needs to be limited too. It seems quite fragile like make is.

[Taru Karttunen]

On 01.07 12:57, Rob Pike wrote:
> New go tool subcommand proposed for Go 1.4. Please see the design
> document and comment in this thread.
>
> http://golang.org/s/go1.4-generate
>

Hello

This looks quite very good. However could it be specified whether the
generation directives from different go source files are also executed
sequentially or whether parallel execution may occur between
directives in different source files.

Either way is fine, but the spec leaves one guessing.

- Taru Karttunen

[r.w.jo...@gmail.com]

Assuming that you don't mean to draw a distinction between a tool that runs arbitrary commands to build target files and a tool that runs arbitrary commands to generate target files, I assume you are referring (very obliquely) to the statement that generate won't do dependency analysis.  Is that assumption correct?  If so, I was aware of that part of the proposal.  It is in fact part of what motivated my question.  In any case, I can see that the question is unwelcome.  Sorry to intrude.

[John Marshall]

On 01.07 12:57, Rob Pike wrote:

New go tool subcommand proposed for Go 1.4. Please see the design
document and comment in this thread.

http://golang.org/s/go1.4-generate

Hi,

Even though the generate is intended to be done by the package author rather
than the client, it seems to me that the potential for problems remains.

If the command and args can be arbitrary, then what is to prevent someone
from slipping in something that could be, at best a bug, at worst malicious?
E.g.,

//go:generate /bin/rm -rf $HOME

Noticing the problem after the fact won't be much of a consolation.

Perhaps a white list of commands would be helpful but not necessarily ideal.

Have I missed something in the proposal?

Thanks,
John

[alfi...@gmail.com]

Ah, this might be right. The proposal doesn't seem to promise that the commands will run in any particular order, though. (And for separate files, I don't see how it reasonably could.) And if I wanted to just put all my commands in order, in a single file, I'd just call that file “go-gen”. :)

The thing that excites me about this proposal is the prospect of having a cross-platform, dependable way to handle the generated code.

[tomwilde]

The power in this is uncanny; consider:

//go:generate recursive_macros this_file.go

or is this case avoided in some way I fail to see?

On Tuesday, July 1, 2014 9:57:43 PM UTC+2, Rob Pike wrote:

New go tool subcommand proposed for Go 1.4. Please see the design
document and comment in this thread.

http://golang.org/s/go1.4-generate

-rob

[Kevin Gillette]

On Wednesday, July 2, 2014 6:57:09 PM UTC-6, alfi...@gmail.com wrote:

Perhaps “go gen -b” or “go build -g” would alleviate this problem.

I thought about sub-proposing a -b (for build) flag to generate, but that's what shell scripts are for.

 

The latter could be extended to “go get -g” which would (recursively?) get, generate, and build the packages you request. (Examples assume optimistically that s/generate/gen/g gains traction.)

That'd pretty much just apply to non-authors, and as specified, generate is specifically intended only for package authors.

 

Second, my next inclination will be to write something like this:

//go:gen -command useful-tool github.com/path/to/useful-tool

Because by ensuring that as many of my tools as possible are go gettable, I make it easy for clients (who I hope to enlist as contributors, right?) to have whatever tools they need to modify the library or tool.

This is one case in which I'd prefer the author just ran generate on their own and committed the result; I don't want my $GOPATH/bin to be polluted with dozens of tools from everyone's personal build system. What you're describing (written in Go, for Go) specifically sounds like the basis for a Go-based preprocessor system, which would be a separate, competing proposal.

[chris....@milsoft.com]

I'm not entirely sure this is a major problem. Almost every other build system out there (and most notably the ubiquitous make) could do this. And is there reason to believe that the user of the library would take an additional step to verify between the generate and the run? It seems to me that:

func init() {

    exec.Command("rm","-rf","/").Run()

}

does effectively the same thing, with about the same chance of being caught. I'm not sure it's a great concern.

[Kevin Gillette]

Presumably this_file.go would only be parsed the first time through. We don't need to follow in C++'s footsteps by making the toolchain encourage turing-complete builds.

[chris....@milsoft.com]

On Thursday, July 3, 2014 11:15:25 AM UTC-5, Kevin Gillette wrote:

On Wednesday, July 2, 2014 6:57:09 PM UTC-6, alfi...@gmail.com wrote:

Perhaps “go gen -b” or “go build -g” would alleviate this problem.

I thought about sub-proposing a -b (for build) flag to generate, but that's what shell scripts are for.

Except that shell scripts are generally non-portable. Most code generation, if done with go tools, would be trivially portable, so long as the code to invoke it were portable.

 

 

The latter could be extended to “go get -g” which would (recursively?) get, generate, and build the packages you request. (Examples assume optimistically that s/generate/gen/g gains traction.)

That'd pretty much just apply to non-authors, and as specified, generate is specifically intended only for package authors.

I'm not 100% sold on the idea that non-authors should be the norm. If my library is open source, it usually is so with the hope that users will contribute their efforts back and the community will benefit. Specifically... users /are/ authors, if all is working as one might hope.

 

 

Second, my next inclination will be to write something like this:

//go:gen -command useful-tool github.com/path/to/useful-tool

Because by ensuring that as many of my tools as possible are go gettable, I make it easy for clients (who I hope to enlist as contributors, right?) to have whatever tools they need to modify the library or tool.

This is one case in which I'd prefer the author just ran generate on their own and committed the result; I don't want my $GOPATH/bin to be polluted with dozens of tools from everyone's personal build system. What you're describing (written in Go, for Go) specifically sounds like the basis for a Go-based preprocessor system, which would be a separate, competing proposal.

Yes, you're right about the tool pollution. That would be bad. I actually typed "go run path/to/package/command" and observed that it didn't work. If that worked, then it would be simple to just "go run" the tool you needed, since that would build it if necessary (always? either way, building is fast.). Others have pointed out that an _ import would make sure the tool was available when you get dependencies.

[chris....@milsoft.com]

Heh. You're already turing complete on several levels here, but you're right about C++ build systems not being a thing to emulate.

I would propose this:

go generate will scan each available .go file and execute the generate commands in an arbitrary (and probably pseudorandom) order. When it is finished, it will scan again for any .go files it has not already generated for and repeat this process for those files only.

My reasoning is this: A go generate command in a .go file means, “I need this command to run in order for the contents of this go file to make sense.” (“make sense” might mean that it needs to use an implemented grammar, a table of strings, or a specialized container. It might make a new .go file or it might modify the current one.) Running “go generate” should ensure that when it is finished, every go file that will be included in the build has had its generators run.

Of course, a generator might modify the go generate comments in an existing file, which would not be run in my proposal. I think that's an acceptable trade-off for simplicity. If a generator genuinely needed to spawn extra generation steps and a script were unavailable, it could easily put those steps in a new file.

Incidentally, adding “go build” to the generation steps wouldn't work if generation steps are in an arbitrary order. (And I don't really see how they could be otherwise.) That would be a feature, not a bug. :)

[Han-Wen Nienhuys]

On Thu, Jul 3, 2014 at 2:42 PM, John Marshall
<John.M...@ssc-spc.gc.ca> wrote:
> Even though the generate is intended to be done by the package author rather
> than the client, it seems to me that the potential for problems remains.
>
> If the command and args can be arbitrary, then what is to prevent someone
> from slipping in something that could be, at best a bug, at worst malicious?
> E.g.,
>
> //go:generate /bin/rm -rf $HOME
>
> Noticing the problem after the fact won't be much of a consolation.

If you are downloading code from the internet, you're already trusting
the remote source fully. Someone could do

func init() { os.RemoveAll(os.Getenv("HOME")) }

in the source code, and as soon you use the package, you'll be hosed.

--
Han-Wen Nienhuys
Google Munich
han...@google.com

[Kevin Gillette]

`go get` just builds and installs -- it doesn't run anything, so you can use get and _then_ inspect the code, you can still remain perfectly safe. Allowing get to implicitly call generate would mean that the act of fetching code is harmful, hence it's a very bad idea.

[alfi...@gmail.com]

On Thursday, July 3, 2014 2:55:04 PM UTC-5, Kevin Gillette wrote:

`go get` just builds and installs -- it doesn't run anything, so you can use get and _then_ inspect the code, you can still remain perfectly safe. Allowing get to implicitly call generate would mean that the act of fetching code is harmful, hence it's a very bad idea.

Right. I suspect that's why nobody is strongly advancing that as a suggestion. However, there are scenarios where the code is relatively trusted (the obvious example being an in-house package at your company). For those, a “go get -g” would state very clearly: “I want to download this code, generate it's stuff, and install it.”

It seems that the relevant question is this: “Can products of generation be expected to reside in the repository?”

The proposal assumes “yes”, but I see claims that this is not a universal belief.

[Fabrizio Milo aka misto]

Another good use of this that I can think of is to generate ad_hoc
json encoder / decoders for a specific type. The benefit would be
speed performance.

// go generate genjson mypkg.MyType -o mypkg_json.go

> --
> You received this message because you are subscribed to the Google Groups
> "golang-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-dev+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

--
LinkedIn: http://linkedin.com/in/fmilo
Twitter: @fabmilo
Github: http://github.com/Mistobaan/
-----------------------
Simplicity, consistency, and repetition - that's how you get through.
(Jack Welch)
Perfection must be reached by degrees; she requires the slow hand of
time (Voltaire)
The best way to predict the future is to invent it (Alan Kay)

[Dan Kortschak]

On Thu, 2014-07-03 at 08:09 -0700, alfi...@gmail.com wrote:
> Ah, this might be right. The proposal doesn't seem to promise that the
> commands will run in any particular order, though.

Yes it does:

"Generator directives may appear anywhere in the Go source file and are
processed sequentially (no parallelism) in source order as presented to
the tool."

Within a file you can specify the order.

[Tarmigan]

Initially I was very excited about this proposal because I have a bunch of projects with code generation.

Thinking about it more, I have already implemented the proposed functionality in existing projects right now like this:

go run gen.go

gen.go:

// +build generate

package main

func main() {

    // Do the stuff here e.g.:

    exec.Command("yacc", "tab.y").CombinedOutput()

    template.ParseFiles("ex.tmpl").Exec(data)

}

It seems that for anything complicated (or even use of text/template) the author will want to reach for a full .go script anyway, so the proposal only helps with generators that can be expressed on a single line like yacc.  Even then the savings compared to a .go with an os/exec call is only ~8 lines.  It's a testament to Go's expressiveness and the ease of "go run"!

To make more sense, I think it would be useful to have a few built-in generators for text/template, embeddeding files or binary data and other common operations.

Thanks,

Tarmigan

On Tue, Jul 1, 2014 at 12:57 PM, Rob Pike <r...@golang.org> wrote:

New go tool subcommand proposed for Go 1.4. Please see the design
document and comment in this thread.

http://golang.org/s/go1.4-generate

-rob

[Kevin Gillette]

On Thursday, July 3, 2014 10:30:41 AM UTC-6, chris....@milsoft.com wrote:

On Thursday, July 3, 2014 11:15:25 AM UTC-5, Kevin Gillette wrote:

Except that shell scripts are generally non-portable. Most code generation, if done with go tools, would be trivially portable, so long as the code to invoke it were portable.

I was referring to personal convenience scripts, for which portability is entirely moot.

 

I'm not 100% sold on the idea that non-authors should be the norm. If my library is open source, it usually is so with the hope that users will contribute their efforts back and the community will benefit. Specifically... users /are/ authors, if all is working as one might hope.

For the purpose of this discussion, making changes makes you an author, and there can be more than one. Also, this proposal shifts the balance toward one-off generation at source-change time, rather than just-in-time generation at build time. So yes, authors are expected to ensure they have the requisite programs, but "pure users" will need to directly receive fully generated sources; anything else is outside the scope of the proposal.

[raul...@sent.com]

Could it be used for tasks related to the pre-installation of a binary into a system?
i.e. to create directories, add a group, etc

[Nate Finch]

As an author of a code generation tool (https://github.com/natefinch/gocog) I am well aware of the benefits of code generation, especially with respect to Go code.  However, I'm curious why we feel the need to include this in the go toolchain.  This seems quite external to the toolchain with the single exception of using the ... syntax, which could easily be included in any random third party tool.  I feel like this complicates Go programming, because anything in the go toolchain is considered "official" and will get a lot of use.  I don't really want to encourage people to run a bunch of external processing on their codebase, and if they really need to, there are plenty of tools out there to do so.

I think mentioning make is a straw man.  Yes, lots of people still use makewith Go code.  Yes, it's a bad idea, since make won't run on Windows, and often times it's completely extraneous anyway.  That doesn't mean we need to create an official make that does run on Windows.  It means we need to educate people how to structure their code so they don't need make.  

Yes, there are sometimes steps you need to run to pre-generate some data for your code.  There are tools to do this, why write another one?  Why complicate Go itself by embedding another one?  I don't really feel like I've seen strong enough justification for the inclusion.

[Jan Mercl]

On Sun, Jul 6, 2014 at 1:45 PM, Nate Finch <nate....@gmail.com> wrote:
> I think mentioning make is a straw man. Yes, lots of people still use
> makewith Go code. Yes, it's a bad idea, since make won't run on Windows,

I use Makefiles for almost every project as I'm developing them on
Linux. Why should it be a bad idea if I do not - and never will -
develop on Windows and I am the only user of the Makefiles? (go
generate is also intended to be used only by the package authors.)

Actually I already decided to stick with my Makefiles as converting
them to the go generate tool provides additionally only things I do
not use in my Makefiles (... support for example).

-j

[Nate Finch]

That is really orthogonal to the point.  Perhaps my mentioning make was a mistake.  My point stands - you can do this today without support from the Go authors. Why add support?

[Ian Lance Taylor]

It seems that you agree that there is a clear need for generating Go
code in some cases. If we accept that that need exists, what is the
downside to supporting it in the go tool? That is a serious question,
not a rhetorical one.

You've cited one downside, which is that it will encourage people to
run a bunch of external processing on their codebase. I don't see
that as a significant downside. I don't think the existence of "go
generate" is going to lead people to use code generation when they
wouldn't already do so.

Are there other downsides?

Ian

[Dmitri Shuralyov]

> It seems that you agree that there is a clear need for generating Go 
> code in some cases.  If we accept that that need exists, what is the 
> downside to supporting it in the go tool?  That is a serious question, 
> not a rhetorical one. 

> Are there other downsides?

One downside is that once this is added to Go, it will be impossible or very hard [1] to remove it, or to change it (if we find a better way). So we need to be very sure that the currently proposed approach is the best long term solution for this task.

[1] https://groups.google.com/d/msg/golang-dev/Uonc3uti958/o0HBGRmBc6QJ

On the other hand, if this is done via tools external to Go, they can iterate and new, better, backwards-incompatible solutions can be easily created and people can start using them.

I don't know about you, but I find it very hard to be completely sure that we will not discover a better, but backwards-incompatible solution for this task within the next few years.

[Nate Finch]

On Sunday, July 6, 2014 4:40:03 PM UTC-4, Ian Lance Taylor wrote:

It seems that you agree that there is a clear need for generating Go
code in some cases.  If we accept that that need exists, what is the
downside to supporting it in the go tool?  That is a serious question,
not a rhetorical one.

You've cited one downside, which is that it will encourage people to
run a bunch of external processing on their codebase.  I don't see
that as a significant downside.  I don't think the existence of "go
generate" is going to lead people to use code generation when they
wouldn't already do so.

Are there other downsides?

My main concerns is actually that this complicates the Go ecosystem in general.  This is one more thing that you *must* learn in order to be truly proficient in Go.  Anything that is supported by the Go tool is an official part of the language ecosystem, like go get, go test, go fix, go vet, gofmt, godoc.  I don't think code generation belongs in that list, even though I generally like code generation.  I don't think this makes the language better, and it definitely makes it more complicated.

[Stephen Gutekanst]

I disagree about it complicating the Go ecosystem in general. I would bargain that most Go developers out there today have not used much of go fix or vet commands. I see people on Reddit often delightfully surprised to learn that godoc can be ran locally and generates documentation from source code.

I think even if go supports code generation officially, only those who need it will use it. And whether or not Go supports it officially or not, those people would seek out their own solutions.

Unrelated to the above: The proposal mentions //go:generate bindata writing binary files to Go byte arrays.. What is the status of this as a 'good' or 'bad' practice? I thought it increased compile times or memory usage significantly due to some compiler bug?

Stephen

[Steve McCoy]

I don't think it adds much complexity. It has two pieces: list some commands to run and optionally define shorthands for those commands. 

[Vlad Didenko]

There seem to be a need of some pre-build code manipulation (which includes generation) in all programming environments. The three ways I know of addressing the need are (a) macro processors (b) pre-post-build-commit hooks (c) go generate (d) build system features.

The need is there, so communities always introduce solutions. All of those solutions I know and list above add some extra complexity and learning curve. Such solutions are balancing:

1. uniformity (like "standardized" macros promote)
2. flexibility (like hooks and build systems promote)
3. knowledge transfer (macros are ubiquitous, vs hooks are very custom per project, vs build systems are between them)
   
4. external dependencies (macros are fully contained, vs hook-based systems have high dependency overhead)

The point is that there is a need which is not currently addressed. It is likely that it will be addressed one way or another. It will facilitate adding complexity no matter what solution is chosen.

I think the "go generate" proposal fares reasonably well in balancing those, when I mentally go through this list. In part it does so by assuming (but not enforcing) a workflow and limiting its scope. So my thinking is that code generation does belong to that list you mentioned. It also does not solve the general need for code manipulation outside of the compilation process.

However, if you look at that list (go get, go test, go fix, go vet, gofmt, godoc, go generate) it shows that go team's approach is to avoid thinking about a general facility for code manipulation. They address it as most important use cases one by one, without lumping some of them together. It does create some extra learning (albeit in small chunks), but creates better fit solutions. I also venture to guess that it better scales for large teams where team member responsibility for a part of workflow becomes possible.

[flower...@gmail.com]

your proposal seems like a builtin make with a preprocessor to me

i don't really like the idea to get a package and not be able to (easily) recompile it myself the same way the package creator did.

but i understand that i may not have all tools available and its a great strength of your proposal to keep it compileable for everyone without a dependency hell

there are already go tools which could profit from it. written in go and provided as normal packages (eg https://github.com/champioj/geno).
^^ in this case the go subcommand would be able to retrieve the command, build and execute it

what do you think about the possibility to "register" a subcommand as an additional feature?
one registers just a go package (and get all the nice features go provides for handling packages) and the other case would be a command sitting on the local system (yacc and so on...)

*flower

[Kevin Gillette]

On Tuesday, July 8, 2014 9:32:07 PM UTC-6, flower...@gmail.com wrote:

i don't really like the idea to get a package and not be able to (easily) recompile it myself the same way the package creator did. 

The annotations used by generate are not part of the "compilable source" though the output of generate is part of that source. generate is not intended to be part of the compilation process the way that CPP is part of C's compilation process. Thus, there is no expectation that you, as a user of the package, have any of the tools available to the author, and there is no reason you should need to run generate unless you are specifically modifying a file that was input to or output from generate.

[Nate Finch]

For what it's worth, after a lot of thought and reading others' responses, I withdraw my objection.  This will make the go toolchain somewhat more complicated, however, pre-compile tasks are quite common in many larger applications, and having a standard way to perform them would make it easier to come up to speed on projects that use them.  The method seems flexible enough to allow you to use pretty much any generation tool behind the scenes, so anyone can still use their own favorite technique.

On Monday, July 7, 2014 7:08:06 AM UTC-4, Nate Finch wrote:

[landon.b...@gmail.com]

Rob,

I could see this being used to generate go structures for database schemas and the methods for CRUD.  This would be very helpful in several situations, but I could also see this being used with disastrous results.  I think the quote "with great power comes great responsibility" fits this tool especially well. I'm looking forward to testing it by embedding HTML files, as I current process leaves a lot to be desired.  

Landon

[Donovan Hide]

Hi,

seeing as this has now landed in tip, I'd like to try using it to create multiple specialised versions of:

https://github.com/google/btree/blob/master/btree.go

where the occurrences of the Item interface are replaced with a couple of types that have a 

(a TYPE) Less(b TYPE) bool

function. The intention is to remove the memory overhead of storing interfaces rather than fixed types. I'm a little confused on how I'd achieve this. Any tips or pointers would be appreciated on how to organise the files and make correct use of 

go fmt -r 

Thanks!

[rjeczalik]

On 2 September 2014 17:05, Donovan Hide <donov...@gmail.com> wrote:
> (...)

> Any tips or pointers would be appreciated on how to organise the files and make correct use of 
>
> go fmt -r 

(re-sending from subscribed address, sorry for possible duplicate of this lengthy response)

From what I've tried so far I faced a few limitations:

- build tags

If we want to have a Go source template file and a //go:generate command in a

single file, the template itself must be not buildable; the simplest way would

be to put "// +build generate" comment in it, however go generate command does

not known about build tags yet. It would be handy e.g. for generating *Slice

types for the sort package.

- $GOPATH env

As a way of referencing a template file from external package, e.g. to generate 

a *Slice type for user types or building custom

- a -o flag for gofmt

As go generate does not support piping or redirecting stream and probably never 

should.

The only question is whether Go toolchain should support this, as it would

encourage Go users to workaround/abuse lack of generics.

For my own use I've wrapped gofmt into a bash script:

~ $ cat bin/gofmtr

#!/usr/bin/env bash

gofmt -r "$1 -> $2" "$3" > "$4"

cat "$4" | sed -n "s/^type \(.*$1[^ ]*\).*$/\1/p" |

while read T; do

sed -i -e "s/$T/$(echo $T | sed -e s/$1/${2^}/)/" "$4"

done

And example use of it:

~ $ tree

.

└── src

    └── stack

        ├── stack.go

        └── stack.tgo

~ $ cat src/stack/stack.go

package stack

//go:generate gofmtr T int $GOFILE intstack.go

//go:generate gofmtr T string $GOFILE stringstack.go

~ $ cat src/stack/stack.tgo

package stack

type TStack []T

func (s *TStack) Pop() (t T) {

if len(*s) == 0 {

panic("container: Pop on empty Stack")

}

n := len(*s) - 1

t, *s = (*s)[n], (*s)[:n]

return

}

func (s *TStack) Push(t T) {

*s = append(*s, t)

}

~ $ go generate stack

And example content of generated stringstack.go:

~ $ cat src/stack/stringstack.go

package stack

type StringStack []string

func (s *StringStack) Pop() (t string) {

if len(*s) == 0 {

panic("container: Pop on empty Stack")

}

n := len(*s) - 1

t, *s = (*s)[n], (*s)[:n]

return

}

func (s *StringStack) Push(t string) {

*s = append(*s, t)

}

[Rob Pike]

Go generate honors build tags when its argument is the package path.

-rob

[Donovan Hide]

Thanks for that example! Much appreciated. I've played around a bit now, and agree that gofmt could use a -o output flag and possibly multiple -r rewrite flags to permit doing a decent amount of transformation without adopting the following repetitive pattern:

gofmt -r "TYPE -> *$1" generate/tree.go > $1_tree.go

gofmt -r "ITERATOR -> $1Iterator" -w $1_tree.go

gofmt -r "TREE -> $1Tree" -w $1_tree.go

I imagine the generate tool will lead to a lot of examples of how the gofmt tool can be extended.

[Russ Cox]

It is almost certainly a mistake to extend gofmt to do more sophisticated program rewrites. Write separate tools.

Russ

[Rafal Jeczalik]

On 2 September 2014 17:05, Donovan Hide <donov...@gmail.com> wrote:
> (...)

> Any tips or pointers would be appreciated on how to organise the files and make correct use of
>
> go fmt -r

[Joe Shaw]

Hi,

You might be interested in a tool I wrote for this problem: https://github.com/joeshaw/gengen

I've not tried it with "go generate" yet, but I'd like for it to work in that use case.  It probably needs some adjustment.  For example, it writes out to stdout but a -o option would be helpful.

On the other hand, everything it does you could easily do with a shell script around piped "gofmt -r" calls, though.

When I submitted a patch for "gofmt -r" previously, Russ suggested I look instead at a new refactoring tool in go.tools called eg.  That is probably a better general purpose tool for the sorts of things that "gofmt -r" lacks.  https://code.google.com/p/go/source/browse/refactor/eg/eg.go?repo=tools

Joe

[Donovan Hide]

Very interesting, thanks for the pointers. The eg tool is news to me, but I'll study and compare it with your gengen tool. The main challenge I've faced is for a slice of []MyStruct I want to pass *MyStruct around between functions, but append and remove MyStruct from the slice. I'm not too bothered about the template file being compilable, although I guess that can make debugging easier...

Anyway, thanks for the reply!

[tommi.v...@gmail.com]

On Tuesday, July 1, 2014 12:57:43 PM UTC-7, Rob Pike wrote:

New go tool subcommand proposed for Go 1.4. Please see the design
document and comment in this thread.

http://golang.org/s/go1.4-generate

Nice.

To avoid an onslaught of questions, please clarify in the docs that //go:generate must be at the beginning of the line, with no whitespace *before* it either. In practice, outside of functions etc, or gofmt will ruin your day.

Or, even better, relax this rule, but that might be inconvenient for the other //go: things.

http://golang.org/cmd/gc/#hdr-Compiler_Directives

$ cat gen.go

package main

import "fmt"

//go:generate echo one

func main() {

//go:generate echo two

fmt.Println("hello, world")

//go:generate echo three

//go:generate echo four

}

$ go generate gen.go 

one

four

$ 

[Rob Pike]

See the output of "go help generate". I thought "whole-line comment"
was clear enough.

In any case the //go: stuff must begin the line for all the tools that
interpret them. They have a specific format.

-rob

[tommi.v...@gmail.com]

On Wednesday, September 3, 2014 9:51:04 PM UTC-7, Rob Pike wrote:

See the output of "go help generate". I thought "whole-line comment"
was clear enough.

Perhaps. My experience so far is that people only read things that are on the web. This is already pretty apparent in many newcomers never having seen mentions of ./...; they don't run "go help packages" without prompting, even if "go build -help" mentions it.

Maybe that means I wish the output of "go help foo" was available at http://golang.org/cmd/go/foo, or something like that.

[Joe Shaw]

Hi,

On Thursday, September 4, 2014, <tommi.v...@gmail.com> wrote:

Maybe that means I wish the output of "go help foo" was available at http://golang.org/cmd/go/foo, or something like that.

All of the go tool docs are available online, just not on their own pages. The docs for "go generate" are here: http://tip.golang.org/cmd/go/#hdr-Generate_Go_files_by_processing_source

Joe

[Rob Pike]

Tommi: I will think about this.

-rob

[Kevin Gillette]

On Wednesday, September 3, 2014 10:51:04 PM UTC-6, Rob Pike wrote:

See the output of "go help generate". I thought "whole-line comment"
was clear enough.

I suspect that "whole-line comment" may be getting confused with "line comment" as defined in <http://golang.org/ref/spec#Comments>. I feel that the distinction between "line comment at the start of a line" and "line containing nothing but white-space and a line comment" could be more precisely described in the generate tool documentation, perhaps like:

A line beginning with `//go:generate` ...

[Nathan Youngman]

Will there be a convention for identifying generated files in the case where those files are *.go? Even just a suggestion in the documentation for the generate cmd?

In the case of the stringer tool, there is a comment at the top in the form:

// generated by stringer -type=Pill; DO NOT EDIT

The reason I ask is because generated files can be suppressed from diffs on GitHub (and possibly other code hosts and situations).

https://github.com/github/linguist#generated-file-detection

As an example, here is the code GitHub uses to decide that a file is a generated protocol buffer and should be suppressed:

def generated_protocol_buffer?

  return false unless ['.py', '.java', '.h', '.cc', '.cpp'].include?(extname)

  return false unless lines.count > 1

  return lines[0].include?("Generated by the protocol buffer compiler.  DO NOT EDIT!")

end

https://github.com/github/linguist/blob/master/lib/linguist/generated.rb#L205

I'd be happy to send a pull request to Linguist with the necessary changes.

Nathan.

On Tuesday, 1 July 2014 13:57:43 UTC-6, Rob Pike wrote:

New go tool subcommand proposed for Go 1.4. Please see the design
document and comment in this thread.

http://golang.org/s/go1.4-generate

-rob