[ANN] gorename: precise type-safe renaming of identifiers in Go source code

[Alan Donovan]

gorename is a new tool for automatic and sound renaming of Go identifiers.

It reports all declaration and reference conflicts so that, given a well-typed Go program as input, it will either report an informative error, or transform the program to another well-typed and semantically equivalent* Go program.

The entity to rename may be specified by position (e.g. from within an editor) or by logical name.  Adding editor support is straightforward, but only Emacs is integrated today; contributions welcome.  

Run 'gorename -help' to see documentation, most of which is shown below.  The complete message includes caveats/TODOs.

Any suggestions much appreciated.

cheers

alan

* The usual static analysis caveats apply; gorename doesn't understand reflection or comments.

% go get code.google.com/p/go.tools/cmd/gorename

% go build code.google.com/p/go.tools/cmd/gorename

% mv ./gorename $GOROOT/bin

% gorename -help

gorename: precise type-safe renaming of identifiers in Go source code.

Usage:

 gorename (-from <spec> | -offset <file>:#<byte-offset>) -to <name> [-force]

You must specify the object (named entity) to rename using the -offset

or -from flag.  Exactly one must be specified.

Flags:

-offset    specifies the filename and byte offset of an identifier to rename.

           This form is intended for use by text editors.

-from      specifies the object to rename using a query notation;

           This form is intended for interactive use at the command line.

A legal -from query has one of the following forms:

  (encoding/json.Decoder).Decode      method of package-level named type

  (encoding/json.Decoder).buf         field of package-level named struct type

  encoding/json.HTMLEscape            package member (const, func, var, type)

  (encoding/json.Decoder).Decode::x   local object x within a method

  encoding/json.HTMLEscape::x         local object x within a function

  encoding/json::x                    object x anywhere within a package

  json.go::x                          object x within file json.go

  For methods attached to a pointer type, the '*' must not be specified.

  [TODO(adonovan): fix that.]

  It is an error if one of the ::x queries matches multiple objects.

-to        the new name.

-force     causes the renaming to proceed even if conflicts were reported.

           The resulting program may be ill-formed, or experience a change

           in behaviour.

           WARNING: this flag may even cause the renaming tool to crash.

           (In due course this bug will be fixed by moving certain

           analyses into the type-checker.)

-dryrun    causes the tool to report conflicts but not update any files.

-v         enables verbose logging.

gorename automatically computes the set of packages that might be

affected.  For a local renaming, this is just the package specified by

-from or -offset, but for a potentially exported name, gorename scans

the workspace ($GOROOT and $GOPATH).

gorename rejects any renaming that would create a conflict at the point

of declaration, or a reference conflict (ambiguity or shadowing), or

anything else that could cause the resulting program not to compile.

Currently, it also rejects any method renaming that would change the

assignability relation between types and interfaces.

Examples:

% gorename -offset file.go:#123 -to foo

  Rename the object whose identifier is at byte offset 123 within file file.go.

% gorename -from '(bytes.Buffer).Len' -to Size

  Rename the "Len" method of the *bytes.Buffer type to "Size".

[Steve McCoy]

I'm psyched to use this.

But, here are two critiques disguised as questions: Why require parentheses in some of the -from queries? And why have a -to flag if it's mandatory?

[Alan Donovan]

On 23 September 2014 15:43, Steve McCoy <mcc...@gmail.com> wrote:

I'm psyched to use this.

Glad to hear it!

 

Why require parentheses in some of the -from queries?

To avoid ambiguity.  It allows the tool to distinguish these two syntaxes:

   importpath.member.method

   importpath.member

without doing any I/O.  (Recall that importpath may contains periods.)

 

And why have a -to flag if it's mandatory?

For symmetry with -from; for better documentation.

Both of these UI issues can be changed fairly easily.

[Steve McCoy]

Ah, right, dots in import paths. Thanks!

[Ian Dawes]

Fantastic, I've been hoping that somebody would build this for quite some time! Sublime Text support would be a dream come true.

[adon...@google.com]

P.S. gorename requires a $GOROOT recent enough to include last week's move of the $GOROOT/src/pkg directory.

[Fatih Arslan]

Vim users can now use it by pulling the latest master from vim-go
repo: https://github.com/fatih/vim-go

Here is quick screencast that shows the implementation: http://quick.as/dgof2p7

There is now the "GoRename" vim command that can be used to rename the
identifier under the cursor. You can directly pass the new name as an
argument or it will prompt to enter you the new identifier if no
arguments are given.

Let me know if you hit problems :)

Regards

> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Fatih Arslan]

Btw, I've forget to add, you can easily install the `gorename` binary
with the ":GoInstallBinaries" command.

[Dmitri Shuralyov]

gorename is an incredibly useful tool, thank you for making it!

For those trying it for the first time, please be aware that it may fail to work or perform more slowly than normal due to an issue in code.google.com/p/go.tools/go/buildutil, which gorename imports indirectly [1].

If you have any git repositories or Go packages with large test suites or vendored packages, buildutil will descend into far too many dirs that it shouldn't (unlike the go tool that respects hidden directories correctly). I've reported the issue at https://code.google.com/p/go/issues/detail?id=8907 and I hope it will be fixed soon because it's potentially just a 3 line change.

[1] http://godoc.org/code.google.com/p/go.tools/cmd/gorename?import-graph&hide=2

[Peter Nguyen]

I tried to use GoRename but got the following error

runtime: failed to create new OS thread (have 2049 already; errno=12)

[Alan Donovan]

On 12 October 2014 04:53, Peter Nguyen <peter.ng...@gmail.com> wrote:

I tried to use GoRename but got the following error

runtime: failed to create new OS thread (have 2049 already; errno=12)

Thanks for reporting this.  (And thanks for Rog Peppe for a previous report and diagnosis.)  There's a known bug whereby it uses unlimited concurrency exploring the file tree in the workspace, resulting in thousands of parallel system calls each on its own OS thread.  I don't think there's an easy workaround (other than: have fewer files!) without changing the code.  I'll get to it this week.

[Peter Nguyen]

My package folder only have 12 Go files though... Does it search outside the folder too?

[Alan Donovan]

Yes, if the renaming changes an exported identifier, the tool must examine all packages that transitively depend upon the defining package, since they may contain references to it.

[Peter Nguyen]

Maybe it could be an option to whether rename only in the current package or all packages?

[robfig]

Great tool, thank you.

I've also been running into "Too many open files" -- the error messages suggest that it is unnecessarily exploring inside ".git" directories.  Perhaps excluding VCS directories could help a little?

e.g.

Package "github.com/codegangsta/martini-contrib/.git/logs": open /Users/robfig/gocode/src/github.com/codegangsta/martini-contrib/.git/logs: too many open files.

[Alan Donovan]

On 14 October 2014 13:03, robfig <rob...@gmail.com> wrote:

Great tool, thank you.

Thanks!

 

I've also been running into "Too many open files" -- the error messages suggest that it is unnecessarily exploring inside ".git" directories.  Perhaps excluding VCS directories could help a little?

Try syncing and building from tip; the fix was committed this morning.

cheers

alan

[drj]

Thank you, it's a great tool!

I made a wrapper around gorename for use with Acme. It's go gettable:

% go get bitbucket.org/drj_/agorn

Let me know what you think.

-david

[mpe...@gmail.com]

Hi Alan, works great with fatih's vim-go plugin.

Is there any way to turn off ".prerename" files littering the filesystem though?  The manual cleanup after a rename is painful.

Mike

[Alan Donovan]

On 15 October 2014 14:03, <mpe...@gmail.com> wrote:

Hi Alan, works great with fatih's vim-go plugin.

Is there any way to turn off ".prerename" files littering the filesystem though?  The manual cleanup after a rename is painful.

Currently, no, but I'll add that feature to my list. 

[Steve McCoy]

Nice! Works just the way I like.

[drj]

Thanks for your interest.

I made some improvements to it. You can 'git pull' the newest version.

I'm interested in your feedback. You can file issues on bitbucket.

-david

[adon...@google.com]

Update:

gorename now supports renaming of abstract methods; it will rename all coupled concrete methods.  (Caveat: unlike other renamings, soundness of this one is undecidable, so while it will never introduce a compile error, there is a risk that it may change the program's dynamic behaviour by causing a concrete type to no longer satisfy some distant interface type to which it is indirectly---but never directly---assigned.  I think this is unlikely to be a problem in practice.)

gorename's -from syntax now requires double-quotes for multi-segment package import paths, and it allows but no longer requires parens for non-pointer types.

gorename now preserves file modes.

cheers

alan

On Tuesday, 23 September 2014 14:55:18 UTC-4, Alan Donovan wrote:

[Ugorji Nwoke]

The tool's help says:

- allow users to specify a scope other than "global" (to avoid being

  stuck by neglected packages in $GOPATH that don't build).

How do I do this? I want to make changes only within the packages under my current directory. 

Thanks.

[Alan Donovan]

On 10 December 2014 at 13:07, Ugorji Nwoke <ugo...@gmail.com> wrote:

The tool's help says:

- allow users to specify a scope other than "global" (to avoid being

  stuck by neglected packages in $GOPATH that don't build).

How do I do this? I want to make changes only within the packages under my current directory. 

Renaming unexported identifiers will work fine.  Renaming exported identifiers will work fine too if everything that depends on the defining package builds ok.  The problematic case is renaming an exported identifier defined in package P, where some broken package depends on P.  In that case, just delete the broken package.  Of course, you needn't check in the deletion, but it will let you proceed with the refactoring, and you can revert it later.

[Ugorji Nwoke]

Thanks.

The feature to define a scope for the changes would have worked very well. I wouldn't have to look through all packages that do not build, and remove them, just to do a rename. I could just specify the subset of packages I care about making changes in, and those are guaranteed to build.

In addition, I wanted to export a function which was previously unexported e.g. change function name from do() to Do(). This should not have to go through the whole GOPATH.

[Alan Donovan]

On 10 December 2014 at 14:22, Ugorji Nwoke <ugo...@gmail.com> wrote:

Thanks.

The feature to define a scope for the changes would have worked very well. I wouldn't have to look through all packages that do not build, and remove them, just to do a rename. I could just specify the subset of packages I care about making changes in, and those are guaranteed to build.

I agree it would be nice. 

 

In addition, I wanted to export a function which was previously unexported e.g. change function name from do() to Do(). This should not have to go through the whole GOPATH.

That's what I thought before I analyzed the problem, but it turns out not to be true.

If you're renaming a package-level type from "p.foo" too "p.Foo", and that type is used as an anonymous field of some exported struct type p.S, and some client of p declares:

  type T struct { q.Foo }

  type U struct { p.S; T }

then the renaming would cause all references to U.Foo to become ambiguous, i.e. an error.  (Previously, they referred to U.T.Foo, now they could refer to U.S.Foo too.)

(Internally gorename calls this a "super-block selection conflict".)

It may sound far-fetched, but examples made from single-letter names never seem convincing, and I wouldn't be surprised if this actually happens in practice.

And besides, when I promised sound, I meant sound. :)

cheers

alan

[Ugorji Nwoke]

Understood. Makes sense.

What I eventually did, was something similar to:

- rename methods like meth1 to zzzzzzzzMeth1, meth2 to zzzzzzzzMeth2, ...

- use sed to read all files and remove all occurrences of zzzzzzzz

It worked well.

Thanks again for the tool. It made my life much easier.

[zbergq...@gmail.com]

I would love to use this tool but can't get it to work.  What am I doing wrong here?

gorename -from "github.com/me/mypackage".TestFunc -to NewTestFunc
Error: -from %!q(MISSING): invalid expression.

I've tried a few variations of -from based on the usage message, and they all give me this result.

I also tried the following form:

gorename -from main.go::TestFunc -to NewTestFunc

That invocation gives me several hundred lines of error messages, that look like:

could not import encoding/json (cannot find package "encoding/json" in ...)

Thanks for any help!

[Alan Donovan]

On 11 January 2015 at 13:30, <zbergq...@gmail.com> wrote:

I would love to use this tool but can't get it to work.  What am I doing wrong here?

gorename -from "github.com/me/mypackage".TestFunc -to NewTestFunc
Error: -from %!q(MISSING): invalid expression.

I've tried a few variations of -from based on the usage message, and they all give me this result.

I also tried the following form:

gorename -from main.go::TestFunc -to NewTestFunc

That invocation gives me several hundred lines of error messages, that look like:

could not import encoding/json (cannot find package "encoding/json" in ...)

Thanks for any help!

For multi-segment import paths such as this one, gorename needs to see the double-quotes, but most likely your shell is evaluating the double-quoted string.  Add another layer of single quotes:

% gorename -from '"github.com/me/mypackage".TestFunc' -to NewTestFunc

Perhaps we need better rules for parsing the import path.

BTW, the "%!q(MISSING)"  bug was fixed a while back; I suggest you sync up and rebuild.  Support for interface method renamings was added since then.

cheers

alan