The Go project in 2017

4,047 views
Skip to first unread message

Brad Fitzpatrick

unread,
Jan 4, 2017, 6:14:17 PM1/4/17
to golang-dev
Happy 2017, Gophers.

Before the Go 1.9 tree opens and before we start the regular mailing list planning thread about who's doing what during the 1.9 dev cycle, I wanted to start this separate thread about the health and tooling of the Go open source project.

As Go continues to grow in usage and contributors, some parts of the Go project are not scaling as well as we'd like.

In particular, there's an increasing number of bugs and code reviews being filed (to say nothing of the mailing list volume) and it's hard for people to keep up.

At the same time, we've heard via surveys and email and indirect anecdotes that many people want to contribute to Go but either don't know how or don't feel welcome.

To be explicit, we want and need your help. There's an increasing number of bugs to fix, bugs to triage, code to review, things to document, and tooling to improve.

And despite the increasing number of bugs and code reviews, I think there's an even greater number of people who'd want to send bugs & code but are intimidated by the process.

My main focus for the first half of the year will be working on the health and tooling for Go as an open source project to make sure it's easy and welcoming for people to send bugs and code, and that we have great tools, process, and documentation for processing incoming bugs and code.

I have my own pet-peeve list of things I'd like to be smoother (below), but my perspective is skewed from being on the maintainer side for many years. I'm going to rely on the community to help point out the areas that are broken or at least not ideal.

What do you find confusing or mysterious or intimidating about the Go project? Please reply here and/or file bugs. Let's fix it over the next few months.

As some examples I can think of:
 
* Flesh out the new "PriorDiscussion" wiki page (https://github.com/golang/go/issues/18430#issuecomment-270491909), so we give better answers and pointers to people in the future.

* Start accepting Github PRs (with Gerrit integration). That is https://github.com/golang/go/issues/18517

* Increase the number of non-Googlers with Github Issue/wiki access and access to various Gerrit permissions (https://golang.org/wiki/GerritAccess), and document more the process and policies for who gets what access, and when. That likely involves creating said policies, since it's somewhat random at the moment.

* Integrate some non-Googlers into our weekly proposal review process more. A/V issues and timezones are always a challenge, but we should do something.

* Make sure the Github issue labels are documented and used consistently. (https://github.com/golang/go/issues/18413)

* Make sure contributors know what to expect, and the sometimes-implicit state diagram of the lifetime of a bug or code review is made explicit. (refining this implicit process as necessary as we document it and over time as we get experience with it)

* (Related to the prior item) Start load balancing the Github & Gerrit review load. Currently a few of us have tried to look at everything, and obviously that can't scale as Go grows. We need to load balance incoming code reviews & bugs for a first-pass review for obvious issues before marking them as ready for more thorough reviews. I don't have an exact plan yet. I'd like to figure this out with the community. To some extent an informal system has evolved (I see many people helping out in Github and Gerrit with incoming stuff, thanks!), but our tooling and documentation around it all is lacking.

* Continue to improve the https://dev.golang.org/ dashboards.

* Get Android and iOS builders owned by people who care. (They're somewhat unloved right now.)

* Improve the build system and trybots (as always)

This is not a complete list by any means.

What else needs work?

Dave Cheney

unread,
Jan 4, 2017, 6:19:27 PM1/4/17
to Brad Fitzpatrick, golang-dev
This is a great list, thanks for starting this discussion Brad.

> What else needs work?

I know Quentin and Russ and working on a performance test harness.
This is near and dear to my heart and I want in.
> --
> You received this message because you are subscribed to the Google Groups
> "golang-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-dev+...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

Dave Cheney

unread,
Jan 4, 2017, 6:21:13 PM1/4/17
to Brad Fitzpatrick, golang-dev
> What else needs work?

Getting the subrepos green and keeping them green, possibly by gating
changes on trybots running the various subrepos.

Jessica Frazelle

unread,
Jan 4, 2017, 6:22:08 PM1/4/17
to Dave Cheney, Brad Fitzpatrick, golang-dev
I'm definitely interested build system and trybots! :D

More testing machines for all the architectures etc. And speeding up the tests for sure!

Kevin Burke

unread,
Jan 4, 2017, 6:29:06 PM1/4/17
to golang-dev
As someone who's mailed in a number of patches (and found/fixed quite a few typos), I would love to get more involved on the code review side, but it's a little intimidating to figure out which patches I can/should be reviewing; I'm not in the habit of checking every day, but I do read the commit history for every commit. I can definitely review anything for grammar/spelling, and general Go/networking/Linux knowledge, but probably not anything super specific about the compiler or about performance tuning.

Daniel Martí

unread,
Jan 4, 2017, 6:32:37 PM1/4/17
to Brad Fitzpatrick, golang-dev
On Wed, Jan 04, 2017 at 15:14:12 -0800, Brad Fitzpatrick wrote:
> * (Related to the prior item) Start load balancing the Github & Gerrit
> review load. Currently a few of us have tried to look at everything, and
> obviously that can't scale as Go grows. We need to load balance incoming
> code reviews & bugs for a first-pass review for obvious issues before
> marking them as ready for more thorough reviews. I don't have an exact plan
> yet. I'd like to figure this out with the community. To some extent an
> informal system has evolved (I see many people helping out in Github and
> Gerrit with incoming stuff, thanks!), but our tooling and documentation
> around it all is lacking.

I think a good way to horizontally scale would be to better define what
people take care of what aspects or packages of the Go project. Then it
would be easier to add granularity and make space for more people,
taking some work off long-term contributors.

For example, I know from memory what people do cmd/compile* stuff, but I
don't think that's documented anywhere. A wiki page might be good.

--
Daniel Martí - mv...@mvdan.cc - https://mvdan.cc/

Kevin Burke

unread,
Jan 4, 2017, 6:42:11 PM1/4/17
to golang-dev
(I would also be interested to know if there are people who would contribute more, but can't because they need money, and/or people looking for mentors; I'd like to give back more to the community if possible and tried posting about that here: https://groups.google.com/forum/m/#!topic/golang-nuts/3ezbJW4gTSk)


On Wednesday, January 4, 2017 at 3:14:17 PM UTC-8, Brad Fitzpatrick wrote:

ron minnich

unread,
Jan 4, 2017, 6:43:59 PM1/4/17
to Kevin Burke, golang-dev
I have a slightly weird problem that I'm looking for ideas on.

harvey is a plan 9 kernel with a small number of differences as far as Go is concerned. The binary format is ELF. The system call calling convention follows the amd64 ABI, not the Plan 9 stack-based one. It's a true 64-bit ABI (Plan 9 ABI is really 32 bits) so we don't need to pass a pointer to Seek for a 64-bit value; it just goes in the 3rd register argument. So the weirdness of Seek for the Plan 9 ABI does not exist on Harvey. The system call numbers are the same as Plan 9.

Aki Nyrhinen and I took a stab at a port to Harvey by doing the long task of finding all places that were plan 9 related (//build, *_plan9.go, etc.) and adding Harvey and it turned into quite a slog, and keeping up with ToT was impossible for our limited group. So, as a different experiment, I created a set of patches (567 lines total) that are applied to ToT, and then
I do a 
GOOS=plan9 make.bash
and get a Go compiler that builds for Harvey and runs on Harvey.

This method has worked for about a year: git pull the upstream master, then rebase a harvey branch on it, and my 3 small commits give me a working compiler. Almost all the tests pass, although I have yet to test in the last month or so.

So the harvey compiler is "just like" plan 9 with small changes to 6 files. 

The effort of doing a full port is more than we have people for. I keep wondering if there's a way to express a port as "just like this other port except for these exceptions" -- or embed it in the naming of things, or in how things are found.

I realize that this way probably lies madness, but thought I'd ask.

thanks

ron

Jaana Burcu Dogan

unread,
Jan 4, 2017, 6:50:17 PM1/4/17
to Daniel Martí, Brad Fitzpatrick, golang-dev
On Wed, Jan 4, 2017 at 3:32 PM, Daniel Martí <mv...@mvdan.cc> wrote:
On Wed, Jan 04, 2017 at 15:14:12 -0800, Brad Fitzpatrick wrote:
> * (Related to the prior item) Start load balancing the Github & Gerrit
> review load. Currently a few of us have tried to look at everything, and
> obviously that can't scale as Go grows. We need to load balance incoming
> code reviews & bugs for a first-pass review for obvious issues before
> marking them as ready for more thorough reviews. I don't have an exact plan
> yet. I'd like to figure this out with the community. To some extent an
> informal system has evolved (I see many people helping out in Github and
> Gerrit with incoming stuff, thanks!), but our tooling and documentation
> around it all is lacking.

I think a good way to horizontally scale would be to better define what
people take care of what aspects or packages of the Go project. Then it
would be easier to add granularity and make space for more people,
taking some work off long-term contributors.

I think this requires a bit of more structure and the presence of sub teams maybe. A few go-to people for each team, a written roadmap, summary of current projects and an accessible communication channel.
 

For example, I know from memory what people do cmd/compile* stuff, but I
don't think that's documented anywhere. A wiki page might be good.

--
Daniel Martí - mv...@mvdan.cc - https://mvdan.cc/
--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.

minux

unread,
Jan 4, 2017, 7:38:25 PM1/4/17
to ron minnich, golang-dev, Kevin Burke


On Jan 4, 2017 6:43 PM, "ron minnich" <rmin...@gmail.com> wrote:
I have a slightly weird problem that I'm looking for ideas on.

harvey is a plan 9 kernel with a small number of differences as far as Go is concerned. The binary format is ELF. The system call calling convention follows the amd64 ABI, not the Plan 9 stack-based one. It's a true 64-bit ABI (Plan 9 ABI is really 32 bits) so we don't need to pass a pointer to Seek for a 64-bit value; it just goes in the 3rd register argument. So the weirdness of Seek for the Plan 9 ABI does not exist on Harvey. The system call numbers are the same as Plan 9.

Aki Nyrhinen and I took a stab at a port to Harvey by doing the long task of finding all places that were plan 9 related (//build, *_plan9.go, etc.) and adding Harvey and it turned into quite a slog, and keeping up with ToT was impossible for our limited group. So, as a different experiment, I created a set of patches (567 lines total) that are applied to ToT, and then
I do a 
GOOS=plan9 make.bash
and get a Go compiler that builds for Harvey and runs on Harvey.

This method has worked for about a year: git pull the upstream master, then rebase a harvey branch on it, and my 3 small commits give me a working compiler. Almost all the tests pass, although I have yet to test in the last month or so.

So the harvey compiler is "just like" plan 9 with small changes to 6 files. 

The effort of doing a full port is more than we have people for. I keep wondering if there's a way to express a port as "just like this other port except for these exceptions" -- or embed it in the naming of things, or in how things are found.

I realize that this way probably lies madness, but thought I'd ask.

This perhaps belongs to another email thread. But I'd like to note that the Android port has exactly the same problem: it's almost Linux, with a few tweaks of its own. The current solution is that GOOS=android also includes the Linux build tag, implicitly. This reduces the code duplication considerably, but also introduces confusion.

I guess the ultimate answer depends on whether you plan to contribute the Harvey port into the upstream Go repo or not. If yes, perhaps we can do the same (GOOS=Harvey includes plan9 build tag implicitly.)

Brendan Tracey

unread,
Jan 4, 2017, 8:50:45 PM1/4/17
to Brad Fitzpatrick, golang-dev

> What do you find confusing or mysterious or intimidating about the Go project? Please reply here and/or file bugs. Let's fix it over the next few months.

1) I know this isn’t your fault, but the github issue tracker isn’t as good as the code.google.com one was, at least for someone who is loosely trying to follow issues. It used to be I could skim the issues every now and then, and easily identify an issue I had previously seen by where I had starred. I’m not sure how to do that effectively in github, especially with the increased frequency of issues.

2) Partly because of the above, I don’t know how to effectively follow the “big ideas” of where Go is going. That place used to be golang-dev, but it seems like a lot more is now only on github. For example, I don’t think the (original) alias proposal was even mentioned on golang-dev until Dave’s email.

3) Another consequence of (1) is I may not see issues I can be helpful with. I just happened to look at the right time to see #17675, but even if I had triage powers, I don’t know that I would have seen it.



ra...@develer.com

unread,
Jan 4, 2017, 9:25:21 PM1/4/17
to golang-dev
Thanks.

It would be great if PRs could always be fully reviewed (up to -2/+2 status), irrespective of the code freeze status. Code freeze matters for maintainers, but contributors have different working cycles.

Also, it's not clear how we can help in Gerrit. I tend not to write reviews because I don't want to suggest something to a contributor, only for a maintainer to later revert/override what I say. Triaging is one thing where contributors can help, but reviews sound harder. Either way, some policy/docs would help.

Giovanni

Joe Tsai

unread,
Jan 5, 2017, 12:32:34 AM1/5/17
to golang-dev
@Kevin Burke, I have a link bookmarked that essentially runs the following query: "status:open (is:starred file:"compress" OR file:"archive" OR reviewer:joe...@digital-static.net) -owner:joetsai" in Gerrit. Every now and then, I just visit that page and see if there's anything new, but it allows me to see what CLs are new in packages that I'm familiar with. You can probably adjust it to fit your own needs.

JT

Ralph Corderoy

unread,
Jan 5, 2017, 6:35:08 AM1/5/17
to golang-dev
Hi minux,

> the Android port has exactly the same problem: it's almost Linux,
> with a few tweaks of its own. The current solution is that
> GOOS=android also includes the Linux build tag, implicitly. This
> reduces the code duplication considerably, but also introduces
> confusion.
...
> perhaps we can do the same (GOOS=Harvey includes plan9 build tag
> implicitly.)

Could confusion be reduced by including the implicit build tag in the
dependant's name? harvey-plan9, or the other way around, linux-android.

--
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

Russ Cox

unread,
Jan 5, 2017, 9:25:33 AM1/5/17
to Brendan Tracey, Brad Fitzpatrick, golang-dev
On Wed, Jan 4, 2017 at 8:50 PM, Brendan Tracey <tracey....@gmail.com> wrote:
2) Partly because of the above, I don’t know how to effectively follow the “big ideas” of where Go is going. That place used to be golang-dev, but it seems like a lot more is now only on github. For example, I don’t think the (original) alias proposal was even mentioned on golang-dev until Dave’s email.

This is our fault. Important proposals should be announced to golang-dev. We should have posted about it and forgot. We'll do better next time.

1) I know this isn’t your fault, but the github issue tracker isn’t as good as the code.google.com one was, at least for someone who is loosely trying to follow issues. It used to be I could skim the issues every now and then, and easily identify an issue I had previously seen by where I had starred. I’m not sure how to do that effectively in github, especially with the increased frequency of issues.

3) Another consequence of (1) is I may not see issues I can be helpful with. I just happened to look at the right time to see #17675, but even if I had triage powers, I don’t know that I would have seen it.

This is a deeper issue of scale. Everyone can't be everywhere at the same time. The best we can hope is that we make sure everyone sees the important things (see above), and not worry as much about maximal exposure for the other things.

Russ

tylerb...@gmail.com

unread,
Jan 5, 2017, 12:50:45 PM1/5/17
to golang-dev, da...@cheney.net, brad...@golang.org
On Wednesday, January 4, 2017 at 4:22:08 PM UTC-7, Jessica Frazelle wrote:
I'm definitely interested build system and trybots! :D

More testing machines for all the architectures etc. And speeding up the tests for sure!

I have an idle Mac Mini I could turn into testing machine/build bot. Would that be helpful?

ron minnich

unread,
Jan 5, 2017, 1:13:11 PM1/5/17
to minux, golang-dev, Kevin Burke
On Wed, Jan 4, 2017 at 4:38 PM minux <mi...@golang.org> wrote:

This perhaps belongs to another email thread.


Shall I start another thread? I expected this discussion to die a quick death, but it seems I'm not the only one with this problem.

Having harvey include the plan 9 tag implicitly would *almost* work, but not quite. Consider this case:

def_plan9_amd64.go

This is one of 5 files with that name pattern that I change for harvey.

 I'm pretty sure just having harvey imply plan 9 would include that file, still, and that would not work.

however, consider this example:
if I have def_harvey_amd64.go, and def_plan9_amd64.go, I could extend GOOS as follows
GOOS=harvey:plan9 make.bash

it would be a search path: given a file named blah_harvey_blah.go, it would take precedence over blah_plan9_blah.go. 

Unfortunately this doesn't play well with build tags, I expect, and also compares of Goos to a name, but maybe there's a solution to that too. 

For those projects with limited time, it would be very nice to be able to say "we're almost completely like this GOOS named 'x' save for a few differences". This actually might also simplify existing ports such as the bsds.

We're doing ok for now with our fork, but I'd be happier were we on the dashboard. But as Aki and I learned in summer of 2015, tracking ToT feels like grabbing the tiger's tail.

thanks

ron

mun...@ca.ibm.com

unread,
Jan 5, 2017, 2:01:33 PM1/5/17
to golang-dev, mi...@golang.org, k...@inburke.com
Shall I start another thread? I expected this discussion to die a quick death, but it seems I'm not the only one with this problem.

I've changed the subject for this reply, I'm not sure if that will start a new thread or not. If not, then probably best to start a new thread.


Having harvey include the plan 9 tag implicitly would *almost* work, but not quite. Consider this case:

def_plan9_amd64.go

This is one of 5 files with that name pattern that I change for harvey.

 I'm pretty sure just having harvey imply plan 9 would include that file, still, and that would not work.

This is pretty easy to do I think. Just include the following line in the vanilla-plan9-only file:

// +build !harvey

This will mean that the file is built on plan9 only if the harvey build tag is not set. As an example the line '// +build !android' is present in src/runtime/internal/sys/zgoos_linux.go to stop it building on android.

ron minnich

unread,
Jan 5, 2017, 2:44:44 PM1/5/17
to mun...@ca.ibm.com, golang-dev, mi...@golang.org, k...@inburke.com
ok, thanks, and we can -golang-dev if this is not of general interest.

the // +build !harvey
idea confuses me. If GOOS is harvey, files named *plan9* won't get built anyway. So adding that to those files is kind of a no op.

IOW, if GOOS is harvey, then all the tests against the OS name, build tags, and file names fails. 
if GOOS is plan9, but our target is really harvey, all but five files are quite usable. It's those five files that are at issue.

thanks

ron



--

mdla...@gmail.com

unread,
Jan 5, 2017, 2:51:25 PM1/5/17
to golang-dev
> * Increase the number of non-Googlers with Github Issue/wiki access and access to various Gerrit permissions (https://golang.org/wiki/GerritAccess), and document more the process and policies for who gets what access, and when. That likely involves creating said policies, since it's somewhat random at the moment.

To echo some previous comments in this thread, even when these permissions are granted (I can run trybots and approve CLs), it's quite difficult to determine whether or not it's "appropriate" for someone outside of Google to make a decision on a given CL.  I'd hate to approve something and then immediately have a Go team member give a -2 code review because something with the CL or idea is totally wrong.

I'd love to see some more clear guidelines in place on when/where it's appropriate for an outside contributor to make that decision.


> * Integrate some non-Googlers into our weekly proposal review process more. A/V issues and timezones are always a challenge, but we should do something.

This is really awesome to hear, and I'd love to get involved in some way if at all possible.
Message has been deleted

mun...@ca.ibm.com

unread,
Jan 5, 2017, 3:41:10 PM1/5/17
to golang-dev, mun...@ca.ibm.com, mi...@golang.org
the // +build !harvey
idea confuses me. If GOOS is harvey, files named *plan9* won't get built anyway. So adding that to those files is kind of a no op.

minux is suggesting that the go build tool be modified so that when GOOS=harvey all plan9 files are also included in the build implicitly (as minux mentioned, this is what GOOS=android does but for linux files).

In your scenario you would then need to create the new harvey-specific file def_harvey_amd64.go and add '// +build !harvey' to the def_plan9_amd64.go file to stop it being built on harvey. All the other plan9 files (that do not contain '// +build !harvey') will be built when GOOS=harvey.

Since this idea would require copying the GOOS=android mechanism I suggest taking a look at src/cmd/dist/build.go and src/go/build/build.go to see how GOOS=android implicitly includes linux files.

Nathan Youngman

unread,
Jan 5, 2017, 5:27:52 PM1/5/17
to golang-dev, mv...@mvdan.cc, brad...@golang.org


On Wednesday, 4 January 2017 16:50:17 UTC-7, Jaana Burcu Dogan wrote:

On Wed, Jan 4, 2017 at 3:32 PM, Daniel Martí <mv...@mvdan.cc> wrote:
I think a good way to horizontally scale would be to better define what
people take care of what aspects or packages of the Go project. Then it
would be easier to add granularity and make space for more people,
taking some work off long-term contributors.

I think this requires a bit of more structure and the presence of sub teams maybe. A few go-to people for each team, a written roadmap, summary of current projects and an accessible communication channel.

Perhaps sub teams working on Go proper is really the same as the idea of special interest groups that's been floating around lately. For example, those people interested in working on the compiler might operate in a similar fashion to those working on a new set of audio packages and interfaces. Hangouts, code reviews, trybots, mailing list, etc. 

It would be a matter of making guidelines and tools available for any team contributing to the Go community based on some set of requirements (what qualifies as a SIG). It doesn't really matter where the source code for those audio libraries live, whether in x/ subrepositories or out in the wild west of GitHub.

Ralph Corderoy

unread,
Jan 5, 2017, 5:50:41 PM1/5/17
to mdla...@gmail.com, golan...@googlegroups.com
Hi mdlayher,

> I'd hate to approve something and then immediately have a Go team
> member give a -2 code review because something with the CL or idea is
> totally wrong.

"Be bold", as Wikipedia says. You wouldn't have been given those
permissions lightly. Sure, it's embarrassing when you get things wrong,
and annoying if you're the kind that doesn't like making mistakes, but
as long as you consider why that happened and what can lessen the chance
of repetition then I expect everything will be fine. Just look at the
number of errors introduced and +2'd by Googlers. ;-)

I review on Gerrit when I get the chance not to +1, but to learn more
about the project's code, because reviewing makes one study more
thoroughly than just reading, to learn good idioms from the experts, and
to read the insights from those that do -2/+2. Apart from the odd
stupid comment made in haste, the thing I've found most worrying is a
novice changeset submitter might think I'm "worthy" of paying attention
to because there are few "unofficial" reviewers.

Edward Muller

unread,
Jan 5, 2017, 6:04:13 PM1/5/17
to Brad Fitzpatrick, golang-dev
It is pretty hard to drink from the firehose that is all Github notifications for github.com/golang/go. With that said, I would like to help with issues and any other areas where I can be useful. 

Marvin Stenger

unread,
Jan 5, 2017, 7:01:57 PM1/5/17
to golang-dev, brad...@golang.org
Once there was the idea of having automatically maintained "owner files" associated for each repository listing the developers most actively or most recently working on that package in order to get them involved again, when new code-reviews come in.

ron minnich

unread,
Jan 5, 2017, 9:02:37 PM1/5/17
to mun...@ca.ibm.com, golang-dev, mi...@golang.org
OK, got it, that's great. I missed the part about harvey implies plan 9, sorry. 

The remaining part I don't quite understand: what about all those places that test for Goos being plan 9? 

Our problems have boiled down to a few simple things:
1. the files to build, which the minux suggestion solves
2. the Goos string
3. the build directives

It seems this idea solves 1 and 3, but I'm not sure about 2. Does the build tool in this case establish an equivalence
between plan 9 and harvey?

thanks

ron

mun...@ca.ibm.com

unread,
Jan 5, 2017, 9:32:29 PM1/5/17
to golang-dev, mun...@ca.ibm.com, mi...@golang.org
Does the build tool in this case establish an equivalence between plan 9 and harvey?

No, unfortunately you'd have to manually update all the runtime.GOOS string comparisons.

alex....@gmail.com

unread,
Jan 5, 2017, 11:43:27 PM1/5/17
to golang-dev
Thanks for working on this, Brad.

Last year I submitted a couple of small CLs to the Go project. I found it quite time-consuming to get up and running with an initial dev loop in which I could write, build, and run a single test. I remember that doing this without each time running the shell script that executes the entire test suite was somewhat nontrivial, and not well documented. For reference I've attached the instructions I wrote for myself to the end of this email -- perhaps from this you will be able to reconstruct what documentation was missing.

It would have been helpful to have a very explicit tutorial on writing and running a test in the core codebase, all the way from cloning the repo to doing the one-time bootstrap stuff, to writing and running the test.

Alex

---
The instructions I wrote for myself were as follows:

First build from source:
GOROOT_BOOTSTRAP=/usr/local/go ./all.bash

Add the resulting binary to PATH:
PATH=/Users/alex/Code/golang/bin:$PATH

unset GOROOT

Then build and run unittests as usual:
cd net
go test


On Wednesday, January 4, 2017 at 3:14:17 PM UTC-8, Brad Fitzpatrick wrote:
Happy 2017, Gophers.

Before the Go 1.9 tree opens and before we start the regular mailing list planning thread about who's doing what during the 1.9 dev cycle, I wanted to start this separate thread about the health and tooling of the Go open source project.

As Go continues to grow in usage and contributors, some parts of the Go project are not scaling as well as we'd like.

In particular, there's an increasing number of bugs and code reviews being filed (to say nothing of the mailing list volume) and it's hard for people to keep up.

At the same time, we've heard via surveys and email and indirect anecdotes that many people want to contribute to Go but either don't know how or don't feel welcome.

To be explicit, we want and need your help. There's an increasing number of bugs to fix, bugs to triage, code to review, things to document, and tooling to improve.

And despite the increasing number of bugs and code reviews, I think there's an even greater number of people who'd want to send bugs & code but are intimidated by the process.

My main focus for the first half of the year will be working on the health and tooling for Go as an open source project to make sure it's easy and welcoming for people to send bugs and code, and that we have great tools, process, and documentation for processing incoming bugs and code.

I have my own pet-peeve list of things I'd like to be smoother (below), but my perspective is skewed from being on the maintainer side for many years. I'm going to rely on the community to help point out the areas that are broken or at least not ideal.

What do you find confusing or mysterious or intimidating about the Go project? Please reply here and/or file bugs. Let's fix it over the next few months.

As some examples I can think of:
 
* Flesh out the new "PriorDiscussion" wiki page (https://github.com/golang/go/issues/18430#issuecomment-270491909), so we give better answers and pointers to people in the future.

* Start accepting Github PRs (with Gerrit integration). That is https://github.com/golang/go/issues/18517

* Increase the number of non-Googlers with Github Issue/wiki access and access to various Gerrit permissions (https://golang.org/wiki/GerritAccess), and document more the process and policies for who gets what access, and when. That likely involves creating said policies, since it's somewhat random at the moment.

* Integrate some non-Googlers into our weekly proposal review process more. A/V issues and timezones are always a challenge, but we should do something.

* Make sure the Github issue labels are documented and used consistently. (https://github.com/golang/go/issues/18413)

* Make sure contributors know what to expect, and the sometimes-implicit state diagram of the lifetime of a bug or code review is made explicit. (refining this implicit process as necessary as we document it and over time as we get experience with it)

* (Related to the prior item) Start load balancing the Github & Gerrit review load. Currently a few of us have tried to look at everything, and obviously that can't scale as Go grows. We need to load balance incoming code reviews & bugs for a first-pass review for obvious issues before marking them as ready for more thorough reviews. I don't have an exact plan yet. I'd like to figure this out with the community. To some extent an informal system has evolved (I see many people helping out in Github and Gerrit with incoming stuff, thanks!), but our tooling and documentation around it all is lacking.

Brad Fitzpatrick

unread,
Jan 6, 2017, 2:27:41 PM1/6/17
to alex....@gmail.com, golang-dev
On Thu, Jan 5, 2017 at 8:04 PM, <alex....@gmail.com> wrote:
Thanks for working on this, Brad.

Last year I submitted a couple of small CLs to the Go project. I found it quite time-consuming to get up and running with an initial dev loop in which I could write, build, and run a single test. I remember that doing this without each time running the shell script that executes the entire test suite was somewhat nontrivial, and not well documented. For reference I've attached the instructions I wrote for myself to the end of this email -- perhaps from this you will be able to reconstruct what documentation was missing.

It would have been helpful to have a very explicit tutorial on writing and running a test in the core codebase, all the way from cloning the repo to doing the one-time bootstrap stuff, to writing and running the test.


For example, https://golang.org/doc/contribute.html#Testing says to run all.bash to run all the tests.

All that's missing from that page is a reference to GOROOT_BOOTSTRAP which is mentioned on https://golang.org/doc/install/source. I filed https://github.com/golang/go/issues/18545 to add to the docs.
 

--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.

minux

unread,
Jan 6, 2017, 6:22:42 PM1/6/17
to Brad Fitzpatrick, golang-dev
A slightly related request, is it possible for Gerrit to show if
the CL author can submit on the CL page?

Whenever I leave a +2 score, I hesitate to submit the CL not
knowing whether the author want to submit or not.

Or perhaps just submit away when you're confident about
the CL and give +2 score?

Brad Fitzpatrick

unread,
Jan 6, 2017, 6:31:45 PM1/6/17
to minux, golang-dev
Good point.

This is another thing that we could automate in addition to: 

https://github.com/golang/go/issues/9858 -- trybots should rebase when testing

... which is also about auto-submit.

We could have a bot that auto-rebases and auto-tests +2ed CLs, and if they're not merged in N hours, it could add a reviewer who has +2 access, leaving a comment at the same time like, "This CL is rebased & tests pass. It needs somebody to press submit."

(unless the CL had some sort of AUTOSUBMIT=true comment, in which case it'd just happen)

Could you file a bug?

Brad Fitzpatrick

unread,
Jan 6, 2017, 6:58:52 PM1/6/17
to Matt Layher, golang-dev
On Thu, Jan 5, 2017 at 11:25 AM, Matt Layher <mdla...@gmail.com> wrote:
> * Increase the number of non-Googlers with Github Issue/wiki access and access to various Gerrit permissions (https://golang.org/wiki/GerritAccess), and document more the process and policies for who gets what access, and when. That likely involves creating said policies, since it's somewhat random at the moment.

As some others have mentioned, even once access is granted, it can be difficult to determine (as an outside contributor) if you are the correct person to provide a review or approve a change.  It would be nice to have some guidelines spelled out on what kind of changes are okay for outside contributors to review.  I'd hate to give a +1/+2 and then immediately see a -2 after from a Go team member because I wasn't aware of some existing rule or the change was totally wrong.

In addition to what Ralph said ("be bold", and not worrying about it), the other thing you can do if you're uncomfortable is only leave comments on the parts you're pretty sure are wrong.

In fact, I would like to get to the point where a +1 just means that the CL is not obviously wrong or misformatted.

If we could get to the point where the owners of various areas of Go only look at properly formatted CLs and aren't involved in the early polish of CLs, that would free up a lot of of our time.

So feel free to slap +1 on a CL when you think it's looks like a typical CL, regardless of its correctness.

I've updated https://github.com/golang/go/wiki/Gardening#pending-cls with a bit more about this.

Nathan Youngman

unread,
Jan 7, 2017, 10:49:39 AM1/7/17
to golang-dev


On Wednesday, 4 January 2017 16:14:17 UTC-7, Brad Fitzpatrick wrote:
* Improve the build system and trybots (as always)

This is not a complete list by any means.

What else needs work?


Hi Brad,

Thanks for this effort. Are you still thinking about making go builders and trybots available to third-party repositories?


Nathan.
Reply all
Reply to author
Forward
0 new messages