Proposal to drop plan9 and dragonfly

[Brad Fitzpatrick]

This is a discussion about dropping broken and unmaintained Go ports from the main tree once the Go 1.3 tree opens.

Specifically, I propose deleting plan9 and dragonfly support.  Neither has many users, nor are they running a builder at http://build.golang.org/.

Both can be maintained by their respective communities out-of-tree.

Once those communities have builders running and passing all tests, they can be considered for inclusion back in the tree.  This is similar to the "tiny" runtime and NaCl support which were once in the tree but now exist only in the hg history.

- Brad

[Dave Cheney]

On Wed, Nov 13, 2013 at 11:27 AM, Brad Fitzpatrick <brad...@golang.org> wrote:
> This is a discussion about dropping broken and unmaintained Go ports from
> the main tree once the Go 1.3 tree opens.
>
> Specifically, I propose deleting plan9 and dragonfly support. Neither has
> many users, nor are they running a builder at http://build.golang.org/.

I don't like the precedent of 'not many users', but the being able to
host a builder sounds like a good yard stick for inclusion.

> Both can be maintained by their respective communities out-of-tree.
>
> Once those communities have builders running and passing all tests, they can
> be considered for inclusion back in the tree. This is similar to the "tiny"
> runtime and NaCl support which were once in the tree but now exist only in
> the hg history.
>
> - Brad
>

> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "golang-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-dev+...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

[Brad Fitzpatrick]

On Tue, Nov 12, 2013 at 4:30 PM, Dave Cheney <da...@cheney.net> wrote:

On Wed, Nov 13, 2013 at 11:27 AM, Brad Fitzpatrick <brad...@golang.org> wrote:
> This is a discussion about dropping broken and unmaintained Go ports from
> the main tree once the Go 1.3 tree opens.
>
> Specifically, I propose deleting plan9 and dragonfly support.  Neither has
> many users, nor are they running a builder at http://build.golang.org/.

I don't like the precedent of 'not many users', but the being able to
host a builder sounds like a good yard stick for inclusion.

Basically, a port just has to contribute more to the project than it costs in maintenance.

Ports can contribute users and/or bug reports (e.g. finding bogus assumptions we make in the runtime or compiler about an OS or architecture).  But if your existence merely causes us maintenance cost and we get neither users nor bug reports, it's better to remove it from the tree.

[Dmitry Vyukov]

On Wed, Nov 13, 2013 at 4:30 AM, Dave Cheney <da...@cheney.net> wrote:

On Wed, Nov 13, 2013 at 11:27 AM, Brad Fitzpatrick <brad...@golang.org> wrote:
> This is a discussion about dropping broken and unmaintained Go ports from
> the main tree once the Go 1.3 tree opens.
>
> Specifically, I propose deleting plan9 and dragonfly support.  Neither has
> many users, nor are they running a builder at http://build.golang.org/.

I don't like the precedent of 'not many users', but the being able to
host a builder sounds like a good yard stick for inclusion.

I agree that 'not many users' is questionable. But we need a builder and an active maintainer(s).

[minux]

On Thu, Nov 14, 2013 at 12:46 AM, Dmitry Vyukov <dvy...@google.com> wrote:

On Wed, Nov 13, 2013 at 4:30 AM, Dave Cheney <da...@cheney.net> wrote:

On Wed, Nov 13, 2013 at 11:27 AM, Brad Fitzpatrick <brad...@golang.org> wrote:
> This is a discussion about dropping broken and unmaintained Go ports from
> the main tree once the Go 1.3 tree opens.
>
> Specifically, I propose deleting plan9 and dragonfly support.  Neither has
> many users, nor are they running a builder at http://build.golang.org/.

I don't like the precedent of 'not many users', but the being able to
host a builder sounds like a good yard stick for inclusion.

I agree that 'not many users' is questionable. But we need a builder and an active maintainer(s).

I agree with both of your points.

A lot of open source projects have the concept of code owner, subsystem maintainer or

something similar, perhaps we can adopt that. The new port is only allowed to enter the tree

if:

1. it has a builder, and is passing all.bash (should we relax this? some of existing port don't pass

all.bash all the time)

2. someone (or a group of people) could take the responsibility to maintain the port in case it

breaks by a runtime/toolchain change. If the builder remains red for certain amount of time,

we wait some more time (perhaps even email the maintainer) and then eliminate the port from

the tree.

What do you think? If we want to try a system like that, we probably could start with DragonFly

BSD and Plan 9. (I don't think we should remove them without first asking for help from respective

communities)

[Brad Fitzpatrick]

To give a concrete example of the overhead: code.google.com/p/go.tools/ssa/interp is currently broken for plan9, and that's affecting me right now for some lame reason (probably something else is broken).  So even though I've never looked at interp and never used plan9, I'm currently looking into how to fix interp on plan9 and looking at syscall_plan9.go.  That's overhead.

What are the plan9 people doing to make Go and/or my life easier?

[andrey mirtchovski]

What are the plan9 people doing to make Go and/or my life easier?

i think the whole point of the go port to plan9 was that it will help revive plan9, not the other way around. this hasn't happened and despite everyone's best intentions not much software for plan9 has been written in Go. only one experimental offshoot of p9 is using Go more heavily, but that's to plan9 what dragonfly is to the bsd's.

chuck it in the archives next to 'tiny'.

[minux]

On Thu, Nov 14, 2013 at 1:05 AM, Brad Fitzpatrick <brad...@golang.org> wrote:

To give a concrete example of the overhead: code.google.com/p/go.tools/ssa/interp is currently broken for plan9, and that's affecting me right now for some lame reason (probably something else is broken).  So even though I've never looked at interp and never used plan9, I'm currently looking into how to fix interp on plan9 and looking at syscall_plan9.go.  That's overhead.

I think I'm missing something here, syscall_plan9.go just contains some stub functions, why is that so much an overhead?

I'm also curious as to why a broken ssa/interp on plan 9 affects a non-Plan 9 user.

[Brad Fitzpatrick]

On Wed, Nov 13, 2013 at 10:25 PM, minux <minu...@gmail.com> wrote:

On Thu, Nov 14, 2013 at 1:05 AM, Brad Fitzpatrick <brad...@golang.org> wrote:

To give a concrete example of the overhead: code.google.com/p/go.tools/ssa/interp is currently broken for plan9, and that's affecting me right now for some lame reason (probably something else is broken).  So even though I've never looked at interp and never used plan9, I'm currently looking into how to fix interp on plan9 and looking at syscall_plan9.go.  That's overhead.

I think I'm missing something here, syscall_plan9.go just contains some stub functions, why is that so much an overhead?

Try to build it.  GOOS=plan9 and waste some time with me.

 

I'm also curious as to why a broken ssa/interp on plan 9 affects a non-Plan 9 user.

Internal tool.  (which I think is also broken, but it led me to this)

[Anthony Martin]

Brad Fitzpatrick <brad...@golang.org> once said:
> Try to build it. GOOS=plan9 and waste some time with me.

That's easy. The code is using a non-portable constant
(syscall.ENOSYS) in a file with no build constraints.

CL is here: https://codereview.appspot.com/26500043

Anthony

[Brad Fitzpatrick]

On Thu, Nov 14, 2013 at 1:56 AM, Anthony Martin <al...@pbrane.org> wrote:

Brad Fitzpatrick <brad...@golang.org> once said:
> Try to build it.  GOOS=plan9 and waste some time with me.

That's easy. The code is using a non-portable constant
(syscall.ENOSYS) in a file with no build constraints.

Easy is the opposite of hard, not the opposite of time-consuming.

 

CL is here: https://codereview.appspot.com/26500043

Thanks

 

[Aram Hăvărneanu]

I am busy with the Solaris port, once I that stabilises I volunteer to run the Plan 9 builders and take over maintenance.

[Aram Hăvărneanu]

On Thursday, November 14, 2013 4:31:04 PM UTC+1, Aram Hăvărneanu wrote:

I am busy with the Solaris port, once that stabilises I volunteer to run the Plan 9 builders and take over maintenance.

CCing dho. 

[Russ Cox]

When deciding what ports we keep in the main tree, the number of users is a factor to consider but it is probably a relatively small one. A much bigger consideration is the effort that will be required to maintain the port over time. Especially for ports with few users, it is important to keep the effort of maintenance small. One very important part of that is to have a builder, so that the dashboard tells us when the system is broken. If there had been a Plan 9 builder, the ssa thing would have been trivial to find out about and deal with. A secondary but still important part of that is to have someone who is willing to take care of issues specific to that system, like when new functionality is added in the runtime and system-specific routines must be written.

I have been somewhat disappointed in the Plan 9 port. There is no builder, but worse no one is driving the development, I don't believe we have ever gotten to the point where the whole tree runs, and when Plan 9-specific problems come up, they often linger for months. If we evict Plan 9 support from the tree, it would be for these reasons, not (directly) due to lack of users.

Dragonfly is a similar situation, but it only just started being integrated into the tree right before the Go 1.2 freeze. (I would have liked some discussion beforehand, so that we could have held it all until afterward, but that's a different topic.) In contrast, the Plan 9 port has had years.

I propose that we leave the code in the tree for the Go 1.3 development cycle, but we reexamine at the end of the cycle. If a port has no builder running by then, we will think very seriously about removing it. During the Go 1.3 cycle, it is okay to break ports without builders, and it is okay not to fix them. Until there's a builder, the development work must be on the person or people driving the port.

Any new ports, including the Solaris port, should have all.bash working completely before being integrated, and there should be a builder running (and presumably failing, because the main repo is missing the port) before we start moving code into the main repo.

Thanks.

Russ

[Robert Griesemer]

+1 for this plan going forward

--

[ron minnich]

If you don't care about the Go port to Plan 9 or other research
kernels, stop here.

Short form: for new ports, I think you should require a working
builder for 3 months and then take code into the repo. I think you
should require working builders for keeping code in the repo and
version bumps. I think taking code in and *then* asking for a builder
is backwards and is not working. Requiring a builder first would
require a demonstration of a level of effort from interested
communities that is appropriate for a port.

Somebody pointed this discussion out to me and I thought I would
relate experiences from 3 different perspectives (ports). I've not
been reading this list for a bit, got consumed with other work.

I first discussed the Go port to Plan 9 with some Bell Labs friends in
summer 2010 -- yep, that's right, over 3 years ago. We felt strongly
that a move to a Go-based user space was the right direction. Since
that time I've also worked on the 64-bit plan 9 port (started 2011)
and, more recently, I've provided some advice to the person doing the
Akaros port.

Anyone from the Plan 9 world who knows a thing or two can get the
basic toolchain going in a couple hours or a day, and will probably
hit hello, world in a week or two if they're really working at it.
That very ease is what fools people. Passing go test is a much bigger
deal. I had one guy claim to me that he was passing go test; I had to
point out to him that in fact he was not, and was less than 20%
through it based on the output he showed me. He realized later that go
test was causing a plan 9 kernel panic. He mistook failure for
completion. It's surprisingly easy to do because in the Plan 9 world
build tasks run in seconds and minutes, not hours. For reference, on
our branch of NIX, a full library/bin/kernel build took under 2
minutes not counting ghostscript. Also, when a reboot takes less than
a second it's easy to miss that it was a reboot, not a new command
prompt. Hence, a 'go test' that runs a very long time is just not part
of the normal experience.

For the 64-bit Plan 9 port we hit interesting issues in Go test. We
ended up adding system calls to set/get TLS using the %gs register
(which made the runtime code incredibly simple and in fact let us
share code with the Linux port); a system call to get the nanosecond
timer (which made time far more accurate and removed a 'leaking file
descriptor' issue in process spawn); and fixed an FP constant in the
standard Plan 9 library that had been wrong for at least 20 years.
That's just a few things, there were many more.

We've been at it long enough to calibrate what it takes to port Go to
a non-Unix-like kernel. The classic toolchain (6g/6c/6l/ta) takes at
most a week, and sometimes a day. Hello world is a tad longer. Passing
go test? Months of sustained effort. In fact, it's much more like a
year than a month. The Akaros effort, which is being done by a very
smart guy, passed the 6 month mark some time ago. And it's required
changes to Akaros. Again, if the target kernel is not a lot like a
Unix kernel, you're going to hit some messy issues.

The Plan 9 efforts of which I am aware have not finished go test (I'm
happy to be shown to be wrong), but for sure they have not got a
builder done. I've talked to several folks about it and the initial
enthusiasm is quickly followed by "oh wow that's more complicated than
I thought" followed by silence. We had planned to do it for 64-bit
Plan 9 once we passed go test. We were close but there were remaining
issues with async signals.

Hence this recommendation. I think you need to invert the ordering of
operations. To date, you've put code in the repo and then waited for
people to finish up a builder. That seems to be causing you
unnecessary trouble. I'd suggest the following:

- to get code into the repo, you have to show a working builder for three months
- to keep code in the repo, you have to sustain that effort. If the
builder breaks for, say, three months, code gets removed
- to make the bump to the next version, you have to show a working
builder for -- say -- 6 months, or the time from the last version,
whichever is less.

I think this would help because it would (hopefully) reduce the
trouble the Go team has had with orphan code, and would focus the new
efforts on getting a builder going s the price of entry.

ron

[Russ Cox]

People seem in general agreement, so I wrote a draft policy at golang.org/wiki/PortingPolicy. Comments and questions can stay here in this thread for now.

The policy means, among other things, that plan9 and dragonfly are on the chopping block if there is no builder running and passing by the Go 1.3 freeze (March 1).

Russ

[first.pl...@googlemail.com]

Specifically, I propose deleting plan9 and dragonfly support.  Neither has many users, nor are they running a builder at http://build.golang.org/.

 
for a distributed operating system the number of users is irrelevant, compared to the number of installations.
i may be the only user of my plan9 installation, but i serve a large backend with that single grid of a few machines.
thats a bit (overgeneralizingly) like kicking x86_32 out of the linux kernel because its number of CPUs in the wild is not high enough, compared to ARM.

and with the BSD`s they already lack maintainers for everything - please mail the mailinglists and ask. i am positively sure they will offer setting up the regression thingie.
most of them dont use something like this because it runs google. i only have a throw-away account in here, too. because data ends up on google.

kind regards,
me
(no use in replying or sending an email)

[Russ Cox]

On Mon, Dec 2, 2013 at 11:30 AM, <first.pl...@googlemail.com> wrote:

Specifically, I propose deleting plan9 and dragonfly support.  Neither has many users, nor are they running a builder at http://build.golang.org/.

the message you are replying to is stale. the new policy is at golang.org/wiki/PortingPolicy, as mentioned in this very thread.

russ

[Jeff Sickel]

On Monday, December 2, 2013 10:39:40 AM UTC-6, rsc wrote:

the message you are replying to is stale. the new policy is at golang.org/wiki/PortingPolicy, as mentioned in this very thread.

The plan9 386 builds started failing due to libbio changes and subsequent dependencies spread across the src tree.  There are two patches that address the issue (14604047 & 15750047).  There are also two other patch sets that are most likely required (14521056, 34640045).  With these patches applied Plan 9 386 builds and passes most of the tests.

Addressing amd64 and other builds will start to be an issue as cmd/dist &| go_bootstrap install go and gofmt in $GOROOT/bin, which without name space changes would just overwrite the 386 build.  The pkg tree does a very nice job at handling $objtype requirements, it would be nice to see the final install phase do the same.

jeff