CLs for the Solaris Port

[Aram Hăvărneanu]

Here are the CLs for the Solaris port:

35900045 cmd/dist, go/build: add support for GOOS=solaris

35980043 cmd/ld, cmd/{6,8}l, lib9, libmach: add support for GOOS=solaris

35990043 runtime: add support for GOOS=solaris

36000043 syscall: add support for GOOS=solaris

36010043 syscall: generate zfiles for GOOS=solaris

36020043 os, os/exec: add support for GOOS=solaris

36030043 net: add support for GOOS=solaris

36040043 all: add solaris build tags

Of course, these will wait after Keith's and Rsc's changes are merged

first, but everyone is welcome to review in the meantime.

They are fairly independent; 35900045 is a prerequisite for all the

others, but other than that they stand on their own. They have only

been tested together, however, so it would be great to submit them at

the same time, once each of them is LGTMd.

Special thanks to Dave Cheney, Shenghou Ma, Devon H. O'Dell, Richard

Lowe, Bryan Cantrill, Joshua M. Clulow, Brendan Gregg, Jonathan Perkin,

Neil Houston; they all contributed in various capacities, either with

code or valuable advice.

-- 

Aram Hăvărneanu

[Aram Hăvărneanu]

I hoped the CL numbers automatically change into URLs, but apparently not:

https://codereview.appspot.com/35900045

35900045 cmd/dist, go/build: add support for GOOS=solaris

https://codereview.appspot.com/35980043

35980043 cmd/ld, cmd/{6,8}l, lib9, libmach: add support for GOOS=solaris

https://codereview.appspot.com/35990043

35990043 runtime: add support for GOOS=solaris

https://codereview.appspot.com/36000043

36000043 syscall: add support for GOOS=solaris

https://codereview.appspot.com/36010043

36010043 syscall: generate zfiles for GOOS=solaris

https://codereview.appspot.com/36020043

36020043 os, os/exec: add support for GOOS=solaris

https://codereview.appspot.com/36030043

36030043 net: add support for GOOS=solaris

https://codereview.appspot.com/36040043

36040043 all: add solaris build tags

Thanks again.

[Aram Hăvărneanu]

Answers to some preemptive questions:

I will maintain the port and Dave Cheney will maintain the builders (with

myself for backup). Solaris/386 is not planned. I might make it work on

Solaris 10 if somebody donates a builder (somebody promised that). Cgo

support is planned. A more performant network poller remains to be done

(now it's using a select-based poller from gccgo).

[minux]

Wonderful! Perhaps the next step is applying for a build key and make

solaris-amd64 build appear in build.golang.org.

On Mon, Dec 2, 2013 at 3:08 PM, Aram Hăvărneanu <ar...@mgk.ro> wrote:

I hoped the CL numbers automatically change into URLs, but apparently not:

https://codereview.appspot.com/35900045

35900045 cmd/dist, go/build: add support for GOOS=solaris

https://codereview.appspot.com/35980043

35980043 cmd/ld, cmd/{6,8}l, lib9, libmach: add support for GOOS=solaris

I went through this CL, and besides some cosmetic comments, LGTM.

 

I remember Russ doesn't like cmd/{6,8}l, and he recommends spell it out in full,

also I suggest we order them in dependency order:

lib9, libmach, cmd/ld, cmd/8l, cmd/6l: add support for GOOS=solaris

this is a pretty long list, I wonder if we could split out the lib9/libmach part and

focus on linker support in this CL.

Also, considering that Russ wants his liblink CLs in first, we probably will need

to wait and merge after those CLs are in (golang.org/s/go13linker).

https://codereview.appspot.com/35990043

35990043 runtime: add support for GOOS=solaris

https://codereview.appspot.com/36000043

36000043 syscall: add support for GOOS=solaris

Procs don't support floating point argument, right? Should we document that?

(This is similar to windows, and we have issues regarding FP argument/results

handling in syscall)

https://codereview.appspot.com/36010043

36010043 syscall: generate zfiles for GOOS=solaris

https://codereview.appspot.com/36020043

36020043 os, os/exec: add support for GOOS=solaris

https://codereview.appspot.com/36030043

36030043 net: add support for GOOS=solaris

why TestSelfConnect hangs on solaris?

what about TestTransportPersistConnLeak?

(bad link on the URL in src/pkg/net/http/transport_test.go line 745) 

should we use `/usr/sbin/ndd /dev/tcp tcp_conn_req_max_q` to query

the real maxListenerBacklog?

https://codereview.appspot.com/36040043

36040043 all: add solaris build tags

I regret we don't have a unix build tag builtin, too late to add one though.

[Dave Cheney]

Yup, that is on my plate. 

I'll wait a few days for the smoke to clear from Carl and Keith's changes before introducing a line of red fails on the dashboard. 

--
 
---
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Aram Hăvărneanu]

> I remember Russ doesn't like cmd/{6,8}l, and he recommends spell it out
> in full, also I suggest we order them in dependency order: lib9, libmach,
> cmd/ld, cmd/8l, cmd/6l: add support for GOOS=solaris
>
> this is a pretty long list, I wonder if we could split out the
> lib9/libmach part and focus on linker support in this CL.

I agree, done.

> Also, considering that Russ wants his liblink CLs in first,
> we probably will need to wait and merge after those CLs are in
> (golang.org/s/go13linker).

Yes, ackowledged in my first post, that's why I haven't hg mail'ed yet.

> Procs don't support floating point argument, right? Should we document
> that?

They don't support floating point, but we don't need floating point. If
we ever need floating point it should be pretty easy to add. I don't
think we should document this low-level trivia. In fact, I'd be happy
with not exposing this stuff at all.

> why TestSelfConnect hangs on solaris? what about
> TestTransportPersistConnLeak?

I don't know, but it seems netpoll related. Since we are going to change
it pretty soon (even before integration maybe), I didn't spend too much
time on this.

We have an events-port version of netpoll in testing.

> I regret we don't have a unix build tag builtin, too late to add one
> though.

Me too. Is it too late to add unix/bsd/posix build tags though? It's
(obviously) too late to add support for magic file_unix.go files, but
build tags? Since we have access to almost all Go code in existence,
it would be pretty easy to assess the effect of this.

--
Aram Hăvărneanu

[Aram Hăvărneanu]

Everything uses liblink now.

Dave, can you please enable our builder? Thanks.

[Dave Cheney]

solaris11 builder is up and failing

On Wed, Dec 11, 2013 at 7:42 AM, Aram Hăvărneanu <ar...@mgk.ro> wrote:
> Everything uses liblink now.
>
> Dave, can you please enable our builder? Thanks.
>

[minux]

On Dec 10, 2013 4:56 PM, "Dave Cheney" <da...@cheney.net> wrote:
> solaris11 builder is up and failing

Great! we are one step closer!

[Aram Hăvărneanu]

> solaris11 builder is up and failing

Thanks, what about the smartos builder?

--
Aram Hăvărneanu

[Russ Cox]

What is smartos?

[Dave Cheney]

As I understand it, smartos is a distribution of Illuminos, which is a fork of opensolaris, which itself is an early fork of Solaris 11.

I'm sure someone will correct me on this.

> On 11 Dec 2013, at 12:53, Russ Cox <r...@golang.org> wrote:
>
> What is smartos?

[Rob Pike]

Smartos are the opposite of stupidos.

-rob

[Aram Hăvărneanu]

On Wed, Dec 11, 2013 at 2:53 AM, Russ Cox <r...@golang.org> wrote:
> What is smartos?

It's a distribution of illumos, which is the open-source successor of
OpenSolaris and also the primary target of this port. Oracle Solaris
happens to work too, binary compatibility and all, but it's not the
primary target.

--
Aram Hăvărneanu

[Aram Hăvărneanu]

On Mon, Dec 2, 2013 at 10:17 PM, minux <minu...@gmail.com> wrote:
> should we use `/usr/sbin/ndd /dev/tcp tcp_conn_req_max_q` to query
> the real maxListenerBacklog?

Unstable interface I'm afraid, both ndd and the underlying ioctl. I'll
just use syscall.SOMAXCONN, like some other ports. (The real observed
values I see in practice are equal to syscall.SOMAXCONN, so we don't
use performance).

--
Aram Hăvărneanu

[minux]

One more thing (CL), we need to add solaris to cmd/api, and then solve

possible failures due to API testing.

(api/* have some entries that are universal before solaris, I suspect they

might not be after solaris is supported)

[Aram Hăvărneanu]

The reason I haven't tackled cmd/api yet is that I am not certain the
current syscall API is the right one. In particular, I want to remove
the Windows-style syscall.Syscall completely.

--
Aram Hăvărneanu

[minux]

yeah, my intent is to try to match the syscall api to other Unix systems.

on the syscall.Syscall issue, do you have specific plans to eliminate them? generate wrappers for each libc call? or just unexport the Syscall api?

[Aram Hăvărneanu]

As an externally-visible change, just unexport syscall.Syscall and
syscall.RawSyscall (which are not used and don't work correctly anyway).

In the internal implementation, I want to remove the usage of dlopen
and dlsym. The linker does lazy loading anyway. This is very easy to
do from C (as we do it in the runtime package). I'd like a way to do
it from Go, but perhaps C will suffice.

--
Aram Hăvărneanu

[minux]

Yeah, I agree. mksyscall.pl should just generate plan 9 c source file

that have required "#pragma dynimport", so that we don't need to use

dlopen in syscall.

can the linker drop unreferenced dynimports? I'd prefer a Go hello world

program doesn't pull in networking symbols from libc.

[Aram Hăvărneanu]

I've updated the CLs to the current code. A major change is that I
have replaced the select-based poller with an event ports version and
I also solved the equivalent of issue 6955 (for gc and event ports,
not gccgo and select).

--
Aram Hăvărneanu

[Mikio Hara]

I've just tried to run tip w/ CLs, 36000043, 36010043 and 36030043, on
solaris/amd64(*) and it passes both short and long-mode tests. I think
solaris port is almost ready to land after runtime/netpoll
review+minor editorial fixes.

*) SunOS vm4 5.11 oi_151a8 i86pc i386 i86pc Solaris

[Dave Cheney]

Thanks for testing Mikio. I hope solaris can land in 1.3 cycle.

[Mikio Hara]

On Mon, Feb 24, 2014 at 10:32 AM, Dave Cheney <da...@cheney.net> wrote:

> Thanks for testing Mikio. I hope solaris can land in 1.3 cycle.

sigh, i can't get a stable test result, still netpoll related tests
fail intermittently.
it needs a deep review to runtime/netpoll w/ experts.

[Aram Hăvărneanu]

> sigh, i can't get a stable test result, still netpoll related tests fail
> intermittently. it needs a deep review to runtime/netpoll w/ experts.

Running go test net in a loop has been very reliable for me, both on
my 2-core laptop and on a 24-way machine.

for i in `seq 30`; do go test net || break; done
ok net 20.073s
ok net 20.356s
ok net 19.911s
ok net 20.162s
ok net 20.160s
ok net 19.995s
ok net 20.180s
ok net 29.973s
ok net 25.109s
ok net 19.795s
ok net 20.055s
ok net 20.084s
ok net 20.086s
ok net 19.938s
ok net 24.910s
ok net 20.093s
ok net 24.745s
ok net 20.114s
ok net 20.168s
ok net 24.943s
ok net 20.189s
ok net 19.923s
ok net 20.305s
ok net 20.135s
ok net 20.332s
ok net 19.953s
ok net 20.224s
ok net 20.381s
ok net 20.212s
ok net 20.159s
: oos:aram;

: oos:net; for i in `seq 100`; do net.test -test.short || break; done
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
: oos:net;

Somewhat mysteriously, running all.bash sometimes fails in net. I can't
reproduce without running all.bash.

It panics as it violates an assertion I've put in there. That assertion
can only fail if there's a race between closing file descriptors and
rearming them in the netpoller.

Thinking about this, I've probably been too cautious and it's probably
safe to remove that assertion. The worst that could happen is a spurious
netpoller wake.

I won't do that however, at least not yet. I want to understand the
problem better before doing anything, and if possible, fix the underlying
cause.

--
Aram Hăvărneanu

[Aram Hăvărneanu]

Everything except 62010043 (which I'm not sure we need) is integrated.

Investigating the race I talked in previous mail.

--
Aram Hăvărneanu