GOOS=fuchsia

[David Crawshaw]

For about a year now I have been working on a port of Go to the
Fuchsia operating system:

https://fuchsia.googlesource.com/third_party/go/

The port has been quite stable for the past three months, with a
long-planned rename going in this week. So I would like to look at
bringing the port upstream into the Go repository, as GOOS=fuchsia.

As Fuchsia is not a Unix operating system, there are a few surprises
in this port. I would like to run them by you all to make sure there
are no strong objections.

First, the kernel interface focuses on several fundamental object
types each with an associated set of system calls. These objects and
syscalls are all prefixed with zx_ in the C library for Zircon (the
Fuchsia micro-kernel), and this lends itself really nicely to a Go
package named syscall/zx. The various objects are called zx.Socket,
zx.Channel, zx.VMO, and so on, with the syscalls specific to each one
are methods on these types.

As the operating system does not provide a syscall interface for
files, the file system is implemented as a protocol spoken over
zx.Channel objects to a file system process. Inside the Fuchsia libc
this is called libfdio, and we implement this protocol in
syscall/zx/fdio. This is used by the syscall, os, and net packages to
provide a layer of file descriptor emulation for the rest of the
standard library.

My intention is the syscall/zx and syscall/zx/fdio packages will be
public packages as they are generally useful when writing programs
targeting fuchsia. Just as the syscall package contains some symbols
only available on some OSs, I intend for syscall/zx and packages under
it to only be available when GOOS=fuchsia. Like the syscall package,
they won't be subject to the Go 1 compatibility guarantee.

Just about everything under syscall/zx is necessary for implementing
the standard library, so I believe it belongs in the go tree, rather
than go.sys.

The syscall/zx package is the largest unusual component of the port.
Any objections?

There are some other small novelties:

- The runtime depends on syscall/zx, as zx has no dependencies and
most of its contents are useful to the runtime. It is a small
modification to cmd/go so the runtime can depend on it. This would be
the first package outside runtime/internal that runtime depends on.

- The fuchsia kernel is running on GCE, with support for network and
disk coming very soon. That's how I intend to run builders for
build.golang.org.

- The OS has not reached version 1. That means system calls may change
and older releases of Go won't be able to build binaries for newer
versions of Fuchsia. This isn't entirely new, it happened recently on
macOS with a system call change, but it is far more likely with an OS
yet to have its first release.

- The os/user package will be stubbed out for a while, as it's not
clear how to give this package meaning on Fuchsia. (GOOS=android is in
a similar state.)

- Some features temporarily only work with cgo, in particular,
os/exec. There's no fundamental reason why this is true, just code
that needs writing. I hope to merge the port before making
CGO_ENABLED=0 fully functional. This is a reasonably high-priority
goal though, as Fuchsia is not yet self hosting and it would be nice
to be able to build Go on Fuchsia before the C/C++ toolchain is ready.

A piece of good news, Fuchsia is ELF-based, so very few changes are
needed to the compiler and linker.

This port is quite important to Fuchsia, as Go is used for some core
services in the operating system. In particular, the network stack is
implemented Go, so for any program to make a TCP connection on
Fuchsia, we need the Go port to be working:

https://fuchsia.googlesource.com/garnet/+/master/go/src/netstack/

If you are interested in more details of the kernel, there is some
documentation:

https://fuchsia.googlesource.com/zircon

Assuming there are no major concerns, I'll start squashing CLs from
our Go port into something ready for code review here. Thanks,

David

[moeh...@google.com]

- The runtime depends on syscall/zx, as zx has no dependencies and
most of its contents are useful to the runtime.

Is there anything in the implicitly generated init functions in zx that calls

runtime functions?

 

- The fuchsia kernel is running on GCE, with support for network and
disk coming very soon. That's how I intend to run builders for
build.golang.org.  

Could we also add a trybot please? Its nice to be able to test not so common 

architectures through trybots especially if they have some not so common properties

like the runtime depending on a syscall package.

- Martin

[Henrik Johansson]

I must say that I am impressed! I thought fuchsia was not at all far enough along to bother with a Go port. Good stuff!

--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[minux]

Hi David,

On Tue, Sep 12, 2017 at 7:06 PM, David Crawshaw <craw...@golang.org> wrote:

My intention is the syscall/zx and syscall/zx/fdio packages will be
public packages as they are generally useful when writing programs
targeting fuchsia. Just as the syscall package contains some symbols
only available on some OSs, I intend for syscall/zx and packages under
it to only be available when GOOS=fuchsia. Like the syscall package,
they won't be subject to the Go 1 compatibility guarantee.

Just about everything under syscall/zx is necessary for implementing
the standard library, so I believe it belongs in the go tree, rather
than go.sys.

The syscall/zx package is the largest unusual component of the port.
Any objections?

I wonder why we don't just use the syscall the provide API from both syscall/zx and syscall/zx/fdio?

Are you worried that it will make syscall vastly different on differently OSes?

But that's already the case for Unix, Windows and Plan 9.

Perhaps an alternative would be to add new packages to runtime/internal/syscall/zx

and then re-export them from the real syscall package.

Regarding Go 1 compatibility guarantee, we can explicit label the zx port as unstable

and exclude from the guarantee until Fuchsia 1.0 is released.

[David Crawshaw]

On Wed, Sep 13, 2017 at 6:41 PM, minux <mi...@golang.org> wrote:
> Hi David,
>
> On Tue, Sep 12, 2017 at 7:06 PM, David Crawshaw <craw...@golang.org> wrote:
>>
>> My intention is the syscall/zx and syscall/zx/fdio packages will be
>> public packages as they are generally useful when writing programs
>> targeting fuchsia. Just as the syscall package contains some symbols
>> only available on some OSs, I intend for syscall/zx and packages under
>> it to only be available when GOOS=fuchsia. Like the syscall package,
>> they won't be subject to the Go 1 compatibility guarantee.
>>
>> Just about everything under syscall/zx is necessary for implementing
>> the standard library, so I believe it belongs in the go tree, rather
>> than go.sys.
>>
>> The syscall/zx package is the largest unusual component of the port.
>> Any objections?
>
>
> I wonder why we don't just use the syscall the provide API from both
> syscall/zx and syscall/zx/fdio?
> Are you worried that it will make syscall vastly different on differently
> OSes?
> But that's already the case for Unix, Windows and Plan 9.

It is possible, however I believe this is a much better package
organization. The zx package comes out with a nice exported interface.
All of the objects in a fuchsia syscall package would have ugly names
like syscall.ZXSocket, syscall.ZXChannel, and so on, and the fdio
package contains a significant body of code that can be isolated
nicely. The syscall package in the Fuchsia port continues to have a
lot of the Unix symbols, as part of Fuchsia's partial POSIX emulation,
so the zx symbols get lost in the documentation.

The nicer interface is valuable because I expect it will be more
common for a program written for Fuchsia to use the zx package then it
is for a typical Unix program to use the syscall package. Several of
the kernel primitives are convenient IPC objects.

Put another way: syscall/zx and syscall/zx/fdio are the packages for
Fuchsia I would build in the go.sys repository. It just happens to be
the case that we also need them for the standard library because we
have no syscall package implementation yet.

> Perhaps an alternative would be to add new packages to
> runtime/internal/syscall/zx
> and then re-export them from the real syscall package.

I think this would be strictly worse than just dumping everything in
syscall, because there would be two ways of naming things for those of
us who work on the stdlib.

> Regarding Go 1 compatibility guarantee, we can explicit label the zx port as
> unstable
> and exclude from the guarantee until Fuchsia 1.0 is released.

That would be ideal. The syscall package already has a special place
in the guarantee, which I believe an unstable zx package fits under
nicely: https://golang.org/doc/go1compat#operating_systems

[David Crawshaw]

On Wed, Sep 13, 2017 at 3:19 AM, moehrmann via golang-dev
<golan...@googlegroups.com> wrote:
>> - The runtime depends on syscall/zx, as zx has no dependencies and
>> most of its contents are useful to the runtime.
>
> Is there anything in the implicitly generated init functions in zx that
> calls
> runtime functions?

Not yet. If it did, I'm not sure that would be a problem, as long as
the appropriate parts of the runtime are initialized. It certainly
could be confusing.

>>
>> - The fuchsia kernel is running on GCE, with support for network and
>> disk coming very soon. That's how I intend to run builders for
>> build.golang.org.
>
> Could we also add a trybot please? Its nice to be able to test not so common
> architectures through trybots especially if they have some not so common
> properties
> like the runtime depending on a syscall package.

I hope so. Hopefully GCE will make that straightforward, as the most
common obstacle to try-bots is enough machines.

[Anthony Martin]

My suggestion is to try as hard as you can to minimize what goes into
the syscall package. There's a bunch of stuff in the Plan 9 port that
I wish we never put in syscall.

I think of the syscall package as the bare minimum that's necessary
to support the os and os/exec packages (and maybe net too). All other
low-level code should be somewhere else, e.g., golang.org/x/sys/foo.

Emulating the Unix ports in syscall is a mistake, in my opinion.

Is there a public repo with your work so far?

Cheers,
Anthony

[David Crawshaw]

Yes, from my first email:
https://fuchsia.googlesource.com/third_party/go (Note that the global
rename s/mx/zx/ is not in this repository yet.)

I agree with your point about minimizing what goes into syscall. All
handwritten code in the fuchsia port's syscall tree is used by the os
package, and I would estimate about 90% of the generated syscall
wrappers. Some of the unused syscalls are extremely rare, I'll look at
removing those.

> Cheers,
> Anthony

[bi...@opptygo.com]

I'm curious what the criteria is for addition of new OS support in Golang?  If I start writing a new open source OS in my garage which has not reached release 1 and is not present in any available product, will I be able to add support for it to Golang?  I'm not aware that Google has even made any official public statements about its plans for Fuchsia?

I'm glad to hear parts of Fuchsia are written in Go, though.  It would be great if Google had an OS, especially a mobile OS, for which Go was the (or at least a) first class development environment.  In the absence of any official statements from Google, though, it's not clear whether Fuchsia will be that.

I'm also curious why Go is only used for "some core services" and not the vast majority of Fuchsia?

[Ian Lance Taylor]

On Thu, Sep 14, 2017 at 8:26 AM, <bi...@opptygo.com> wrote:
>
> I'm curious what the criteria is for addition of new OS support in Golang?
> If I start writing a new open source OS in my garage which has not reached
> release 1 and is not present in any available product, will I be able to add
> support for it to Golang? I'm not aware that Google has even made any
> official public statements about its plans for Fuchsia?

(I don't know that much about Fuschia.)

The Go project's policy for new ports can be found at
https://github.com/golang/go/wiki/PortingPolicy . But I don't think
we've ever discussed what to do about a project from a garage. I
think we would first want to be persuaded that the port was going to
survive, and that we wouldn't just be removing it again in a year.

Ian

[bi...@opptygo.com]

On Thursday, September 14, 2017 at 1:51:55 PM UTC-4, Ian Lance Taylor wrote:

>>  I think we would first want to be persuaded that the port was going to survive, and that we wouldn't just be removing it again in a year.

And how do we know that about Fuchsia, especially given what seems to be the minimal official communication about it?  After all, it's not like Google has never scrapped a project, and projects with no users are particularly easy to scrap.  If Uber or Baidu proposed integrating a new OS with no official communication, no committed products and no production-ready release timeline or target, would we accept that?  (Apologies if those things exist for Fuchsia and I just haven't been able to find them.)

[Ian Lance Taylor]

Those are good questions. Note that we have not accepted the Fuschia
port as yet.

I think it does clearly count in David's favor that he is an existing
significant contributor to the project. Even if Fuschia disappears,
it seems likely that David would clean up after it.

Ian

[David Crawshaw]

Typically, the Go community writes down policy in response to specific
issues that impact a release. For example, the current porting policy
came about because some OS ports lacked builders, and it was a big
problem when it came to making a release. So we got a wiki page saying
that new ports need a working builder before they can be integrated.

I believe the issue you are raising is hypothetical. That is, no one
has argued that an OS port should be excluded from the Go tree because
the OS is not significant enough. (Unless you are saying that
specifically Fuchsia should be excluded?) A concrete policy to address
ephemeral operating systems would be easier to write well when we are
faced with a concrete example. Personally, I don't expect it to come
up. Running builders is not easy. The only good way to do it is as a
VM, and if a tiny OS project can get that working well, then I would
be impressed and be happy to see their port integrated into Go. So our
current policy does most, if not all, of the work.

On Fuchsia specifically: it doesn't have release numbers. Builds from
HEAD are quite stable. You can run it today the same way I do, all the
documentation I use to do development is public:
https://fuchsia.googlesource.com/magenta/+/master/docs/targets/acer12.md
I usually netboot the acer with the extra details here:
https://fuchsia.googlesource.com/zircon/+/HEAD/docs/getting_started.md#network-booting
There are no products running Fuchsia, but I believe that is true of
other ports too, like OpenBSD. And if Fuchsia disappeared, I would
delete the Go port (if for nothing else then for the sheer joy of
deleting code).

[john...@gmail.com]

From a quick look at the Fuchsia repository on GitHub, Go is the # 3 language, behind C and C++ and ahead of Dart and Python.

Also, from what David said above, Go is likely to be the first language that will self-host on Fuchsia; they don't have the C/C++ tool chain working (or working well enough) under Fuchsia yet.

As for why not use it for more of the kernel, frankly I think it's wonderful that they're using it for anything! Kernel programming is not like application programing.

[Daniel Morsing]

Cool stuff. Just one question for now.

Does the fuchsia "syscall/zx/fdio" concepts and interface differ enough from the "os" file interface that it cannot be used for this? You could for example do the fuchsia-specific setup with the "syscall/zx" package and then pass it into os.NewFile to get a file abstraction.

[bi...@opptygo.com]

On Thursday, September 14, 2017 at 3:08:36 PM UTC-4, John Roth wrote:

>>As for why not use it for more of the kernel, frankly I think it's wonderful that they're using it for anything! Kernel programming is not like application programing.

Good point.  I can imagine that GC wouldn't be very compatible with kernel development.  Still, there are more parts to an OS than just the kernel, but maybe that's the primary focus of development at this stage.

[john...@gmail.com]

 Good point as well. From that fast look, they've got a lot of stuff that's traditionally in the kernel out in userspace, including the file systems and the network stack.

[bi...@opptygo.com]

First off, David, I'd like to say that I agree with Ian that significant contributors to the language should be given more leeway.  

For the sake of discussion and In hopes of maybe having a clearer precedent for future requests, and making it more obvious that we don't just rubber stamp requests that originate from Goggle, I'll rebut some of your points:

Typically, the Go community writes down policy in response to specific
issues that impact a release. For example, the current porting policy
came about because some OS ports lacked builders, and it was a big
problem when it came to making a release. So we got a wiki page saying
that new ports need a working builder before they can be integrated.

Makes sense.  Now, the issue is whether we want to allow support in production quality Golang for an OS that has had very little official communication shared.  What little there has been (that I've seen) characterized it as an "experiment".  So one question is whether we want to support every experimental OS integration request we may get.  The answer could be "yes, until such time as that policy causes issues that impact a release." 

Another answer could be "no, but Fuchsia is different from your average experimental OS."  If so, then how?

 

A concrete policy to address ephemeral operating systems would be easier to write well when we are
faced with a concrete example.

I would argue that Fuchsia is a concrete example of an ephemeral operating system.

all the documentation I use to do development is public:  https://fuchsia.googlesource.com/magenta/+/master/docs/targets/acer12.md 

That depends on what you mean by "to do development".  If doing development includes understanding the project vision, goals and timelines, that's clearly not all of the documentation required (which contributes to applying the "ephemeral" label above).

There are no products running Fuchsia, but I believe that is true of other ports too, like OpenBSD.

A quick search of "OpenBSD products" suggests there are several products based on OpenBSD, as well as a diverse and active community.

 

[Juliusz Chroboczek]

> I can imagine that GC wouldn't be very compatible with kernel development.

Why?

[Devon H. O'Dell]

2017-09-14 15:30 GMT-07:00 Juliusz Chroboczek <j...@irif.fr>:
>> I can imagine that GC wouldn't be very compatible with kernel development.
>
> Why?

In kernels, there are several problems with having your stuff stopped.
You usually can't go to sleep when handling an interrupt because you
mayn't get another one that wakes you up again. Also, Fuschia is a
real-time operating system, which means that it has deadlines to meet
around scheduling. With most GC schemes, these deadlines can be
unpredictable. Managing memory outside the purview of a GC can also be
problematic.

Go (or any GC'ed language) is, on the other hand, fine for
non-realtime processes running on the system. If the GC was pauseless,
perhaps these would be less of a problem. But in general it's not
worth the effort, since one of the things the OS has to do anyway at
the hardware level is memory management.

--dho

[ron minnich]

On Thu, Sep 14, 2017 at 11:58 AM David Crawshaw <craw...@golang.org> wrote:

I believe the issue you are raising is hypothetical. That is, no one
has argued that an OS port should be excluded from the Go tree because
the OS is not significant enough.

well, actually, it's happened. To me. When we proposed a NIX port, back in 2012, I was told that NIX was not really a significant enough kernel to rate the support required, even though most of the work would be borne outside the Go team. The key word there being 'most': there's always a non-zero impac on the core groupt. So, yes, this happens, and in the NIX case I understood the argument and did not push back, because the argument makes sense.

ron

  

[Devon H. O'Dell]

2017-09-14 12:54 GMT-07:00 <bi...@opptygo.com>:
>> There are no products running Fuchsia, but I believe that is true of other
>> ports too, like OpenBSD.
>
> A quick search of "OpenBSD products" suggests there are several products
> based on OpenBSD, as well as a diverse and active community.

What about Plan 9? I guess you could point out Coraid, but if there
are few products based on OpenBSD and few OpenBSD users, Plan 9 is
surely even lower.

There've been several discussions on the list to this point. The issue
has effectively boiled down to, "Will the additional system support
place a burden on developers who don't use / care about that system?"
In some cases, OSes like Plan 9 and DragonFly BSD have almost been
removed because they were not up to date and spent huge amounts of
time failing. I suspect if someone wanted to port Go to her hobby OS,
she'd have to convince the Go team that she'd be able to maintain the
port. This is totally fair, though I suspect probably easier than you
suggest in your hypothetical: this person has already demonstrated an
individual ability to write an OS capable of hosting Go, which is no
small feat. I don't suspect the "convincing" would be very hard if
this hypothetical person was serious.

Some of the same distrust about the adoption of / impetus for /
supportability of Go was present in 2009 and 2010. People had these
same arguments about Go being a project that would be directed by
Google and last at Google's whim. I ported Go to FreeBSD within the
first couple months of its release (with a lot of help from Russ and
others). Within a couple years, I didn't have time to maintain it
anymore, and other folks stepped up. I don't think I've contributed to
the language or runtime since 2012. Yet FreeBSD support isn't really a
burden.

I think nearly 8 years after the public release of the language, it's
fair to say that's probably not true. So I don't think this is a
useful argument. Maybe it's worth codifying some rules for import of a
new system, but both the infrequency and nuance of adding a new system
are good reasons for just treating it on a case-by-case basis.

--dho

[David Crawshaw]

On Thu, Sep 14, 2017 at 3:12 PM, Daniel Morsing
<daniel....@gmail.com> wrote:
> Does the fuchsia "syscall/zx/fdio" concepts and interface differ enough from
> the "os" file interface that it cannot be used for this? You could for
> example do the fuchsia-specific setup with the "syscall/zx" package and then
> pass it into os.NewFile to get a file abstraction.

The fdio package implements file descriptors for Fuchsia. (If you take
a look at the existing port, it's called mxio there.) A file
descriptor is a data structure made up of a zx.Handle to the kernel
object you communicate over, and the sub-protocol type. The syscall
package maps integer fd values onto these data structures, for the
convenience of the os package that expects to be able to meaningfully
refer to file descriptors via an int. The fdio package is also heavily
used by net, because sockets are files.

[David Crawshaw]

On Thu, Sep 14, 2017 at 3:54 PM, <bi...@opptygo.com> wrote:
> First off, David, I'd like to say that I agree with Ian that significant
> contributors to the language should be given more leeway.
>
> For the sake of discussion and In hopes of maybe having a clearer precedent
> for future requests, and making it more obvious that we don't just rubber
> stamp requests that originate from Goggle, I'll rebut some of your points:
>
>>
>> Typically, the Go community writes down policy in response to specific
>> issues that impact a release. For example, the current porting policy
>> came about because some OS ports lacked builders, and it was a big
>> problem when it came to making a release. So we got a wiki page saying
>> that new ports need a working builder before they can be integrated.
>
>
> Makes sense. Now, the issue is whether we want to allow support in
> production quality Golang for an OS that has had very little official
> communication shared. What little there has been (that I've seen)
> characterized it as an "experiment". So one question is whether we want to
> support every experimental OS integration request we may get. The answer
> could be "yes, until such time as that policy causes issues that impact a
> release."
>
> Another answer could be "no, but Fuchsia is different from your average
> experimental OS." If so, then how?

The only reason I see to oppose adding an OS port to the Go tree is
the extra work for the team working on Go. Luckily, I'm part of the
team working on Go and want to take it on, so that's not a problem.
The only added effort is finding someone willing to do code reviews,
and I asked someone else on the team who has agreed to help.

While I originally responded to the points about (what wikipedia might
call) the notability of the OS, on reflection I don't think that
matters. What matters is, as Ron pointed out, the extra work for the
people on the Go team. A hobby or experimental OS deserves a little
extra scrutiny only because it's more likely to be removed from the
tree later, which means more to do. In the case of Fuchsia I'm also
signing up for that, so it's newness shouldn't be a problem.

So I believe the criteria for adding an OS is something like: There
someone who is both willing and able to fix bugs for each release.

[Daniel Morsing]

Ah, I see fdio is an abstraction on the concept of file descriptors themselves, not just OS files.

Only issue I see there is that it would require a file descriptor table and leave the implementation open to fd leaks (like with all the other unix-likes, not a knock on fuchsia in particular). An alternative would be to make NewFile be able to take a pointer and have the clean up finalizer on the fdio structs themselves.

[Ian Lance Taylor]

On Tue, Sep 12, 2017 at 4:06 PM, David Crawshaw <craw...@golang.org> wrote:
>
> As the operating system does not provide a syscall interface for
> files, the file system is implemented as a protocol spoken over
> zx.Channel objects to a file system process. Inside the Fuchsia libc
> this is called libfdio, and we implement this protocol in
> syscall/zx/fdio. This is used by the syscall, os, and net packages to
> provide a layer of file descriptor emulation for the rest of the
> standard library.
>
> My intention is the syscall/zx and syscall/zx/fdio packages will be
> public packages as they are generally useful when writing programs
> targeting fuchsia. Just as the syscall package contains some symbols
> only available on some OSs, I intend for syscall/zx and packages under
> it to only be available when GOOS=fuchsia. Like the syscall package,
> they won't be subject to the Go 1 compatibility guarantee.
>
> Just about everything under syscall/zx is necessary for implementing
> the standard library, so I believe it belongs in the go tree, rather
> than go.sys.
>
> The syscall/zx package is the largest unusual component of the port.
> Any objections?

I'm mildly concerned by putting a not-complete-stable API under the
standard library syscall package. I know that the syscall package is
explicitly omitted from the Go 1 compatibility guarantee, but it
doesn't follow that it's OK for it to be completely unstable. When we
omitted the syscall package, we at the same time froze the syscall
package and created golang.org/x/sys as a place where we could change
things faster, and in particular where we could change things without
having to wait for the Go release cycle. I think the latter in
particular is an important concern: as Fuschia continues to evolve,
people using Fuschia syscalls directly should not have to wait for the
Go release cycle.

Also I have to say that names like syscall/zx and syscall/zx/fdio do
not to me say "Fuschia." They seem rather cryptic. And it sounds
like the names have already changed once.

So in an ideal world I think that we would not add new syscall
packages at all, but would instead add golang.org/x/sys/fuschia,
perhaps with zx and fdio packages underneath it for more convenient
naming by users of the packages. Of course we would need to access
this functionality from the standard library runtime and perhaps the
syscall package, but ideally we would do that using internal packages
that are not part of the Go standard library API.

At this point I need to say that one thing I do not understand is to
what degree the current syscall package is usable on Fuschia.

So I would like to understand whether it is possible to create
internal packages in the standard library that implement the features
needed by the standard library, and to have golang.org/x/sys/fuschia
packages that provide the useful functionality for user packages by
reaching into the standard library. Perhaps this packages could be
largely generated using build tags and go:linkname tricks, much as
golang.org/x/sys/unix is largely generated.

I'm not saying we must do things this way, but I'd like to at least consider it.

Ian

[David Crawshaw]

Thanks for the review,

Part of the benefit I believe of syscall/zx is the
potentially-completely-unstable interface is not mixed into the
syscall package. As is, the syscall package with GOOS=fuchsia
introduces very few new exported symbols, and those I believe meet a
decent standard of stability. That means there's a package-level
cordon, zx, on which we can write "here be dragons."

> Also I have to say that names like syscall/zx and syscall/zx/fdio do
> not to me say "Fuschia." They seem rather cryptic. And it sounds
> like the names have already changed once.

Names are distracting, but the short story: this change has been a few
months in the works, and approximately no-one was excited about it.
Another top-level rename like this would be extraordinary, and I would
bet against it. (I delayed this email until it was almost done, and
now wish I had waited another week until I had converted the Go port
and never mentioned it.)

I suppose the names are cryptic. The micro-kernel and a minimal set of
drivers and user-land programs to get a text terminal are bundled
together in a sub-project called Zircon. It works quite well on its
own, as a single git repository with its own makefile-based build
system, independent of the rest of Fuchsia. So the name tends to get
used often.

The short-name for Zircon is zx. The x stands for awesome.

Inside Zircon, libfido is implements the protocols for supporting a
subset of POSIX file descriptors. The C library is tiny and almost
everyone uses it via libc, so not a lot of effort was put into making
its name easily recognizable. That's a good argument for saying this
package should be internal to the Go standard library, the reason I
didn't originally suggest it is several other Go programs in Fuchsia
use it for various interesting reasons. (One is a file system
implementation, and so serves the fdio protocol.)

> So in an ideal world I think that we would not add new syscall
> packages at all, but would instead add golang.org/x/sys/fuschia,
> perhaps with zx and fdio packages underneath it for more convenient
> naming by users of the packages. Of course we would need to access
> this functionality from the standard library runtime and perhaps the
> syscall package, but ideally we would do that using internal packages
> that are not part of the Go standard library API.

Answered further below,

> At this point I need to say that one thing I do not understand is to
> what degree the current syscall package is usable on Fuschia.

Some of the higher-level wrappers work now, such as syscall.Open,
syscall.OpenAt, syscall.ParseDirent. (That's where the fake fd mapping
table lives, using fdio.)

Others will work at some point, such as syscall.Pipe (which would use
fdio to create a Unix-looking pipe out of a zx.Socket object).

Even more could theoretically work, such as the network syscalls that
look like libc functions such as Accept4. However I skipped over those
making the net package work, by directly calling into the fdio
package. I think it makes the net implementation better.

Unix-isms such as Getpid, Getgid, etc will never be meaningful.
Linux-isms are way out.

> So I would like to understand whether it is possible to create
> internal packages in the standard library that implement the features
> needed by the standard library, and to have golang.org/x/sys/fuschia
> packages that provide the useful functionality for user packages by
> reaching into the standard library. Perhaps this packages could be
> largely generated using build tags and go:linkname tricks, much as
> golang.org/x/sys/unix is largely generated.
>
> I'm not saying we must do things this way, but I'd like to at least consider it.

I think that is possible. It will complicate the plumbing through the
Fuchsia build system in a couple of ways which is why I haven't
considered it until now. In particular, it has been very useful to
have one place to update Go system calls that a standard call to gn,
the build tool, would propagate through the entire system. To keep
that property the act of copying golang.org/x/sys into the standard
library will have to be part of the Fuchsia build system.

So it will be some work, but I don't see any major problems yet. I
will spend some time experimenting and see if any big problems come
up.

On the topic of the src/vendor/golang_org tree, looking at some git
history it appears these files are updated by writing new commits,
with messages that do not always mention what commit hash in the
original repository the code came from. (Not to pick on anyone in
particular, here's the first example that comes up in a reverse
search: https://go-review.googlesource.com/40711, many of the other
upstream references are curtailed-hashes) That seems odd given that
git is quite good at merging, and I would expect these update CLs to
be merge CLs (two parent commit hashes, one the top of the go
repository, the other the top of the other repository being merged
in). Do we avoid that for some reason? A gerrit limitation?

[paulo....@gmail.com]

Oberon was quite successful workstation OS used during the 90's at ETHZ.

So nothing forbids an OS implemented in Go to eventually achieve the same feature level than Native Oberon System 3 with its Gadgets toolkit.

http://www.progtools.org/article.php?name=oberon&section=compilers&type=tutorial

If you enjoy reading OS source, the book for the complete OS was revised in 2013 (originally from 1992) and the Ceres CPU re-targeted to an FPGA, so
anyone can build a full Oberon based computer (http://oberonstation.x10.mx/).

https://www.inf.ethz.ch/personal/wirth/ProjectOberon/index.html

http://www.projectoberon.com/

Or maybe a 32-bit ARM7 board instead.

https://www.elektormagazine.com/labs/easy-sceptre-programming-with-oberon-07

In these kind of OSes the GC is just yet another OS service (https://www.inf.ethz.ch/personal/wirth/ProjectOberon/Sources/Kernel.Mod.txt).

Naturally I am picking Oberon as an example, given its influence in Go, but there are quite a few examples of successful GC enabled OSes on research labs all the way back to Mesa/Cedar at Xerox PARC, which failed the transition to the industry more due to politics than any technical challenge.

See Joe Duffy's keynote at RustConf 2017 about the internal hurdles to fight against prejudice.

So looking forward to which role Go will play in Fuchsia beyond the TCP/IP stack.

--

Paulo

[Juliusz Chroboczek]

>>> Why?

>> In kernels, there are several problems with having your stuff stopped.
>> You usually can't go to sleep when handling an interrupt because you
>> mayn't get another one that wakes you up again.

> Oberon was quite successful workstation OS used during the 90's at ETHZ.

I was actually thinking about the MIT/Symbolics Lisp Machines, which had
all of the OS written in Lisp. Even the GC, if I'm recalling correctly.

I'll see if I can find any papers about the Oberon OS.

-- Juliusz

[paulo....@gmail.com]

You can start by following the links I provided, the ones from my website have lots of screenshots about how it used to look like,
including relevant papers.

Oberon and Lisp Machines are just two examples, there are quite a few more.

--
Paulo

[Florian Weimer]

On 09/15/2017 11:55 PM, Juliusz Chroboczek wrote:
>>>> Why?
>
>>> In kernels, there are several problems with having your stuff stopped.
>>> You usually can't go to sleep when handling an interrupt because you
>>> mayn't get another one that wakes you up again.
>
>> Oberon was quite successful workstation OS used during the 90's at ETHZ.
>
> I was actually thinking about the MIT/Symbolics Lisp Machines, which had
> all of the OS written in Lisp. Even the GC, if I'm recalling correctly.

Note that these old systems used GC merely as a programmer convenience,
and not as a way to (dynamically) ensure memory safety. The Common Lisp
standard defines quite a few things which are unsafe and does *not*
require that safe alternative are used when compiler safety is increased
beyond a certain level.

I think what is expected from Go today is quite different.

Thanks,
Florian

[tanishq...@yahoo.com]

I just wish that there shall be a desktop PC version launch as soon as possible which supports running of . Exe extension files and also the. Apk stuff

[contacts...@gmail.com]

Hi All,

i am trying to build Fuchsia, following the link, https://fuchsia.googlesource.com/docs/+/master/getting_started.md#Ubuntu
At fx full-build, I get build failure..and wondering if anyone can show me some pointers as Make process is also new. Attached the build failure screenshot.

Thanks,
Sri

[Brad Fitzpatrick]

Fuchsia's support for Go is currently owned by the Fuchsia team. It has not been merged into the official Go tree (yet?).

And this mailing list doesn't really have any Fuchsia people on it (maybe 1?), so the Fuchsia mailing lists are probably a better venue for such questions for now.

--
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.