Help Porting to DragonFly BSD

[vari...@gmail.com]

All I want for Christmas is to be able to program in Go (with gc and cgo) on DragonFly BSD. I've started porting Go and I can both fix easy errors and clean up my code once I'm done making a mess of it, but when it comes to tough C and assembly errors, although I'm probably technically capable of eventually figuring them out, from a practical perspective my aversion to those languages prevents it from actually happening. So could somebody help me resolve my current error? DragonFly BSD installs very quickly in VirtualBox and I've posted information about the error at the following address. https://github.com/varialus/godfly/

Thanks,
Varialus

[Brad Fitzpatrick]

At the system call level, how much has it diverged from FreeBSD?  (Where Go already runs)

How far have you gotten already?

--

---
You received this message because you are subscribed to the Google Groups "golang-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-dev+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Dmitry Vyukov]

On Fri, Aug 16, 2013 at 5:22 PM, <vari...@gmail.com> wrote:

All I want for Christmas is to be able to program in Go (with gc and cgo) on DragonFly BSD. I've started porting Go and I can both fix easy errors and clean up my code once I'm done making a mess of it, but when it comes to tough C and assembly errors, although I'm probably technically capable of eventually figuring them out, from a practical perspective my aversion to those languages prevents it from actually happening. So could somebody help me resolve my current error? DragonFly BSD installs very quickly in VirtualBox and I've posted information about the error at the following address. https://github.com/varialus/godfly/

Program received signal SIGSEGV, Segmentation fault.
runtime.settls () at /root/go/src/pkg/runtime/sys_dragonflybsd.s:252
252 MOVL $0xf1, 0xf1 // crash


I am looking at freebsd code for runtime.settls, you must have something similar:

 // set tls base to DI

TEXT runtime·settls(SB),NOSPLIT,$8

        ADDQ    $16, DI // adjust for ELF: wants to use -16(FS) and -8(FS) for g and m

        MOVQ    DI, 0(SP)

        MOVQ    SP, SI

        MOVQ    $129, DI        // AMD64_SET_FSBASE

        MOVQ    $165, AX        // sysarch

        SYSCALL

        JCC     2(PC)

        MOVL    $0xf1, 0xf1  // crash

        RET

Crash here means either that this syscall is not supported on dragonfly (bad) or you use wrong constants (good).

Try to figure out what is errno, it must be probably in some register.

[Dmitry Vyukov]

does it have strace? it would probably help debugging this.

[Brad Fitzpatrick]

On Fri, Aug 16, 2013 at 9:37 AM, Dmitry Vyukov <dvy...@google.com> wrote:

does it have strace? it would probably help debugging this.

http://leaf.dragonflybsd.org/cgi/web-man?command=truss&section=1

[Varialus]

DragonFly forked from FreeBSD 4.8 ten years ago. So it's very divergent but it's closer than any other OS that Go has been ported to. Still, from an api perspective dfly is a fairly typical BSD, so I would expect that porting Go to it wouldn't be very difficult for someone who is more proficient in C than myself. Threading has been rewritten from scratch and it works completely differently within the kernel, but my understanding is that they have some fairly standard threading API's exposed. I have search through the dfly source looking for code related to thread local storage and I have posted links to files which looked related within the Notes section at https://github.com/varialus/godfly .

Tangent: I'm not a very experience dfly user. I primarily like it for its HAMMER file system, but I also like other aspects of it. I previously tried and failed to switch to it from Debian for everyday computing. However, lately I have started to prefer permissive copyright as opposed to copyleft because although I don't particularly care for big business, GPL inadvertently ends up mainly hurting lowly developers working for big business. So I want try to use and contribute to BSD/MIT/Apache licensed software.

I have gotten to where the build script tries to build cmd/go and in the process calls:

"$GOTOOLDIR"/go_bootstrap clean -i std

Here's a summary of what I've done so far.

1. Within the 'experimental' branch I started by just trying to duplicate all of the FreeBSD references and changing them to dfly references. This felt too haphazard and I didn't know whether my changes were affecting whichever error I was currently trying to fix, so I abandoned this branch early on.
   
2. I then created the 'development' branch to fix errors as I encountered them. There were two errors that gave me trouble. One was that along the way, building on other OS's broke, and the second was the current settls error. While struggling with these two errors I reverted to haphazardly duplicating FreeBSD references and changing them to dfly references which ended up fixing the first error, so it now builds on other OS's. I documented some assembly instructions and and pseudo-instructions within the code right above the settls routine. Unfortunately I have a hard time pushing myself to become proficient in C and assembly. *whiny voice* I just want to program in Go!
3. The 'patched' branch contains two patch files merged in which I got from joris in the #dragonflybsd channel on the EFNET IRC network. It may be higher quality than my port and it may be more customized to dfly than my code, but when I tried to build using it, it didn't get as far along in the build process, so I didn't end up using it.
4. The 'dev1.0.1' branch contains a from scratch attempt which I started after reading a message about somebody on an old version of Linux encountering a similar error while upgrading Go from 1.0.x to 1.1.x. It was at this point when I asked the dfly IRC folks whether they'd prefer I refer to dfly within the Go source code as dragonflybsd or just dragonfly and they said just plain dragonfly is better. I had been referring to it as dragonflybsd within the 'development' branch, which I'll clean up amongst other things once I get Go running. I was able to get this branch to the current settls error but unfortunately starting from go1.0.1 didn't resolve the error. This branch does not contain the haphazard duplication of FreeBSD references changed to dfly references.

Also, I think it's worth noting that on 64-bit dfly, gdb from their binary repository is not configured to debug FreeBSD branded ELF binaries, but it works fine on 32-bit dfly. But I think that's just gdb. The default ELF brand on dfly is ELFOSABI_NONE/ELFOSABI_SYSV, but as I understand it ELFOSABI_FREEBSD should also work at the OS level. I was playing around with this yesterday within the dev1.0.1 branch to make 6l use ELFOSABI_NONE and 8l use ELFOSABI_FREEBSD.

[Varialus]

I've started over starting from Go 1.1.2 with the patch files applied which I got from another dfly user. I'm trying to not copy over any FreeBSD code, but rather slowly pull stuff from the dfly source code. I haven't caught up to where I previously was, but for the moment everything is going well. I'll let you know whether I encounter this settls error once I get to that part of the build process.

[Varialus]

Same error. :-( I pulled a lot more code directly from dfly this time and I was really hoping it would fly past without triggering the error. I did notice that sys/kern/syscalls.master has a set_tls_area() function. I tried to comment the places where I copied code from the FreeBSD port of Go. I ran that truss program and got the following output. I'm retyping the error, so there may be typos, but I'll try to avoid them.

bash-4.2# truss ../pkg/tool/dragonfly_amd64/go_bootstrap clean -i std

open("/proc/curproc/mem",0x1,0400032600400)      = 3 (0x3)

fcntl(0x3,0x2,0x1)                               = 0 (0x0)

ioctl(3,PIOCBIS,0x21)                            = 0 (0x0)

ioctl(3,PIOCSFL,0x1)                             = 0 (0x0)

execve(<missing argument>,<missing argument>,<missing argument>)sysarch(0x81,0x7ffffffff640)

SIGNAL 11

SIGNAL 11

Process stopped because of:  16

process exit, rval = 139

Segmentation fault: 11

bash-4.2# Aug 17 07:37:51  kernel: pid 22655 (go_bootstrap), uid 0: exited on signal 11 (core dumped)

Aug 17 07:37:51  kernel: pid 22654 (truss), uid 0: exited on signal 11

I stayed up all night because I hoped that my current effort would prevent this error. Please help!

Thanks,

Varialus

[Dmitry Vyukov]

On Sat, Aug 17, 2013 at 5:52 PM, Varialus <vari...@gmail.com> wrote:

Same error. :-( I pulled a lot more code directly from dfly this time and I was really hoping it would fly past without triggering the error. I did notice that sys/kern/syscalls.master has a set_tls_area() function. I tried to comment the places where I copied code from the FreeBSD port of Go. I ran that truss program and got the following output. I'm retyping the error, so there may be typos, but I'll try to avoid them.

bash-4.2# truss ../pkg/tool/dragonfly_amd64/go_bootstrap clean -i std

open("/proc/curproc/mem",0x1,0400032600400)      = 3 (0x3)

fcntl(0x3,0x2,0x1)                               = 0 (0x0)

ioctl(3,PIOCBIS,0x21)                            = 0 (0x0)

ioctl(3,PIOCSFL,0x1)                             = 0 (0x0)

execve(<missing argument>,<missing argument>,<missing argument>)sysarch(0x81,0x7ffffffff640)

SIGNAL 11

SIGNAL 11

Process stopped because of:  16

process exit, rval = 139

Segmentation fault: 11

bash-4.2# Aug 17 07:37:51  kernel: pid 22655 (go_bootstrap), uid 0: exited on signal 11 (core dumped)

Aug 17 07:37:51  kernel: pid 22654 (truss), uid 0: exited on signal 11

I stayed up all night because I hoped that my current effort would prevent this error. Please help!

check out with gdb from where execve is called and  from where sysarch is called

why does execve say missing argument? this is quite suspicious 

check that 0x81 is the correct sysarch number

check that address passed to sysarch is addressable

[Ian Lance Taylor]

On Sat, Aug 17, 2013 at 6:52 AM, Varialus <vari...@gmail.com> wrote:
>
> bash-4.2# truss ../pkg/tool/dragonfly_amd64/go_bootstrap clean -i std

Most current truss versions have a -f option. If the DragonFly one
supports -f, use it.

Ian

[Varialus]

I'm not very familiar with Google Groups or writing to mailing lists. This reply is intended for both Dmitry and Ian.

On Saturday, August 17, 2013 8:10:33 AM UTC-6, Dmitry Vyukov wrote:

check out with gdb from where execve is called and  from where sysarch is called

why does execve say missing argument? this is quite suspicious 

check that 0x81 is the correct sysarch number

check that address passed to sysarch is addressable

 

Dmitry, I don't know assembly pretty much at all. I very likely could have done that part incorrectly. I'll try to figure out how to do what you suggest tomorrow.

On Saturday, August 17, 2013 8:12:57 AM UTC-6, Ian Lance Taylor wrote:

Most current truss versions have a -f option.  If the DragonFly one 
supports -f, use it. 

Ian 

 

Ian, it does not support the -f option. If I can simply re-compile an newer version I'll try that tomorrow, but if it's a customized port, I may not get that done.

Thanks,

Alex

[Dmitry Vyukov]

On Sat, Aug 17, 2013 at 6:35 PM, Varialus <vari...@gmail.com> wrote:

I'm not very familiar with Google Groups or writing to mailing lists. This reply is intended for both Dmitry and Ian.

On Saturday, August 17, 2013 8:10:33 AM UTC-6, Dmitry Vyukov wrote:

check out with gdb from where execve is called and  from where sysarch is called

why does execve say missing argument? this is quite suspicious 

check that 0x81 is the correct sysarch number

check that address passed to sysarch is addressable

 

Dmitry, I don't know assembly pretty much at all. I very likely could have done that part incorrectly. I'll try to figure out how to do what you suggest tomorrow.

$ gdb --args go_boostrap clean -i std

then in gdb execute

"r" to run the program

when it crashes

"bt" to print stack traceback

"disass" to print disassembly of the current funtion

"p /x $rax" to print rax register if necessary

where did you get that 0x81 from?

 

On Saturday, August 17, 2013 8:12:57 AM UTC-6, Ian Lance Taylor wrote:

Most current truss versions have a -f option.  If the DragonFly one 
supports -f, use it. 

Ian 

 

Ian, it does not support the -f option. If I can simply re-compile an newer version I'll try that tomorrow, but if it's a customized port, I may not get that done.

Thanks,

Alex

--

[Varialus]

I've posted the output from the commands which you recommended at https://github.com/varialus/godfly and formatted it with code blocks to make it more readable than it was before.

[Dmitry Vyukov]

OK, now it's clear that this piece of code needs to be changed for dragonfly:

// set tls base to DI

TEXT runtime·settls(SB),NOSPLIT,$8

        ADDQ    $16, DI // adjust for ELF: wants to use -16(FS) and -8(FS) for g and m

        MOVQ    DI, 0(SP)

        MOVQ    SP, SI

        MOVQ    $129, DI        // AMD64_SET_FSBASE

        MOVQ    $165, AX        // sysarch

        SYSCALL

        JCC     2(PC)

        MOVL    $0xf1, 0xf1  // crash

        RET

At best you just need to change the $129 constant. I don't know where dragonfly syscall are described.

Another possibility isthat threads work differently -- runtime·thr_start expects to receive pointer to M in DI register. 0x7ffffffff640 does not look completely wrong, but it may be...

[Dmitry Vyukov]

On Sat, Aug 17, 2013 at 9:11 PM, Dmitry Vyukov <dvy...@google.com> wrote:

OK, now it's clear that this piece of code needs to be changed for dragonfly:

// set tls base to DI

TEXT runtime·settls(SB),NOSPLIT,$8

        ADDQ    $16, DI // adjust for ELF: wants to use -16(FS) and -8(FS) for g and m

        MOVQ    DI, 0(SP)

        MOVQ    SP, SI

        MOVQ    $129, DI        // AMD64_SET_FSBASE

        MOVQ    $165, AX        // sysarch

        SYSCALL

        JCC     2(PC)

        MOVL    $0xf1, 0xf1  // crash

        RET

At best you just need to change the $129 constant. I don't know where dragonfly syscall are described.

Another possibility isthat threads work differently -- runtime·thr_start expects to receive pointer to M in DI register. 0x7ffffffff640 does not look completely wrong, but it may be...

I guess you need dragonfly's systems programmer. Probably you can get some help on dragonfly forums.

[Devon H. O'Dell]

This needs to look like:

// set tls base to DI

TEXT runtime·settls(SB),7,$16

        ADDQ    $16, DI // adjust for ELF: wants to use -16(FS) and -8(FS) for g and m

        MOVQ    DI, 0(SP) // DI is first arg of syscall in amd64 calling convention, need to stash

        MOVQ    $16, 8(SP) // size of TLS area

        MOVQ    SP, SI // SP represents struct tls_info *

        MOVQ    $0, DI // "which"

        MOVQ    $16, DX // sizeof (struct tls_info)

        MOVL    $472, AX        // set_tls_area(which, tls_info, infosize)

        SYSCALL

        JCC     2(PC)

        MOVL    $0xf1, 0xf1  // crash

        RET

DragonFly has diverged significantly from FreeBSD (which has also diverged significantly from FreeBSD 4), so these sorts of low-level interfaces are expected to change.

Varialus, I figured this out by looking at the set_tls_area manpage and /usr/src/sys/kern/syscalls.master. I suspect that several runtime syscalls have different constants for the syscall entry and perhaps for arguments. I'll keep going with this until something works a little better and send a pull request. We'll see if I can still remember how to port Go :S.

--dho

2013/8/17 Dmitry Vyukov <dvy...@google.com>

[Devon H. O'Dell]

I sent a pull request to the repo. The go tool is still broken; I suspect it's due to bad sizes in the zfiles in pkg/syscall for dfbsd. It looks like these were just copied from FreeBSD. The only DF system I have access to is leaf, so I'm somewhat limited in my administrative capabilities to get things working. That said:

leaf:/home/dho/godfly> cat foo.go && pkg/tool/dragonfly_amd64/6g foo.go && pkg/tool/dragonfly_amd64/6l foo.6 && ./6.out

package main

import "fmt"

func main() {

fmt.Println("Hi");

}

Hi

leaf:/home/dho/godfly>

--dho

2013/8/18 Devon H. O'Dell <devon...@gmail.com>

[Varialus]

Thanks to dho's help make.bash finishes successfully! w00t! w00t! The actual go tool isn't getting built for some reason and the tests aren't running for some reason, but go_bootstrap builds and runs without error now.

Varialus

[andrey mirtchovski]

> Thanks to dho's help make.bash finishes successfully! w00t! w00t! The actual
> go tool isn't getting built for some reason and the tests aren't running for
> some reason, but go_bootstrap builds and runs without error now.

don't you need 'all.bash' for the tests?

[Varialus]

That is correct. Thanks for the correction. I had just worded that poorly while trying to emphasize which parts are and are not finishing without error. It actually is all.bash which is finishing without error, but the tests aren't being run. I haven't really looked into it yet. Although pkg/tool/dragonfly_amd64/go isn't being built or linked or whatever, there are lots of *.a file in pkg/dragonfly_amd64, including in pkg/dragonfly_amd64/go.

[Dmitry Vyukov]

go command must be in bin/ dir

[Varialus]

The go binary now builds and the tests start running, but many tests fail and I think one test gets stuck, but I suppose it could just be taking a really long time. Yesterday it turned out that the build was not without errors, it just wasn't crashing on errors. As soon as I fixed all the errors, the tests started running.

[Devon H. O'Dell]

Great. Need any more assistance or is this stuff you've got?

2013/8/19 Varialus <vari...@gmail.com>:

[Varialus]

I don't "need" assistance at the moment, but I will soon once I run into more more grizzly assembly bugs. But if you or anyone else wants to help with the busywork, there's a ton still needs to be done, so I'll gladly and liberally accept pull requests.

I've disabled a bunch of tests in order to prevent the tests from hanging. They'll all need to be re-enabled eventually, I just don't want the hangy tests to block the others from running.

[Brad Fitzpatrick]

On Mon, Aug 19, 2013 at 4:27 PM, Varialus <vari...@gmail.com> wrote:

The go binary now builds and the tests start running, but many tests fail and I think one test gets stuck, but I suppose it could just be taking a really long time. Yesterday it turned out that the build was not without errors, it just wasn't crashing on errors. As soon as I fixed all the errors, the tests started running.

all.bash is probably too large to be running at this point.

If you can't figure out what's hanging or crashing, pick it apart and run the individual tests one at a time to figure out what's reliable and what's not.

[Joel Sing]

I guess this could be considered as an early present then... :)

https://codereview.appspot.com/13176043/

The syscall/passfd test is disabled due to sendmsg returning EINVAL,
otherwise all tests pass. I've also not yet enabled cgo, however that
should be a minimal amount of work.

[Dmitry Vyukov]

WOW
the last missing piece is DragonFly builder ;)

[Varialus]

Thanks Joel!

This is the first time I've attempted to get a significantly sized new feature added to an important mainstream open source project, so for attribution and education, would you mind letting me know how useful or not my effort was? How instrumental to your effort were my request for help and my attempt to port it myself. Did you start from scratch or from where I was at? If you started from scratch, how helpful to you was what I had already done.

I acknowledge that I'm not the first person to have worked on the port and others' efforts were helpful to me. The FreeBSD port, which dho worked on, was vitally important as it served as a template for what needed to be done. GaryS had attempted a port before me and had gotten to where go_bootstrap builds but crashes when run as part of the build process, which is as far as I had gotten before dho fixed a bit of assembly for me. Also, joris sent me some patch files with some initial work done which at first I didn't use much, but later I did end up using, and having them was very helpful. That's just the people to whom I know to give direct attribution. Others also definitely helped either directly or indirectly such as dillon and the dfly crew, who made the HAMMER file system with low memory block level deduplication, which is what keeps pulling me back to the very innovative and just plain cool DragonFly BSD operating system and Rob and the golang crew who have made such an awesomely balanced programming language.

The comment in that code review mentions the Go 1.2 freeze. I hadn't realized it was approaching, so I really appreciate that you were able to get it submitted upstream with more than a week to get it reviewed before the freeze which looks to be targeted for the 1st of September. Do you happen to know whether adding adding support for a new OS or adding support for cgo would be too big a change for a minor 1.2.x Go update? If either are too big for a minor 1.2.x release, then I hope they get (written, in the case of cgo,) reviewed and accepted before then. I suppose it's not too terribly vital since building from source is easy enough, but it would be nice to not have to maintain a separate branch for the the entire life of Go 1.2.

Thanks again!

Varialus