cgo: how to generate crash report/coredump on macOS for SIGSEGV

[aihui zhu]

I have enabled coredump with `ulimit -c unlimited`, and run the binary with GOTRACEBACK=crash, but it only outputs the go stack trace when crash. 

thanks.

[Joop Kiefte]

CGo is not Go -- C-land crash information fully depends on your C-compiler.

Joop Kiefte - Chat @ Spike

On October 2, 2020 at 0:39 GMT, aihui zhu <mr.z...@gmail.com> wrote:

I have enabled coredump with `ulimit -c unlimited`, and run the binary with GOTRACEBACK=crash, but it only outputs the go stack trace when crash. 

thanks.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/8a93e022-f9a2-4b12-9b1d-e50640dd5573n%40googlegroups.com.

[Kurtis Rader]

On Thu, Oct 1, 2020 at 5:39 PM aihui zhu <mr.z...@gmail.com> wrote:

I have enabled coredump with `ulimit -c unlimited`, and run the binary with GOTRACEBACK=crash, but it only outputs the go stack trace when crash. 

Have you verified that you can generate a core dump from a pure C program that simply dereferences a NULL pointer? I ask because a couple of years ago I was surprised that SIGSEGV did not result in a core dump on my macOS server despite having run "ulimit -c unlimited". I eventually traced it to the core dump directory not being writable by non-root users. Check the permissions which should look like this:

> ls -ld /cores

drwxrwxrwt 2 root wheel 64 Aug 10 11:21 /cores/

--

Kurtis Rader

Caretaker of the exceptional canines Junior and Hank

[Ian Lance Taylor]

On Thu, Oct 1, 2020 at 5:39 PM aihui zhu <mr.z...@gmail.com> wrote:
>

> I have enabled coredump with `ulimit -c unlimited`, and run the binary with GOTRACEBACK=crash, but it only outputs the go stack trace when crash.

There are two different things. There is the stack trace emitted when
a program crash. And there is the core dump file generated by the
kernel when a program crashes (if GOTRACEBACK=crash is set). It is
normal for the stack trace to only show Go functions (though you could
try importing github.com/ianlancetaylor/cgosymbolizer). I'm not
clear: are you saying that you see only the stack trace, and that you
do not see a core dump file?

Ian

[aihui zhu]

yes, there are only go stack trace, and no core file was generated. with a pure C program, it could generate core file in /cores directory.

[aihui zhu]

on a linux virtual machine GOTRACEBACK=crash works fine and would generates core files in working directory for me.

[Kurtis Rader]

On Thu, Oct 1, 2020 at 8:36 PM aihui zhu <mr.z...@gmail.com> wrote:

on a linux virtual machine GOTRACEBACK=crash works fine and would generates core files in working directory for me.

I just tested this trivial Go (not CGo) program:

package main

func main() {

  panic("WTF")

}

I ran the above program on Linux distro OpenSuse 42. With the specified env var results I get a core dump. The same action on macOS 10.15 (Catalina) only produces a backtrace -- no core dump. Looks like a bug (or unimplemented feature) to me.

[Ian Lance Taylor]

On Thu, Oct 1, 2020 at 9:05 PM Kurtis Rader <kra...@skepticism.us> wrote:
>
> On Thu, Oct 1, 2020 at 8:36 PM aihui zhu <mr.z...@gmail.com> wrote:
>>
>> on a linux virtual machine GOTRACEBACK=crash works fine and would generates core files in working directory for me.
>
>
> I just tested this trivial Go (not CGo) program:
>
> package main
> func main() {
> panic("WTF")
> }
>
>
> I ran the above program on Linux distro OpenSuse 42. With the specified env var results I get a core dump. The same action on macOS 10.15 (Catalina) only produces a backtrace -- no core dump. Looks like a bug (or unimplemented feature) to me.

Can you get a core dump from a C program on macOS 10.15?

Ian

[aihui zhu]

C program could generate a core dump file, my os is Big Sur beta.

➜  ~ ./a.out

[1]    37539 segmentation fault  ./a.out

➜  ~ ulimit -c unlimited

➜  ~ ./a.out

[1]    37596 segmentation fault (core dumped)  ./a.out

➜  ~ cat a.c

int main() {

    int a = *((int*)(0));

    return a;

}

➜  ~ ls /cores -lh

total 2.9G

-r-------- 1 zah wheel 2.9G Oct  2 13:37 core.37596

[Ian Lance Taylor]

On Thu, Oct 1, 2020 at 10:44 PM aihui zhu <mr.z...@gmail.com> wrote:
>
> C program could generate a core dump file, my os is Big Sur beta.

Thanks.

I see now that for darwin-amd64 we do not generate core dumps. Sorry
for forgetting about that.

We have this comment:

// OS X core dumps are linear dumps of the mapped memory,
// from the first virtual byte to the last, with zeros in the gaps.
// Because of the way we arrange the address space on 64-bit systems,
// this means the OS X core file will be >128 GB and even on a zippy
// workstation can take OS X well over an hour to write (uninterruptible).
// Save users from making that mistake.

It may be worth investigating whether this is still a problem on
current macOS systems. The comment in question is in
runtime/signal_unix.go.

Ian

[aihui zhu]

thank you.  i see that there are also .crash file generated at ~/Library/Logs/DiagnosticReports

```

➜  ~ ls Library/Logs/DiagnosticReports/a.out_2020-10-02-1337* -lh

-rw------- 1 zah _analyticsusers 22K Oct  2 13:37 Library/Logs/DiagnosticReports/a.out_2020-10-02-133721_hac.crash

-rw------- 1 zah _analyticsusers 21K Oct  2 13:37 Library/Logs/DiagnosticReports/a.out_2020-10-02-133751_hac.crash

```

could go generates .crash file?

[Kurtis Rader]

On Fri, Oct 2, 2020 at 5:29 PM aihui zhu <mr.z...@gmail.com> wrote:

thank you.  i see that there are also .crash file generated at ~/Library/Logs/DiagnosticReports

```

➜  ~ ls Library/Logs/DiagnosticReports/a.out_2020-10-02-1337* -lh

-rw------- 1 zah _analyticsusers 22K Oct  2 13:37 Library/Logs/DiagnosticReports/a.out_2020-10-02-133721_hac.crash

-rw------- 1 zah _analyticsusers 21K Oct  2 13:37 Library/Logs/DiagnosticReports/a.out_2020-10-02-133751_hac.crash

```

could go generates .crash file?

Those are simple text files that contain a report of the event that resulted in an application crash. They are created by macOS. The program that crashes does not create that file. Which means that Go should not attempt to create that file. Also, Go already emits most of what you need to know when a Go program crashes.

 

[Kurtis Rader]

It  looks like this is still a problem on macOS (at least as of 10.15 Catalina). A trivial C program that does nothing more than

char *p = 0;

*p = 'x';

results in a 2.1 GB core dump.

It might be worth documenting this limitation in the https://golang.org/pkg/runtime/ documentation. :-)

-- 

[aihui zhu]

I mean that is there any way to expose SIGSEGV to macOS system just like a pure C program, so that the system could generate .crash file automatically.

currently, seems that go itself catch the SIGSEGV signal, and prints go trace back, so it doesn't trigger system default behavior for SIGSEGV.

please correct me if i'm wrong at something,  thank you.

[Kurtis Rader]

On Fri, Oct 2, 2020 at 9:23 PM aihui zhu <mr.z...@gmail.com> wrote:

I mean that is there any way to expose SIGSEGV to macOS system just like a pure C program, so that the system could generate .crash file automatically.

currently, seems that go itself catch the SIGSEGV signal, and prints go trace back, so it doesn't trigger system default behavior for SIGSEGV.

That is correct. And, no, there is no way for that to result in a core dump unless you build a custom Go runtime that changes the behavior of the "crash()" function in src/runtime/signal_unix.go. That is for the reason explained in the doc string in that function: You would end up with a huge core dump file that would probably take an hour or more to create. Do you actually have enough disk space (> 128 GB free space) and are willing to wait upwards of an hour for the core file to be created? If so then I, and no doubt others, would be interested in hearing from you why letting Go behave that way on macOS is sensible for 99% of people running Go programs on macOS.

[Robert Engels]

A crash file is not a core dump. It does not contain a memory dump. 

On Oct 2, 2020, at 11:36 PM, Kurtis Rader <kra...@skepticism.us> wrote:



--

You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CABx2%3DD9-qWpfrHzo6SHspAfxbk7EymGmSbj2Wb4squ1iRur-hg%40mail.gmail.com.

[aihui zhu]

.crash file is also acceptable for me, if it contains a stack trace.

[Kurtis Rader]

On Fri, Oct 2, 2020 at 9:41 PM Robert Engels <ren...@ix.netcom.com> wrote:

A crash file is not a core dump. It does not contain a memory dump. 

Yes. I addressed that in a different reply than the one you responded to. So I'm a wee bit confused what to take away from your comment, Robert. :-)

[Kurtis Rader]

On Fri, Oct 2, 2020 at 9:49 PM aihui zhu <mr.z...@gmail.com> wrote:

.crash file is also acceptable for me, if it contains a stack trace.

No, it does not. At least not for me when running a trivial C program that simply dereferences a NULL pointer. It does contain some information (e.g., the contents of the CPU registers) that coule be useful in deducing the state of the program but a macOS ".crash" file does not contain a stack trace.

The default Go behavior for reporting panics does output stack traces. However, I have no idea what CGo does and would be surprised if a panic from the "C" code resulted in a useful backtrace of that code written to stdout/stderr. So the problem seems to be that you are melding Go and C code on macOS, the C code is causing a fatal error (most likely a SIGSEGV), and you're trying to find the bug in the C code. Yes?

[aihui zhu]

thank you and yes, i'm trying to debug a SIGSEGV occurred in C code. 

i'm going changed my C code to some other way, since it's a bit difficult to retrieve the stack trace for debugging..

[Robert Engels]

A crash file on OS X should contain a stack dump for each running thread, along with register information and mapped libraries. 

It would be nice to have in addition to the Go trace back because since Go uses green threads the state of these would not be captured in the crash file. 

On Oct 3, 2020, at 12:41 AM, aihui zhu <mr.z...@gmail.com> wrote:

thank you and yes, i'm trying to debug a SIGSEGV occurred in C code. 

--

You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/e5d4de0f-8e5e-409e-87db-4f56369fad57n%40googlegroups.com.