Segmentation Fault, how to debug?

7,748 views
Skip to first unread message

Kyle

unread,
Feb 8, 2010, 12:43:53 PM2/8/10
to nodejs
Hi All,

I'm writing a node app, and when benchmarking with ab, I come across a
segmentation fault after about 60 requests (with a concurrency of
around 10). What's the best way to go about debugging this? The
error doesn't say anything other than "segmentation fault", so I'm not
sure what part of node it's coming from, or if it's because of my own
code.

Thanks,
Kyle

Ryan Dahl

unread,
Feb 8, 2010, 12:50:10 PM2/8/10
to nod...@googlegroups.com

Do something like this:

./configure --debug && make
gdb --args ./node_g myscript.js
run
(make it crash)
backtrace

Kyle

unread,
Feb 8, 2010, 5:41:27 PM2/8/10
to nodejs
On Feb 8, 12:50 pm, Ryan Dahl <coldredle...@gmail.com> wrote:
> Do something like this:
>
>   ./configure --debug && make
>   gdb --args ./node_g myscript.js
>   run
>   (make it crash)
>   backtrace

Here's my backtrace, this is on v0.1.27, pulled from Git, running on
OS X, Snow Leopard. Let me know what I can do to help out with the
issue--I'm not too familiar with C, so I don't quite know what I'm
looking at here :)

dyn026014:node kyle$ gdb --args ~/src/node/node_g header_proxy.js
GNU gdb 6.3.50-20050815 (Apple version gdb-1344) (Fri Jul 3 01:19:56
UTC 2009)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and
you are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for
details.
This GDB was configured as "x86_64-apple-darwin"...Reading symbols for
shared libraries ... done

(gdb) run
Starting program: /Users/kyle/src/node/node_g header_proxy.js
Reading symbols for shared libraries ++. done
Reading symbols for shared libraries .... done
Server running at http://127.0.0.1:8000

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: 13 at address: 0x0000000000000000
0x0000000105874fb8 in ?? ()
(gdb) backtrace
#0 0x0000000105874fb8 in ?? ()
#1 0x0000000105879b3d in ?? ()
#2 0x0000000105801132 in ?? ()
#3 0x000000010581649f in ?? ()
#4 0x000000010580bd1c in ?? ()
#5 0x000000010583d0b0 in ?? ()
#6 0x000000010583c288 in ?? ()
#7 0x000000010587a42c in ?? ()
#8 0x0000000105814418 in ?? ()
#9 0x00000001058005cb in ?? ()
#10 0x000000010008602a in v8::internal::Invoke (construct=false,
func={location_ = 0x101017218}, receiver={location_ = 0x10072f368},
argc=0, args=0x0, has_pending_exception=0x7fff5fbfe2df) at /Users/kyle/
src/node/deps/v8/src/execution.cc:97
#11 0x000000010008655e in v8::internal::Execution::Call
(func={location_ = 0x101017218}, receiver={location_ = 0x10072f368},
argc=0, args=0x0, pending_exception=0x7fff5fbfe2df) at /Users/kyle/src/
node/deps/v8/src/execution.cc:124
#12 0x0000000100050172 in v8::Function::Call (this=0x101017218,
recv={val_ = 0x10072f368}, argc=0, argv=0x0) at /Users/kyle/src/node/
deps/v8/src/api.cc:2426
#13 0x000000010001e451 in node::ReallyEmit (self={val_ = 0x10072f368},
event={val_ = 0x10072e198}, argc=0, argv=0x0) at ../src/node_events.cc:
65
#14 0x000000010001e5d6 in node::EventEmitter::Emit (this=0x100c00080,
event={val_ = 0x10072e198}, argc=0, argv=0x0) at ../src/node_events.cc:
100
#15 0x000000010002ad91 in node::Timer::OnTimeout (watcher=0x100c00098,
revents=256) at ../src/node_timer.cc:69
#16 0x000000010002eb85 in ev_invoke_pending () at ../deps/libev/ev.c:
1997
#17 0x000000010002f23b in ev_loop (flags=0) at ../deps/libev/ev.c:2359
#18 0x000000010001065f in node::Loop (args=@0x7fff5fbfe680) at ../src/
node.cc:406
#19 0x000000010006b56a in v8::internal::HandleApiCallHelper<false>
(args={<v8::internal::Arguments> = {<v8::internal::Embedded> = {<No
data fields>}, length_ = 2, arguments_ = 0x7fff5fbfe870}, <No data
fields>}) at /Users/kyle/src/node/deps/v8/src/builtins.cc:451
#20 0x000000010006b627 in v8::internal::Builtin_Impl_HandleApiCall
(args={<v8::internal::Arguments> = {<v8::internal::Embedded> = {<No
data fields>}, length_ = 2, arguments_ = 0x7fff5fbfe870}, <No data
fields>}) at /Users/kyle/src/node/deps/v8/src/builtins.cc:468
#21 0x000000010006b64d in v8::internal::Builtin_HandleApiCall
(args={<v8::internal::Arguments> = {<v8::internal::Embedded> = {<No
data fields>}, length_ = 2, arguments_ = 0x7fff5fbfe870}, <No data
fields>}) at /Users/kyle/src/node/deps/v8/src/builtins.cc:467
#22 0x000000010580022a in ?? ()
#23 0x0000000105842a23 in ?? ()
#24 0x0000000105814418 in ?? ()
#25 0x00000001058005cb in ?? ()
#26 0x000000010008602a in v8::internal::Invoke (construct=false,
func={location_ = 0x1010171c8}, receiver={location_ = 0x101016258},
argc=1, args=0x7fff5fbfec70, has_pending_exception=0x7fff5fbfeb8f) at /
Users/kyle/src/node/deps/v8/src/execution.cc:97
#27 0x000000010008655e in v8::internal::Execution::Call
(func={location_ = 0x1010171c8}, receiver={location_ = 0x101016258},
argc=1, args=0x7fff5fbfec70, pending_exception=0x7fff5fbfeb8f) at /
Users/kyle/src/node/deps/v8/src/execution.cc:124
#28 0x0000000100050172 in v8::Function::Call (this=0x1010171c8,
recv={val_ = 0x101016258}, argc=1, argv=0x7fff5fbfec70) at /Users/kyle/
src/node/deps/v8/src/api.cc:2426
#29 0x000000010001264c in node::Load (argc=2, argv=0x7fff5fbff740)
at ../src/node.cc:1032
#30 0x0000000100012995 in main (argc=2, argv=0x7fff5fbff740) at ../src/
node.cc:1161

Ryan Dahl

unread,
Feb 8, 2010, 5:54:32 PM2/8/10
to nod...@googlegroups.com
> Here's my backtrace, this is on v0.1.27, pulled from Git, running on
> OS X, Snow Leopard.  Let me know what I can do to help out with the
> issue--I'm not too familiar with C, so I don't quite know what I'm
> looking at here :)

This looks pretty strange. Is it possible to send me a script that
demonstrates the bug?

Kyle

unread,
Feb 8, 2010, 10:48:58 PM2/8/10
to nodejs

Ok, I did some more investigating, and it seems to actually be
libxmljs that's causing the issues. When I try to work with an
XMLDocument, it causes the segmentation fault, and when I comment
those lines out, it goes away. I'm going to do some more testing, then
hopefully follow up whoever maintains that code.

-Kyle

metamind

unread,
Mar 10, 2010, 12:11:45 PM3/10/10
to nodejs
I get a segmentation fault as well when using libxmljs.

I am running node-v0.1.31 on Ubuntu 9.10 client (inside a windows
virtual pc on windows 7). The libxmljs versioin is: sprsquish-
libxmljs-1dc0797

the script is:

=================
var sys = require('sys')
var http = require('http')
var libxml = require('./libxmljs')

http.createServer(function (req, res) {
res.sendHeader(200, {'Content-Type': 'text/plain'});
res.write('Hello XML\n');

// var doc = libxml.parseXmlString('<xml><child><boing>a</boing></
child></xml>');

var doc = libxml.parseXmlFile('./data/test.xml');
//res.sendBody('.' + doc.toString() + '.\n')
res.write(doc.child(0).name() + ' .\n')
res.write(doc.child(0).child(0).name() + ' .\n')
res.write(doc.child(0).child(0).text() + ' .\n')
res.write('.\n')

res.close();
}).listen(8001);
sys.puts('Server running at http://127.0.0.1:8001/');
----------------------------------

./data/test.xml is:
=================
<xml><stuff>hello</stuff></xml>
----------------------------------

The segmentation fault happens after calling the node server from a
web browser.

The browser returns:
=================
Hello XML
stuff .
text .
hello .
.
----------------------------------

which is what you would expect. But the console output is:
=================
Server running at http://127.0.0.1:8001/
Segmentation fault
----------------------------------

Not too sure how to get the backtrace stuff to work. This is what I
did:

=================
ed@ceUbuntu:~/node/node-v0.1.31$ gdb --args ./node_g xmlTest.js
GNU gdb (GDB) 7.0-ubuntu
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/
gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show
copying"
and "show warranty" for details.
This GDB was configured as "i486-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /home/ed/node/node-v0.1.31/node_g...done.
(gdb) ^C.Quit
(gdb) /node_g xmlTest.js
Undefined command: "". Try "help".
(gdb) node xmlTest.js
Undefined command: "node". Try "help".
(gdb)
----------------------------------

I will email Jeff about this post.

metamind

unread,
Mar 10, 2010, 12:33:05 PM3/10/10
to nodejs
OK, got the backtrace to work:

=============================
[New Thread 0xb5f49b70 (LWP 3425)]


Server running at http://127.0.0.1:8001/

Program received signal SIGSEGV, Segmentation fault.
0x003cbd41 in ?? ()
(gdb) backtrace
#0 0x003cbd41 in ?? ()
#1 0x003f5498 in ?? ()
#2 0x003d6799 in ?? ()
#3 0x003c7842 in ?? ()
#4 0x081baa38 in Invoke (construct=false, func=..., receiver=...,
argc=2, args=0xbfffc950,
has_pending_exception=0xbfffc89f) at /home/ed/node/node-v0.1.31/
deps/v8/src/execution.cc:97
#5 0x081bab25 in v8::internal::Execution::Call (func=...,
receiver=..., argc=2, args=0xbfffc950,
pending_exception=0xbfffc89f) at /home/ed/node/node-v0.1.31/deps/
v8/src/execution.cc:124
#6 0x0818d462 in v8::Function::Call (this=0x8482e9c, recv=...,
argc=2, argv=0xbfffc950)
at /home/ed/node/node-v0.1.31/deps/v8/src/api.cc:2439
#7 0x0816010f in ReallyEmit (self=..., event=..., argc=2,
argv=0xbfffc950) at ../src/node_events.cc:65
#8 0x0816046f in node::EventEmitter::Emit (args=...) at ../src/
node_events.cc:93
#9 0x081aae90 in HandleApiCallHelper<false> (args=...) at /home/ed/
node/node-v0.1.31/deps/v8/src/builtins.cc:783
#10 0x081a8e20 in Builtin_Impl_HandleApiCall (args=...) at /home/ed/
node/node-v0.1.31/deps/v8/src/builtins.cc:800
#11 0x081a8e06 in Builtin_HandleApiCall (args=...) at /home/ed/node/
node-v0.1.31/deps/v8/src/builtins.cc:799
#12 0x003c638e in ?? ()
#13 0x00402ed5 in ?? ()
#14 0x00401ca0 in ?? ()
#15 0x003d6799 in ?? ()
#16 0x003c7842 in ?? ()
#17 0x081baa38 in Invoke (construct=false, func=..., receiver=...,
argc=1, args=0xbfffce0c,
has_pending_exception=0xbfffccff) at /home/ed/node/node-v0.1.31/
deps/v8/src/execution.cc:97
#18 0x081bab25 in v8::internal::Execution::Call (func=...,
receiver=..., argc=1, args=0xbfffce0c,
pending_exception=0xbfffccff) at /home/ed/node/node-v0.1.31/deps/
v8/src/execution.cc:124
#19 0x0818d462 in v8::Function::Call (this=0x8482e7c, recv=...,
argc=1, argv=0xbfffce0c)
at /home/ed/node/node-v0.1.31/deps/v8/src/api.cc:2439
#20 0x0816010f in ReallyEmit (self=..., event=..., argc=1,
argv=0xbfffce0c) at ../src/node_events.cc:65
#21 0x0816054a in node::EventEmitter::Emit (this=0x8485a20, event=...,
argc=1, argv=0xbfffce0c)
at ../src/node_events.cc:100
#22 0x081672a8 in node::HTTPConnection::on_headers_complete
(parser=0x8485ae0) at ../src/node_http.cc:321
#23 0x0817c8a5 in headers_complete_callback (parser=0x8485ae0) at ../
deps/http_parser/http_parser.c:85
#24 0x0817e48c in http_parser_execute (parser=0x8485ae0,
data=0xbfffcf7c "GET / HTTP/1.1\r\nhost: ceubuntu:8001\r\nuser-
agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-GB; rv:1.9.2) Gecko/
20100115 Firefox/3.6\r\naccept: text/html,application/xhtml
+xml,application/xml;q=0.9,*/*"..., len=389) at ../deps/http_parser/
http_parser.c:1283
#25 0x08165f9c in node::HTTPConnection::OnReceive (this=0x8485a20,
buf=0xbfffcf7c, len=389)
at ../src/node_http.cc:112
#26 0x0816c00e in node::Connection::on_read (s=0x8485a3c,
buf=0xbfffcf7c, len=389) at ../src/node_net.h:127
#27 0x0817a596 in stream_recv__data (stream=0x8485a3c) at ../deps/
evcom/evcom.c:562
#28 0x0817b2b8 in stream_event (w=0x8485a4c, revents=1) at ../deps/
evcom/evcom.c:1038
#29 0x0817353a in ev_invoke_pending () at ../deps/libev/ev.c:1997
#30 0x08173f94 in ev_loop (flags=0) at ../deps/libev/ev.c:2359
#31 0x0814fa89 in Loop (args=...) at ../src/node.cc:423
#32 0x081aae90 in HandleApiCallHelper<false> (args=...) at /home/ed/
node/node-v0.1.31/deps/v8/src/builtins.cc:783
#33 0x081a8e20 in Builtin_Impl_HandleApiCall (args=...) at /home/ed/
node/node-v0.1.31/deps/v8/src/builtins.cc:800
#34 0x081a8e06 in Builtin_HandleApiCall (args=...) at /home/ed/node/
node-v0.1.31/deps/v8/src/builtins.cc:799
#35 0x003c638e in ?? ()
#36 0x00407a2a in ?? ()
#37 0x003d6799 in ?? ()
#38 0x003c7842 in ?? ()
#39 0x081baa38 in Invoke (construct=false, func=..., receiver=...,
argc=1, args=0xbffff3a8,
has_pending_exception=0xbffff33f) at /home/ed/node/node-v0.1.31/
deps/v8/src/execution.cc:97
#40 0x081bab25 in v8::internal::Execution::Call (func=...,
receiver=..., argc=1, args=0xbffff3a8,
pending_exception=0xbffff33f) at /home/ed/node/node-v0.1.31/deps/
v8/src/execution.cc:124
#41 0x0818d462 in v8::Function::Call (this=0x8482d74, recv=...,
argc=1, argv=0xbffff3a8)
at /home/ed/node/node-v0.1.31/deps/v8/src/api.cc:2439
#42 0x08153a67 in Load (argc=2, argv=0xbffff7e4) at ../src/node.cc:
1142
#43 0x08154108 in main (argc=2, argv=0xbffff7e4) at ../src/node.cc:
1318
(gdb)

francisco treacy

unread,
Mar 15, 2010, 4:44:05 PM3/15/10
to nod...@googlegroups.com
Though I haven't debugged it, I'm also getting a "Segmentation fault"
error with node v0.1.32, using libxmljs 1d8cb7.

Anyone still with the same problem? The issue ought to be reported
against libxmljs?

Francisco


2010/3/10 metamind <no...@craft-e.com>:

> --
> You received this message because you are subscribed to the Google Groups "nodejs" group.
> To post to this group, send email to nod...@googlegroups.com.
> To unsubscribe from this group, send email to nodejs+un...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/nodejs?hl=en.
>
>

francisco treacy

unread,
Mar 15, 2010, 6:22:38 PM3/15/10
to nod...@googlegroups.com
Ok, this is the stacktrace: am I debugging badly or what does this
mean? A bug in node?

=> gdb --args ~/work/sandbox/node/node_g importer.js
GNU gdb 6.3.50-20050815 (Apple version gdb-962) (Sat Jul 26 08:14:40 UTC 2008)


Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.

There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-apple-darwin"...Reading symbols for
shared libraries .... done

(gdb) run
Starting program: /Users/ftreacy/Documents/work/sandbox/node/node_g importer.js
Reading symbols for shared libraries +++. done
deprecation warning: process.mixin will be removed from node-core
future releases.


Reading symbols for shared libraries .... done

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x0baddeac
0x033c2c9a in ?? ()
(gdb) backtrace
#0 0x033c2c9a in ?? ()
#1 0x00000000 in ?? ()
(gdb)

2010/3/15 francisco treacy <francisc...@gmail.com>:

Nate Agrin

unread,
Mar 15, 2010, 5:25:09 PM3/15/10
to nod...@googlegroups.com
I also see lots of seg faults wiith libxmljs but don't have the build
numbers handy. I'm also not exactly sure how to get the backtrace
working either.

-Nate

Jeff Smick

unread,
Mar 18, 2010, 1:19:14 AM3/18/10
to nod...@googlegroups.com
whoa.. sorry for not replying to this earlier.

An issue has been filed here:
http://github.com/sprsquish/libxmljs/issues#issue/18
The most recent HEAD of libxmljs shouldn't be segfaulting nearly as
much. For the last couple weeks (I only get about 2hrs/day) I've been
trying to hunt down and stop the leaks. If there are any c/c++ hackers
here that would be willing to lend a hand, I'd be very much
appreciative.

I've almost got it. Right now I'm down to trying to reconcile 3
garbage collectors (v8, libxml2, libxmljs) which has been a pretty big
pain. If anyone can help please send a message to the libxmljs group:
http://groups.google.com/group/libxmljs

-- Jeff

rooster 808

unread,
Mar 20, 2010, 9:00:00 PM3/20/10
to nodejs
+1

On Mar 15, 2:25 pm, Nate Agrin <n8ag...@gmail.com> wrote:
> I also see lots of seg faults wiith libxmljs but don't have the build
> numbers handy.  I'm also not exactly sure how to get the backtrace
> working either.
>
> -Nate
>

> On Mon, Mar 15, 2010 at 1:44 PM, francisco treacy<francisco.tre...@gmail.com> wrote:
> > Though I haven't debugged it, I'm also getting a "Segmentation fault"
> > error with node v0.1.32, using libxmljs 1d8cb7.
>
> > Anyone still with the same problem? The issue ought to be reported
> > against libxmljs?
>
> > Francisco
>

> > 2010/3/10 metamind <n...@craft-e.com>:


> >> OK, got the backtrace to work:
>
> >> =============================
> >> [New Thread 0xb5f49b70 (LWP 3425)]

> >> Server running athttp://127.0.0.1:8001/

> >> For more options, visit this group athttp://groups.google.com/group/nodejs?hl=en.

Scott Taylor

unread,
Mar 21, 2010, 12:35:50 AM3/21/10
to nod...@googlegroups.com

On Mar 20, 2010, at 9:00 PM, rooster 808 wrote:

> +1
>
> On Mar 15, 2:25 pm, Nate Agrin <n8ag...@gmail.com> wrote:
>> I also see lots of seg faults wiith libxmljs but don't have the build
>> numbers handy. I'm also not exactly sure how to get the backtrace
>> working either.

The v8 debugger protocol has a backtrace function, which isn't totally implemented yet in ndb. I should be getting to it sometime in the next week.

Scott

Reply all
Reply to author
Forward
0 new messages