Getting GDB Python extensions for SpiderMonkey debugging into the source tree

[Jim Blandy]

Hi, folks. I wrote a bunch of Python plug-in code for GDB to
automatically display SpiderMonkey types in a meaningful way; for example:

$ gdb obj~/js
(gdb) b math_atan2
Breakpoint 1 at 0x537b42: file /home/jimb/moz/archer/js/src/jsmath.cpp, line 215.
(gdb) run -e "Math.atan2('schnoo')"
Starting program: /home/jimb/moz/archer/js/src/obj~/js -e "Math.atan2('schnoo')"

Breakpoint 1, math_atan2 (cx=0xc31470, argc=1, vp=0x7ffff17300a0) at /home/jimb/moz/archer/js/src/jsmath.cpp:215
215 if (argc <= 1) {
(gdb) print vp[2]
$1 = $jsval("schnoo")
(gdb) print vp[1]
$2 = $jsval(0x7ffff1505080 [Object Math])
(gdb) print vp[0]
$3 = $jsval(0x7ffff1511d00 [Object Function "atan2"])
(gdb)


There's similar support for random stuff like parse nodes, property
cache entries, and so on.

For a long time, these have lived at:
http://hg.mozilla.org/users/jblandy_mozilla.com/archer-mozilla/ However,
I've started putting together a patch to integrate them into the Mozilla
tree, so that anyone who uses a Python-capable GDB will get them
automatically.

Does this sound like a good idea?

Possible questions and concerns:

* Which versions of GDB will this work with? I'm adapting the
pretty-printers to work with the stock GDB shipped with recent
Linuxes; no custom build should be required. Even recent Macintoshes
seem to ship with a Python-ignorant GDB, so it shouldn't affect them.
* What happens when a Python pretty-printer encounters an error, due
to a bug, say? Nowadays, GDB disables the pretty-printer, so things
degrade gracefully. I'd make sure this happens before landing anything.
* Who can I bug when there's a problem? I volunteer to be on call for
any GDB-Python related issues; there are some other folks on the
team who've hacked on this as well, and I hope they'll volunteer too.
* How are we going to keep the pretty-printers up to date with the
current SpiderMonkey code? SpiderMonkey has changed quickly over the
last couple of years, and the printers have been prone to bit-rot.
Here, I'm asking to impose on the rest of you: I suspect that having
these printers in-tree will make problems show up more quickly, so
we can stay on top of things.

I'm picking up Bug 506717
<https://bugzilla.mozilla.org/show_bug.cgi?id=506717> for the process of
integration.

My current work in progress is at
https://github.com/jimblandy/mozilla-central. In particular, take a look
at the integrate-archer
<https://github.com/jimblandy/mozilla-central/tree/integrate-archer> branch.

[Benjamin Peterson]

2012/8/2 Jim Blandy <ji...@mozilla.com>:

> Hi, folks. I wrote a bunch of Python plug-in code for GDB to
> automatically display SpiderMonkey types in a meaningful way; for example:
>
> $ gdb obj~/js
> (gdb) b math_atan2
> Breakpoint 1 at 0x537b42: file /home/jimb/moz/archer/js/src/jsmath.cpp, line 215.
> (gdb) run -e "Math.atan2('schnoo')"
> Starting program: /home/jimb/moz/archer/js/src/obj~/js -e "Math.atan2('schnoo')"
>
> Breakpoint 1, math_atan2 (cx=0xc31470, argc=1, vp=0x7ffff17300a0) at /home/jimb/moz/archer/js/src/jsmath.cpp:215
> 215 if (argc <= 1) {
> (gdb) print vp[2]
> $1 = $jsval("schnoo")
> (gdb) print vp[1]
> $2 = $jsval(0x7ffff1505080 [Object Math])
> (gdb) print vp[0]
> $3 = $jsval(0x7ffff1511d00 [Object Function "atan2"])
> (gdb)
>
>
> There's similar support for random stuff like parse nodes, property
> cache entries, and so on.
>
> For a long time, these have lived at:
> http://hg.mozilla.org/users/jblandy_mozilla.com/archer-mozilla/ However,
> I've started putting together a patch to integrate them into the Mozilla
> tree, so that anyone who uses a Python-capable GDB will get them
> automatically.
>
> Does this sound like a good idea?

Yes! CPython has similar gdb hooks and they're really helpful.

> * How are we going to keep the pretty-printers up to date with the
> current SpiderMonkey code? SpiderMonkey has changed quickly over the
> last couple of years, and the printers have been prone to bit-rot.
> Here, I'm asking to impose on the rest of you: I suspect that having
> these printers in-tree will make problems show up more quickly, so
> we can stay on top of things.

Can tests be written for them, such that engine changes that break
debugging hooks will become immediately known?

--
Regards,
Benjamin

[Josh Matthews]

Nice. Glad to see my printers are going in as well, although it's a
teeny bit weird to have all the nsFoo printers residing under js/. I
volunteer to consult, if JS folk want to fix things but don't know
much about the GDB side of things.

Cheers,
Josh

> Possible questions and concerns:
>
> * Which versions of GDB will this work with? I'm adapting the
> pretty-printers to work with the stock GDB shipped with recent
> Linuxes; no custom build should be required. Even recent Macintoshes
> seem to ship with a Python-ignorant GDB, so it shouldn't affect them.
> * What happens when a Python pretty-printer encounters an error, due
> to a bug, say? Nowadays, GDB disables the pretty-printer, so things
> degrade gracefully. I'd make sure this happens before landing anything.
> * Who can I bug when there's a problem? I volunteer to be on call for
> any GDB-Python related issues; there are some other folks on the
> team who've hacked on this as well, and I hope they'll volunteer too.

> * How are we going to keep the pretty-printers up to date with the
> current SpiderMonkey code? SpiderMonkey has changed quickly over the
> last couple of years, and the printers have been prone to bit-rot.
> Here, I'm asking to impose on the rest of you: I suspect that having
> these printers in-tree will make problems show up more quickly, so
> we can stay on top of things.
>

> I'm picking up Bug 506717
> <https://bugzilla.mozilla.org/show_bug.cgi?id=506717> for the process of
> integration.
>
> My current work in progress is at
> https://github.com/jimblandy/mozilla-central. In particular, take a look
> at the integrate-archer
> <https://github.com/jimblandy/mozilla-central/tree/integrate-archer> branch.

> _______________________________________________
> dev-tech-js-engine-internals mailing list
> dev-tech-js-en...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-tech-js-engine-internals

[Jim Blandy]

On 08/02/2012 06:37 PM, Benjamin Peterson wrote:
> Can tests be written for them, such that engine changes that break
> debugging hooks will become immediately known?

There are unit tests for the pretty-printers.

I'd suggest that those tests *not* be hooked up to 'make check' to run
by default, because they can't be run on OSX and Windows; developers on
those platforms could break them, not find out until they hit try, and
then be stuck updating debugging tools they don't even use.

I could add a configuration flag to add the GDB tests to 'make check',
so that developers could 'opt in' to getting trouble reports for the
pretty-printers. At least the way I work, I have scripts to run
configure for me in the ways I usually need, so this would be easy for
me to enable for all my builds.

[Benjamin Peterson]

2012/8/2 Jim Blandy <ji...@mozilla.com>:

> On 08/02/2012 06:37 PM, Benjamin Peterson wrote:
>> Can tests be written for them, such that engine changes that break
>> debugging hooks will become immediately known?
> There are unit tests for the pretty-printers.
>
> I'd suggest that those tests *not* be hooked up to 'make check' to run
> by default, because they can't be run on OSX and Windows; developers on
> those platforms could break them, not find out until they hit try, and
> then be stuck updating debugging tools they don't even use.

Couldn't you detect gdb in the test?

>
> I could add a configuration flag to add the GDB tests to 'make check',
> so that developers could 'opt in' to getting trouble reports for the
> pretty-printers. At least the way I work, I have scripts to run
> configure for me in the ways I usually need, so this would be easy for
> me to enable for all my builds.

Just as long as we find out quickly when things break.


--
Regards,
Benjamin

[Wes Garland]

> Couldn't you detect gdb in the test?

You would have to detect gdb, archer-awareness, python and python version
IIUC.

Wes

[Jim Blandy]

On 08/02/2012 11:36 PM, Benjamin Peterson wrote:
> Couldn't you detect gdb in the test?

Certainly --- the concern isn't that the test would always fail on OSX
and Windows; it's not difficult to detect whether they're appropriate to
run, and skip them if not.

The pretty-printers know about the details of the implementation; they
break when the implementation changes, and need to be brought up to
date. So I'm concerned that, from time to time, a developer on
OSX/Windows will change the implementation; run tests locally (skipping
the pretty-printer tests) and see that all is okay; and then push to try
and see that the GDB pretty-printer tests are now failing because of
their change.

It seems unfair to expect people to fix tools they don't use. The burden
of maintenance should be on the users of that tool. But if the general
consensus is that I'm worrying too much about this, and that everyone on
the JS team is expected to be fluent in all four platforms - or perhaps
it just so happens that none of us avoid Linux - then we can just hook
it all up.

[Brendan Eich]

Experience throughout Mozilla's history with single- or minority-share
platform (or one tinderbox/buildbot) checking is -- not surprising --
people will break it. Then someone has to fix, ideally by testing first
on the supporting configuration.

I'm thinking of long ago when we had strange Mac OS 9 or older stuff
people broke. Also the big schism between fatal assertions on Linux and
Mac vs. assertions-as-warnings on Windows. And of course the
treehydra-scripted static analysis tinderbox.

If everyone pitches in to fix, and no one is put out by having to fix...
But the workload likely falls on Linux-first developers, which could be
only a relative few. It would be interesting to see primary-platform
poll results.

/be

[Wes Garland]

Given the availability and cost of Linux VMs -- is the burden of
"Linux-only" code anywhere near as high as other-platform-only code?

Does Moz have VM images for Linux development that can be turned on really
quickly?

I guess part of the equation is also familiarity with the toolchain, but
given that we already use mozilla-build on Windows I don't think it would
be a significant burden for someone with win32 as their mother tongue,
provided we can test the pretty-printers with a shell-only build.

[Jim Blandy]

On 08/03/2012 09:28 AM, Brendan Eich wrote:
> Experience throughout Mozilla's history with single- or minority-share
> platform (or one tinderbox/buildbot) checking is -- not surprising --
> people will break it. Then someone has to fix, ideally by testing
> first on the supporting configuration.

That's exactly the pattern I'd anticipate here, if we enable the tests
whenever the platform supports it. Thus my suggestion of opt-in testing,
so only volunteers get hit.

> If everyone pitches in to fix, and no one is put out by having to
> fix... But the workload likely falls on Linux-first developers, which
> could be only a relative few. It would be interesting to see
> primary-platform poll results.

Yes, I'd like to see that, too. I think Naveed was either collecting
this information, or was interested, too; I'll ask him.

[Brendan Eich]

Problem here is a category mistake -- people don't all test on Linux and
we do not require that (and won't). So arguing about the price of Linux
isn't going to help, because it can't be zero.

/be

[Steve Fink]

On 08/02/2012 03:22 PM, Jim Blandy wrote:
> For a long time, these have lived at:
> http://hg.mozilla.org/users/jblandy_mozilla.com/archer-mozilla/ However,
> I've started putting together a patch to integrate them into the Mozilla
> tree, so that anyone who uses a Python-capable GDB will get them
> automatically.
>
> Does this sound like a good idea?

I don't know how people survive without these. I use them and love 'em.

I understand the reluctance to add even more flakiness to gdb. But in
practice, I find that when a pretty-printer barfs on something, it's
fairly clear what's going on (it'll insert an error message into the
pretty-printed value's display, which doesn't prevent you from seeing
the rest of the data.) The biggest problem I've had is that something
can go wrong and I'll only get the hex address of certain types, without
the type name. But as long as you notice that things are going off the
rails, you just do a p/r.

> * Who can I bug when there's a problem? I volunteer to be on call for
> any GDB-Python related issues; there are some other folks on the
> team who've hacked on this as well, and I hope they'll volunteer too.

I volunteer to help.

> * How are we going to keep the pretty-printers up to date with the
> current SpiderMonkey code? SpiderMonkey has changed quickly over the
> last couple of years, and the printers have been prone to bit-rot.
> Here, I'm asking to impose on the rest of you: I suspect that having
> these printers in-tree will make problems show up more quickly, so
> we can stay on top of things.

We can do the same thing as we're doing the warnings-and-errors and
rooting analysis builds -- create a single-platform job type on
mozilla-inbound (at least) that only fires off when something is changed
under js/src, and hide it by default. The hiding by default is
unfortunate, but it's currently the recognized way to label something as
"you don't have to backout if this fails. Yet."

Note that in practice, those hidden spidermonkey jobs are only
moderately effective. We do seem to fix them when they break, but
frequently there's a 1-2 week lag in doing so. And the gdb/python stuff
could be a bit worse, since the pool of people familiar enough to fix
them is smaller.

Come to think of it, we should teach tbpl to display a different type of
build: visible by default but informational only. And create a
persistent preference to display spidermonkey builds in that state. I
could do the implementation part of that; not sure about the UI/design part.

> My current work in progress is at
> https://github.com/jimblandy/mozilla-central. In particular, take a look
> at the integrate-archer
> <https://github.com/jimblandy/mozilla-central/tree/integrate-archer> branch.

Oh, cool. I have a couple of things piled on top of your original
archer-mozilla repo (eg stack frames and some stuff for moving GCthing
pointers), currently in a half-broken state. I should switch over.

But... why is "archer" still in the name?