Multi-language debugging
James Abley
Part of the attraction of having languages other than Java on the JVM
is that arguably more powerful, apt, glue languages can be used for
certain parts of an application, and then use Java (either for legacy
reasons) or performance, etc.
One thing that I've encountered recently is the inability to debug
Ruby code (set breakpoints, etc) along with breakpoints in Java code -
I can only debug one layer (tried with Eclipse and IDEA; can't do it
in Netbeans either, but I don't know Netbeans very well).
.
I'm curious what experiences other language implementers have had in
providing support for this functionality, both in sharing ideas /
implementation experiences and to consider the possibility of any
common requirements that might need to be pushed up into the JVM, a la
invokedynamic? Is there more than JSR-45 required?
Cheers,
James
N.B. I'm aware that there is sometimes a chasm to cross in terms of
attracting an audience to a language [1]; I think that languages on
the JVM have to contend with the fact that Java the Language offers
pretty good tooling and so that is a box that needs to be checked in
order to attract significant mind-share, unless the other features are
so compelling.
Eugene Kuleshov
All languages compiled to JVM bytecode and support JSR-45 should
work in regular Java debuggers, including Eclipse and IDEA.
While ago I've played with a JRuby prototype Charlie worked on and
here is how it look like in the Eclipse debugger [1]. Though I am not
sure if that stuff went into JRuby-proper or nor.
There is of course, a catch, e.g. some stack frames does not
directly map to any ruby nor Java code and this approach won't work
with interpreting languages or when just in time compiler didn't kick
in and there isn't really any bytecode.
regards,
Eugene
[1] http://img200.imageshack.us/img200/6825/rubyc.png
rssh
// In termware for debug mode special debug chunk (class with all
work: wrap call of interpreter from java-class with correct JSR-45
pmap) is
//created for each line.
So, this issue can be solved in fully interpreted version, but in
extremely ugly way.
[It would be good to include entries for trace manipulation in next
version of JSR-233 API]
James Abley
>
>
> // In termware for debug mode special debug chunk (class with all
> work: wrap call of interpreter from java-class with correct JSR-45
> pmap) is
> //created for each line.
>
> So, this issue can be solved in fully interpreted version, but in
> extremely ugly way.
> [It would be good to include entries for trace manipulation in next
> version of JSR-233 API]
>
> On Jun 18, 7:48 am, Eugene Kuleshov <ekules...@gmail.com> wrote:
>> All languages compiled to JVM bytecode and support JSR-45 should
>> work in regular Java debuggers, including Eclipse and IDEA.
>>
>> While ago I've played with a JRuby prototype Charlie worked on and
>> here is how it look like in the Eclipse debugger [1]. Though I am not
>> sure if that stuff went into JRuby-proper or nor.
>>
>> There is of course, a catch, e.g. some stack frames does not
>> directly map to any ruby nor Java code and this approach won't work
>> with interpreting languages or when just in time compiler didn't kick
>> in and there isn't really any bytecode.
>>
>> regards,
>> Eugene
>>
>> [1]http://img200.imageshack.us/img200/6825/rubyc.png
is being able to see the state of org.jruby objects.
From a user perspective, I would like something like the following:
1. Write an app in a language other than Java that runs on the JVM.
2. Have good debug facilities at the language level; e.g. show Ruby
variables, call-stack, etc.
3. Be able to step into other Java libraries that are being used in
the application; e.g. apache libraries, or more likely, home-spun
com.acme libraries that are less widely used and more likely to have
bugs in them [2]. Typically, Java IDEs offer a list of packages to
exclude stepping into when debugging code written in Java, so that you
could similarly avoid stepping into org.jruby code. For the majority
of users, I don't see that stepping into the non-Java language
implementation (org.jruby, org.python, etc) is desirable.
AFAIK, this isn't possible for JRuby [1]. As previously stated, I
haven't managed to get Netbeans, Eclipse or IDEA to do this for JRuby
(that could well be due to my lack of understanding or knowledge; has
anyone else had better success? Not that I can determine thus far.)
From discussions on #jruby, I believe that the machinery is there in
terms of JRuby language support / JSR-45 implementation. What is
lacking is tooling support for this. Now, if the tooling support isn't
there, then that will hinder a language's adoption in some quarters.
When I tell Netbeans that I want to launch and debug a Ruby app on
JRuby, it should be starting up the process so that ruby debugging is
on port X and Java debugging is on port Y, and let me do both, if I
want that option. There are a lot of legacy Java libraries out there
and that is one of the selling points of writing a language for the
JVM.
As a data point, I intend to write a Lift app and maybe Django and see
what Scala and Jython offer respectively.
I believe that what I'm asking should Just Work. So I'd like to know
if this support is offered in some JVM languages and not others, and
what common things can be learned to improve the shared state of the
art.
Cheers,
James
[1] http://www.netbeans.org/issues/show_bug.cgi?id=135357
[2] http://gist.github.com/132520
rssh
On Jun 19, 12:29 pm, James Abley <james.ab...@gmail.com> wrote:
> 2009/6/19 rssh <Rus...@shevchenko.kiev.ua>:
>
>
>
> > // In termware for debug mode special debug chunk (class with all
> > work: wrap call of interpreter from java-class with correct JSR-45
> > pmap) is
> > //created for each line.
>
> > So, this issue can be solved in fully interpreted version, but in
> > extremely ugly way.
> > [It would be good to include entries for trace manipulation in next
> > version of JSR-233 API]
>
> Did you mean JSR-223 - Scripting?
>
> on port X and Java debugging is on port Y, and let me do both, if I
> want that option. There are a lot of legacy Java libraries out there
> and that is one of the selling points of writing a language for the
> JVM.
>
which can be different from Java.
I'm not sure, that maintaining 'separate' view of program state in
two different
languages is possible and more useful than one mixed state. I. e.
usually we
debug mix of Java and other languages and one mixed state give us more
information.
(From other side - of course it depends, may be for jruby we have
other situation)
>
> I believe that what I'm asking should Just Work. So I'd like to know
> if this support is offered in some JVM languages and not others, and
> what common things can be learned to improve the shared state of the
> art.
>
debugger user JODA VM API:
http://java.sun.com/javase/6/docs/jdk/api/jpda/jdi/index.html
JDI is defined in term of mirrors for constructions, specified in Java
programming language.
What can be near ideal solution in long-term perspective: is to
define set of 'language-neutral'
analogues of JDI interfaces (let's name one Language Debug
Interface, LDI) in such case, that
language implementer can register set of callback interfaces and one
root object
('view for VM state from language X) in some registry, so debugger
will be able call
LNI methods provided by language, to show current stack frame, state
of variables, etc.
More realistic and 'hacky' approch is to emulate Java debugging, by
providing JDWP
(http://java.sun.com/javase/6/docs/technotes/guides/jpda/jdwp-
spec.html) interface
implemented in own debug server.
Attila Szegedi
job. It turned out rather well. I came up with a JSON-based TCP
protocol for sending commands, responses, and asynchronous events
(mostly about suspension of execution on a breakpoint) and built a
simple CLI client. It supports multiple debug sessions at once, and
even provides some additional niceties for debugging across
continuation restarts. It has the usual step in/over/out, breakpoints,
watchpoints, exception breakpoints, remote eval, and stack traces.
Anyway, it might be interesting here for following reasons:
1. I managed to come up with a design where large chunks of the code
are actually language independent. The debug session management, the
wire protocol implementation, the breakpoint maintenance and so on are
all language independent. The Rhino-specific tidbits are clearly
separated into separate set of classes, all coupled loosely via Java
interfaces to the language-independent parts. With a bit of extra
work, it'd actually be possible for multiple language runtimes that
are all aware of the API to share a debugging session when one runtime
calls into the other (well, we'd need a working MOP for that first,
wouldn't we?)
2. Some folks started asking around for just this thing on Rhino list
about a month ago (a Rhino remote debugger, not a multi-language
debugger, mind you). I asked for a permission from my employer to open
source the debugger, and got it after some time. I'm now cleansing up
the source code for general release. The Rhino thread is here if you
care: <http://groups.google.com/group/mozilla.dev.tech.js-engine.rhino/browse_thread/thread/a4d26551dff6b9e8#
>. I just posted a message there linking to this thread here so the
interested parties in the two crowds can be aware of each other.
3. Someone on the Rhino list pointed to this: <http://code.google.com/p/v8/wiki/DebuggerProtocol
>. This is the remote debugger protocol for the V8 JavaScript engine
that ships in Google Chrome. I didn't know about it when I embarked on
my own debugger implementation, but it's actually strikingly similar
to what I came up with - asynchronous JSON over TCP, even the command/
response/event structures are fairly similar. As part of the code
cleansing in part 2, I'm also changing the protocol to conform to V8
protocol. I'm shooting for as close compatibility as possible. (Not a
big deal, as it was fairly similar to begin with; mostly just renaming
identifiers...) Since I wasn't aware of them, I ended up reinventing
that wheel, but now I'm un-inventing it. I'm not in any way actually
committed to V8 protocol, it's just that they got out to public with a
good enough spec of a JSON-based wire protocol for debugging a dynamic
language, so I thought I'll be a sport, overcome the NIH syndrome and
not bring out yet another incompatible protocol.
Now, in contrast with my other efforts that aren't going anywhere
lately due to lack of time (MOP equivalent for invokedynamic most
notably, which exists in a half-baked state on my local machine), this
is something I think I can actually deliver fairly quickly, seeing how
it's something I actually got to spend few weeks of my paid time on,
and had enough resources to round it out nicely.
Now, this ain't exactly an all-encompassing silver bullet. Problems are:
1. It works best with interpreted, not compiled languages. I'm not
saying it couldn't work with bytecode-compiled ones, but it's
considerably harder to achieve. Stack inspection, breakpoints, etc are
easier
2. While it can work in-process, it's really geared towards remote
(out-of-process) debugging.
3. You might ask: why a new JSON based protocol? What's wrong with
JDWP? Well, nothing, except implementing JDWP is a quite massive effort.
4. It would need some more work (although not significantly more) to
support lookup of debug sessions for the thread, so theoretically a
call from one language runtime (with established session) to another
on the same thread could be debugged in the same remote session, if
both support the API
5. Doesn't support debugging of code on newly spun threads from
already-debugged threads. I don't see this as a huge problem, but it'd
be nice to round out over time. Requires thinking of how'd it work.
Right now one debug session (TCP connection from a debugger client)
will always be bound to at most one thread of execution.
6. V8 protocol does have some JS specifics (i.e. inspection of
scopes). I might end up not even implementing that part. We can also
end up adding some proprietary extensions.
Some of the above problems might make you think it's not the answer to
the topic of this thread, and it's fine. I just thought I'll make you
aware of some of the happenings in the field since the following three
things curiously coincided: (a) I just happened to write a remote
debugger for a dynamic language on JVM three months ago (b) people
started asking for the same on Rhino list a month ago and (c)
debugging suddenly came up as a topic here :-)
Attila.
--
twitter: http://twitter.com/szegedi
weblog: http://constc.blogspot.com
Charles Oliver Nutter
> One thing that I've encountered recently is the inability to debug
> Ruby code (set breakpoints, etc) along with breakpoints in Java code -
> I can only debug one layer (tried with Eclipse and IDEA; can't do it
> in Netbeans either, but I don't know Netbeans very well).
Ok, so I've been AWOL this whole week and going on vacation next, but
I wanted to jump in here with a few thoughts.
1. Debugging across languages still totally depends on having real
bytecode *somewhere*. You could debug JRuby's interpreted mode, but it
would be just about the most painful thing ever. I'd love to see the
JVM enhanced to provide debugging hooks for alternative languages in
interpreted modes, but I know of no such work.
2. JRuby can run in a mode where it is 100% compiled (modulo some
problem areas like larger-than-64k bodies), and then Ruby code shows
up with .rb code and line numbers right in the stack trace. It's
actually pretty clean:
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:258)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:77)
at test.method__2$RUBY$baz(test.rb:10)
at testInvokermethod__2$RUBY$bazFixed0.call(test#baz)
at testInvokermethod__2$RUBY$bazFixed0.call(test#baz)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:258)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:77)
at test.method__1$RUBY$bar(test.rb:6)
at testInvokermethod__1$RUBY$barFixed0.call(test#bar)
at testInvokermethod__1$RUBY$barFixed0.call(test#bar)
And with "inlineDyncalls", a somewhat tighter call protocol:
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:258)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:77)
at test.method__2$RUBY$baz(test.rb:10)
at testInvokermethod__2$RUBY$bazFixed0.call(test#baz)
at testInvokermethod__2$RUBY$bazFixed0.call(test#baz)
at test.method__1$RUBY$bar(test.rb:6)
at testInvokermethod__1$RUBY$barFixed0.call(test#bar)
at testInvokermethod__1$RUBY$barFixed0.call(test#bar)
3. The problem, in JRuby's case, is that even when running in
fully-compiled mode, none of the available Ruby tooling recognizes
Ruby code as Java-debuggable. All efforts to support Ruby debugging
have focused on what would work in both CRuby and JRuby, so it's all
Ruby's debug protocols. That all works great, but there's no
integration with Java debugging as a result. HOWEVER...I don't think
it would be hard to hook into any given debugger. Any new language
getting going on the JVM faces exactly the same problem, so I suspect
this will move toward a general solution soon.
I've done it myself with jdb, when I know what the class names I'll
generate will be:
4. Another smaller problem with JRuby is that because we've always
focused on having a mixed-mode execution, a lot of the filenames
embedded in the bytecode are either bogus or complete canonical paths,
neither of which helps a Java debugger out very much. But that's a
solvable problem, possibly by just embedding a better path to the
source or by embedding a couple different paths using JSR-45.
Ultimately, the only reason JRuby has complications here is because we
opted to support an interpreted mode forever, which on the JVM isn't
supported particularly well. But this is an area we recognize we need
improvement, and the next version or two of JRuby are going to have a
much tighter focus on integrating with the rest of the platform. This
means getting debugging working nicely (probably by contributing to
the IDEs a bit or working with them), getting profiling a bit cleaner
and more usable (it works now, but again the interpreted mode skews
the results a lot; knowing that CallNode.interpret is hit hard isn't
really helpful), and perhaps most importantly, pulling Ruby and Java
type systems closer together at the JVM level, to solve the "last
mile" of platform integration.
We welcome help on any and all of these, because they're problems any
language from off-platform will face...and making the JVM a better
home for unusual off-platform languages is perhaps the most important
task ahead of us.
- Charlie