dynamic testing for 1.9isms under Ruby 1.8
Rich Morin
http://goruco2009.confreaks.com/30-may-2009-11-00-where-is-ruby-really-heading-gregory-brown.html,
he talks about migrating Ruby 1.8.6 code to Ruby 1.9.*. In
particular, he talks about setting up a test environment to
figure out what code "works" under which versions of Ruby.
I have a notion which I think is synergistic with this and may
also be relevant to Unity, RubySpec, etc. Basically, I'd like
to have a way to dynamically test code, running under 1.8, for
things that would not work properly on 1.9 (1.9.*, 2.0, ...).
For purposes of argument, let's assume that the code already
has a reasonably comprehensive test suite. So, we _could_
run it on 1.9 and "beat on it" until no errors are detected.
However, this approach has a few problems:
* Its effectiveness is limited by the completeness of the
test suite. If no test covers the problem, we lose.
* The errors that are detected may not be easy to connect
to the underlying problems in the code. That is, if a
1.8ism causes bad data to be handed off to other code,
the detected error might be several methods away.
* It requires a LOT of work. There needs to be a 1.9
test framework for the code, the code must be brought
to the point where it is capable of running under 1.9,
and any errors found by the tests must be resolved. If
someone is just "testing the waters" of migrating to
1.9, they may want a less demanding approach.
It would be nice to have an approach that runs under 1.8, on
our existing infrastructure, and simply warns us about code
that is unlikely to work properly under 1.9. I see two ways
to build such a tool: static analysis and dynamic checking.
Because Ruby is a dynamic language, static analysis is both
difficult to do and unable to test many types of errors. I'd
be happy to hear of static test suites, but I wouldn't expect
them to be all that effective.
So, I'm looking into creating a dynamic checking facility.
That is, a library that links in with an application and
watches for method calls that wouldn't work properly on 1.9.
Loading this library and running the test suite (and/or some
sort of limited production) should tell us quite a bit.
In order to create such a library, we'll need a definition of
Ruby 1.8 usage that is not valid under 1.9. RubySpec's lists
of "guarded specs" provide this, at least implicitly. (If a
spec is guarded against 1.9, it probably relies on a 1.8ism.)
However, simply extracting the relevant specs isn't as easy
as it could be (I'm working on that :-) and figuring out how
to create a method call "watcher" from a spec is likely to
take a bit of work. Given that there are about 900 guarded
specs (and counting), this work could add up.
Nonetheless, I think this is a useful approach to consider,
and I'd be happy to get comments and suggestions. Are there
ways that Unity could aid this effort? Are there existing
tools which might be relevant? Like that...
-r
P.S. I have some detailed discussion of this notion, with
some "proof of concept" code, in the thread:
http://groups.google.com/group/connascence/browse_thread/thread/628dcc3794e1b3f0
--
http://www.cfcl.com/rdm Rich Morin
http://www.cfcl.com/rdm/resume r...@cfcl.com
http://www.cfcl.com/rdm/weblog +1 650-873-7841
Technical editing and writing, programming, and web development