interesting test tool... looking for opinions.

550 views
Skip to first unread message

Jeff Wood

unread,
Aug 3, 2005, 2:42:14 PM8/3/05
to
There is wonderful java-based testing unit framework called TestNG.

I'd love to see it's extensions to the standard xUnit style testing
get incorporated into our TestUnit objects...

Anyways, if/when any of you that care to, have the time, please take a
look at TestNG and let me know what you think of the idea of something
like that becoming the standard for Ruby?

Thanks ... I know I'm not in any place of power, but I just wanted to
see what people think.

j.

--
"So long, and thanks for all the fish"

Jeff Wood


Austin Ziegler

unread,
Aug 3, 2005, 5:45:52 PM8/3/05
to
On 8/3/05, Jeff Wood <jeff.da...@gmail.com> wrote:
> There is wonderful java-based testing unit framework called TestNG.
>
> I'd love to see it's extensions to the standard xUnit style testing
> get incorporated into our TestUnit objects...
>
> Anyways, if/when any of you that care to, have the time, please take a
> look at TestNG and let me know what you think of the idea of something
> like that becoming the standard for Ruby?
>
> Thanks ... I know I'm not in any place of power, but I just wanted to
> see what people think.

http://www.beust.com/testng/ appears to be the home page. What might
be useful is forwarding this to Nathaniel Talbott to see what he's
done toward creating the test/unit next generation code that he
presented last year at RubyConf and maybe see if the ideas with this
can be expressed in code that fits with his ideas. It might also be
worth going over the ideas that he's had with test/unit from last
year's presentation. I *think* Ryan Davis archives the presentations
at zenspider.org.

-austin
--
Austin Ziegler * halos...@gmail.com
* Alternate: aus...@halostatue.ca


Ryan Leavengood

unread,
Aug 3, 2005, 7:10:53 PM8/3/05
to
Jeff Wood said:
> There is wonderful java-based testing unit framework called TestNG.
>
> I'd love to see it's extensions to the standard xUnit style testing
> get incorporated into our TestUnit objects...
>
> Anyways, if/when any of you that care to, have the time, please take a
> look at TestNG and let me know what you think of the idea of something
> like that becoming the standard for Ruby?
>
> Thanks ... I know I'm not in any place of power, but I just wanted to
> see what people think.

I've taken just a very cursory look, but I imagine a Ruby equivalent could
be created fairly easily. In fact, a Ruby version could be much better.

For example, since TestNG test classes do not subclass from anything, one
is required to use the Java assert keyword for all assertions. That means
all those handy assertions provided by JUnit can no longer be used (unless
I missed some trick.) In Ruby, the Test::Unit::Assertions module could
just be mixed into Object and everything could use them.

Another possible issue is the fact that there is no equivalent to Java
annotations, though I've done some prototyping and figured out a way to
fairly easily have the equivalent in Ruby, at least for methods.

Still, I balk when I see XML config files, even if they are fairly simple

Austin Ziegler

unread,
Aug 3, 2005, 7:16:35 PM8/3/05
to

Alexandru Popescu

unread,
Aug 3, 2005, 8:00:39 PM8/3/05
to
#: Austin Ziegler changed the world a bit at a time by saying on 8/4/2005 1:16 AM :#

> On 8/3/05, Ryan Leavengood <mrc...@netrox.net> wrote:
>> Still, I balk when I see XML config files, even if they are fairly simple.
>> I imagine a Ruby version would use YAML instead, but I still don't like
>> the idea of external files. But maybe that is part of the power of TestNG.
>> Without making a lot of tests using TestNG I can't quite see all the
>> benefits or power.
>
> I prefer to think that a Ruby version would use Ruby.
>
> While I will often use configuration files (XML, YAML, or something
> else) when they are suitable (e.g., you don't want executable code),
> test configurations are all about running code. Therefore, these
> should themselves be executable code. At least IMO.
>
> -austin

Hi!

I am part of the TestNG team and your messages just raised my pulse (pretty happy right now).

I am a newbie to Ruby, but I really think I would like to work with somebody on porting the TestNG
functionality in Ruby.

Please let me know if somebody is interested in guiding me through Ruby. I can pay back with some
TestNG internal ;-).

thanks,
:alex |.::the_mindstorm::.|


Jeff Wood

unread,
Aug 3, 2005, 8:32:28 PM8/3/05
to
well, first we'll need to get matz input on adding attributes @ the
function level.

I mean, we know that methods are objects, and therefore can have their
own values & methods ( methods on methods ) <evilgrin> ... So, we just
need a good way to provide and use those attributes.

so:

class TestSuite

def testOne() :isa=test , :group="group1","group2"
# do stuff here
end

def testTwo() :isa=test , :group="group1" , :depends_on=testOne
# do stuff here too
end

def runTests()
methods( :isa, "test", :include, "group1", :exclude, "group2" ).run
end

end

The ability to categorize and filter a set of functions... then, to be
able to execute them all one after the other...

.. or something like that.

j.

Austin Ziegler

unread,
Aug 3, 2005, 9:03:01 PM8/3/05
to
On 8/3/05, Jeff Wood <jeff.da...@gmail.com> wrote:
> Well, first we'll need to get matz input on adding attributes @
> the function level.

I don't think so.

> I mean, we know that methods are objects, and therefore can have

> their own values & methods (methods on methods) <evilgrin> ... So,


> we just need a good way to provide and use those attributes.

> so:
> class TestSuite
> def testOne() :isa=test , :group="group1","group2"
> # do stuff here
> end
>
> def testTwo() :isa=test , :group="group1" , :depends_on=testOne
> # do stuff here too
> end
>
> def runTests()
> methods( :isa, "test", :include, "group1", :exclude, "group2" ).run
> end
> end

No need for attributes this way. Try this:

class MyTestSuite
extend Test::Unit::Testable # Hypothetical testable module

test_groups :group1, :group2
test_method
def test_one
# ...
end

test_groups :group1
test_dependency :test_one
test_method
def test_two
# ...
end
end

Alternately:

class MyTestSuite
extend Test::Unit::Testable

test_method :test_one, :groups => [ :group1, :group2 ] do
# ...
end

test_method :test_two, :groups => [ :group1 ], :depends => :test_one do
# ...
end
end

This latter uses a Rake-style test declaration and uses the block as
the test method.

Jeff Wood

unread,
Aug 3, 2005, 9:10:58 PM8/3/05
to
I like your second one better ... my only question would then be your
suggestion for how to deal with the invocation based on inclusion &|
exclusion of groups

.. I wasn't too far from yours but i agree yours is superiour.

j.

Austin Ziegler

unread,
Aug 3, 2005, 10:08:22 PM8/3/05
to
On 8/3/05, Jeff Wood <jeff.da...@gmail.com> wrote:
> I like your second one better ... my only question would then be your
> suggestion for how to deal with the invocation based on inclusion &|
> exclusion of groups
>
> ... I wasn't too far from yours but i agree yours is superiour.

Probably not much different than yours. test_method would add this
information to something attached to the class itself. I was playing
with the idea more than trying to create an implementation. See pages
25 and beyond of Nathaniel Talbott's 2004 presentation:

http://zenspider.com/dl/rubyconf2004/TestUnit.pdf

Alexandru Popescu

unread,
Aug 4, 2005, 2:45:44 AM8/4/05
to
#: Austin Ziegler changed the world a bit at a time by saying on 8/4/2005 4:08 AM :#

> On 8/3/05, Jeff Wood <jeff.da...@gmail.com> wrote:
>> I like your second one better ... my only question would then be your
>> suggestion for how to deal with the invocation based on inclusion &|
>> exclusion of groups
>>
>> ... I wasn't too far from yours but i agree yours is superiour.
>
> Probably not much different than yours. test_method would add this
> information to something attached to the class itself. I was playing
> with the idea more than trying to create an implementation. See pages
> 25 and beyond of Nathaniel Talbott's 2004 presentation:
>
> http://zenspider.com/dl/rubyconf2004/TestUnit.pdf
>
> -austin

Is there any metadata that can be associated with a method? I know it is an object, and maybe you
can attach some metadata to it. From that point on the things will be more simple.

:alex |.::the_mindstorm::.|


Pit Capitain

unread,
Aug 4, 2005, 2:55:40 AM8/4/05
to
Alexandru Popescu schrieb:

> Is there any metadata that can be associated with a method? I know it is
> an object, and maybe you can attach some metadata to it. From that point
> on the things will be more simple.

No, you can't attach metadata to methods, but you can attach them to the
class or module where the methods are defined. I think this should be
enough for something like TestNG. For more info look into the archives
of ruby-talk or ruby-core (I don't remember where).

Regards,
Pit


Jeff Wood

unread,
Aug 4, 2005, 3:06:18 AM8/4/05
to
Yeah, Why's got a great article in his blog on metaprogramming.

j.

Alexandru Popescu

unread,
Aug 4, 2005, 3:25:53 AM8/4/05
to
#: Jeff Wood changed the world a bit at a time by saying on 8/4/2005 9:06 AM :#

> Yeah, Why's got a great article in his blog on metaprogramming.
>
> j.
>

I think I have tried that article a couple of times, but my Ruby understanding was pretty young at
that moment. Maybe I will try it again.

:alex |.::the_mindstorm::.|

Jeff Wood

unread,
Aug 4, 2005, 3:33:06 AM8/4/05
to
There is also a really good section in PickAxe2 on metaprogramming.

j.

Reyn Vlietstra

unread,
Aug 4, 2005, 6:07:42 AM8/4/05
to
Attributes, metadata on methods are one of the things C# gave me that
I truely miss.
Maybe because I dont know ruby that well yet ...


--
Reyn Vlietstra


Alexandru Popescu

unread,
Aug 4, 2005, 6:59:37 AM8/4/05
to
#: Reyn Vlietstra changed the world a bit at a time by saying on 8/4/2005 12:07 PM :#

Does anyone still has the reference to the why blog entry about metadaata? thanks a lot

:alex |.::the_mindstorm::.|


Ryan Leavengood

unread,
Aug 4, 2005, 10:56:14 AM8/4/05
to
Ryan Leavengood said:
>
> Another possible issue is the fact that there is no equivalent to Java
> annotations, though I've done some prototyping and figured out a way to
> fairly easily have the equivalent in Ruby, at least for methods.

I see there has been quite a few additional discussions of this. I decided
to code up a more thorough prototype of how Java method annotations could
be simulated in Ruby:

class Class
attr_reader :configs, :groups, :test_methods
def config(hash)
@config_method = true
@temp_configs = []
hash.each do |key, value|
if value
@temp_configs << key
end
end
end

def test(hash)
@test_method = true
@current_groups = hash[:groups]
end

def method_added(method)
if @config_method or @test_method
if @current_groups
@groups||={}
@current_groups.each do |group|
(@groups[group]||=[]) << method
end
end
if @config_method
@configs||={}
@temp_configs.each do |config|
(@configs[config]||=[]) << method
end
end
if @test_method
(@test_methods||=[]) << method
end
end
@config_method = false
@current_groups = nil
@temp_configs=nil
@test_method = false
end
end

class Test
config :before_test => true
def setup
puts "Test#setup called"
end

config :after_test => true
def teardown
puts "Test#teardown called"
end

test :groups => ['one']
def test_method1
puts "Test#test_method1 called"
end

test :groups => ['two']
def test_method2
puts "Test#test_method2 called"
end
end

class Test2
config :before_test => true
def setup
puts "Test2#setup called"
end

config :after_test => true
def teardown
puts "Test2#teardown called"
end

test :groups => ['one']
def test_method1
puts "Test2#test_method1 called"
end

test :groups => ['two']
def test_method2
puts "Test2#test_method2 called"
end
end

def run_tests(tests, test_instance, test_class)
tests.each do |method|
if test_class.configs[:before_test]
test_class.configs[:before_test].each do |config_method|
test_instance.__send__(config_method)
end
end
test_instance.__send__(method)
if test_class.configs[:after_test]
test_class.configs[:after_test].each do |config_method|
test_instance.__send__(config_method)
end
end
end
end

if $0 == __FILE__
%w{Test Test2}.each do |klass|
klass = Module.const_get(klass)
test = klass.new
run_tests(klass.test_methods, test, klass)
klass.groups.each do |group, list|
puts "\nTesting group #{group}:"
run_tests(list, test, klass)
end
puts
end
end
__END__

Ryan


Cedric

unread,
Aug 4, 2005, 12:01:13 PM8/4/05
to
Hi everyone,

I am the creator of TestNG, and this is a very interesting discussion!

I'm a big Ruby fan myself and while I have toyed with the idea of
porting TestNG to Ruby, the need has never presented itself so far.
Good to see this is changing.

I am not too worried about the annotations part, since it can be easily
simulated, either with Ryan's trick above, or with similar ones such as
Aslak's ( http://tinyurl.com/6xjnj ).

The TestNG runtime is fairly heavy, though, although I am convinced it
would be much simpler to implement in Ruby, but it would probably still
represent most of the porting effort, especially if you want to support
some of the advanced functionalities (dependent methods or parallel
runs).

An interesting angle could also be to do away with testng.xml and
replace it with a Ruby definition file, which would probably be easier
to handle.

Feel free to email me if you'd be interested in undertaking such a
project, there has been a lot of progress on the TestNG tools front
these past weeks (IDEA and Maven plug-in mostly) and I would love to
see some of the innovative ideas that we put inside TestNG cross
borders and reach the Ruby community as well.

--
Cedric
http://testng.org

Jeff Wood

unread,
Aug 4, 2005, 1:34:36 PM8/4/05
to

Jeff Wood

unread,
Aug 4, 2005, 1:55:57 PM8/4/05
to
I'd love to be involved, we just need to come up with a good standard
way to do the attributes.

j.

Phil Tomson

unread,
Aug 4, 2005, 2:52:38 PM8/4/05
to
In article <42F1BBE4...@capitain.de>,

Why couldn't you add metadata to methods?

class Metod
def metadata
@metadata
end

def metadata= data
@metadata = data
end
end

def foo
"foo"
end


foometh = self.method(:foo)

foometh.metadata = "This method just says foo"

puts foometh.metadata #=> This method just says foo


Phil


Alexandru Popescu

unread,
Aug 4, 2005, 3:46:09 PM8/4/05
to
#: Phil Tomson changed the world a bit at a time by saying on 8/4/2005 9:16 PM :#
That was exactly what I have thought ... but haven't the means to express it. Thanks Phil.

:alex |.::the_mindstorm::.|


Jeff Wood

unread,
Aug 4, 2005, 4:12:11 PM8/4/05
to
The only thing I don't like about that solution is that you define the
metadata outside the method body.

I really think we should keep them together. Otherwise we're just
asking for things to get out of sync.

j.

On 8/4/05, Alexandru Popescu <the_mi...@evolva.ro> wrote:

Joel VanderWerf

unread,
Aug 4, 2005, 4:28:41 PM8/4/05
to
Phil Tomson wrote:
..

> Why couldn't you add metadata to methods?
>
> class Metod
> def metadata
> @metadata
> end
>
> def metadata= data
> @metadata = data
> end
> end
>
> def foo
> "foo"
> end
>
>
> foometh = self.method(:foo)
>
> foometh.metadata = "This method just says foo"
>
> puts foometh.metadata #=> This method just says foo

foometh2 = self.method(:foo)

puts foometh2.metadata #=> nil

The metadata doesn't really stay with the method, only with the Method
instance.

--
vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407


Phil Tomson

unread,
Aug 4, 2005, 4:08:28 PM8/4/05
to
In article <42F27E7F...@evolva.ro>,

Even better:

irb(main):001:0> class Object
irb(main):002:1> def metadata
irb(main):003:2> @metadata
irb(main):004:2> end
irb(main):005:1> def metadata= data
irb(main):006:2> @metadata = data
irb(main):007:2> end
irb(main):008:1> end
irb(main):021:0> class Foo
irb(main):022:1> def sayfoo
irb(main):023:2> "foo I said"
irb(main):024:2> end
irb(main):025:1> end
=> nil
irb(main):026:0> Foo.metadata = "This is a Foo class"
=> "This is a Foo class"
irb(main):027:0> Foo.metadata
=> "This is a Foo class"


That way you can put metadata on (just about) any object.

Phil

Jeff Wood

unread,
Aug 4, 2005, 4:59:24 PM8/4/05
to
if we do it that way, all we then need for the function attributes is
a way to get a reference to the method we're in the middle of
defining.

def foo
foo.metadata = "the foo function"
end

or something like that.

j.

Niki

unread,
Feb 6, 2013, 3:18:20 AM2/6/13
to
Is this implementation available now? I am newbie to ruby and have built a automation framework using selenium webdriver in ruby. Is it possible to use TestNg with ruby? Are there some tutorials available?

Thanks!!

garcia...@gmail.com

unread,
Apr 28, 2014, 11:56:00 AM4/28/14
to
I am also very interested in a TestNG solution for Ruby. Please let me know if there one

Thanks
Reply all
Reply to author
Forward
0 new messages