There is wonderful java-based testing unit framework called TestNG.
I'd love to see it's extensions to the standard xUnit style testing get incorporated into our TestUnit objects...
Anyways, if/when any of you that care to, have the time, please take a look at TestNG and let me know what you think of the idea of something like that becoming the standard for Ruby?
Thanks ... I know I'm not in any place of power, but I just wanted to see what people think.
On 8/3/05, Jeff Wood <jeff.darkli...@gmail.com> wrote:
> There is wonderful java-based testing unit framework called TestNG.
> I'd love to see it's extensions to the standard xUnit style testing > get incorporated into our TestUnit objects...
> Anyways, if/when any of you that care to, have the time, please take a > look at TestNG and let me know what you think of the idea of something > like that becoming the standard for Ruby?
> Thanks ... I know I'm not in any place of power, but I just wanted to > see what people think.
http://www.beust.com/testng/ appears to be the home page. What might be useful is forwarding this to Nathaniel Talbott to see what he's done toward creating the test/unit next generation code that he presented last year at RubyConf and maybe see if the ideas with this can be expressed in code that fits with his ideas. It might also be worth going over the ideas that he's had with test/unit from last year's presentation. I *think* Ryan Davis archives the presentations at zenspider.org.
> There is wonderful java-based testing unit framework called TestNG.
> I'd love to see it's extensions to the standard xUnit style testing > get incorporated into our TestUnit objects...
> Anyways, if/when any of you that care to, have the time, please take a > look at TestNG and let me know what you think of the idea of something > like that becoming the standard for Ruby?
> Thanks ... I know I'm not in any place of power, but I just wanted to > see what people think.
I've taken just a very cursory look, but I imagine a Ruby equivalent could be created fairly easily. In fact, a Ruby version could be much better.
For example, since TestNG test classes do not subclass from anything, one is required to use the Java assert keyword for all assertions. That means all those handy assertions provided by JUnit can no longer be used (unless I missed some trick.) In Ruby, the Test::Unit::Assertions module could just be mixed into Object and everything could use them.
Another possible issue is the fact that there is no equivalent to Java annotations, though I've done some prototyping and figured out a way to fairly easily have the equivalent in Ruby, at least for methods.
Still, I balk when I see XML config files, even if they are fairly simple
#: Austin Ziegler changed the world a bit at a time by saying on 8/4/2005 1:16 AM :#
> On 8/3/05, Ryan Leavengood <mrc...@netrox.net> wrote: >> Still, I balk when I see XML config files, even if they are fairly simple. >> I imagine a Ruby version would use YAML instead, but I still don't like >> the idea of external files. But maybe that is part of the power of TestNG. >> Without making a lot of tests using TestNG I can't quite see all the >> benefits or power.
> I prefer to think that a Ruby version would use Ruby.
> While I will often use configuration files (XML, YAML, or something > else) when they are suitable (e.g., you don't want executable code), > test configurations are all about running code. Therefore, these > should themselves be executable code. At least IMO.
> -austin
Hi!
I am part of the TestNG team and your messages just raised my pulse (pretty happy right now).
I am a newbie to Ruby, but I really think I would like to work with somebody on porting the TestNG functionality in Ruby.
Please let me know if somebody is interested in guiding me through Ruby. I can pay back with some TestNG internal ;-).
well, first we'll need to get matz input on adding attributes @ the function level.
I mean, we know that methods are objects, and therefore can have their own values & methods ( methods on methods ) <evilgrin> ... So, we just need a good way to provide and use those attributes.
so:
class TestSuite
def testOne() :isa=test , :group="group1","group2" # do stuff here end
def testTwo() :isa=test , :group="group1" , :depends_on=testOne # do stuff here too end
> #: Austin Ziegler changed the world a bit at a time by saying on 8/4/2005 1:16 AM :# > > On 8/3/05, Ryan Leavengood <mrc...@netrox.net> wrote: > >> Still, I balk when I see XML config files, even if they are fairly simple. > >> I imagine a Ruby version would use YAML instead, but I still don't like > >> the idea of external files. But maybe that is part of the power of TestNG. > >> Without making a lot of tests using TestNG I can't quite see all the > >> benefits or power.
> > I prefer to think that a Ruby version would use Ruby.
> > While I will often use configuration files (XML, YAML, or something > > else) when they are suitable (e.g., you don't want executable code), > > test configurations are all about running code. Therefore, these > > should themselves be executable code. At least IMO.
> > -austin
> Hi!
> I am part of the TestNG team and your messages just raised my pulse (pretty happy right now).
> I am a newbie to Ruby, but I really think I would like to work with somebody on porting the TestNG > functionality in Ruby.
> Please let me know if somebody is interested in guiding me through Ruby. I can pay back with some > TestNG internal ;-).
> I mean, we know that methods are objects, and therefore can have > their own values & methods (methods on methods) <evilgrin> ... So, > we just need a good way to provide and use those attributes. > so: > class TestSuite > def testOne() :isa=test , :group="group1","group2" > # do stuff here > end
> def testTwo() :isa=test , :group="group1" , :depends_on=testOne > # do stuff here too > end
> def runTests() > methods( :isa, "test", :include, "group1", :exclude, "group2" ).run > end > end
No need for attributes this way. Try this:
class MyTestSuite extend Test::Unit::Testable # Hypothetical testable module
test_groups :group1, :group2 test_method def test_one # ... end
test_groups :group1 test_dependency :test_one test_method def test_two # ... end end
Alternately:
class MyTestSuite extend Test::Unit::Testable
test_method :test_one, :groups => [ :group1, :group2 ] do # ... end
test_method :test_two, :groups => [ :group1 ], :depends => :test_one do # ... end end
This latter uses a Rake-style test declaration and uses the block as the test method.
I like your second one better ... my only question would then be your suggestion for how to deal with the invocation based on inclusion &| exclusion of groups
.. I wasn't too far from yours but i agree yours is superiour.
j.
On 8/3/05, Austin Ziegler <halosta...@gmail.com> wrote:
> On 8/3/05, Jeff Wood <jeff.darkli...@gmail.com> wrote: > > Well, first we'll need to get matz input on adding attributes @ > > the function level.
> I don't think so.
> > I mean, we know that methods are objects, and therefore can have > > their own values & methods (methods on methods) <evilgrin> ... So, > > we just need a good way to provide and use those attributes.
> > so: > > class TestSuite > > def testOne() :isa=test , :group="group1","group2" > > # do stuff here > > end
> > def testTwo() :isa=test , :group="group1" , :depends_on=testOne > > # do stuff here too > > end
> > def runTests() > > methods( :isa, "test", :include, "group1", :exclude, "group2" ).run > > end > > end
> No need for attributes this way. Try this:
> class MyTestSuite > extend Test::Unit::Testable # Hypothetical testable module
On 8/3/05, Jeff Wood <jeff.darkli...@gmail.com> wrote:
> I like your second one better ... my only question would then be your > suggestion for how to deal with the invocation based on inclusion &| > exclusion of groups
> ... I wasn't too far from yours but i agree yours is superiour.
Probably not much different than yours. test_method would add this information to something attached to the class itself. I was playing with the idea more than trying to create an implementation. See pages 25 and beyond of Nathaniel Talbott's 2004 presentation:
#: Austin Ziegler changed the world a bit at a time by saying on 8/4/2005 4:08 AM :#
> On 8/3/05, Jeff Wood <jeff.darkli...@gmail.com> wrote: >> I like your second one better ... my only question would then be your >> suggestion for how to deal with the invocation based on inclusion &| >> exclusion of groups
>> ... I wasn't too far from yours but i agree yours is superiour.
> Probably not much different than yours. test_method would add this > information to something attached to the class itself. I was playing > with the idea more than trying to create an implementation. See pages > 25 and beyond of Nathaniel Talbott's 2004 presentation:
Is there any metadata that can be associated with a method? I know it is an object, and maybe you can attach some metadata to it. From that point on the things will be more simple.
> Is there any metadata that can be associated with a method? I know it is > an object, and maybe you can attach some metadata to it. From that point > on the things will be more simple.
No, you can't attach metadata to methods, but you can attach them to the class or module where the methods are defined. I think this should be enough for something like TestNG. For more info look into the archives of ruby-talk or ruby-core (I don't remember where).
> Alexandru Popescu schrieb: > > Is there any metadata that can be associated with a method? I know it is > > an object, and maybe you can attach some metadata to it. From that point > > on the things will be more simple.
> No, you can't attach metadata to methods, but you can attach them to the > class or module where the methods are defined. I think this should be > enough for something like TestNG. For more info look into the archives > of ruby-talk or ruby-core (I don't remember where).
> On 8/3/05, Pit Capitain <p...@capitain.de> wrote: >> Alexandru Popescu schrieb: >> > Is there any metadata that can be associated with a method? I know it is >> > an object, and maybe you can attach some metadata to it. From that point >> > on the things will be more simple.
>> No, you can't attach metadata to methods, but you can attach them to the >> class or module where the methods are defined. I think this should be >> enough for something like TestNG. For more info look into the archives >> of ruby-talk or ruby-core (I don't remember where).
> #: Jeff Wood changed the world a bit at a time by saying on 8/4/2005 9:06 AM :# > > Yeah, Why's got a great article in his blog on metaprogramming.
> > j.
> I think I have tried that article a couple of times, but my Ruby understanding was pretty young at > that moment. Maybe I will try it again.
> :alex |.::the_mindstorm::.|
> > On 8/3/05, Pit Capitain <p...@capitain.de> wrote: > >> Alexandru Popescu schrieb: > >> > Is there any metadata that can be associated with a method? I know it is > >> > an object, and maybe you can attach some metadata to it. From that point > >> > on the things will be more simple.
> >> No, you can't attach metadata to methods, but you can attach them to the > >> class or module where the methods are defined. I think this should be > >> enough for something like TestNG. For more info look into the archives > >> of ruby-talk or ruby-core (I don't remember where).
> There is also a really good section in PickAxe2 on metaprogramming.
> j.
> On 8/4/05, Alexandru Popescu <the_mindst...@evolva.ro> wrote: > > #: Jeff Wood changed the world a bit at a time by saying on 8/4/2005 9:06 AM :# > > > Yeah, Why's got a great article in his blog on metaprogramming.
> > > j.
> > I think I have tried that article a couple of times, but my Ruby understanding was pretty young at > > that moment. Maybe I will try it again.
> > :alex |.::the_mindstorm::.|
> > > On 8/3/05, Pit Capitain <p...@capitain.de> wrote: > > >> Alexandru Popescu schrieb: > > >> > Is there any metadata that can be associated with a method? I know it is > > >> > an object, and maybe you can attach some metadata to it. From that point > > >> > on the things will be more simple.
> > >> No, you can't attach metadata to methods, but you can attach them to the > > >> class or module where the methods are defined. I think this should be > > >> enough for something like TestNG. For more info look into the archives > > >> of ruby-talk or ruby-core (I don't remember where).
> Attributes, metadata on methods are one of the things C# gave me that > I truely miss. > Maybe because I dont know ruby that well yet ...
> On 8/4/05, Jeff Wood <jeff.darkli...@gmail.com> wrote: >> There is also a really good section in PickAxe2 on metaprogramming.
>> j.
>> On 8/4/05, Alexandru Popescu <the_mindst...@evolva.ro> wrote: >> > #: Jeff Wood changed the world a bit at a time by saying on 8/4/2005 9:06 AM :# >> > > Yeah, Why's got a great article in his blog on metaprogramming.
>> > > j.
>> > I think I have tried that article a couple of times, but my Ruby understanding was pretty young at >> > that moment. Maybe I will try it again.
>> > :alex |.::the_mindstorm::.|
>> > > On 8/3/05, Pit Capitain <p...@capitain.de> wrote: >> > >> Alexandru Popescu schrieb: >> > >> > Is there any metadata that can be associated with a method? I know it is >> > >> > an object, and maybe you can attach some metadata to it. From that point >> > >> > on the things will be more simple.
>> > >> No, you can't attach metadata to methods, but you can attach them to the >> > >> class or module where the methods are defined. I think this should be >> > >> enough for something like TestNG. For more info look into the archives >> > >> of ruby-talk or ruby-core (I don't remember where).
>> > >> Regards, >> > >> Pit
>> -- >> "So long, and thanks for all the fish"
>> Jeff Wood
Does anyone still has the reference to the why blog entry about metadaata? thanks a lot
> Another possible issue is the fact that there is no equivalent to Java > annotations, though I've done some prototyping and figured out a way to > fairly easily have the equivalent in Ruby, at least for methods.
I see there has been quite a few additional discussions of this. I decided to code up a more thorough prototype of how Java method annotations could be simulated in Ruby:
class Class attr_reader :configs, :groups, :test_methods def config(hash) @config_method = true @temp_configs = [] hash.each do |key, value| if value @temp_configs << key end end end
def test(hash) @test_method = true @current_groups = hash[:groups] end
def method_added(method) if @config_method or @test_method if @current_groups @groups||={} @current_groups.each do |group| (@groups[group]||=[]) << method end end if @config_method @configs||={} @temp_configs.each do |config| (@configs[config]||=[]) << method end end if @test_method (@test_methods||=[]) << method end end @config_method = false @current_groups = nil @temp_configs=nil @test_method = false end end
class Test config :before_test => true def setup puts "Test#setup called" end
config :after_test => true def teardown puts "Test#teardown called" end
test :groups => ['one'] def test_method1 puts "Test#test_method1 called" end
test :groups => ['two'] def test_method2 puts "Test#test_method2 called" end end
class Test2 config :before_test => true def setup puts "Test2#setup called" end
config :after_test => true def teardown puts "Test2#teardown called" end
test :groups => ['one'] def test_method1 puts "Test2#test_method1 called" end
test :groups => ['two'] def test_method2 puts "Test2#test_method2 called" end end
def run_tests(tests, test_instance, test_class) tests.each do |method| if test_class.configs[:before_test] test_class.configs[:before_test].each do |config_method| test_instance.__send__(config_method) end end test_instance.__send__(method) if test_class.configs[:after_test] test_class.configs[:after_test].each do |config_method| test_instance.__send__(config_method) end end end end
if $0 == __FILE__ %w{Test Test2}.each do |klass| klass = Module.const_get(klass) test = klass.new run_tests(klass.test_methods, test, klass) klass.groups.each do |group, list| puts "\nTesting group #{group}:" run_tests(list, test, klass) end puts end end __END__
I am the creator of TestNG, and this is a very interesting discussion!
I'm a big Ruby fan myself and while I have toyed with the idea of porting TestNG to Ruby, the need has never presented itself so far. Good to see this is changing.
I am not too worried about the annotations part, since it can be easily simulated, either with Ryan's trick above, or with similar ones such as Aslak's ( http://tinyurl.com/6xjnj ).
The TestNG runtime is fairly heavy, though, although I am convinced it would be much simpler to implement in Ruby, but it would probably still represent most of the porting effort, especially if you want to support some of the advanced functionalities (dependent methods or parallel runs).
An interesting angle could also be to do away with testng.xml and replace it with a Ruby definition file, which would probably be easier to handle.
Feel free to email me if you'd be interested in undertaking such a project, there has been a lot of progress on the TestNG tools front these past weeks (IDEA and Maven plug-in mostly) and I would love to see some of the innovative ideas that we put inside TestNG cross borders and reach the Ruby community as well.
> #: Reyn Vlietstra changed the world a bit at a time by saying on 8/4/2005 12:07 PM :# > > Attributes, metadata on methods are one of the things C# gave me that > > I truely miss. > > Maybe because I dont know ruby that well yet ...
> > On 8/4/05, Jeff Wood <jeff.darkli...@gmail.com> wrote: > >> There is also a really good section in PickAxe2 on metaprogramming.
> >> j.
> >> On 8/4/05, Alexandru Popescu <the_mindst...@evolva.ro> wrote: > >> > #: Jeff Wood changed the world a bit at a time by saying on 8/4/2005 9:06 AM :# > >> > > Yeah, Why's got a great article in his blog on metaprogramming.
> >> > > j.
> >> > I think I have tried that article a couple of times, but my Ruby understanding was pretty young at > >> > that moment. Maybe I will try it again.
> >> > :alex |.::the_mindstorm::.|
> >> > > On 8/3/05, Pit Capitain <p...@capitain.de> wrote: > >> > >> Alexandru Popescu schrieb: > >> > >> > Is there any metadata that can be associated with a method? I know it is > >> > >> > an object, and maybe you can attach some metadata to it. From that point > >> > >> > on the things will be more simple.
> >> > >> No, you can't attach metadata to methods, but you can attach them to the > >> > >> class or module where the methods are defined. I think this should be > >> > >> enough for something like TestNG. For more info look into the archives > >> > >> of ruby-talk or ruby-core (I don't remember where).
> >> > >> Regards, > >> > >> Pit
> >> -- > >> "So long, and thanks for all the fish"
> >> Jeff Wood
> Does anyone still has the reference to the why blog entry about metadaata? thanks a lot
> I am the creator of TestNG, and this is a very interesting discussion!
> I'm a big Ruby fan myself and while I have toyed with the idea of > porting TestNG to Ruby, the need has never presented itself so far. > Good to see this is changing.
> I am not too worried about the annotations part, since it can be easily > simulated, either with Ryan's trick above, or with similar ones such as > Aslak's ( http://tinyurl.com/6xjnj ).
> The TestNG runtime is fairly heavy, though, although I am convinced it > would be much simpler to implement in Ruby, but it would probably still > represent most of the porting effort, especially if you want to support > some of the advanced functionalities (dependent methods or parallel > runs).
> An interesting angle could also be to do away with testng.xml and > replace it with a Ruby definition file, which would probably be easier > to handle.
> Feel free to email me if you'd be interested in undertaking such a > project, there has been a lot of progress on the TestNG tools front > these past weeks (IDEA and Maven plug-in mostly) and I would love to > see some of the innovative ideas that we put inside TestNG cross > borders and reach the Ruby community as well.
In article <42F1BBE4.8020...@capitain.de>, Pit Capitain <p...@capitain.de> wrote:
>Alexandru Popescu schrieb: >> Is there any metadata that can be associated with a method? I know it is >> an object, and maybe you can attach some metadata to it. From that point >> on the things will be more simple.
>No, you can't attach metadata to methods, but you can attach them to the >class or module where the methods are defined. I think this should be >enough for something like TestNG. For more info look into the archives >of ruby-talk or ruby-core (I don't remember where).
Why couldn't you add metadata to methods?
class Metod def metadata @metadata end
def metadata= data @metadata = data end end
def foo "foo" end
foometh = self.method(:foo)
foometh.metadata = "This method just says foo"
puts foometh.metadata #=> This method just says foo
> In article <42F1BBE4.8020...@capitain.de>, > Pit Capitain <p...@capitain.de> wrote: >>Alexandru Popescu schrieb: >>> Is there any metadata that can be associated with a method? I know it is >>> an object, and maybe you can attach some metadata to it. From that point >>> on the things will be more simple.
>>No, you can't attach metadata to methods, but you can attach them to the >>class or module where the methods are defined. I think this should be >>enough for something like TestNG. For more info look into the archives >>of ruby-talk or ruby-core (I don't remember where).
> Why couldn't you add metadata to methods?
> class Metod > def metadata > @metadata > end
> def metadata= data > @metadata = data > end > end
> def foo > "foo" > end
> foometh = self.method(:foo)
> foometh.metadata = "This method just says foo"
> puts foometh.metadata #=> This method just says foo
> Phil
That was exactly what I have thought ... but haven't the means to express it. Thanks Phil.
> #: Phil Tomson changed the world a bit at a time by saying on 8/4/2005 9:16 PM :# > > In article <42F1BBE4.8020...@capitain.de>, > > Pit Capitain <p...@capitain.de> wrote: > >>Alexandru Popescu schrieb: > >>> Is there any metadata that can be associated with a method? I know it is > >>> an object, and maybe you can attach some metadata to it. From that point > >>> on the things will be more simple.
> >>No, you can't attach metadata to methods, but you can attach them to the > >>class or module where the methods are defined. I think this should be > >>enough for something like TestNG. For more info look into the archives > >>of ruby-talk or ruby-core (I don't remember where).
> > Why couldn't you add metadata to methods?
> > class Metod > > def metadata > > @metadata > > end
> > def metadata= data > > @metadata = data > > end > > end
> > def foo > > "foo" > > end
> > foometh = self.method(:foo)
> > foometh.metadata = "This method just says foo"
> > puts foometh.metadata #=> This method just says foo
> > Phil
> That was exactly what I have thought ... but haven't the means to express it. Thanks Phil.
>#: Phil Tomson changed the world a bit at a time by saying on 8/4/2005 >9:16 PM :# >> In article <42F1BBE4.8020...@capitain.de>, >> Pit Capitain <p...@capitain.de> wrote: >>>Alexandru Popescu schrieb: >>>> Is there any metadata that can be associated with a method? I know it is >>>> an object, and maybe you can attach some metadata to it. From that point >>>> on the things will be more simple.
>>>No, you can't attach metadata to methods, but you can attach them to the >>>class or module where the methods are defined. I think this should be >>>enough for something like TestNG. For more info look into the archives >>>of ruby-talk or ruby-core (I don't remember where).
>> Why couldn't you add metadata to methods?
>> class Metod >> def metadata >> @metadata >> end
>> def metadata= data >> @metadata = data >> end >> end
>> def foo >> "foo" >> end
>> foometh = self.method(:foo)
>> foometh.metadata = "This method just says foo"
>> puts foometh.metadata #=> This method just says foo
>> Phil
>That was exactly what I have thought ... but haven't the means to >express it. Thanks Phil.
Even better:
irb(main):001:0> class Object irb(main):002:1> def metadata irb(main):003:2> @metadata irb(main):004:2> end irb(main):005:1> def metadata= data irb(main):006:2> @metadata = data irb(main):007:2> end irb(main):008:1> end irb(main):021:0> class Foo irb(main):022:1> def sayfoo irb(main):023:2> "foo I said" irb(main):024:2> end irb(main):025:1> end => nil irb(main):026:0> Foo.metadata = "This is a Foo class" => "This is a Foo class" irb(main):027:0> Foo.metadata => "This is a Foo class"
That way you can put metadata on (just about) any object.
