2. The TeSLa docs give this example, comparing Test::Unit and TesLa:
class TestCatalog < Test::Unit::TestCase def test_add_item catalog = Catalog.new() catalog.items = [:item1, :item2, :item3] catalog.add_item :item4 assert_equal(catalog.size, 4, "length should be 4") end end
would look like this in TeSLa
test_method :add_item => [:item4] do requires {@items = [:item1, :item2, :item3]} assert {size == 4} end
How does the TeSLa version know what :item4 is for, or where it goes, or what 'size' refers to? The example has no reference to Catalog; how does the code know to test that class?
> How does the TeSLa version know what :item4 is for, or where > it goes, or what 'size' refers to? The example has no > reference to Catalog; how does the code know to test that class?
I know nothing about TeSLA, but I can make a guess.
> test_method :add_item => [:item4] do > requires {@items = [:item1, :item2, :item3]} > assert {size == 4} > end
This creates a testcase that will call the "add_item" method on an instance of the class (I'm guessing that you define test_method within the context of a class). Before calling the method, it will set the instance variable @items to the value [:item1, :item2, :item3]. It will then pass that arguments [:item4] to the method, and afterwards call the size method and ensure that it equals 4. It does the equivalent of:
> -----Original Message----- > From: James Britt [mailto:jame...@neurogami.com] > Sent: Monday, 31 October 2005 4:41 PM > To: ruby-talk ML > Subject: Re: <ANN> TeSLa, a Domain Specific Language for Unit Testing
> javierg1...@gmail.com wrote: > > Hi > > I just posted version 0.1.0 of TeSLa, a Domain Specific > Language (DSL) > > for Unit Testing. > > You can download TeSLa along with a small example script from > > http://theniceweb.com/projects/tesla/tesla.zip (zip) or here > > http://theniceweb.com/projects/tesla/tesla.tar.gz (tar.gz) I also > > posted a small article/tutorial explaining the rationale and use of > > TeSLa here
> 2. The TeSLa docs give this example, comparing Test::Unit and TesLa:
> class TestCatalog < Test::Unit::TestCase > def test_add_item > catalog = Catalog.new() > catalog.items = [:item1, :item2, :item3] > catalog.add_item :item4 > assert_equal(catalog.size, 4, "length should be 4") > end > end
> would look like this in TeSLa
> test_method :add_item => [:item4] do > requires {@items = [:item1, :item2, :item3]} > assert {size == 4} > end
> How does the TeSLa version know what :item4 is for, or where > it goes, or what 'size' refers to? The example has no > reference to Catalog; how does the code know to test that class?
########################################################################### ########## This email has been scanned by MailMarshal, an email content filter. ########################################################################### ##########
Well, it doesn't quite works that way (the implementation, that is... I tried to avoid using 'eval' as long as I could) but yes, all references to properties or methods are to an instance of the class where the tests are specified. If you take a look at the example.rb script included in the package, you'll notice
test_method :add_item => [:item4] do requires {@items = [:item1, :item2, :item3]} assert {size == 4} end
is indeed declared just bellow the 'add_item' method in class Catalog and that 'items' is a property of that class. 'test_method' will create a clean instance of class Catalog and execute the 'add_item' method with :item4 as a parameter and 'assert' will check that 'size' (of the Catalog instance) is indeed 4 as long of the 'precondition' stated by requires was met. As for behavior-driven development, I never knew it existed, thanks a lot for the link, I'm going to have a look at it right away. By the way, I never intended for TeSLa to be a "pure" unit testing framework, maybe even include concepts from Eiffel's Design By Contract (so far that's just an idea I'm toying with) so let me know of any interesting concept you might have about testing in general and I'll take a look at it. Again, let me know of ways you think the syntax might be more readable/intuitive (even though I know that's a subject where people will never totally agree) Thanks a lot for the feedback
obscured by code wrote: > Well, it doesn't quite works that way (the implementation, that is... I > tried to avoid using 'eval' as long as I could) but yes, all references > to properties or methods are to an instance of the class where the > tests are specified. If you take a look at the example.rb script > included in the package, you'll notice
> test_method :add_item => [:item4] do > requires {@items = [:item1, :item2, :item3]} > assert {size == 4} > end
> is indeed declared just bellow the 'add_item' method in class Catalog > and that 'items' is a property of that class. 'test_method' will create > a clean instance of class Catalog and execute the 'add_item' method > with :item4 as a parameter and 'assert' will check that 'size' (of the > Catalog instance) is indeed 4 as long of the 'precondition' stated by > requires was met.
Ah. Thank you for the clarification. I often put my unit tests in the same source file as the class code, and being able to have tests right next to methods is quite slick.
> As for behavior-driven development, I never knew it existed, thanks a > lot for the link, I'm going to have a look at it right away. > By the way, I never intended for TeSLa to be a "pure" unit testing > framework, maybe even include concepts from Eiffel's Design By Contract > (so far that's just an idea I'm toying with) so let me know of any > interesting concept you might have about testing in general and I'll > take a look at it.
I'm interested in the idea of having executable comments, so that the rdoc for a method both renders as examples for the reader, and can be run as unit tests. I have no idea how your work might fit into that, though. One out-of-the-blue idea: an rdoc directive call that grabs the TeSLa code and translates it into an example using the normal calling syntax.
So,
test_method :add_item => [:item4] do requires {@items = [:item1, :item2, :item3]} assert {size == 4} end
obscured by code wrote: > test_method :add_item => [:item4] do > requires {@items = [:item1, :item2, :item3]} > assert {size == 4} > end
> is indeed declared just bellow the 'add_item' method in class Catalog > and that 'items' is a property of that class. 'test_method' will create > a clean instance of class Catalog and execute the 'add_item' method > with :item4 as a parameter and 'assert' will check that 'size' (of the > Catalog instance) is indeed 4 as long of the 'precondition' stated by > requires was met.
So what part of the code makes {@items = [:...]} true?
No part of the code makes that line true, that line is a precondition to the success of the assert once the method is run. That is, requires {@items = [:item1, :item2, :item3]} is executed *before* add_item => [:item4] Look at it as initialization code if you like. In other words, it is *required* for the 'items' property to hold items 1 to 3 so the addition of a fourth one would make the assert {size == 4} line true. Below is the entire Catalog class from example.rb. Perhaps that would help make things clearer
class ShoppingCart
def add_item item @items << item end
def size @items.length end
attr_accessor :items
test_method :add_item => [:item5] do
# requires makes sure certain conditions are met before the # assert is run
Well, I think your confusion stems from my use of the word "precondition". Thinking back, I realize it might have been better if I'd have used a different wording. In Eiffel a precondition is indeed (and I quote from Meyer's OOSC) "The boolean expression which defines the domain", but in this thread I've been using the word in the sense of: "A condition that must exist or be established before something can occur or be considered; a prerequisite." The word 'requires' in the code is therefore not used in the sense of the Eiffel keyword but rather to imply that what follows it is a *requirement* for 'assert' to succeed. In plain English, the following segment of code
test_method :add_item => [:item4] do requires {@items = [:item1, :item2, :item3]} assert {size == 4} end
should be read as, "it is required for the 'items' property of any Catalog instance to hold three items if the condition of 'size' being 4 is to be met after the method 'add_item' is called with an 'item' as parameter". All this leads me to think maybe a better syntax is, well... required (no pun intended). Any suggestions regarding this will be very much welcomed. As an aside note, I've been reading Dave Astels' proposal of a Behavior Driven approach (thanks James) and tend to agree with much of what he proposes. Perhaps I could use some ideas (and syntax) from BDD to make TeSLa more readable/intuitive. I'd still use Test::Unit underneath, though, as maintaining tool integration is still very important to me. Again, I'm very open to suggestions.
James Britt <jame...@neurogami.com> writes: > Ryan Leavengood wrote:
>> What about: >> test_method :add_item => [:item4] do >> before {@items = [:item1, :item2, :item3]} >> after {size == 4} >> end >> Or variations thereof.
> Such as
> test_method :add_item => [:item4] do > given {@items = [:item1, :item2, :item3]} > expected {size == 4} > end
Continuing the brainstorm (I used "assuming" before "given", but that reads better):
On 11/3/05, Christian Neukirchen <chneukirc...@gmail.com> wrote:
> I'm still not sure whether instance_eval'ing tests is a good idea.
That's my concern too. I'm not sure I trust the validity of of the tests since they are applied in a way that they are not used. This type of testing also ties the tests to a particular implementation. If you change the implementation (not the functionality), you'll likely have to change the testing since you are directly setting and accessing instance variables.
On the other hand, with some types of objects, you may not want to expose very much internal state at the API and this type of testing may make it much easier. I can draw an analogy from this to the hardware world - DFT (design for test) and ATPG (automatic test pattern generation) - where it is common practice to add control and observe points for testing (most state points at least). That is testing for defects, but even for functional tests it is common to at least have observability at some internal points in a chip. BTW, this is where I draw many of my software testing ideas from. I think the software world can learn some things from the hardware world in terms of testing.
Eric Mahurin <eric.mahu...@gmail.com> writes: > On 11/3/05, Christian Neukirchen <chneukirc...@gmail.com> wrote:
>> I'm still not sure whether instance_eval'ing tests is a good idea.
> That's my concern too. I'm not sure I trust the validity of of the tests > since they are applied in a way that they are not used. This type of testing > also ties the tests to a particular implementation. If you change the > implementation (not the functionality), you'll likely have to change the > testing since you are directly setting and accessing instance variables.
Also, how do you develop test-first with non-existing instance variables?
At least, one should make two groups of test, one that test "blackbox", and one that test the actual implementation. I'd write the blackbox test first, and the whitebox tests in parallel to the implementation. That way, you can oversee the programming and still test for implementation details. The latter tests will need to be thrown away on a reimplementation, of course.
> On the other hand, with some types of objects, you may not want to expose > very much internal state at the API and this type of testing may make it > much easier. I can draw an analogy from this to the hardware world - DFT > (design for test) and ATPG (automatic test pattern generation) - where it is > common practice to add control and observe points for testing (most state > points at least). That is testing for defects, but even for functional tests > it is common to at least have observability at some internal points in a > chip. BTW, this is where I draw many of my software testing ideas from. I > think the software world can learn some things from the hardware world in > terms of testing.
I recently searched a bit about "behavior driven design" and found a paper on "spec driven design"---which turned out to be about hardware. -- Christian Neukirchen <chneukirc...@gmail.com> http://chneukirchen.org
-- -- Jim Weirich j...@weirichhouse.org http://onestepback.org ----------------------------------------------------------------- "Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas)
Thank you all for your responses, I though on posting here some of my opinions but it turned out to be too big so I decided I'd better blog about it. You can find the article here http://www.theniceweb.com/JaviersBlog/2005/11/ruby-20-and-tesla-upcom... Oh, would you guys please elaborate on "I'm not sure I trust the validity of the tests since they are applied in a way that they are not used." I don't really understand what you mean by that.
> Thank you all for your responses, I though on posting here some of my > opinions but it turned out to be too big so I decided I'd better blog > about it. You can find the article here
I know it's not exactly what you asked for when you've written this post, but your comments on your blog about possible mandatory parentheses in Ruby2 concern me. Can people read Javier's blog and comment on that part? (if possible by saying that it ain't gonna happen so!) Thanks in advance. -- Christophe Grandsire.
On 11/4/05, obscured by code <javierg1...@gmail.com> wrote:
> Thank you all for your responses, I though on posting here some of my > opinions but it turned out to be too big so I decided I'd better blog > about it. You can find the article here
Sometimes, you can get different results by calling the method without a receiver (what you do in this testing infrastructure) vs. with a receiver (what you do when you use this class). I think the main problem revolves around the use of #method_missing. If you declare a private method #foo and also #method_missing, calling #foo without a receiver will give you the private method #foo and calling #foo with a receiver will put you in #method_missing. In general you are not testing with the protections that you have from outside the class. There may be other suttle issues to be concerned about. I don't think this invalidates your approach, but it should be a concern.
Jim Weirich <j...@weirichhouse.org> writes: > On Thursday 03 November 2005 10:41 am, Christian Neukirchen wrote: >> I recently searched a bit about "behavior driven design" and found a >> paper on "spec driven design"---which turned out to be about hardware.
Christophe Grandsire <christophe.grands...@free.fr> writes: > Selon obscured by code <javierg1...@gmail.com>:
>> Thank you all for your responses, I though on posting here some of my >> opinions but it turned out to be too big so I decided I'd better blog >> about it. You can find the article here
> I know it's not exactly what you asked for when you've written this post, but > your comments on your blog about possible mandatory parentheses in Ruby2 > concern me. Can people read Javier's blog and comment on that part? (if > possible by saying that it ain't gonna happen so!) Thanks in advance.
I don't think the behavior will be changed compared to 1.8.2: assert raises(ZeroDivisionError) and attr_accessor :items will stay valid. (IANM, but I'd be very sad if parens are enforced.)
> Yeah, I've seen those already. It's all rehashing the same content. > ;) I'd like to see code...
gem install rspec
-- -- Jim Weirich j...@weirichhouse.org http://onestepback.org ----------------------------------------------------------------- "Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas)
>"Sometimes, you can get different results by calling the method without a >receiver (what you do in this testing infrastructure)"
Well, it might be a matter of wording, maybe I'm not quite getting what you say (English is not, after all, my mother tongue), but I totally fail to see why you think methods get called without a receiver inside TeSLa. As far as my understanding of what a receiver is, they don't. In line 31 of tesla.rb you can see the following @instance = self.new self being the class where the tests are being defined. Later on, inside method #assert (line 105 to be precise) you find the following result = @instance.send(@method_name, *@params) As you can see, there's no eval_ing of the method being tested, the method is being sent to a previously created instance of the class in which both the test and the method being tested are defined. It's true *right now* you can directly test private methods, but that is intentional, as the inability to test private functionality is a common complain from people using other testing frameworks, and as such, something I wanted TeSLa to offer. Also, testing a private method will always end up in the private method being called, regardless of #method_missing being defined in the class or not. I stressed *right now* because in Ruby 1.9 you can't use Object#send with private methods, so as it's currently implemented TeSLa won't be able to do that. By the time Ruby 1.9 is out (in a stable form, that is) I'll be forced to either create an explicit syntax for doing tests on private methods of trying to work some inner magic with the newly introduced Object#funcall. By the way, the only thing that gets eval_ed in TeSLa are the Test::Unit test methods, but that's only because I want to rely on Test::Unit reporting and tool integration. If TeSLa were to do its own test report (which by the way it does, only it's commented out) there would be no evals in the code.
