 |
Newsgroups: perl.perl6.compiler
From: ste...@iinteractive.com (Stevan Little)
Date: Tue, 3 May 2005 10:36:40 -0400
Local: Tues, May 3 2005 10:36 am
Subject: Yet Another TODO Model for Test.pm
The ongoing search to find a good model for TODO tests in Pugs
continues ... To start with, the idea of some kind of "switch" to turn all failures Second, I would like to move away from the force_todo() usage, as it It was determined (on #perl6 and various emails) that what we really - TODO :todo, :todo(1) Sometimes, it just doesn't fit anywhere else, so in that case it is - TODO bug :todo<bug>, todo => 'bug', :todo('bug') This is either an outright bug in Pugs, or it is a feature which - TODO feature :todo<feature>, todo => 'feature', :todo('feature') These are tests written for perl6 features which are not yet - TODO parsefail :todo<parsefail>, todo => 'parsefail', :todo('parsefail') These are tests which fail to parse correctly. These are usually tested - TODO hardfail :todo<hardfail>, todo => 'hardfail', :todo('hardfail') These are tests which parse okay, but die with an uncatchable error. I My goal was to support these different kinds of TODOs with Test.pm, and So currently (in the latest revision), Test.pm will support the ok(bad_function(), "... testing bad_function", :todo<bug>); this will produce the following TAP output: not ok 1 - ... testing bad_function # TODO bug I have also altered our local copy of Test::Harness (in inc/) so that t/my_test_file....ok This allows a simple top-level overview of not only the number of TODO Now all that is left is for people to start changing all the :todo As always comments, question and suggestions are always welcome. Thanks, Stevan You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
| |||||||||||||
|
Newsgroups: perl.perl6.compiler
From: kolib...@graystudios.org (Nathan Gray)
Date: Tue, 3 May 2005 12:32:53 -0400
Local: Tues, May 3 2005 12:32 pm
Subject: Re: Yet Another TODO Model for Test.pm
On Tue, May 03, 2005 at 10:36:40AM -0400, Stevan Little wrote: Good. > The ongoing search to find a good model for TODO tests in Pugs > continues ... > To start with, the idea of some kind of "switch" to turn all failures > Second, I would like to move away from the force_todo() usage, as it That also sounds good. If not sure, wait a bit. > has a number of obvious problems. However, I cannot dismiss it's > usefulness as a quick-fix during release preparation, so i will not yet > remove it. > It was determined (on #perl6 and various emails) that what we really This sounds like a really good plan: specify why a test is TODO. > need in Pugs is a multi-layered TODO model. In Pugs we have several > different kinds of TODO tests, along with the plain old TODO. Here is a > list of some of the more common ones and examples of how the attributes > might look; > - TODO So, vanilla. > :todo, :todo(1) > Sometimes, it just doesn't fit anywhere else, so in that case it is > - TODO bug So this is a test which should be working, but isn't. Still, the > :todo<bug>, todo => 'bug', :todo('bug') > This is either an outright bug in Pugs, or it is a feature which definition for this one is a little hazy in my mind. > - TODO feature This is what TODO is all about: this is going to be implemented, but > :todo<feature>, todo => 'feature', :todo('feature') > These are tests written for perl6 features which are not yet isn't yet, so I'm marking it TODO until it is implemented. So maybe the 'bug' one means: this was implemented, but now seems to be > - TODO parsefail So this is like 'feature', except that the parser does not understand > :todo<parsefail>, todo => 'parsefail', :todo('parsefail') > These are tests which fail to parse correctly. These are usually tested the syntax yet. Maybe it could be called 'syntax', which might make more sense. > - TODO hardfail This one is hazy in my mind, too. I'm guessing it means the syntax > :todo<hardfail>, todo => 'hardfail', :todo('hardfail') > These are tests which parse okay, but die with an uncatchable error. I parses, but is still not understood, so the interpretter dies when trying to execute. It's the difference between understanding (the syntax) and comprehending (the meaning). English doesn't really have good words to describe it. Dutch uses 'verstaan' (for words/syntax) and 'begrijpen' (for comprehension/meaning). > My goal was to support these different kinds of TODOs with Test.pm, and Very nice. > also make them visible through Test::Harness (and eventually in our > smoke test). In exploring Test::Harness I noticed that TODO tests can > also have a "reason", just like SKIP tests do (although Test::Harness > seems to ignore them) and I have exploited this feature to accomplish > my goal. That looks really useful. We might just be able to leave the TODO tests as TODO, once we have all the reasons in place. > Now all that is left is for people to start changing all the :todo Can you put the definitions of each attribute in one of the README > tests to :todo<some_attribute>. files? > As always comments, question and suggestions are always welcome. And gladly given. Good work, Stevan. -kolibrie You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
| |||||||||||||
|
Newsgroups: perl.perl6.compiler
From: ste...@iinteractive.com (Stevan Little)
Date: Tue, 3 May 2005 13:37:43 -0400
Local: Tues, May 3 2005 1:37 pm
Subject: Re: Yet Another TODO Model for Test.pm
Nathan,
On May 3, 2005, at 12:32 PM, Nathan Gray wrote: > On Tue, May 03, 2005 at 10:36:40AM -0400, Stevan Little wrote: Yeah it was a cheap hack, so best left undone :) >> The ongoing search to find a good model for TODO tests in Pugs >> continues ... >> To start with, the idea of some kind of "switch" to turn all failures > Good. >> Second, I would like to move away from the force_todo() usage, as it Agreed. Even cheap hacks can be useful :) >> has a number of obvious problems. However, I cannot dismiss it's >> usefulness as a quick-fix during release preparation, so i will not >> yet >> remove it. > That also sounds good. If not sure, wait a bit. This one is a little hazy for me too. Obviously if it is a bug, then we label it as such (most of these are in t/pugsbugs), but sometimes there are edge cases which just have not been fleshed out enough, and this is where this attr comes in. And to be honest, I think it is okay that it is a little hazy, as the >> - TODO feature >> :todo<feature>, todo => 'feature', :todo('feature') >> These are tests written for perl6 features which are not yet > This is what TODO is all about: this is going to be implemented, but > So maybe the 'bug' one means: this was implemented, but now seems to be such) and they are likely to change more as the OO model gets put in too. >> - TODO parsefail >> :todo<parsefail>, todo => 'parsefail', :todo('parsefail') >> These are tests which fail to parse correctly. These are usually > So this is like 'feature', except that the parser does not understand combination/ordering that the parser cannot comprehend. These types of TODOs tend to not stay around for very long either. This is one Autrijus suggested. It is basically this: Pugs can parse the code. There is still a difference between Pugs-catchable errors and >> Now all that is left is for people to start changing all the :todo > Can you put the definitions of each attribute in one of the README using the TODO "reason" attribute is that I am not limiting us to a strict sub-set of TODO attributes. So if you so desire, you can make them up on your own (or write them in Dutch :)). >> As always comments, question and suggestions are always welcome. > And gladly given. Good work, Stevan. And thanks for the comments as well. Stevan You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
| |||||||||||||
|
Newsgroups: perl.perl6.compiler
From: kolib...@graystudios.org (Nathan Gray)
Date: Wed, 4 May 2005 12:00:38 -0400
Local: Wed, May 4 2005 12:00 pm
Subject: Re: Yet Another TODO Model for Test.pm
On Tue, May 03, 2005 at 01:37:43PM -0400, Stevan Little wrote: Ah, flexibility! Very nice. > On May 3, 2005, at 12:32 PM, Nathan Gray wrote: > >Can you put the definitions of each attribute in one of the README > >files? > I will put these common ones in there yes. However, the nice aspect of -kolibrie You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
| |||||||||||||