Recently we've gone over 41% on test coverage of lines of code. There are a bit more than 28,000 lines of code which means that 2800 more covered lines would get us to 50% which would be amazing considering where we were last July. Not that we are so shallow as to focus on mindless statistics (and they are kind of mindless) ... but if 28 people each would cover 100 lines we would be there. There is definitely some low hanging fruit out there in the form of tests that are not so hard to write and classes that have just a few missing pieces especially in some of the older classes. The older classes are where we should be worried about backward compatibility issues as they get updated so it would be really great to get some done.
So that's the challenge, pitch in an help get us to 50%.
Since I've been working on learning how to write better tests I was wondering if there would be any interest in an irc test camp where people could work on tests together one weekend soon.
I guess that one good way to do it do what Mark Dexter do on JBS. He do
webnars and put on Youtube [
https://groups.google.com/forum/#!msg/joomlabugsquad/h3scl1UyXUo/7Wkm... ].
But for this case, since this task depends much more on code, better or
explain line by line of one already good standard test, or even start from
scratch a new now to simple simple test and show a list of other complex
tests that have a good standard to follow. So, ok that can be done like
webnar, but can also be better just do it as a offline tutorial
Even if some IDEs help a bit on how to create draft of tests, can be a good
idea avoid these features if will not show how to install and use on at
least 2 IDEs, like Eclipse and Netbeans. But this is a case of a diferent
tutorial or just a few links of how to install.
And, if someone will do and make available on Youtube, would be really good
with some help of others, and who does not necessarily make the video, made
English subtitles. This helps those who do not understand very well spoken
English, apparently youtube even allows you to translate the subtitles. Not
that this is the main target, but it helps who is deaf, is without earphone
or (do not laugh) who is listening to music.
> Recently we've gone over 41% on test coverage of lines of code. There are
> a bit more than 28,000 lines of code which means that 2800 more covered
> lines would get us to 50% which would be amazing considering where we were
> last July. Not that we are so shallow as to focus on mindless statistics
> (and they are kind of mindless) ... but if 28 people each would cover
> 100 lines we would be there. There is definitely some low hanging fruit out
> there in the form of tests that are not so hard to write and classes that
> have just a few missing pieces especially in some of the older classes. The
> older classes are where we should be worried about backward compatibility
> issues as they get updated so it would be really great to get some done.
> So that's the challenge, pitch in an help get us to 50%.
> Since I've been working on learning how to write better tests I was
> wondering if there would be any interest in an irc test camp where people
> could work on tests together one weekend soon.
Well, I hate IRC, so I'll post here instead. I'm working on patches for countModules - so I switched gears for the moment. Here is the process I am using: gmJoomla\joomla-platform - my github clone of the joomla-platform my branches: -->evalFix: my initial pass on coding a fix for this untested method -->countModulesUnitTests: a new clean branch for just adding the unit tests
I need to rewrite the evalFix patch, but have put that on hold and am working on the unitTests as a seperate patch/pull request. Is this the correct process[submit unit tests seperately, then fixes] - or should they be combined?
I've added tests, so this code now has unit test coverage. These tests are not yet complete because there are many other conditions which need testing - so I changed the "this test has not been implemented" text mark to "this test has not been completed".
Because there are so MANY conditions to test for to provide an adequate level of coverage, I've only included 17 new tests out of approximately 50 tests which are needed.
Some questions I have if anyone knows the answer, otherwise I'll look them up later: There are times when we need to determine if the result of a test was boolean or integer. Ie sometimes it returns false, and sometimes it returns 0. Is the PHPUnittest method isEqualTo sufficient or do I need to use a different method there?
There is one test where there won't be a return value, but instead an exception will be thrown[if your countModules parameter is 'dummy1 / dummy2' and the dummy2 position is empty, then that is a divide by zero exception] - how do you catch exceptions in unitTests for testing?
Lastly, is the correct process so far? And will partial implementations of unit tests be accepted? It's rather tedious and tiresome to write 50+ conditions before knowing if your even doing things correctly, so getting the first 17 in would at least provide motivation and guidance for the remaining 30+. :-)
> Recently we've gone over 41% on test coverage of lines of code. There are > a bit more than 28,000 lines of code which means that 2800 more covered > lines would get us to 50% which would be amazing considering where we were > last July. Not that we are so shallow as to focus on mindless statistics > (and they are kind of mindless) ... but if 28 people each would cover > 100 lines we would be there. There is definitely some low hanging fruit out > there in the form of tests that are not so hard to write and classes that > have just a few missing pieces especially in some of the older classes. The > older classes are where we should be worried about backward compatibility > issues as they get updated so it would be really great to get some done.
> So that's the challenge, pitch in an help get us to 50%.
> Since I've been working on learning how to write better tests I was > wondering if there would be any interest in an irc test camp where people > could work on tests together one weekend soon.
I tend to write a lot of conditions, but that's partly the kind of tests I
work on and that match my skill level (lots of form fields and rules so
dealing with user input). But don't think you need to check every possible
condition, what you need to do is say did the general piece work? Don't
fall into the pit of unit testing php -- you just have to make sure the
joomla parts work. I'd pay more attention to things like what happens if
the property doesn't exist?
I am trying to do TDD in the sense of having the discipline to write the
failing tests for bugs, but I'm not always succeeding, partly because
sometimes that means writing a test from scratch. Tests with out code ---
beautiful. Code without tests not so much.
On Fri, Jun 22, 2012 at 1:56 PM, garyamort <garyam...@gmail.com> wrote:
> Well, I hate IRC, so I'll post here instead. I'm working on patches for
> countModules - so I switched gears for the moment. Here is the process I
> am using:
> gmJoomla\joomla-platform - my github clone of the joomla-platform
> my branches:
> -->evalFix: my initial pass on coding a fix for this untested method
> -->countModulesUnitTests: a new clean branch for just adding the unit tests
> I need to rewrite the evalFix patch, but have put that on hold and am
> working on the unitTests as a seperate patch/pull request. Is this the
> correct process[submit unit tests seperately, then fixes] - or should they
> be combined?
> I've added tests, so this code now has unit test coverage. These tests
> are not yet complete because there are many other conditions which need
> testing - so I changed the "this test has not been implemented" text mark
> to "this test has not been completed".
> Because there are so MANY conditions to test for to provide an adequate
> level of coverage, I've only included 17 new tests out of approximately 50
> tests which are needed.
> Some questions I have if anyone knows the answer, otherwise I'll look them
> up later:
> There are times when we need to determine if the result of a test was
> boolean or integer. Ie sometimes it returns false, and sometimes it
> returns 0. Is the PHPUnittest method isEqualTo sufficient or do I need to
> use a different method there?
> There is one test where there won't be a return value, but instead an
> exception will be thrown[if your countModules parameter is 'dummy1 /
> dummy2' and the dummy2 position is empty, then that is a divide by zero
> exception] - how do you catch exceptions in unitTests for testing?
> Lastly, is the correct process so far? And will partial implementations
> of unit tests be accepted? It's rather tedious and tiresome to write 50+
> conditions before knowing if your even doing things correctly, so getting
> the first 17 in would at least provide motivation and guidance for the
> remaining 30+. :-)
> On Thursday, June 21, 2012 8:18:57 PM UTC-4, Elin Waring wrote:
>> Recently we've gone over 41% on test coverage of lines of code. There
>> are a bit more than 28,000 lines of code which means that 2800 more
>> covered lines would get us to 50% which would be amazing considering where
>> we were last July. Not that we are so shallow as to focus on mindless
>> statistics (and they are kind of mindless) ... but if 28 people each
>> would cover 100 lines we would be there. There is definitely some low
>> hanging fruit out there in the form of tests that are not so hard to write
>> and classes that have just a few missing pieces especially in some of the
>> older classes. The older classes are where we should be worried about
>> backward compatibility issues as they get updated so it would be really
>> great to get some done.
>> So that's the challenge, pitch in an help get us to 50%.
>> Since I've been working on learning how to write better tests I was
>> wondering if there would be any interest in an irc test camp where people
>> could work on tests together one weekend soon.
> Lastly, is the correct process so far? And will partial implementations of unit tests be accepted? It's rather tedious and tiresome to write 50+ conditions before knowing if your even doing things correctly, so getting the first 17 in would at least provide motivation and guidance for the remaining 30+. :-)
Yes, you're following the correct process :) You don't need to write an assertion for every possible situation (it's great if you do thou ;) ) but we're happy to get forward however we can.
Your patch actually revealed a more fundamental problem so it may take a bit longer than usual to get in, but that's not your fault.
On Friday, June 22, 2012 4:22:18 PM UTC-4, Rouven Weßling wrote:
> On 22.06.2012, at 19:56, garyamort wrote:
> Lastly, is the correct process so far? And will partial implementations > of unit tests be accepted? It's rather tedious and tiresome to write 50+ > conditions before knowing if your even doing things correctly, so getting > the first 17 in would at least provide motivation and guidance for the > remaining 30+. :-)
> Yes, you're following the correct process :) You don't need to write an > assertion for every possible situation (it's great if you do thou ;) ) but > we're happy to get forward however we can.
> Your patch actually revealed a more fundamental problem so it may take a > bit longer than usual to get in, but that's not your fault.
In this case I have another pull request, https://github.com/joomla/joomla-platform/pull/1296 to eliminate an eval call altogether but since there are no tests at the moment to check for correct functionality, I need to add the tests and make sure their correct, then I can revisit 1296 to see if it passes.
Almost a chicken/egg situation with almost but not quite new functionality added in 1296 and 1307 - and 1303 containing the tests to ensure the functionality works. [Not sure if it should be called "new functionality" - since it is in line with the current API documentation, it just removes some of the undocumented functionality in countModules. For example: countModules('pos1 ==1;eval(base64_decode('arbitraryphpcommandsbase64encoded');1== pos2') will currently execute the php code and countModules('pos1 && pos2') will work even though it is not documented as an acceptable operator.
On Friday, June 22, 2012 6:16:45 PM UTC-4, garyamort wrote:
> On Friday, June 22, 2012 4:22:18 PM UTC-4, Rouven Weßling wrote:
>> On 22.06.2012, at 19:56, garyamort wrote:
>> Lastly, is the correct process so far? And will partial implementations >> of unit tests be accepted? It's rather tedious and tiresome to write 50+ >> conditions before knowing if your even doing things correctly, so getting >> the first 17 in would at least provide motivation and guidance for the >> remaining 30+. :-)
>> Yes, you're following the correct process :) You don't need to write an >> assertion for every possible situation (it's great if you do thou ;) ) but >> we're happy to get forward however we can.
>> Your patch actually revealed a more fundamental problem so it may take a >> bit longer than usual to get in, but that's not your fault.
> In this case I have another pull request, > https://github.com/joomla/joomla-platform/pull/1296 to eliminate an eval > call altogether but since there are no tests at the moment to check for > correct functionality, I need to add the tests and make sure their correct, > then I can revisit 1296 to see if it passes.
> Almost a chicken/egg situation with almost but not quite new functionality > added in 1296 and 1307 - and 1303 containing the tests to ensure the > functionality works. [Not sure if it should be called "new functionality" > - since it is in line with the current API documentation, it just removes > some of the undocumented functionality in countModules. For example: > countModules('pos1 > ==1;eval(base64_decode('arbitraryphpcommandsbase64encoded');1== pos2') will > currently execute the php code and countModules('pos1 && pos2') will work > even though it is not documented as an acceptable operator.
