Please Test Test.Simple

[David E. Wheeler]

Howdy,

I'm prepping a new version of Test.Simple and related modules, and
would greatly appreciate it if everyone would check it out from SVN
and run the tests in their favorite browsers. You can do a quick
online run here:

http://justatheory.com/code/Test.Simple/tests/index.html

The repo is here:

svn://svn.openjsan.org/openjsan/users/theory/Test.Simple/trunk

I've run the tests in:

Camino 1.5.4
Firefox 2.0.0.11
Safari 3.1

And all tests pass in those browsers. How about for you? IE users
especially wanted.

Thanks,

David

[Dan Brook]

David E. Wheeler wrote:
> I've run the tests in:
>
> Camino 1.5.4
> Firefox 2.0.0.11
> Safari 3.1
>
> And all tests pass in those browsers. How about for you? IE users
> especially wanted.
>

Seems to get caught in a timeout loop in IE6 (SP2 on XP) in the
builder_death.js test within
the Test.Harness.Browser.prototype.runTests method (the scoped wait
function specifically).

Cheers,
Dan

[David E. Wheeler]

On Mar 31, 2008, at 21:29, Dan Brook wrote:

> Seems to get caught in a timeout loop in IE6 (SP2 on XP) in the
> builder_death.js test within
> the Test.Harness.Browser.prototype.runTests method (the scoped wait
> function specifically).

Marvin, can you have a look at this? I changed that test to call
_finish() instead of _ending() to eliminate an error in Firefox, but
maybe that's not the right thing to do for IE?

Thanks,

David

[Marvin Humphrey]

On Apr 1, 2008, at 9:47 AM, David E. Wheeler wrote:
>> Seems to get caught in a timeout loop in IE6 (SP2 on XP) in the
>> builder_death.js test within
>> the Test.Harness.Browser.prototype.runTests method (the scoped wait
>> function specifically).
>
> Marvin, can you have a look at this? I changed that test to call
> _finish() instead of _ending() to eliminate an error in Firefox, but
> maybe that's not the right thing to do for IE?

I can confirm that on IE6 Win2k, calling _ending() completes the test
cleanly while calling _finish() hangs.

What sort of a problem have you run into with FireFox? With the
original version (using _ending()), the builder_death.js test passes
on...

Win2k SP4: IE 6.0.2800.1106 FF 2.0.0.7
Vista: IE 7.0.6000.16609 FF 2.0.0.11
OS X 10.5: Safari 3.1 (5525.13) FF 2.0.0.13

Background on builder_death.js here: http://xrl.us/biqyj (Link to
groups.google.com)

Rectangular Research
http://www.rectangular.com/

[David E. Wheeler]

On Apr 1, 2008, at 13:23, Marvin Humphrey wrote:

> I can confirm that on IE6 Win2k, calling _ending() completes the test
> cleanly while calling _finish() hangs.

Cute.

> What sort of a problem have you run into with FireFox?

I get:

Error: Test is not defined

All tests pass, so I'm not sure what's up with that. Hrm…looking more
closely, if I skip the builder_death test, I don't get the error. But
if I put it in, the error happens after curr_test runs, not at the end
of builder_death. There must be some side-effect.

> With the
> original version (using _ending()), the builder_death.js test passes
> on...
>
> Win2k SP4: IE 6.0.2800.1106 FF 2.0.0.7
> Vista: IE 7.0.6000.16609 FF 2.0.0.11
> OS X 10.5: Safari 3.1 (5525.13) FF 2.0.0.13

Great.

Thanks,

David

[David E. Wheeler]

On Apr 1, 2008, at 14:38, David E. Wheeler wrote:

>> With the
>> original version (using _ending()), the builder_death.js test passes
>> on...
>>
>> Win2k SP4: IE 6.0.2800.1106 FF 2.0.0.7

Are you certain that the call to _ending() is necessary? See this bit
in Test.Harness.Browser:

// IE 6 SP 2 doesn't seem to run the onload() event, so we force
the
// issue.
Test.Builder._finish(Test);

So the harness should finish the job, no?

Thanks,

David

[Marvin Humphrey]

On Apr 1, 2008, at 2:46 PM, David E. Wheeler wrote:

> Are you certain that the call to _ending() is necessary? See this bit
> in Test.Harness.Browser:
>
> // IE 6 SP 2 doesn't seem to run the onload() event, so we force
> the
> // issue.
> Test.Builder._finish(Test);
>
> So the harness should finish the job, no?

Yes, leaving it out works -- in all 6 browsers, and with no errors
reported in the FF console.

Actually, I note that the last version of the patch I submitted,
archived at <http://xrl.us/biq9g>, didn't include the force. Forcing
was only needed in the earlier versions where the the test was
packaged as an html file, builder_death.html, rather than as
builder_death.js.

In other news, async.js dies after the first test in Vista IE7.

Unfortunately it's hard to see that it's failing when you're running
the suite via the Harness, because the Harness doesn't notice that the
test died and that not enough tests ran. Harness still reports "All
tests successful" and just indicates a lower number than it should.
The false negative arises because of the long-standing, hard to fix
bug discussed at <http://xrl.us/birbc>.

Marvin Humphrey
Rectangular Research
http://www.rectangular.com/

[David E. Wheeler]

On Apr 1, 2008, at 16:00, Marvin Humphrey wrote:

>> // IE 6 SP 2 doesn't seem to run the onload() event, so we force
>> the
>> // issue.
>> Test.Builder._finish(Test);
>>
>> So the harness should finish the job, no?
>
> Yes, leaving it out works -- in all 6 browsers, and with no errors
> reported in the FF console.

Oh, good. Removing it, then. Thanks!

> Actually, I note that the last version of the patch I submitted,
> archived at <http://xrl.us/biq9g>, didn't include the force. Forcing
> was only needed in the earlier versions where the the test was
> packaged as an html file, builder_death.html, rather than as
> builder_death.js.

Yeah, builder_death.html is where I copied the code from, somewhere in
the archives, since it wasn't in the new svn. Good spot, thanks.

> In other news, async.js dies after the first test in Vista IE7.

Bleh.

> Unfortunately it's hard to see that it's failing when you're running
> the suite via the Harness, because the Harness doesn't notice that the
> test died and that not enough tests ran. Harness still reports "All
> tests successful" and just indicates a lower number than it should.
> The false negative arises because of the long-standing, hard to fix
> bug discussed at <http://xrl.us/birbc>.

Yes, I thought TestResult was supposed to fix that. But I guess it was
mainly just a refactoring, eh?

So we nee some way to get that data to the harness from Test.Builder.

Thanks,

David

[Marvin Humphrey]

BTW, this commit message isn't accurate:

------------------------------------------------------------------------
r9 | theory | 2008-03-31 15:35:42 -0700 (Mon, 31 Mar 2008) | 3 lines

* Restored missing files that Marvin forgot to commit to the old
repository.
* Set `svn:keywords` on the class files.

------------------------------------------------------------------------

According to my local copy, those files *were* committed:

/Volumes/Bruin/Users/marvin/projects/ $ cp -R Test.Simple/
Test.Simple.old
/Volumes/Bruin/Users/marvin/projects/ $ cd Test.Simple.old/
/Volumes/Bruin/Users/marvin/projects/Test.Simple.old/ $ svn stat
M tests/instance.js
M tests/harness.js
M doc/pod/Test/Harness/Browser.pod
/Volumes/Bruin/Users/marvin/projects/Test.Simple.old/ $ svn revert -R .
Reverted 'tests/instance.js'
Reverted 'tests/harness.js'
Reverted 'doc/pod/Test/Harness/Browser.pod'
/Volumes/Bruin/Users/marvin/projects/Test.Simple.old/ $ rm tests/
builder_death.js
/Volumes/Bruin/Users/marvin/projects/Test.Simple.old/ $ svn stat
! tests/builder_death.js
/Volumes/Bruin/Users/marvin/projects/Test.Simple.old/ $

Something got f***ed up somewhere else -- my guess is that whoever re-
imported the svn repository did so from an out-of-date local copy.
Whoever that is should check the revision number of their copy and see
what else may have gotten lost.

PS: Google Groups sux because it will have auto-rewrapped many of the
lines above.

[David E. Wheeler]

On Apr 1, 2008, at 16:36, Marvin Humphrey wrote:

> Something got f***ed up somewhere else -- my guess is that whoever re-
> imported the svn repository did so from an out-of-date local copy.

Yep. Did you commit those files in a commit separate from when you
modified the other files that referred to them? For example, all the
code had been updated to use TestResult, it's just that the file
wasn't there.

Thanks,

David

[Marvin Humphrey]

On Apr 1, 2008, at 4:28 PM, David E. Wheeler wrote:

>> Unfortunately it's hard to see that it's failing when you're running
>> the suite via the Harness, because the Harness doesn't notice that
>> the
>> test died and that not enough tests ran. Harness still reports "All
>> tests successful" and just indicates a lower number than it should.
>> The false negative arises because of the long-standing, hard to fix
>> bug discussed at <http://xrl.us/birbc>.
>
> Yes, I thought TestResult was supposed to fix that. But I guess it was
> mainly just a refactoring, eh?

It was a start. The idea was to beef up the OO discipline level a
bit, and break a chunk out of the gnarly center. It's pretty hard to
follow what's going on in there -- and the original Perl Test::Simple
is harder still. It's going to take more refactoring yet to get to
the point where internal test result tracking makes sense to an
ordinary human brain.

I think you should release anyway. async.js fails in IE7 Vista in the
current release (0.21), so you're not losing anything AFICT.

[Marvin Humphrey]

It's hard to know. Since the old repository is gone, I can't perform
an "svn log" or an "svn diff" and AFAIK the commits weren't being
archived anywhere at that point. From spelunking the .svn/ files, it
looks like TestResult.js got added during 518.

I performed a "diff -ru" from my old local copy against the new repos
version (attached). Allowing for the other changes since mine, it
looks like we're square.

[David E. Wheeler]

On Apr 1, 2008, at 17:15, Marvin Humphrey wrote:

> I think you should release anyway. async.js fails in IE7 Vista in the
> current release (0.21), so you're not losing anything AFICT.

Agreed.

Best,

David

[David E. Wheeler]

On Apr 1, 2008, at 17:58, Marvin Humphrey wrote:

> I performed a "diff -ru" from my old local copy against the new repos
> version (attached). Allowing for the other changes since mine, it
> looks like we're square.

Huh. I don't see TestResult.js or builder_death.js in that diff. I'd
think that TestResult.js would be showing up as deleted, and
builder_death.js surely must have *some* differences…

But I agree that things look good.

Best,

David

[Marvin Humphrey]

On Apr 1, 2008, at 8:17 PM, David E. Wheeler wrote:
>
> On Apr 1, 2008, at 17:58, Marvin Humphrey wrote:
>
>> I performed a "diff -ru" from my old local copy against the new repos
>> version (attached). Allowing for the other changes since mine, it
>> looks like we're square.
>
> Huh. I don't see TestResult.js or builder_death.js in that diff. I'd
>> think that TestResult.js would be showing up as deleted,

TestResult.js was inside lib/Test/Builder/, and that directory's been
scotched:

Only in Test.Simple.old/lib/Test: Builder

> and builder_death.js surely must have *some* differences…

builder_death.js had a difference: one newline:

diff -ru Test.Simple.old/tests/builder_death.js Test.Simple.new/tests/
builder_death.js
--- Test.Simple.old/tests/builder_death.js 2008-04-01
17:56:28.000000000 -0700
+++ Test.Simple.new/tests/builder_death.js 2008-04-01
17:33:50.000000000 -0700
@@ -19,4 +19,3 @@
catch (ex) {
T.ok((ex.message === "baz"), "expected 'baz', got '" +
ex.message + "'");
}
-

> But I agree that things look good.

rokn.

[David E. Wheeler]

On Apr 1, 2008, at 21:39, Marvin Humphrey wrote:

>> Huh. I don't see TestResult.js or builder_death.js in that diff. I'd

>> think that TestResult.js would be showing up as deleted,
>
> TestResult.js was inside lib/Test/Builder/, and that directory's been
> scotched:

Oh. Right. Duh.

> builder_death.js had a difference: one newline:

Okay, good.

>> But I agree that things look good.
>
> rokn.

Thanks,

David