Making reftest expectation conditional on a pref

[Henri Sivonen]

I'm now working with the assumption that the HTML5 parser and the old parser will be run side-by-side on tinderboxen for a while. This is a problem with reftests that only pass with the new parser. (Fortunately, there's only one such test in the tree currently It' now an expected fail and I had previously prepared to make it an expected pass when turning the HTML5 parser on by default.)

Should such tests be marked to be "random" (even though they consistently pass or fail depending on the HTML parser)? Or is there a way to make the reftest expectation depend on the value of the html5.enable pref?

--
Henri Sivonen
hsiv...@iki.fi
http://hsivonen.iki.fi/

[Ehsan Akhgari]

On Mon, Apr 19, 2010 at 8:16 AM, Henri Sivonen <hsiv...@iki.fi> wrote:
> I'm now working with the assumption that the HTML5 parser and the old parser will be run side-by-side on tinderboxen for a while. This is a problem with reftests that only pass with the new parser. (Fortunately, there's only one such test in the tree currently It' now an expected fail and I had previously prepared to make it an expected pass when turning the HTML5 parser on by default.)
>
> Should such tests be marked to be "random" (even though they consistently pass or fail depending on the HTML parser)? Or is there a way to make the reftest expectation depend on the value of the html5.enable pref?

I think this is what you want:

fails-if(!prefs.getBoolPref("html5.enable")) test.html ref.html

--
Ehsan
<http://ehsanakhgari.org/>

[Zack Weinberg]

Ehsan Akhgari <ehsan....@gmail.com> wrote:

If there's going to be a lot of these, it would be good to
create shorthand, so it can just be

fails-if(!html5) test.html ref.html

You can do that by adding another boolean variable to the sandbox
created in ReadManifest() in layout/tools/reftest/reftest.js.

zw

[crowder]

> fails-if(!prefs.getBoolPref("html5.enable")) test.html ref.html

I only implemented getIntPref() in my original landing. If
getBoolPref doesn't exist yet, you can add it similarly.

[Ehsan Akhgari]

I've added it on trunk and 1.9.2 as part of bug 555289.

--
Ehsan
<http://ehsanakhgari.org/>

[Henri Sivonen]

"Zack Weinberg" <zwei...@mozilla.com> wrote:

> Ehsan Akhgari <ehsan....@gmail.com> wrote:
>
> > On Mon, Apr 19, 2010 at 8:16 AM, Henri Sivonen <hsiv...@iki.fi>
> > wrote:
> > > I'm now working with the assumption that the HTML5 parser and the
> > > old parser will be run side-by-side on tinderboxen for a while.
> > > This is a problem with reftests that only pass with the new
> parser.
> > > (Fortunately, there's only one such test in the tree currently
> It'
> > > now an expected fail and I had previously prepared to make it an
> > > expected pass when turning the HTML5 parser on by default.)
> > >
> > > Should such tests be marked to be "random" (even though they
> > > consistently pass or fail depending on the HTML parser)? Or is
> > > there a way to make the reftest expectation depend on the value
> of
> > > the html5.enable pref?
> >
> > I think this is what you want:
> >

> > fails-if(!prefs.getBoolPref("html5.enable")) test.html ref.html

Thanks.
fails-if(!prefs.getBoolPref("html5.enable")) == 311366-unknown-block-3.html 311366-unknown-block-3-ref.html
works.

> If there's going to be a lot of these, it would be good to
> create shorthand, so it can just be
>
> fails-if(!html5) test.html ref.html

I think it would be an overkill to create a shorthand now when this is only about one reftest. If we end up running the two parser side-by-side for a long time on the tinderboxen, a larger number of tests that fail with the old parser could crop up, but let's address that if it really becomes an annoyance to use fails-if(!prefs.getBoolPref("html5.enable")).

[Zack Weinberg]

Henri Sivonen <hsiv...@iki.fi> wrote:

> "Zack Weinberg" <zwei...@mozilla.com> wrote:
>
> > If there's going to be a lot of these, it would be good to
> > create shorthand, so it can just be
> >
> > fails-if(!html5) test.html ref.html
>
> I think it would be an overkill to create a shorthand now when this
> is only about one reftest. If we end up running the two parser
> side-by-side for a long time on the tinderboxen, a larger number of
> tests that fail with the old parser could crop up, but let's address
> that if it really becomes an annoyance to use
> fails-if(!prefs.getBoolPref("html5.enable")).

Fine with me.

zw