In my experience testharness.js is nicer than js-tests because the way we have it set up (thanks to some unnamed hero... Dirk?) you don't need expectation files for success.The downside is that you have less visibility that your tests actually ran. On the whole I like the way we have testharness.js set up. It would not be crazy for someone to write all of their tests using testharness.js.I think we will have >1 test harnesses for some time unless we have a big LayoutTest fixit.
> On the whole I like the way we have testharness.js set up. It would not> beI wonder if it would be crazy to entirely retire js-tests? I am trying
> crazy for someone to write all of their tests using testharness.js.
to understand what we would need for that to happen or why it can't
happen.
> I think it would be a lot of work to convert the tests over, and it's unclear if the reward would be worth it.
For any tests that could be contributed to the general W3C web-platform-tests suite, run by multiple browsers, I think there would be a big upside for the ecosystem, if not for Blink directly. It could be a good "beginner bug" even, or test-the-web-forward project.
I don't have enough knowledge or context to know what fraction of the js-tests that encompasses, however.
On Wed, Jul 2, 2014 at 9:17 AM, Mounir Lamouri <mou...@lamouri.fr> wrote:
On Thu, 3 Jul 2014, at 01:45, Dirk Pranke wrote:Converting the old tests would likely be a time waste. I was mostly
> On Wed, Jul 2, 2014 at 2:59 AM, Mounir Lamouri <mou...@lamouri.fr> wrote:
>
> > > On the whole I like the way we have testharness.js set up. It would not
> > > be
> > > crazy for someone to write all of their tests using testharness.js.
> >
> > I wonder if it would be crazy to entirely retire js-tests? I am trying
> > to understand what we would need for that to happen or why it can't
> > happen.
> >
>
> I think it would be a lot of work to convert the tests over, and it's
> unclear if the reward would be worth it.
wondering if there was a reason why we should allow more of the js-tests
to be added?If you were testing blink-specific things and needed test hooks that js-tests provide and testharness doesn't, it seems like there's not a super-compelling reason to use testharness.
On Wed, 2 Jul 2014, at 15:37, Dominic Cooney wrote:What do you mean?
> The downside is that you have less visibility that your tests actually
> ran.
Is it just a tooling issue? Maybe we could show
results in the terminal like we do for gtest-based suites?
On Wed, Jul 2, 2014 at 6:59 PM, Mounir Lamouri <mou...@lamouri.fr> wrote:
On Wed, 2 Jul 2014, at 15:37, Dominic Cooney wrote:What do you mean?
> The downside is that you have less visibility that your tests actually
> ran.
I mean we have this testharnessreport.js thing and it just prints a one line "success" message. So if you add a test, you're not certain it ran unless you have omnipotent JavaScript powers, console.log, or a failure in there
On Fri, Jul 25, 2014 at 12:50 AM, Takeshi Yoshino <tyos...@chromium.org> wrote:Referring to the Chromium C++ style would be better than Google C++
> Thanks all for input. I've updated the site.
>
> Please take a look and feel free to add new items there directly or propose
> at this thread.
>
> https://sites.google.com/a/chromium.org/dev/blink/coding-style/layout-test-style-guidelines
style IMO (even if they are close).
I also don't think that the new policy gives us much: by trying to
accommodate the status quo (which is that we have at least 4 competing
styles), it has lost any potential value apart from asking people to
be consistent. I am probably missing something but It would be
immensely better to settle on one style for internal tests and one for
external ones. I would support settling on a single style but I could
see that as a tough call in our current situation.
The following sections either don't have anything to do with style or don't match the common practice that I've seen, so I'd remove them:Comparison (=== vs. ==)
Asynchronous tests
Writing HTTP tests
Stable tests
TestExpectation family files
License boilerplate
On Tue, Jul 29, 2014 at 3:04 PM, Ojan Vafai <oj...@chromium.org> wrote:
The following sections either don't have anything to do with style or don't match the common practice that I've seen, so I'd remove them:Comparison (=== vs. ==)Do you mean that this is not about style but just general JS common practice?
License boilerplateWhat's your opinion about this? Can we officially say that license texts are unnecessary in layout tests?
On Mon, Jul 28, 2014 at 11:31 PM, Takeshi Yoshino <tyos...@chromium.org> wrote:
On Tue, Jul 29, 2014 at 3:04 PM, Ojan Vafai <oj...@chromium.org> wrote:
The following sections either don't have anything to do with style or don't match the common practice that I've seen, so I'd remove them:Comparison (=== vs. ==)Do you mean that this is not about style but just general JS common practice?It is about style, but it's not common practice in layout tests. The vast majority use == and !=. The point of the style guide is to avoid people wasting time dealing with inconsistent style nits in code reviews. Has this really come up in code reviews?
License boilerplateWhat's your opinion about this? Can we officially say that license texts are unnecessary in layout tests?I don't really have an opinion about whether this should go in the style guide. In practice we don't put licenses in layout tests. I don't see this coming up in code reviews either though, so I don't know that it needs an explicit rule.
I would cast my vote in favour of js-test.js as I find the testharness.js way of doing asynchronous tests to be more verbose and harder to understand than the simpler method used by js-test.js. In particular, the test.step_func() function wrappers just look like noise to me.
I would cast my vote in favour of js-test.js as I find the testharness.js way of doing asynchronous tests to be more verbose and harder to understand than the simpler method used by js-test.js. In particular, the test.step_func() function wrappers just look like noise to me.While I initially felt that the *-expected.txt files were a gregarious example of over-specified testing, they uncovered quite a few bugs in the new WebSocket implementation, so I am grudgingly in favour of them.Would it be possible to at least get a consensus against ad-hoc tests like http/tests/websocket/frame-lengths.html? It's cool that you can replicate the functionality of a test framework in three lines of Javascript, and I'm sure your way is better, now please stop. It's enough of a maintenance nightmare having two test frameworks without random tests doing their own thing.Also, tests should not duplicate functionality from the test framework. The most common thing I see is<div id="description"></div><div id="console"></div>included in the HTML when js-test.js adds them automatically. It may necessary for layout-sensitive tests to specify where those should go, but I've never encountered a case where it was necessary in a pure JS test.
On 30 July 2014 16:04, Takeshi Yoshino <tyos...@chromium.org> wrote:
On Wed, Jul 30, 2014 at 3:17 AM, Ojan Vafai <oj...@chromium.org> wrote:
On Mon, Jul 28, 2014 at 11:31 PM, Takeshi Yoshino <tyos...@chromium.org> wrote:
On Tue, Jul 29, 2014 at 3:04 PM, Ojan Vafai <oj...@chromium.org> wrote:
The following sections either don't have anything to do with style or don't match the common practice that I've seen, so I'd remove them:Comparison (=== vs. ==)Do you mean that this is not about style but just general JS common practice?It is about style, but it's not common practice in layout tests. The vast majority use == and !=. The point of the style guide is to avoid people wasting time dealing with inconsistent style nits in code reviews. Has this really come up in code reviews?I heard from one person that he's asked to use === in layout test files by some reviewer.
Codesearch under LayoutTests/ dir with extension of .js or .html returns
For / == / 5028For / === / 1929For / != / 3384For / !== / 32372% of equals are ==91% of not equals are !=OK, majority is double one, but count for === is not negligible. I'll rewrite it not to suggest triple ones strongly but say ok to use triple ones when it makes sense and use double ones if it's sufficient. Does it make sense?
should follow the blink style. As we change Blink coding style to
match Chromium's, we can have the tests follow but I wouldn't do the
other way around.
Julien
On Tue, Jul 29, 2014 at 2:37 PM, Mounir Lamouri <mou...@lamouri.fr> wrote:
On Wed, 30 Jul 2014, at 04:17, Ojan Vafai wrote:Wouldn't that be better to make it explicit that the tests are in the
> > License boilerplate
> >>
> >
> > What's your opinion about this? Can we officially say that license texts
> > are unnecessary in layout tests?
> >
>
> I don't really have an opinion about whether this should go in the style
> guide. In practice we don't put licenses in layout tests. I don't see
> this
> coming up in code reviews either though, so I don't know that it needs an
> explicit rule.
Public Domain? WHATWG recommend using Public Domain, CC0 1.0 Public
Domain to be precise [1] [2]. FWIW, Mozilla has been putting Public
Domain headers on its tests [3] [4] in order to ease contributions to
W3C/WHATWG test suites.
[1] http://wiki.whatwg.org/wiki/Testsuite#How_to_license_your_test_suite
[2] http://creativecommons.org/publicdomain/zero/1.0/
[3]
http://mxr.mozilla.org/mozilla-central/source/content/html/content/test/reflect.js
[4]
http://mxr.mozilla.org/mozilla-central/search?string=public+domain&find=test&findi=&filter=%5E%5B%5E%5C0%5D*%24&hitlimit=&tree=mozilla-centralI want to keep tests as minimal and simple as possible. Comments at the top of the file are not a noop. They actually affect what the test does. I'd be OK with someone putting a license at the top of the LayoutTests directory that applies to the whole subtree.
I'm not qualified to say whether that sort of license is actually useful though.In either case, this is a separate discussion. It's clearly not standard practice right now. I'm OK with the style guide explicitly saying that it's not required.
On Wed, Jul 30, 2014 at 2:46 AM, Adam Rice <ri...@chromium.org> wrote:
On 30 July 2014 16:04, Takeshi Yoshino <tyos...@chromium.org> wrote:
On Wed, Jul 30, 2014 at 3:17 AM, Ojan Vafai <oj...@chromium.org> wrote:
On Mon, Jul 28, 2014 at 11:31 PM, Takeshi Yoshino <tyos...@chromium.org> wrote:
On Tue, Jul 29, 2014 at 3:04 PM, Ojan Vafai <oj...@chromium.org> wrote:
The following sections either don't have anything to do with style or don't match the common practice that I've seen, so I'd remove them:Comparison (=== vs. ==)Do you mean that this is not about style but just general JS common practice?It is about style, but it's not common practice in layout tests. The vast majority use == and !=. The point of the style guide is to avoid people wasting time dealing with inconsistent style nits in code reviews. Has this really come up in code reviews?I heard from one person that he's asked to use === in layout test files by some reviewer.That was probably me. I'll try and stop.Codesearch under LayoutTests/ dir with extension of .js or .html returns
For / == / 5028For / === / 1929For / != / 3384For / !== / 32372% of equals are ==91% of not equals are !=OK, majority is double one, but count for === is not negligible. I'll rewrite it not to suggest triple ones strongly but say ok to use triple ones when it makes sense and use double ones if it's sufficient. Does it make sense?[rant]Using == in JS either indicates you don't understand the funky type coercion that may be going on, or you do understand it and are confident that the values you are comparing aren't subject to coercion and want to save a character. Neither are convincing arguments to me for using it; it's sloppy, like relying on ASI. If tests are *code*, they shouldn't be sloppy. Ergo, always use ===.
But consistency should win, so I'll shut up.[/rant]