Browserscope RichText 2 test suite and unit tests

[Roland Steiner]

Hi all,

A quick FYI: as it has been requested a few times, I refactored the Browserscope RichText 2 test suite to allow easier inclusion into unit testing. Now, this is only a "proof-of-concept" and will likely require more convenience functionality to be really useful (e.g., allowing to run single tests rather than whole test groups). In detail:

-) the test Python files contained in categories/richtext2/tests/ now can be re-used as JS files directly (some hack, but works and is fast).

-) instead of using comments in those files I sub-grouped the tests and added in-object annotations/descriptions.

See categories/richtext2/unittestexample.html for an example. Details may change however, esp. with regard to the handling of the <iframe>s.

Other changes in the last push:

-) Simplified the CSS "logic" for the output somewhat

-) a few very minor corrections and additions to the tests. Also, some tests moved around due to the above mentioned grouping.

As always, please let me know about any additional requirements, bug reports, etc.!

Cheers,

- Roland

[Ehsan Akhgari]

This looks very good, thanks!

Is it also possible to get individual results, instead of only aggregate results?

--
Ehsan
<http://ehsanakhgari.org/>

[Lindsey Simon]

You can hit www.browserscope.org/browse?category=richtext2

> --
> You received this message because you are subscribed to the Google Groups
> "Browserscope" group.
> To post to this group, send email to browse...@googlegroups.com.
> To unsubscribe from this group, send email to
> browserscope...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/browserscope?hl=en.
>

[Ehsan Akhgari]

Sure, but I meant from within a unit test framework.  Specifically, looking at unittestexample.html, it can report aggregate results, but not individual results.

--
Ehsan
<http://ehsanakhgari.org/>

[Roland Steiner]

At the moment you cannot run individual tests, only whole suites (e.g., 'Apply Tests'). However, the complete results for every individual test should be reported in the 'results' object, along with aggregate results:

{

  // aggregate over all tests

  count: <int>

  selscore: <int>

  valscore: <int>

  // Suites by ID (e.g., 'A' for 'Apply Tests')

  A: {

    count: <int>

    selscore: <int>

    valscore: <int>

    time: <time>

    // Types of tests (Proposed, RFC or Finalized):

    Proposed: {

      // aggregate results for all tests of this type

      count: <int>

      selscore: <int>

      valscore: <int>

      // Individual test results

      UC_U_U-1_SL: {

        selscore: <0 or 1>

        valscore: <0 or 1>

        selresult: <int, see SELRESULT_xxx in variables.js>

        valresult: <int, see VALRESULT_xxx in variables.js>

        // detailed results per test container ('dM', 'body', 'div'):

        dM: {

          selscore: <0 or 1>

          valscore: <0 or 1>

          selresult: <int, see SELRESULT_xxx in variables.js>

          valresult: <int, see VALRESULT_xxx in variables.js>

          output: <string/bool>

          // for HTML tests:

          innerHTML: <string>

          outerHTML: <string>

          bodyInnerHTML: <string>

          bodyOuterHTML: <string>

        }

      }

  }

}

Cheers,

- Roland

[Ehsan Akhgari]

I just wanted to drop a quick note saying that today I finally
integrated the Richtext2 suite into Mozilla's automated testing
system:

https://bugzilla.mozilla.org/show_bug.cgi?id=642704

This is an amazing project, and the test coverage is very valuable for
us. I'd like to thank everybody who worked on it, and I look forward
to future awesomeness in this area! :-)

Cheers,
--
Ehsan
<http://ehsanakhgari.org/>

[Roland Steiner]

Very cool! :D

Please let me know if I can do anything to make your life easier.
Also, if Mozilla's testing finds issues with the test suite or any of
its tests, please do complain! ;)

Cheers,

- Roland

[Ehsan Akhgari]

Hi Roland,

Thanks! Integrating the suite was quite easy, thanks for the
unittestexample.html file, and also to the efforts that you'd made in
order to report granular test results for both value and selection.

I'll get back to you in case I find any issues, and I'm also planning
to update the suite from upstream from time to time to pick up the
work done on the browserscope side.

Cheers,
--
Ehsan
<http://ehsanakhgari.org/>