Is Richtext2 being used by browser vendors as a regression detection test suite?

[Ehsan Akhgari]

As you know, I've integrated richtext2 (in addition to the older
richtext suite) into Mozilla's unit test framework which gets run
after every check-in, to ensure that changes to our code won't regress
any of the editing operations covered by these tests. I'm also
looking into the possibility of using richtext2 as a preliminary
editing performance test suite too.

I'm interested to know if you're aware of WebKit (or browsers based on
it), Opera and IE also using richtext2 for these purposes?

Cheers,
--
Ehsan
<http://ehsanakhgari.org/>

[Roland Steiner]

[+Ryosuke, WebKit editing specialist]

- Roland

--
You received this message because you are subscribed to the Google Groups "Browserscope" group.
To post to this group, send email to browse...@googlegroups.com.
To unsubscribe from this group, send email to browserscope...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/browserscope?hl=en.

[Roland Steiner]

Ehsan is already doing this for Mozilla, so it hopefully should be straightforward to do it in a similar way for WebKit.

As for performance testing (Ehsan's original question of the thread): Do you have suggestions for a setup to do this?

- Roland

On Tue, Jul 19, 2011 at 11:06 AM, Ryosuke Niwa <rn...@webkit.org> wrote:

That'll certainly be an interesting idea.

Roland, do you know if RTE2 test suite can easily be converted to layout tests?

- Ryosuke

[Ehsan Akhgari]

On Mon, Jul 18, 2011 at 10:22 PM, Roland Steiner <roland...@google.com> wrote:

Ehsan is already doing this for Mozilla, so it hopefully should be straightforward to do it in a similar way for WebKit.

I have integrated both richtext and richtext2 into the Mozilla testing framework for some time now.  Let me show you how I did it.  The main richtext2 test driver is this file: <http://mxr.mozilla.org/mozilla-central/source/editor/libeditor/html/tests/browserscope/test_richtext2.html?force=1>.  As some of those tests currently fail, we want a framework to make sure that the passing tests do not regress, and we also want to know if a failing test starts to pass (for example, when a bug is fixed).  To make that work, we need to record the current status of the test results with a given browser build.  In order to bootstrap this process, I set the UPDATE_TEST_RESULTS variable in the test driver to true, and run the tests once.  This will create a huge json blob which records the current results of the test.  Then I use that json blob to create <http://mxr.mozilla.org/mozilla-central/source/editor/libeditor/html/tests/browserscope/lib/richtext2/currentStatus.js> which records the current status of the test results, and then I set UPDATE_TEST_RESULTS back to false.  The test driver will use currentStatus.js to measure the differences between the results on another build and what's been recorded in currentStatus.js, and generates the correct regression/improvement notification using the Mozilla framework if needed.

Let me know if you need more info.
 

As for performance testing (Ehsan's original question of the thread): Do you have suggestions for a setup to do this?

The Mozilla Automation team has started a project for automated speed tests across browsers <http://cloquewerk.livejournal.com/22400.html>.  For now I've just asked them to measure the amount of time it takes to complete the whole richtext2 suite across different browsers (the project has not been finished yet, so there are no results to see right now).  Once we have those results, we can choose whether they make sense in that form at all, and what other types of tests would be useful.  But I'll definitely would like to hear other ideas too!