writing test cases faster?

[Hallvord Reiar Michaelsen Steen]

Hi all,
maybe just because I'm new to it.. but it seems to be very time consuming
to write tests for devtools. I make a tweak, run "mach mochitest
path/to/some/test.js", wait - sometimes for a long time if I'm making
mistakes that cause timeouts. Rinse and repeat. Do you have any tricks for
speeding this up? Is there for example a way to load a "scratchpad" running
in the right context with head.js injected to be able to try out commands
immediately?
-Hallvord

[James Long]

That's a cool idea. I don't think we have it setup to do that right now,
but I'm definitely interested in making it easier to right tests. One
problem is that if you want to run a test through, it basically needs to
start a new "instance" of devtools each time. I'm not sure what we can
improve other than maybe just making mochitests faster to start up.

But maybe we can make something like scratchpad that's evaluated in the
testing environment, at least just to test specific helper functions and
such.

> _______________________________________________
> dev-developer-tools mailing list
> dev-devel...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-developer-tools
>

[Brian Grinstead]

A couple tips:

* I wouldn’t say this speeds up startup (quite the opposite), but you can run `./mach mochitest path/to/some/test.js —jsdebugger` and throw a debugger statement at the start of the test case. Once it pauses you will have access to all of the head.js functionality so you can run code and try things out.
* I often do a `yield new Promise(() => {});` inside of a task to get execution to stop and let me investigate the state of the UI. You can from there open up a Browser Console / Browser Toolbox and take a look at errors in console / run code (global scope) if you need to dig in further. Then once I figure out what I needed, I ctrl+c on the terminal that started the test and repeat. Or `yield new Promise(resolve => {setTimeout(resolve, 1000)});` if you want things to slow down a bit and watch what is happening.
* Or as I just discovered looking at the help for the command, there is a --debug-on-failure option that is intended to pause the debugger when a test fails (haven’t tried that yet).

Brian

> On Oct 4, 2015, at 12:55 PM, Hallvord Reiar Michaelsen Steen <hst...@mozilla.com> wrote:
>
> Hi all,
> maybe just because I'm new to it.. but it seems to be very time consuming
> to write tests for devtools. I make a tweak, run "mach mochitest
> path/to/some/test.js", wait - sometimes for a long time if I'm making
> mistakes that cause timeouts. Rinse and repeat. Do you have any tricks for
> speeding this up? Is there for example a way to load a "scratchpad" running
> in the right context with head.js injected to be able to try out commands
> immediately?
> -Hallvord
> _______________________________________________
> dev-developer-tools mailing list
> dev-devel...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-developer-tools

[J. Ryan Stinnett]

We should record "test tips" somewhere on MDN / wiki.

- Ryan

[Hallvord Reiar Michaelsen Steen]

On Mon, Oct 5, 2015 at 6:19 PM, Brian Grinstead <bgrin...@mozilla.com>

wrote:

> A couple tips:
>
> * I wouldn’t say this speeds up startup (quite the opposite), but you can
> run `./mach mochitest path/to/some/test.js —jsdebugger` and throw a
> debugger statement at the start of the test case. Once it pauses you will
> have access to all of the head.js functionality so you can run code and try
> things out.
>

This excellent tips unfortunately doesn't work with the talos-test command
:-/

"It looks like you passed an unrecognized argument into mach.

The talos-test command does not accept the arguments: --jsdebugger"

Can we add it?
-Hallvord

[J. Ryan Stinnett]

On Tue, Jan 5, 2016 at 4:56 AM, Hallvord Reiar Michaelsen Steen
<hst...@mozilla.com> wrote:
> This excellent tips unfortunately doesn't work with the talos-test command
> :-/
>
> "It looks like you passed an unrecognized argument into mach.
>
> The talos-test command does not accept the arguments: --jsdebugger"
>
> Can we add it?

I don't know very much about Talos specifically. Someone would need to
extend the Talos harness to support this. As an example, here's how
the XPCShell harness handles it:

https://dxr.mozilla.org/mozilla-central/rev/29258f59e5456a1a518ccce6b473b50c1173477e/testing/xpcshell/head.js#447-482

- Ryan

[Hallvord Reiar Michaelsen Steen]

On Wed, Jan 6, 2016 at 12:53 AM, J. Ryan Stinnett <jry...@gmail.com> wrote:

> > The talos-test command does not accept the arguments: --jsdebugger"
>

> I don't know very much about Talos specifically. Someone would need to
> extend the Talos harness to support this. As an example, here's how
> the XPCShell harness handles it:
>
>
> https://dxr.mozilla.org/mozilla-central/rev/29258f59e5456a1a518ccce6b473b50c1173477e/testing/xpcshell/head.js#447-482

Hm.. looks a bit complicated. I don't know Talos either. I just reported a
bug - somebody might pick it up some day:
https://bugzilla.mozilla.org/show_bug.cgi?id=1237215