testing our polymer elements

[Jelte Liebrand]

Hi,

We've been adding a bunch of polymer elements to our code (kudos btw, polymer rocks!), and have been following the same pattern for testing as the polymer elements themselves.

One problem with this approach is that since the tests run inside an iframe (when run  from the commandline), it is hard to debug any errors. We've extended our local copy of htmlTest.js and mocha-htmlTest.js a bit to improve this by

a- allowing the test inside the iframe to use mocha (with a special reporter that postMessages results to the parent frame)

b- serialising Error objects such that callstacks are preserved when sent to the parent frame

c- posting console.log messages to the parent frame

But even with these changes, it's not always straight forward to spot what is going wrong. On top of that, our "hacks" are somewhat bolted on to the side, rather than something that is very extensible.

Do you (or perhaps the karma team) have any plans for improvements here? Or should we continue to tweak the existing parent vs iframe communication a bit more to help with developing/debugging elements?

/Cheers,

Jelte

Jelte Liebrand | Software Engineer | je...@google.com | +1 (415) 318 6507

[Rob Dodson]

Hey Jelte,

I'm feeling your pain. I just wrote a ton of tests for a11y and it is a pain having to generate new html files just so I can know which test was failing. I spoke with the team and they said it's something we need to improve but I don't think there are any immediate concrete plans in place. 

Follow Polymer on Google+: plus.google.com/107187849809354688692
---
You received this message because you are subscribed to the Google Groups "Polymer" group.
To unsubscribe from this group and stop receiving emails from it, send an email to polymer-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/polymer-dev/CAOOg8mr-KDB0SoMu2XJZxRKaZTiEaSmuPnDKfV6rpDngZYPD0g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Steve Orvell]

Hey Jelte, 

I just wanted to make sure I got back to you on this, I thought I did verbally but in case I didn't... This feedback was very helpful. We're currently in the midst of a host of testing improvements that should make it more pleasant to both write and run tests. We're hoping to have some progress around the time of the next polymer point release.

To view this discussion on the web visit https://groups.google.com/d/msgid/polymer-dev/CAJj5OwDeq3kdBDW5HS6iFk%3DM6GqHbRWQMrDP6xHTUdJdXk6KQw%40mail.gmail.com.

[Jelte Liebrand]

That sounds very promising; I'm very curious so I'll keep my eyes open. Will it still be based on the iframe solution and thus run each test individually (with all load overhead), or will it be more karma-like in that all code/tests is loaded once, and then all tests are run. The latter requiring your code to be a good citizen and not leave global state around, but giving much faster tests?

Jelte Liebrand | Software Engineer | je...@google.com | +1 (415) 318 6507

[Scott Miles]

>> not leave global state around

Currently we can't control the custom element registry and therefore cannot clean up the state of registrations (i.e. you can only register one x-foo ever). This is the primary reason we are using the iframes, and we are stuck with this for the time being.

The work underway right now is primarily aimed at making it (dramatically) easier to run cross-device tests and to allow us to do a lot of automated (cloud) testing on commits or pull-requests. I suspect neither of these features are exactly what you are looking for, but I know that (1) your valuable feedback has been involved in the discussions (2) there are lots of general improvements and (3) this is just the start of evolving a robust testing story.

There should be more information coming next week.

Scott

To view this discussion on the web visit https://groups.google.com/d/msgid/polymer-dev/CAOOg8mpr_dOBg%3DkBxyo4%3D75%3DynjVtFQwDWMAOi7xFQQz_N1A1Q%40mail.gmail.com.

[Jelte Liebrand]

Awesome! Thanks for the update!

Jelte Liebrand | Software Engineer | je...@google.com | +1 (415) 318 6507

[jamesnew...@gmail.com]

I'm in the middle of this right now too - trying to get mocha to work with saucelabs - there seems to be some libraries out there that I can connect up with the grunt file but having issues - any advice on how to get all that working you can provide would be appreciated!

Thanks!

- Devin

[Ian MacLeod]

Take a look at https://github.com/Polymer/web-component-tester (which we use to run mocha on Sauce Labs), I think it'll cover your needs!

To view this discussion on the web visit https://groups.google.com/d/msgid/polymer-dev/eef950da-3d63-4f64-ab05-799e3dedfff9%40googlegroups.com.