Proposal: Write unit tests for JavaScript

Trey Hunner

unread,

Apr 16, 2014, 5:57:07 PM4/16/14

to django-d...@googlegroups.com

I saw a previous discussion about JavaScript testing in Django but it looks like there hasn't been any progress in a few years.

Some of my thoughts on this issue:

This would require choosing a JavaScript testing framework. There are many good ones out there. A popular one should probably used for easier community support.

Unit testing JavaScript (ideally) should not require running the Django server.

JavaScript tests will probably require introducing Node.js into the automated testing process. Tests can be run manually from the browser, but automated JavaScript tests tend to require Node.js and sometimes PhantomJS (for headless testing).

The JavaScript tests should be run as part of the CI testing process. If the tests are run standalone this should be easy to do using a single command (possibly requiring grunt or a similar task runner).

This seems like it would be a big change, but I think it could be done in small steps. Setting up the testing framework is the first big step.

What do others think about this issue?

--
Trey Hunner

Russell Keith-Magee

unread,

Apr 16, 2014, 9:20:12 PM4/16/14

to Django Developers

I have two thoughts:

1) More testing doubleplus good. :-)

2) Is there anything that can save us from the Node.js kudzu? :-)

As with your previous question about linting, I don't have any firm opinions about this, beyond "Yes, we should do it".

My suggestion here would be to proceed as if we all agree that this is a good idea (more testing good, etc), and make a concrete proposal. It's a lot easier to discuss a concrete proposal than to just kick around buzzwords and hope some of them stick :-).

You've definitely identified that this is a long term project; so if you can lay out a map for the way forward, with an indication of the end goal, that would be a fantastic start IMHO.

Yours,

Russ Magee %-)

Matthew Lauber

unread,

Apr 17, 2014, 8:36:52 AM4/17/14

to django-d...@googlegroups.com

Regarding running javascript tests, could we consider something like rhino? Just an off the top of my head suggestion. Would avoid node.js, not sure if it's an improvement though.

Matt Lauber

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAJxq84_2u%3DtUpEKwOQ0fPDFcOB80w0K%3Dh%2By_Kg6DM7T8LkSPOg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

Carl Meyer

unread,

Apr 17, 2014, 1:30:17 PM4/17/14

to django-d...@googlegroups.com

I agree that this would be a good idea - thanks for working on it!

A DEP might be a good format to summarize the thinking that goes into
picking a particular tech stack for JS tests.

(FWIW, on my company's projects we unit-test JS using Node, PhantomJS,
Grunt, QUnit, and Istanbul for test coverage measurement, so that's the
stack I'm familiar with. It's worked very well for us; there's a
grunt-qunit-istanbul plugin that brings the pieces together nicely. But
I didn't make those choices and am not familiar with the alternatives;
there may be better options.)

Carl

Trey Hunner

unread,

Apr 18, 2014, 5:09:44 PM4/18/14

to django-d...@googlegroups.com

> On 04/16/2014 07:20 PM, Russell Keith-Magee wrote:
>
> 2) Is there anything that can save us from the Node.js kudzu? :-)

Yes. Removing JavaScript from Django. :-)

JSHint requires Node.js and running automated JavaScript tests
typically requires Node.js and PhantomJS.

Node.js is pretty easy to install and it would only be required for
JavaScript linting and automated testing. You wouldn't necessarily
need Node.js to manually run the tests locally in a browser, but you
would need it for running tests from the command line.

> You've definitely identified that this is a long term project; so if you
> can lay out a map for the way forward, with an indication of the end
> goal, that would be a fantastic start IMHO.

Agreed. I will try formulating a more concrete proposal. I work
full-stack but I have limited experience with JavaScript testing
frameworks, so my personal preferences include a sample size of one.
I will try to keep my suggestions to the community standards as I
perceive them.

On Thu, Apr 17, 2014 at 10:30 AM, Carl Meyer <ca...@oddbird.net> wrote:
>
> A DEP might be a good format to summarize the thinking that goes into
> picking a particular tech stack for JS tests.

I will look at the DEPs and try to propose a concrete example there.

> (FWIW, on my company's projects we unit-test JS using Node, PhantomJS,
> Grunt, QUnit, and Istanbul for test coverage measurement, so that's the
> stack I'm familiar with. It's worked very well for us; there's a
> grunt-qunit-istanbul plugin that brings the pieces together nicely. But
> I didn't make those choices and am not familiar with the alternatives;
> there may be better options.)

I have a very similar setup. I chose qunit because it seemed popular
among front-end JavaScript projects and because jQuery uses it.

QUnit and Jasmine seem like the winners in this space at the moment.

We may also want to consider looking at the Karma test runner by the
Angular.js team (it supports a number of test frameworks including
QUnit and Jasmine). I know little about it, so feedback on this would
be helpful. This may be something that can be added later.

I analyzed the test runners used by the top 12 front-end JavaScript
libraries on Github (according to the API). Here are the numbers:

QUnit test framework is used by:
- Backbone
- Ember.js
- Knockout
- Reveal.js
- jQuery
- three.js
- jQuery-File-Upload

Jasmine test framework is used by:
- Knockout
- Spine
- Brackets
- Angular.js (using Karma)

Vows test framework is used by:
- d3

I'll start working on that DEP.

--
Trey Hunner

Marc Tamlyn

unread,

Apr 18, 2014, 7:17:30 PM4/18/14

to django-d...@googlegroups.com

Having worked with both, I'd like to put in a vote for Jasmine over QUnit. Unit test is a Java construct which can feel out of place in Python, never mind JS. I find QUnit clunky and difficult to write, whereas jasmine (as a behavioural tool) is pleasant to write and likely comes closer to the kind of tests we would run - checking user interactions with the admin do the correct thing.

As an aside, there may be a way of avoiding the NodeJS problem for devs - I think a static HTML file which includes all the jasmine test files (or QUnit) would work in any (good) browser for local manual testing. We would then either use Phantom for Jenkins, or in theory we could open that aforementioned file using Selenium and check it runs all the JS tests properly. This is probably vile though.

The absolute best solution is to use Jasmine in browser opened via selenium or similar but report back individual test results via a socket to a custom python unit test reporter - reporting each JS test with details to the console as it if were a Python test. This is effectively what karma.js does from what I remember. This would not only be great for our test suite, but allow easily writing front end tests for any Django application. Several projects I have worked on has two completely distinct test suites one for JS and one for python. Being able to run both together must be better - and may also allow use of LiveServer like functionality within your JS (leaky) "unit" tests.

Marc

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CACuWcAxsRN6YwSM5GfA42TEgMv3cWeJ27L3gW5Yh%2Bpm5ASwMag%40mail.gmail.com.

Jannis Leidel

unread,

Apr 19, 2014, 7:50:19 AM4/19/14

to django-d...@googlegroups.com

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Trey, all,

I know we shortly talked about that at PyCon but I forgot to mention
that a while ago we took a stab at that already. Sean Bleier was spear
heading it and I helped out here and there:
https://github.com/sebleier/django/compare/4f3ad28a9b4ffc3ae9866d14f242844d5720b3be...qunit

Sadly we never finished it, even though it was supposed to be merged
together with the LiveServerTestCase. It was providing a "jstest"
management command that was just runserver in disguise with an
enforced URLconf with a few views to render JS test pages.

The actual core part was a collector for JS test suites
(conventionally placed in <app>/tests/javascript). Each suite could
test any kind of local and remote files and used staticfiles for
serving them.

I'm not suggesting to re-use this code but wanted to mention that
we've worked on it and found it pretty good (at the time). We just
lacked the time to finish it. Also, some parts are specific to QUnit
but others (like the suite collector) could be reused for other test
runners.

All in all, I'm all for supporting some way for writing JS tests as
part of Django apps but would be a lot more careful about what
technique/tool to use given the fast paced changes in JS land. I admit
I don't have experience with Jasmine though so take my advise with a
grain of salt :)

Cheers,
Jannis

> -- You received this message because you are subscribed to the
> Google Groups "Django developers" group. To unsubscribe from this
> group and stop receiving emails from it, send an email to
> django-develop...@googlegroups.com

> <mailto:django-develop...@googlegroups.com>. To post to

> this group, send email to django-d...@googlegroups.com

> <mailto:django-d...@googlegroups.com>. Visit this group at

> http://groups.google.com/group/django-developers. To view this
> discussion on the web visit

> https://groups.google.com/d/msgid/django-developers/CACuWcAwHVJ7HfeWOui3pAT3nQJeABP_Vt5WQe1N5%2BvDs%2Bnt8GQ%40mail.gmail.com
>
>
<https://groups.google.com/d/msgid/django-developers/CACuWcAwHVJ7HfeWOui3pAT3nQJeABP_Vt5WQe1N5%2BvDs%2Bnt8GQ%40mail.gmail.com?utm_medium=email&utm_source=footer>.

> For more options, visit https://groups.google.com/d/optout.

Kamil Gałuszka

unread,

Apr 22, 2014, 7:31:09 AM4/22/14

to django-d...@googlegroups.com

Hi Trey,

Trey Hunner

unread,

Apr 24, 2014, 3:51:16 AM4/24/14

to django-d...@googlegroups.com

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Thanks for the information Jannis. I hadn't realized that project got
to the point of a pull request.

Personally I prefer running unit tests without LiveServerTestCase. It
seems like LiveServerTestCase should be used for functional tests (using
selenium) and not unit tests.

I had not anticipated taking up the task of creating a custom
Python-based test harness for JavaScript tests. I am not necessarily
opposed to this idea, but it seems like a complex solution to a simple
problem. I'm more concerned with getting the JavaScript properly tested
than I am with creating a Python-powered JavaScript test harness. I
think that problem (if it is one) could be resolved later without much
hassle in migration.

I disagree with Marc's suggestion that being able to run both Python and
JS tests at the same time "must be better". I see these tests as
distinct from the Python tests. I am certainly biased from my past
experience attempting to run JavaScript tests through Python and giving
up in favor of the more common solution of using Node and PhantomJS.

The great thing about JavaScript tests is that they just run in HTML
files, so if JS unit tests are run through PhantomJS and Node initially
they can be migrated to use a Selenium and Python wrapper later if
desired. Also, regardless of the solution chosen, it should always be
possible to manually run the tests with a web browser.

Sorry for the gap in my reply time. I have been busy this week and
haven't had time to put together a DEP yet.

Trey Hunner
http://treyhunner.com

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJTWMJ0AAoJEOpnfp/NreonPDUIAJZzgCu7YNWz2dLlndlmOQ4P
lRIOSdQcVX4RDAEkZheaQRcymTdXA6GfpzHhQ46RZMW6V8RCuoIwCTAy+My3wEgO
IbMNufIcUh7Kn8sLWxoD/sOCcBjhqTAtCQqfMywYMNtHfCSdUJ/lhUHyRc5CRHYC
CmsXR3rsapc6SHcg0tVCsMQL2V63RLZi1+xAnlBv7CVVcUfSsrEh1k1hw22eFGvP
257Tjb+bl81K48QvFKZoH9vefRnbTKkX7sxxbPsWEC4/T2+Zb6MRU8qzmq+8fUEP
blF3QXNrFudbhI1eAGOKPmmKRhuzHfz+Bx4fVf/JowQuG5cMlk8oLIYpyzmhA5M=
=Fhuu
-----END PGP SIGNATURE-----

Trey Hunner

unread,

Apr 24, 2014, 4:01:49 AM4/24/14

to django-d...@googlegroups.com

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Kamil,

I started another thread related to JS linting:
https://groups.google.com/forum/#!topic/django-developers/GUgRMnnC0dM

Here is a related pull request: https://github.com/django/django/pull/2577

I have not yet changed the code style of any JavaScript files because I
wasn't certain what style should be used.

At this point I think the jshint tool should be used to enforce the
default JSHint code style. If style exceptions arise then they can be
documented in the .jshintrc file and re-factored later.

>> django-develop...@googlegroups.com <javascript:>
>> <mailto:django-develop...@googlegroups.com
> <javascript:>>. To post to
>> this group, send email to django-d...@googlegroups.com <javascript:>
>> <mailto:django-d...@googlegroups.com <javascript:>>. Visit this
> group at
>> http://groups.google.com/group/django-developers
> <http://groups.google.com/group/django-developers>. To view this

>> discussion on the web visit
>
> https://groups.google.com/d/msgid/django-developers/CACuWcAwHVJ7HfeWOui3pAT3nQJeABP_Vt5WQe1N5%2BvDs%2Bnt8GQ%40mail.gmail.com
> <https://groups.google.com/d/msgid/django-developers/CACuWcAwHVJ7HfeWOui3pAT3nQJeABP_Vt5WQe1N5%2BvDs%2Bnt8GQ%40mail.gmail.com>
>
>
>

> <https://groups.google.com/d/msgid/django-developers/CACuWcAwHVJ7HfeWOui3pAT3nQJeABP_Vt5WQe1N5%2BvDs%2Bnt8GQ%40mail.gmail.com?utm_medium=email&utm_source=footer

> <https://groups.google.com/d/msgid/django-developers/CACuWcAwHVJ7HfeWOui3pAT3nQJeABP_Vt5WQe1N5%2BvDs%2Bnt8GQ%40mail.gmail.com?utm_medium=email&utm_source=footer>>.
>
>> For more options, visit https://groups.google.com/d/optout

> <https://groups.google.com/d/optout>.

>
> --
> You received this message because you are subscribed to the Google
> Groups "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-develop...@googlegroups.com

> <mailto:django-develop...@googlegroups.com>.

> To post to this group, send email to django-d...@googlegroups.com
> <mailto:django-d...@googlegroups.com>.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/django-developers/19e76ec5-4e27-4012-864a-f73fe8c73d81%40googlegroups.com
> <https://groups.google.com/d/msgid/django-developers/19e76ec5-4e27-4012-864a-f73fe8c73d81%40googlegroups.com?utm_medium=email&utm_source=footer>.

> For more options, visit https://groups.google.com/d/optout.

- --
Trey Hunner
http://treyhunner.com

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJTWMTtAAoJEOpnfp/NreonC08IAIPg4vrs4udoWBro3UcjU8bR
2oPre4HMSKNSxM3iAnP7uFfPpV9ZzeuJS/TNasxPPhPpLBLSyNVXnOEaMw75mra8
rgCool0RmC7lstgzs2F2jJWx/0gfu2yie/E4p5+9TbbCvve4PScOQRewDFmFaPcQ
DSrZbUT6K4qBTHLuiVj8cqSgA+vDQgWcxKG7vAGbX526aN+WgxrWoAR5gBS54TLW
vvHy4XVc/fMeiKWppQ8CbWC4ycMGdVheaBhEY2u1QUmR224l8EcPYryvWm7G6qzB
bkF+wVYMLA5bdPaOFA3uir28FCVpJklY/KiNYUr2xGGImfPqsYE9Ly9yJYrgu0I=
=Y9ER
-----END PGP SIGNATURE-----

Trey Hunner

unread,

May 5, 2014, 2:19:54 AM5/5/14

to django-d...@googlegroups.com

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I started a rough draft of a DEP for adding unit tests for Django's
JavaScript: https://github.com/django/deps/pull/4

I am suggesting that a native JavaScript test framework be used
*without* attempting to use an adapter to run the tests under the Django
test framework. I anticipate that this suggestion may require further
explanation and debate, so I have submitted a pull request for
discussion while I continue to extend the DEP.

- --
Trey Hunner
http://treyhunner.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJTZy2KAAoJEOpnfp/Nreona8oIAKBE0T05wmnCP0tq/nvmQIlC
0KWB0bX47FHn3TeOjEK8dHrxebP9UDL+wmdZ8F23iOQnW8OiJIO3dJjSjqLOAXxg
dNh5imq02kY2rvjVB6ypZp0h+INkQoaQMae5xMdN4RozLnbNrgXln7vbuSrjIB/8
z7h983vqiRp/ofa+2urTnUnC63730gg6vBbXE5EQ+5WeJxqx2yahdbeSmB3oCnhD
7TMRdHYY+Tx1bdsHF3KOniKoHXA0qoeV16RK+J7EWRHMo3eKlH1zOpcbNX8klwed
qF2zZSKFqMIj0H/mBOVWy5uePXUOfIOP4iLTjEuSy+6sPDNPCiPAvCE5R9jyz/k=
=/i9B
-----END PGP SIGNATURE-----

Nick Sandford

unread,

Aug 1, 2014, 8:29:55 PM8/1/14

to django-d...@googlegroups.com

So, I have been working a little on a PR for this (https://github.com/django/django/pull/3003) and having spoken to Tim briefly about getting this supported in jenkins I just wanted to check that the approach was OK. The tests use node, grunt, qunit, istanbul and jshint. Is this a reasonable approach? Some of the responses here are using a similar stack.

-----END PGP SIGNATURE-----

--
You received this message because you are subscribed to the Google Groups "Django developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.

Visit this group at http://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/53672D8A.2050005%40treyhunner.com.

Trey Hunner

unread,

Aug 2, 2014, 1:22:12 AM8/2/14

to django-d...@googlegroups.com, nick.s...@gmail.com

Hi Nick,

Personally, I think this is a great start. I would like to hear input
from others.

I appreciate the work you put into manually resolving those jshint
warnings. The qunit test style looks good to me too.

I added some comments on the pull request with some suggestions for
improvements.

The biggest suggestions I made in the PR:
1. The Gruntfile.js and package.json should be moved to the top level
2. Use 4 spaces for all code in the PR (it's mostly 2 spaces currently)

Thanks for taking the initiative on this!

On 08/01/2014 05:29 PM, Nick Sandford wrote:
> So, I have been working a little on a PR for this
> (https://github.com/django/django/pull/3003) and having spoken to Tim
> briefly about getting this supported in jenkins I just wanted to check
> that the approach was OK. The tests use node, grunt, qunit, istanbul and
> jshint. Is this a reasonable approach? Some of the responses here are
> using a similar stack.
>
>
> On Mon, May 5, 2014 at 7:19 AM, Trey Hunner <tr...@treyhunner.com
> <mailto:tr...@treyhunner.com>> wrote:
>
> I started a rough draft of a DEP for adding unit tests for Django's
> JavaScript: https://github.com/django/deps/pull/4
>
> I am suggesting that a native JavaScript test framework be used
> *without* attempting to use an adapter to run the tests under the Django
> test framework. I anticipate that this suggestion may require further
> explanation and debate, so I have submitted a pull request for
> discussion while I continue to extend the DEP.
>
>

> --
> You received this message because you are subscribed to the Google
> Groups "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to django-develop...@googlegroups.com

> <mailto:django-developers%2Bunsu...@googlegroups.com>.

> To post to this group, send email to
> django-d...@googlegroups.com

> <mailto:django-d...@googlegroups.com>.

> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/53672D8A.2050005%40treyhunner.com.
> For more options, visit https://groups.google.com/d/optout.
>
>

> --
> You received this message because you are subscribed to the Google
> Groups "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-develop...@googlegroups.com

> <mailto:django-develop...@googlegroups.com>.

> To post to this group, send email to django-d...@googlegroups.com

> <mailto:django-d...@googlegroups.com>.

> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/django-developers/CAMwcc5WuxZ5yVQQH8wVbX51VHyaXN-ZLyrK_oJeS3e4vo5kqvA%40mail.gmail.com
> <https://groups.google.com/d/msgid/django-developers/CAMwcc5WuxZ5yVQQH8wVbX51VHyaXN-ZLyrK_oJeS3e4vo5kqvA%40mail.gmail.com?utm_medium=email&utm_source=footer>.

> For more options, visit https://groups.google.com/d/optout.

Reply all

Reply to author

Forward