TDD style assert aliases for chai plugins

551 views
Skip to first unread message

Bart van der Schoor

unread,
Jun 7, 2013, 8:22:21 AM6/7/13
to cha...@googlegroups.com
I would like to use some of the chai plugins in TDD-style asserts but I see most of them are only available in should- and expect-mode.

Because I see this could be tricky to add in one namespace I started the effort by making a big list indexing the assertions from the plugins and comparing them to the existing core assertions.

It is saved it on github as a markdown document: https://github.com/Bartvds/chai-tdd-plugins

I also had a first pass at trying to fit them in the one assert.<n>() namespace, seems to work out well enough. I added a node.js script to de-dupe the names from the markdown (was bored :) There's also the list of core assertions I had lying around from generating TypeScript declarations for tdd-assert's to add to DefinitelyTyped (tdd-assert's work nicer then expect or should in auto-suggest enabled editors like WebStorm).

As I needed chai-fuzzy in my TypeScript project I implemented those in the plugin and send a pull request to the maintainer. I'm not sure how to proceed next, I think I'll just expand this when I need them myself.

It could get create quite big chunks of aliases for the large plugins, but does that matter?

And how much should we test the aliases if I added them to the plugins? The matchers themself are already fully tested so is only a little bit of alias-integrity-testing enough? Here's what I did for chai-fuzzy: https://github.com/Bartvds/chai-fuzzy/commit/30dfa3ccc0bc8a3eb071e969120a4085e11b0c70

Does somebody have an opinion on this whole thing? (I feel a bit crazy now I look at the document :)

Grtz,

Bart



Jake L.

unread,
Jun 8, 2013, 6:19:49 PM6/8/13
to cha...@googlegroups.com
Hi Bart,

This is a very impressive and thorough evaluation of where TDD/assert stands in plugin-world. 

One specific note on cross-plugin collisions: in some scenarios this "should" be perfectly acceptable. You list chai-spies and sinon-chai as a pain-point but I don't see it that way. Both of these tools address the same test mechanics in different ways: sinon-chai is a vendor integration for an already existing tool while chai-spies takes a lighter approach to a subset of that tool (for those who aren't interested in using sinon.js). It is expected that there should be some overlap and I think that is OK.

I think the same should be said for dom-related tools. Though we only currently have a plugin for jQuery, I can see future implementations that use Zepto or document.querySelector. You shouldn't have to rewrite all your tests if you switch tools... or relearn an API because you are working/consulting on a project that uses different mechanics. This, of course, is a personal philosophy and not one I will enforce on the community of plugin developers - however, I do want to make this easier for everyone.

That being said, prefixing within each tool to make it easier to understand and avoid collision with generics (such as min/max) should be encouraged. IE: "assert.spyCallledMin()" or "assert.domClass()" or "assert.$class()" makes total sense to me. But again, up to the plugin authors.

The big point that you make in this evaluation is one that I whole-heartedly agree with: plugins are notoriously bad at providing compliant assert/tdd assertions in their tools. I am also a perpetrator of this as chai-spies and chai-http are mine; and admitting you have a problem is the first step on the road to remedy. Unfortunately, in chai's current state, it is incredible hard to motivate plugin developers to do this. Not only is it more code to maintain, its an exponentially larger number of tests to worry about (as you pointed at with chai-fuzzy).

One of the things I would like to see in Chai's future is a better plugin system. My current line of thinking is that a plugin author should write one assertion and test that assertion regardless of interface and then chai-core will then build out the interface for the end user based on alias hints from the plugin. This would allow for new and interesting interfaces (such as stringed expect) without plugin authors having to build in compliance. But this thought is still in early requirements gathering phase...

The first/current iteration of the plugin system is embarrassingly targeted at chain-able APIs. As an assert user, core/plugin contributor, and possible future plugin author: what features/tools/requirements would make that next generation plugin API an appealing and functional replacement for TDD users and plugin authors?

Thank you for putting together such a thorough piece of documentation and I look forward to your thoughts and discussion on the long-term future of TDD within Chai.

Kindest Regards,
Jake (GitHub: @logicalparadox)

Bart van der Schoor

unread,
Jun 9, 2013, 6:15:13 AM6/9/13
to cha...@googlegroups.com
Thanks for your extensive reply.

I agree with the notion that colliding names don't have to be a problem if they'd collide in alternate sets like you describe, maybe they should indeed be grouped a bit with a 'domain' prefix like dom.. or spy... or similar.

The main reasons I favour TDD assert() over expect() and .should is because it's less wordy and easier on the keyboard. And also because plays very nice with typing layers like TypeScript or tern.js because the 'actual' parameter is defined on the assert function so it can be properly shadow typed.

But even in plain JavaScript I personally don't care very much for the descriptive dot notation of expect() and .should (and tickles my DRY sensor to type .to.have.bleh.bla so much). It's the same with the proposed string-interface, it's too much, although I can see how strings could offer some interesting possibilities in automation or generators.

What would be nice is an reworked interface that is a middle-ground between stark TDD single-term fucntions and the wordy BDD sentences. Short and efficient but with all the features, and generated from same assertion verbs.

It probably should use chained not-negations so we don't have to create/maintain them by hand:
    assert.equal(actual, expected, msg)
    assert.not.equal(actual, expected, msg)

Personally I could do without most of the other language chaining sugar as the docs say it usually doesn't add functionality anyway. For the cases where plugins overwrite the keywords there could be a argument for name-concatenating scheme with auto-camelcasing.

If I look at chai-stats this sort-of happens already:
    expect(actual).to.almost.equal(..)
    expect(actual).to.deepAlmostEqual(..)

So if you glue things'up automatically you can get something like:
    expect(actual).to.have.deviation.almost.equal(...)
ditch the sugar and become:
    assert.deviationAlmostEqual(actual, ...)

There likely should still be support for a sugar layer to append some exotic other assertions without generating to much aliases.

Like if I look at chai-things it'll get messy how it wraps other asserts:
    expect(actual).to.include.some.equal(expected) 
It could become:
    assert.some.equal(actual, expected)

But maybe the concat/camel-casing scheme is not needed at all if we opt for a light purely functional 'tight' chaining flow? The current assertion verbs seem usable enough in itself so we might as well use them.

I'm not sure how it works internally but maybe chai does already have the required data to generate an light interface like this?

Regards,

Bart

Op zondag 9 juni 2013 00:19:49 UTC+2 schreef Jake L. het volgende:

Todd Wolfson

unread,
Jun 9, 2013, 6:08:50 PM6/9/13
to cha...@googlegroups.com
Putting in my 2 cents. I am glad we are re-evaluating chaining on `assert`. It feels like the camelCase is lipstick and short-sighted at that.

At chai's core, it is all about chaining and if a plugin requires chaining (e.g. grab an element, grab its child, look at color of it), then that should be supported on all fronts. Moving to a camel case model solves the first resolution but does not solve any chaining problems without severely bloating the getters/setters.

This problem is caused in-part by the fragmented language between `assert` and `expect/should`. One uses camelCase and the other uses chaining. While this might be great for IE6 support, it has brought us to this point and a re-evaluation of the fragmentation might be in order.

 - Todd

Bart van der Schoor

unread,
Jun 10, 2013, 4:09:48 AM6/10/13
to cha...@googlegroups.com
IE6, oh my, well, I completely forgot about it (finally! :), and how it doesn't support getters. I could see how you'd want to keep some basic support for it, although many users don't seem to care for testing on it and go all out with the chaining.

Still I think a terse style with less natural-language sugar but including the useful combination chaining and having the "actual"-parameter moved to the assertion verb would be a very nice middle ground.

But is that still fit to be the assert keyword? Can assert have chaining or is there some fixed community convention to only have it use plain functions? I wouldn't mind chaining to it instead of inventing a new term but to be honest I don't care overly for hanging on to idioms if they aren't practical (I mix TDD assert with BDD describe/it, and it works fine as chai will report readable messages anyway)

Of course a new hybrid would only fragment things more :)

- Bart

ps: related: http://xkcd.com/927/

2013/6/10 Todd Wolfson <to...@twolfson.com>
--
You received this message because you are subscribed to a topic in the Google Groups "Chai.js" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/chaijs/TXtkOAGyetU/unsubscribe?hl=en.
To unsubscribe from this group and all its topics, send an email to chaijs+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

Stepan Riha

unread,
Jun 10, 2013, 12:42:23 PM6/10/13
to cha...@googlegroups.com
We too use TDD asserts with BDD tests - and for exactly the same reasons.  BDD does a great job of structuring tests and TDD asserts are nicely terse.  IMO, the fluent "expect" assertions introduce a lot of sugary noise between the salient arguments.

FWIW, I do see the appeal of both styles, but adding chaining to asserts muddies the issue.  Chaining makes sense with promise asserts, but that's because of the async natures and not because of syntax sugar.

We usually get around the missing assert variants by using assert.isTrue(...) and echoing the test expression in the message string, e.g.:

assert.isTrue($('.foo').is(':checked'), "$('.foo').is(':checked')");

The duplication isn't optimal, but c'est la vie.  If the main concern is namespace pollution between plugins, then I would prefer for it to be in one of these forms (jq is short of jQuery):

// Using a "jq" prefix
assert.jqIsChecked('.foo');
// Using a "jq" namespace
assert.jq.isChecked('.foo');

I think I'd prefer the second one.

- Stepan

On Monday, June 10, 2013 3:09:48 AM UTC-5, Bart van der Schoor wrote:
[...]

Bart van der Schoor

unread,
Jun 10, 2013, 5:21:47 PM6/10/13
to cha...@googlegroups.com
Noise reduction and dryness is a large part of what I think would be a desirable aspect of an assertion style, but I think there could be a little (purely functional) chaining to keep things workable.

Some plugins use a chaining keyword to expand the assertion (like chai-things), if we could detect/filter those cases the style generator could only add those chains to the terse hybrid and ignore the rest. With a bit of luck this generator can work with the same assertion verbs as the BDD style, especially if a plugin could specify a short 'namespace' like you described. I'd favour the chained namespace as it'll would work beautiful with typing hints.

On the isTrue()workaround: it is certainly valid but personally I love the the enhanced error messages you get when you can use specialized assertions, even without specifying a custom assertion message. And proper assertions are also better if you want both the description access options like the diff's).


2013/6/10 Stepan Riha <non...@gmail.com>

--

Jake L.

unread,
Jun 12, 2013, 8:50:31 AM6/12/13
to cha...@googlegroups.com
Thank you all for your input. I think with the proper planning of a plugin manager the different variations expressed here can be realized. This is not the final spec, but a starting point for further API specifics.

PLUGIN API

I see three types of language slugs....

1. Chains
Noop "getters" that are used in chainable interfaces. "to", "is", "and", "that". Needed for BDD but may not be needed for TDD.

2. Modifiers
Op "getters" that are used to change the behavior of the methods: "not", "include", "contain", "deep". There are two subtypes of modifiers:
- "global": can be used for any assertion. IE: "not"
- "targeted": used only to modify some methods: IE: "deep" for "equal" or "property", etc.

The two types will allow us to construct camelCase interfaces or warn in chainable interfaces if you are using a modifier with an incompatible method. Can be used in the generation of pure camelCase APIs: "assert.notDeepEqual(a, b)".

3. Methods
Assertions that accept input: "equal", etc.

Note 1: We already have the ability to have a language element in the chain to function as both modifier and a method. This would need to be retained for the BDD interfaces.
Note 2: We already have the ability to stack assertions. This is how "gte"/"lte" can function for both numbers or the count of calls made to a spy based on the input.. This would need to be retained for both TDD/BDD.

INTERFACES

With this we can easy create many different types of interfaces.

- assert-classic

assert.equal(a, b);
assert.notEqual(a, b);
assert.deepEqual(a, b);
assert.notDeepEqual(a, b);

- assert-hybrid

assert.equal(a, b);
assert.not.equal(a, b);
assert.deepEqual(a, b);
assert.not.deepEqual(a, b);

- expect

expect(a).to.equal(b);
expect(a).to.not.equal(b);
expect(a).to.deep.equal(b);
expect(a).to.not.deep.equal(b);

- should

a.should.equal(b);
// etc..

- expect-legacy

expect(a).toEqual(b);
expect(a).toNotEqual(b);
expect(a).toDeepEqual(b);
expect(a).toNotDeepEqual(b);

Note 1: If done correctly, "assert-classic" and "expect-legacy" will work in older browsers (IE7+) the same way they work in modern browsers.

Note 2: For performance reasons it is best to construct the interface once as opposed to for every assertion. Therefor, plugins will needed to be "loaded" before the interface is exported.

var chai = require('chai')
  , assert;

chai.use(require('chai-spies'));
chai.use(require('chai-jquery'));

assert = chai.interface('assert-classic');

NAMESPACING

Namespacing is an interesting concept because it is only really needed in BDD. Though this isn't final, this is how I see it. For example:

expect(emitterSpy).to.have.been.called.gte(3);
assert.spyCalledGte(emitterSpy, 3);

In this example the namespace is "spy" but it is not required in the BDD because it is self descriptive. Whereas it might be needed in the TDD. If we didn't have the namespace you would get:

assert.calledGte(emitterSpy, 3);

The second "assert" example isn't as descriptive. Therefor, the namespace is applied to "called" and it is included when the interface is constructed as a TDD style interface. 

=====

Obviously this needs some more work, but I think it is a starting point for technical discussion. Thoughts?

Bart van der Schoor

unread,
Jun 12, 2013, 12:35:44 PM6/12/13
to cha...@googlegroups.com
Oh wow, that's the kind of thing I personally was hoping for. I don't see anything obviously wrong with it either.

If you can hammer this into an generator/data API you can also cover that string based interface mentioned in the Issues, and allow people create other experimental/exotic styles.

If possible it could be awesome if there's support to declare _all_ meta data directly in the assertion declaration, like parameter names & types, description, error message etc.

We can then access that programmatically and do stuff with it, like generating API docs, give runtime help, auto-extract assertions for reuse in odd niche projects or output declarations for TypeScript and similar.

Looks promising so far!

B.

2013/6/12 Jake L. <jake...@gmail.com>

Todd Wolfson

unread,
Jun 13, 2013, 1:08:10 AM6/13/13
to cha...@googlegroups.com
Seems legitimate and a reachable goal. My concern is the amount of effort it will take to get there; the logic is pretty straight forward but there are a lot of cases.

Just to make the obvious questions:

- Are we guaranteed to present `assert`, `expect`, and `should` as default properties of `chai` without `use`?
- Why no `should-legacy`? The prototypes should be extensible in all browsers and hopefully people who use it understand what they are opting in for.

For namespacing, I am going to play the devil's advocate. It seems like an awesome concept but if we are sniffing for `instanceof` for specific methods to automatically become accessible then we will run into trouble if the assertion is going to fail.

For example, if we are expecting a `spy` and receive `undefined` in the `expect` block, then we won't present any of the `spy` methods and the accessors will fall on their faces. Part of the reason I love `expect` so much is that it will give me an AssertionError no matter what I feed it since the `actualItem` is always wrapped.

I am a little worried about the permutation power set for `expect-legacy` since ordering can be flipped around (e.g. `toDeepNotEqual`, `toNotDeepEqual`).

In the end, this seems to umbrella everyone's problems and hopefully mitigate them.

 - Todd

mi...@bocoup.com

unread,
Jul 11, 2013, 5:00:04 PM7/11/13
to cha...@googlegroups.com
This seems like the right place to build in an assertion counter. If assertions in Chai are defined in such a rigorous way, it seems like Chai could also provide an API for keeping track of assertions. With support like this, we could integrate expected assertion count into our test runner of choice. I'm not seeing any discussion around such functionality on the list--is it even in-scope?

Bart van der Schoor

unread,
Aug 7, 2013, 1:31:10 PM8/7/13
to cha...@googlegroups.com
I was wondering if this assertion-style generator thing with IE support and everything that we talked about is still being considered. Not that I must have it right now or anything but it is an interesting solution to a recurring problem.

Like I did have an encounter with dread IE8 again and had to use proclaim instead of chai; clients keep asking for it. Sad face :(

I'm actually eyeballing Jasmine over Mocha sometimes: Jasmine 1.x does have IE support, but I really dislike their odd async test pattern (waitFor / runs). They fixed that in Jasmine 2.0 to work similar to Mocha, but alas they also dropped old IE support. So no-go there.

I looked into Vows as well but I dislike that it is a Object structure instead of functions. And the grunt plugins feel shoddy. There are a few other test runners that work on Node.js AND browsers but mos tof them are niche or also do not support old IE's.

So safest way to get all-in-one testing proper async for both Node.js as browser is still Mocha, but it sucks to have to use proclaim as assertion library for some projects; it's just not the same as chai.

So what do you think? How do you guys handle this?

Op donderdag 11 juli 2013 23:00:04 UTC+2 schreef mi...@bocoup.com:
Reply all
Reply to author
Forward
0 new messages