understanding RSpec's design

[bone...@boneskull.com]

Hi,

I'm a current maintainer of Mocha, which (AFAIK) was inspired by RSpec.  I am not the author of Mocha; the author is long gone, or I'd ask him about this.

My question regards a paradigm which seems common to BDD-style test frameworks.  I assume that RSpec works similarly, but I apologize if I'm incorrect.

It's most easily illustrated with a pseudocode example (sorry, I'm unfamiliar with Ruby):

begin suite 'foo'

  begin test 'bar'
    print 1
  end test
 
  begin suite 'baz'

    begin test 'quux'
      print 2
    end test

    print 3

  end suite
end suite

In Mocha (and I assume RSpec), the following would be printed:

1
3
2

This is difficult for some users to reason about.  I'm unable to explain what this algorithm buys us, other than perhaps better control over disabling tests and suites.  Can anyone explain why it works as it does?

thanks

Chris

[Myron Marston]

Assuming I understand your pseudocode correctly, your example would be written like this:

``` ruby

RSpec.describe "foo" do

  it "bar" do

    puts 1

  end

  describe "baz" do

    it "quux" do

      puts 2

    end

    puts 3

  end

end

```

This would print the numbers in the following order:

3

1

2

...which, noticably, is different from the order of mocha.  I can't comment on Mocha's design (having never used it) but for RSpec, we intentionally load all the specs first (which involves evaluating the `describe` blocks), then apply spec ordering, filtering, etc, and then run the specs (the `it` blocks).  Thus, the `3` is printed first (as it got printed while specs were being defined) and then the specs ran and 1 and 2 are printed.

HTH,

Myron

--
You received this message because you are subscribed to the Google Groups "rspec" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rspec+un...@googlegroups.com.
To post to this group, send email to rs...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/rspec/872f0ecb-4ca1-4440-bb3f-27e111c80d89%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[bone...@boneskull.com]

Gah, I screwed that up.  No, Mocha works like how you're describing; 3 1 2 would be the order.

So, now that I've confirmed it works the same, my answer is "why"?

[Myron Marston]

I tried to answer the "why" when I said:

...but for RSpec, we intentionally load all the specs first (which involves evaluating the `describe` blocks), then apply spec ordering, filtering, etc, and then run the specs (the `it` blocks).  Thus, the `3` is printed first (as it got printed while specs were being defined) and then the specs ran and 1 and 2 are printed.

If that doesn't answer your question, can you explain what order you would expect?  The order RSpec operates in feels completely natural to me as it enables features like spec ordering, filtering, etc...but I've never really considered another ordering, so I'm not sure what order you have in mind that you are contrasting RSpec's ordering to.

Myron

To view this discussion on the web visit https://groups.google.com/d/msgid/rspec/cdfe4dcd-aea0-43ef-84fe-64603aa2c64f%40googlegroups.com.

[Jon Rowe]

I would only operate in that order in “defined” order mode, meaning that specs are executed in the order they are defined. In other (including the automatically suggested random) ordering setups only the 3 would be guaranteed to the be first, the 1 and 2 would depend on the spec execution order.

Jon Rowe

---------------------------

ma...@jonrowe.co.uk

jonrowe.co.uk

To view this discussion on the web visit https://groups.google.com/d/msgid/rspec/CADUxQmsz1%3DzSgfKfVYcdFbNtVCZtMNCFTpvzGjrwdT8Tie9tew%40mail.gmail.com.

[bone...@boneskull.com]

Sorry, I think my reading comprehension was poor, as it seems you did answer this indirectly.

OK, it looks like the "why" would then be to offer ordering and filtering functionality.  Is this correct?

I would assume (remember, I'm not expecting different behavior; some users are) the "alternative" would be that a test is executed when it's encountered, as opposed to "collecting" all the suites (or "groups", "describes", etc.) beforehand, then choosing which tests (or "specs", "examples", etc.) to run (and in what order).

I'm going to guess that the "inclusion" filtering is not possible without using the current strategy, because the "alternative" strategy would not necessarily know that an "inclusion" filter was in use--until it was too late.  

How would the "alternative" strategy inhibit RSpec's ordering functionality, if at all?

Thanks for helping me out.

Chris

[Jon Rowe]

The “alternative” strategy you suggest isn’t really possible for us, but in it wouldn’t allow any kind of ordering other than “defined"

Jon Rowe

---------------------------

ma...@jonrowe.co.uk

jonrowe.co.uk

To view this discussion on the web visit https://groups.google.com/d/msgid/rspec/2bf301e8-19b0-41b2-8917-7957fed96df4%40googlegroups.com.

[Myron Marston]

To expand on what Jon said: we offer random ordering as a way to surface accidental ordering dependencies.  It's not possible to offer random ordering if the specs are executed as they are defined, because the order they are defined is the same on every run of the suite.

To view this discussion on the web visit https://groups.google.com/d/msgid/rspec/39AEAC5DB783416FBDBB2596BE3E1A13%40jonrowe.co.uk.

[bone...@boneskull.com]

Alright.  Thanks Myron and Jon; you've answered my questions.