[Opinions Needed] Solution: Running only a single spec/suite

[Kristian Calhoun]

Hello all,

In reference to the following unanswered question:
http://groups.google.com/group/jasmine-js/browse_thread/thread/d522098457385249,
I modified Jasmine to be able to run only a single spec or suite of
specs. Similar to xit and xdescribe, a spec or suite can now be
flagged to be the only one to be tested by changing the function name
to either oit or odescribe. It currently works as follows:

*If there are multiple oits, then they will each run and all other non-
oit specs will be skipped.
*When using odescribe, all specs within that suite will be tested,
including those in any nested suites. However, if there is an oit
within an odescribe, then the oit gets priority and no other specs
within the suite will run (except for other oits).
*Moreover, if the oit is nested in another suite within the odescribe,
then that oit is the only spec that’s ran within that inner suite—the
rest of the specs outside of that suite, but still within the scope of
the original odescribe, also run.

I wanted to get others' thoughts on whether or not they think this is
a useful feature that would be worth contributing to the open source
project. If so, does the behavior described above sound right or
should it be changed to meet some other requirements?

I look forward to your feedback!

Thanks,
Kristian

[Frank Schwieterman]

Sounds great! :)

> --
> You received this message because you are subscribed to the Google Groups "Jasmine" group.
> To post to this group, send email to jasmi...@googlegroups.com.
> To unsubscribe from this group, send email to jasmine-js+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/jasmine-js?hl=en.
>
>

[Rajan Agaskar]

I don't know if I'm a fan of adding syntax to focus a spec or suite. I'd prefer to see this interaction happen on via the runner/reporter. 

For those not familiar with how jasmine currently focuses specs: we essentially skip running the function that defines 'unfocused' specs rather than just collecting and running focused specs, which probably adds significant time to a suite. We also continue to load all specs in a suite, even when you've focused a single one (skippable via the server by visiting the __suite__ specific page). 

Hopefully it should be possible for you to release as a pluggable additional behavior for jasmine (ie, you load jasmine-core, then load the jasmine-ofocus behavior logic). Let us know there are any missing hooks in jasmine-core that you would need to achieve this.  

[fschwiet]

I'd always assume the right way to do this was to take the target
test as a querystring parameter, like you. That definitely is cleaner
in that it doesn't impact the test code... I had not considered
Kristian's approach until it was posted. Then thinking how I run
debug tests, actually using oit/odescribe would be more convenient. I
would be happy with either approach.

I didn't realize that focusing a spec would limit execution of
unrelated it() blocks. It doesn't limit execution of unrelated
beforeEach()/afterEach() blocks too does it?

On Apr 7, 4:34 am, Rajan Agaskar <ra...@pivotallabs.com> wrote:
> I don't know if I'm a fan of adding syntax to focus a spec or suite. I'd
> prefer to see this interaction happen on via the runner/reporter.
>
> For those not familiar with how jasmine currently focuses specs: we
> essentially skip running the function that defines 'unfocused' specs rather
> than just collecting and running focused specs, which probably adds
> significant time to a suite. We also continue to load all specs in a suite,
> even when you've focused a single one (skippable via the server by visiting
> the __suite__ specific page).
>
> Hopefully it should be possible for you to release as a pluggable additional
> behavior for jasmine (ie, you load jasmine-core, then load the
> jasmine-ofocus behavior logic). Let us know there are any missing hooks in
> jasmine-core that you would need to achieve this.
>

> On Wed, Apr 6, 2011 at 4:37 PM, Frank Schwieterman <fschw...@gmail.com>wrote:
>
> >  Sounds great!  :)
>
> > On Wed, Apr 6, 2011 at 12:46 PM, Kristian Calhoun
> > <kristian.calh...@gmail.com> wrote:
> > > Hello all,
>
> > > In reference to the following unanswered question:
>

> >http://groups.google.com/group/jasmine-js/browse_thread/thread/d52209...
> > ,

[Rajan Agaskar]

On Thu, Apr 7, 2011 at 4:14 PM, fschwiet <fsch...@gmail.com> wrote:

 I'd always assume the right way to do this was to take the target
test as a querystring parameter, like you.  That definitely is cleaner
in that it doesn't impact the test code...  I had not considered
Kristian's approach until it was posted.  Then thinking how I run
debug tests, actually using oit/odescribe would be more convenient.  I
would be happy with either approach. 

 I didn't realize that focusing a spec would limit execution of
unrelated it() blocks.  It doesn't limit execution of unrelated
beforeEach()/afterEach() blocks too does it?

 It will execute all beforeEach/afterEach blocks that would have normally been run for that spec.  

[Patrick Cozzi]

I have been using Kristian's oit/odescribe for a few weeks now and am
very with it. It is really easy to write a spec and only run it, or,
maybe more commonly, only run a single suite while I continue to add
new specs. The syntax is really convenient and mirrors xit/
xdescribe. It also easily allows running multiple, but not all, specs/
suites.

In practice, the main downside is sometimes I forget to strip the o-
prefix from an oit/odescribe before submitting to source control. I'm
not sure if this is a big enough deal to make it a show stopper.

Are alternative designs also as convenient for the test developer?
TestDriven.NET is a really nice tool in the .NET world that basically
lets you right click on a test and run it. I wonder if that can
provide any inspiration.

Patrick



On Apr 8, 1:55 pm, Rajan Agaskar <ra...@pivotallabs.com> wrote:

[Davis Frank]

The runner HTML does allow for focused spec running - you can run a single spec or a single describe pretty easily.

--dwf

[fschwiet]

Davis I was surprised when you said this, as when I select a single
Jasmine test it seems to run them all. I have over 900 tests that
take almost 10 seconds to run on Chrome. They're slow mainly due to
DOM manipulation. When I select a single test, the URL does change to
something like "js/testrunner.html?spec=CustomJasmineMatchers
%20toDeepEqual%20identifies%20matches.". However, every test still
runs. It takes around 10 seconds still, and if other tests not
focused fail the failures still show up in the result. The summary
shows only 1 spec ran, but if I clicked the "Show passed" or "Show
skipped" dialogs I see every test in one or the other.

Any idea why it may not be working? Where can I set a breakpoint to
start following the spec focus behavior? I just grabbed the latest
version of Jasmine, it didn't make any difference for me.

On Apr 11, 11:02 am, Davis Frank <dwfr...@pivotallabs.com> wrote:
> The runner HTML does allow for focused spec running - you can run a single
> spec or a single describe pretty easily.
>
> --dwf
>

[Davis Frank]

Well, they "run", but are skipped.

It may be that all the befores run, which may contribute to the lack of change in the execution time. I'd have to go dig into the depths to better understand that (it's been a while since we've played around in that code).

You can follow down the execution from the Jasmine environment #exec function.

--dwf

[fschwiet]

Ahh ok. Hrmm does Kristian's oit()/odescribe() help with execution
time? Kristian where can I find your sourcecode?

I did find where the spec filtering is done, it is applied to the
test results when displayed, but not during execution. I suppose the
existing querystring-based filter could be applied to execution as
well.

On Apr 13, 8:10 pm, Davis Frank <dwfr...@pivotallabs.com> wrote:
> Well, they "run", but are skipped.
>
> It may be that all the befores run, which may contribute to the lack of
> change in the execution time. I'd have to go dig into the depths to better
> understand that (it's been a while since we've played around in that code).
>
> You can follow down the execution from the Jasmine environment #exec
> function.
>
> --dwf
>

[Rajan Agaskar]

Huh. Interesting. It's been awhile since I looked at this, but it does appear that the focused running doesn't pass anything to jasmine core. We could probably do better here by giving the reporter the ability to ask Jasmine for a spec rather than Jasmine happily plowing through whatever suites it can find. That may mean we need to do some remapping of how suites are loaded but that could be a win anyways, as our large in memory rep of suites has caused probs in the past. I'll think about this a little more in the coming weeks. 

[Kristian Calhoun]

As Davis mentioned, the beforeEach blocks can account for a lot of the
execution time. With my modifications, only the relevant beforeEach
blocks run for the specs in a suite that have been flagged by oit()/
odescribe(), thereby saving time.
While the HTML runner does allow you to select a single spec or suite
to run, one has to load that page first (and therefore the unwanted
specs/suites) before being able to select the option to focus on a
single spec/suite. Using oit()/odescribe avoids this annoyance, as
well as allows one to easily run multiple specs/suites while avoiding
all others, something that's missing from the runner (without having
to use a plethora of calls to xit());.

@fschwiet - The source code can be found here: http://db.tt/Ozpkk9d
The general idea is that each spec is given a new property named
run_flag that is set after calling oit()/odescribe(). The specFilter
function, which previously existed, then catches whether or not any
specs have been flagged and is used to determine which specs, if any,
should be skipped. If there is an easier or more efficient way of
accomplishing this, I'm more than open to ideas.

@Rajan - Take a look at the source code as well and let me know if you
have any ideas for how to make this functionality a separate plugin.
However, I'm not entirely sure that doing so is necessary as the
current implementation works just the same if the user is oblivious
to, or chooses not to use the oit()/odescribe() functionality. Though
there are probably ways to clean up the code to keep this
functionality, but make it more modular and less intrusive to the core
Jasmine code..

Thanks for all of the replies,

Kristian

> ...
>
> read more »

[Kristian Calhoun]

Has anyone had the chance to take a look at the source code I posted
in my last reply? What did you think? Should I move forward with
submitting it to the GitHub?

I have been using this modified version of Jasmine for the past month
and, with over 500 tests and growing, it has proved to be invaluable
in terms of running tests for only the portions of the code I'm
currently working on. So much time has been saved by being able to
avoid running the tests that have the longest running times that I
know have already passed; it's been much easier to simply odescribe a
single suite rather than xdescribing 50 suites.


On Apr 15, 11:43 am, Kristian Calhoun <kristian.calh...@gmail.com>
wrote:

> ...
>
> read more »

[Doug Reeder]

Something with that functionality would be very useful to me. I don't know the Jasmine source well enough to comment on whether your approach is the right one.

On May 5, 2011, at 4:30 PM, Kristian Calhoun wrote:

> Has anyone had the chance to take a look at the source code I posted
> in my last reply? What did you think? Should I move forward with
> submitting it to the GitHub?
>
> I have been using this modified version of Jasmine for the past month
> and, with over 500 tests and growing, it has proved to be invaluable
> in terms of running tests for only the portions of the code I'm
> currently working on. So much time has been saved by being able to
> avoid running the tests that have the longest running times that I
> know have already passed; it's been much easier to simply odescribe a
> single suite rather than xdescribing 50 suites.

Doug Reeder
reed...@gmail.com
http://reeder29.livejournal.com/
https://twitter.com/reeder29

https://twitter.com/hominidsoftware
http://outlinetracker.com

[Rajan Agaskar]

As I mentioned previously, I'm reluctant to add additional syntax that people will depend on going forward. That said, there's definitely a need to provide this sort of functionality. We've been spending some time getting the jasmine-gem project up to speed and will probably turn our attention to the jasmine core libraries next, and we'll certainly consider this patch. 

In the meantime, is there any possibility you can offer this functionality as pluggable behavior? A lot of core jasmine functionality should be monkey-patchable by including additional js files in your suite. This would permit a simple path for people to use this functionality and not require you to maintain a fork.  

Thanks!

Rajan

[Toby Ho]

Hello all,

I've used Kristian's fork in my work for a few days and think this is really valuable: I have a test suite that runs in around 30 seconds, and being able to focus to a single spec or a single suite quickly is really huge - and this method is way quicker than running all the tests first and then picking the one you want from a list on the test page.

I would like to restart this thread and try to get this feature into Jasmine. To start, I've re-implemented odescribe and oit in a branch

https://github.com/airportyh/jasmine/tree/odescribe

and the commit is

https://github.com/airportyh/jasmine/commit/6911d6c9d178841df572a5f5f4adbf47f0111e77

I re-implemented this because I reviewed Kristian's code(on his dropdox account - his github branch seems incomplete) and thought it could be done a simpler way, but the functionality and strategy is basically the same.

Toby

[Toby Ho]

I have since starting recommending the ddescribe/iit fork by vojtajina

https://github.com/pivotal/jasmine/pull/181

which you can get the built version here

https://raw.github.com/vojtajina/jasmine/ddescribe-iit-build/lib/jasmine-core/jasmine.js

Try that and see.

Toby

On Wed, Dec 5, 2012 at 6:21 AM, Валерий Васин <inv...@gmail.com> wrote:
> Hi, guys!
>
> Tell me please, how could I run special suite / spec?
>
> HtmlReporter works incorrect for me. And tests don't run if I filter (via
> link click).
>
> But on http://pivotal.github.com/jasmine/ it works fine for me.. What the
> reason is?

>
> --
> You received this message because you are subscribed to the Google Groups
> "Jasmine" group.

> To view this discussion on the web visit
> https://groups.google.com/d/msg/jasmine-js/-/glZ4UaO_RZoJ.