rspec runs indeterminate number of tests

[Dan Alvizu]

Hi!

I recently upgraded an ancient rails 3.2 app to rails 4.1. Everything looks good, except my rspec doesn't want to run all tests now!

If I run rspec --dry-run, I see 429 tests available. However I never see rspec actually run all of these tests. It runs what seems to be an indeterminate number of tests: sometimes as many as 237, sometimes as few as 2!

I'm running rspec v 3.5.2-rails and rspec 3.5

Is there any config or condition which would cause rspec to skip an entire test suite?

[Myron Marston]

There are a variety of ways to filter specs, and one can imagine configuring the filtering to be randomized to achieve the kind of behavior you're seeing.  That said, you'd almost certainly have to set that up intentionally to get that behavior and I can't imagine you're doing that.

A more likely possibility is that you've got an `exit` or `exit!` call somewhere in your spec suite or in code being called from your suite.  Such a method call will cause the ruby interpreter to exit, cutting short the run (at least on recent 3.x releases; we changed things to not interfere with `exit` and `exit!` calls since they are core ruby methods we didn't feel like we should interfere with).  My suggestion is to see what is the last spec when your suite exits (running with `--format doc` will help figure that out).  That spec is probably calling `exit` or calling code that calls `exit`.  If you can find the `exit` call (or comment out the spec) it will hopefully fix the problem.

HTH,

Myron

--
You received this message because you are subscribed to the Google Groups "rspec" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rspec+unsubscribe@googlegroups.com.
To post to this group, send email to rs...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/rspec/edf89a5d-cc4f-4240-aa8b-20407f8cc8d4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Dan Alvizu]

Aha!

Running with `--format doc` allowed me to see what the last test was run an identify the issue. I didn't have exit, but I did have a test using backticks to restart a service I didn't have, which presumably caused the suit to exit. 

Thanks!

On Friday, January 6, 2017 at 11:27:43 AM UTC-7, Myron Marston wrote:

There are a variety of ways to filter specs, and one can imagine configuring the filtering to be randomized to achieve the kind of behavior you're seeing.  That said, you'd almost certainly have to set that up intentionally to get that behavior and I can't imagine you're doing that.

A more likely possibility is that you've got an `exit` or `exit!` call somewhere in your spec suite or in code being called from your suite.  Such a method call will cause the ruby interpreter to exit, cutting short the run (at least on recent 3.x releases; we changed things to not interfere with `exit` and `exit!` calls since they are core ruby methods we didn't feel like we should interfere with).  My suggestion is to see what is the last spec when your suite exits (running with `--format doc` will help figure that out).  That spec is probably calling `exit` or calling code that calls `exit`.  If you can find the `exit` call (or comment out the spec) it will hopefully fix the problem.

HTH,

Myron

On Fri, Jan 6, 2017 at 10:19 AM, 'Dan Alvizu' via rspec <rs...@googlegroups.com> wrote:

Hi!

I recently upgraded an ancient rails 3.2 app to rails 4.1. Everything looks good, except my rspec doesn't want to run all tests now!

If I run rspec --dry-run, I see 429 tests available. However I never see rspec actually run all of these tests. It runs what seems to be an indeterminate number of tests: sometimes as many as 237, sometimes as few as 2!

I'm running rspec v 3.5.2-rails and rspec 3.5

Is there any config or condition which would cause rspec to skip an entire test suite?

--
You received this message because you are subscribed to the Google Groups "rspec" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rspec+un...@googlegroups.com.

[Myron Marston]

Using backticks should spawn a subshell and run the provided command. I didn’t think it could cause your current ruby process to exit. Can you provide more detail about what was in the backticks? And can you see if it was raising an exception (via a snippet like this)?

def safe_backticks(command)
  `#{command}`
rescue Exception => ex
  puts "#{ex.class}: #{ex.message}"
end

​

To unsubscribe from this group and stop receiving emails from it, send an email to rspec+unsubscribe@googlegroups.com.

To post to this group, send email to rs...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/rspec/cac38f21-2ef7-4609-9d71-d6316412f78b%40googlegroups.com.

[Dan Alvizu]

I was incorrect - it was not backtickets that was the problem.

I am using Stalker (https://github.com/han/stalker) and a call to Stalker.prep when beanstalkd was not running was causing the ruby process to exit

[Myron Marston]

Right, I can see that here:

https://github.com/han/stalker/blob/18a725d42cbbe87361f3bc0dcad6e43a90bdc6dc/lib/stalker.rb#L112

IMO, libraries generally shouldn't do this; it should be up to the host application to decide what to do when an error occurs.  Causing the OS process to exit (and giving the host app no opportunity to change that) is really constraining and problematic.  It would be much better if it raised an exception.  That's what they are for, after all.  Can you imagine if every library decided to force the process to exit on every error instead of raising exceptions?

To unsubscribe from this group and stop receiving emails from it, send an email to rspec+unsubscribe@googlegroups.com.

To post to this group, send email to rs...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/rspec/0fbdde5e-ebcb-4686-9a95-b027f1ea4530%40googlegroups.com.