[rspec-users] return code question

[Ben Fyvie]

We are trying to automate the running of our rspec tests for our Rails app on a build server using Capistrano.  The problem is that Capistrano seems to think that the command called to run the model tests failed when in fact I believe it succeeded. I believe the problem has to do with the return code from the command I am running.. The command I am running is:

ruby script/spec spec/models -f html:model_test_results.html

The output file seems to be complete as it has both the opening and closing html tags and it has the results of all of our tests. But when I run $? To get the return code of the last command it says it was “1” which I believe signifies that the command was not successful and is what is causing our Capistrano task to abort.  We are using RSpec 1.1.4 and in case you’re curious the output from Capistrano looks like this:

 

  * executing "cd /data/nightingalenotes/releases/20080812160703 && ruby script/

spec spec/models -f html:public/rspec_test_results/model_test_results.html"

    servers: ["buildnexus.champ.net"]

    [buildnexus.champ.net] executing command

    command finished

*** [deploy:update_code] rolling back

  * executing "rm -rf /data/nightingalenotes/releases/20080812160703; true"

    servers: ["buildnexus.champ.net"]

    [buildnexus.champ.net] executing command

    command finished

command "cd /data/nightingalenotes/releases/20080812160703 && ruby script/spec s

pec/models -f html:public/rspec_test_results/model_test_results.html" failed on

buildnexus.champ.net

 

 

Thanks!

 

Ben Fyvie

 

[David Chelimsky]

On Tue, Aug 12, 2008 at 1:22 PM, Ben Fyvie <ben....@champsoftware.com> wrote:
> We are trying to automate the running of our rspec tests for our Rails app
> on a build server using Capistrano. The problem is that Capistrano seems to
> think that the command called to run the model tests failed when in fact I
> believe it succeeded. I believe the problem has to do with the return code
> from the command I am running.. The command I am running is:
>
> ruby script/spec spec/models -f html:model_test_results.html
>
>
>
> The output file seems to be complete as it has both the opening and closing
> html tags and it has the results of all of our tests. But when I run $? To
> get the return code of the last command it says it was "1" which I believe
> signifies that the command was not successful and is what is causing our
> Capistrano task to abort.

I just did this in a Rakefile:

task :models do
`ruby script/spec spec/models -f html:model_test_results.html`
puts $?
end

And I get this when I run 'rake models'

$ rake models
(in /Users/david/projects/ruby/mycastory)
0

I'm using the latest from git (naturally).

How are you checking the exit code?

> We are using RSpec 1.1.4 and in case you're
> curious the output from Capistrano looks like this:
>
>
>
> * executing "cd /data/nightingalenotes/releases/20080812160703 && ruby
> script/
>
> spec spec/models -f html:public/rspec_test_results/model_test_results.html"
>
> servers: ["buildnexus.champ.net"]
>
> [buildnexus.champ.net] executing command
>
> command finished
>
> *** [deploy:update_code] rolling back
>
> * executing "rm -rf /data/nightingalenotes/releases/20080812160703; true"
>
> servers: ["buildnexus.champ.net"]
>
> [buildnexus.champ.net] executing command
>
> command finished
>
> command "cd /data/nightingalenotes/releases/20080812160703 && ruby
> script/spec s
>
> pec/models -f html:public/rspec_test_results/model_test_results.html" failed
> on
>
> buildnexus.champ.net
>
>
>
>
>
> Thanks!
>
>
>
> Ben Fyvie
>
>
>

> _______________________________________________
> rspec-users mailing list
> rspec...@rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users
>
_______________________________________________
rspec-users mailing list
rspec...@rubyforge.org
http://rubyforge.org/mailman/listinfo/rspec-users

[Ben Fyvie]

The command I was running to get the return code was "echo $?" After reading
over my email it seems I left out a bit of rather important information.

You see we expect the command to return a return code of "0" if the command
was successful in it's task which was to generate an html file with the
results of running the model tests. In our case the command is successful
and it generates the desired output and puts it into the HTML file as
expected. However we find that the return code is changed to "1" if any
single test fails. To me this seems incorrect because the result of a test
should not influence the return code as the return code should be associated
with whether or not the command executed successfully or not.

Our current situation is a great example of why the return code should only
represent the success or failure of the command and not the results of the
command. Whenever a test fails the return code of "1" causes our build to
break and therefore nobody even knows a test is failing until we notice that
the build server is no longer updating on regular intervals.

Ben Fyvie