Hiding backtraces and using symlinks

62 views
Skip to first unread message

Cliff Rosson

unread,
Dec 31, 2015, 12:52:45 AM12/31/15
to rspec
Hi Folk,

Hopefully this is a very simple question. I use RSPEC as a functional test for network maintenances. It allows me to loop a series of tests during a maintenance providing a clean format and notification of any behavioral changes on the network. 

Often times I expect some things to fail and in an attempt to keep results clean I often want to hide the full backtrace. If I run my rspec with the config.full_backtrace argument set to false everything works great. If I symlink the file however the backtraces return.

I wrote this example to illustrate my issue.
#!/usr/bin/ruby
require 'rspec/autorun'
require 'net/ping'


RSpec.configure do |config|
  config.full_backtrace=false
end

describe "TEST A >> " do
  it "1:Ping is true >> " do 
    host = Net::Ping::External.new("www.google.com")
    host.ping?.should be_false
  end
end

if I run this directly it works great.

/test_rspec.rb

F


Failures:


  1) TEST A >>  1:Ping is true >> 

     Failure/Error: Unable to find matching line from backtrace

       expected: false value

            got: true

     # ./test_rspec.rb:13:in `block (2 levels) in <main>'


Finished in 0.01479 seconds

1 example, 1 failure


Failed examples:


rspec ./test_rspec.rb:11 # TEST A >>  1:Ping is true >>


Nice and clean output.
However if I symlink the file I get the full backtrace which I really just would rather ignore.

ls -l ~/bin/

total 24

lrwxr-xr-x  1 cliff.rosson  USERS\Domain Users  49 Dec 30 16:56 test_rspec -> /Users/cliff.rosson/Desktop/scratch/test_rspec.rb



And run the symlink

test_rspec 

F


Failures:


  1) TEST A >>  1:Ping is true >> 

     Failure/Error: host.ping?.should be_false

       expected: false value

            got: true

     # /Library/Ruby/Gems/2.0.0/gems/rspec-expectations-2.14.5/lib/rspec/expectations/fail_with.rb:32:in `fail_with'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-expectations-2.14.5/lib/rspec/expectations/handler.rb:36:in `handle_matcher'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-expectations-2.14.5/lib/rspec/expectations/syntax.rb:53:in `should'

     # /Users/cliff.rosson/bin/test_rspec:13:in `block (2 levels) in <main>'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-2.14.8/lib/rspec/core/example.rb:114:in `instance_eval'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-2.14.8/lib/rspec/core/example.rb:114:in `block in run'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-2.14.8/lib/rspec/core/example.rb:254:in `with_around_each_hooks'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-2.14.8/lib/rspec/core/example.rb:111:in `run'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-2.14.8/lib/rspec/core/example_group.rb:390:in `block in run_examples'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-2.14.8/lib/rspec/core/example_group.rb:386:in `map'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-2.14.8/lib/rspec/core/example_group.rb:386:in `run_examples'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-2.14.8/lib/rspec/core/example_group.rb:371:in `run'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-2.14.8/lib/rspec/core/command_line.rb:28:in `block (2 levels) in run'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-2.14.8/lib/rspec/core/command_line.rb:28:in `map'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-2.14.8/lib/rspec/core/command_line.rb:28:in `block in run'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-2.14.8/lib/rspec/core/reporter.rb:58:in `report'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-2.14.8/lib/rspec/core/command_line.rb:25:in `run'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-2.14.8/lib/rspec/core/runner.rb:80:in `run'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-2.14.8/lib/rspec/core/runner.rb:17:in `block in autorun'


Finished in 0.01452 seconds

1 example, 1 failure


Failed examples:


rspec /Users/cliff.rosson/bin/test_rspec:11 # TEST A >>  1:Ping is true >>

 
Any-thoughts why this setting doesn't carry over when referencing the symlink?

Thanks everyone.

Cliff Rosson

unread,
Dec 31, 2015, 12:52:41 PM12/31/15
to rspec
Versions.

rspec (2.14.1)

rspec-core (2.14.8)

rspec-expectations (2.14.5)

rspec-mocks (2.14.6)

Jon Rowe

unread,
Dec 31, 2015, 7:08:09 PM12/31/15
to rs...@googlegroups.com
Full backtrace is actually designed to force a full backtrace display, turning it off merely uses the default inclusion / exclusion filters, by default (if I remember correctly) this includes stuff based on your working directory; so if you manually add the directory (even if you use ruby to generate it from the file rather than the working directory) to the exclusion filter it should be removed no matter where you run the test from.

There’s been a fair amount of work on improving this behaviour in RSpec 3 and given that RSpec 2 isn’t supported I heartily recommend you upgrade to the latest version, if you install 2.99 first it will be relatively painless!

Cheers
Jon

Jon Rowe
---------------------------

--
You received this message because you are subscribed to the Google Groups "rspec" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rspec+un...@googlegroups.com.
To post to this group, send email to rs...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/rspec/62b49200-336a-4f42-8537-a82d536ecf8b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Cliff Rosson

unread,
Dec 31, 2015, 7:51:18 PM12/31/15
to rspec
Thanks Jon. I'll give it a shot. I ran into some semantic issues when rspec released 3.0 some time ago but in reading the docs nothing really stands out on why rspec3 wouldn't work for me.

Myron Marston

unread,
Dec 31, 2015, 7:52:42 PM12/31/15
to rs...@googlegroups.com
Our website has detailed upgrade instructions:


Besides upgrading to 2.99 first, the other important thing is using transpec.

HTH,
Myron

To view this discussion on the web visit https://groups.google.com/d/msgid/rspec/4f6e5a17-e808-4f4c-8cfd-f734c9bed45a%40googlegroups.com.

Cliff Rosson

unread,
Dec 31, 2015, 8:07:31 PM12/31/15
to rspec
Thanks everyone. So I just made some changes to my example file and tested again. I still have the same issue on rspec3. Any thoughts? 

#!/usr/bin/ruby
require 'rspec/autorun'
require 'net/ping'


RSpec.configure do |config|
  config.full_backtrace=false
end

describe "TEST A >> " do
  it "1:Ping is true >> " do 
    host = Net::Ping::External.new("www.google.com")
    expect(host.ping?).to be false
  end
end

If I call the file directly I get no traceback. If I call the symlink I do. If I call the symlink from my bin directory, where it exists, then I don't.

Cliffords-iMac:Desktop crosson$ /Users/crosson/Desktop/scratch/example.rb

F


Failures:


  1) TEST A >>  1:Ping is true >> 

     Failure/Error: expect(host.ping?).to be false


       expected false

            got true

     # ./scratch/example.rb:13:in `block (2 levels) in <main>'


Finished in 0.03627 seconds (files took 0.14898 seconds to load)

1 example, 1 failure


Failed examples:


rspec ./scratch/example.rb:11 # TEST A >>  1:Ping is true >> 


Cliffords-iMac:Desktop crosson$ example 

F


Failures:


  1) TEST A >>  1:Ping is true >> 

     Failure/Error: expect(host.ping?).to be false


       expected false

            got true

     # /Library/Ruby/Gems/2.0.0/gems/rspec-support-3.4.1/lib/rspec/support.rb:87:in `block in <module:Support>'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-support-3.4.1/lib/rspec/support.rb:96:in `call'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-support-3.4.1/lib/rspec/support.rb:96:in `notify_failure'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-expectations-3.4.0/lib/rspec/expectations/fail_with.rb:27:in `fail_with'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-expectations-3.4.0/lib/rspec/expectations/handler.rb:40:in `handle_failure'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-expectations-3.4.0/lib/rspec/expectations/handler.rb:50:in `block in handle_matcher'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-expectations-3.4.0/lib/rspec/expectations/handler.rb:27:in `with_matcher'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-expectations-3.4.0/lib/rspec/expectations/handler.rb:48:in `handle_matcher'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-expectations-3.4.0/lib/rspec/expectations/expectation_target.rb:54:in `to'

     # /Users/crosson/bin/example:13:in `block (2 levels) in <main>'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-3.4.1/lib/rspec/core/example.rb:236:in `instance_exec'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-3.4.1/lib/rspec/core/example.rb:236:in `block in run'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-3.4.1/lib/rspec/core/example.rb:477:in `block in with_around_and_singleton_context_hooks'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-3.4.1/lib/rspec/core/example.rb:434:in `block in with_around_example_hooks'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-3.4.1/lib/rspec/core/hooks.rb:478:in `block in run'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-3.4.1/lib/rspec/core/hooks.rb:616:in `run_around_example_hooks_for'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-3.4.1/lib/rspec/core/hooks.rb:478:in `run'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-3.4.1/lib/rspec/core/example.rb:434:in `with_around_example_hooks'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-3.4.1/lib/rspec/core/example.rb:477:in `with_around_and_singleton_context_hooks'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-3.4.1/lib/rspec/core/example.rb:233:in `run'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-3.4.1/lib/rspec/core/example_group.rb:581:in `block in run_examples'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-3.4.1/lib/rspec/core/example_group.rb:577:in `map'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-3.4.1/lib/rspec/core/example_group.rb:577:in `run_examples'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-3.4.1/lib/rspec/core/example_group.rb:543:in `run'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-3.4.1/lib/rspec/core/runner.rb:115:in `block (3 levels) in run_specs'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-3.4.1/lib/rspec/core/runner.rb:115:in `map'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-3.4.1/lib/rspec/core/runner.rb:115:in `block (2 levels) in run_specs'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-3.4.1/lib/rspec/core/configuration.rb:1680:in `with_suite_hooks'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-3.4.1/lib/rspec/core/runner.rb:114:in `block in run_specs'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-3.4.1/lib/rspec/core/reporter.rb:77:in `report'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-3.4.1/lib/rspec/core/runner.rb:113:in `run_specs'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-3.4.1/lib/rspec/core/runner.rb:89:in `run'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-3.4.1/lib/rspec/core/runner.rb:73:in `run'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-3.4.1/lib/rspec/core/runner.rb:41:in `invoke'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-3.4.1/lib/rspec/core/runner.rb:34:in `perform_at_exit'

     # /Library/Ruby/Gems/2.0.0/gems/rspec-core-3.4.1/lib/rspec/core/runner.rb:20:in `block in autorun'

     # 

     #   Showing full backtrace because every line was filtered out.

     #   See docs for RSpec::Configuration#backtrace_exclusion_patterns and

     #   RSpec::Configuration#backtrace_inclusion_patterns for more information.


Finished in 0.03689 seconds (files took 0.15154 seconds to load)

1 example, 1 failure


Failed examples:


rspec /Users/crosson/bin/example:11 # TEST A >>  1:Ping is true >> 


Cliffords-iMac:Desktop crosson$ ls ~/bin/

example

Cliffords-iMac:Desktop crosson$ ls -l ~/bin/

total 8

lrwxr-xr-x  1 crosson  staff  41 Dec 31 16:59 example -> /Users/crosson/Desktop/scratch/example.rb




Cliffords-iMac:Desktop crosson$ cd ~/bin/

Cliffords-iMac:bin crosson$ example 

F


Failures:


  1) TEST A >>  1:Ping is true >> 

     Failure/Error: expect(host.ping?).to be false


       expected false

            got true

     # ./example:13:in `block (2 levels) in <main>'


Finished in 0.03574 seconds (files took 0.16152 seconds to load)

1 example, 1 failure


Failed examples:


rspec ./example:11 # TEST A >>  1:Ping is true >>  

Cliff Rosson

unread,
Dec 31, 2015, 8:11:10 PM12/31/15
to rspec
Re-reading Jon's post. I guess it only excludes the directory it runs from? I'd need to add each directory I am working in? 

Just to spell it out, sorry if I am being dense, can I just have ruby read the directory I am currently in then add that folder to the exclusion list?

Jon Rowe

unread,
Dec 31, 2015, 8:40:47 PM12/31/15
to rs...@googlegroups.com
>  I guess it only excludes the directory it runs from? 

It bases patterns from that yes.

> I'd need to add each directory I am working in? 

No, you need to add the directory the spec files are in. You can get that with `File.expand_path(__FILE__, ’/..’)` 

Jon Rowe
---------------------------

Cliff Rosson

unread,
Dec 31, 2015, 8:56:37 PM12/31/15
to rs...@googlegroups.com
Something else is going on. Adding the folder in the exclusion pattern doesn't help especially since in the backtrace my working directory isn't even whats shown (observed above).

I would expect the same behavior when calling the symlink as I would when calling the file directly. Clearly when calling the file directly it is working as expected. What else might be going on? 

I was able to easily reproduce this with the above ./example.rb file. Creating a symlink and calling the symlink.


--
You received this message because you are subscribed to a topic in the Google Groups "rspec" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/rspec/RwkPXk3y7vE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to rspec+un...@googlegroups.com.

To post to this group, send email to rs...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/rspec/0064257E07314AA7B36E1EB82257017C%40jonrowe.co.uk.

For more options, visit https://groups.google.com/d/optout.



--
- Cliff Rosson

Myron Marston

unread,
Dec 31, 2015, 8:57:59 PM12/31/15
to rs...@googlegroups.com
Cliff, can you put together a reproducible example and open an issue on the rspec-core issue tracker?  That would be the easiest way for us to dig into this.

Thanks,
Myron

To view this discussion on the web visit https://groups.google.com/d/msgid/rspec/CAKST4vw9hT40C1wg5HvK1SZhDuUrEOTQsnE6tMezzajJEZTvsA%40mail.gmail.com.

Cliff Rosson

unread,
Dec 31, 2015, 9:17:01 PM12/31/15
to rs...@googlegroups.com
Will do. Starting the new years festivities here soon so tomorrow I'll throw something together. I wonder if the line, "Showing full backtrace because every line was filtered out", is the key. Perhaps everything is getting excluded when I am in another directory. Maybe I need to include working and file directories. Maybe that is what Jon was referring to. I just kept trying to exclude exclude exclude! :)

Tomorrow I'll test this once more. If my final attempts at figuring this out fails I'll and post something up on the issue tracker. Thank you everyone.


To view this discussion on the web visit https://groups.google.com/d/msgid/rspec/CADUxQmvK5Bq9kRaAQaP3EOCudGFt9ZsXGHwCP%2BAnjoLXy50Svg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.



--
- Cliff Rosson

Cliff Rosson

unread,
Dec 31, 2015, 9:22:16 PM12/31/15
to rs...@googlegroups.com
Ha yea ok that did the job!

RSpec.configure do |config|
  config.full_backtrace=false
  path = File.expand_path(__FILE__)
  config.backtrace_inclusion_patterns << /#{path}/
end

Thank you Jon and Myron. 

The symlink is in my BIN folder and I can call it from anywhere. No backtrace issues. 
--
- Cliff Rosson
Reply all
Reply to author
Forward
0 new messages