Logging screenshots in Cucumber's HTML format

[Brad Taylor]

Hey all,

Just thought I'd share a handy thing I cooked up today.  We're using Cucumber's HTML output format (cucumber --format html) to show the results of tests in our CI.  Rather than try to gather up the screenshot files and reprocess the HTML file to include them, I'm using cucumber's embed functionality and data: urls to embed them directly in the script output.  Here's the snippet that I used to take a screenshot after every step and embed it:

AfterStep do |scenario|

  # Capture a screenshot and embed it into the step

  res = http({:method => :get, :path => 'screenshot'})

  image_data = Base64.encode64(res).gsub("\n", "")

  embed("data:image/png;base64,#{image_data}", "image/png")

end

You can put this in your support/hooks.rb file.

Hope this is helpful!

Best,

-Brad

[Nicholas Albion]

Thanks Brad,

I've updated the source for calabash-android to support screenshots in Scenario Outlines:

https://github.com/nalbion/calabash-android/commit/ece8b6f2f1267d632ac2654cd1c17287de9718dc 

...I don't suppose you know how to have the screenshots scaled down for the report?

[Krukow]

You could use ImageMagick, see:

http://www.imagemagick.org/script/convert.php

[Rakesh]

Brad,

I copy pasted the step in the support/hooks.rb file. But i doen't seems to embedded the picture.

I'm using MAC , Calabash 0.9.80.pre.5. 

Here is the screenshot of the html report.

Command line: DEVICE=ipad OS=ios5 DEVICE_ENDPOINT=http://192.168.29.67:37265 cucumber --format html --out report.html  NO_LAUNCH=1

I can see a screenshot being generated, also see "Saved screenshot: screenshot_5.png" on the report , but i don't see the image.

--Rakesh

[Kavi]

Hi All,

I get the following error

undefined method `force_encoding' for #<String:0x102e64b70> (NoMethodError)

./support/hooks.rb:28:in `AfterStep'
Test.feature:34:in `Given the app is running'

26  raw = scenario.raw_steps[CALABASH_COUNT[:step_index]]
27  CALABASH_COUNT[:step_line] = raw.line unless raw.nil?
28    res = http({:method => :get, :path => 'screenshot'})
29    image_data = Base64.encode64(res).gsub("\n", "")
30    embed("data:image/png;base64,#{image_data}", "image/png")
31# gem install syntax to get syntax highlighting

What could be the problem???

[Karl Krukow]

Sorry I made a bugfix that worked for Ruby 1.9  only. 

https://github.com/calabash/calabash-ios/issues/55

Please try updating to 0.9.96

https://github.com/calabash/calabash-ios/wiki/B1-Updating-your-Calabash-iOS-version

[Kavi]

Hi Karl,

After the update i am not able to run multiple scenarios with html formatting. Getting the error as below

class or module required (TypeError)

/Library/Ruby/Gems/1.8/gems/builder-3.0.0/lib/builder/xmlbase.rb:43:in `kind_of?'

/Library/Ruby/Gems/1.8/gems/builder-3.0.0/lib/builder/xmlbase.rb:43:in `method_missing'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/formatter/html.rb:467:in `build_step'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/formatter/html.rb:264:in `step_name'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/tree_walker.rb:173:in `__send__'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/tree_walker.rb:173:in `send_to_all'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/tree_walker.rb:171:in `each'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/tree_walker.rb:171:in `send_to_all'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/tree_walker.rb:166:in `broadcast'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/tree_walker.rb:112:in `visit_step_name'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/tree_walker.rb:105:in `visit_step_result'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/tree_walker.rb:163:in `broadcast'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/tree_walker.rb:104:in `visit_step_result'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/step_invocation.rb:43:in `visit_step_result'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/step_invocation.rb:39:in `accept'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/tree_walker.rb:99:in `visit_step'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/tree_walker.rb:163:in `broadcast'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/tree_walker.rb:98:in `visit_step'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/step_collection.rb:15:in `accept'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/step_collection.rb:14:in `each'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/step_collection.rb:14:in `accept'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/tree_walker.rb:93:in `visit_steps'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/tree_walker.rb:163:in `broadcast'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/tree_walker.rb:92:in `visit_steps'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/scenario.rb:55:in `accept'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/runtime.rb:80:in `with_hooks'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/runtime.rb:96:in `before_and_after'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/runtime.rb:79:in `with_hooks'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/runtime/support_code.rb:120:in `call'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/runtime/support_code.rb:120:in `around'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/language_support/language_methods.rb:9:in `around'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/language_support/language_methods.rb:91:in `call'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/language_support/language_methods.rb:91:in `execute_around'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/language_support/language_methods.rb:8:in `around'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/runtime/support_code.rb:119:in `around'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/runtime/support_code.rb:117:in `call'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/runtime/support_code.rb:117:in `around'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/runtime.rb:91:in `around'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/runtime.rb:78:in `with_hooks'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/scenario.rb:53:in `accept'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/scenario.rb:108:in `with_visitor'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/scenario.rb:47:in `accept'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/tree_walker.rb:51:in `visit_feature_element'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/tree_walker.rb:163:in `broadcast'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/tree_walker.rb:50:in `visit_feature_element'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/feature.rb:43:in `accept'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/feature.rb:42:in `each'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/feature.rb:42:in `accept'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/tree_walker.rb:20:in `visit_feature'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/tree_walker.rb:163:in `broadcast'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/tree_walker.rb:19:in `visit_feature'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/features.rb:29:in `accept'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/features.rb:17:in `each'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/features.rb:17:in `each'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/features.rb:28:in `accept'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/tree_walker.rb:14:in `visit_features'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/tree_walker.rb:163:in `broadcast'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/ast/tree_walker.rb:13:in `visit_features'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/runtime.rb:46:in `run!'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/cli/main.rb:43:in `execute!'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/../lib/cucumber/cli/main.rb:20:in `execute'

/Library/Ruby/Gems/1.8/gems/cucumber-1.2.1/bin/cucumber:14

/usr/bin/cucumber:23:in `load'

/usr/bin/cucumber:23

What could be the reason???

[Karl Krukow]

I don't have this problem locally and this stacktrace doesn't have Calabash in it. So  it is hard for me to see what is wrong.

Please check your hooks.

Could you paste your hooks file here?

- Karl

[Kavi]

My Hooks 

Before do |scenario|

  begin

      @scenario_is_outline = (scenario.class == Cucumber::Ast::OutlineTable::ExampleRow) 

    CALABASH_COUNT[:step_index] = 0

      

      if @scenario_is_outline 

          scenario = scenario.scenario_outline 

      end

      CALABASH_COUNT[:step_line] = scenario.raw_steps[CALABASH_COUNT[:step_index]].line

  rescue Exception => e

    puts "#{Time.now} - Exception:#{e}"

  end

end

AfterStep do |scenario|

    CALABASH_COUNT[:step_index] = CALABASH_COUNT[:step_index] + 1

    

    if @scenario_is_outline 

        scenario = scenario.scenario_outline 

    end 

    

    raw = scenario.raw_steps[CALABASH_COUNT[:step_index]]

    CALABASH_COUNT[:step_line] = raw.line unless raw.nil?

    res = http({:method => :get, :path => 'screenshot'})

    image_data = Base64.encode64(res).gsub("\n", "")

    embed("data:image/png;base64,#{image_data}", "image/png")

end

[Karl Krukow]

I'm sorry,  I am unable to reproduce this problem.

I've just run with your hooks and calabash-cucumber version 0.9.96

cucumber --format html --out a.html

and it works just fine.

Could you create a small sample project that I can access that reproduces the problem?

- Karl

[Kavi]

Are you able to run multiple scenarios in a single feature file??? It was working fine all these days. The problem araised once i updated Cucumber to version : 1.2.1 and calabash-ios to 0.9.96. Could this be because of cucumber update??

[Kavi]

The tests are working fine without formatting :(

[Karl Krukow]

Yes, I've run multiple scenarios with your hooks and html formatter

krukow:~/github/talk-with-simple-test$ cucumber --version

1.2.1

krukow:~/github/talk-with-simple-test$ cucumber --format html --out a.html

/Users/krukow/github/talk-with-simple-test/features/support/hooks.rb:1: warning: already initialized constant CALABASH_COUNT

-------------------------------------

Auto detected APP_BUNDLE_PATH:

APP_BUNDLE_PATH=/Users/krukow/Library/Developer/Xcode/DerivedData/LPSimpleExample-dmfxejwasmrjgzgkdqccfrlwwgbm/Build/Products/Debug-iphonesimulator/LPSimpleExample-cal.app

Please verify!

If this is wrong please set it as APP_BUNDLE_PATH in features/support/launch.rb

...

gives a nice html report with screenshots

[Kavi]

Karl,

After updating the calabash ios to 0.9.96 i didnt update the framework file in the application build. Could that be the problem? I guess its running 0.9.80 framework

[Karl Krukow]

I'm not entirely sure if this could be the cause, but you have to update in either case.

Each release of the calabash-cucumber gem is matched by a particular version of the calabash.framework, this is why you should always follow the recommended update procedure closely:

https://github.com/calabash/calabash-ios/wiki/B1-Updating-your-Calabash-iOS-version

[Kavi]

Identified the problem :) I had a argument in the step definition as Symbol. Its a constant defined in  Builder gem and I Modified it

[Rakesh]

Is there a way to get a screenshot when a test fails as well? 

Thanks,

Rakesh

[Krukow]

On Tuesday, August 28, 2012 12:57:16 AM UTC+2, Rakesh wrote:

Is there a way to get a screenshot when a test fails as well? 

Usually you do this using the "screenshot_and_raise" method. Alternatively, you could do it in an After hook. 

[Kavi]

Hi Karl,

I am trying to embed screenshot only for the step which resulted in the failure of the scenario. How can this be done?  I Tried 

 if(scenario.failed?)

        res = http({:method => :get, :path => 'screenshot'})

        image_data = Base64.encode64(res).gsub("\n\n", "")

        embed("data:image/png;base64,#{image_data}", "image/png")

 end

But it embeds screenshots for all the steps??? how o achieve my requirement???

Thanks in advance

[Krukow]

The latest version of Calabash supports screenshots much better (the cucumber way!).

I would recommend that you upgrade to the latest version of Calabash iOS:

https://github.com/calabash/calabash-ios/wiki/B1-Updating-your-Calabash-iOS-version

After upgrading, you should delete the file features/support/hooks.rb unless there is more code that what you show below.

Then Calabash takes a screenshot by default for most step failures.

/Karl

[Rama Sadineni]

Hi Karl

I am using the latest calabash-cucumber gem 0.9.167.  And wanted to generate  reports in a folder called reports.  screen shots are generated and link showing in report as well, but they don't open from report.  If I just use screenshot_embed, then i can open the attachments.  

Couldn't understand why they are not opening when i use naming convention, could you please help.

I am using this code in launch.rb file

After do |scenario|

    if scenario.failed?

       screenshot_embed(:prefix => "./reports/", :name => "sc_#{Time.now}")

end

Thanks

Rama