Customising reports

[michaelw]

Hi Mehdi

I've been meaning to write for awhile just to say how much I'm
enjoying using bddify. I've used StoryQ quite a bit, and really liked
it, but I prefer bddify now because of the extra flexibility it
provides. In fact, one of the biggest challenges has been choosing the
best way to write tests with the multiple styles bddify offers.

I have a question regarding customising the report. I want to get rid
of the stack trace for not implemented methods and just display the
icon. What's the best way to go about doing that?

Thanks
Michael

[Mehdi Khalili]

Hi Micheal,

Thanks for the nice comments. I am glad to hear you like the
framework. I guess you sent me an email a while back about the
framework. Is that right? That email helped me find a nasty bug in a
very specific corner case. So thanks for that.

With regards to customising the report, in V0.7 I externalized the
report styling and the css were moved into a file called bddify.css
which is saved next to the html report. If you want to customize the
report the best way would be to change the css file and overwrite it
after each test run (as part of your build script). I just did a very
quick try and it worked for me. I changed

div.FailedException,
div.NotImplementedException,
div.InconclusiveException
{
padding:5px 0px 3px 50px;
}

with

div.FailedException,
div.InconclusiveException
{
padding:5px 0px 3px 50px;
}

div.NotImplementedException
{
display:None;
}

If this does not do what you want, please let me know what is not
working so I can provide a better answer.

With regards to best way to use the framework I get quite a lot of
questions from developers who start using it. I tried to make the
framework very simple; but the number of available options make it a
bit confusing :(

So feel free to post your questions and code snippets here and I will
try to respond in a reasonable time.

Hope it helps.
Mehdi

[michaelw]

Hi Mehdi

Yes, I sent you an email awhile back about the framework. I'm glad it
helped!

I wouldn't worry about the number of options making it confusing to
get started with. I would rather have too many options than not
enough. You have done a lot of documentation which is really helpful.
Though I did struggle to see how to plug new listeners in via the
config that you mention in the 0.8 release notes.

Thanks for your CSS solution for the report. It did solve the problem.
I suppose I could put a post-build step in to copy the CSS file over.
What would be nice would be to be able to have the CSS file in the
test project and be able to tweak it as necessary. It would also be
good to be able to set the HtmlReport to a different Razor file.

I pulled the source project into my solution and tweaked the Razor
HtmlReport to not display the exception message and stack trace if
typeof(NotImplementedException) and felt the report was much cleaner
(I think this is how your original reports were and how StoryQ
reports). I also tried making the HtmlReportTraceListener CssFile
field public and was able to set it to a different CSS file and run
it, but the HtmlReport was still set to bddify.css so that wasn't much
help.

Bddify.Reporters.HtmlReportTraceListener.CssFile = new Lazy<string>(()
=> "Custombddify.css");
new my_scenario()
.Bddify<my_story>("Scenario 1...");

Interested to hear what you think.

Thanks
Michael

[Mehdi Khalili]

Hi Michael,

Here is a few good news for you. I just made some changes on the code
that you can download (not released yet):
- Now if bddify.css file exists next to the report file, bddify will
not export it again. This means that you can have a css file in your
test project that you copy to output folder and bddify uses that
instead of overwriting it. Of course the file should be called
'bddify.css' which I do not think is a big problem.
- I was quite uncertain about whether I should show the exceptions on
NotImplemented and Inconclusive steps or not. I like cleaner reports
too; so I changed it to the way it was before. Now only failed steps
show the exception.

Setting the css file on the listener through a public field has a few
issues:
1. Bddify resolves the trace listeners from config file (and defaults
if there is no config file). This means that it may not use the html
trace listener at runtime. So you take dependency on an inner working
that may not even be used at runtime.
2. You do not have a single place to set that public field which means
you will have to set that after every single test. To avoid code
duplication you can write your own extension method that does that;
but it would not be a clean solution. Just my personal preference, of
course.

I hope this helps.

Thanks for the feedback.

Cheesr,
Mehdi

[michaelw]

Hey Mehdi

Thanks, that's awesome. That's just how I would expect the report to
look. I compiled the latest source and copied the new DLL over the
Bddify DLL in my solution's nuget package, which my test project
picked up.

Our workflow is to put all our scenarios in as acceptance tests at the
start of the sprint as not implemented, and then see them go green as
the sprint progresses. I like this report change for a couple of
reasons. Firstly, it's aesthetically nicer to have the icons
differentiate the states and not to have lots of red stack traces.
More importantly though, I think conceptually in acceptance testing
Not Implemented is different from an exception, even though an
exception is used as the technical mechanism to derive it, and I think
it is important to distinguish the two.

I don't really need to change the CSS any more, but I figured you
might appreciate the feedback, so I copied bddify.css into the root of
my test project, and set the "Copy to Output" property to "Copy
Always." I made a couple of tweaks to the CSS and they showed up in
the report as expected. I like what you've done with the reports in
the last couple of releases, externalising the CSS, making them more
extensible, and being able to categorise reports/change their name. I
think it helps making the decision to adopt a framework if you know
that extensibility is there should you need it.

I've missed the info about bddify's use of a config file and how to
plug in new listeners. Is that documented somewhere or could you
explain that a bit more please?

Thanks again for implementing the changes.

Michael

[Mehdi Khalili]

Hi Michael,

It is good to know it is working. I am with you with regards to red/
yellow/green states of tests.

With regards to tracelistener, I have not documented anything about
them. However, they work exactly as you would expect .net
tracelistener work and the trace source name you subscribe to is
'Bddify.Reporter'. I will write this up somewhere. For now, it should
look something like (not tested):

<configuration>
<system.diagnostics>
<sources>
<source name="Bddify.Reported" switchValue="Information">
<listeners>
<add name="traceListener"
type="YourTraceListenerGoesHere" />
</listeners>
</source>
</sources>
</system.diagnostics>
</configuration>

please note that as soon as you register a listener Bddify drops its
convention and trusts your configuration. So you may have to add back
the console and html tracelisteners.

I will release these changes hopefully next week.

Thanks for the feedback.

Cheers,
Mehdi