Which one is better SpecFlow or Fitnesse?

[Ravi Yadav]

I am doing a comparison between Fitnesse and SpecFlow for my project. Lately I have heard a lot of things about Fitnesse. Can you please help me if I have covered all the necessary features for SpecFlow in order to make an informed decision about the recommendation of the tool.

Here are some good things about Fitnesse:

1. I have heard that Fitnesse is more then just a BDD tool and as per my evaluation it does provide a lot of functionality. FitNesse is quite a bit more flexible in my view.

2. It is a wiki based tool so it is platform independent. Can support C#,C++,Java and quite a few others. Do not need IDE to run the tests.

3. Can edit test in any IDE as they are just text files. Or if you do not want to use IDE then you can edit tests in any text editor. Testers or business can access it alike.

4. It eases collaboration between its users.

5. It also acts as a test management tool.

6. Its table definition type test format provides information leaves no grey areas. Infact while trying to develop the decision tables I have found many disconnects between the team members which are escalated and resolved during decision table development.

Bad things about Fitnesse:

1. Haven't found support for regex in method names when mapping them to cell names(called steps in SpecFlow)

2. Inability to generate skeleton code from fixtures(called Scenarios in specflow)

Good things about SpecFlow:

1. Is like any other BDD tool so most of the users are familiar with it.

2. Generate Skeleton code from the scenarios/

3. Is easier to integrate with a CI(TeamCity)

4. Provide regex support.

5. Lot of support online

6. Easier to learn

7. Test scenario can be in plain text file.

Bad things about SpecFlow:

1. Need IDE to run the tests and some programming knowledge.

Any other feature or a lack of feature which I am missing about Specflow??

Please help.

Regards,

Ravi

This email is confidential.  If you have received this email in error please notify us immediately by return email and delete this email and any attachments.  

VIX accepts no liability for any damage caused by this email or any attachments due to viruses, interference, interception, corruption or unauthorised access.

[Mark Levison]

Ravi - Do you like Apples, Oranges or Strawberries more? Let the team run a series of experiments and decide for themselves. 

In addition also consider RobotFramework, perhaps even Concordian.

The pros and cons of all of these tools are very subjective. Perhaps more important than the mechanics of the tool is how your team likes to practice Spec By Example/BDD/ATDD.

Cheers

Mark

--
You received this message because you are subscribed to the Google Groups "SpecFlow" group.
To unsubscribe from this group and stop receiving emails from it, send an email to specflow+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Mark Levison | 1 (877) 248-8277 | Twitter | LinkedIn | Facebook Certified ScrumMaster Training: Vancouver | Edmonton | Ottawa | Montreal | Toronto Certified Product Owner & Private Training also available ~ Our Training Schedule Agile Pain Relief Consulting | Notes from a Tool User Proud Sponsor of Agile Tour Gatineau Ottawa and Agile Coach Camp Canada

[Umair Chagani]

SpecFlow is not an automation tool. It is a collaboration tool. If your team is not collaborating on user stories and their acceptance criteria then you will get very little benefit from SpecFlow.

[Mark Levison]

Umair agreed but the same is true if Fitnesse and RobotFramework.

Cheers
Mark

[Umair Chagani]

I've never had the need to use those so I can't comment. I simply wanted to inform Ravi that unless the collaboration process is followed, the overhead that SpecFlow and other such frameworks require is not worth it.

[Ravi Yadav]

Thanks for your comments Mark and Uchagani.

I can see that Specflow(and other BDD tools similar to specflow) and Fitnesse are very similar in what they do. There is almost nothing which can be achieved from Specflow and not with Fitnesse(except the regex support and skeleton code-gen). This has made it hard for me to make a decision regarding the tool. I come from Specflow background and now since I have also worked with Fitnesse, I know that Fitnesse has a slight upper hand in the features which it provides. But the problem is that Fitnesse is a tool based on a completly different idea and this makes it hard to learn and master, also not to forget the support and documentation for fitnesse is very sparse.

Whereas SpecFlow does lack in a feature or two when compared with Fitnesse but the learning curve is not that steep and you can find plenty of support online.

So I guess it is a tool's usability vs its feature-rich capabilities comparison.

Also we are introducing these tools in our project because we want our testers to be more involved in the process and gradually reach a stage where they are comfortable in coding the tests themselves(using a common automation framework library developed in house). So not sure if any of these two tools provide any kind of greater advantage on that front.

Mark we are fixated on Fitnesse and BDD(SpecFlow/JBehave) at the moment. I do not think we will be adding any other tool to the mix at this stage. Though I have had a look at Robot Framework looks like  a mix of Fitnesse and Specflow(I might be wrong).

Regards,

Ravi

[Mark Levison]

Ravi - clearly you have a deeper understanding of BDD style than I realized when I read your original email. That makes me lean even more strongly run a bake off in your team(s). Get them to choose the tool by spending a couple of Sprints trying each and then they will discover which style of tool better meets their needs. 

BTW my personal experience with FitNesse is the C# version is tough to setup, I tried and failed once.

Cheers

Mark

[Tim Murphy]

Ravi

You do not need an IDE to run the SpecFlow tests. The test framework that SpecFlow generates test code for will a command line option.

Tim

Sent from phone.

[Ravi Yadav]

Hi Tim,

Thankyou for your comment. 

Yes you are right specflow does not require essentially an ide to run.

The idea was to compare which one was easier for non-technical(product owners)/semi-technical(testers) users to run tests in. 

Options to run SpecFlow tests are:

1. Command Prompt

2. IDE

Where as for Fitnesse the options are:

1. wiki

2. IDE

I think wiki is more friendly and something with which the users can relate to very easily.

Regards,

Ravi

[Ravi Yadav]

Mark,

I agree with you that we should do a demo run with both the tools and let the users decide.

Apparently we did exactly that and the decision was to go with Fitnesse. Since our company uses a multitude of different technologies and platforms, we needed a tool which was very flexible. Fitnesse caters for higher collaboration(wiki server), test management, can act as a documentation server and also can act as a scenario(gwt) based tool. 

But the lack of documentation and the issues faced at every stage is forcing us to re-think about our decision. I think it will be important for the users to draft the requirements that will define what they require from the the tool. These requirements should be prioritised and then based on this documentation a decision should be made. This company is new to agile and bdd, the decision to use fitnesse was based on the feature comparison with SpecFlow.

Regards,

Ravi