Junit and automation - some help/inputs needed

[Sushrut Bidwai]

Did not seem like a suitable question for stackoverflow so here it is. 

I have bunch of text input files which provided input as well as expected output for set of programs. One line is one combination of input+expected result (one unit test in a way)

I have written one testcase which reads these files, runs the program and then asserts validity of results. 

Problem is -

When I run the unit test, it stops at first failure, of course. Even though practically it is bunch of unit tests I am writing, code wise it is a single unit test and hence on first failure it halts. How do I get around this? Is there a better way to write this sort of thing? I cant hard code all inputs and expected values as lot of them are created by mechanical turkers. 

I thought about writing a small utility which will generated junit cases ie code , but it did not seems like a correct way. 

Thank in advances for any inputs.

~Sushrut

http://simplycious.com

simplifying shopping

[Ashish Belagali (Gmail)]

Dear Sushrut,

Can you not modify the program to just record the failure and go ahead? (Catch the exception.) At the end of it, you should get a report telling you which passed and which failed. This may be done with JUnit but even without it.

Give me a call if you have further queries on this.

Regards,
/Ashish

--
You received this message because you are subscribed to the Google Groups "Java User Group For Pune, India" group.
To unsubscribe from this group and stop receiving emails from it, send an email to javapune+u...@googlegroups.com.
To post to this group, send email to java...@googlegroups.com.
Visit this group at http://groups.google.com/group/javapune?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
  http://www.communication-tracker.com - Organize communications centrally!
  http://www.headtracker.in - Recruit efficiently!
  http://www.acism.com - Outsource your software development!

[Tushar Joshi]

Hi Sushrut,

On Thu, Jan 31, 2013 at 10:52 PM, Sushrut Bidwai <bidwai....@gmail.com> wrote:

I have written one testcase which reads these files, runs the program and then asserts validity of results. 

Your use case is an exact use case for using Fitnesse framework http://fitnesse.org/
Please go through the documentation of this tool and it will be useful for you

--
with regards
    Tushar


Tushar Joshi, Nagpur: http://www.tusharvjoshi.com

* MCSD_NET C#, SCJP, RHCE, ZCE, PMP, CSM

* Architect: http://www.persistentsys.com
* NetBeans Dream Team Member: http://dreamteam.netbeans.org
* DZone Most Valued Blogger: http://netbeanside61.blogspot.com
* Java User Group Nagpur Founder and Leader: http://www.jugnagpur.com

* LinkedIn: http://www.linkedin.com/in/tusharvjoshi

[Ashish Belagali (Gmail)]

Tushar,

Since he has already taken the effort to manually read the file line-by-line, it may not be worth migrating to fitnesse. But Fitnesse will be perfect for this. I agree.

Best,
/Ashish

[dexterous]

On Thursday, January 31, 2013 10:52:34 PM UTC+5:30, Sushrut Bidwai wrote:

I thought about writing a small utility which will generated junit cases ie code , but it did not seems like a correct way.

https://github.com/KentBeck/junit/wiki/Parameterized-tests

Since you are using JUnit and the essential test instrumentation code is same for all cases, you might want to look at Parameterized tests. That approach seems like the best fit for your use case.

If you're dataset isn't too large and doesn't change much, you could simply stick it into the source; or, if you want to maintain it in the text file, the @Paramters method could read from there too.

- d

[basavara...@gmail.com]

Sushrut, 

Fitnesse is ideal for data tests with data flowing in from text files. I have used this earlier and have to say once you get the framework setup you can see your tests grow real fast. 

Parameterized tests are also another way to implement but don't seem suitable where your test data crosses say more than 100 scenarios. Also debugging test failures seemed difficult for me. In this regard Fitneese offers nice looking reports which can give you a better picture and also help you to publish those good looking results.

Let me know if you need help on setting up Fitneese.

Thanks,

Basu

[Sushrut Bidwai]

Thanks everyone for suggestions.

Fitness - looked at it does not seem like a solution. I am not testing a webapp, plain Java code.

@ashish - My testing setup is closely nit with deployment automation and it will be difficult to integrate a simple program as that means I lose out on lot of things junit offers. Like code coverages, notifications. Test data is also far too huge (80 files each with 500+ testdatasets growing every day) which entails a proper reporting to happen when something breaks to reduce turn around time for a fix. I also need to run tests along side production environment for certain things.

@saager - ParameterizedTests is good and perfect solution. Thanks a tonne for the tip. 

[Basavaraj Kalloli]

Hi Sushrut,

Fitness as you said is a testing tool for web applications. But I had used it like you for plain java code testing with junit - http://fitnesse.org/FitNesse.UserGuide.RunningFromJunit

So in the future if you do feel like using it, do take a look at it. 

Thanks !

[Sushrut Bidwai]

Thanks for offer to help. All set with parameterized tests now but will pick your brain if I need to come back to fitness for sure.  

[dexterous]

On Friday, February 1, 2013 5:11:13 PM UTC+5:30, Sushrut Bidwai wrote:

Thanks everyone for suggestions.

@saager - ParameterizedTests is good and perfect solution. Thanks a tonne for the tip.

One is glad to be of service.

- d 

[dexterous]

On Friday, February 1, 2013 5:16:46 PM UTC+5:30, Basavaraj Kalloli wrote:

Fitness[sic] [...] is a testing tool for web applications. But I had used it like you for plain java code testing with junit - http://fitnesse.org/FitNesse.UserGuide.RunningFromJunit

So in the future if you do feel like using it, do take a look at it.

FitNesse is a good tool in it's own right, and we should all definitely give it a spin.

However, it has a slightly different objective from JUnit. JUnit is a test runner, in that it runs tests and generates reports (period). FitNesse was built with writing business-use-friendly test fixtures as one of the primary objectives. That's where the focus on tables, columns and cells comes in. There was a design decision to separate test process from test data. JUnit is agnostic of how test data gets to the tests. It simply ignores that aspect and leaves it upto the developer to figure out. This is a perfectly fine set of decisions, given JUnit was originally intended as a unit testing framework while FitNesse is primarily intended for acceptance (system) testing.

My pet peeves about FitNesse (which I'm lising here for completeness of the discussion considering the number of people suggesting FitNesse on this thread) are:

• It puts you in this somewhat weird (at least to me) world of tables, rows and columns which seemed superflous to and distracting from the actual testing effort
• Running a single test (single row in a table)  is just an exercise in hair pulling!
  
• Feedback cycle was usually pretty long, which is fine for long-running system tests; but coupled with the fact that running subsets of tests was hard, this was a pain point for our in-system acceptance/integration tests.
• People using it for unit testing! This is more of a social issue that that of the framework. People just need to read a little better, it says it right there on the home page: fully integrated standalone wiki, and acceptance testing framework (emphasis mine)

There have been other attempts at frameworks allowing business-user-friendly (BDD-compliant, if there be such a term) tests, notable among them are:

• JBehave, from BDD vangaurd Dan North
  
• cuke4duke, a Java port of Ruby's cucumber
  
• Twist (as an ex-TWer I couldn't resist the shameless plug) actually this is cucumber+fitnesse packaged into eclipse
  

A comparative analysis of the above would be a separate discussion altogether

HTH

- d

[mohan kumar Muddana]

Hi Sushrut,

Have a look at JUnitParams. It might fit your requirements. We have used it sometime back to get parameters from a csv file.
http://code.google.com/p/junitparams/

It is similar to Kent Beck's tool as Sagar suggested.

It has few more features

• more explicit - params are in test method params, not class fields
• less code - you don't need a constructor to set up parameters
• you can mix parameterised with non-parameterised methods in one class
• params can be passed as a CSV string or from a parameters provider class
• parameters provider class can have as many parameters providing methods as you want, so that you can group different cases
• you can have a test method that provides parameters (no external classes or statics anymore)
• you can see actual parameter values in your IDE (in JUnit's Prameterized it's only consecutive numbers of parameters):

Thanks

Mohan

[Altaf Ahmed Dhoteghar]

Hi,

you can try by throwing custom exception for each failure, instead of
using assert.
may be this will help you to carry out your test scenario.

On Thu, Jan 31, 2013 at 10:52 PM, Sushrut Bidwai
<bidwai....@gmail.com> wrote:

> --
> You received this message because you are subscribed to the Google Groups
> "Java User Group For Pune, India" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to javapune+u...@googlegroups.com.
> To post to this group, send email to java...@googlegroups.com.
> Visit this group at http://groups.google.com/group/javapune?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>



--

A. M. Dhoteghar
Ph:9096266769

[Sushrut Bidwai]

Sorry been extremely busy with work in last week and hence could not get back.

Junitparameters does look interesting and I will give it a go. Thank you for that suggestion.

@altaf Throwing custom exception is not a solution as its difficult to accommodate it in all automation that is required.