Comma Separated Values (CSV) Files as DataProviders

2,727 views
Skip to first unread message

Jacob Robertson

unread,
Dec 12, 2006, 12:17:08 PM12/12/06
to testng...@googlegroups.com
In my team, our business analysts and testers collaborate to establish test cases, including the test data. What I would like to do is allow the BA and tester to own that data, and create/modify it in Excel (for example). If they can then hand that data off to me in a CSV file, I can (theoretically) shove it into my parameterized tests as a DataProvider. I have put together a little prototype that does this, but I'm surprised that this type of facility doesn't already exist. It seems like this would be a pretty common scenario. yet, in all the JUnit or TestNG documentation I've read all the examples of DataProviders pretty much use data that is hard-coded in java. Am I missing something? My perfect world would be to be able to do something like the following, or as close to it as possible.

[code]
/**
* @testng.test
* csvDataProviderFile="path/myparameters.csv"
*/
public void testSomeParameters(String p1, String p2, boolean r1, String r2) {
// test goes here
}
[/code]
---------------------------------------------------------------------
Posted via Jive Forums
http://forums.opensymphony.com/thread.jspa?threadID=53200&messageID=106987#106987

Alexandru Popescu

unread,
Dec 12, 2006, 12:27:21 PM12/12/06
to testng...@googlegroups.com
I am wondering if the following is not better:

@DataProvider
public Object[][] loadCSVData() {
}

@Test
public void yourTestMethod(...) {
}

Adding lots of different types of sources to @Test annotation will
definitely scale in the long run, but the above will always work (and
you can pick your data from whatever source you like).

HTH,

./alex
--
.w( the_mindstorm )p.
TestNG co-founder
EclipseTestNG Creator

Cédric Beust ♔

unread,
Dec 12, 2006, 12:27:13 PM12/12/06
to testng...@googlegroups.com
Hi Jacob,

Indeed, there is no easy way to do exactly this right now.  Some ideas:
  • Put the name of the CSV in a .properties file or somewhere else and read it inside your data provider
  • Use your own annotation (@CsvFile) to pass the file name to the data provider
I'm wondering what would be the easiest way to provide you with the functionality you are requesting, though...  For example, maybe we could make the Xml file available in its parsed form the the Data Provider, this way you could look up anything you want.  Or maybe we could just pass the <parameter> portion.

Thoughts?

--
Cedric


On 12/12/06, Jacob Robertson <testng...@opensymphony.com > wrote:



--
Cédric

Jacob Robertson

unread,
Dec 12, 2006, 12:54:50 PM12/12/06
to testng...@googlegroups.com
Thanks to both of you for your quick responses.

Let me elaborate a little on what I've put together that is at least working. I want to emphasize that the reason I posted to this forum is that I didn't want to build or use code that had already been written, or was not the right way to go. Secondly, if the code hadn't already been written, but was a worthwhile idea to promote to TestNg users in general, then I'm interested in that conversation.

The following is the snippet from my actual test case class. It does in fact reference a csv provider that I've written.

[code]
/**
* @testng.test
* dataProvider="CsvDataProvider"
* dataProviderClass="CsvDataProvider"


*/
public void testSomeParameters(String p1, String p2, boolean r1, String r2) {
// test goes here
}
[/code]

The following is the CsvDataProvider implementation I put together as a proof of concept. The idea here is to have a lazy-loading Iterator that provides the lines from the csv file one at a time. I wanted to be able to either easily find the csv file through convention, or allow the user to create their own data provider method that would explicitly reference the csv file they wanted to use.

(not sure why the forum software is converting some of my brackets incorrectly, but not all of them...)

[code]
import org.apache.commons.beanutils.ConvertUtils;
import org.apache.commons.beanutils.Converter;

import au.com.bytecode.opencsv.CSVReader;


/**
* Convenience class to provide data sets for unit tests within TestNG.
*
* @author cjsr
*/
public class CsvDataProvider implements Iterator {

/**
* @testng.data-provider name="CsvDataProvider"
* @param method the method the TestNg passes to you
* @return an Iterator of Object[]
*/
public static Iterator getDataProvider(Method method) throws IOException {
return getDataProvider(method.getDeclaringClass(), method);
}

/**
* Call this directly when necessary, to avoid issues
* with the method's declaring class not being the test class.
* @param cls The actual test class - matters for the CsvFileName and the class loader
* @return an Iterator of Object[]
*/
public static Iterator getDataProvider(Class cls, Method method) throws IOException {
String className = cls.getName();
String dirPlusPrefix = className.replace('.', '/');
String fileName = method.getName() + ".csv";
String filePath = dirPlusPrefix + "." + fileName;

return new CsvDataProvider(cls, method, filePath);
}

private CSVReader reader;
private String[] last;
private Class[] parameterTypes;
private Converter[] parameterConverters;

/**
* Basic constructor that will provide the data from the given file for the given method
* @throws IOException when file io fails
*/
public CsvDataProvider(Class cls, Method method, String csvFilePath) throws IOException {
InputStream is = cls.getClassLoader().getResourceAsStream(csvFilePath);
InputStreamReader isr = new InputStreamReader(is);
reader = new CSVReader(isr);
parameterTypes = method.getParameterTypes();
int len = parameterTypes.length;
parameterConverters = new Converter[len];
for (int i = 0; i < len; i++) {
parameterConverters[i] = ConvertUtils.lookup(parameterTypes[i]);
}
}

/**
* @return see Iterator contract.
*/
public boolean hasNext() {
return (getNextLine() != null);
}

/**
* Get the next line, or the current line if it's already there.
* @return the line.
*/
private String[] getNextLine() {
if (last == null) {
try {
last = reader.readNext();
} catch (IOException ioe) {
throw new RuntimeException(ioe);
}
}
return last;
}

/**
* @return the Object[] representation of the next line
*/
public Object next() {
String[] next;
if (last != null) {
next = last;
} else {
next = getNextLine();
}
last = null;
Object[] args = parseLine(next);
return args;
}

/**
* @return the correctly parsed and wrapped values
* @todo need a standard third-party CSV parser plugged in here
*/
private Object[] parseLine(String[] svals) {
int len = svals.length;
Object[] ovals = new Object[len];
for (int i = 0; i < len; i++) {
ovals[i] = parameterConverters[i].convert(parameterTypes[i], svals[i]);
}
return ovals;
}

/**
* Always throws UnsupportedOperationException.
*/
public void remove() {
throw new UnsupportedOperationException();


}
}
[/code]
---------------------------------------------------------------------
Posted via Jive Forums

http://forums.opensymphony.com/thread.jspa?threadID=53200&messageID=107006#107006

Cédric Beust ♔

unread,
Dec 12, 2006, 1:16:56 PM12/12/06
to testng...@googlegroups.com
Hi Jacob,

Wow, that's a great mix of various advanced features:
  • Data Providers in different classes than the methods they feed
  • Data Providers using the method name of the test they are about to feed
  • Iterators for lazy loading
  • Dynamic look up of the file to parse based on the calling class
Very neat!

--
Cedric


On 12/12/06, Jacob Robertson <testng...@opensymphony.com> wrote:

Cédric Beust ♔

unread,
Dec 12, 2006, 3:20:30 PM12/12/06
to testng...@googlegroups.com
Jacob, would you mind contacting me privately?  There's something I'd like to ask you (cbeust at google).

Thanks!

--
Cedric
--
Cédric

christo nexus

unread,
Feb 8, 2015, 10:18:58 PM2/8/15
to testng...@googlegroups.com
Hi Jacob,

I am new to selenium, do you have any simple example (eg login) on
pass parameter to dataprovider in testng from csv file
(I need to read and write data into csv file)
Appreciate if you can help. thanks.

Can you contact me via gmail: christ...@gmail.com
I have some question on Selenium CSV dataProvider if you don't mind. Thanks.
Reply all
Reply to author
Forward
0 new messages