How to ignore rows that cannot be parsed

55 views
Skip to first unread message

tonyp...@gmail.com

unread,
Nov 12, 2017, 10:50:03 PM11/12/17
to SimpleFlatMapper
I would like to know if there is a way to ignore rows that cannot be parsed. Some of the rows in my source data have missing data. In those cases, I would like to simply ignore the row. I haven't found a way to do this so far. Please let me know if there is a way to do this.

Thanks again,

-Tony

Arnaud Roger

unread,
Nov 13, 2017, 10:02:26 AM11/13/17
to SimpleFlatMapper
It is possible to ignore the field that fail to parse, but not the row.
The trick is to build a CsvMapper with a specified FieldMapperErrorHandler.

ie

@Test
public void testIgnoreFailedMapRow() throws IOException {
CsvMapper<AB> csvMapper = CsvMapperFactory
.newInstance()
.fieldMapperErrorHandler((key, source, target, error) -> {
System.out.println("error : " + key + "," + source + "," + target +"," + error);
})
.newMapper(AB.class);
CsvParser.mapWith(csvMapper).stream("a,b\n1,2\nfreddy is coming for you\n3,4\nbetter lock your door").forEach(r-> System.out.println("row = " +r));
}

public static class AB {
public final int a;
public final int b;

public AB(int a, int b) {
this.a = a;
this.b = b;
}

@Override
public String toString() {
return "AB{" +
"a=" + a +
", b=" + b +
'}';
}
}

it should be possible to add a feature that ignore row that have an issue, but it's not that trivial.

tonyp...@gmail.com

unread,
Nov 13, 2017, 10:15:29 AM11/13/17
to SimpleFlatMapper

Thanks for the fast reply! Here is an example of a data file where I would like to ignore the rows that have lots of missing fields:



+1 for adding the feature to allow ignoring rows with parse errors. 


Thanks for your work on this very useful library.


-Tony

Reply all
Reply to author
Forward
0 new messages