Error generating Graph.obj when a comma in name

[m.gro...@gmail.com]

I got following error, if I try to generate a Graph.obj:

12:12:06.253 WARN (IndividualCsvEntityReader.java:110) expected and actual number of csv fields differ: type=org.onebusaway.gtfs.model.Stop line # 13 expected=6 actual=7
Exception in thread "main" org.onebusaway.csv_entities.exceptions.CsvEntityIOException: io error: entityType=org.onebusaway.gtfs.model.Stop path=java.io.InputStreamReader@53706eb7 lineNumber=13
    at org.onebusaway.csv_entities.CsvEntityReader.readEntities(CsvEntityReader.java:161)
    at org.onebusaway.csv_entities.CsvEntityReader.readEntities(CsvEntityReader.java:120)
    at org.onebusaway.csv_entities.CsvEntityReader.readEntities(CsvEntityReader.java:115)
    at org.onebusaway.csv_entities.CsvEntityReader.readEntities(CsvEntityReader.java:108)
    at org.opentripplanner.graph_builder.impl.GtfsGraphBuilderImpl.loadBundle(GtfsGraphBuilderImpl.java:220)
    at org.opentripplanner.graph_builder.impl.GtfsGraphBuilderImpl.buildGraph(GtfsGraphBuilderImpl.java:156)
    at org.opentripplanner.graph_builder.GraphBuilderTask.run(GraphBuilderTask.java:127)
    at org.opentripplanner.graph_builder.GraphBuilderMain.main(GraphBuilderMain.java:55)
Caused by: java.lang.IndexOutOfBoundsException: Index: 6, Size: 6
    at java.util.ArrayList.rangeCheck(ArrayList.java:571)
    at java.util.ArrayList.get(ArrayList.java:349)
    at org.onebusaway.csv_entities.IndividualCsvEntityReader.readEntity(IndividualCsvEntityReader.java:123)
    at org.onebusaway.csv_entities.IndividualCsvEntityReader.handleLine(IndividualCsvEntityReader.java:96)
    at org.onebusaway.csv_entities.CsvEntityReader.readEntities(CsvEntityReader.java:157)
    ... 7 more

Line 13 is as follow:
112, "Berliner Str.,Schule", 50.566265, 10.420705, 1,

As you can see there is a comma in the name, but it is surrounded by quotes, so it shouldn't be a problem.
I also checked my .gtfs file with feedvalidator.py which shows no error. Also the specification of gtfs says that names can have commas, if it is surrounded by quotes. So I guess this is an error in the parser.
What do you think?

[Andrew Byrd]

On 06/30/2013 12:17 AM, m.gro...@gmail.com wrote:
> Line 13 is as follow:
> 112, "Berliner Str.,Schule", 50.566265, 10.420705, 1,
>
> As you can see there is a comma in the name, but it is surrounded
> by|quotes, so it shouldn't be a problem.

This is due to a strict interpretation of a grammar in the RFC defining
CSV files. The OBA CSV parser expects the opening quote to be the first
character in the field, but your GTFS has spaces after the commas.

That same RFC encourages CSV consumers to be tolerant, so ideally we
would ignore whitespace after commas. We submitted a patch to OBA a
while back to fix this but I don't think it has been incorporated yet.

If the input GTFS is fixed such that the opening quotes immediately
follow the commas, it will load.

-Andrew

[m.gro...@gmail.com]

thanks Andrew for your reply. This was indeed the problem. I eliminated all spaces and now it is working. 

-Michael