OTP2 Kryo serialisation bug

[Philip Kamenarsky]

Hi all,

We seem to be hitting a long-standing serialisation bug when trying to generate a graph for the UK, both with OTP1 and OTP2:

17:32:48.797 ERROR (OTPMain.java:45) An uncaught error occurred inside OTP: java.lang.NegativeArraySizeException: -2147483645 Serialization trace: serviceDatesByServiceId (org.opentripplanner.model.calendar.CalendarServiceData) services (org.opentripplanner.routing.graph.Graph) graph (org.opentripplanner.routing.graph.SerializedGraphObject) com.esotericsoftware.kryo.KryoException: java.lang.NegativeArraySizeException: -2147483645

(For more context, see here https://github.com/opentripplanner/OpenTripPlanner/issues/3128).

One of the suggestions in that thread was that it might be worth trying updating to Kryo 5  (even though it was assumed that that particular bug has been fixed in Kryo 4 already, though apparently not).

Unfortunately, updating to Kryo 5 seems to be a more complicated endeavour than just swapping version numbers and fixing compiler errors, as suggested here https://github.com/opentripplanner/OpenTripPlanner/pull/3230#issuecomment-722189107.

Is anybody else hitting the same bug? Maybe the community knows of other workarounds worth trying before biting the bullet and attempting a proper migration to Kryo 5, which might not even fix the original problem.

Thanks,

Phil

[Michael Heilig]

Hi,

I am receiving the same error message when trying to build a graph from a German wide GTFS feed (gtfs.de) with both versions 1.5.0 and 2.0.0. In other forums and in the OTP issues on Github, I also saw a similar error message using large gtfs files. Could that be an issue?

Best,

Michael

[Andrew Byrd]

Hello,

The library (Kryo) that we use to save files to disk uses 32-bit signed integers for addressing, which imposes some limits on the number of items that can be serialized. See: https://github.com/EsotericSoftware/kryo#reference-limits

It might be possible to circumvent this by saving the data in separate chunks that don't have references between them, but I don't know of any completed work in this direction.

-Andrew

-- 
You received this message because you are subscribed to the Google Groups "OpenTripPlanner Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to opentripplanner...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/opentripplanner-dev/814f51d7-8df9-454d-8282-205cfce2b65an%40googlegroups.com.

[Michael Heilig]

Hi Andrew,

 

thank you for you quick reply. Based on your reference about Kryo, I solved the problem by filtering unnecessary elements from the OSM file (removed all tracks), so that the graph file is below the maximum size.

 

Best

Michael