Atlanta-area trip plan crashes OTP 1.4.x

82 views
Skip to first unread message

ma...@rideamigos.com

unread,
Sep 9, 2021, 9:39:57 PM9/9/21
to OpenTripPlanner Developers
I have a transit trip plan query in the Atlanta area which crashes OTP 1.4.x. I didn't find an an issue in the issue tracker which seems like an exact match.

I have the exact the query but I'm not making it public because it causes a denial of service attack on my own OTP cluster.  Below is the stack trace it generates. 

I tried using the same origin/destination on MARTA's Atlanta-area OpenTripPlanner instance but it did not appear to crash it. 

-------

java.lang.IndexOutOfBoundsException: Index: 2, Size: 2
        at java.util.ArrayList.rangeCheck(ArrayList.java:659) ~[na:1.8.0_292]
        at java.util.ArrayList.get(ArrayList.java:435) ~[na:1.8.0_292]
        at org.opentripplanner.routing.edgetype.Timetable.getTripTimes(Timetable.java:310) ~[otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.opentripplanner.routing.edgetype.TripPattern.getResolvedTripTimes(TripPattern.java:278) ~[otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.opentripplanner.routing.edgetype.TripPattern.getResolvedTripTimes(TripPattern.java:271) ~[otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.opentripplanner.routing.edgetype.PatternInterlineDwell.traverse(PatternInterlineDwell.java:142) ~[otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.opentripplanner.routing.algorithm.AStar.iterate(AStar.java:172) ~[otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.opentripplanner.routing.algorithm.AStar.runSearch(AStar.java:249) ~[otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.opentripplanner.routing.algorithm.AStar.getShortestPathTree(AStar.java:297) ~[otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.opentripplanner.routing.algorithm.AStar.getShortestPathTree(AStar.java:84) ~[otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.opentripplanner.routing.impl.GraphPathFinder.getPaths(GraphPathFinder.java:154) ~[otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.opentripplanner.routing.impl.GraphPathFinder.getGraphPathsConsideringIntermediates(GraphPathFinder.java:437) ~[otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.opentripplanner.routing.impl.GraphPathFinder.graphPathFinderEntryPoint(GraphPathFinder.java:340) ~[otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.opentripplanner.api.resource.PlannerResource.plan(PlannerResource.java:68) ~[otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_292]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_292]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_292]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_292]
        at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:308) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:291) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1140) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:375) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:565) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:545) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at java.lang.Thread.run(Thread.java:748) [na:1.8.0_292]
00:57:34.974 WARN (PlannerResource.java:77) Error while planning path:
java.lang.IndexOutOfBoundsException: Index: 2, Size: 2
        at java.util.ArrayList.rangeCheck(ArrayList.java:659) ~[na:1.8.0_292]
        at java.util.ArrayList.get(ArrayList.java:435) ~[na:1.8.0_292]
        at org.opentripplanner.routing.edgetype.Timetable.getTripTimes(Timetable.java:310) ~[otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.opentripplanner.routing.edgetype.TripPattern.getResolvedTripTimes(TripPattern.java:278) ~[otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.opentripplanner.routing.edgetype.TripPattern.getResolvedTripTimes(TripPattern.java:271) ~[otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.opentripplanner.routing.edgetype.PatternInterlineDwell.traverse(PatternInterlineDwell.java:142) ~[otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.opentripplanner.routing.algorithm.AStar.iterate(AStar.java:172) ~[otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.opentripplanner.routing.algorithm.AStar.runSearch(AStar.java:249) ~[otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.opentripplanner.routing.algorithm.AStar.getShortestPathTree(AStar.java:297) ~[otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.opentripplanner.routing.algorithm.AStar.getShortestPathTree(AStar.java:84) ~[otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.opentripplanner.routing.impl.GraphPathFinder.getPaths(GraphPathFinder.java:154) ~[otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.opentripplanner.routing.impl.GraphPathFinder.getGraphPathsConsideringIntermediates(GraphPathFinder.java:437) ~[otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.opentripplanner.routing.impl.GraphPathFinder.graphPathFinderEntryPoint(GraphPathFinder.java:340) ~[otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.opentripplanner.api.resource.PlannerResource.plan(PlannerResource.java:68) ~[otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_292]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_292]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_292]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_292]
        at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:308) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:291) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1140) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:375) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:565) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:545) [otp-1.4.0-SNAPSHOT-shaded.jar:1.1]
        at java.lang.Thread.run(Thread.java:748) [na:1.8.0_292]

ma...@rideamigos.com

unread,
Sep 9, 2021, 9:57:18 PM9/9/21
to OpenTripPlanner Developers
This issue from the bug tracker seems related, because it mentions both `java.lang.IndexOutOfBoundsException` and the last method called in the Stack trace is: `getTripTimes`:


That ticket says the bug discussion is considered in another issue:

Alighting at the wrong stop

That issue is still open, and someone else reported that they had a related problem in December, 2020. It appears the issue may be affecting some other instances in active use.  

Andrew Byrd

unread,
Sep 10, 2021, 2:46:57 AM9/10/21
to ma...@rideamigos.com, OpenTripPlanner Developers
Hello,

When you say this crashes OTP, do you mean that the entire server stops responding? I would normally expect an exception generating a stack trace like this to cause a single request to fail, while the server continues to run and accept requests. Can you describe in more detail the effects observed on your server? If an unexpected exception while processing a single request takes down the whole server, there is some deeper problem here we need to identify.

I should mention that this 1.4 release of OTP is over two years old, and most work on OTP since then has been on OTP 2.x which has a completely new transit routing engine, where it is unlikely that code related to this problem still exists. It is not certain that developer effort will be directed to debugging and patching this older version. If you can demonstrate that the problem exists with the 1.5 tag or the dev-1.x branch, then it is more likely to receive attention as a bugfix on the final maintenance branch of OTP 1.x. 

Thanks for identifying existing Github issues that seem related. Indeed the stack trace looks similar to some reported in issue #1794, specifically PatternInterlineDwell.traverse calling TripPattern.getResolvedTripTimes, in turn calling Timetable.getTripTimes with an array index equal to the array length. The issue #2153 that is linked from #1794 seems to involve alighting at the wrong stop without the request outright failing with a stack trace. My sense from a quick read-through is that the problem causing the exceptions was solved (at least partially) in #1794 and discussion moved on to a possibly related problem in #2153.

Please confirm whether the problem exists with identical inputs and configuration on current versions of OTP, specifically 1.5 and/or 1.x, and ideally OTP 2. If the problem does exist with these later versions a new issue can be created, referencing the old ones but starting a new discussion -- the existing issues are over 5 years old and have large numbers of comments on them that may or may not be related. 

For debugging purposes it would be necessary to share all input data and configuration information, as well as the specific request that triggers the problem. If this request actually crashes the entire server then I can understand your hesitation to share it publicly - please document clearly on the issue what the effects are on your server. That said, if someone really wanted to DDOS your server they could probably do it without using this particular URL. It’s going to be much harder for anyone to understand the source of the problem if they can’t trace execution for a specific request.

Thanks,
Andrew



--
You received this message because you are subscribed to the Google Groups "OpenTripPlanner Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to opentripplanner...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/opentripplanner-dev/bb5d8ff1-45fd-4b02-ad21-4aaed17dbca7n%40googlegroups.com.

ma...@rideamigos.com

unread,
Sep 10, 2021, 11:11:08 AM9/10/21
to OpenTripPlanner Developers


When you say this crashes OTP, do you mean that the entire server stops responding?

We do get back an HTTP 200 response from the query, but it includes this JSON:

"error": { 23 "id": 500, 24 "message": "SYSTEM_ERROR", 25 "msg": "We're sorry. The trip planner is temporarily unavailable. Please try again later.", 26 "noPath": true 27 },

Further, the server at least temporarily stops responding to basic requests after this. I noticed the condition because monitoring detected at this router was "down" after the query came in, meaning basic queries no longer returned results.

I should mention that this 1.4 release of OTP is over two years old, and most work on OTP since then has been on OTP 2.x which has a completely new transit routing engine, where it is unlikely that code related to this problem still exists. It is not certain that developer effort will be directed to debugging and patching this older version. If you can demonstrate that the problem exists with the 1.5 tag or the dev-1.x branch, then it is more likely to receive attention as a bugfix on the final maintenance branch of OTP 1.x. 

I will look at 1.5 and 2.x as possible upgrade paths. It looks like 1.5 has just a few change in it vs 1.4, though and none of them seem related to this issue:

 
Please confirm whether the problem exists with identical inputs and configuration on current versions of OTP, specifically 1.5 and/or 1.x, and ideally OTP 2. If the problem does exist with these later versions a new issue can be created, referencing the old ones but starting a new discussion -- the existing issues are over 5 years old and have large numbers of comments on them that may or may not be related. 

I'll follow up when I'm able to test with newer versions. Because we serve about 500 feeds across 20 routers upgrades are a bit of project. I can see about spinning up small instance for debugging to test, though.  

For debugging purposes it would be necessary to share all input data and configuration information, as well as the specific request that triggers the problem. If this request actually crashes the entire server then I can understand your hesitation to share it publicly - please document clearly on the issue what the effects are on your server. That said, if someone really wanted to DDOS your server they could probably do it without using this particular URL. It’s going to be much harder for anyone to understand the source of the problem if they can’t trace execution for a specific request.

I've shared the crashing query details privately.

Google Maps and MARTA's OTP trip planner in the area detect no viable transit routes between the two points, but it does seem that only a transit request crashes. We also ran walk and bike routing requests between the two points and didn't se the same pattern. 

Thanks,

       Mark

ma...@rideamigos.com

unread,
Sep 10, 2021, 12:55:08 PM9/10/21
to OpenTripPlanner Developers
I built a minimal graph with Georgia's OSM extract the MARTA and GRTA as well as a private Emory GTFS feed and tested against OTP 1.5.x. It also throws an exception there. The process doesn't exit and appears to handle further incomng requests unlike my experience with the extra-large instance we use in production. Here's the Stack Trace from 1.5:

2:49:35.380 ERROR (PlannerError.java:48) exception planning trip:
java.lang.IndexOutOfBoundsException: Index 2 out of bounds for length 2
at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64) ~[na:na]
at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70) ~[na:na]
at java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:248) ~[na:na]
at java.base/java.util.Objects.checkIndex(Objects.java:372) ~[na:na]
at java.base/java.util.ArrayList.get(ArrayList.java:459) ~[na:na]
at org.opentripplanner.routing.edgetype.Timetable.getTripTimes(Timetable.java:437) ~[otp-1.5.0-shaded.jar:1.1]
at org.opentripplanner.routing.edgetype.TripPattern.getResolvedTripTimes(TripPattern.java:277) ~[otp-1.5.0-shaded.jar:1.1]
at org.opentripplanner.routing.edgetype.TripPattern.getResolvedTripTimes(TripPattern.java:270) ~[otp-1.5.0-shaded.jar:1.1]
at org.opentripplanner.routing.edgetype.PatternInterlineDwell.traverse(PatternInterlineDwell.java:142) ~[otp-1.5.0-shaded.jar:1.1]
at org.opentripplanner.routing.algorithm.AStar.iterate(AStar.java:172) ~[otp-1.5.0-shaded.jar:1.1]
at org.opentripplanner.routing.algorithm.AStar.runSearch(AStar.java:249) ~[otp-1.5.0-shaded.jar:1.1]
at org.opentripplanner.routing.algorithm.AStar.getShortestPathTree(AStar.java:297) ~[otp-1.5.0-shaded.jar:1.1]
at org.opentripplanner.routing.algorithm.AStar.getShortestPathTree(AStar.java:84) ~[otp-1.5.0-shaded.jar:1.1]
at org.opentripplanner.routing.impl.GraphPathFinder.getPaths(GraphPathFinder.java:169) ~[otp-1.5.0-shaded.jar:1.1]
at org.opentripplanner.routing.impl.GraphPathFinder.getGraphPathsConsideringIntermediates(GraphPathFinder.java:468) ~[otp-1.5.0-shaded.jar:1.1]
at org.opentripplanner.routing.impl.GraphPathFinder.graphPathFinderEntryPoint(GraphPathFinder.java:370) ~[otp-1.5.0-shaded.jar:1.1]
at org.opentripplanner.api.resource.PlannerResource.plan(PlannerResource.java:68) ~[otp-1.5.0-shaded.jar:1.1]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:308) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.jersey.internal.Errors.process(Errors.java:315) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.jersey.internal.Errors.process(Errors.java:297) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.jersey.internal.Errors.process(Errors.java:267) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:291) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1140) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:375) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:565) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:545) ~[otp-1.5.0-shaded.jar:1.1]
at java.base/java.lang.Thread.run(Thread.java:829) ~[na:na]
12:49:35.383 WARN (PlannerResource.java:77) Error while planning path:
java.lang.IndexOutOfBoundsException: Index 2 out of bounds for length 2
at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64) ~[na:na]
at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70) ~[na:na]
at java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:248) ~[na:na]
at java.base/java.util.Objects.checkIndex(Objects.java:372) ~[na:na]
at java.base/java.util.ArrayList.get(ArrayList.java:459) ~[na:na]
at org.opentripplanner.routing.edgetype.Timetable.getTripTimes(Timetable.java:437) ~[otp-1.5.0-shaded.jar:1.1]
at org.opentripplanner.routing.edgetype.TripPattern.getResolvedTripTimes(TripPattern.java:277) ~[otp-1.5.0-shaded.jar:1.1]
at org.opentripplanner.routing.edgetype.TripPattern.getResolvedTripTimes(TripPattern.java:270) ~[otp-1.5.0-shaded.jar:1.1]
at org.opentripplanner.routing.edgetype.PatternInterlineDwell.traverse(PatternInterlineDwell.java:142) ~[otp-1.5.0-shaded.jar:1.1]
at org.opentripplanner.routing.algorithm.AStar.iterate(AStar.java:172) ~[otp-1.5.0-shaded.jar:1.1]
at org.opentripplanner.routing.algorithm.AStar.runSearch(AStar.java:249) ~[otp-1.5.0-shaded.jar:1.1]
at org.opentripplanner.routing.algorithm.AStar.getShortestPathTree(AStar.java:297) ~[otp-1.5.0-shaded.jar:1.1]
at org.opentripplanner.routing.algorithm.AStar.getShortestPathTree(AStar.java:84) ~[otp-1.5.0-shaded.jar:1.1]
at org.opentripplanner.routing.impl.GraphPathFinder.getPaths(GraphPathFinder.java:169) ~[otp-1.5.0-shaded.jar:1.1]
at org.opentripplanner.routing.impl.GraphPathFinder.getGraphPathsConsideringIntermediates(GraphPathFinder.java:468) ~[otp-1.5.0-shaded.jar:1.1]
at org.opentripplanner.routing.impl.GraphPathFinder.graphPathFinderEntryPoint(GraphPathFinder.java:370) ~[otp-1.5.0-shaded.jar:1.1]
at org.opentripplanner.api.resource.PlannerResource.plan(PlannerResource.java:68) ~[otp-1.5.0-shaded.jar:1.1]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:308) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.jersey.internal.Errors.process(Errors.java:315) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.jersey.internal.Errors.process(Errors.java:297) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.jersey.internal.Errors.process(Errors.java:267) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:291) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1140) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:375) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:565) ~[otp-1.5.0-shaded.jar:1.1]
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:545) ~[otp-1.5.0-shaded.jar:1.1]
at java.base/java.lang.Thread.run(Thread.java:829) ~[na:na]

ma...@rideamigos.com

unread,
Sep 10, 2021, 1:04:03 PM9/10/21
to OpenTripPlanner Developers
I ran the private Emory GTFS feed through a validator and it did have one out-of-bounds error, so maybe that's related:

Notice{filename='frequencies.txt', level='ERROR', code='10', title='Out of range integer value', description='Invalid value for field:`headway_secs` of entity with id:`null` -- min:0 max:3600 actual:4800', extra='{fieldName=headway_secs, rangeMax=3600, rangeMin=0, actualValue=4800}'},

ma...@rideamigos.com

unread,
Sep 10, 2021, 4:51:05 PM9/10/21
to OpenTripPlanner Developers
I corrected this validation issue, but the Emory feed still triggers the issue. I'm investigating whether I'm able to share the feed.

ma...@rideamigos.com

unread,
Sep 13, 2021, 1:36:11 PM9/13/21
to OpenTripPlanner Developers

I should mention that this 1.4 release of OTP is over two years old, and most work on OTP since then has been on OTP 2.x 

For a number of issues that concern us, there's no indication that open issues that they are confirmed-fixed in OTP 2, or they are simply not implemented:

  *  We submitted a patch to make GTFS file loading deterministic. While RideAmigos and IBI use in production, no comment was ever made on whether the issue was fixed in OTP 2: https://github.com/opentripplanner/OpenTripPlanner/pull/3022
 *  The GBFS auto-discovery spec is not followed. IBI partially fixed this. No comment if OTP-2 resolves this: https://github.com/opentripplanner/OpenTripPlanner/issues/2814
 * Isochrone support was removed. 

Generally when I have looked to solutions to our problems, I've been more likely to find the solutions in the IBI 1.x branch then I have in the mainline 2.0 branch. 

Andrew Byrd

unread,
Sep 15, 2021, 11:32:25 AM9/15/21
to ma...@rideamigos.com, opentripplanner-dev
Hi Mark,

My responses are inline below.

On 10 Sep 2021, at 23:11, ma...@rideamigos.com <ma...@rideamigos.com> wrote:

When you say this crashes OTP, do you mean that the entire server stops responding?

We do get back an HTTP 200 response from the query, but it includes this JSON:

"error": { 23 "id": 500, 24 "message": "SYSTEM_ERROR", 25 "msg": "We're sorry. The trip planner is temporarily unavailable. Please try again later.", 26 "noPath": true 27 },

The text stating that the planner is "temporarily unavailable" is just a stock message produced whenever an unhandled exception reaches a final catch-all block while handling a request. It does not mean that the whole server is down or unable to handle any request other than the one that just failed. The id:500 indicates this is analogous to an HTTP 500 (server error). It appears that this unexpected exception was cleanly handled in the catch clause near the end of the PlannerResource#plan method.

(I'm not saying it makes sense to return a 500 message inside an HTTP 200 response, just describing the way things are. I think this behavior is a relic of the early days of OTP over a decade ago.)

Further, the server at least temporarily stops responding to basic requests after this. I noticed the condition because monitoring detected at this router was "down" after the query came in, meaning basic queries no longer returned results.

This failure to respond is unexpected behavior, separate from the out of bounds exception. I don't know why this would happen. Any such exception should cause only a single request to fail. If you can isolate the problem to OTP itself, rather than the environment OTP is running in (caching or load balancing layers over OTP for example) this would deserve a Github issue of its own.

I will look at 1.5 and 2.x as possible upgrade paths. It looks like 1.5 has just a few change in it vs 1.4, though and none of them seem related to this issue:

I wasn't suggesting an upgrade, just emphasizing that any problem must be shown to exist on the latest release of 1.x before there would be grounds to investigate it further, and that the existence of a problem on 2.x would be most likely to receive attention.

Reproducing the problem in these environments (as you did in follow-up messages) is the best way to help people understand where the problem exists and whether it's something they should prioritize.

- Andrew

Andrew Byrd

unread,
Sep 15, 2021, 12:44:22 PM9/15/21
to ma...@rideamigos.com, opentripplanner-dev
Hi Mark,

Following your message we discussed these points at Tuesday's meeting, and I see that Leonard and Hannes commented on #2814 showing where that issue has been addressed in OTP2. For situations like this where you are in need of a progress update or straightforward information, please don't hesitate to bump issues or PRs and mention people by name. There is a constant flow of new material to review, and sometimes we just need to be reminded.

There is indeed not feature parity between OTP 1.x and 2.x. Thomas Gran has been making an effort to keep this documented at http://docs.opentripplanner.org/en/latest/Version-Comparison/ and I know he's looking forward to reintroducing a few important 1.x features in the next release of 2.x. Some other features have been removed due to an intentional focus on passenger-facing transit trip planning in 2.x.

Regarding maintenance branches of OTP 1.x, I would like to describe the current situation in hopes of making things clearer for the community.

There are several people working full-time on OTP now, and as far as I know most of them are working almost exclusively on OTP2. They need to follow the priorities of their employers and clients, which lead them to allocate most time to specific capabilities of the 2.x branch. Anything else is essentially a volunteer effort (personal or organizational) out of good will to society or their professional community.

Considering this, at a previous OTP Project Leadership Committee meeting we came to a clear decision: if some organizations would like to continue development and maintenance of OTP1 they should propose a team specifically for this purpose, consisting of at least two organizations that take responsibility for the 1.x code base, perform code reviews etc.

Currently, changes to OTP1 will be applied to the final maintenance branch of OTP1, which an unknown number of people around the world depend on. Even when OTP2-focused reviewers have time to look at OTP1 contributions, they feel an obligation to exercise caution - the whole rest of the community will have to deal indefinitely with the maintainability or stability implications of new features that are convenient for some users. The resulting process of review and coordination with contributors can be very time consuming.

It's completely fine to use and maintain a third-party branch of OTP1 that suits the needs of your deployment and others like it. Changes on such a branch can proceed more quickly and without formal review, because they won't affect the final 1.x maintenance branch most visible to the rest of the community. If such a branch is shared by multiple organizations which have sufficient time to take responsibility for maintenance and code review, then the OTP committee is clearly open to proposals to create a 1.x maintenance team.

I hope this clarifies the situation a bit.

Regards,
Andrew


--
You received this message because you are subscribed to the Google Groups "OpenTripPlanner Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to opentripplanner...@googlegroups.com.

Mark Stosberg

unread,
Sep 15, 2021, 3:39:16 PM9/15/21
to Andrew Byrd, opentripplanner-dev
Great clarity. Thank you!
--

Mark Stosberg (he/him)

Director of Systems & Security

mark@rideamigos.com | 765.277.1916

https://www.rideamigos.com

Changing the way the world commutes.


Reply all
Reply to author
Forward
0 new messages