How to update graph with new GTFS-static data?

[Enrico Rotundo]

I'm deploying OpenTripPlanner 1.2.0 using this Docker container. I need graph persistence so I'm not using the --inMemory flag: first I run `--build` and then `--server`. At building time I have a GTFS (static) file in the build directory that contains scheduled trips of roughly 3 months; how do I update the graph in the background and re-load it into OpenTripPlanner?

I could do:

1. Fetch the latest archive with `wget http://gtfs.ovapi.nl/new/gtfs-nl.zip`
2. Build graph
3. Re-load OTP instance with new graph

Is there an automatic procedure I can use instead?

[Andrew Byrd]

Hello,

You have a few options.

1. Build the new graph, restart OTP with the new graph (causes downtime while the new instance starts up).

2. Build the new graph, start a new instance of OTP with that new graph, switch your reverse proxy over to the new instance once it’s loaded (avoids downtime but would require some manual effort)

3. Build the new graph, then signal the server to reload the new graph and swap it over the existing one. This can be done with downtime (while the new graph loads) or without downtime (by loading the entire new graph before it’s swapped in, which takes twice as much memory).

4. Push a bundle of data to the running OTP server, which will then build a graph and swap it in.  

You can probably come up with even more workflows. See class org.opentripplanner.api.resource.Routers for this “routers API” for options 3 and 4 and relevant endpoints. These endpoints are obviously dangerous, so are only accessible with authentication over HTTPS. See http://opentripplanner.readthedocs.io/en/latest/Security/ for how to set that up.

However if you just want a highly available OTP router for the whole Netherlands you may want to check with Plannerstack, who work closely with OVAPI to provide such a router. You can find more information at http://www.plannerstack.com/. This is a commercial service but run by a group of FOSS-friendly people so depending on your goals and motivations lower-cost collaboration may be possible.

Regards,

Andrew

--
You received this message because you are subscribed to the Google Groups "OpenTripPlanner Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to opentripplanner-...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Andrew Byrd]

> On 25 Oct 2017, at 13:51, Andrew Byrd <and...@fastmail.net> wrote:
> 3. Build the new graph, then signal the server to reload the new graph and swap it over the existing one. This can be done with downtime (while the new graph loads) or without downtime (by loading the entire new graph before it’s swapped in, which takes twice as much memory).

Another thought for developers:

Many other daemon (background server) processes respond to the POSIX signal SIGHUP by reloading configuration data. We could consider making OTP respond to this signal so you wouldn’t have to configure secure API access to trigger a graph reload.

Andrew

[Liam Neville]

Apologies for reviving such an old thread but I'm also looking to implement this type of functionality. For the app I'm working on, it's possible to update timetables or add/remove routes. Since routes will be user generated, I anticipate frequent changes. Are the above options this still the best way to do this a couple of years on? How will this affect OTP2?

Thanks,

Liam

On Wednesday, 25 October 2017 12:51:44 UTC+1, Andrew Byrd wrote:

Hello,

You have a few options.

1. Build the new graph, restart OTP with the new graph (causes downtime while the new instance starts up).

2. Build the new graph, start a new instance of OTP with that new graph, switch your reverse proxy over to the new instance once it’s loaded (avoids downtime but would require some manual effort)

3. Build the new graph, then signal the server to reload the new graph and swap it over the existing one. This can be done with downtime (while the new graph loads) or without downtime (by loading the entire new graph before it’s swapped in, which takes twice as much memory).

4. Push a bundle of data to the running OTP server, which will then build a graph and swap it in.  

You can probably come up with even more workflows. See class org.opentripplanner.api.resource.Routers for this “routers API” for options 3 and 4 and relevant endpoints. These endpoints are obviously dangerous, so are only accessible with authentication over HTTPS. See http://opentripplanner.readthedocs.io/en/latest/Security/ for how to set that up.

However if you just want a highly available OTP router for the whole Netherlands you may want to check with Plannerstack, who work closely with OVAPI to provide such a router. You can find more information at http://www.plannerstack.com/. This is a commercial service but run by a group of FOSS-friendly people so depending on your goals and motivations lower-cost collaboration may be possible.

Regards,

Andrew

On 25 Oct 2017, at 10:03, Enrico Rotundo <enrico...@gmail.com> wrote:

I'm deploying OpenTripPlanner 1.2.0 using this Docker container. I need graph persistence so I'm not using the --inMemory flag: first I run `--build` and then `--server`. At building time I have a GTFS (static) file in the build directory that contains scheduled trips of roughly 3 months; how do I update the graph in the background and re-load it into OpenTripPlanner?

I could do:

1. Fetch the latest archive with `wget http://gtfs.ovapi.nl/new/gtfs-nl.zip`
2. Build graph
3. Re-load OTP instance with new graph

Is there an automatic procedure I can use instead?

--
You received this message because you are subscribed to the Google Groups "OpenTripPlanner Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to opentripplanner-users+unsub...@googlegroups.com.

[Leonard Ehrenfried]

The outline by Andrew is still valid a few years on but you could also try using the GTFS-RT facilities that OTP has. If you don't need to add new stations, but only new routes between existing ones it could be sufficient.

We are doing something similar with user added carpool "routes" and it works ok.

This approach means that you are abusing the GTFS-RT spec ever so slightly and depending on your use case have to bend OTP to your will. That said, it took us less than a week to get this working.

Best

Leonard

--

To unsubscribe from this group and stop receiving emails from it, send an email to opentripplanner-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/opentripplanner-users/b0f221cf-096c-4369-bdbf-10de0018b523%40googlegroups.com.

[Liam Neville]

I have been looking at GTFS-RT and wondered if I could use this. I'm not sure it would be possible though because in my app, users or transport providers can create, update and delete routes (based on a 'from' and 'to' gelocation) and also update timetable information for these routes (it will all be private transport). I'm currently thinking along the lines of building something that creates GTFS from our database on maybe a daily or hourly basis and then the OTP service will be re-deployed with the new files. Making sure there is no down time would be the biggest issue here. Do you have any idea of the technical detail of what options 3 and 4 would involve?

Thanks,

Liam

To unsubscribe from this group and stop receiving emails from it, send an email to opentripplanner-users+unsub...@googlegroups.com.

[Leonard Ehrenfried]

If you don't need the update instantly but you can wait up to an hour, you can do it as you are planning to: 

- build graph

- drop it into the correct folder

- tell OTP to reload the graph (don't know the API calls by heart, but they are in the documentation

As Andrew mentioned,if you want to have zero downtime, you need twice as much memory.

Or you use a load balancer, start a second instance with the updated data and then switch your traffic to that instance.

--

  Leonard Ehrenfried

To unsubscribe from this group and stop receiving emails from it, send an email to opentripplanner-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/opentripplanner-users/f5596f64-1a44-4cf0-9811-47e136de997e%40googlegroups.com.