Indexing Schedules for quick look up

[scfy12]

Hello there folks,

I have started dabbling with the Network Rail  SCHEDULE feed by taking a full download.

Could someone give me a pointer or two on how to index the data for quick lookup (eg lookup Schedules at Plymouth for 1000-1200 for Sun 27 Nov)?

Assume some heavy duty caching will be involved.

My first thought is to move the Saturday and Sunday schedules into their own database for quicker lookup, and then create groups of TIPLOCs (e.g. everything from PLYMTH - LISKARD) to be used to for a lookup?

Probably naive on my part, but just wondering how to begin?

Thanks in advance

Shaun

[Chris Wraith]

HI Shaun,

I wrote a Java library a few years ago that downloads, parses and caches the schedule data. It uses an in-memory model - so it needs to run on something with at least 2GB of RAM but it does mean you can do lookups and updates quickly. I've open-sourced it and you can find the code here and it's in Maven Central too - so you can just use it in another Java project if you like.

It's not perfect and there are likely some bugs that I don't know about and a few edge cases it doesn't handle too well but having a read through might help get you started. Any feedback is welcome of course.

All the best

Chris

--
You received this message because you are subscribed to the Google Groups "A gathering place for the Open Rail Data community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openraildata-t...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/openraildata-talk/CAF-z2GcdEzjkWWb81K4Nm4DXJ9enG-0%2Bgw%3DPGGFGjzma_zfouQ%40mail.gmail.com.

[scfy12]

Hello there Chris,

Thank you for this - I have started to have a prod round.

It has given me a little inspiration - will experiment by cutting up the schedules in different ways (perhaps by each day of the timetable period, and then create smaller Map structures by location somehow).

Thank you again!

Shaun

To view this discussion on the web, visit https://groups.google.com/d/msgid/openraildata-talk/CABEBWFHEefzGq2oNGaNZwh252np4omZvwOkx37xCsLZdcKAm6A%40mail.gmail.com.

[SimonAlexander]

@Chris nice, does that code deal with bank holidays when checking if a schedule is running? And I presume it only deals with full CIFs, not deltas?

[Chris Wraith]

Hi Simon,

The logic for the app I used to use this code in would download the full CIF every day - so the delta logic wasn't extensively tested but from a quick look at the code, it does respect the CIF Transaction Type so you should be able to load a CIF delta file after having loaded a full CIF and get the right results.

For bank holiday running, this looks like a gap. The code that parses the CIF file certainly extracts the Bank Holiday Running field but isn't passed up into the model. Changing that would be relatively easy as long as the calling code also implemented the logic for whether or not to run on a bank holiday.

All the best

Chris

To view this discussion on the web, visit https://groups.google.com/d/msgid/openraildata-talk/4711382b-b595-44ea-91d1-94455b2f2ddbn%40googlegroups.com.

[Ian Sargent]

" For bank holiday running, this looks like a gap. The code that parses the CIF file certainly extracts the Bank Holiday Running field but isn't passed up into the model. Changing that would be relatively easy as long as the calling code also implemented the logic for whether or not to run on a bank holiday."

It should be noted that the presence or absence of a code in the Bank Holiday field does not actually affect the train running dates. Its sole purpose is to add the BHX symbol to the relevant column in a printed timetable. Actual changes for bank holiday workings for each train schedule are indicated by STP overlays or STP cancellations.

[SimonAlexander]

" It should be noted that the presence or absence of a code in the Bank Holiday field does not actually affect the train running dates.   "

Ah, that's interesting - so in theory the best overlay for a schedule should always be C for a bank holiday?

[SimonAlexander]

...unless the train does actually run on the BH

[Tom Cairns]

Yep, if there’s no C and the train should otherwise be running from the rest of the information then the train is scheduled to run regardless of what any other field suggests…

 

Tom

To view this discussion on the web, visit https://groups.google.com/d/msgid/openraildata-talk/acda7943-7c9d-453a-bca7-0fa815809d60n%40googlegroups.com.