Archives of Darwin data via NRDP

340 views
Skip to first unread message

Peter Hicks (Poggs)

unread,
Jul 5, 2017, 4:37:15 PM7/5/17
to A gathering place for the Open Rail Data community
All,

I’m archiving all Darwin messages being sent out via the NRDP (thanks to a spare box and some disk space), and I’m considering making the archive public.

I currently have logs going back to 21st May, and I can upload June’s data in a few days. Incidentally, if anyone else has been archiving messages, timetable/reference data and snapshots, please drop me an email off list if you can help me tidy up my archive of data for 23rd - 26th June.

Nobody else seems to be doing this publicly, and storage is reasonably cheap. I’d be happy to extend my code to upload files to Amazon S3 each day, making a week or a month’s worth of data available. More than that will be “please contact me”.

Does anyone have views on the above?


Peter

Linus Norton

unread,
Jul 5, 2017, 4:40:03 PM7/5/17
to Peter Hicks (Poggs), A gathering place for the Open Rail Data community
Hi Peter,

That would be potentially useful for me (and many other people too probably).

Cheers,

Linus

--
You received this message because you are subscribed to the Google Groups "A gathering place for the Open Rail Data community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openraildata-t...@googlegroups.com.
To post to this group, send an email to openrail...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Kevin Fullerton

unread,
Jul 6, 2017, 4:02:29 AM7/6/17
to A gathering place for the Open Rail Data community
I've been doing a similar thing with the Network Rail data (daily/full
timetables in CIF format and TRAIN_MVT and VSTP feed messages) which
get uploaded to S3 daily - I've got a pretty complete set of messages
from this year (with a few gaps in the feeds where my STOMP
reconnection logic didn't work for whatever reason) which may or may
not be useful at some point to some one as well?

Cheers

Kevin
> To post to this group, send email to openrail...@googlegroups.com.

petermount

unread,
Jul 6, 2017, 4:50:59 AM7/6/17
to A gathering place for the Open Rail Data community
Same here:
  • Darwin:  D3 PushPort, daily reference/ timetable files 
  • NR: TimeTable CIF's, TRUST, TD, RTPPM
Some like trust only go back a few months when I started logging them, but some go back a year or so.

Peter

Samuel Brown

unread,
Jul 6, 2017, 5:26:19 AM7/6/17
to A gathering place for the Open Rail Data community
Hi all, 

I would be very interested in accessing an archive of raw TRUST and TD messages for Kent and Sussex over the last couple of months -- there are a couple of periods where our database went down and I missed a few days' worth of messages. I don't want to inconvenience anybody, but let me know if anyone has this set up in an easily accessible location.

Thanks
Sam Brown

petermount

unread,
Jul 6, 2017, 5:59:53 AM7/6/17
to A gathering place for the Open Rail Data community
I've only got them to the entire UK not just Kent/Sussex From March 10th

Anyhow I've generated a file list of what I've got available:
Files ending in .tbz2 are tar files, compressed with bzip2.

Samuel Brown

unread,
Jul 6, 2017, 6:12:14 AM7/6/17
to A gathering place for the Open Rail Data community
Brilliant, thank you very much!

Samuel Brown

unread,
Sep 7, 2017, 9:21:56 AM9/7/17
to A gathering place for the Open Rail Data community
Hi Peter, 

Thanks again for these helpful file links from a couple of months ago. I was wondering whether you had the same data archived for August? We had a total outage from 25th - 30th August, and I would like to try and fill that gap if possible.

Kind regards
Sam

On Thursday, July 6, 2017 at 10:59:53 AM UTC+1, petermount wrote:

petermount

unread,
Sep 7, 2017, 10:03:25 AM9/7/17
to A gathering place for the Open Rail Data community
They should be there, just try changing the url to the correct date.


I've had a couple of issues on some days not being archived, but the archive should be in sync up to the start of this week.

Samuel Brown

unread,
Sep 7, 2017, 10:18:03 AM9/7/17
to A gathering place for the Open Rail Data community
OK brill, thank you!
Sam

Samuel Brown

unread,
Sep 7, 2017, 10:27:07 AM9/7/17
to A gathering place for the Open Rail Data community
Peter, 

The URLs seem to work great from 31st August onwards, but I'm getting an error in the TD and Trust feeds (which is what I'm after) for dates between 25th to 30th August. e.g. https://cdn.area51.onl/archive/rail/td/2017/8/25.tbz2 gives a "This XML file does not appear to have any style information associated with it" error. Do you know what could be going on here?

Thanks again
Sam

Andrew Belcher

unread,
Mar 21, 2018, 5:39:12 AM3/21/18
to A gathering place for the Open Rail Data community
Hi Peter,

Do you still have these running and available? I'm interested in the last couple months, but in particular yesterday (21/3/18).

Cheers,
Andrew

Mike Kynaston

unread,
May 9, 2018, 6:05:04 PM5/9/18
to A gathering place for the Open Rail Data community
Hi Peter,
Is it possible to get access to your NR CIF and TRUST archive files at all please?

Cheers
Mike

Edwin Reynolds

unread,
May 21, 2018, 5:53:30 AM5/21/18
to A gathering place for the Open Rail Data community

Hi,

Thanks Peter for your putting up the data at https://networkrail.opendata.opentraintimes.com/mirror/tsr/. It has been really useful to me. Would anybody be willing to put up some SCHEDULE files? (as at the moment I can only see how to get them from accessing the feeds, and I don't really need live data at all).

Thanks,
Edwin


On Wednesday, July 5, 2017 at 9:37:15 PM UTC+1, Peter Hicks wrote:
Reply all
Reply to author
Forward
0 new messages