National Rail Data Portal - Update from December

[Rail Delivery Group]

Dear All,

Please find attached notification from Rail Delivery Group of a significant upgrade to the National Rail Data Portal (NRDP) service, due in December this year. Enhancements in this release include:

• Darwin feed will be upgraded from Version 12 to Version 16 (latest current version, upgraded back end connectivity between NRDP and Darwin to improve stability)

• New TLMS (Train Location & Movement Service feed), including:
  

• Connecting Train Identifier (CTI): Linking physical train to a Darwin logical schedule using GSM-R + Darwin information.
• Customer Information GPS Processor feed- receiving GPS positional information from on-board trains and outputting this onto a line of route against a RID.

• Rail Settlement Plans DTD Portal Feeds (Routeing, Fares and Timetable feeds, updated weekly)

It is important that anyone currently consuming Darwin data from the NRDP read this notification as the upgrade will mean existing method of providing Darwin data in v12 format will cease 3 months after the upgraded service goes live.

Please feel free to post any questions or comments here or send to nationalra...@raildeliverygroup.com.

Regards,

RDG

[Tom Thorpe]

This seems like great news :-) It's going to be great to have this extra data.

Having had a quick read through the "P75301004 Push Port Data Interface Specification Issue 21" doc, it does seem like version 16 just layers extra data on top of the existing version 12 and no fields have really been removed or renamed/restructed, so it's likely not many - if any - changes will be needed for someone currently running version 12, other than to implement the new data if desired? 

Seems like perhaps the bigger differences will be with how we connect - is that a fair conclusion or have I missed anything big?

Thanks,

Tom

Please feel free to post any questions or comments here or send to nationalraildatafeeds@raildeliverygroup.com.

Regards,

RDG

[petermount]

That's about what I've manage to extract from those docs & a brief scan of the xml schemas. I think the biggest difference will be how we connect, but I think I need to sit down & identify what's changed inside the schemas.

[Peter Hicks]

As far as I'm aware, it's still a Stomp or OpenWire connection to either a queue or a topic.  You can ignore the document about connecting - the means of access via the National Rail Data Portal (NRDP) will be broadly in line with what it is now.

The document in the ZIP file deals with connections directly to Darwin, which are handled slightly differently.

Peter

Please feel free to post any questions or comments here or send to nationalra...@raildeliverygroup.com.

Regards,

RDG

--
You received this message because you are subscribed to the Google Groups "A gathering place for the Open Rail Data community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openraildata-t...@googlegroups.com.
To post to this group, send email to openrail...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

OpenTrainTimes Ltd. registered in England and Wales, company no. 09504022.

Registered office: 13a Davenant Road, Upper Holloway, London N19 3NW

[Rail Delivery Group]

Hi all,

Just to confirm what Peter is saying, the full specification files sent out with this notification include details of connecting to Darwin's own AMQ service. The NRDP handles this for you and you connect to NRDP. This model allows Darwin to be isolated so that any issues in the open data channel does not impact on other services powered by Darwin (e.g. NRE website/apps, on-station boards, TOC apps, etc...)

There will be some changes between how you connect to NRDP for Darwin data now, and how you connect in the future, although they should hopefully not be massively different - full details will follow in due course.

Regards,

RDG

[Chris Bailiss]

Hello RDG

With December now being only a couple of weeks away, are you in a position now to firm up dates and any other details around changes to how we connect?

Thanks

Chris

[Rail Delivery Group]

Hi Chris,

 

Thanks for your message and apologies for not updating sooner. Regarding timescales, we are anticipating that the following upgrades will be made by w/c 17^th December:

 

• Darwin feed will be upgraded from Version 12 to Version 16 (latest current version, upgraded back end connectivity between NRDP and Darwin to improve stability)

• Rail Settlement Plans DTD Portal Feeds (Routeing, Fares and Timetable feeds, updated weekly)

Owing to slight delays with the Train Location and Movement Service Project, we anticipate those feeds will be made available in New Year (January).

 

Conscious that the 17^th December isn’t too far away, we’ll make sure to provide connection details for the newest version of Darwin (V16) in the next couple of weeks. Additionally, in the next couple of days, we are arranging (with Peter and at his suggestion, credit where credit is due!) to provide 24hrs of recorded v16 data for people to become familiar with.

 

Thanks

RDG

[Tom Thorpe]

Hi RDG,

When you mention "Darwin feed will be upgraded from Version 12 to Version 16" w/c 17th Dec, would you mind confirming (from the original announcement) that v12 and v16 will be available in parallel for a short time after launch, and that nothing has changed from that point?

The test data that was provided is much appreciated, but definitely cant beat the upgraded version being available in parallel for any unanticipated issues

Thanks,

Tom

[Rail Delivery Group]

Hi,

Yes, we will definitely be making v12 and v16 available in parallel for a period of 3 months after we go live with v16. So v12 should remain available via existing connections until roughly the end March 2019. Anyone who hasn't migrated by then will unfortunately lose their access (we had hoped to provide two versions in parallel as part of the standard service, rather than just a limited time, but were unable to do so for technical/commercial reasons).

RDG

[Tom Thorpe]

Thats great, thanks :-)

[WantStuff]

Hi RDG,

What’s going to happen with the Pport log and snapshot files during this parallel period? I assume the files will also be produced in parallel but what will we need to do to differentiate between the two versions?

Thanks
Martin

[Arun Kumar]

Will this update include TLS v1.2 support?

Please feel free to post any questions or comments here or send to nationalraildatafeeds@raildeliverygroup.com.

Regards,

RDG

[Alex Holland]

Can you confirm whether this will affect OpenLDBWS, or just Darwin Push Port?

If OpenLDBWS is affected, will this only apply to certain older versions of the schema? We have some legacy applications which have not been updated in some time and are still using the 2012-01-13 (ldb5) and 2016-02-16 (ldb9) versions of the schema.

Cheers,

Alex

[Rail Delivery Group]

@WantStuff - I can confirm that the v16 snapshot and log files will be added to the same FTP server but in a different folder. The v12 snapshots and logs will continue in the existing folder exactly as now (during the parallel run). Note, the snapshot and logging process for v16 is slightly different to the existing process for v12. Under normal operation for v16, a snapshot will be taken every three hours and log files will be created every 5 mins thereafter, until the next snapshot. This should allow users who disconnect for longer than the queue (currently 5 mins) to only have to consume a maximum of 3 hours worth of data to get back in sync.

@Arun - I think this question relates to the LDB and LDBSV APIs. This upgrade relates to the push port.

@Alex - Similar answer to above, this upgrade doesn't impact on the LDB or LDBSV APIs. This is because NRDP only provides links to sign-up to these services - once tokens have been generated the APIs are provided by a ring-fenced part of Darwin.

RDG

[Arun Kumar]

Hi RDG,

  Thanks for you're response. Our website (airport) currently uses openldbws and uses TLS1.0 for connection to openldbws. However, we are currently undergoing a re platform of our site and the new solution (SaaS) does not allow us to have TLS 1.0, which means we cannot use openldbws to display the current travel information for our passengers. 

  Do you know what is the roadmap for openldbws to support TLS 1.2.

Thanks

Arun

[Rail Delivery Group]

Hi Arun,

We are due to go live with Darwin v4 later this month which should finally enable TLS 1.2 for all LDB and LDBSV channels, including the open versions. I've attached a couple of screenshots from an SSL Labs server test on our staging site which already has v4 installed and this indicates a drastic improvement in overall grade and TLS support when compared to the current live site.

Regards,

RDG

[Rail Ale Fan]

Hi RDG,

As much as I'm looking forward to the update, the last week before Christmas is a very strange time to go live with a major platform upgrade.

Lots of us are totally reliant upon Push Port now with our websites and apps having outgrown the web services and whilst I know that the upgrade is being brought in at least in some part to address the recent issues, things are for now at least stable.

I personally would be quite happy to wait until January when we're all back to work as normal and can expect the usual fast response on here from yourselves, your suppliers and each other rather than worry about things going wrong, let's say last thing on Friday 21st and having no idea when things might return to normal!

Cheers!

[Arun Kumar]

Hi RDG,

   Thanks for your update. Do we have a exact date when Darwin v4 will be available and is there any option for getting access to early adaptors for a development environment. We have a live project in development which is currently waiting for TLS 1.1 support. It would be useful to know the date of upgrade and early access possible before the upgrade for development purpose.

Thanks

Arun

[petermount]

Going by an earlier email they are deploying to to live tonight from 9pm so it would be good to know when we'll have access to it & the new connection details.

Here's the last one I got:

Rail Delivery Group - Online Services - Upcoming changes to Darwin Hello - this email is to notify you of some up coming changes being release this week

Darwin Release 4.0 deployment –  On Wednesday the 12th of December we will be deploying the latest build of Darwin   We will be deploying to live in the evening starting 21:00 and should finish circa 00:00.  This is a major release so will cause some disruption to services and feeds. It is recommended customers disconnect from Darwin during this time:     Push Port Users (including Darwin CIS) – It is recommended you disconnect completely at 21:00 until the outage is complete however if you stay connected then you may experience disconnection and reconnection multiple times as we deploy the new build across our environments. Live Departure Boards Feeds (CCLDB) - From 21:00, for approximately 2-3 hours, information will show however it may start to go stale and remain so until the deployment is over. NationalRail.co.uk & RTJP Web service - During this time the Realtime Journey Planning Webservice, CCJP and the NRE website will not contain any realtime information. Timetable journey planning will remain in operation. History (LDB History & History Lite) - It is recommended you do not use History during this time.   An email will be sent to confirm when Darwin is back up.

New Functionality 
 
This release is a major release to the infrastructure and inner workings of Darwin to modernise and future proof the way Darwin works in the background. The Key benefits of this build is:
 

• The Darwin Infrastructure will now have the ability to 'auto scale' in high demand. This means rather than having computing power to handle a set amount of demand all the time, we can now scale up automatically during a major rail/weather disruption that causes huge spikes in demand, the computing power automatically increases meaning in theory we now have unlimited capacity without the system going down.
• Changes and improvements to system resilience and performance.
• Users of the LDB/LDBSV web services will be able to use TLS1.2 security.
• Some prerequisites to prepare for Darwin using industry GPS data in its predictions

 
Bug Fixes included in this Release
 

• INC0039877 - Darwin History "Advance arrows" not functioning correctly
• INC0049744 – Issue accessing services from Historic Departure Board
• INC0050095 / INC0054102 - Cannot see data in historic departure boards (No CRS code)

Train Loading Push port output is missing wta, etd, pta and ptd fields

[Rail Delivery Group]

For those that haven't see the notification, the Darwin upgrade went in as planned last night. There are some teething issues (as one would expect with the biggest upgrade in 3 years, nearly 12 months in the making) but most things went as planned. This includes the enablement of TLS1.2 (finally) for the LDB/LDBSV APIs

@Rail Ale Fan - the intention is to deploy the new version of NRDP alongside the existing version during a 3-month transition window, without touching the operation of the existing service. The two services will then run in parallel until the old version is finally switched off. Any issues with the new version should therefore have no impact on the operation of the existing version. That said, it is now looking likely the release will be postponed until after Christmas anyway - further information will be posted here in due course.

RDG

[Arun Kumar]

Thats great stuff RDG!!! 

I tested the LDBWS on TLSv1.2 and it is working fine. I understand the documentation site is down as discussed in the other thread, are we expecting any new version of WSDL or the current (2017-10-01) will be the latest?

Thanks

Arun