Open Rail Data Outage 14th May 2025
77 views
Skip to first unread message
CACI Open Data Support
Jun 27, 2025, 12:55:47 PMJun 27
to A gathering place for the Open Rail Data community
Dear Users,
As you may already be aware, Network Rail experienced a major incident that impacted one of their data centres on 14th May. This had a downstream impact on the CACI Public Data Feed platform, as our systems were no longer receiving messages from the data centre our platform was consuming from.
Following this incident, we have carried out a review of our current setup and identified improvements we will be making to mitigate the impact of such scenarios in the future:
• Firstly, we have already implemented additional monitoring and alerts to notify our service desk team if the connection to the Network Rail data centre we are consuming messages from is interrupted. This will trigger a manual process to switch the connection to the second data centre, allowing for recovery as quickly as possible.
• Secondly, we found that our current setup, which uses DNS records to connect to Network Rail’s data centres, is inadequate for disaster recovery scenarios. As such, we are working with Network Rail to implement an alternative setup using IP addresses, which should allow for much faster automatic failover in similar circumstances going forward.
This is an ongoing process of continual improvement, and further refinements should be in place by the end of Q3.
As always, if you have any questions or queries, please feel free to reach out to the team at dsg_nrod...@caci.co.uk.
Following this incident, we have carried out a review of our current setup and identified improvements we will be making to mitigate the impact of such scenarios in the future:
• Firstly, we have already implemented additional monitoring and alerts to notify our service desk team if the connection to the Network Rail data centre we are consuming messages from is interrupted. This will trigger a manual process to switch the connection to the second data centre, allowing for recovery as quickly as possible.
• Secondly, we found that our current setup, which uses DNS records to connect to Network Rail’s data centres, is inadequate for disaster recovery scenarios. As such, we are working with Network Rail to implement an alternative setup using IP addresses, which should allow for much faster automatic failover in similar circumstances going forward.
This is an ongoing process of continual improvement, and further refinements should be in place by the end of Q3.
As always, if you have any questions or queries, please feel free to reach out to the team at dsg_nrod...@caci.co.uk.
Reply all
Reply to author
Forward
0 new messages