NR public data feeds outage
132 views
Skip to first unread message
James Uppington
Jun 6, 2023, 12:07:14 PM6/6/23
to A gathering place for the Open Rail Data community
I was only thinking the other day about how stable the NR public data feeds have been recently, but it appears we're suffering from another data feed issue at the moment.
No messages received on any realtime feed between 16:29 at 16:58 and a "peer did not send his wire format" error on auto reconnect.
TM feed looks to have recovered and is acting fairly normally now. TD feed looks to be very flaky with very low message rate and high latency.
Status dashboard (https://nrodcaci.grafana.net/public-dashboards/960fa54d94884dc7abd1f5ab9c70df7e?orgId=1) also shows some odd stats at the moment with -1 million messages per 5 minutes!
Frustrating.
James Uppington
Jun 8, 2023, 8:01:45 AM6/8/23
to A gathering place for the Open Rail Data community
Anyone else seeing another issue on publicdatafeeds at the moment?
Connecting fine, but with no messages being received on TM or TD feeds. VSTP is working.
Last messages on both feeds received at 12:26.
Status dashboard looks normal and shows messages flowing.
Stuart Gordon
Jun 8, 2023, 8:07:57 AM6/8/23
to A gathering place for the Open Rail Data community
Yes, same issue here, some of the sub queues were still working though TRAIN_MVT_XX_TOC, but not all areas etc
James Uppington
Jun 8, 2023, 8:10:10 AM6/8/23
to A gathering place for the Open Rail Data community
Just come back at 13:09.
Stuart Gordon
Jun 8, 2023, 9:43:59 AM6/8/23
to A gathering place for the Open Rail Data community
Feed is still unstable for me
Although - any chance the dashboard has got its timezones mixed up and actually only shows status delayed by an hour? Screenshot taken at 14:40 BST, issue on here shows issue starting at 13:20 BST, but it actually started at 12:20 BST? So grafana is an hour behind?
James Uppington
Jun 8, 2023, 9:51:40 AM6/8/23
to A gathering place for the Open Rail Data community
Yes, it's been incredibly unstable since for me too with wild swings in latency and message rates, and some messages going missing completely.
I think the dashboard timezone is correct, but I suspect it's measuring messages going onto topic queues, not whether or not anyone is capable of receiving them.
The outage with no messages on some queues earlier was likely after the point the dashboard is measuring what's going on, and the instability since is likely something before than the topic queues we subscribe to, hence why the graph is showing the instability but not the earlier issue.
Reply all
Reply to author
Forward
0 new messages