Fwd: Network Rail Open Data Platform - Notification of upcoming change

[Peter Hicks (Poggs)]

All,

Did anyone *not* receive this email yesterday?  I think it could do with saying *what* the criteria for blocking are, something I don’t think anyone’s been able to extract from CACI.

Peter

Begin forwarded message:

From: DSG_NROD.Support <DSG_NROD...@caci.co.uk>

Subject: Network Rail Open Data Platform - Notification of upcoming change

Date: 4 August 2023 at 16:42:41 BST

Dear Network Rail Open Data User, You are receiving this email as a registered user of the Network Rail Open Data platform on https://publicdatafeeds.networkrail.co.uk/ and/or https://datafeeds.networkrail.co.uk/. This email is to inform you that the platform is being updated to introduce stability improvements and some filtering of problematic connections (high volumes of constantly repeated connections or connection attempts).  The update will be deployed on Wednesday 9th August. Clients properly connecting to the Network Rail Open Data real-time feeds should not encounter any changes or differences, however, if your client application (using OpenWire or STOMP protocols) does not properly manage its connections, then your connections may be blocked until you have corrected your client configuration. If you have any queries please respond to this email.   Network Rail Open Data Support.

 

 

This electronic message contains information from CACI International Inc or
subsidiary companies, which may be confidential, proprietary,
privileged or otherwise protected from disclosure.  The information is
intended to be used solely by the recipient(s) named above.  If you are not
an intended recipient, be aware that any review, disclosure, copying,
distribution or use of this transmission or its contents is prohibited.  If
you have received this transmission in error, please notify us immediately
at postm...@caci.co.uk
Viruses: Although we have taken steps to ensure that this e-mail and 
attachments are free from any virus, we advise that in keeping with good 
computing practice the recipient should ensure they are actually virus free.

CACI Limited. Registered in England & Wales. Registration No. 1649776. CACI House, Avonmore Road, London, W14 8TS

[petermount]

I did get it, I presumed it went out to everyone

[Juhani Pirttilahti]

Peter,

I agree with you. The definition of connecting properly is not clear. I got the bridge setup with exponential backoff and all that one could say are proper... and it still gets blocked on occassion.

Juhani

[Chris Wraith]

I agree with what has been said already - it'd be good for the criteria for being blocked to be clear to all.

I also think it is important that any user who is blocked is notified in a timely fashion using their registered email or contact details. In an ideal world, they could be notified before they are actually blocked to give them a chance to resolve the issue first provided it isn't causing immediate stability problems for the NROD platform. Users can also be informed of the process to get unblocked if they have been blocked: it's a good opportunity to provide guidance on best practices.

All the best

Chris

--
You received this message because you are subscribed to the Google Groups "A gathering place for the Open Rail Data community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openraildata-t...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/openraildata-talk/29e1a805-d4b7-4899-b527-407ec67c2420n%40googlegroups.com.

[Adrian Bradshaw]

Chris

It was good to see the layout all together today. It's a startling achievement.

As promised, here are the stats as of this morning, but I'll get them updated to include this weekend.

As of 9am this morning...

• Quad Cam has had 11,800 live views and 13400 in total.
• Cycle Cam has had 14600 live and 16200 total.
• Aerial Cam has had 6000 live and 7700 total.
• The MT3 webage should hit 20000 today.

I'll send further figures as we move through the 6 week run.

I left 7 boxes of leaflets under the fiddle-yard section. Hopefully that will be enough!

Cheers,

Adrian

---

Adrian Bradshaw – Director

Railcam UK Limited

@: adr...@railcam.uk

Railcam.uk – Bringing The Lineside To Your Armchair

Railcam UK Limited | Registered in England and Wales | registration number 10167844

Thomas House | Meadowcroft Business Park | Pope Lane | Whitestake | Preston | PR4 4AZ

-------- Original Message --------

Subject:	Re: [openraildata-talk] Re: Network Rail Open Data Platform - Notification of upcoming change
Date:	2023-08-05 17:33
From:	Chris Wraith <ch...@indigenously.co.uk>
To:	openrail...@googlegroups.com

To view this discussion on the web, visit https://groups.google.com/d/msgid/openraildata-talk/CABEBWFFH8ThYn80axP%2BgmeM9qWfn-Tmqm1KMb42HuWE-VDp%3DyA%40mail.gmail.com.

[Adrian Bradshaw]

Apologies.... I seem to have had a senior moment!!!

Please ignore the message about website stats and leaflets (which relate to Pete Waterman's Making Tracks III model railway layout at Chester).

---

Adrian

-------- Original Message --------

Subject:	Re: [openraildata-talk] Re: Network Rail Open Data Platform - Notification of upcoming change

Date:	2023-08-05 19:06
From:	Adrian Bradshaw <adr...@railcam.uk>
To:	openrail...@googlegroups.com

To view this discussion on the web, visit https://groups.google.com/d/msgid/openraildata-talk/f89fffa0f6032c8b245cb6405c8a2aef%40railcam.uk.

[Evelyn Snow]

Hi all,

I'd echo everything said in this thread, if CACI wants us to adhere to their
definition of good behaviour, it might be helpful if they'd tell us what that
is.

The wording of the email was ambiguous to me, it could be construed either as
blocking a client temporarily automatically until it behaves, or banning a
user from connecting entirely until they can commit to following these unknown
rules.

The first would be in line with what they did with the old public data feeds
platform, in banning clients which connected too rapidly and were answered with
an ERROR frame. I'm unsure whether the new feeds platform does this, but this
made durable subscriptions unpleasant to use, because you'd be banned for an
hour or so if you reconnected before their end realised the connection had
dropped.

Of course, if they're proposing indefinite bans, that would be a significant
escalation.

Evelyn

2023-08-05T17:33:35+0100 Chris Wraith <ch...@indigenously.co.uk>:

[Tom Cairns]

There was an action from the early July forum if they could actually update the Good Practice guide on the wiki with their latest view on what that was with the recent upgrades. I note they haven't updated that yet - but I wonder if they felt that there was no need to update it.

Tom

> -----Original Message-----
> From: openrail...@googlegroups.com <openraildata-
> ta...@googlegroups.com> On Behalf Of Evelyn Snow
> Sent: Monday, August 7, 2023 12:24 PM
> To: openrail...@googlegroups.com
> Subject: Re: [openraildata-talk] Re: Network Rail Open Data Platform -
> Notification of upcoming change
>

> --
> You received this message because you are subscribed to the Google Groups
> "A gathering place for the Open Rail Data community" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to openraildata-t...@googlegroups.com.
> To view this discussion on the web, visit
> https://groups.google.com/d/msgid/openraildata-

> talk/ZNDUbXK5lJGl0TbY%40evelyn.moe.

[Tom Cairns]

Peter and I had a chat with CACI arranged by Network Rail just now and touched on this email and the outage from a few weeks ago. We have another chat next week so please let either/both of us know if there's anything that anyone wants to raise. We are aware that we are the privileged few in being able to have these discussions - and someone from NR did ask us to make sure that we are getting the whole communities view across.

To pick up on my previous email, they were reminded again about the action to update the Good Practice guide and they did nod that it needs reviewing.

The upgrade on Wednesday looks like it is for an upgrade to ActiveMQ 5.18.2 and there was mention of the following issue as being of interest: https://issues.apache.org/jira/browse/AMQ-9283. There is a plan to reissue the comms about the update with a bit more detail and also supply a time window.

The cause of the approved platform outage ... basically "an issue" with a memory leak somewhere in the platform relating to their specific ActiveMQ configuration setup and user behaviour. (Or at least that's what I noted down!) CACI stated that they have to monitor it and some of the service restarts we see are directly related to trying to prevent a lockout of the nature that we saw. An RCA has gone to NR which we made comments about sharing with the wider community. The upgrade to the public platform on Wednesday is one attempt at looking at resolving that but they do not know whether it'll completely solve it.

They are looking at other upgrades including a move to ActiveMQ Artemis on their brokers and there are some intentions to test that. There is specific interest in anyone with *STOMP* client processing data direct from the brokers and not bridging the data into a local broker - so if you are interested on helping out on some testing please let us know and we can pass your details along.

They are keen on re-earning everyone's confidence in the platform(s); and to be fair the conversation was fairly open and it felt like a step that we have been waiting for some time. Whether this bears fruit remains to be seen.

> talk/PRAPR02MB785855A7E6B02A157BC4BB4CE70CA%40PRAPR02MB7858.e
> urprd02.prod.outlook.com.

[Evelyn Snow]

Seems that CACI updated the good practice guide on the wiki themselves yesterday

Change diff here: https://wiki.openraildata.com/index.php?title=Good_Practice&curid=308&diff=3195&oldid=1872

Evelyn

2023-08-07T16:57:48+0000 Tom Cairns <t...@swlines.co.uk>:

> To view this discussion on the web, visit https://groups.google.com/d/msgid/openraildata-talk/PRAPR02MB785826B0C145C5E659F4F0A9E70CA%40PRAPR02MB7858.eurprd02.prod.outlook.com.

[Peter Hicks (Poggs)]

> On 9 Aug 2023, at 09:52, Evelyn Snow <eve...@evelyn.moe> wrote:
>
> Seems that CACI updated the good practice guide on the wiki themselves yesterday
>
> Change diff here: https://wiki.openraildata.com/index.php?title=Good_Practice&curid=308&diff=3195&oldid=1872

Good spot! I think their description is a little bit off though and doesn’t make sense.

“Loop on receive, not on connection” - I think their intent is to say “Use long-lived connections, rather than connecting, receiving, processing and disconnecting all the time”. I’m not sure what they mean by “one connection per feed”, because it’s entirely possible to have a single TCP connection and receive messages from multiple topics. I think their use of “feed” is too broad here.

“Use separate client ids per client” - if you duplicate a client ID, the second time you reconnect, your attempt will be rejected as client IDs are designed to be unique. In fact, ActiveMQ will log errors and the previous version of NROD before the ‘big rebuild’ handled these very badly indeed, apparently

I’ll reach out to them and suggest rewording.


Peter

[Tom Cairns]

Following up here after another call yesterday.

The patch last week to AMQ has *not* resolved the memory leak. The alternative route of ActiveMQ Artemis is therefore being pursued and this is now in testing. For those who have been in touch with me about it, thank you - CACI should be in touch soon if they haven't already for testing. The plan is that this is to be evaluated for a few weeks and then it will be thrown out wider.

We once again highlighted the comms around the outages late last week and this has been noted and will be followed up on.

It would also be really useful more generally if people were able to say here whether they are subscribing via STOMP or OpenWire to the service, and what libraries/clients you are using to do that.

> talk/PRAPR02MB785826B0C145C5E659F4F0A9E70CA%40PRAPR02MB7858.eu
> rprd02.prod.outlook.com.

[Russell Bowman]

Hi,  

I'm using OpenWire Apache.NMS.ActiveMQ 2.1.0 installed via Visual Studio's NuGet Package Manager. My use is fairly limited in scope TD, Trust, VSTP over one connection bridging to a local broker, but I have not seen any of the stability problems I used to see (3/4 years ago) when I was using STOMP.

Russell

[Chris Wraith]

I use the same bridged ActiveMQ over Openwire approach that Russell described above to consume TD, TRUST, VSTP and TSR data. This has been the same for a few years now.

Anecdotally, I’ve found that Openwire seems to handle error and recovery scenarios better than STOMP.

On 15 Aug 2023, at 17:34, Russell Bowman <russell....@gmail.com> wrote:

Hi,  

To view this discussion on the web, visit https://groups.google.com/d/msgid/openraildata-talk/e2cc32bf-f6c9-4b00-9876-4f9a19f70521n%40googlegroups.com.

[Evelyn Snow]

Hi Tom,

When I get around to adding the Network Rail feeds back to my relay, it'll be
over STOMP, using the library Barytherium.

I'm currently also consuming the Darwin push port (OW, Camel 3.18.2), and KB
incidents (STOMP, Barytherium 0.5.0) feeds.

I have a strong preference against OW (there's no libraries for
Erlang/Elixir), I use STOMP when possible, I'd likely use AMQP if it were
offered.

Evelyn

2023-08-15T16:24:09+0000 Tom Cairns <t...@swlines.co.uk>:

> To view this discussion on the web, visit https://groups.google.com/d/msgid/openraildata-talk/PRAPR02MB7858A28EB4738B92C50C8769E714A%40PRAPR02MB7858.eurprd02.prod.outlook.com.

[WantStuff]

Same as Russel; OpenWire on .NET using the Apache.NMS.ActiveMQ 2.1 libraries.

Similarly, I saw very poor reliability with STOMP especially around the durable subscription reconnections. I would not recommend STOMP.