KnowledgeBase Live Incidents Feed

[Malcolm Bovey]

Hi,

Does anyone else have a subscription to the KnowledgeBase Live Incidents feed?  I recently setup a listener service to this expecting this to be a continuous feed of individual incident messages.  Instead it seems like the messages are sent in "batches" of 20-30 messages that are sent approx. every 8 hours, after which my client seems to get disconnected from the feed.  Just wondering if anyone can confirm if this is "normal" behaviour?

Thanks
M

[Matt Jones]

Hi,

No, this is not normal. I've been subscribing to this feed for some months and normally messages are pushed as and when they are created/modified by NRES. This can be observed by comparing changes with what's on nationalrail.co.uk.

I note that the static incidents document (https://opendata.nationalrail.co.uk/api/staticfeeds/5.0/incidents) is also out of date.

This behaviour of "batching" has started in the last few days or so.

I have raised this as a ticket with CACI (https://caci.freshservice.com/support/tickets/26717)

So glad I am not the only one! :-)

Matt

[Malcolm Bovey]

Thanks - good to know and typical of my luck to start subscribing just as it breaks!   Hopefully CACI fix it soon

[Matt Jones]

I'll post on here with any updates.

[Matt Jones]

As a matter of interest, what client are you using to connect? I'm connecting using C#/dotnet with the Apache NMS library.

[Malcolm Bovey]

I'm using a Python-baed client using Stomp.py, running on AWS Fargate.  I was able to use pretty much the same container I use to subscribe to Darwin PushPort with a few tweaks.

[Matt Jones]

OK that's cool. Am using dotnet here, so at least that proves it's not a problem with the client stack.

[Matt Jones]

Are there anyone working for NRES (or with contacts at NRES) who can help push a solution along? I've raised a ticket with CACI and provided them with detail but I'm not sure anything is being done. If anyone can help get some movement on this I would be extremely grateful.

Matt

[Peter Hicks]

Hi Matt

Can you describe the impact it's having on your systems? That usually gets a response!

Peter

--
You received this message because you are subscribed to the Google Groups "A gathering place for the Open Rail Data community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openraildata-t...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/openraildata-talk/db16d789-eea6-4252-b6c5-3e6a6c05f285n%40googlegroups.com.

OpenTrainTimes Ltd. registered in England and Wales, company no. 09504022.

Registered office: Suite 1-3, Hop Exchange, 24 Southwark Street, London SE1 1TY

[Matt Jones]

Thanks Peter.

I've put this on the CACI ticket.

It means that an incident (a new or updated incident) may not be distributed to consumers for up to two hours, which means that consuming systems may be up to two hours out of date at any point in time. For instance, if an incident is cleared at 1pm and not sent out to consumers until 3pm - for two hours consumers (and their users) will think that the incident is still ongoing, when it's all gone!

M

[Malcolm Bovey]

If it helps, here is a visual display of how activity on the feed has looked over the past few days - long periods of inactivity with occasional sudden spikes before immediately falling back to zero

[Matt Jones]

Here's my chart which goes back a little further. You can clearly see a change in the profile of how incidents are received around November 5th/6th (Friday/Saturday)

[Matt Jones]

CACI writes...

"Hi Matt,

We investigated this issue and determined that this was an upstream problem. We were in the process of reporting this to National Rail but has since recieved a notification from their Service Desk that there is an issue causing out of data information on the KB feed. They are currently investigating this."

 Hopefully resolved soon.

Matt

 

[Malcolm Bovey]

Yes, saw there was an email to the National Rail list at 1325 claiming that the issue has been resolved.  However within the last few minutes another batch of 11 message has arrived all at once, so I'm not so sure!

[Malcolm Bovey]

Another burst just came through, definitely doesn't seem to be fixed :(

[Matt Jones]

Yeah same here. I'll find someone at NR to chase it with.

[Matt Jones]

Hmm. Looks like incidents are starting to come through separately and not in batches. Are you seeing the same thing Malcolm?

[Malcolm Bovey]

Yes the behaviour is definitely looking more like how I expected, although I wasn't subscribing to the feed when things were "normal" so don't have a reference.  I'll keep an eye on it

[Malcolm Bovey]

I'm still finding my client is getting regularly disconnected from the feed, about once an hour.  Is this happening for you too?  Perhaps I need to set a higher heartbeat timeout, but I have no problems running (essentially) the same code to subscribe to Darwin PushPort

[Matt Jones]

Hmm. I don't seem to be getting that - I'm running dotnet inside a docker container using the Apache NMS dotnet client. I need to put in a bit more logging because I don't think I can actually see if it is disconnecting.

Are you ACK'ing the messages when they are received?

[Malcolm Bovey]

Yes, ACKing all the messages as they come in.  My client is configured to automatically reconnect so it's not really a problem but would ideally this behaviour would not occur.  Perhaps the heartbeat timeout of 15s is too low for such a low volume feed.

[Malcolm Bovey]

After a brief period of "normal" behaviour this morning, the feed seems to have reverted to posting messages in batches

[Matt Jones]

I see the same thing :-(

I'll re-open the ticket.

[Malcolm Bovey]

Looks like there has been some improvement since 1800 this evening.  Hopefully more sustained than last time!

[Matt Jones]

An update on this.

I re-opened the ticket (26717) with CACI on Wednesday 17th November and indeed the incidents started flowing again.

They asked me today whether I'd be happy to close the ticket; I agreed since the feed had been well-behaved since then.

Then this afternoon, more or less as soon as the ticket was closed, it started happening again. It's starting to feel like someone is doing this deliberately :-(

I've reopened the ticket and offered my help to CACI and National Rail so we can solve this on a more permanent basis.

Matt

[Malcolm Bovey]

Hi Matt,

Yes, exactly the same experience here :(

M

--
You received this message because you are subscribed to a topic in the Google Groups "A gathering place for the Open Rail Data community" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/openraildata-talk/mCGZL7auj3w/unsubscribe.
To unsubscribe from this group and all its topics, send an email to openraildata-t...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/openraildata-talk/2ff887ec-0e0a-446e-a96d-5859f27c5835n%40googlegroups.com.