TD Latency

[David Higginbottom]

Hi, we're seeing some latency creeping in on the TD feeds for NROD

Our metrics looks fine, and the server is healthy so we're having to investigate where the bottle neck is.

If people could report the issues they are seeing here it would be useful for our diagnosis.

Thanks

[Phil Wieland]

Looks like the same problem as yesterday morning and afternoon.

The latency of about 60 seconds isn't so bad, the real problem is the missing messages which leave duplicate trains all over the diagrams.

Phil

[Where-in-Sussex]

Ditto what Phil says.

It seems we have about a minute or so of lag, not really increasing or decreasing, but messages are being missed.

I'm not sure, but it seems like there is a message processing issue somewhere as yesterday morning and evening peaks were the same, but it was ok off peak, when the message rate was lower.

The missing messages are not something we usually see with just a 1 minute lag, as it usually takes 5 minutes before they start dropping off; so for me this is not one of the more regular issues.

[Where-in-Sussex]

Interestingly the lag pattern for the last two mornings has been almost identical... blue yesterday and red today

[Mark Rawlinson]

Did you manage to get the the bottom of what was causing the morning TD feed failures last week in the end?

If so, is it the same 'root cause' this week, too?

MARK

On Tuesday, April 10, 2018 at 8:13:07 AM UTC+1, David Higginbottom wrote:

[Adrian Bradshaw]

It seems to me that this isn't a bottleneck as such. If it were, then the latency would vary through the day, according to the prevailing message-rate - as we've seen i the past - latency climbing to many minutes or even hours, until someone reboots/restarts something somewhere and the recovery begins, albeit frustratingly slowly.

 

Starting last week, we have seen this new symptom, whereby the latency sits consistently at about 60 seconds. I'm not sure if the number of dropped messages is greater at busier times - perhaps someone can comment?

 

Adrian Bradshaw – Administrator (volunteer)

Railcam UK Limited

@: adr...@railcam.uk

Railcam.uk – Bringing The Lineside To Your Armchair

Railcam UK Limited | A company limited by guarantee | registration number 10167844

3rd Floor | 207 Regent Street | London W1B 3HH

Railcam UK Ltd is operated on a not-for-profit basis entirely by volunteers

--
You received this message because you are subscribed to the Google Groups "A gathering place for the Open Rail Data community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openraildata-t...@googlegroups.com.
To post to this group, send email to openrail...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.