Gateway timeout for Darwin HSP (on RDM)

[ycohui]

Hello everyone,

Recently I am trying around with the Darwin HSP but quite a significant portion results are HTTP 504 Gateway Timeout.

I wonder anyone had faced the same problems or this might related with the recent downtime of Darwin Push port?

Thanks,

ycohui

[Jez Smith]

Looking into this now.  Will come back with any updates.

Jez

[Jez Smith]

Ycohui

This one is resolved. Apparently, sometimes due to heavy load or data size the publisher system takes time to respond, we increased the holding time. instead of giving a timeout we are waiting for 5 mins.

Hopefully this should alleviate the issues, but if they persist, please just raise a support ticket.

Cheers

Jez

[D K Ntiamoah]

Hi Ycohui,

This endpoint https://api1.raildata.org.uk/1010-historical-service-performance-_hsp_v1/api/v1/serviceMetrics is still responding 504 Gateway Timeout for me, also with a Rate Limit of one request per second that is really restrictive for batch processing unless there is another source to collect all historic performance data for each day. I am currently using the serviceDetails operation on the API but I’m running into 429 Spike Alert violation exceptions unless I limit my application to 1 request per second which means for over 24k requests per weekday the process is running for a long time. The hsp-prod.rockshore api for serviceDetails is still available and doesn’t seem to have any rate limit, but I don’t want to risk it going down mid operations. Anyone with a similar issue or a work around/ solution would be greatly appreciated.

Regards

Danny

--
You received this message because you are subscribed to the Google Groups "A gathering place for the Open Rail Data community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openraildata-t...@googlegroups.com.
To view this discussion, visit https://groups.google.com/d/msgid/openraildata-talk/204103c5-0c77-4d00-af33-c1f587fdcd88n%40googlegroups.com.

[Peter Hicks]

Hi Danny

On Thursday, 19 March 2026 at 09:03, 'D K Ntiamoah' via A gathering place for the Open Rail Data community <openrail...@googlegroups.com> wrote:

This endpoint https://api1.raildata.org.uk/1010-historical-service-performance-_hsp_v1/api/v1/serviceMetrics is still responding 504 Gateway Timeout for me, also with a Rate Limit of one request per second that is really restrictive for batch processing unless there is another source to collect all historic performance data for each day. I am currently using the serviceDetails operation on the API but I’m running into 429 Spike Alert violation exceptions unless I limit my application to 1 request per second which means for over 24k requests per weekday the process is running for a long time. The hsp-prod.rockshore api for serviceDetails is still available and doesn’t seem to have any rate limit, but I don’t want to risk it going down mid operations. Anyone with a similar issue or a work around/ solution would be greatly appreciated.

HSP was never designed for bulk scraping. I worked on design and testing of the original system in 2011/2012 and although it never went live, it was​ load-tested to ensure it handle a certain number of requests per second. Although it didn't go live, the replacement HSP was a new codebase and I never got to do any testing on it. Throughout the years, it's been 'unavailable' for extended periods due to 'high load' but I don't think anyone's ever found out how many requests per second it can​ handle, and how many it is handling at any one time.

Some years ago, I suggested publishing an extract of historical data in CSV format to allow others to do bulk computation without scraping the API, but (like most other ideas, such as 'monitoring NROD' and 'communicating outages with the user community') it got ignored. My personal opinion is that nobody wanted to expose the fact that it's probably got substantial gaps in data and errors, but that's my personal view.

I apologise for the negativity - but watch this space, hopefully I'll convince the industry to go forward with some of my 'low hanging fruit' ideas for making the right data available in the right format for bulk analysis soon!

Peter

[D K Ntiamoah]

Hi Peter,

Thanks you so much for responding to my query, I have a work around for the bulk collection of HSP serviceDetails.

However I’m curious as to why both API/s are responding with errors  for serviceMetrics

https://hsp-prod.rockshore.net/api/v1/serviceMetrics -> 504 Gateway Time-out

https://api1.raildata.org.uk/1010-historical-service-performance-_hsp_v1/api/v1/serviceMetrics -> 504 Gateway Timeout

I’m stuck, which of these can I use for one off querying for my application. All help greatly appreciated.

Regards

Danny

--
You received this message because you are subscribed to the Google Groups "A gathering place for the Open Rail Data community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openraildata-t...@googlegroups.com.

To view this discussion, visit https://groups.google.com/d/msgid/openraildata-talk/U6W6_bGQXvb5jd45rYY1h1_z_ABY21irFYVnibdUL5txrWMlyV5cCY5QGfNBFrsz4c8R0BZh1efq2oNk1fTBeuJvrd2h0Onj4hktK4o-mg0%3D%40poggs.co.uk.

[Peter Hicks]

Hi Danny

On Thursday, 19 March 2026 at 11:28, 'D K Ntiamoah' via A gathering place for the Open Rail Data community <openrail...@googlegroups.com> wrote:

However I’m curious as to why both API/s are responding with errors  for serviceMetrics

https://hsp-prod.rockshore.net/api/v1/serviceMetrics -> 504 Gateway Time-out

https://api1.raildata.org.uk/1010-historical-service-performance-_hsp_v1/api/v1/serviceMetrics -> 504 Gateway Timeout

I’m stuck, which of these can I use for one off querying for my application. All help greatly appreciated.

The RDM URL (raildata.org.uk) is simply a proxy for the hsp-prod URL.

Peter

[D K Ntiamoah]

Thanks Peter,

 

Is there a more reliable API for serviceMetrics that you know of?

Regards

Dany 

--
You received this message because you are subscribed to the Google Groups "A gathering place for the Open Rail Data community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openraildata-t...@googlegroups.com.

To view this discussion, visit https://groups.google.com/d/msgid/openraildata-talk/UTYzeYGKA-E5H27Qvpxuty6xPZ94CqPEICb53CAYcn434KCLT2aj9J0-UPJvxkp9zFDgipRcFYcKbtgpZD5BQv0rQ7Qgl8rLRvkhFcGpjtM%3D%40poggs.co.uk.

[ycohui]

Hi Jez,

Thanks for the update. I just had a chance to test the HSP endpoint again, and it still appears to be down.
I also noticed that both the RDM and NRDP endpoints are still timing out (504) after about 1 minute. Could you please confirm whether this might be due to an API configuration issue on my end, or if the service is still experiencing high request loads?

Thanks,

ycohui