HSP Service Details & Cancellation Structure

[Nikhil Datta]

Hi all,

I have a couple of clarification questions about how cancellations are represented across CIF and the HSP Service Details API.

1. CIF present, but missing from HSP Service Details API

If a passenger service appears in the CIF timetable but does not appear in the HSP Service Details API for a given date, is it safe to assume that the service was cancelled?

More specifically:

• Does absence from the HSP Service Details API imply the service was cancelled sufficiently early that it never entered Darwin?

• If so, is there a documented or de facto notion of what “sufficiently early” means (e.g. cancelled before activation, before schedule upload to Darwin, etc.)?

2. Blank actual times in HSP Service Details API

Within the HSP Service Details API, if a service has scheduled (expected) arrival/departure times at a stop, but the corresponding actual times are blank, does this imply that the stop was cancelled?

From analysis of a full day of HSP Service Details data:

• The only observed values in actual_ta / actual_td are numeric HHMM times or blanks

• There appear to be no alphabetic markers (e.g. “CANC”) indicating stop-level cancellation

If blank actual times do not imply stop cancellation:

• How (if at all) are cancelled stops represented in the HSP Service Details API?

• More generally, how should one identify whether a service was fully cancelled versus partially run / curtailed, using only the HSP Service Details feed?

Any clarification on the intended interpretation of these cases would be much appreciated.

Thanks in advance.

[Peter Hicks]

Hi Nikhil

On Friday, 12 December 2025 at 18:16, Nikhil Datta <nikhil....@gmail.com> wrote:

I have a couple of clarification questions about how cancellations are represented across CIF and the HSP Service Details API.

I would urge you to exercise caution when using the HSP - I don't believe it is sufficiently accurate.  The original version of HSP (which I worked on well over a decade now) never ended up going live, and a reworked version was launched some years later.  I have my doubts over the quality of the codebase and little confidence it handles all scenarios correctly.  No hard evidence though I'm afraid.

1. CIF present, but missing from HSP Service Details API

If a passenger service appears in the CIF timetable but does not appear in the HSP Service Details API for a given date, is it safe to assume that the service was cancelled?

Don't try to match up the CIF timetable with what's in Darwin - it may not be consistent.  For example, service A12345 may be in the CIF but not in Darwin - however both services may be the 1000 from A to F, calling at C, D and E, but the times may be slightly different.

More specifically:

• Does absence from the HSP Service Details API imply the service was cancelled sufficiently early that it never entered Darwin?

No, it just means that HSP doesn't have details of it.  It may have gone in to Darwin but not reached HSP due to an outage, or a coding issue.

2. Blank actual times in HSP Service Details API

Within the HSP Service Details API, if a service has scheduled (expected) arrival/departure times at a stop, but the corresponding actual times are blank, does this imply that the stop was cancelled?

I see scheduled times as what was planned in advance, and expected times as the forecasts/predictions that Darwin makes.  Since HSP is supposed to report actuals, I wouldn't expect it to show anything that isn't a concrete report.  However, if the arrival times are blank, it could mean anything - train not reported, train cancelled, or train diverted.  Or perhaps HSP not processing the message correctly.

Peter

[Nikhil Datta]

Hi Peter

Thanks a lot for the info. It’s very helpful. 

Are there any alternative data sources which would be more accurate, or is HSP the only possibility with the relevant micro performance data? 

Thanks

Nikhil 

--
You received this message because you are subscribed to a topic in the Google Groups "A gathering place for the Open Rail Data community" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/openraildata-talk/8gV0y9-XDb4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to openraildata-t...@googlegroups.com.
To view this discussion, visit https://groups.google.com/d/msgid/openraildata-talk/E1Nwe9q5KSXb0aGmSd1oAAE91GLK4wIVkgcUkyPr14o_LrH5WzYrNIwVDsLLiVsKHnsOFECE-4yzW-dBd-GiyLAjy4b0P0H2qAIPJOj-x7s%3D%40poggs.co.uk.

[Gaelan Steele]

Archived data from the Darwin push port might be an option. Peter Mount (“area51”) used to operate an extensive archive, but sadly no longer does due to health reasons; I’m not sure if a public replacement has sprung up. Depending on your exact requirements it might be good enough to start archiving yourself today. 

Best wishes,

Gaelan

On Dec 12, 2025, at 19:24, Nikhil Datta <nikhil....@gmail.com> wrote:



You received this message because you are subscribed to the Google Groups "A gathering place for the Open Rail Data community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openraildata-t...@googlegroups.com.
To view this discussion, visit https://groups.google.com/d/msgid/openraildata-talk/CAG%2BGG2MJQfwcck7_gR%2BSFSs5xhLT1WZnMGS5Tcuxc60Y%3DPJx2Q%40mail.gmail.com.

[Nikhil Datta]

Hi Gaelan

I think I’m likely to start archiving this myself as you suggested, but I was hoping to get some historic information as well.

Thanks

Nikhil 

To view this discussion, visit https://groups.google.com/d/msgid/openraildata-talk/108E48A0-AC68-4A5A-BDE8-B0D4CEA82381%40canishe.com.

[Gaelan Steele]

Ah yes, I’ve just remembered that David Wheatley now operates an archive at https://ilovetrains.co.uk/. Thanks for taking that on!

On Dec 12, 2025, at 19:29, Gaelan Steele <g...@canishe.com> wrote:



To view this discussion, visit https://groups.google.com/d/msgid/openraildata-talk/108E48A0-AC68-4A5A-BDE8-B0D4CEA82381%40canishe.com.

[Peter Hicks]

On Friday, 12 December 2025 at 19:29, Gaelan Steele <g...@canishe.com> wrote:

Archived data from the Darwin push port might be an option. Peter Mount (“area51”) used to operate an extensive archive, but sadly no longer does due to health reasons; I’m not sure if a public replacement has sprung up. Depending on your exact requirements it might be good enough to start archiving yourself today. 

The good thing about HSP is that it's the right idea - being able to look up historical running information and get some basic multi-day stats.  The bad thing is that it's slow (remember the days where it was under "high load" and returning errors for a long time?) and you can only make specific queries.

Rather than just archives of data, I think we need a bulk set of as-run data from Darwin so that somebody can download it and process it on their own infrastructure in their own way, without the complication of having to process a stream of data.

I'll mention it to RDG.

Peter

[David Wheatley]

If someone has more historical data from before 7 Sept (the earliest date in my archive) I'd be happy to take it on too.

My archive preserves the exact data I've received, but I hadn't realised that sometimes the RDM push port will send multi-line XML messages, thus the one message per line system I had set up doesn't always work properly... If you split on every <?xml I'd imagine you'd work around the issues I've caused you!

To view this discussion, visit https://groups.google.com/d/msgid/openraildata-talk/D323716B-BDB3-4539-9D02-C3492A8BBB55%40canishe.com.

[Peter Hicks]

On Friday, 12 December 2025 at 19:37, 'David Wheatley' via A gathering place for the Open Rail Data community <openrail...@googlegroups.com> wrote:

My archive preserves the exact data I've received, but I hadn't realised that sometimes the RDM push port will send multi-line XML messages, thus the one message per line system I had set up doesn't always work properly... If you split on every <?xml I'd imagine you'd work around the issues I've caused you!

Hard relate here - remember the thread on "How do you want the Passenger Train Allocation messages to be archived?".

Peter

[Nikhil Datta]

Might be a silly question but for archiving precise performance metrics akin to what is meant to be in HSP would it make more sense to use TRUST over the Darwin push port?

Cheers

N

--

You received this message because you are subscribed to a topic in the Google Groups "A gathering place for the Open Rail Data community" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/openraildata-talk/8gV0y9-XDb4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to openraildata-t...@googlegroups.com.

To view this discussion, visit https://groups.google.com/d/msgid/openraildata-talk/Z82HT4mg7aIkwqkTCjLLKXG7itBN-gLOGbEyUD4WpPavk1-Jb9CiQhTTXWPg5q59ZO0j3UVSi8hUIQQ-OCD5KlDFDHfXRjl9zpal9FjB4jk%3D%40poggs.co.uk.

[Peter Hicks]

On Friday, 12 December 2025 at 19:56, Nikhil Datta <nikhil....@gmail.com> wrote:

Might be a silly question but for archiving precise performance metrics akin to what is meant to be in HSP would it make more sense to use TRUST over the Darwin push port?

It all depends on what you're trying to do.  Darwin will give you the passenger view, so will contain additional and removed stops.  TRUST will give you the official timing figures, but with the TRUST messages published as Open Data, won't show you added or removed stops unless they're part of a VSTP schedule.

Peter

[Nikhil Datta]

Thanks. If anyone knows of an archive for TRUST as well please let me know. Thanks David Wheatley for the Darwin archiving.

Best

N

--

You received this message because you are subscribed to a topic in the Google Groups "A gathering place for the Open Rail Data community" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/openraildata-talk/8gV0y9-XDb4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to openraildata-t...@googlegroups.com.

To view this discussion, visit https://groups.google.com/d/msgid/openraildata-talk/knnoRE4roT2fR8H8VCghBHPdN-af1tc8DJWJwmMx-iyiyT3Rtim5iejniTgLAh3f-uklHWj6MixfTaBnbFnT9HhebIGrRDNTcRn9DEp6nv0%3D%40poggs.co.uk.