How to get rid of obfuscated headcodes?

[Juhani Pirttilahti]

Hi all,

During recent days I've seen quite a few enthusiastic sites go in the open showing freight headcodes as is. Are these sites accessing feed from NR directly or how is this possible? What should one do to get access?

Also, I've noted that some freight services have now appeared unobfuscated on the open data feeds but not all.

Kind Regards,

Juhani

[Matthew Burdett]

The new data feed is only available to a limited number of people at the moment before public release.

As for headcodes prior to activation, presumably a reverse lookup for last activation of that service and then filling in the blanks for the rest of the schedules validity 

--
You received this message because you are subscribed to the Google Groups "A gathering place for the Open Rail Data community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openraildata-t...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/openraildata-talk/a779dda1-1c25-4140-8b8a-afc1d18a7f44n%40googlegroups.com.

[Peter Hicks]

Hi Juhani

On 29 Jul 2022, at 21:20, Juhani Pirttilahti <juhani.pi...@gmail.com> wrote:

During recent days I've seen quite a few enthusiastic sites go in the open showing freight headcodes as is. Are these sites accessing feed from NR directly or how is this possible? What should one do to get access?

There is work underway to build a new infrastructure for NROD.  We’re finally at the stage where it’s gone live - for some of the most visible sites such as OTT, RTT and Railcam.  There may well be others at this early stage.

I’ve already given the new platform the seal of approval from my side, and the plan is to make the same code live on a separate and unconnected platform.  Think of it like a production and development environment - the data coming in is identical, but the production environment only has ‘production' systems connected to it.  I think we can all agree that the amount of damage you can do by having rogue or wonkily-coded Stomp client - or a downstream system that processes messages one by one rather than de-queueing messages locally, has resulted in NROD falling over a lot before.

What’s not been worked out yet is how to identify what systems are ‘production’ - but I understand the process for giving a user access to the ‘production’ platform is going to be straightforward and I hope, well-publicised and easy to do.  My feeling is that when communication, generally, about the new platform goes out to all users, there will be a “How do I get qualified for the production system?” section to it.

Also, I've noted that some freight services have now appeared unobfuscated on the open data feeds but not all.

There was some work done in the last week or two about obfuscation - do you know what’s changed?  Is it more GBRf services being un-obfuscated?  If so, that is likely because the de-obfuscation of FOC data is done by train service code, and if that list of services codes isn’t regularly updated, it’ll result in trains running under new service codes being obfuscated by default.

As an aside, I was told that there was no obfuscation on the new platform - but there is.  I’m using my standing to actively challenge this, by putting more data out in the clear in an attempt to get the data in the clear for everyone, not just me.  From what I’ve heard so far, we might be winning.

Peter

[Juhani Pirttilahti]

Hi Peter,

There is work underway to build a new infrastructure for NROD.  We’re finally at the stage where it’s gone live - for some of the most visible sites such as OTT, RTT and Railcam.  There may well be others at this early stage.

I’ve already given the new platform the seal of approval from my side, and the plan is to make the same code live on a separate and unconnected platform.  Think of it like a production and development environment - the data coming in is identical, but the production environment only has ‘production' systems connected to it.  I think we can all agree that the amount of damage you can do by having rogue or wonkily-coded Stomp client - or a downstream system that processes messages one by one rather than de-queueing messages locally, has resulted in NROD falling over a lot before.

I am seeing the big picture now. There is a new platform?! I wish it was communicated to the community earlier. :) Yes, wonkily coded clients can do some harm. I've unintentionally done just that myself! I did not ACK each message immediately and it caused a lot of latency spikes if I recall correctly.

 

What’s not been worked out yet is how to identify what systems are ‘production’ - but I understand the process for giving a user access to the ‘production’ platform is going to be straightforward and I hope, well-publicised and easy to do.  My feeling is that when communication, generally, about the new platform goes out to all users, there will be a “How do I get qualified for the production system?” section to it.

 

Production ready or not is obviously a question that only a developer can answer. And the opinion can be slightly biased. And in my opinion the provision of a sandbox for testing would demonstrate good practice. Have you considered the possibility of sharing code for screening before access to the production feed is granted?

 

Also, I've noted that some freight services have now appeared unobfuscated on the open data feeds but not all.

There was some work done in the last week or two about obfuscation - do you know what’s changed?  Is it more GBRf services being un-obfuscated?  If so, that is likely because the de-obfuscation of FOC data is done by train service code, and if that list of services codes isn’t regularly updated, it’ll result in trains running under new service codes being obfuscated by default.

I did some research on the services I started seeing unobfuscated yesterday and identified them as Freightliner services. But not all Freightliner are treated the same! Is it possible that some contracts held by GBRf in the past have now been transferred to Freightliner and the whitelist was not updated to reflect that?

I did not have time to do full research. Hundreds of services appeared unobfuscated over last Wednesday and Thursday but now on Friday things are nearly back to as it was before.

As an aside, I was told that there was no obfuscation on the new platform - but there is.  I’m using my standing to actively challenge this, by putting more data out in the clear in an attempt to get the data in the clear for everyone, not just me.  From what I’ve heard so far, we might be winning.

This is something we have done in Finland from the beginning and no business has been affected to date. :)

Kind Regards,

Juhani

[Tom Cairns]

The obfuscation changes on the main platform last week were to remove it from Freightliner services, but it's done on a service code level so they may not have got everything covered.

Tom

---

From: openrail...@googlegroups.com <openrail...@googlegroups.com> on behalf of Juhani Pirttilahti <juhani.pi...@gmail.com>
Sent: Friday, July 29, 2022 11:28 pm
To: A gathering place for the Open Rail Data community <openrail...@googlegroups.com>
Subject: Re: [openraildata-talk] How to get rid of obfuscated headcodes?

 

--
You received this message because you are subscribed to the Google Groups "A gathering place for the Open Rail Data community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openraildata-t...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/openraildata-talk/13532022-c60e-4a7b-a64a-5b060b65c604n%40googlegroups.com.