Masters project query

[Jacob Marron]

Good Afternoon,

I am currently doing a masters project using the network rail data feed. I intend to gather a variety of data sources from network and national rail. 

 I've attempted to apply to "Rail data marketplace" as a student, and had my application rejected. Stating an academic representative from my university would be required to apply rather than myself.

 I also noticed that there is national rail OpenData, by the same company, rail delivery group. My query is, will I need to make two seperate applications for these two sites, through an academic representative? I apologise if this is a bit of an ignorant question, this is all new to me, and I need to try and develop an understanding quickly.

My other query, is I am trying to use the STOMP client for network rail, and it seems that the module 'pyxb' has been discontinued, and as a result it won't generate the classes using pyxbgen. Did anyone have any luck using a different python module getting this to work?

Thanks

Jacob Marron

[Gaelan Steele]

Hi Jacob,

For National Rail Open Data, the sign-up process is fully self-serve - once you fill out the form, you get access, with no further trouble. (An exception is the Live Departure Board Web Service, which has its own sign-up process, currently involving an email and several weeks’ delay.)

For the Rail Data Marketplace, that’s quite unfortunate - if they will be making a habit of this, it’s a significant downgrade from the old “open data” systems where anyone can sign up with no ceremony. In any case, many of the data feeds on the Rail Data Marketplace are licensed freely (under the Open Government License, “OGL3”), and therefore can be freely redistributed by anyone who obtains it. If the data you’re after falls into this category, I’ll gladly get a copy under my own RDM account and pass it on to you, if that would be helpful.

Best wishes,

Gaelan

--
You received this message because you are subscribed to the Google Groups "A gathering place for the Open Rail Data community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openraildata-t...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/openraildata-talk/dce74f71-c64e-430e-8bb3-d0104616d66fn%40googlegroups.com.

[Adam Williams]

> I've attempted to apply to "Rail data marketplace" as a student, and had my application rejected

My personal opinion: It's actually quite concerning to hear that sign-ups are being arbitrarily denied to this new platform for reasons that really don't make any sense to me. Rail Delivery Group have been given a significant amount of money by the taxpayer (circa £5m, which it would appear Tata Consultancy Services have done quite well out of) to develop RDM as an open data platform in order to further plans to "simplify data access" and deliver on recommendations from the William-Shapps Plan for Rail to modernise contracts and to enable "open data sharing".

Is anyone aware of where this eligibility criteria for sign-ups is actually documented, if it's written down at all? It frankly feels like a lot of unnecessary red tape that didn't used to exist and acts as a brand new barrier to entry. This is rather ironic to me when the stated aim of the RDM platform is "to remove barriers to data sharing, [open] up access to rail industry data and [encourage] innovation" (https://raildata.org.uk/helpAndInformation/aboutRDM).

The Open Data Institute are pretty clear about this with their definition of open data:

> Open data is data that anyone can access, use or share. Simple as that. When big companies or governments release non-personal data, it enables small businesses, citizens and medical researchers to develop resources which make crucial improvements to their communities.

I'm not really sure which part of "anyone can access, use or share" is difficult to grasp.

Jacob - did you sign up as an "Individual not affiliated with a company" or an "Academic organisation"? It's very much a shot in the dark, but I wonder if you picked the latter, that that might've been the issue. There seems to be a desire to tie every account to an organisation (I have no idea why), and I can just about see why, if you're requiring that, you'd want it to be managed by an academic member of staff.

Adam

[Gaelan Steele]

On Feb 9, 2024, at 8:19 PM, Adam Williams <adam.lu....@gmail.com> wrote:

My personal opinion: It's actually quite concerning to hear that sign-ups are being arbitrarily denied to this new platform for reasons that really don't make any sense to me. Rail Delivery Group have been given a significant amount of money by the taxpayer (circa £5m, which it would appear Tata Consultancy Services have done quite well out of) to develop RDM as an open data platform in order to further plans to "simplify data access" and deliver on recommendations from the William-Shapps Plan for Rail to modernise contracts and to enable "open data sharing”.

Yes, very much agreed.

The RDM’s general approach makes sense for data feeds that are paid or otherwise carry restrictive licensing, as you want to ensure users are authorized to enter into contracts on behalf of the institutions they supposedly represent. But for OGL-licensed data, the amount of paper tape - both major things like the manual signup process, and small papercuts like having to individually “subscribe” to each feed, is rather silly. Freely licensed data should be freely downloadable - Network Rail’s releases of things like delay attribution data (on their own website - they’ve also released it on the RDM, with the normal issues) and the sectional appendices are more or less the gold standard here.

I’ve said as much on the RDM feedback form - hopefully they’ll at least consider it.

Gaelan

[Evelyn Snow]

Very much agreed with everything said.

The RDM is a very strange beast, it's overbuilt on the one hand, but lacking in every
way that really matters, and full of bugs so obvious that reporting them feels pointless. It seems
like it was created by someone who has no idea what data - never mind rail data - is, no idea who's
using it, and no idea what it's being used for (so, a consultancy!)

The signup process assumes that you're an organisation, the security stance assumes that your
account will have paid licences attached (2FA for an account on a platform which presently offers
me nothing of value is a ridiculous concept to me). There's not even any way to automatically
download files, a gaping - and incredibly obvious - hole in the featureset.

I also think that the RDM belongs to a mindset which supposes that the main obstacles to data
sharing are technical, rather than being related to institutional cowardice; Of course, publishing
data isn't technically difficult, nor is it expensive, but I have to imagine that the RDG are
less than willing to confront the idea that they themselves are a significant obstacle.

I'll believe that the RDG has seen the light when they reverse their U-turn and finally offer CTI
through NRDP, but I don't think that day will ever come.

Not important (though I found it funny), the RDM newsletter from this month claimed 1100 "active
data contracts". If that includes open licenses, I represent about 2% of that number.

Evelyn

[Jacob Marron]

Hello,

Since my query I've managed to get an academic representative to sign up on my behalf so I am awaiting this to be processed.

Thanks

Jacob 

To view this discussion on the web, visit https://groups.google.com/d/msgid/openraildata-talk/33bc6ae3-d363-4d35-9a9e-0ffff1e68174n%40googlegroups.com.

[Tom Cairns]

From my recent discussions with NR, they are being actively pushed into releasing new data (and moving all existing data releases) solely into RDM. I’ve already had discussions with NR along the lines of increased friction, and the use of data is more constrained in RDM due to its platform agreements, which I have to say I’m finding a very disappointing step.

 

Particularly if these series of events are down to what appears to be a largely spurious reason relating to the that they’ve built into the platform.

 

FWIW, I am not registered to RDM due to various qualms with the platform both - contractual and technical. Particularly the fact that it’s run by RDG and they don’t *appear* to have any boundaries about entities within RDG / RSP that can see the details of commercial revenues given they’re party to the fees(!), etc. For open stuff, that’s fine – but for commercial stuff I’d be worried about that leaking outside of solely RDM and that information then being used for other reasons.

 

But maybe that’s just me 😊

 

Tom

 

--

You received this message because you are subscribed to the Google Groups "A gathering place for the Open Rail Data community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to

openraildata-t...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/openraildata-talk/9A697F2F-6702-4DE7-903C-937B430F19D5%40canishe.com.

[Guido Eco]

Good evening Jacob,

This is an interesting. As others have stated I also see this as an unhelpful and potentially retrogressive step. 

For what it's work, I have an RDM account set up on the basis of a self-managed personal pseudo-organization, rather than through my place of work or affiliated academic institution. As such, I would be interested to see if an application from, say, the "Rail Innovation Group" or "The People's Front for the Liberation of Rail Data" would then be allowed.

I then note that as an arms-length government organization that Network Rail's has a "data is open by default" policy and is subject to Freedom of Information requests. Given the current funding of passenger operators, I do wonder how long it will be before the ONS make a similar ruling.

Kind regards,

Will

[Peter Hicks]

Hi Jacob

On Fri, 9 Feb 2024 at 14:09, Jacob Marron <jacobm...@gmail.com> wrote:

 

My other query, is I am trying to use the STOMP client for network rail, and it seems that the module 'pyxb' has been discontinued, and as a result it won't generate the classes using pyxbgen. Did anyone have any luck using a different python module getting this to work?

Sorry for missing this initially - but I have good news.

pyxb (https://github.com/pabigot/pyxb) has indeed been deprecated, and there is a fork called PyXB-X (https://github.com/renalreg/PyXB-X) available.  An open issue (https://github.com/renalreg/PyXB-X/issues/13) reports that pyxbgen does not work on Python 3.12.

The solution is easy - don't use Python 3.12 for the moment.  I've updated the sample code at https://github.com/openraildata/stomp-client-python/commit/3b3bd37a17623978c5733683091a61f79080e5ec to reflect this.

You may be able to use Python 3.11 to run pyxbgen and generate the code, but you'll be on your own if you do.

Peter