Unable to access the Station XML dataset

[Greg Williams]

Dear All,

I've just registered onto the NR Data Portal, but am unable to access the stations XML dataset.  

My registration appears to be complete apart from the fields at the bottom

(see attached screen shot 1)

I am wondering what would count a valid input for any of these fields below.  I've tried a variety of texts in all and some of the fields, but the form still thinks I need to enter something else.

Following this "registration" and then unable to access the XML dataset.  The result is shown in Screenshot 2.

Can anyone help out a newbie please?

Since I'm behind a Govt Firewall, might this cause issues on this point?

"Information in KB is made available in XML files that are downloaded through a web browser. Access to the URLs which contain the XML is enabled by whitelisting IP addresses, meaning the user must have a static IP Address to gain access to the data. A list of all available Knowledgebase XML feeds is shown below."

Regards

Greg Williams

[NRE Online Services]

Instructions on how to access this feed from the NRDP as open data are found here (including authentication)

http://nrodwiki.rockshore.net/index.php/KnowledgeBase#On_Demand_Data_Feeds

The section you have quoted relates to accessing these feeds under licence directly from NRE, which is a different mechanism (and has the IP restriction which the Open data through NRDP does not)

NRE Online team

[Greg Williams]

Dear NRE Team,

I've made some progress this evening with trying to make a POST request, but am still not getting a response from the Authentication server.  (Screenshot given below with password and username obscured for information).

From reading other posts here and your comments below, it appears that a direct registration with NR is required to access the station information dataset.  What is the process of registering for access to this dataset?  and what other steps might I be missing to get a token?

Regards,

Greg Williams

[Peter Hicks]

Hi Greg

Try it again without HTTP/1.1 on the URL.

Peter

--
You received this message because you are subscribed to the Google Groups "A gathering place for the Open Rail Data community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openraildata-t...@googlegroups.com.
To post to this group, send email to openrail...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Greg Williams]

Dear Peter,

Thanks for the quick response.  I tried it again without HTTP/1.1 on the URL but the end result was the same.  Any further advice?

• SSL certificates are turned off
  
• The timeout setting are set to infinity
• I've guessed that the client certificate is the same as the my user crentials.  If not, how do I obtain a client certificate?

A screenshot is attached (credentials are obscured)

Regards,

Greg

To unsubscribe from this group and stop receiving emails from it, send an email to openraildata-talk+unsubscribe@googlegroups.com.
To post to this group, send email to openraildata-talk@googlegroups.com.

[Peter Hicks]

Hi Greg

The next thing to check is whether you can access the same URL through a web browser.

If you can get ant kind of reaponse, then you're possibly using a proxy server that you need to configure Postman to use.  And if you can't, that suggests a networking issue, probably specific to you or the network you're on.

Petet

To unsubscribe from this group and stop receiving emails from it, send an email to openraildata-t...@googlegroups.com.
To post to this group, send email to openrail...@googlegroups.com.

[Greg Williams]

Dear Peter,

I can access the URL through a web browser, but I am unsure of the format needed to pass the keys and values as part of the URL.  Without them, the site obviously rejects them (Http Response attached below)

Trying in the office  (rather than at home) I get a different error message (Postman (as chrome app rather than desktop app) response attached below)

"Error while sending request: Failed to execute 'set Request header' on "XMLHttp Request: 'Raw Payload is not a valid HTTP header field name.'"

Thanks for your patience.  This is all bit of a steep learning curve for me.

Regards,

Greg

Petet

To unsubscribe from this group and stop receiving emails from it, send an email to openraildata-talk+unsubscribe@googlegroups.com.
To post to this group, send email to openraildata-talk@googlegroups.com.

[Peter Hicks (Poggs)]

Hi Greg

Don’t worry, we all need to learn and you’re in the right place!

I’ll go over the whole thing here, as it’ll likely help somebody else in the future.  I’ll also put this on https://wiki.openraildata.com/ when I get a chance.

Importing the attached JSON file to Postman using the ‘Import’ button at the top of the window.  Then, click on the ‘gear’ icon at the top right and select ‘Manage Environments’ - create one such as “National Rail Data Portal”, and add the following keys and values:

Key: hostname

Value: datafeeds.nationalrail.co.uk

Key: username

Value: the username you subscribed with

Key: password

Value: the password you’ve set

Then, select the environment you created and make a POST call using the ‘Authenticate’ call.  You’ll see output such as:

{

    "username": “user...@example.com",

    "roles": {

        "Role(id=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx, name='ROLE_KB_USER')": true,

        "Role(id=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx, name='ROLE_D3_USER')": true,

        "Role(id=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx, name='ROLE_HSP_USER')": true

    },

    "token": “<token>"

}

If you don’t see ROLE_KB_USER, ensure you’ve clicked ‘On Demand Data Feeds’ after you log in to https://datafeeds.nationalrail.co.uk/ - you’ll need to click Update to save it - then try again.

Copy the whole string from ‘token’ and add this in to the environment on Postman with the key ‘token’.  Then, make another API call - you should see some XML come back rather than an error from Apache Tomcat.

Let me know how you get on.

Kind regards,

Peter

[Greg Williams]

Dear Peter,

Could you please clarify what you mean by "make a POST call using the ‘Authenticate’ call" -is this the same as Authorisation:Basic?

I've imported the JSON file and created the environment as outlined, but I think my authentication call is not properly formatted.  Screen shots below, along with my registration for the NR Data Portal.

Incidentally, putting these exchanges onto the public wiki would be a *very* good idea, provided you change the names to protect the innocent!

Regards,

Greg

<HTTP Response.docx><Postman Reponse.docx>

[Peter Hicks (Poggs)]

Hi Greg

It doesn’t look like you’re using the collection you’ve imported.

If you click on ‘Collections’ to the right of ‘History’, you should see something like:

Double-click on Authenticate and make sure it looks like this:

Then, click ‘Send’.

For the Stations XML feed, just copy an existing request to https://{{hostname}}/stations

Peter

On 21 Jul 2017, at 11:47, Greg Williams <greggwi...@gmail.com> wrote:

Dear Peter,

[Greg Williams]

Dear Peter,

Thank you very much for this.  I've been able to extract the XML file now and the next step is to get it into Excel.  If you like, I could try and write up a note explaining how I did this for inclusion into the wiki.

Regards,

Greg

[Dan Sinclair]

Hi All,

I'm relatively new to this API, and thanks to this thread in particular I was able to get this working a couple of months ago.

However, I now can't seem to be able to get a list of stations. Nothing is returned. I can get data from all other endpoints (TOCs, Ticket Restricktions, Network Service Indicator, Incidents) using Postman but not from the stations endpoint.

I noticed it not working in my application, and thinking that it was my code I thought I would try it in Postman, but to no avail.

Any thoughts?

Thanks in advance.

/Dan

[Rail Ale Fan]

Hi Dan,

Stations XML feed seems to be working OK at the moment, have a go with the following JavaScript/WSH

user = "YOUR_USERNAME";
pass = "YOUR_PASSWORD";
feed = "https://datafeeds.nationalrail.co.uk/api/staticfeeds/4.0/stations";
http = new ActiveXObject("WinHttp.WinHttpRequest.5.1");
http.Open("POST","https://datafeeds.nationalrail.co.uk/authenticate",false);
http.SetRequestHeader("Content-Type","application/x-www-form-urlencoded");
http.Send("username="+user+"&password="+pass);
json = http.ResponseText;
eval('auth='+json);
http = new ActiveXObject("WinHttp.WinHttpRequest.5.1");
http.Open("GET",feed,false);
http.SetRequestHeader("X-Auth-Token",auth.token);
http.Send();
WSH.Echo(http.ResponseText);

Edit YOUR_USERNAME and YOUR_PASSWORD and save as stations.js and then from a Windows command prompt:

cscript /NoLogo stations.js > stations.xml

Cheers

[Michael Flynn]

>>  have a go with the following JavaScript/WSH

Spent half the morning trying to get access with Postman but with no success.  Took me just half a minute using Rail Ale Fan's JavaScript/WSH.  Maybe wiki should include this method as an option.  Thanks :)

[Andy West]

Hi, Any idea how to do this in C#?

[Allan Blundell]

WebRequest request = WebRequest.Create( "https://datafeeds.nationalrail.co.uk/authenticate");

request.Method = "POST";

request.Credentials = new WebCredentials("{Username}", "{Password}");

WebResponse response = request.GetResponse();

To get the response body:

string body = string.Empty();

using (StreamReader reader = new StreamReader(response.GetResponseStream())){

        body = reader.ReadToEnd();

}

How you process the body and get the token is down to you

Allan

[Allan Blundell]

Here is a .NET fiddle with the code:

https://dotnetfiddle.net/TAd2Qi

Replace the username and password with your details 

Allan

[Simon Jones]

In case anyone else wants to use curl, it looks like the urlencoding for username and password works when you separate the fields:

curl --data-urlencode "username=m...@gmail.com" --data-urlencode "password=foo$@bar99!" https://datafeeds.nationalrail.co.uk/authenticate

Hope this helps,

Simon

blackpepper.co.uk

We collaborate with clients to create not just the best software, but the right software. On time. On budget. On your terms. With Black Pepper it’s your software, your way.

Black Pepper Software Limited, registered in England and Wales. Registered number: 03763392. 

Registered office: Warwick House, 32 Clarendon Street, Leamington Spa, Warwickshire CV32 4PG