500 - Internal Server Errors (Hourly Obs)

[Kevin Davies]

Hi All,

We've been picking up hourly observations for some time with no problem, but over the last couple of days we've started to receive 500 errors:

http://datapoint.metoffice.gov.uk/public/data/val/wxobs/all/json/all?res=hourly&key=***KEY***&time=2015-08-19T6Z

(Key removed.)

Any ideas?

Cheers,

Kev

[brian.gaze]

I've raised a separate but probably related query regarding the hourly obs. In my experience it looks like the files are being populated with data up to several hours after they first appear on DataPoint. For example today I made the same call at 06:40am and 07:50am for the 04GMT file:

http://datapoint.metoffice.gov.uk/public/data/val/wxobs/all/json/all?res=hourly&time=2015-08-19T04Z&key=*****

At 06:40 the returned file was about 5KB and by 07:50am it was 48KB. My understanding was the files appear on the DataPoint server only once they contain all of the available data.

[colin.corkerton]

Hi,

I've already asked our support team to look at this. They have requested the following information to help with their investigations:

Please can you  provide:

HTTP request headers
HTTP response headers
HTTP response body

These can be gained using the command line tool "curl"

In the meantime I will pass this additional information to them.

Regards,

Colin

[Brian Gaze]

Colin,

I provided the Curl information to you yesterday. Here it is again if you didn't receive:

HTTP/1.1 200 OK

Cache-Control: private

Content-Length: 483

Content-Type: text/html; charset=utf-8

Server: Microsoft-IIS/7.5

X-AspNet-Version: 4.0.30319

X-Powered-By: ASP.NET

Date: Tue, 18 Aug 2015 06:00:13 GMT

Thanks

Brian

--
You received this message because you are subscribed to the Google Groups "Met Office DataPoint" group.
To unsubscribe from this group and stop receiving emails from it, send an email to metoffice-datap...@googlegroups.com.
To post to this group, send email to metoffice...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Corkerton, Colin]

Brian,

 

Thanks. My apologies for not picking this up. I’ve passed it on.

 

Regards,

Colin

 

 

Colin Corkerton  Forecasting and Service Delivery Team  (PWS)
Met Office  FitzRoy Road  Exeter  Devon  EX1 3PB  United Kingdom
Tel: 01392 886548 Fax: +44 (0)1392 885681
E-mail: colin.c...@metoffice.gov.uk   http://www.metoffice.gov.uk

[colin.corkerton]

The cause of the observations issue has been found.

The observations file is being created on an hourly basis close to the hour. This data is then cached for an hour until it is updated again. Therefore you are only seeing those observations available close to the hour and you won’t see the complete set of observations until an hour later. We propose to amend the caching time to 5 minutes, but this will require a release to make this change and this needs to be arranged with our release team. The change has come about following the DataPoint release last Friday, 14^th, which included a change to the source of the observations data, and the caching of the data for an hour was not picked up.

 

In the meantime it will be best to either delay the collection of observations data until just over an hour after the observations time, or routinely make a second call after an hour.

Colin

On Wednesday, August 19, 2015 at 8:44:21 AM UTC+1, Kevin Davies wrote: