Empty Forecast Responses

[Tim H]

Hello,

Since 10:18 this morning we have started to see large a number of site responses returning no data.

For example:

    Heathrow site 3772
    http://datapoint.metoffice.gov.uk/public/data/val/wxfcs/all/json/3772?res=3hourly&key=APIKEY

Please could you investigate?

Thanks in advance,

Tim

[Jeremy Holmes]

Also seeing this right now. Using a json query with this url

http://datapoint.metoffice.gov.uk/public/data/val/wxfcs/all/json/351448?res=3hourly&key=<My key removed>

returns this

{"SiteRep":{"Wx":{"Param":[{"name":"F","units":"C","$":"Feels Like Temperature"},{"name":"G","units":"mph","$":"Wind Gust"},{"name":"H","units":"%","$":"Screen Relative Humidity"},{"name":"T","units":"C","$":"Temperature"},{"name":"V","units":"","$":"Visibility"},{"name":"D","units":"compass","$":"Wind Direction"},{"name":"S","units":"mph","$":"Wind Speed"},{"name":"U","units":"","$":"Max UV Index"},{"name":"W","units":"","$":"Weather Type"},{"name":"Pp","units":"%","$":"Precipitation Probability"}]},"DV":{"dataDate":"2017-05-27T08:00:00Z","type":"Forecast"}}}

This pulled directly from a browser window. No code involved at this end.

Key bit missing is in here "DV":{"dataDate":"2017-05-27T10:00:00Z","type":"Forecast"} which should also include the "Period" object with the forecast data in.

During my own analysis (thinking it was at my end :) I noticed that it "comes and goes" in that perhaps after five or ten mins it comes back before going away again.

hth

Jeremy

[Jeremy Holmes]

On Saturday, 27 May 2017 12:56:17 UTC+1, Jeremy Holmes wrote:

Also seeing this right now. Using a json query with this url

http://datapoint.metoffice.gov.uk/public/data/val/wxfcs/all/json/351448?res=3hourly&key=<My key removed>

returns this

{"SiteRep":{"Wx":{"Param":[{"name":"F","units":"C","$":"Feels Like Temperature"},{"name":"G","units":"mph","$":"Wind Gust"},{"name":"H","units":"%","$":"Screen Relative Humidity"},{"name":"T","units":"C","$":"Temperature"},{"name":"V","units":"","$":"Visibility"},{"name":"D","units":"compass","$":"Wind Direction"},{"name":"S","units":"mph","$":"Wind Speed"},{"name":"U","units":"","$":"Max UV Index"},{"name":"W","units":"","$":"Weather Type"},{"name":"Pp","units":"%","$":"Precipitation Probability"}]},"DV":{"dataDate":"2017-05-27T08:00:00Z","type":"Forecast"}}}

This pulled directly from a browser window. No code involved at this end.

Key bit missing is in here "DV":{"dataDate":"2017-05-27T10:00:00Z","type":"Forecast"} which should also include the "Period" object with the forecast data in.

During my own analysis (thinking it was at my end :) I noticed that it "comes and goes" in that perhaps after five or ten mins it comes back before going away again.

hth

Jeremy

Quick update. If I use the xml variant then it appears to work! 

[Jeremy Holmes]

Sorry. Just some more thoughts. Don't want to teach my grandmother to suck eggs or anything but if you are natively processing in XML and then parsing to json at the last minute I would suggest that the json parser you are using is getting grumpy about some of the data content that is being fed to it. That of course makes it tricky to debug. If you guys are a Microsoft shop I have had trouble with the native json parser in the past and had to resort to a third party item (a while ago so can't remember the name) 

Just to help I thought I would include the json (non) data and the associated xml data to show the xml variant in case when the time comes to track this down it all works a treat!

json data

{"SiteRep":{"Wx":{"Param":[{"name":"F","units":"C","$":"Feels Like Temperature"},{"name":"G","units":"mph","$":"Wind Gust"},{"name":"H","units":"%","$":"Screen Relative Humidity"},{"name":"T","units":"C","$":"Temperature"},{"name":"V","units":"","$":"Visibility"},{"name":"D","units":"compass","$":"Wind Direction"},{"name":"S","units":"mph","$":"Wind Speed"},{"name":"U","units":"","$":"Max UV Index"},{"name":"W","units":"","$":"Weather Type"},{"name":"Pp","units":"%","$":"Precipitation Probability"}]},"DV":{"dataDate":"2017-05-27T10:00:00Z","type":"Forecast"}}}

xml data

This XML file does not appear to have any style information associated with it. The document tree is shown below.
<SiteRep>
<Wx>
<Param name="F" units="C">Feels Like Temperature</Param>
<Param name="G" units="mph">Wind Gust</Param>
<Param name="H" units="%">Screen Relative Humidity</Param>
<Param name="T" units="C">Temperature</Param>
<Param name="V" units="">Visibility</Param>
<Param name="D" units="compass">Wind Direction</Param>
<Param name="S" units="mph">Wind Speed</Param>
<Param name="U" units="">Max UV Index</Param>
<Param name="W" units="">Weather Type</Param>
<Param name="Pp" units="%">Precipitation Probability</Param>
</Wx>
<DV dataDate="2017-05-27T11:00:00Z" type="Forecast">
<Location i="351448" lat="51.3155" lon="0.891" name="FAVERSHAM" country="ENGLAND" continent="EUROPE" elevation="9.0">
<Period type="Day" value="2017-05-27Z">
<Rep D="S" F="16" G="29" H="79" Pp="0" S="11" T="17" V="VG" W="1" U="1">360</Rep>
<Rep D="SSE" F="18" G="18" H="78" Pp="51" S="16" T="20" V="MO" W="3" U="5">540</Rep>
<Rep D="SSW" F="19" G="29" H="70" Pp="4" S="16" T="22" V="GO" W="1" U="7">720</Rep>
<Rep D="SW" F="17" G="29" H="64" Pp="0" S="18" T="21" V="EX" W="1" U="4">900</Rep>
<Rep D="SW" F="16" G="22" H="71" Pp="3" S="13" T="18" V="VG" W="7" U="1">1080</Rep>
<Rep D="SW" F="13" G="20" H="82" Pp="1" S="9" T="15" V="VG" W="2" U="0">1260</Rep>
</Period>
<Period type="Day" value="2017-05-28Z">
<Rep D="SW" F="12" G="16" H="93" Pp="0" S="7" T="13" V="GO" W="0" U="0">0</Rep>
<Rep D="WSW" F="12" G="9" H="95" Pp="2" S="4" T="12" V="GO" W="0" U="0">180</Rep>
<Rep D="W" F="14" G="4" H="88" Pp="2" S="2" T="14" V="GO" W="3" U="1">360</Rep>
<Rep D="NNE" F="17" G="7" H="74" Pp="3" S="4" T="17" V="VG" W="7" U="5">540</Rep>
<Rep D="NE" F="19" G="9" H="66" Pp="9" S="4" T="19" V="VG" W="7" U="7">720</Rep>
<Rep D="ENE" F="18" G="13" H="70" Pp="39" S="9" T="19" V="VG" W="10" U="3">900</Rep>
<Rep D="ENE" F="17" G="11" H="69" Pp="12" S="4" T="18" V="VG" W="7" U="1">1080</Rep>
<Rep D="E" F="14" G="16" H="77" Pp="33" S="7" T="15" V="VG" W="9" U="0">1260</Rep>
</Period>
<Period type="Day" value="2017-05-29Z">
<Rep D="E" F="13" G="22" H="78" Pp="36" S="11" T="15" V="VG" W="9" U="0">0</Rep>
<Rep D="E" F="13" G="22" H="83" Pp="32" S="9" T="15" V="VG" W="7" U="0">180</Rep>
<Rep D="SSE" F="17" G="11" H="87" Pp="8" S="4" T="17" V="VG" W="1" U="1">360</Rep>
<Rep D="SW" F="21" G="13" H="80" Pp="2" S="7" T="20" V="GO" W="3" U="4">540</Rep>
<Rep D="WSW" F="20" G="22" H="74" Pp="5" S="11" T="22" V="GO" W="7" U="6">720</Rep>
<Rep D="SW" F="20" G="22" H="76" Pp="7" S="9" T="21" V="GO" W="7" U="3">900</Rep>
<Rep D="SW" F="18" G="20" H="82" Pp="7" S="9" T="19" V="VG" W="7" U="1">1080</Rep>
<Rep D="SSW" F="14" G="20" H="87" Pp="18" S="11" T="16" V="VG" W="7" U="0">1260</Rep>
</Period>
<Period type="Day" value="2017-05-30Z">
<Rep D="SW" F="13" G="18" H="91" Pp="5" S="9" T="14" V="VG" W="7" U="0">0</Rep>
<Rep D="SW" F="12" G="18" H="91" Pp="1" S="9" T="14" V="VG" W="2" U="0">180</Rep>
<Rep D="WSW" F="13" G="18" H="85" Pp="2" S="9" T="14" V="VG" W="3" U="1">360</Rep>
<Rep D="WSW" F="15" G="22" H="68" Pp="2" S="11" T="17" V="VG" W="3" U="4">540</Rep>
<Rep D="WSW" F="17" G="25" H="59" Pp="8" S="11" T="19" V="VG" W="8" U="6">720</Rep>
<Rep D="WSW" F="17" G="25" H="62" Pp="4" S="11" T="19" V="VG" W="7" U="3">900</Rep>
<Rep D="WSW" F="16" G="20" H="69" Pp="4" S="11" T="18" V="VG" W="3" U="1">1080</Rep>
<Rep D="SW" F="14" G="16" H="86" Pp="3" S="9" T="15" V="VG" W="0" U="0">1260</Rep>
</Period>
<Period type="Day" value="2017-05-31Z">
<Rep D="W" F="12" G="13" H="87" Pp="2" S="7" T="13" V="VG" W="0" U="0">0</Rep>
<Rep D="W" F="10" G="9" H="88" Pp="1" S="7" T="11" V="VG" W="0" U="0">180</Rep>
<Rep D="WNW" F="12" G="11" H="81" Pp="2" S="4" T="13" V="VG" W="1" U="1">360</Rep>
<Rep D="WSW" F="15" G="13" H="65" Pp="1" S="4" T="16" V="VG" W="1" U="5">540</Rep>
<Rep D="WSW" F="18" G="16" H="55" Pp="0" S="7" T="19" V="VG" W="1" U="7">720</Rep>
<Rep D="WSW" F="19" G="16" H="52" Pp="1" S="7" T="20" V="VG" W="3" U="4">900</Rep>
<Rep D="ENE" F="18" G="11" H="58" Pp="1" S="4" T="19" V="VG" W="1" U="1">1080</Rep>
<Rep D="SSE" F="14" G="9" H="78" Pp="0" S="4" T="15" V="VG" W="0" U="0">1260</Rep>
</Period>
</Location>
</DV>
</SiteRep>

Just to be doubly sure, I flicked back to the json call and it's still not working so hopefully this will help you to track the issue down. Best of luck!

Jeremy

[Douglas Morrison]

Unfortunately, I'm getting empty responses on my xml feed on a regular basis as well, so not sure that it is an xml -> json issue

http://datapoint.metoffice.gov.uk/public/data/val/wxfcs/all/xml/352172?res=3hourly&key=MYAPIKEY currently returning

<?xml version="1.0" encoding="UTF-8"?><SiteRep><Wx><Param name="F" units="C">Feels Like Temperature</Param><Param name="G" units="mph">Wind Gust</Param><Param name="H" units="%">Screen Relative Humidity</Param><Param name="T" units="C">Temperature</Param><Param name="V" units="">Visibility</Param><Param name="D" units="compass">Wind Direction</Param><Param name="S" units="mph">Wind Speed</Param><Param name="U" units="">Max UV Index</Param><Param name="W" units="">Weather Type</Param><Param name="Pp" units="%">Precipitation Probability</Param></Wx><DV dataDate="2017-05-27T16:00:00Z" type="Forecast"></DV></SiteRep>

[Dave Murchison]

I've been getting the same problem. Some queries returned responses over the last few days (perhaps as much as 30%) but today none of them are returning results. Instead I'm getting as shown below. Can we get an official response as to what's happening? I understand there's problems moving to the new API, but we were told it would run in tandem with this one, at least for a while.

[Greg]

I too am experiencing the same problem.

I contacted the Met Office Service Desk on 26 May, who raised an internal ticket with their Internet Applications Team. The reference number for this is INC612016.

My findings have been that this problem:

a) is intermittent;
b) is affecting both XML and JSON responses;
c) only affects queries for the '3hourly' forecast ('daily' appears unaffected);
d) only affects queries without a 'time' parameter.

Attached is a screenshot that I submitted with the ticket to illustrate the problem.

I have sent them another email just now requesting an update, so will keep you posted.

[Greg]

I have received the following reply:

We can confirm that we have now received a response back from our Internet Applications team. They have advised that they are currently unable to replicate the issue you have described with all responses returning with no faults.

I am no longer able to replicate the issue either. Is anyone else still having problems?

[Douglas Morrison]

I have a script that has been logging the success or failures of the calls to the api. The last failure occurred on Monday. Seems to have been trouble free since then. Fingers crossed the problem has been solved.

Thanks for chasing it up with them

[Mike Watson]

Its happening again. Also noticed their own mountain forecast was exhibiting similar issues.

[Greg]

It's happening for me too. I've reopened the ticket and provided the HTTP headers and response for both a successful and unsuccessful request.

Will keep you all posted.

[Greg]

The new incident number has been confirmed as INC000000614820. It is currently under investigation by the Met Office technical team.

[Greg]

I've received a follow up:
 

Our technical teams have had a look and restarted the server and tested that all is working now.
 
Please can you try again and let us know how you get on?

Seems to be working for me now. Any comments?

[Mike Watson]

Nope, still knackered. Still it gave me a chance to test my code for handling such an event works.

[Greg]

Thanks for confirming. It's failing for me too. I've just captured another HTTP request/response and passed it back to them for further investigation.

[colinsjonesuk]

I have empty XML 4 hour forecast.

[Greg]

Latest update:

We have been able to replicate the issue this morning. We have restarted the servers again and they then appeared. We will continue to monitor the situation.

 

Please can you check on your end again and see if the issue is resolved?

It seems to be okay for me at the moment.

[colinsjonesuk]

It seems fine now. Thanks for your response Greg.

[John Dolan]

Hi there,

Is anyone experiencing this problem today? I'm getting the same error as above in the browser.

Thanks

[colinsjonesuk]

XML protocol seems fine.

[John Dolan]

I'm getting this:

This XML file does not appear to have any style information associated with it. The document tree is shown below.

<SiteRep>

<Wx>

<Param name="F" units="C">Feels Like Temperature</Param>

<Param name="G" units="mph">Wind Gust</Param>

<Param name="H" units="%">Screen Relative Humidity</Param>

<Param name="T" units="C">Temperature</Param>

<Param name="V" units="">Visibility</Param>

<Param name="D" units="compass">Wind Direction</Param>

<Param name="S" units="mph">Wind Speed</Param>

......etc.

 

I get this in the browser when I enter the following URl: http://datapoint.metoffice.gov.uk/public/data/val/wxfcs/all/xml/3840?res=3hourly&key=(MY API KEY)

Any ideas?

Thanks.

[Phil Lee]

Hi

I've been seeing this as well, been happening on and off since I've been using their API, thought it was a problem with my coding but then on seeing the problem this morning I thought I would check running the URL call directly in the browser and realised it is their response missing the forecast periods.

It looks like they have never got to the bottom of this problem and just reboot the server and say it's fixed.

Phil

[ahmed]

Getting the same error too for almost all site ids (for some reason site id =1, 2 and a few others up to 250 work, but none of the ones i want).

[peakweather]

This has been happening since early this year on and off, so its not fixed and not very reliable, in that every month it breaks.

While the whole service is not down this sort of thing should be fixed within 24 hours, but unfortunately last time, it went on for weeks and this was in the summer. 

This call along with many others returns incomplete XML

http://datapoint.metoffice.gov.uk/public/data/val/wxfcs/all/xml/310069?res=3hourly&key=<apikey>

[peakweather]

Nearly all 3 hourly forecast XML calls are part empty now. 

I've emailed the MO enquiries and service desk. I think i'm going to have find another data provider as this is not being fixed permanently.