Question about the Realtime Gauge-Data Extension

[Alec Bennett]

I want to build a live weather widget for a website similar to what Weather Underground displays on their pages:

https://www.wunderground.com/personal-weather-station/dashboard?ID=KCAPETAL93

Note the wind direction changing every few seconds, assuming my weather station is behaving.

I found the Realtime Gauge-Data Extension, and *think* it might enable that sort of thing, except that I can't figure out how to upload gauge-data.txt more often than the other files in my public_html folder. To get true real time weather data it would need to be uploaded as often as Weather Underground is updated in "rapidfire" mode.

Is such a thing currently possible?

[gjr80]

Hi,

The short answer to your question is no, nothing like that exists at the moment. If you have your web server (or process that wil use gauge-data.txt) on your weeWX machine then it is a simple matter for the generated file to be saved wherever you want, but if you need to transfer the file in near realtime to another machine then no that capability does not exist at the moment.

There is no technical reason why a loop based ftp or rsync service could not be written, I know that some have been tinkering with it. There are no weeWX issues raised on GitHub in this regard nor is there anything in the weeWX roadmap so I don't expect you will see such a service in the immediate future (3.7.0 has been soaking up most people's effort). I think it is fair to say there is some recogniton that some sort of loop based transfer mechanism is needed to support near realtime the likes of SteelSeries Gauges, Saratoga and Leuven templates and others but at the moment no effort has been put into creating a supported solution.

I expect Tom and Matthew may have well have further views on this.

Gary

[Alec Bennett]

The short answer to your question is no, nothing like that exists at the moment. If you have your web server (or process that wil use gauge-data.txt) on your weeWX machine then it is a simple matter for the generated file to be saved wherever you want, but if you need to transfer the file in near realtime to another machine then no that capability does not exist at the moment.

I added the ability to post live_data.json to a remote webserver, if anyone needs the ability, it's here:

https://github.com/wrybread/weewx-realtime_gauge-data

It works by posting all the data it writes to live_data.json to a PHP script on a remote webserver using a GET statement. Note the PHP script here, that receives the data:

https://github.com/wrybread/weewx-realtime_gauge-data/tree/master/PHP%20Script

If anyone wants me to convert it to Python for consistency, let me know.

To use, set the following value in the RealtimeGaugeData section of your weewx config file:

remote_server_url = http://yourwebsite.com/path_to/receive_conditions.php

It's my first time posting a fork to github, so please let me know if I did anything dumb. And let me know if anyone wants me to change anything.

[Alec Bennett]

I made a quick page that updates some data from the real time gauge using Ajax:

sinkingsensation.com/wind

It takes some data from live_data.json (currently wspeed, wgust, bearing, domwinddir) and displays it, updating every 2 seconds.

I'd like to add the highest gust of the day though, and I'm wondering if anyone can see an easy way to do that. In the reports, that would be $day.windGust.max.

Also, would questions like this be better on the dev list?

[Alec Bennett]

And comparing the page I just made (sinkingsensation.com/wind) to my Weather Underground page (https://www.wunderground.com/personal-weather-station/dashboard?ID=KCAPETAL93#history), the data is slightly different. Does anyone know what data the WU updater uses, and why it would be different?

And would I be better off modifying the Weather Underground uploader so that it uploads to my server from there, instead of the Real Time Gauge plugin?

My ultimate goal is to reproduce the wind gauge from the Weather Underground site. 

Incidentally, live camera feed from the beach right next to our weather gauge, with a nice storm moving in:

http://sinkingsensation.com/weather/

[gjr80]

Good work, I must admit I had been stuck in the 'file transfer' paradigm rather than 'data transfer' and whilst I have had loop based RSYNC working on my LAN there was too much contention for my liking. I will have have a look at this and see how your approach works for me.

Gary

[gjr80]

On Tuesday, 21 March 2017 12:28:02 UTC+10, Alec Bennett wrote:

I made a quick page that updates some data from the real time gauge using Ajax:

sinkingsensation.com/wind

It takes some data from live_data.json (currently wspeed, wgust, bearing, domwinddir) and displays it, updating every 2 seconds.

I'd like to add the highest gust of the day though, and I'm wondering if anyone can see an easy way to do that. In the reports, that would be $day.windGust.max.

field wgustTM should already provide that.
 

Also, would questions like this be better on the dev list?

 
Certainly doesn't worry me, can't speak for anyone else though.

Gary

[gjr80]

Since you have rapidfire on WU should be being fed the loop windSpeed values. The loop windSpeed values appear in gauge-data.txt in the wlatest field not the wspeed field you are using; wspeed is the 'average wind speed' which will be the average wind speed over an archive period or 5 minutes depending on settings. The wgust used by your page is the highest wind speed seen over the last 10 minutes, that seems approriate though I am not sure exactly what WU uses.

Direction wise you are using fields bearing and domwinddir, bearing will be the direction corresponding to the loop windSpeed value so that seems appropriate but domwinddir is the vector average wind direction for today. Your brackets around domwinddir seem to imply it is the compas point equvalent of bearing which it isn't, if that is what you intend then perhaps you need to express bearing as a compass point either in the code on your page or in the source gauge-data.txt.

Gary

[Alec Bennett]

On Mon, Mar 20, 2017 at 8:34 PM, gjr80 <gjrod...@gmail.com> wrote:

Since you have rapidfire on WU should be being fed the loop windSpeed values. The loop windSpeed values appear in gauge-data.txt in the wlatest field not the wspeed field you are using; wspeed is the 'average wind speed' which will be the average wind speed over an archive period or 5 minutes depending on settings. The wgust used by your page is the highest wind speed seen over the last 10 minutes, that seems approriate though I am not sure exactly what WU uses.

Direction wise you are using fields bearing and domwinddir, bearing will be the direction corresponding to the loop windSpeed value so that seems appropriate but domwinddir is the vector average wind direction for today. Your brackets around domwinddir seem to imply it is the compas point equvalent of bearing which it isn't, if that is what you intend then perhaps you need to express bearing as a compass point either in the code on your page or in the source gauge-data.txt.

Thanks for the clarification, I updated the code so:

wlatest = current wind speed

wgust = 10 minute high wind gust

bearing = current wind direction

wgustTM = today's highest wind gust

TwgustTM = time of today's highest wind gust

Incidentally, for anyone else who comes down this path, there's good documentation on the variables used by the Real Time Gauge here:

http://wiki.sandaysoft.com/a/Realtime.txt

[Ruben Navarro Huedo]

A lot of thank's for your great work.
Please... could we have a copy of this example wind page?
Thank's a lot.

[gjr80]

Alec,

Just wondering if there was any particular reason you used a GET rather than a POST? I think the code to support passing the data via GET is a little simpler than a POST but since GET includes all of the data in the URL my nginx access log is getting hammered. I estimate about 55Mb per day.

Gary

[Alec Bennett]

Just wondering if there was any particular reason you used a GET rather than a POST? I think the code to support passing the data via GET is a little simpler than a POST but since GET includes all of the data in the URL my nginx access log is getting hammered. I estimate about 55Mb per day.

Very interesting, I hadn't considered that. I'll switch to POST later today. 

Gary

On Tuesday, 21 March 2017 13:09:56 UTC+10, gjr80 wrote:

Good work, I must admit I had been stuck in the 'file transfer' paradigm rather than 'data transfer' and whilst I have had loop based RSYNC working on my LAN there was too much contention for my liking. I will have have a look at this and see how your approach works for me.

Gary

On Tuesday, 21 March 2017 04:41:49 UTC+10, Alec Bennett wrote:

The short answer to your question is no, nothing like that exists at the moment. If you have your web server (or process that wil use gauge-data.txt) on your weeWX machine then it is a simple matter for the generated file to be saved wherever you want, but if you need to transfer the file in near realtime to another machine then no that capability does not exist at the moment.

I added the ability to post live_data.json to a remote webserver, if anyone needs the ability, it's here:

https://github.com/wrybread/weewx-realtime_gauge-data

It works by posting all the data it writes to live_data.json to a PHP script on a remote webserver using a GET statement. Note the PHP script here, that receives the data:

https://github.com/wrybread/weewx-realtime_gauge-data/tree/master/PHP%20Script

If anyone wants me to convert it to Python for consistency, let me know.

To use, set the following value in the RealtimeGaugeData section of your weewx config file:

remote_server_url = http://yourwebsite.com/path_to/receive_conditions.php

It's my first time posting a fork to github, so please let me know if I did anything dumb. And let me know if anyone wants me to change anything.

--
You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/ws7u2itvJMw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to weewx-user+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Alec Bennett]

A lot of thank's for your great work.
Please... could we have a copy of this example wind page?

Sure, I'll add a stripped down version to my GitHub. 

Note that all the code is a available currently by viewing the source. 

[Thomas Keffer]

On the general problem of what makes a good RESTful API, I found the document RESTful Service Best Practices, Recommendations for Creating Web Services very informative. It would definitely recommend a POST.

-tk

To unsubscribe from this group and all its topics, send an email to weewx-user+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscribe@googlegroups.com.

[Alec Bennett]

Hallelujia, figured out the Weather Underground style wind gauge:

http://sinkingsensation.com/wind

(Note that the above page is set up for mobile, and lets you swipe to get to a second page).

Compare it to my PWS page on Weather Underground:

https://www.wunderground.com/cgi-bin/findweather/getForecast?query=pws:KCAPETAL93

All working nicely with the Real Time Gauge extension.

A stripped down version is on my Github if anyone has interest:

https://github.com/wrybread/Weather-Underground-Style-Real-Time-Gauge

[gjr80]

Thanks Tom, that was good to read, will tuck that away for reference in future.

Gary

On Wednesday, 22 March 2017 01:46:21 UTC+10, Tom Keffer wrote:

On the general problem of what makes a good RESTful API, I found the document RESTful Service Best Practices, Recommendations for Creating Web Services very informative. It would definitely recommend a POST.

-tk

On Tue, Mar 21, 2017 at 8:40 AM, Alec Bennett <wryb...@gmail.com> wrote:

Just wondering if there was any particular reason you used a GET rather than a POST? I think the code to support passing the data via GET is a little simpler than a POST but since GET includes all of the data in the URL my nginx access log is getting hammered. I estimate about 55Mb per day.

Very interesting, I hadn't considered that. I'll switch to POST later today. 

Gary

On Tuesday, 21 March 2017 13:09:56 UTC+10, gjr80 wrote:

Good work, I must admit I had been stuck in the 'file transfer' paradigm rather than 'data transfer' and whilst I have had loop based RSYNC working on my LAN there was too much contention for my liking. I will have have a look at this and see how your approach works for me.

Gary

On Tuesday, 21 March 2017 04:41:49 UTC+10, Alec Bennett wrote:

The short answer to your question is no, nothing like that exists at the moment. If you have your web server (or process that wil use gauge-data.txt) on your weeWX machine then it is a simple matter for the generated file to be saved wherever you want, but if you need to transfer the file in near realtime to another machine then no that capability does not exist at the moment.

I added the ability to post live_data.json to a remote webserver, if anyone needs the ability, it's here:

https://github.com/wrybread/weewx-realtime_gauge-data

It works by posting all the data it writes to live_data.json to a PHP script on a remote webserver using a GET statement. Note the PHP script here, that receives the data:

https://github.com/wrybread/weewx-realtime_gauge-data/tree/master/PHP%20Script

If anyone wants me to convert it to Python for consistency, let me know.

To use, set the following value in the RealtimeGaugeData section of your weewx config file:

remote_server_url = http://yourwebsite.com/path_to/receive_conditions.php

It's my first time posting a fork to github, so please let me know if I did anything dumb. And let me know if anyone wants me to change anything.

--
You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/ws7u2itvJMw/unsubscribe.

To unsubscribe from this group and all its topics, send an email to weewx-user+...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

[gjr80]

I have added HTTP POST to rtgd.py using a similar approach to that taken in the weeWX RESTful API in restx.py. It seems to work well. The php I am using is:

<?php
// we are only interested in HTTP POST
if($_SERVER['REQUEST_METHOD'] == 'POST') {  
    // get the data
    $data = file_get_contents("php://input");
    // write the data to file
    file_put_contents("json/gauge-data.txt", $data);
    // send something back indicating success
    echo "success";
}
?>

If OK with you Alec I have mentioned you in the comments as the source of the idea and referenced your repo. I will publish 0.2.11 later today.

Gary

[Alec Bennett]

On Tue, Mar 21, 2017 at 4:49 PM, gjr80 <gjrod...@gmail.com> wrote:

I have added HTTP POST to rtgd.py using a similar approach to that taken in the weeWX RESTful API in restx.py. It seems to work well. The php I am using is:

<?php
// we are only interested in HTTP POST
if($_SERVER['REQUEST_METHOD'] == 'POST') {  
    // get the data
    $data = file_get_contents("php://input");
    // write the data to file
    file_put_contents("json/gauge-data.txt", $data);
    // send something back indicating success
    echo "success";
}
?>

If OK with you Alec I have mentioned you in the comments as the source of the idea and referenced your repo. I will publish 0.2.11 later today.

Fantastic! And yes of course.