Has anyone managed to upload data to Weathercloud

[Richard Truran]

I've been looking at a new online weather logging service WeatherCloud (https://weathercloud.net/) and I was wondering if anyone has been able to adapt one of the existing weather service upload configurations to upload to this service?

[Jim Easterbrook]

I've just had a quick look round their web site and couldn't find
anything about their API.
--
Jim Easterbrook <http://www.jim-easterbrook.me.uk/>

[Richard Truran]

Thanks for looking, I wasn't able to find anything either.

Richard

[Richard Truran]

Jim,

Looking through the website I've found in the FAQ section (https://weathercloud.net/info/compatibility) a table detailing the weather stations and software it is compatible, firstly it is compatible with the Fine Offset station we all have and second they seem to allow uploading by a large number of different software solutions, I would therefore imagine they will let pywws users upload data too. 

There is a contact form on the about us page (https://weathercloud.net/site/about), I can contact them regarding uploading weather data from pywws if you are OK with that, I may need further assistance depending on their answer, obviously if you want to contact them yourself as the author of the software then I'll leave it to you. If you'd rather I didn't contact them then that's OK as it is your software and I'm just a grateful user.

Thanks for all your effort and help with this great software.

Richard

[atrophi...@gmail.com]

Would anyone like to decipher the weewx script to create a to.service template & ini file? I would but I guarantee it wouldn't work...

[Richard Truran]

Thank for finding that, unfortunately it is way beyond my skills to decipher and port to pywws :(

However it does reference API documentation, performing a quick generic search engine search of the internet I found a comment on a the weathercloud blog which confirmed that you can request the API documentation from their support group using the form on the abut us page.

Richard

[atrophi...@gmail.com]

It would appear that I have my pywws uploading to weathercloud with several items being logged. However before I share my works I'd like to refine it a little and confirm a few things.

My .ini file has the [config] section as below, with pywws sending the data using a 10min cron set up in the weather.ini file;

[config]

url             = http://api.weathercloud.net/v01/set

auth_type       = http

catchup         = 0

result          = []

use get         = True

interval        = 0

So can I improve the following.

• Weathercloud accept at 10 minute intervals only, should the interval in the config be set to 600 instead? If so should this service then be placed under [logged] in the weather.ini
• Weathercloud has 4 return codes, I'd like the text string to show in the log file instead of the number, I couldn't understand the examples in other service.ini files

A couple of values I'd like to include in the template file are the average wind speed & average wind direction (10-min average values). And the highest wind speed (10-min maximum gust of wind). I know I can use the #calc # expression but unsure if it then needs a #loop # or #goto # expression?

One final though at this time, Weathercloud needs a version number when uploading data, I've currently hand written my pywws version in the .ini file but can I pull this from the installed pywws to cater for other versions being used?

[Jim Easterbrook]

On 29/08/17 15:41, atrophi...@gmail.com wrote:
> It would appear that I have my pywws uploading to weathercloud with
> several items being logged. However before I share my works I'd like to
> refine it a little and confirm a few things.
>
> My .ini file has the [config] section as below, with pywws sending the
> data using a 10min cron set up in the weather.ini file;
> |
>
> [config]
>

> url =http://api.weathercloud.net/v01/set

>
> auth_type = http
>
> catchup = 0
>
> result = []
>
> use get = True
>
> interval = 0
>
> |
> So can I improve the following.
>

> * Weathercloud accept at 10 minute intervals only, should the interval

> in the config be set to 600 instead? If so should this service then
> be placed under [logged] in the weather.ini

Yes, set the interval to 600 (or 590 to have a margin), but don't worry
about which section in weather.ini you add the service to. The interval
setting ensures data isn't sent to frequently.

> * Weathercloud has 4 return codes, I'd like the text string to show in

> the log file instead of the number, I couldn't understand the
> examples in other service.ini files

It's a bit of a cludge using a list of regular expressions to match each
line in the text response. Well designed servers that make proper use of
error codes are easier to handle.

> A couple of values I'd like to include in the template file are
> the average wind speed & average wind direction (10-min average values).
> And the highest wind speed (10-min maximum gust of wind). I know I can
> use the #calc # expression but unsure if it then needs a #loop # or
> #goto # expression?

Using #calc# on anything but the current record is difficult. You can't
use a loop or goto to accumulate results.

> One final though at this time, Weathercloud needs a version number when
> uploading data, I've currently hand written my pywws version in the .ini
> file but can I pull this from the installed pywws to cater for other
> versions being used?

There isn't currently any way to do this as far as I can tell.

[Richard Truran]

Hi, have you had any luck getting the uploading to weathercloud working.
I've managed to source the official API documentation from them which I can share to those working on making pywws work with the service.
Please let me know if you think the API documentation will help and I send it to you.

Kind Regards

Richard

[atrophi...@gmail.com]

Yes, I've been uploading to Weathercloud for a while now, currently averaging a 97.3% success rate over the last 30 days. It's not perfect as it doesn't include the 10 minute average data they would prefer and I'm a little wary of the THW Index calc, too many conversions.

I've adapted 3 files that need to go into the Services folder of your pywws install (/usr/local/lib/python2.7/dist-packages/pywws/services), you'll also need to add an entry into your weather.ini file with your weathercloud device details, and of course add weathercloud to a section, I've put mine in logged along with underground:

[weathercloud]
deviceid = YOURDEVICEID
devicekey = YOURDEVICEKEY
template = default

[Richard Truran]

Thanks,

This is perfect. Would you like a copy of the official API documentation for your reference? I can send it privately to you as I've been asked to only send it to those working to make pywws upload to Weathercloud.

Regards

Richard

[Jim Easterbrook]

On 17/01/18 10:44, Richard Truran wrote:
>
> This is perfect. Would you like a copy of the official API documentation
> for your reference? I can send it privately to you as I've been asked to
> only send it to those working to make pywws upload to Weathercloud.

If either of you would like to submit a pull request to have your work
included in pywws I'd be happy to accept it, once you've got it working
satisfactorily.
http://pywws.readthedocs.io/en/latest/copyright.html#contributing-to-pywws

I'm suspicious of a service that wants to keep its API secret. If they
want to limit access to their servers there are better ways of going
about it.

[atrophi...@gmail.com]

If it's v0.6 then I already have it ;)

[Richard Truran]

Yes it is version 0.6

To answer Jim's point, I didn't get the impression they wanted to maintain its API secret, and as you can see they haven't.  When I requested the API through their website they supplied it to me immediately with no questions asked, they just said I was allowed to distribute to anyone working to make pywws work with weathercloud, perhaps I'm being too cautious and could easily post it up here as this is a members only forum for users of pywws.

[Jim Easterbrook]

On 17/01/18 11:22, Richard Truran wrote:
>
> To answer Jim's point, I didn't get the impression they wanted to
> maintain its API secret, and as you can see they haven't.  When I
> requested the API through their website they supplied it to me
> immediately with no questions asked, they just said I was allowed to
> distribute to anyone working to make pywws work with weathercloud,
> perhaps I'm being too cautious and could easily post it up here as this
> is a members only forum for users of pywws.

Don't do that - this forum is publicly visible. I'm just slightly
surprised the API isn't on their website. But I'm often surprised, e.g.
I recently visited an online shop that wanted to charge £4 to download
their catalogue. I took my business elsewhere.

[Richard Truran]

Dear Jim,

OK thanks for the clarification. I won't post.

Thanks

Richard

[Richard Truran]

Jim is in process of performing a large update of pywws which will mean that it will no longer upload to weathercloud until a new service uploader script is written specifically for each service.  Jim has already written scripts for all the officially supported services but one of these will need to adapted to work with weathercloud. I have looked to do this, but it is beyond my ability.

Would you be interested in helping out,, as you successfully got pywws to upload to weathercloud for the current version of pywws.

Thanks 

On Wednesday, 17 January 2018 10:42:01 UTC, atrophi...@gmail.com wrote:

[Richard Truran]

OK, so I need to have a bit more faith in my ability. I appear to have managed to convert the wunderground.py service script to produce a functioning (well, returns a 200 code) script. However, it doesn't return the 'pywws.service.weathercloud:OK' message can anyone see what is wrong?

Jim,

I've copied this to the service directory, will this be deleted when I install new commits from github?

R

[Jim Easterbrook]

On 21/04/18 20:26, Richard Truran wrote:
> OK, so I need to have a bit more faith in my ability. I appear to have
> managed to convert the wunderground.py service script to produce a
> functioning (well, returns a 200 code) script. However, it doesn't
> return the 'pywws.service.weathercloud:OK' message can anyone see what
> is wrong?

At the end of upload_data you're doing a test for the response being
"success". This is what the Weather Underground server returns but may
not apply to other servers.

The services that give a simple "OK" message are ones where the server
doesn't send any response. Have a look at the other service modules to
see how other servers are handled.

PS You've also managed to insert some tab characters into your Python
source. Never use tabs in Python, always use spaces.

> I've copied this to the service directory, will this be deleted when I
> install new commits from github?

If it's in your source it'll be OK ('git status' will show it as an
untracked file). If you've only got it in the installation directory
(e.g. /usr/local/lib/python3.4/dist-packages/...) it will probably get
deleted when you next install.

[Richard Truran]

Dear Jim,

Thanks for the information and advice, I'll look to replacing tabs with spaces and research a bit on the correct handling of server responses for weather cloud.

R

[Richard Truran]

I've now fixed the weathercloud service uploader, it correctly identifies the error codes from the server and displays a descriptive message in the log. I have also hopefully removed all the errant tabs in the code (I used the untabify function in IDLE).

[Richard Truran]

Dear Jim,

Given your recent updates to make the metoffice uploaded work correct, should it be possible to calculate the 10 minute average values now?
Also is there any way to upload the live values weathercloud requires.

Thanks

Richard

[Jim Easterbrook]

On 16/05/18 20:00, Richard Truran wrote:
>
> Given your recent updates to make the metoffice uploaded work
> correct, should it be possible to calculate the 10 minute average
> values now?

It should be easy enough to adapt the bits I've added to the Met Office
uploader. Given the particular requirements of all these different
services I'm not sure it's worth adding too many different functions to
the template module itself.

> Also is there any way to upload the live values weathercloud
> requires.

Adding the service to the [live] section of weather.ini (and running
live logging) is as close as you can get.

[Jim Easterbrook]

On 22/04/18 08:43, Jim Easterbrook wrote:
> On 21/04/18 20:26, Richard Truran wrote:
>> OK, so I need to have a bit more faith in my ability. I appear to have
>> managed to convert the wunderground.py service script to produce a
>> functioning (well, returns a 200 code) script. However, it doesn't
>> return the 'pywws.service.weathercloud:OK' message can anyone see what
>> is wrong?
>
> At the end of upload_data you're doing a test for the response being
> "success". This is what the Weather Underground server returns but may
> not apply to other servers.
>
> The services that give a simple "OK" message are ones where the server
> doesn't send any response. Have a look at the other service modules to
> see how other servers are handled.

I've added a bit of documentation about writing your own service
uploader(s).
http://pywws.readthedocs.io/en/latest/guides/integration.html#writing-your-own-uploader

>> I've copied this to the service directory, will this be deleted when I
>> install new commits from github?
> If it's in your source it'll be OK ('git status' will show it as an
> untracked file). If you've only got it in the installation directory
> (e.g. /usr/local/lib/python3.4/dist-packages/...) it will probably get
> deleted when you next install.

I've also recently added the ability to put your uploader(s) in a
"modules" directory of your own choosing.

[Richard Truran]

Thanks Jim,

I’ll look to setting up a “modules” directory soon as I just updated to the latest development version.

I still need to update the uploaded to produce 10 minute averages and test if uploading live data works before I would consider submitting this for inclusion into the stable version of pywws.

Richard