weewx on QNAP

[David Schulz]

I wanted to run Weewx on my QNAP TS119 to connect to my Fine Offset WH1080 weather station. This is how I achieved that.

Hardware

I purchased a 'Wireless Weather Sation' from Jaycar (http://www.jaycar.com.au/productView.asp?ID=XC0348). This station is sold under a variety of names but is generally referred to in most cases as a Fine offset WH1080. The 1080 has a USB port that allows it to be connected to a computer to download the data from the station. It is supplied with some Windows software but I don't have a PC.

The best solution for me was to connect it to my QNAP NAS. This acts as my home 'server' and is always on, making it the best place to connect the weather station to.

Dead Ends

The QNAP uses a highly customised version of Linux with a graphical user interface to configure and operate it. This is the main selling point for me for whom the command line is a foreign country. The QNAP system is extensible with its own package management system (QPKGs) which makes adding additional software literally as simple as a single click. This is my kind of Linux!

So first port of call was to find weather station software that was available as a QPKG. I downloaded and installed wfrog which installed OK. It relied on another package called pywws to actually get the data with wfrog primarily acting as a presentation layer (from what I understand?). Despite great support from the author of the packages I never could get pywws working.

Software

The user community for both wfrog and pywws is quite small. A number of people pointed me at weewx which has a comparatively much larger support base and I knew people running this. I decided weewx was the best bet for the software part of my solution.

And the Bad News...

The downside of this decision was there is no QPKG. The worse news was that a number of posts on the weewx user forums said that because the QNAP was such a specialised Linux environment it would simply not be possible to get weewx working on it.

The 'smart money' seemed to be to grab a Raspberry Pi or a Beaglebone Black and run up a standard Linux system to act as a dedicated controller for the weather station. I've been itching for an excuse to buy one of these mini marvels for a while so I wasn't averse to that.

But I really like the idea of getting some more value from the QNAP system. I had it, so what was the harm in having a go at getting it running?

The Journey

As Linux novice fearful of the command line, I found the weewx documentation to be really quite good. Rather than the usual instructions laden with assumed knowledge, the document contained clear commentary on what needed to be achieved in each step, along with the actual commands required with variants for a range of Linux distributions - but sadly not for the QNAP. But it did give me a really good idea of what I needed to achieve to get this going.

QNAP and QPKGS

The first steps were to turn on embedded capabilities in the QNAP and to grab the things that were available as QPKGs. I opted to use the MySQL database for my weather data. The default is SQLite which doesn't require any setup. I kinda liked the idea of being able to use PHPMyAdmin to look at the data directly so opted for MySQL. The weewx system is a Python application so naturally enough Python is required. All these things are able to be installed and setup from the QNAP graphical interface. Yay!

MySQL

Enabling MySQL is as simple as logging into the QNAP QTS Desktop, opening the Control Panel, navigating to Applications and selecting MySQL Server. Click the checkbox to enable MySQL Server.

PHPMyAdmin

PHPMyAdmin gives you a graphical UI to manage your databases, permissions and you can execute queries if you can manage the odd “select * from...”.  In the QTS Desktop, open the App Centre and find the PHPMyAdmin QPKG and click “Add to QTS'. The package will download and install.

Python

Weewx requires Python. As at the time of writing, Python 3 is not supported. Similarly, the Python QPKG installs version 2.7 so this aligns nicely. Find the Python QPKG and click 'Add to QTS'. The package will download and install.

Pre-PreRequisites

The weewx documentation lists the prerequisites including a number of Python modules. In practice I found a couple of these modules would not install without a compiler. So I first needed to install the gcc compiler before these would install successfully.

GCC Module

A compiler is needed to install the Cheetah and PIL modules (from memory). The way I installed he GCC compiler was from an IPKG. There is a QPKG that allows you to use IPKGs. The first step is to install the Optware QPKG. Open the App Centre on the QTS Desktop and find the Optware package and click 'Add to QTS'. It will download and install.

Click the Optware IPKG icon on the QTS Desktop. This will open the IPKG system. In the “Filter' box type 'gcc' and click Submit. To the left of the gcc IPKG you will see the option to 'Install' Click this to install the compiler.

We're Not in Kansas Anymore Dorothy

So here ends the part where I was in my comfort zone. The next steps are executed from the command line. I used a terminal window on OS X to connect to the QNAP. When your terminal window opens, connect to the QNAP using the following command

ssh ad...@nnn.nnn.nnn.nnn

where the last part is the IP address of the QNAP. If you don't know this, you can find the IP address from the Network page in System Settings of the Control Panel.

You will be prompted for the admin password and then you will see the unnerving, vacuous blinking of the command line prompt “$”; telling you nothing, taking you nowhere.

When you are done you can type 'exit' to disconnect.

PreRequisites

To PIP or not to PIP

Python has a number of package management system designed to make it simple to download and install Python modules. The most current method from what I read is PIP. I used these nice instructions to get PIP setup: http://rdkevinchien.blogspot.com.au/2013/08/how-to-setup-my-python-developing.html

As it turned out I was never able to get modules installed using PIP. PIP enforces an SSL connection for downloading the package which is a good thing. The self issued certificate on the QNAP meant the download always failed. It is beyond me to sort out the certificate thing and I couldn't find a pip option or switch to override the secure download.

There is a nice set of instructions for purchasing and installing a certificate for your QNAP. See my references at the end. This would probably make the PIP installs works and will be bit simper if you want to go down that path.

My solution was to use the commands to download and install pip as a template to download the modules manually and build and install them manually.
I used the admin home folder to store the downloads into.

cd /share/homes/admin

I used the Python module pages to grab the URL to download each module. The Python Package Index allows you to search by module name: https://pypi.python.org/pypi

For example the configobj module page is: https://pypi.python.org/pypi/configobj/5.0.5 From this page you can find the download link and copy the link address.  Paste the link address into the download commands below to make it easier and avoid messing about retyping the url without making an error.

I downloaded, unzipped, built and installed each of the module dependencies in turn.

Configobj

First download

wget https://pypi.python.org/packages/source/c/configobj/configobj-5.0.5.tar.gz#md5=00af246754b4bb41a4e9c625c60dca5c --no-check-certificate

Extract the archive

tar xvfz configobj-5.0.5.tar.gz

Change to the folder where the file was extracted.

cd configobj-5.0.5

Build the module

python setup.py build

Install the module

python setup.py install

Repeat for the remaining dependencies listed in the weewx documentation: http://www.weewx.com/docs/setup.htm

I installed the following modules:

• configobj
• cheetah
• pil
• pyusb
• pyephem

Nearly There

I needed just a couple of last steps to finally get things running.

The MySQL Gotcha

Walking the path less travelled had one last little bump in the road that needed to be ironed out. I was following the pip install instructions in the weewx documentation. When I tried to run weewx for the first time it failed with an error message saying mysqldb module not found.

No dramas I thought, feeling quite confident have smashed all the other module installs.  Well, once I finally sorted my workaround method out...  After downloading and expanding the module as expected, the build command failed with an error saying a config was not found. I had horrible premonitions about needing to manually configure a build file of some sort. Fortunately MySQL Boy had solved this particular problem. Just a path needed to be added: http://blog.mysqlboy.com/2010/08/installing-mysqldb-python-module.html

This command let me build and install the mysqldb module successfully

export PATH=$PATH:/usr/local/mysql/bin

Download and Install Weewx

Now I was able to download and install weewx as per the standard documentation.  http://www.weewx.com/docs/setup.htm

I found the weewx documentation explaining the weewx.conf file clear and straight forward. Similarly the documentation covering configuring the weewx outputs is generally clearly explained.

When I first saw the lines of data from the weather station scrolling up the terminal window I thought it was a problem and weewx was writing to screen instead of the database. A quick peek at the tables in MySQL using the PHPMyAdmin confirmed the data was writing there. So I concluded the writing to screen was normal behaviour.

It will take a few minutes but you will be able to point your browser at http://nnn.nnn.nnn.nnn/weewx and see your weather data. The nnn.nnn.nnn.nnn part is the IP address of the QNAP. It was great to see this page populated with data from my weather station!

I kept the terminal window with weewx running in it open on the Mac for a few days as I wasn't game to just close it in case it stopped the process. It eventually disconnected and the web pages keep updating so it seems OK to just close the terminal window once weewx is up and running.

Apache Config and Restart

The Weewx documentation provides instructions to get weewx talking to the web server (Apache) in the case it was not installed from a package. This was certainly the case for me. These instructions needed some slight modifications to reflect the location of apache on the QNAP and a different commend to restart apache.

The command in the document:

cp util/apache/conf.d/weewx.conf /etc/apache2/conf.d

Needs to be:

cp util/apache/conf.d/weewx.conf /etc/config/apache/conf.d

The equivalent commas to restart apache on the QNAP is:

/etc/init.d/Qthttpd.sh restart

Outstanding Tasks

I have the data coming off the weather station onto the QNAP and an elementary web page displaying our weather data locally. There are still quite a few tasks still to sort out.

Autorun

So far I have just started weewx from the command line. If the QNAP is restarted I will manually need to restart weewx.  Adding things to start on boot on the QNAP is not straightforward. It looks like I'll need to declare a dummy QPKG as the recommended solution form QNAP but that looks like another journey of discovery...
http://wiki.qnap.com/wiki/Running_Your_Own_Application_at_Startup

Graphs

None of the graphs on the web page are showing. I thought this may be to do with a font specified in the skin.conf and not on the QNAP. I have downloaded the required font but still not getting any plots.

Skins

I'm not a huge fan of the layout and look and feel of the web page 'out of the box'.  I need to invest some time in understanding how to modify the skins and maybe some css mods to be able to customise this.

Publish to Web

So far the web page is generated to the QNAP. Weewx provides a report to publish to FTP to make our weather data available outside our network. Need to configure the ftp report options to make this happen.

Publish to Weather Underground and ...

Weewx also allows data from my weather station to be published to crowd sourced weather data systems like Weather Underground and WOW. I've setup a Weather Underground account and station. Need to configure to publish to that account.

References and Recognition

I am nowhere near knowledgeable enough to have solved any of this myself. All I have done is piece together a lot of disparate threads leveraging the knowledge and contributions of people who actually know what they are doing. The following sources were absolutely vital in me nutting this out. Thanks to all the authors!

PIP Setup on QNAP
http://rdkevinchien.blogspot.com.au/2013/08/how-to-setup-my-python-developing.html

Workaround for PIP Download Problem
http://gleenders.blogspot.com.au/2014/04/install-python-pip-on-qnap-nas.html

MySQLDB Build Fail
http://blog.mysqlboy.com/2010/08/installing-mysqldb-python-module.html

QNAP Autorun
http://wiki.qnap.com/wiki/Running_Your_Own_Application_at_Startup

[Simon]

Thanks for the info i have a TS121 with Debian on it running weewx and blitzortung at present. weewx FTPs the data to my TS459 (web server) though if this is possible with the TS459 (x86 architecture) I might try this instead.

Simon

[Simon]

btw what OS firmware version you running on the QNAP?

[David Schulz]

Hi Simon

Apologies for the delay in responding.  I am running the latest 4.1 firmware on the QNAP.

[Simon]

Thanks david.  btw you mentioned the issue with fonts.  did you use

fcache -f -v 

after you downloaded/installed the fonts.  i had the same issue with my QNAP TS-121 running debian.

Simon