Re: Tutrorial: Setting up WeeWX on Ubuntu Linux with a Davis station

[vds]

A minor disagreement - the cable you mention for $30 is basically just a serial cable.  The more expensive Davis data logger is that plus an interim storage device that can hold a week or so of data.  Weewx uses this data to gracefully catch up its local database as well as any uploads to downstream sites such as Weather Underground.  Power outages happen. So for me the extra $75 was money 'very' well spent and not a waste in my opinion.

That said, nice tutorial.

[Thomas Keffer]

Gosh, thanks so much for putting this together, Tony! It's always gratifying to see user content.

May I offer a few suggestions?

1. To avoid doing everything as root and/or using 777 permissions everywhere, I create the /home/weewx directory and then make myself the owner:

sudo mkdir /home/weewx

sudo chown -R tkeffer:tkeffer /home/weewx

(substituting, of course, your user name for 'tkeffer').

At this point, almost everything can now be done without root privileges.  The exception is that you may have to give yourself permission to access the /dev entry for the console. The easiest, permanent, way to do that is to add yourself to the group that owns the /dev entry, usually 'dialout':

sudo adduser tkeffer dialout

(again, substituting your user name for 'tkeffer').

2. In your section 'Install the serial cable' you left out the step of plugging the cable into the console!

3. Question: if you are going to put the generated files into /var/www/myserver/myweather, why would you need the directory /home/weewx/public_html at all? In any case, there is no need to create the latter manually --- it is automatically done by weewx.

4. If you follow my instructions #1 above, you will not need root privileges in the "start up weewx" section.

5. Finally, how about an actual link to the cable you used? There is a lot of confusion here, because some 3rd parties are beginning to offer cables, but they do not have a memory chip, while there are some online instructions that do have a chip.

Thanks for taking this on, Tony!

-tk

On Thu, Nov 29, 2012 at 7:18 PM, Tony Perkins <tonyp...@gmail.com> wrote:

 I've created a tutorial on getting a Davis Vantage Vue running with:

WeeWX
A non-logging serial cable
Ubuntu Linux
MySQL

Hopefully this can help some other folks out along the way!

Up to date tutorial link: 
http://www.blueskybust.com/homewx.html

Original contents copied to this post... sorry for the poor formatting.

Getting your weather station online using a Davis Vantage Vue, a SLOweather serial cable, and WeeWX on Ubuntu Linux.

Updated 11/2012

I wanted to get a Davis Vantage Vue on the Internet. Since I already have an Ubuntu Linux PC running 24 hours a day, this is where I chose to send the data from the station. The Davis product to interface with a PC, called WeatherLink, only contains software for Macs or Windows. Essentially, you spend $100+ for a serial cable and software you throw away.

Fortunately, community grown solutions exist to do this for much cheaper. Through Google I found a homebew serial cable to interface between the Davis console and the Ubuntu PC. This cable can be found by googling for SLOweather and Davis serial cable. I got my for around $30. There are two kinds for purchase, a serial cable, and a USB interface. I have read that the serial interface is rock solid compared to the USB interface, and since my PC has a serial port, I went with that.

Steps to set up

Prerequisite steps:

1. Your Davis Vantage weather station is operations and the console is displaying the weather data
2. You have an Ubuntu Linux PC
3. You have MySQL installed and running on Ubuntu

Disclaimer: I found that some things needed to be run as sudo to work on Ubuntu, or that certains files couldn't be edited because I wasn't logged in as 'weewx'. My quick fixes are to change the permissions of those files to 777, and to run a lot of stuff as sudo. It works to get things going. I know it's not the secure way to do things.

Install the serial cable:

1. Turn off the console and remove the batteries so the power is completely off to it. I don't know if this is required, but I did it.
2. Hook the other end up to the Ubuntu PC with a serial port.
3. Turn the console back on. Press 'Done' on it to get out of setup mode.

Test the connection to Ubuntu

Install the program minicom on Ubuntu:
sudo apt-get install minicom

Find your serial port identified in the device list with this command:
dmesg | grep ttyS

For me, this returned the information of:
[ 0.510838] serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
[ 0.532564] 00:08: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A

The serial port device to use based on this is ttyS0.

Setup minicom for the test

sudo minicom -s

Go to the menu option Serial port setup.
Change setting of "Serial Device" to be what was in the previous step, with /dev/ prepended.
Example: /dev/ttyS0

Change setting of "Bps/Par/Bits" to:
Speed: 19200
Parity: None
Stopbits: 8-N-1

When done, save the config on the first menu with Save setup as dfl.

Test with minicom

Start minicom with:
sudo minicom

When it starts, type TEST in all capitals. You wont be seeing what you are typing. If the word TEST is replied back, then it works.
Next, type LOOP in all caps. You should see gibberish characters come across. This is the weather data being sent. Exit the command with Ctrl-C.

You have now verified that the console is sending data to the Ubuntu PC.

Install WeeWx

WeeWx can decode the data and generate useful graphs and HTML pages with it. It also stores the data in a MySQL database, and can post the data to internet sites.

I downloaded the latest version using wget, which was weewx-2.0.2.tar.gz:
wget http://sourceforge.net/projects/weewx/files/weewx-2.0.2.tar.gz/download

Open the quick guide setup steps here. It's titled for Debian but works on Ubuntu:

http://www.weewx.com/docs/debian.htm

Follow the Prerequisite install steps as is.
Follow the Installing section as is.
For the "Configure" section: Edit the configuration file (I use vi, use what you like):

sudo chmod 777 /home/weewx/weewx.conf
vi /home/weewx/weewx.conf

Here are the changes I made to work with the SLOWeather serial cable and mysql:
Enable debugging. You may need it, so it doesn't hurt. You can turn it back to 0 after everything is working.
debug = 1

In the [Station] section:
location = (enter any text you want to describe your location)
latitude = (your stations latitude)
longitude = (your stations longitude)
altitude = (your stations altitude, foot) ( Example: altitude=654,foot )
station_type = Vantage (I think this was the default, but it's important to verify)

In the [Vantage] section:
port = (Use the /dev/ttyS0 entry you found when testing with mini com)
Ex: port = /dev/ttyS0

In the [StdReport] section:
HTML_ROOT = public_html
(Since I run a web server on the same Ubuntu box, I changed this from public_html to a path like /var/www/myserver/myweather. You can just keep it at public_html for initial testing and change it later if it needs to go to another directory.)

In the [StdArchive] section:
Change the entries so it uses mysql:

archive_database = archive_mysql
stats_database = stats_mysql
record_generation = software

I needed to change record_generation to software in order for it to work with the SLOweather serial cable. I think it's because the cable doesn't store records itself.

Nothing else needs to change in the config file. I used the default username and password listed in the [[archive_mysql]] and [[stats_mysql]] sections of weewx / weewx.

Create the public_html directory

This didn't exist on my installation, so create it:

sudo mkdir /home/weewx/public_html
sudo chmod 777 /home/weewx/public_html

Create the MySQL database user of weewx

Start the mysql client:

mysql -u root -p
(Enter your root password for mysql. Can't help you here)

CREATE USER 'weewx'@'localhost' IDENTIFIED BY 'weewx.;
GRANT select, update, create, insert, delete on weewx.* to weewx@localhost;
GRANT select, update, create, insert, delete on stats.* to weewx@localhost;
quit

Start up weewx

sudo /home/weewx/bin/weewxd.py /home/weewx/weewx.conf

If things are working, you'll see data being printed to your window.
If nothing shows after 15 seconds or so, exit using Ctrl-C and then check the weewx debug output in the syslog:
grep weewx /var/log/syslog | more

If it's working ok too, do Ctrl-C to stop weewx.

Verify that mysql is logging data

Run this command again, but as the weewx user instead of root:

mysql -u weewx -p
(Enter password of weewx, which you probably created above as 'weewx')
use stats;
select * from outTemp;

You should see a record or two in the tables here, like this:
+------------+------+------------+------+------------+------------------+-------+
| dateTime | min | mintime | max | maxtime | sum | count |
+------------+------+------------+------+------------+------------------+-------+
| 1353736800 | 27.2 | 1353819100 | 28.8 | 1353822986 | 986.258918640156 | 35 |
| 1353823200 | 27.5 | 1353862112 | 30.6 | 1353830478 | 4726.0591897815 | 164 |
+------------+------+------------+------+------------+------------------+-------+

2 rows in set (0.00 sec)

If you don't see aything, then something isn't set up right with the mysql logging. Type quit and check the weewx syslog with:
grep weewx /var/log/syslog | more

If you saw a record, chances are the archive database is working too. From still in mysql:

use weewx;
select * from archive;

You should see one or more records listed. If you don.t, it may just be that weewx wasn.t running for 5 minutes before we stopped the initial test. If weewx is running later for 15+ minutes and you still see nothing in here, check the weewx debug syslog:
grep weewx /var/log/syslog | more

type quit to exit mysql

You should also be seeing files generated in the public_html directory. Open a few of the .html files with your web browser to check.

Add weewx as a service that starts up automatically

Now that it.s all working, let.s set it up so weewx starts on system startup, and that it uses the familiar init.d start and stop functions.

sudo cp /home/weewx/start_scripts/Debian/weewx /etc/init.d/
sudo chmod +x /etc/init.d/weewx
sudo update-rc.d weewx defaults 98

Now you can start and stop weewx with:

sudo /etc/init.d/weewx start
sudo /etc/init.d/weewx stop
sudo /etc/init.d/weewx restart

Remember to turn off debugging with debug=0 in the weewx.conf file if you want, and then restart weewx.

That's it. Questions, comments, or improvements, send me an email at the permanent link at the top.

--
 
 

--

Tom Keffer

kef...@threefools.org
+1 541-386-8891 (h)
+1 541-490-9507 (c)

Skype: tkeffer

[Tony Perkins]

VDS:  Thank you for the comment, you are correct.  If you need to be immune from a PC outage then yes, the Davis logger is a great option.

Tom:  Thanks for the 'correct' way to do the permissions on the weewx user.  I'll get that modification in there. 

For adding a cable link, I didn't have the thread a link to the cable info in it.  Now I do and I'll add it in there.

And about the public_html directory, I left it's use recommended for a basic up-and-running test before configuring it to go somewhere else.  And it did not exist after the weewx install, it was the one missing directory.  Likely weewx failed to get to the point where it creates it (while still dealing with some setup issues) before I created it myself.  I just assumed it wasn't part of the install.

[Tony Perkins]

 Suggested changes have been made on the linked page.  Thanks!

[Ted Garrison]

Steve- 

 Do you have Apache running on the machine?  You'll need that to serve up all those files in Public_html.

On Wed, Feb 6, 2013 at 6:12 PM, Steve Jackson <inven...@gmail.com> wrote:

Newbie user here,

Liked the step by step lesson, but I still can't get this darned thing working right.

Completely fresh install.

I have a Ubuntu 10.04 server I built that is my SoftRAID5 array for home storage, so I figured I'd add on my weewx setup to it. 
I do not use MySQL, assuming the sql lite would do for my needs (what do I know?)

I am Linux-aware enough to be extremely dangerous, I have no programming background or software experience, whatsoever.
My single attempt at it, writing a BASIC program, was 30+ years ago and I was able only to lock up the computer.

However, I did succeed in making this old Dell 800 into a 4.5 TB samba server for the house, by myself, so ... maybe I'm more frustrated than hopeless.

My weewx Goals:

1)  make the weather console information (Davis WeatherVue) available to other PCs on my home network, via browser
2)  start reporting my weather to CWOP/ APRS FindU  (I am a ham operator)
3)  store the data off the weather station for analysis, etc.

Here is what I cannot do so far:

1)  figure out how to access weewx from any other computer in the house
2)  see any sort of weewx graphical display at all on the server itself
3)  see any reports being logged on the internet servers I am sending to
4)  get the weewx to start up by itself when the server is rebooted

What is working:

1)  png renditions are being created in public_html, although I can only see them by clicking on them locally
2)  local terminal window shows the weather station data updating every so many seconds

One user on the Ubuntu box,
"steve"

I used to have this whole setup in a Windows XP computer running software I forget the name of, but that machine bit the dust and so did the weather station.
I had Davis re-build the VantageVue setup and decided to not restore the Widows setup if I could avoid it.
The Windows application I used sucked up 50% of the CPU power at all times.  Horrible.


Any tips are appreciated. 
You could also tell me to find someone who has suitable background to make this work, since I clearly do not, and I would not take offense.
My tech skills lie elsewhere.


Steve

***

--
You received this message because you are subscribed to the Google Groups "Weewx Weather Station Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Thomas Keffer]

Exactly right. See this thread: https://groups.google.com/d/msg/weewx-user/TVj_T6LKKBY/8lskvzc6kLQJ

-tk

[Steve Jackson]

Ah, I see. 

I actually thought that, and checked for Apache running using the command <ps -aux | grep apache>  (my meager linux fu is showing) and I thought that was what I needed to have. 
Guess not!

I went to the thread mentioned, but my version of OS appears to not be compatible ... I get some kind of error message sequence telling me the files/server I am looking for aren't there.

I also thought I might use a simpler web server, like Ruby, but that doesn't seem to install either.

Hmm.  I figured this project would either work fine, right off, or rapidly get beyond my skill set.  It's the latter.  Alas. 

I **really** don't want to break my server, it took me *weeks* to figure out how to make it work, and I'd best not ruin it now.
I back-rev'd to Ubuntu 10.04 to make the Samba / SoftRAID server because newer versions of the OS just would not work right.

I guess my options now are (a) get another raspberry Pi and dedicate it to the weewx setup or (b) just go back to Windows like I had before.  (Ugh.)


***

[John Canfield]

Would this be any help for you?  I've never messed with a Raspberry but I can't imagine it wouldn't run Apache (or maybe I don't understand the problem?)

If you aren't familiar with Webmin, I highly recommend it for a GUI admin interface to your box.  I also use a nifty visual file manager app called Midnight Commander (MC) that makes it very easy to move around your file system and view/edit/change permissions, etc.

John
weather.janeandjohn.org

[Steve Jackson]

John

Thanks for the tips.

The problem appears to be that the apt-get install process for apache2 points to IP addresses that come back 404.  I assume this is because my Ubuntu 10.10 OS is no longer supported.  (?)

The links go to some security.ubuntu.org site ... the error messages are kind of cryptic, but basically they tell me the server directories that should have the install files are not there.

So, net is, I cannot figure out how to get a web server running on the computer I was trying to use. 

The weewx setup is working fine, otherwise.  I can see the local web page, the graphs, etc., just fine.  ( Well, mostly fine; it does show a moving weather satellite image of western Oregon for some reason.)   Because weewx is running fine, I thought the other work would be simple.

The top level problem ...  I cannot seem to get the weather to report 'out' of the computer, to CWOP (no idea why not) or as an for-the-home-LAN web page (due to no httpd).  And, if I can't make those functions work, then there's no point in trying to run weewx on the computer in question.  It's easier (for me) to just buy a different computer for the task.

Steve

[John Canfield]

I should buy a Raspberry Pi and play around with it - how can you go wrong for a few bucks?  It's been a year or two since I've set up a Ubuntu box so I'm scratching my head on this end.  One thing you could look at is adding distribution repositories to

/etc/apt/sources.list - see here.

In looking around the Ubuntu site earlier, I could not find a 10.10 release so it's possible Ubuntu has removed that from the distribution repositories, 
so if that's the case, you're out of luck for support.  I'm running 10.04 (LTS) on two servers and it still is getting support.

Any reason why you're not running Raspbian instead of Ubuntu?  Like I said, I don't know much about the Raspberry so maybe that's a dumb question.

On my two PC-Fit slims and a PC-FIT2 I think they are ARM CPUs and 10.04 is the latest release that will run on them.  Anyway, just a few random thoughts.

John
weather.janeandjohn.org

[Steve Jackson]

The computer I am using now is a Dell PowerServer 800, configured with Ubuntu 10.10, SoftRAID5 (several 2 TB drives), and Samba. 

It's the household server for everything from storage to backups to etc.

It runs this Ubuntu release because that's the only one that is stable with *both* Samba *and* the SoftRAID5 utilities.  I pulled out lots of hair trying 11.x and 12.x versions.  Unfortunately, I'm not a CS major, and computers are neither my job nor my hobby, so, I'm really at a big skill disadvantage and not really the target market for Linux.   One either knows what one is doing, or, one doesn't.

I was trying to run weewx on this Dell computer since it runs 24x7, has resilient power, and so forth.  But as you are likely accurately painting the situation, they have pulled support and therefore the repositories are gone.  Can't load a web server daemon ... gee ... seems implausible, but it appears to be so.

I have one Raspberry Pi but it's for a different purpose.  I have built a few kernels for it, on different flash drives, so if I get another one, I bet I could get this right and just dedicate it to the weewx setup.  I'd even mount a volume on the Dell server, across the network, for weather log storage.

[Thomas Keffer]

I'm sure it's possible to get Apache running on 10.10. Best bet is to try to use the 10.04 version. Do so by adding the following line to /etc/apt.sources.list:

deb http://security.ubuntu.com/ubuntu lucid-security main

Then try apt-get install apache2

For more info, see this: http://packages.ubuntu.com/lucid/web/apache2

-tk

[Steve Jackson]

Tom

Thanks.  I found that apt.sources.list file in /etc/apt/ and added the line you mentioned, but no joy.

Here's what comes back:

Err http://us.archive.ubuntu.com/ubuntu/ maverick-updates/main apache2-mpm-worker i386 2.2.16-1ubuntu3.5
  404  Not Found [IP: 91.189.91.14 80]
Err http://security.ubuntu.com/ubuntu/ maverick-security/main apache2-mpm-worker i386 2.2.16-1ubuntu3.5
  404  Not Found [IP: 91.189.92.200 80]
Err http://security.ubuntu.com/ubuntu/ maverick-security/main apache2 i386 2.2.16-1ubuntu3.5
  404  Not Found [IP: 91.189.92.200 80]
Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/a/apache2/apache2-mpm-worker_2.2.16-1ubuntu3.5_i386.deb  404  Not Found [IP: 91.189.92.200 80]
Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/a/apache2/apache2_2.2.16-1ubuntu3.5_i386.deb  404  Not Found [IP: 91.189.92.200 80]


As you mentioned, this sort of help is way above and beyond weewx help!

I am rather impressed with the crew here and the program is quite handy too.  I think I'll move it to a different OS / environment where it can have the resources needed.

regards

Steve


***

[Thomas Keffer]

Oh, well. 

If you're looking for an inexpensive head-less server to run these kinds of processes, I bought one of these and have been pretty happy with it. Price was $233, including shipping.

-tk

[mwall]

On Saturday, February 9, 2013 10:02:01 AM UTC-5, Tom Keffer wrote:

 

If you're looking for an inexpensive head-less server to run these kinds of processes, I bought one of these and have been pretty happy with it. Price was $233, including shipping.

we have had very good results with these:

http://www.ionicsplug.com/products.html

US$100 for the nimbus.

[David Hathaway]

I just wanted to thank you for this post, a year later.  I bought my wife an Oregon Scientific WMR200A for Christmas, and used your guide to connect it to MySQL.  It was a snap on my Ubuntu 12.04 desktop.  I simply followed your guide for setting up the MySQL database (I had an existing MySQL setup, so that was very easy).

There is a small but important typo: the correct command to create the weewx user is

CREATE USER 'weewx'@'localhost' IDENTIFIED BY 'weewx';

You had ...IDENTIFIED BY 'weewx.;

The following period needs to be an apostrophe.  The typo also exists on the linked page.

FWIW, I use 'Emma' to check out the data via a GUI.  I could probably use that to manage the creation of tables and users, but the command line is just so very useful and easy to document. :-)

Cheers for the help!

Dave

On Thursday, November 29, 2012 9:18:24 PM UTC-6, Tony Perkins wrote:

 I've created a tutorial on getting a Davis Vantage Vue running with:

WeeWX
A non-logging serial cable
Ubuntu Linux
MySQL

Hopefully this can help some other folks out along the way!

Up to date tutorial link: 
http://www.blueskybust.com/homewx.html

*snipped for brevity* 

[david hardy]

You seem to have left out  how to create the mysql database and tables. I can't seem to find any info on how that happens.
.
david

[David Hathaway]

Once you grant permission to MySQL for user weewx to do so, it creates the databases and tables by itself.

mysql -u root -p

(Enter your root password for mysql. Can't help you here)

CREATE USER 'weewx'@'localhost' IDENTIFIED BY 'weewx';

GRANT select, update, create, insert, delete on weewx.* to weewx@localhost;

GRANT select, update, create, insert, delete on stats.* to weewx@localhost;

quit

You can look in archive.py and stats.py if curious. 

HTH,

Dave