reading a cvs sentence from an Arduino weather station into weewx

[Yann Chemin]

Hi,

I am just starting to read so I must have not yet looked into the right place. So any reading pointer would be very appreciated.

I have an Arduino station that gives a print I can gather from pySerial as follow:

import sys

import serial

import numpy as np

print("give /dev/USB0 or /dev/tty0 or /dev/ACM0 as argument to this code")

ser = serial.Serial(sys.argv[1], 9600)

while True:

data=[]

received=ser.readline()

for word in received.split(",")

data.append(word)

array=np.asarray(data)

#Step 1: import array data in SQL and/or

#Step 2: import array data in display library

#Arduino organisation of the output sentence in csv format

#Serial.print("lon,lat,altitude,sats,date,GMTtime,winddir");

#Serial.print(",windspeedms,windgustms,windspdms_avg2m,winddir_avg2m,windgustms_10m,windgustdir_10m");

#Serial.print(",humidity,tempc,raindailymm,rainhourmm,rain5mmm,rainindicate,pressure");

I would like this sentence (or any other style as needed) to be read by weewx.

So far I can see only stations and single sensor attempts. Is there an internal Python parser that is configurable to absorb such kind of communication string?

Thank you for any help.

Yann

[mwall]

On Saturday, March 7, 2015 at 12:54:41 AM UTC-5, Yann Chemin wrote:

So far I can see only stations and single sensor attempts. Is there an internal Python parser that is configurable to absorb such kind of communication string?

string.split will turn a comma-delimited string into an array of values.  but that is probably not the question you are really asking...

is your intent to run weewx on the arduino to which the serial device is attached?

do you intend to have one program write data to a file, then have weewx read from that file?  (see the fileparse extension in weewx if you want to do that)

do you intend to create a weewx driver to read directly from the serial device?  (see the ws1 driver for the simplest example of how to read from a serial device)

m

[Yann Chemin]

Hi m

On Mar 7, 2015, at 6:48 PM, mwall <mw...@users.sourceforge.net> wrote:

On Saturday, March 7, 2015 at 12:54:41 AM UTC-5, Yann Chemin wrote:

So far I can see only stations and single sensor attempts. Is there an internal Python parser that is configurable to absorb such kind of communication string?

string.split will turn a comma-delimited string into an array of values.  but that is probably not the question you are really asking…

Ah yes indeed, thx for that.

is your intent to run weewx on the arduino to which the serial device is attached?

I intend to run this script and weewx on a Raspberry PI (with Raspbian) for an educational purpose and plug the Arduino with the USB cable.

do you intend to have one program write data to a file, then have weewx read from that file?  (see the fileparse extension in weewx if you want to do that)

I will possibly extend the script to store in a database/file indeed, but the main principal purpose is to display the weather information on a screen continuously for school children. File parsing may take longer time with increasing size.

do you intend to create a weewx driver to read directly from the serial device?  (see the ws1 driver for the simplest example of how to read from a serial device)

That may be a good option actually, let me read.

Thank you,

yann

m

--
You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/hB1knRFdSvY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to weewx-user+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Yann Chemin]

Hi,

I got a modification of ws1.py done

and included it in the drivers directory, 

and added its name to the weewx.conf file instead of the Simulator.

pi@raspberrypi /home/weewx/bin $ ./weewxd ../weewx.conf

Traceback (most recent call last):

  File "./weewxd", line 61, in <module>

    weewx.engine.main(options, args)

  File "/home/weewx/bin/weewx/engine.py", line 831, in main

    engine = EngineClass(config_dict)

  File "/home/weewx/bin/weewx/engine.py", line 69, in __init__

    self.setupStation(config_dict)

  File "/home/weewx/bin/weewx/engine.py", line 87, in setupStation

    driver = config_dict[stationType]['driver']

  File "/usr/lib/python2.7/dist-packages/configobj.py", line 554, in __getitem__

    val = dict.__getitem__(self, key)

KeyError: 'mws'

It seems to ask for a missing declaration of the driver in a dictionary. 

How do I do that? More importantly: where?

Thanks,

Yann

PS: https://github.com/YannChemin/MWS/tree/master/weewx/mws.py is the mws driver

[Yann Chemin]

Had to put the driver in the SVN tree and build+install again, then add a section [MWS] in weewx.conf with the driver = ... thinggy. Daemon is up, waiting for data...

[Yann Chemin]

Nothing comes up... no updated index.html

[mwall]

On Wednesday, June 24, 2015 at 4:56:55 AM UTC-4, Yann Chemin wrote:

Had to put the driver in the SVN tree and build+install again, then add a section [MWS] in weewx.conf with the driver = ... thinggy. Daemon is up, waiting for data...

what is 'SVN tree'?

this is how to install your own driver:

1) do a standard weewx install
2) put your driver in the user directory, e.g., /home/weewx/bin/user/mws.py
3) modify weewx.conf:

[Station]
    station_type = MWS
[MWS]
    driver = user.mws
    ...    # options for your driver here

4) start weewx

btw, you should start with the ws1 driver from the weewx repository at github.

your project is very interesting!  please let us know how it progresses.

http://www.iwmi.cgiar.org/resources/mobile-weather-stations/

m

[Andrew Milner]

SVN is a version control system

[Thomas Keffer]

Andrew, I think Matthew knows this, seeing as how he used it not only in his previous life, but for 3 years with weewx.

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

[Andrew Milner]

tom - he did, for some reason, ask what an svn tree was though!!!!

[Yann Chemin]

OK guys,

will get at it today hopefully,

Cheers,

Yann

[Yann Chemin]

Things get interesting, Arduino resets when the serial port is opened.

Is there a way in weewx to not open and close to collect data, but keep the port opened and sample when needed?

[Yann Chemin]

ok there seem to be a possibility by setting up a 10microF capacitor between pin RESET and pin GND to disable that function hardware side.
http://arduino.stackexchange.com/questions/439/why-does-starting-the-serial-monitor-restart-the-sketch

[Yann Chemin]

OK got data string from the new user driver by launching it manually and both read() and get_readings() work (see prints  below). However it does not report anything in the index.html, and it does not reload anymore.

Any help to get more info where the bottleneck is, would be great,

thank you,

Yann

(modified functions from ws1.py)
------------------------------------------

    def read(self):
        global last
        buf = ''
        n = 0 #test for minimum length sentence
        count = 0 #keep track of how many read lines
        while n < 114:
                try:
                        buf = self.serial_port.readline()
                        n = len(buf)#maybe useful for debug in future
                        count += 1
                        print(buf) <= first print of the list
                        if DEBUG_READ:
                                logdbg(buf)
                        if n >= 114 and count >= 2:
                                break
                except:
                        #do nothing
                        count += 0
        return buf

    def get_readings(self):
        b = []
        bad_byte = False
        while True:
            c = self.read()
            b.append([cc.strip() for cc in c.split(',')])
            print(b[0]) <= second print of the list content
        if DEBUG_READ:
            logdbg(b[0])
        return b[0]


Manual start
-----------------
yann@yann-OK:/home/weewx$ PYTHONPATH=bin python bin/user/mws.py
1435342424.14

000000,0.000000,0.00,0,2000-00-00,00:00:00,-1,0.0,0.0,0,0.0,0,0.0,0,49.6,26.4,0.00,0.00,0,0,100865.00,4.03,0.00,0,3.94,0.00,

['000000', '0.000000', '0.00', '0', '2000-00-00', '00:00:00', '-1', '0.0', '0.0', '0', '0.0', '0', '0.0', '0', '49.6', '26.4', '0.00', '0.00', '0', '0', '100865.00', '4.03', '0.00', '0', '3.94', '0.00', '']
0.000000,0.000000,0.00,0,2000-00-00,00:00:00,-1,0.0,0.0,0,0.0,0,0.0,0,49.6,26.4,0.00,0.00,0,0,100865.00,4.03,0.00,

['000000', '0.000000', '0.00', '0', '2000-00-00', '00:00:00', '-1', '0.0', '0.0', '0', '0.0', '0', '0.0', '0', '49.6', '26.4', '0.00', '0.00', '0', '0', '100865.00', '4.03', '0.00', '0', '3.94', '0.00', '']
0.000000,0.000000,0.00,0,2000-00-00,00:00:00,-1,0.0,0.0,0,0.0,0,0.0,0,49.7,26.4,0.00,0.00,0,0,100865.25,3.96,0.00,

['000000', '0.000000', '0.00', '0', '2000-00-00', '00:00:00', '-1', '0.0', '0.0', '0', '0.0', '0', '0.0', '0', '49.6', '26.4', '0.00', '0.00', '0', '0', '100865.00', '4.03', '0.00', '0', '3.94', '0.00', '']
0.000000,0.000000,0.00,0,2000-00-00,00:00:00,-1,0.0,0.0,0,0.0,0,0.0,0,49.8,26.4,0.00,0.00,0,0,100869.00,3.97,0.00,

['000000', '0.000000', '0.00', '0', '2000-00-00', '00:00:00', '-1', '0.0', '0.0', '0', '0.0', '0', '0.0', '0', '49.6', '26.4', '0.00', '0.00', '0', '0', '100865.00', '4.03', '0.00', '0', '3.94', '0.00', '']
0.000000,0.000000,0.00,0,2000-00-00,00:00:00,-1,0.0,0.0,0,0.0,0,0.0,0,49.7,26.4,0.00,0.00,0,0,100864.25,3.94,0.00,

['000000', '0.000000', '0.00', '0', '2000-00-00', '00:00:00', '-1', '0.0', '0.0', '0', '0.0', '0', '0.0', '0', '49.6', '26.4', '0.00', '0.00', '0', '0', '100865.00', '4.03', '0.00', '0', '3.94', '0.00', '']
0.000000,0.000000,0.00,0,2000-00-00,00:00:00,-1,0.0,0.0,0,0.0,0,0.0,0,49.3,26.4,0.00,0.00,0,0,100863.50,3.94,0.00,

['000000', '0.000000', '0.00', '0', '2000-00-00', '00:00:00', '-1', '0.0', '0.0', '0', '0.0', '0', '0.0', '0', '49.6', '26.4', '0.00', '0.00', '0', '0', '100865.00', '4.03', '0.00', '0', '3.94', '0.00', '']
0.000000,0.000000,0.00,0,2000-00-00,00:00:00,-1,0.0,0.0,0,0.0,0,0.0,0,49.3,26.4,0.00,0.00,0,0,100866.75,4.00,0.00,

['000000', '0.000000', '0.00', '0', '2000-00-00', '00:00:00', '-1', '0.0', '0.0', '0', '0.0', '0', '0.0', '0', '49.6', '26.4', '0.00', '0.00', '0', '0', '100865.00', '4.03', '0.00', '0', '3.94', '0.00', '']
0.000000,0.000000,0.00,0,2000-00-00,00:00:00,-1,0.0,0.0,0,0.0,0,0.0,0,49.3,26.4,0.00,0.00,0,0,100866.00,3.94,0.00,

['000000', '0.000000', '0.00', '0', '2000-00-00', '00:00:00', '-1', '0.0', '0.0', '0', '0.0', '0', '0.0', '0', '49.6', '26.4', '0.00', '0.00', '0', '0', '100865.00', '4.03', '0.00', '0', '3.94', '0.00', '']
0.000000,0.000000,0.00,0,2000-00-00,00:00:00,-1,0.0,0.0,0,0.0,0,0.0,0,49.1,26.4,0.00,0.00,0,0,100869.75,3.99,0.00,

['000000', '0.000000', '0.00', '0', '2000-00-00', '00:00:00', '-1', '0.0', '0.0', '0', '0.0', '0', '0.0', '0', '49.6', '26.4', '0.00', '0.00', '0', '0', '100865.00', '4.03', '0.00', '0', '3.94', '0.00', '']

[Yann Chemin]

Hi,

got it to work:
https://plus.google.com/+YannCheminDr/posts/13GG8rSNVmP?pid=6167101800376248786&oid=100102165245262460868

For info:
Finally got a self-rolled driver for the Arduino Weather Station (http://www.iwmi.cgiar.org/resources/mobile-weather-stations/) to connect with weewx (www.weewx.com), here is a night data accumulation. Code here: https://github.com/YannChemin/MWS/tree/master/weewx