Re: Vantage Vue problem

[Thomas Keffer]

Sorry, but I have no experience with the Vantage Vue. I can only rely on Davis's assurances that it shares the same serial protocol as the VP2.

I doubt that this is a USB problem as your LOOP data is coming in as expected.

Can you set debug=1 in your weewx.conf configuration file and then run it? Then, the next time the archive records stop getting logged, send me the output from your system log (/var/log/syslog in Ubuntu).

-tk

On Mon, Jan 24, 2011 at 9:29 AM, barton <bartonp...@gmail.com> wrote:

I have a Vantage Vue and while it seems to work OK for a while it
hangs often. The symptoms: after a while (12 to 24 hours) the
archiving stops. It looks like the LOOP continues to work ok but as
archive records trigger most actions, everything stops working.

I suspect my USB-data-logger. It doesn't look like the ttyUSB0 is
changing like some have experienced as LOOP still works and I see no
indication that the ports has changed.

I have tried running on batteries only but the same problems happens.

Has anyone else experienced this or used the new Vantage Vue. The Vue
looks like the VP2 in its data format for LOOP etc.

I have contacted Davis but am still waiting.

Thanks

--
You received this message because you are subscribed to the Google Groups "Weewx Weather Station Discussion" group.
To post to this group, send email to weewx...@googlegroups.com.
To unsubscribe from this group, send email to weewx-user+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/weewx-user?hl=en.

[Joe Percival]

Have you checked to see if maybe it's the same problem some were having with Davis USB noise? search the forum for USB or USB0. There are 2 levels of fixes, one hardware, and one software. Supposedly the hardware fix is something like 90% and the software fix 110% but I could not get the software fix to work (though I did not try very hard).
joe

[barton]

I will do that. I have debug=1 now and the program has been running OK
for 24hrs plus.

Another quick question RE my post about AWEKAS. It looks like all I
have to do is add an [[[AWEKAS]]] template to the Standard Skins. My
only problem is I need to strip off the trailing type information like
37F etc. AWEKAS just wants the raw values without the F, mph etc. Also
how would I get the Month, Day, Year, Time?

Thanks

On Jan 24, 10:32 am, Thomas Keffer <tkef...@gmail.com> wrote:
> Sorry, but I have no experience with the Vantage Vue. I can only rely on
> Davis's assurances that it shares the same serial protocol as the VP2.
>
> I doubt that this is a USB problem as your LOOP data is coming in as
> expected.
>
> Can you set debug=1 in your weewx.conf configuration file and then run it?
> Then, the next time the archive records stop getting logged, send me the
> output from your system log (/var/log/syslog in Ubuntu).
>
> -tk
>

> On Mon, Jan 24, 2011 at 9:29 AM, barton <bartonphill...@gmail.com> wrote:
> > I have a Vantage Vue and while it seems to work OK for a while it
> > hangs often. The symptoms: after a while (12 to 24 hours) the
> > archiving stops. It looks like the LOOP continues to work ok but as
> > archive records trigger most actions, everything stops working.
>
> > I suspect my USB-data-logger. It doesn't look like the ttyUSB0 is
> > changing like some have experienced as LOOP still works and I see no
> > indication that the ports has changed.
>
> > I have tried running on batteries only but the same problems happens.
>
> > Has anyone else experienced this or used the new Vantage Vue. The Vue
> > looks like the VP2 in its data format for LOOP etc.
>
> > I have contacted Davis but am still waiting.
>
> > Thanks
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Weewx Weather Station Discussion" group.
> > To post to this group, send email to weewx...@googlegroups.com.
> > To unsubscribe from this group, send email to

> > weewx-user+...@googlegroups.com<weewx-user%2Bunsu...@googlegroups.com>
> > .

[Thomas Keffer]

Hello, Barton

I'm not familiar with AWEKAS, but it sounds like it is an XML or HTML format? If so, you're on the right track.

Extensive information on how to control formatting of tags is in the guide "Customizing weewx", which is included in your distribution or available here. You want Section 2. You should really read it, but let me summarize what you can do (NOT tested):

If, for example, you want the yearly high temperature with no trailing unit label, it would be available as:

  $year.outTemp.max.formatted

This would use the default format (something like "%.1f") specified in your skin configuration file skin.conf. If you want explicit formatting , you can use:

  $year.outTemp.max.format("%.2f", "Not Available")

Which would use the indicated format for valid values, and the label "Not Available" for null values.

In a similar manner, times can be formatted using strftime formatting. So, if you wanted the month and year of that max temperature, you could say something like:

  $year.outTemp.maxtime.format("%B %Y")

which would return "January 2011". And so on.

-tk

To unsubscribe from this group, send email to weewx-user+...@googlegroups.com.

[barton]

Wonderful. I figured out that I could use $current.outTemp.raw etc. to
get the temperature without the label. I did originally read the
customisation document but with so much to absorb I forgot some of it.
Thanks I think I have the AWEKAS figured out. I will attach the
template once I have it working with a little info on how AWEKAS
works.

Thanks

On Jan 24, 2:10 pm, Thomas Keffer <tkef...@gmail.com> wrote:
> Hello, Barton
>
> I'm not familiar with AWEKAS, but it sounds like it is an XML or HTML
> format? If so, you're on the right track.
>
> Extensive information on how to control formatting of tags is in the

> guide *"Customizing
> weewx"*, which is included in your distribution or available
> here<http://www.weewx.com/docs/customizing.htm>.

> You want Section 2. You should really read it, but let me summarize what you
> can do (NOT tested):
>
> If, for example, you want the yearly high temperature with no trailing unit
> label, it would be available as:
>
>   $year.outTemp.max.formatted
>
> This would use the default format (something like "%.1f") specified in your
> skin configuration file skin.conf. If you want explicit formatting , you can
> use:
>
>   $year.outTemp.max.format("%.2f", "Not Available")
>
> Which would use the indicated format for valid values, and the label "Not
> Available" for null values.
>
> In a similar manner, times can be formatted using

> strftime<http://docs.python.org/library/datetime.html#strftime-strptime-behavior>formatting.

> So, if you wanted the month and year of that max temperature,
> you could say something like:
>
>   $year.outTemp.maxtime.format("%B %Y")
>
> which would return "January 2011". And so on.
>
> -tk
>

> > <weewx-user%2Bunsu...@googlegroups.com<weewx-user%252Buns...@googlegroups.com>

[Thomas Keffer]

Sounds good!

One bit of caution: the ".raw" suffix literally pumps out the raw value. Python then converts it to "something sensible". Normally, this doesn't cause a problem with the one notable exception of value "None", that is a null value. You're better off using suffix .formatted.

-tk

To unsubscribe from this group, send email to weewx-user+...@googlegroups.com.

[barton]

Here is the little file I called awekas.html.tmp

AWEKAS_Template_start<br>
$current.outTemp.formatted<br>
$current.outHumidity.formatted<br>
$current.barometer.formatted<br>
$current.rainRate.formatted<br>
$current.windSpeed.formatted<br>
$current.windDir.formatted<br>
$current.dateTime.format("%H:%M:%S")<!-- this wants to look like
13:30:10--><br>
$current.dateTime.format("%Y%m%d")<!-- this wants to look like
20110124--><br>
Template_V1.5<br>

By adding a [[[AWEKAS]]] to [[ToDate]] with template = awekas.html.tmp
I get a file called awekas.html. I then make a symlink to my apache
web server's root directory. At AWEKAS (http://www.awekas.at/en/
index.php) you can create an account (at the bottom of the page
"Register here"). Set up your user information, most of it is self
explanatory. The only things that might not be are:
1 "Report Mode" you want wview
2 "Path to the datafile: http://" this is where the html file resides,
for my system it is "bartonphillips.dyndns.org/weather/awekas.html"
3 "Only for Virtual Weather Station and Davis Weatherlink Software"
section right below #2 above. Here you need to specify English values:
F, inHg, inch, miles/h, and YYYYMMDD.
4 Under "Information about your weather station" select your Vantage
Pro2 (I use Vantage Vue). Then select one for each, Temperature, Air
pressure, Precipitation and Anemometer.

Now check that you "agree" and click save. You can now click test to
see if everything is OK.

What AWEKAS does is read the awekas.html file from your server every
once in a while so you don't need to do any FTP etc.

I hope this is enough information.

Thanks

On Jan 24, 2:57 pm, Thomas Keffer <tkef...@gmail.com> wrote:
> Sounds good!
>
> One bit of caution: the ".raw" suffix literally pumps out the raw value.
> Python then converts it to "something sensible". Normally, this doesn't
> cause a problem with the one notable exception of value "None", that is a
> null value. You're better off using suffix .formatted.
>
> -tk
>

> > > > <weewx-user%2Bunsu...@googlegroups.com<weewx-user%252Buns...@googlegroups.com>
> > <weewx-user%252Buns...@googlegroups.com<weewx-user%25252Bun...@googlegroups.com>

[Thomas Keffer]

I'll give it a try (I'm on the road right now). It looks simple enough. Thanks for doing the leg work on this!!

-tk

To unsubscribe from this group, send email to weewx-user+...@googlegroups.com.

[barton]

RE my hang problem. I set debug to 1. I got an EMI error after about
12 hours of running so added the udev script to make a symlink to
vpro, which seems to work OK. I got another hang last night but it
looks different from previous ones as I still get the following:

Getting archive packets since 2011-01-25 22:30:00 MST (1296019800)
successfully woke up console
Retrieving 296 page(s); starting index= 0
successfully woke up console
Running reports for latest time in the database.
Requesting 200 LOOP packets.

This sequence continues every loop cycle but not new images etc or
RESTfull actions. I have stopped and restarted the service several
time but it just does the same thing.

Just out of curiosity, why don't you use the LOOP data instead of the
archive records? It would seem that all of the data is available from
the loop data if for some reason the archive records are not
available.

[barton]

I added some code to the VantagePro.py to see where the archive logic
was going. I added two syslog entries for 1) page never used (FF's) 2)
time stamp declining. What I get is the page never used exit.

I am not a Python programmer but I think I kinda understand some of
the logic used. This exit confirms what I see when I use minicom to
poke around with the LOOP and DMP commands. When the problem happens
the LOOP command still works OK but the DMP returns all 0xff's. When
everything is OK the DMP command returns a lot of binary data.

I think the problem is the USB-data-logger but want to convince myself
before having Davis swap it out. Any ideas?

[Thomas Keffer]

Before you give up on your logger, try clearing its memory.

  (stop weewx)

  cd /home/weewx
  ./bin/configure.py --clear-VantagePro weewx.conf

  Then restart weewx.

Let me know what's in the log. By the way, what version are you running? 1.9.2?

-tk

To unsubscribe from this group, send email to weewx-user+...@googlegroups.com.

[Thomas Keffer]

Just out of curiosity, why don't you use the LOOP data instead of the
archive records? It would seem that all of the data is available from
the loop data if for some reason the archive records are not
available.

It's a fair question.

The answer is that when I started this project I didn't have the confidence that I'd achieve the kind of stability weewx has shown. My experience was with wview which would crash about once a week. So, I was reluctant to give up the onboard storage that the VP2 is capable of. This was the simplest way to do it.

If I were to do it again, I think I'd rely on the LOOP data (which, after all, is all that most weather stations offer), with a strategy of using the archive data as a backstop in case of a crash. In fact, that's what I've been thinking for Version 2.0.

By the way, any experts on event-driven architecture out there? Weewx uses one, but with a severely constrained hard-wired state machine. I'd like to make it a little more flexible, but I'm rapidly out of my experience zone!

-tk

[barton]

I did try clearing the archive data but with wview's vpconfig program.
It didn't seem to make any difference. Next time I'll try it the weewx
way. The station has run all day yesterday and over night, so we will
see.

As for my LOOP question and your answer, that is kinda what I thought.
I have been looking at your code a lot and find it well designed and
commented. As I said I am not a Python programmer but have been
looking at some Python documentation and may take a crack at some
modifications. One that looks like it might be pretty easy is to
provide for Weather Underground's "rapid fire" uploads. It looks like
it would require a mod to the LOOP logic to put the data into the
queue and a flag in the RESTful section of the config file etc. The
StdPrint logic already outputs the LOOP data on every loop iteration
so something like that in the loop packet logic maybe. Always fun to
try something new.

Thanks

[Thomas Keffer]

That sounds pretty reasonable. The only thing you have to be careful of is that you only have 2 seconds until the next LOOP data is due. So, it's best that the rapidfire posts happen in a separate thread. If you choose to extend the RESTful service, this is already done for you so you're good there. Should you choose to write your own service (like StdPrint), you'll have to create your own thread.

It may also be necessary to have some "catchup" logic should the net connection go down for an extended period of time and clog the queue. You'll probably want to inspect the queue every once in a while and make sure there aren't hundreds of backed up posts in it!

Finally: just a word of warning, this part of weewx is very likely to change in V2. No reason not to experiment, and it could be a l-o-n-g time before I get to it (!), but be prepared to adapt your code to a new architecture.

Have fun! This is exactly the kind of experimentation I was hoping to encourage by doing an open, extensible system like weewx!

-tk

--
You received this message because you are subscribed to the Google Groups "Weewx Weather Station Discussion" group.
To post to this group, send email to weewx...@googlegroups.com.

To unsubscribe from this group, send email to weewx-user+...@googlegroups.com.

[barton]

Well the problem happened again last night after running for two days.
Here is the syslog entries around the time when it happened.

Jan 27 22:45:21 dell weewx[24627]: restful: Skipped record 2011-01-27
22:45:00 MST (1296193500) to CWOP station DW6375
Jan 27 22:45:21 dell weewx[24627]: **** CWOP: Wait interval (600)
has not passed.
Jan 27 22:50:04 dell weewx[24627]: VantagePro: LOOP #144; buffer not
full (0)... retrying
Jan 27 22:50:04 dell weewx[24627]: last message repeated 3 times
Jan 27 22:50:04 dell weewx[24627]: VantagePro: Max retries exceeded
while getting LOOP packets
Jan 27 22:50:04 dell kernel: [531982.984038] hub 6-0:1.0: port 2
disabled by hub (EMI?), re-enabling...
Jan 27 22:50:04 dell kernel: [531982.984043] usb 6-2: USB disconnect,
address 12
Jan 27 22:50:04 dell kernel: [531982.984219] cp210x ttyUSB0: cp210x
converter now disconnected from ttyUSB0
Jan 27 22:50:04 dell kernel: [531982.984232] cp210x 6-2:1.0: device
disconnected
Jan 27 22:50:04 dell weewx[24627]: Shut down RESTful thread.
Jan 27 22:50:04 dell weewx[24627]: Shut down StdReportService thread.
Jan 27 22:50:04 dell weewx[24627]: wxengine: Caught WeeWxIOError:
While getting LOOP packets
Jan 27 22:50:04 dell weewx[24627]: **** Waiting 60 seconds then
retrying...
Jan 27 22:50:04 dell kernel: [531983.224014] usb 6-2: new full speed
USB device using uhci_hcd and address 13
Jan 27 22:50:04 dell kernel: [531983.388360] usb 6-2: configuration #1
chosen from 1 choice
Jan 27 22:50:04 dell kernel: [531983.394291] cp210x 6-2:1.0: cp210x
converter detected
Jan 27 22:50:04 dell kernel: [531983.505013] usb 6-2: reset full speed
USB device using uhci_hcd and address 13
Jan 27 22:50:05 dell kernel: [531983.656386] usb 6-2: cp210x converter
now attached to ttyUSB1
Jan 27 22:51:04 dell weewx[24627]: wxengine: retrying...
Jan 27 22:51:04 dell weewx[24627]: wxengine: Using configuration file /
extra/weewx/weewx-1.9.2/weewx.conf.
Jan 27 22:51:04 dell weewx[24627]: VantagePro: successfully woke up
console
Jan 27 22:51:05 dell weewx[24627]: VantagePro: successfully woke up
console
Jan 27 22:51:05 dell weewx[24627]: wxengine: List of services to be
run:
Jan 27 22:51:05 dell weewx[24627]: ****
weewx.wxengine.StdCalibrate
Jan 27 22:51:05 dell weewx[24627]: **** weewx.wxengine.StdQC
Jan 27 22:51:05 dell weewx[24627]: archive: archive database /extra/
weewx/weewx-1.9.2/archive/weewx.sdb already exists.
Jan 27 22:51:05 dell weewx[24627]: stats: statistical database /extra/
weewx/weewx-1.9.2/archive/stats.sdb already exists.
Jan 27 22:51:05 dell weewx[24627]: stats: stats database up to date.
Jan 27 22:51:05 dell weewx[24627]: **** weewx.wxengine.StdArchive
Jan 27 22:51:05 dell weewx[24627]: ****
weewx.wxengine.StdTimeSynch
Jan 27 22:51:05 dell weewx[24627]: **** weewx.wxengine.StdPrint
Jan 27 22:51:05 dell weewx[24627]: wxengine: Data will be posted to
Wunderground
Jan 27 22:51:05 dell weewx[24627]: wxengine: Data will be posted to
PWSweather
Jan 27 22:51:05 dell weewx[24627]: wxengine: Data will be posted to
CWOP
Jan 27 22:51:05 dell weewx[24627]: wxengine: Started thread for
RESTful upload sites.
Jan 27 22:51:05 dell weewx[24627]: **** weewx.wxengine.StdRESTful
Jan 27 22:51:05 dell weewx[24627]: ****
weewx.wxengine.StdReportService
Jan 27 22:51:05 dell weewx[24627]: VantagePro: Getting archive packets
since 2011-01-27 22:45:00 MST (1296193500)
Jan 27 22:51:05 dell weewx[24627]: VantagePro: successfully woke up
console
Jan 27 22:51:06 dell weewx[24627]: VantagePro: <blp> Retrieving 356
page(s); starting index= 0
Jan 27 22:51:06 dell weewx[24627]: VantagePro: page never used exit
Jan 27 22:51:06 dell weewx[24627]: VantagePro: successfully woke up
console
Jan 27 22:51:06 dell weewx[24627]: wxengine: Starting main packet
loop.
Jan 27 22:51:07 dell weewx[24627]: VantagePro: successfully woke up
console
Jan 27 22:51:07 dell weewx[24627]: VantagePro: successfully woke up
console
Jan 27 22:51:07 dell weewx[24627]: VantagePro: Clock error is -1.67
seconds (positive is fast)
Jan 27 22:51:07 dell weewx[24627]: VantagePro: Requesting 200 LOOP
packets.
Jan 27 22:51:08 dell weewx[24627]: VantagePro: successfully woke up
console
Jan 27 22:55:16 dell weewx[24627]: VantagePro: new archive record due.
Canceling loop
Jan 27 22:55:17 dell weewx[24627]: VantagePro: successfully woke up
console
Jan 27 22:55:17 dell weewx[24627]: VantagePro: Getting archive packets
since 2011-01-27 22:45:00 MST (1296193500)
Jan 27 22:55:17 dell weewx[24627]: VantagePro: successfully woke up
console
Jan 27 22:55:17 dell weewx[24627]: VantagePro: <blp> Retrieving 356
page(s); starting index= 0
Jan 27 22:55:17 dell weewx[24627]: VantagePro: page never used exit
Jan 27 22:55:18 dell weewx[24627]: VantagePro: successfully woke up
console
Jan 27 22:55:18 dell weewx[24627]: reportengine: Running reports for

latest time in the database.

Jan 27 22:55:18 dell weewx[24627]: VantagePro: Requesting 200 LOOP
packets.
Jan 27 22:55:18 dell weewx[24627]: reportengine: Running report
StandardReport
Jan 27 22:55:18 dell weewx[24627]: reportengine: Found configuration
file /extra/weewx/weewx-1.9.2/skins/Standard/skin.conf for report
StandardReport


As you can see an EMI event happened at Jan 27 22:50:04. I have the
udev script that symlinks to vpro.

It looks like even though the USB port changed to ttyUSB1 the udev
script kept the vpro symlink correct and weewx restart OK -- but the
archive data is always all 0xff's as the new line I added (<blp>)
shows:

Jan 27 22:51:05 dell weewx[24627]: VantagePro: Getting archive packets
since 2011-01-27 22:45:00 MST (1296193500)
Jan 27 22:51:05 dell weewx[24627]: VantagePro: successfully woke up
console
Jan 27 22:51:06 dell weewx[24627]: VantagePro: <blp> Retrieving 356
page(s); starting index= 0
Jan 27 22:51:06 dell weewx[24627]: VantagePro: page never used exit
Jan 27 22:51:06 dell weewx[24627]: VantagePro: successfully woke up
console

I think I am going to have Davis swap out my USB-data-logger and I
will also try to find some fetus beads for my USB cable. It may be
that this data logger is just very very sensitive to EMI flashes.

Any other ideas?

On Jan 27, 9:44 am, Thomas Keffer <tkef...@gmail.com> wrote:
> That sounds pretty reasonable. The only thing you have to be careful of is
> that you only have 2 seconds until the next LOOP data is due. So, it's best
> that the rapidfire posts happen in a separate thread. If you choose to
> extend the RESTful service, this is already done for you so you're good
> there. Should you choose to write your own service (like StdPrint), you'll
> have to create your own thread.
>
> It may also be necessary to have some "catchup" logic should the net
> connection go down for an extended period of time and clog the queue. You'll
> probably want to inspect the queue every once in a while and make sure there
> aren't hundreds of backed up posts in it!
>
> Finally: just a word of warning, this part of weewx is very likely to change
> in V2. No reason not to experiment, and it could be a l-o-n-g time before I
> get to it (!), but be prepared to adapt your code to a new architecture.
>
> Have fun! This is exactly the kind of experimentation I was hoping to
> encourage by doing an open, extensible system like weewx!
>
> -tk
>

> > weewx-user+...@googlegroups.com<weewx-user%2Bunsu...@googlegroups.com>
> > .

[Thomas Keffer]

I'm on the road with just an Android phone but I think your diagnosis is correct. You're definitely getting the CP2101 hang problem and it seems it is also affecting the loggers ability to arxhive.

> To unsubscribe from this group, send email to weewx-user+...@googlegroups.com.

[barton]

I spoke to Davis and we spent some time on the WeatherLink Windows
software and they concur that it looks like the USB-data-logger is the
problem. They are going to send a new one out next week. I hope that
solves the problem so I can play with the "rapid fire" logic. Looking
forward to experimenting with Python it looks like an interesting
little language.

On Jan 28, 12:38 pm, Thomas Keffer <tkef...@gmail.com> wrote:
> I'm on the road with just an Android phone but I think your diagnosis is
> correct. You're definitely getting the CP2101 hang problem and it seems it
> is also affecting the loggers ability to arxhive.
>

> ...
>
> read more »

[Julian]

On Jan 27, 1:21 am, Thomas Keffer <tkef...@gmail.com> wrote:
> By the way, any experts on event-driven architecture out there? Weewx uses
> one, but with a severely constrained hard-wired state machine. I'd like to
> make it a little more flexible, but I'm rapidly out of my experience zone!

I know Twisted[1] fairly well, in fact I'd recommend it over using
threads as it adds a lot of goodness to Python and makes talking over
sockets using FTP/HTTP etc very easy.

[1] http://twistedmatrix.com/trac/

[Marijn Vriens]

Another possibiltiy is to use Multiprocessing that comes as part of the Python standard library from 2.6 onwards ( http://docs.python.org/library/multiprocessing.html ). I found the learning curve fairly gentle.

--
You received this message because you are subscribed to the Google Groups "Weewx Weather Station Discussion" group.
To post to this group, send email to weewx...@googlegroups.com.

To unsubscribe from this group, send email to weewx-user+...@googlegroups.com.

[Jason Rennie]

I've also suggested twisted :)  It's certainly a great fit for weewx, but Thomas was rightly hesitant to change since it'd be a lot of code reworking for little practical benefit.

Jason

On Wed, Jun 15, 2011 at 5:15 PM, Julian <bigj...@gmail.com> wrote:

I know Twisted[1] fairly well, in fact I'd recommend it over using

threads as it adds a lot of goodness to Python and makes talking over
sockets using FTP/HTTP etc very easy.

[1] http://twistedmatrix.com/trac/

--
You received this message because you are subscribed to the Google Groups "Weewx Weather Station Discussion" group.
To post to this group, send email to weewx...@googlegroups.com.

To unsubscribe from this group, send email to weewx-user+...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/weewx-user?hl=en.

--
Jason Rennie
Research Scientist

Google/ITA Software
+1 617-446-3651