Version 3.6.0 hits the streets

[Thomas Keffer]

Many thanks to Matthew Wall and Gary Roderick for getting this release out!

There are several new features, but the two biggest were both provided by Gary:

• Added the ability to run reports using a cron-like notation, instead of with every report cycle. See User's Guide for details. 
• Added the ability to easily import CSV, Weather Underground, and Cumulus data using a new utility, wee_import.

In addition, there are lots of fixes for the newer drivers, particularly the CC3000 and TE923. We can thank Matthew's dogged determination for finding workarounds for the many mysterious glitches in these instruments.

With this release we have rearranged the User's Guide, and broken out the hardware-specific information into its own Hardware Guide.

You can download the new version from the usual places.

-tk

[vince]

On Friday, October 7, 2016 at 5:12:02 PM UTC-7, Tom Keffer wrote:

With this release we have rearranged the User's Guide, and broken out the hardware-specific information into its own Hardware Guide.

upgraded (debian setup.py method) from 3.5.0 - perfect as always.

The documentation reorganization especially the Hardware Guide is 'fabulous'. 

[Andrew Milner]

..... not quite perfect ......

Just upgraded, and thought I would correct some errors from a previous import by using wee_import from WU with full qc checking - as available in the now standard wee_import.

Created a wuimport.conf, and modified the qc section of weewx.conf

The --dry-run proceeded to run looking OK, but when I then went to do the real import I got:

unit system of incoming record (0x01) differs from 'archive' table in 'archive' database (0x10)

**** nothing done, exiting

I AM using a metric database, but my WU data is also in metric - so where/how do I tell wee_import the format of the incoming record?

PS I had previously used wee_import without problems, but before the creation of the now mandatory wee_import config file and the latest qc mods.

[gjr80]

Andrew,

My bad, wee_import was being told to write to a US database irrespective of the underlying database units, the QC department is being fired as I write this. The attached weeimport.py should fix the issue, usual deal:

• move the original weewx/bin/weeimport/weeimport.py aside by renaming it to something like weeimport_orig.py
• copy the attached weeimport.py in its place
• run wee_import
• let us know the outcome

Gary

[Andrew Milner]

Gary

I hasve got into a total mess now .... getting errors all over the place

After the 3.6 instasll I had a file called wee_import in /bin which I renamed to wee_import.orig and moved the new wee_import.py in its place. 

THEN I saw there is also a weeimport directory, which contains wee_import.py

I am still getting all sorts of errors .......

What have I done wrong?  I tried putting wee_import.orig back to wee_import

no joy

I have no touched the files in the /bin/weeimport directory yet

heeeeeeeeeeeeeeeeeeeeeeeelp ......

--
You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/mlw2k1encLU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to weewx-user+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[gjr80]

Ok, that is a mess. Have another read at the dot point instructions I provided.

On 8 October 2016 at 07:37, gjr80 <gjrod...@gmail.com> wrote:

Andrew,

My bad, wee_import was being told to write to a US database irrespective of the underlying database units, the QC department is being fired as I write this. The attached weeimport.py should fix the issue, usual deal:

• move the original weewx/bin/weeimport/weeimport.py aside by renaming it to something like weeimport_orig.py
• copy the attached weeimport.py in its place
• run wee_import
• let us know the outcome

Gary

The bug that caused your original problem is in the file weewx/bin/weeimport/weeimport.py (note the .py extension), that is the file you need to move aside and replace. The file wee/bin/wee_import (note, no .py extension) is fine. You will need to put things back how they were and then move aside original weewx/bin/weeimport/weeimport.py (note the .py extension) aside and replace it with the previously attached weeimport.py

Gary

[Andrew Milner]

Gary

My big bad ........

OK - an import of 20 months is back under way ..... let's see if the result is better than last time as qc is now invoked!!

Andrew

phew .... good job this is just a test system!!!!

I do wish we would go back to using the word 'bad' as an adjective rather than a noun!!!!!!

So .... that was a Big bad mistake of mine - not reading the instructions carefully enough!!  At least I read the new instructions in the new user guides - fantastic documentation that is I must say!!! 

Oh yes - somewhere in the instructions (or help, not sure which) - the time in the --date command is in brackets.  I presume this is because the time is optional - but why the brackets?  An option in syntax definitions is (I thought) indicated by | whilst the inclusion of brackets implies the time should be enclosed in brackets - which is wrong (I think).

[gjr80]

 

Oh yes - somewhere in the instructions (or help, not sure which) - the time in the --date command is in brackets.  I presume this is because the time is optional - but why the brackets?  An option in syntax definitions is (I thought) indicated by | whilst the inclusion of brackets implies the time should be enclosed in brackets - which is wrong (I think).

It is the Utilities guide and the --help usage text. The hh:mm in a range is optional.  I will get some input from Tom and Matthew, can you have nested [ ], if not you end up with a very long line.

Gary

[Andrew Milner]

Square brackets perhaps??  For some reason the round brackets looks as though they should be included (to sill me at least)!!

This may be helpful .....

http://pcsupport.about.com/od/commandlinereference/a/command-syntax.htm

no doubt there is a definition for python somewhere too....

[Andrew Milner]

Just seen an 'oddity' .....

accessing the rpi via ssh from windows via putty

when the window is small I get two lines per period ...

one line for first tranche of 250 records with last timestamp date and time

one line for remaining records of period with last timestamp date and time

In full screen window this becomes much easier to read and has, per period

one line for all records, with last timestamp date and time AND TIMEZONE AND EPOCH TIME

Not sure who 'loses' the timezone and epoch time in the small window .....

Andrew

[Peter_F]

Thanks Tom, Matthew and Gary,

I upgraded without any issues.  One thing I noticed though is that the Weather Underground RapidFire rest posts are now logged by default.  I added 'log_success = False' to my weewx.conf to disable it.  I have a Davis VantagePro 2 and it was logging these every couple of seconds.

Cheers,

Peter

[mwall]

On Saturday, October 8, 2016 at 7:05:51 AM UTC-4, Peter_F wrote:

Thanks Tom, Matthew and Gary,

I upgraded without any issues.  One thing I noticed though is that the Weather Underground RapidFire rest posts are now logged by default.  I added 'log_success = False' to my weewx.conf to disable it.  I have a Davis VantagePro 2 and it was logging these every couple of seconds.

rapidfire is supposed to default to log_success=false, but there was some bad logic in get_site_dict

fixed at commit c9a9ce6

as you noted, the workaround is:

[StdRESTful]
    [[Wunderground]]
        ...
        log_success = false

m

[mwall]

On Saturday, October 8, 2016 at 1:46:03 AM UTC-4, gjr80 wrote:

 

Oh yes - somewhere in the instructions (or help, not sure which) - the time in the --date command is in brackets.  I presume this is because the time is optional - but why the brackets?  An option in syntax definitions is (I thought) indicated by | whilst the inclusion of brackets implies the time should be enclosed in brackets - which is wrong (I think).

It is the Utilities guide and the --help usage text. The hh:mm in a range is optional.  I will get some input from Tom and Matthew, can you have nested [ ], if not you end up with a very long line.

the current (explicit) form is:

[--date=YYYY/MM/DD|'YYYY/MM/DD hh:mm'|'YYYY/MM/DD (hh:mm)-YYYY/MM/DD (hh:mm)']

 
technically that should be:

[--date=(YYYY/mm/dd|'YYYY/mm/dd HH:MM'|'YYYY/mm|dd (HH:MM)-YYYY/mm/dd (HH:MM)')]

a more compact form would be:

[--date="YYYY/mm/dd[ HH:MM][-YYYY/mm/dd[ HH:MM]]"]

with the caveat that quotes are required only if the date string contains any spaces.

note that month should be mm because minutes is MM

m

[Thomas Keffer]

For separating date and time, one strategy I've seen is to use a "T". Then you don't need the quotes. So, a date range would be:

--date=2016/09/28T03:00-2016/10/02T12:00

But I can see another issue. wunderfixer uses a different format:

--date=YYYY-mm-DD

Personally, I prefer this to YYYY/mm/DD, but using hyphens here means we can't use them for date ranges. 

So, I would suggest that when one wants a date range, you use a different set of options:

--from=YYYY-mm-DDTHH:MM --to=YYYY-mm-DDTHH:MM

An example would look like:

--from=2016-09-28T03:00 --to=2016-10-02T12:00

You use either --date or --from/--to, but not both.

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscribe@googlegroups.com.

[gjr80]

I must confess that initially the use of --date or --to and/or --from and the use of the "T" struck me as complicated and not very intuitive. As far as I am aware wee_import and wunderfixer are the only utilities that use a date or date-time parameter (wee_reports accepts a timestamp), though I note that issue #117 will likely see wee_database need a date (not a date-time) range and one could make a case for wunderfixer to accept a date (again not a date-time) range as well. I guess it would be good to lay down a standard now. That being I am coming aroun dto the --date/--to/--from usage outlined below as it's no more difficult to process and arguably makes input a little simpler for the user.

In terms of wee_import, at the moment the --date option is coded that it will accept a single d/m/y date OR a single d/m/y H:M date-time OR a date range d1/m1/y1-d2/m2/y2 OR a date-time range d1/m1/y1 H1:M1-d2/m2/y2 H2:M2 only - anything else is rejected. The subtle point being that a date - date-time range (ie d1/m1/y1-d2/m2/y2 H2:M2 or d1/m1/y1 H1:M1-d2/m2/y2) is not accepted. That makes the format for the option a little more complex. It is easily fixed though, just need to recode the --date option parsing.

Gary

To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.