problems with wee_import from csv
76 views
Skip to first unread message
Astrid
Apr 3, 2020, 8:01:00 AM4/3/20
to weewx-user
Hello!
I am using weewx 3.9.2 and set
it up a few weeks ago.
Now all is working very well, but some
strugling in between.
So I now want to fill some missing values
in the weewx-db with getting it from a csv-file.
This are the first lines of /var/tmp/data.csv
mydateTime, myoutTemp, myoutHumidity, mydewpoint, mybarometer, mywindSpeed, mywindGust, mywindDir, myheatindex
1585692060, 33.08, 65, 22.64, 30.330773774365, 4.60312, 4.60312, 315, 33.08
1585692120, 33.08, 65, 22.64, 30.330773774365, 4.60312, 4.60312, 315, 33.08
1585692180, 33.08, 65, 22.64, 30.330773774365, 4.60312, 4.60312, 315, 33.08
1585692240, 33.08, 65, 22.64, 30.330773774365, 4.60312, 4.60312, 315, 33.08
I stop weewx-daemon and then:
wee_import --import-config=mycsv.conf --dry-run --log=-
But it does not work and I do not know, why? :(
This appears for all fields:
wee_import --import-config=mycsv.conf --dry-run --log=-
Starting wee_import...
A CSV import from source file '/var/tmp/data.csv' has been requested.
Using database binding 'wx_binding', which is bound to database 'weewx.sdb'
Destination table 'archive' unit system is '0x01' (US).
Missing derived observations will be calculated.
All WeeWX UV fields will be set to None.
All WeeWX radiation fields will be set to None.
This is a dry run, imported data will not be saved to archive.
Warning: Import field 'myheatindex' is mapped to WeeWX field 'heatindex'
but the import field could not be found.
WeeWX field 'heatindex' will be set to 'None'.
Warning: Import field 'mybarometer' is mapped to WeeWX field 'barometer'
but the import field could not be found.
WeeWX field 'barometer' will be set to 'None'.
Warning: Import field 'mydewPoint' is mapped to WeeWX field 'dewpoint'
but the import field could not be found.
In my conf I give such line as:
[[FieldMap]]
dateTime = mydateTime, unix_epoch
usUnits =
interval =
barometer = mybarometer, inHg
pressure =
altimeter =
inTemp =
outTemp = myoutTemp, degree_F
inHumidity =
outHumidity = myoutHumidity, percent
windSpeed = mywindSpeed, mile_per_hour
windDir = mywindDir, degree_compass
windGust = mywindGust, mile_per_hour
windGustDir =
rainRate =
rain =
dewpoint = mydewPoint, degree_F
windchill =
heatindex = myheatindex, degree_F
ET =
radiation =
UV =
How I can get it work?
Regards, Astrid
I am using weewx 3.9.2 and set
it up a few weeks ago.
Now all is working very well, but some
strugling in between.
So I now want to fill some missing values
in the weewx-db with getting it from a csv-file.
This are the first lines of /var/tmp/data.csv
mydateTime, myoutTemp, myoutHumidity, mydewpoint, mybarometer, mywindSpeed, mywindGust, mywindDir, myheatindex
1585692060, 33.08, 65, 22.64, 30.330773774365, 4.60312, 4.60312, 315, 33.08
1585692120, 33.08, 65, 22.64, 30.330773774365, 4.60312, 4.60312, 315, 33.08
1585692180, 33.08, 65, 22.64, 30.330773774365, 4.60312, 4.60312, 315, 33.08
1585692240, 33.08, 65, 22.64, 30.330773774365, 4.60312, 4.60312, 315, 33.08
I stop weewx-daemon and then:
wee_import --import-config=mycsv.conf --dry-run --log=-
But it does not work and I do not know, why? :(
This appears for all fields:
wee_import --import-config=mycsv.conf --dry-run --log=-
Starting wee_import...
A CSV import from source file '/var/tmp/data.csv' has been requested.
Using database binding 'wx_binding', which is bound to database 'weewx.sdb'
Destination table 'archive' unit system is '0x01' (US).
Missing derived observations will be calculated.
All WeeWX UV fields will be set to None.
All WeeWX radiation fields will be set to None.
This is a dry run, imported data will not be saved to archive.
Warning: Import field 'myheatindex' is mapped to WeeWX field 'heatindex'
but the import field could not be found.
WeeWX field 'heatindex' will be set to 'None'.
Warning: Import field 'mybarometer' is mapped to WeeWX field 'barometer'
but the import field could not be found.
WeeWX field 'barometer' will be set to 'None'.
Warning: Import field 'mydewPoint' is mapped to WeeWX field 'dewpoint'
but the import field could not be found.
In my conf I give such line as:
[[FieldMap]]
dateTime = mydateTime, unix_epoch
usUnits =
interval =
barometer = mybarometer, inHg
pressure =
altimeter =
inTemp =
outTemp = myoutTemp, degree_F
inHumidity =
outHumidity = myoutHumidity, percent
windSpeed = mywindSpeed, mile_per_hour
windDir = mywindDir, degree_compass
windGust = mywindGust, mile_per_hour
windGustDir =
rainRate =
rain =
dewpoint = mydewPoint, degree_F
windchill =
heatindex = myheatindex, degree_F
ET =
radiation =
UV =
How I can get it work?
Regards, Astrid
gjr80
Apr 3, 2020, 8:24:54 AM4/3/20
to weewx-user
Hi,
You need to double check the field names in the first line of your data file against the import field names in the field map in your import config file. They must match exactly. In your data file you have ‘mydewpoint’ but in your field map you have ‘mydewPoint’. Make sure you check the other two problem fields. If you cannot find the problem please post a copy of your data file and import config file.
Gary
You need to double check the field names in the first line of your data file against the import field names in the field map in your import config file. They must match exactly. In your data file you have ‘mydewpoint’ but in your field map you have ‘mydewPoint’. Make sure you check the other two problem fields. If you cannot find the problem please post a copy of your data file and import config file.
Gary
Astrid
Apr 3, 2020, 10:24:00 AM4/3/20
to weewx-user
Hello!
Thanx. Yes at the dewpoint was a typo, arrrrgh to me...
This is my (shorten, but same errormessages for _all_ fields... :( )
mydatetime, myouttemp, myouthumidity, mydewpoint, mybarometer, mywindspeed, mywindgust, mywinddir, myheatindex
Thanx. Yes at the dewpoint was a typo, arrrrgh to me...
This is my (shorten, but same errormessages for _all_ fields... :( )
mydatetime, myouttemp, myouthumidity, mydewpoint, mybarometer, mywindspeed, mywindgust, mywinddir, myheatindex
1585692060, 33.08, 65, 22.64, 30.330773774365, 4.60312, 4.60312, 315, 33.08
1585692120, 33.08, 65, 22.64, 30.330773774365, 4.60312, 4.60312, 315, 33.08
1585692180, 33.08, 65, 22.64, 30.330773774365, 4.60312, 4.60312, 315, 33.08
1585692240, 33.08, 65, 22.64, 30.330773774365, 4.60312, 4.60312, 315, 33.08
1585692300, 32.9, 65, 22.46, 30.3337271116361, 4.60312, 4.60312, 315, 32.9
1585692360, 32.9, 65, 22.46, 30.3337271116361, 4.60312, 4.60312, 315, 32.9
1585692420, 32.9, 65, 22.46, 30.330773774365, 4.60312, 4.60312, 315, 32.9
1585692480, 32.9, 65, 22.46, 30.330773774365, 4.60312, 4.60312, 315, 32.9
and this my conf:
# EXAMPLE CONFIGURATION FILE FOR IMPORTING FROM CSV FILES
#
# Copyright (c) 2009-2016 Tom Keffer <tkeffer gmail.com> and Gary Roderick.
# See the file LICENSE.txt for your rights.
##############################################################################
# Specify the source. Available options are:
# CSV - import obs from a single CSV format file
# WU - import obs from a Weather Underground PWS history
# Cumulus - import obs from a one or more Cumulus monthly log files
# Format is:
# source = (CSV | WU | Cumulus)
source = CSV
##############################################################################
[CSV]
# Parameters used when importing from a CSV file
# Path and name of our CSV source file. Format is:
# file = full path and filename
file = /var/tmp/data.csv
# If there is no mapped interval field how will the interval field be
# determined for the imported records. Available options are:
# derive - Derive the interval field from the timestamp of successive
# records. This setting is best used when the imported records
# are equally spaced in time and there are no missing records.
# conf - Use the interval setting from weewx.conf. This setting is
# best used if the records to be imported have been produced by
# WeeWX using the same archive interval as set in weewx.conf on
# this machine.
# x - Use a fixed interval of x minutes for every record. This
# setting is best used if the records to be imported are
# equally based in time but there are some missing records.
#
# Note: If there is a mapped interval field then this setting will be
# ignored.
# Format is:
# interval = (derive | conf | x)
interval = derive
# Should the [StdQC] max/min limits in weewx.conf be applied to the
# imported data. This may be useful if the source has extreme values that
# are clearly incorrect for some observations. Available options are:
# True - weewx.conf [StdQC] max/min limits are applied.
# False - weewx.conf [StdQC] max/min limits are not applied.
# Format is:
# qc = (True | False)
qc = True
# Should any missing derived observations be calculated from the imported
# data if possible. Available options are:
# True - Any missing derived observations are calculated.
# False - Any missing derived observations are not calculated.
# Format is:
# calc_missing = (True | False)
calc_missing = True
# Specify how imported data fields that contain invalid data (eg a numeric
# field containing non-numeric data) are handled. Available options are:
# True - The invalid data is ignored, the WeeWX target field is set to
# None and the import continues.
# False - The import is halted.
# Format is:
# ignore_invalid_data = (True | False)
# Default is True.
ignore_invalid_data = True
# Imported records are written to archive in transactions of tranche
# records at a time. Increase for faster throughput, decrease to reduce
# memory requirements. Format is:
# tranche = x
# where x is an integer
tranche = 250
# Specify whether a UV sensor was used to produce any UV observations.
# Available options are:
# True - UV sensor was used and UV data will be imported.
# False - UV sensor was not used and any UV data will not be imported.
# UV fields will be set to None/NULL.
# For a CSV import UV_sensor should be set to False if a UV sensor was
# NOT present when the import data was created. Otherwise it may be set to
# True or omitted. Format is:
# UV_sensor = (True | False)
UV_sensor = False
# Specify whether a solar radiation sensor was used to produce any solar
# radiation observations. Available options are:
# True - Solar radiation sensor was used and solar radiation data will
# be imported.
# False - Solar radiation sensor was not used and any solar radiation
# data will not be imported. radiation fields will be set to
# None/NULL.
# For a CSV import solar_sensor should be set to False if a solar radiation
# sensor was NOT present when the import data was created. Otherwise it may
# be set to True or omitted. Format is:
# solar_sensor = (True | False)
solar_sensor = False
# Date-time format of CSV field from which the WeeWX archive record
# dateTime field is to be extracted. wee_import first attempts to interpret
# date/time info in this format, if this fails it then attempts to
# interpret it as a timestamp and if this fails it then raises an error.
# Uses Python strptime() format codes.
# raw_datetime_format = Python strptime() format string
raw_datetime_format = %Y-%m-%d %H:%M:%S
# Does the imported rain field represent the total rainfall since the last
# record or a cumulative value. Available options are:
# discrete - rain field represents total rainfall since last record
# cumulative - rain field represents a cumulative rainfall reset at
# midnight
# rain = (discrete | cumulative)
rain = cumulative
# Lower and upper bounds for imported wind direction. It is possible,
# particularly for a calculated direction, to have a value (eg -45) outside
# of the WeeWX limits (0 to 360 inclusive). Format is:
#
# wind_direction = lower,upper
#
# where :
# lower is the lower limit of acceptable wind direction in degrees
# (may be negative)
# upper is the upper limit of acceptable wind direction in degrees
#
# Imported values from lower to upper will be normalised to the range 0 to
# 360. Values outside of the parameter range will be stored as None.
# Default is -360,360.
wind_direction = 0,360
# Map CSV record fields to WeeWX archive fields. Format is:
#
# weewx_archive_field_name = csv_field_name, weewx_unit_name
#
# where:
# weewx_archive_field_name - An observation name in the WeeWX database
# schema.
# csv_field_name - The name of a field from the CSV file.
# weewx_unit_name - The name of the units, as defined in WeeWX,
# used by csv_field_name. wee_import will do
# the necessary conversions to the unit system
# used by the WeeWX archive.
# For example,
# outTemp = Temp, degree_C
# would map the CSV field Temp, in degrees C, to the archive field outTemp.
#
# If a field mapping exists for the WeeWX usUnits archive field then the
# units option may be omitted for each mapped field.
#
# WeeWX archive fields that do not exist in the CSV data may be omitted.
# Any omitted fields that are derived (eg dewpoint) may be calculated
# during import using the equivalent of the WeeWX StdWXCalculate service
# through setting the calc-missing parameter above.
[[FieldMap]]
dateTime = mydatetime, unix_epoch
1585692360, 32.9, 65, 22.46, 30.3337271116361, 4.60312, 4.60312, 315, 32.9
1585692420, 32.9, 65, 22.46, 30.330773774365, 4.60312, 4.60312, 315, 32.9
1585692480, 32.9, 65, 22.46, 30.330773774365, 4.60312, 4.60312, 315, 32.9
and this my conf:
# EXAMPLE CONFIGURATION FILE FOR IMPORTING FROM CSV FILES
#
# Copyright (c) 2009-2016 Tom Keffer <tkeffer gmail.com> and Gary Roderick.
# See the file LICENSE.txt for your rights.
##############################################################################
# Specify the source. Available options are:
# CSV - import obs from a single CSV format file
# WU - import obs from a Weather Underground PWS history
# Cumulus - import obs from a one or more Cumulus monthly log files
# Format is:
# source = (CSV | WU | Cumulus)
source = CSV
##############################################################################
[CSV]
# Parameters used when importing from a CSV file
# Path and name of our CSV source file. Format is:
# file = full path and filename
file = /var/tmp/data.csv
# If there is no mapped interval field how will the interval field be
# determined for the imported records. Available options are:
# derive - Derive the interval field from the timestamp of successive
# records. This setting is best used when the imported records
# are equally spaced in time and there are no missing records.
# conf - Use the interval setting from weewx.conf. This setting is
# best used if the records to be imported have been produced by
# WeeWX using the same archive interval as set in weewx.conf on
# this machine.
# x - Use a fixed interval of x minutes for every record. This
# setting is best used if the records to be imported are
# equally based in time but there are some missing records.
#
# Note: If there is a mapped interval field then this setting will be
# ignored.
# Format is:
# interval = (derive | conf | x)
interval = derive
# Should the [StdQC] max/min limits in weewx.conf be applied to the
# imported data. This may be useful if the source has extreme values that
# are clearly incorrect for some observations. Available options are:
# True - weewx.conf [StdQC] max/min limits are applied.
# False - weewx.conf [StdQC] max/min limits are not applied.
# Format is:
# qc = (True | False)
qc = True
# Should any missing derived observations be calculated from the imported
# data if possible. Available options are:
# True - Any missing derived observations are calculated.
# False - Any missing derived observations are not calculated.
# Format is:
# calc_missing = (True | False)
calc_missing = True
# Specify how imported data fields that contain invalid data (eg a numeric
# field containing non-numeric data) are handled. Available options are:
# True - The invalid data is ignored, the WeeWX target field is set to
# None and the import continues.
# False - The import is halted.
# Format is:
# ignore_invalid_data = (True | False)
# Default is True.
ignore_invalid_data = True
# Imported records are written to archive in transactions of tranche
# records at a time. Increase for faster throughput, decrease to reduce
# memory requirements. Format is:
# tranche = x
# where x is an integer
tranche = 250
# Specify whether a UV sensor was used to produce any UV observations.
# Available options are:
# True - UV sensor was used and UV data will be imported.
# False - UV sensor was not used and any UV data will not be imported.
# UV fields will be set to None/NULL.
# For a CSV import UV_sensor should be set to False if a UV sensor was
# NOT present when the import data was created. Otherwise it may be set to
# True or omitted. Format is:
# UV_sensor = (True | False)
UV_sensor = False
# Specify whether a solar radiation sensor was used to produce any solar
# radiation observations. Available options are:
# True - Solar radiation sensor was used and solar radiation data will
# be imported.
# False - Solar radiation sensor was not used and any solar radiation
# data will not be imported. radiation fields will be set to
# None/NULL.
# For a CSV import solar_sensor should be set to False if a solar radiation
# sensor was NOT present when the import data was created. Otherwise it may
# be set to True or omitted. Format is:
# solar_sensor = (True | False)
solar_sensor = False
# Date-time format of CSV field from which the WeeWX archive record
# dateTime field is to be extracted. wee_import first attempts to interpret
# date/time info in this format, if this fails it then attempts to
# interpret it as a timestamp and if this fails it then raises an error.
# Uses Python strptime() format codes.
# raw_datetime_format = Python strptime() format string
raw_datetime_format = %Y-%m-%d %H:%M:%S
# Does the imported rain field represent the total rainfall since the last
# record or a cumulative value. Available options are:
# discrete - rain field represents total rainfall since last record
# cumulative - rain field represents a cumulative rainfall reset at
# midnight
# rain = (discrete | cumulative)
rain = cumulative
# Lower and upper bounds for imported wind direction. It is possible,
# particularly for a calculated direction, to have a value (eg -45) outside
# of the WeeWX limits (0 to 360 inclusive). Format is:
#
# wind_direction = lower,upper
#
# where :
# lower is the lower limit of acceptable wind direction in degrees
# (may be negative)
# upper is the upper limit of acceptable wind direction in degrees
#
# Imported values from lower to upper will be normalised to the range 0 to
# 360. Values outside of the parameter range will be stored as None.
# Default is -360,360.
wind_direction = 0,360
# Map CSV record fields to WeeWX archive fields. Format is:
#
# weewx_archive_field_name = csv_field_name, weewx_unit_name
#
# where:
# weewx_archive_field_name - An observation name in the WeeWX database
# schema.
# csv_field_name - The name of a field from the CSV file.
# weewx_unit_name - The name of the units, as defined in WeeWX,
# used by csv_field_name. wee_import will do
# the necessary conversions to the unit system
# used by the WeeWX archive.
# For example,
# outTemp = Temp, degree_C
# would map the CSV field Temp, in degrees C, to the archive field outTemp.
#
# If a field mapping exists for the WeeWX usUnits archive field then the
# units option may be omitted for each mapped field.
#
# WeeWX archive fields that do not exist in the CSV data may be omitted.
# Any omitted fields that are derived (eg dewpoint) may be calculated
# during import using the equivalent of the WeeWX StdWXCalculate service
# through setting the calc-missing parameter above.
[[FieldMap]]
dateTime = mydatetime, unix_epoch
usUnits =
interval =
barometer = mybarometer, inHg
pressure =
altimeter =
inTemp =
outTemp = myouttemp, degree_F
inHumidity =
outHumidity = myouthumidity, percent
windSpeed = mywindspeed, mile_per_hour
windDir = mywinddir, degree_compass
windGust = mywindgust, mile_per_hour
inHumidity =
outHumidity = myouthumidity, percent
windSpeed = mywindspeed, mile_per_hour
windDir = mywinddir, degree_compass
windGust = mywindgust, mile_per_hour
windGustDir =
rainRate =
rain =
dewpoint = mydewpoint, degree_F
windchill =
heatindex = myheatindex, degree_F
ET =
radiation =
UV =
Regards, Astrid
Astrid
Apr 3, 2020, 1:33:25 PM4/3/20
to weewx-user
Hello!
I now can find it out now...
Blanks seems to be forbidden in csv...
With this, it works.
mydatetime,myouttemp,myouthumidity,mydewpoint,mybarometer,mywindspeed,mywindgust,mywinddir,myheatindex
1585692060,33.08,65,22.64,30.330773774365,4.60312,4.60312,315,33.08
1585692120,33.08,65,22.64,30.330773774365,4.60312,4.60312,315,33.08
1585692180,33.08,65,22.64,30.330773774365,4.60312,4.60312,315,33.08
1585692240,33.08,65,22.64,30.330773774365,4.60312,4.60312,315,33.08
1585692300,32.9,65,22.46,30.3337271116361,4.60312,4.60312,315,32.9
1585692360,32.9,65,22.46,30.3337271116361,4.60312,4.60312,315,32.9
1585692420,32.9,65,22.46,30.330773774365,4.60312,4.60312,315,32.9
1585692480,32.9,65,22.46,30.330773774365,4.60312,4.60312,315,32.9
Perhaps you can give a special hint in documentation of wee_import,
that blanks are forbidden?
Regards, Astrid
I now can find it out now...
Blanks seems to be forbidden in csv...
With this, it works.
mydatetime,myouttemp,myouthumidity,mydewpoint,mybarometer,mywindspeed,mywindgust,mywinddir,myheatindex
1585692060,33.08,65,22.64,30.330773774365,4.60312,4.60312,315,33.08
1585692120,33.08,65,22.64,30.330773774365,4.60312,4.60312,315,33.08
1585692180,33.08,65,22.64,30.330773774365,4.60312,4.60312,315,33.08
1585692240,33.08,65,22.64,30.330773774365,4.60312,4.60312,315,33.08
1585692300,32.9,65,22.46,30.3337271116361,4.60312,4.60312,315,32.9
1585692360,32.9,65,22.46,30.3337271116361,4.60312,4.60312,315,32.9
1585692420,32.9,65,22.46,30.330773774365,4.60312,4.60312,315,32.9
1585692480,32.9,65,22.46,30.330773774365,4.60312,4.60312,315,32.9
Perhaps you can give a special hint in documentation of wee_import,
that blanks are forbidden?
Regards, Astrid
Andrew Milner
Apr 3, 2020, 1:33:49 PM4/3/20
to weewx-user
does your csv file have a blank line after the header line which should be deleted?
Astrid
Apr 3, 2020, 1:46:34 PM4/3/20
to weewx-user
Thanx, but no blank line after header has been there.
Regards, Astrid
Regards, Astrid
Astrid
Apr 3, 2020, 1:56:23 PM4/3/20
to weewx-user
Hello!
I'm afraid, that my happyness was to early...
Now I have the csv without blanks.
I want now to get back some missing data of a special date
to my weewx-db.
I stopped weewxd, make a copy of sdb, let run:
wee_import --import-config=mycsv.conf --log=log1.log
I'm afraid, that my happyness was to early...
Now I have the csv without blanks.
I want now to get back some missing data of a special date
to my weewx-db.
I stopped weewxd, make a copy of sdb, let run:
wee_import --import-config=mycsv.conf --log=log1.log
Starting wee_import...
A CSV import from source file '/var/tmp/data.csv' has been requested.
Using database binding 'wx_binding', which is bound to database 'weewx.sdb'
Destination table 'archive' unit system is '0x01' (US).
Missing derived observations will be calculated.
All WeeWX UV fields will be set to None.
All WeeWX radiation fields will be set to None.
Starting import ...
1439 records identified for import.
Proceeding will save all imported records in the WeeWX archive.
Are you sure you want to proceed (y/n)? y
Records processed: 1439; Unique records: 1439; Last timestamp: 2020-04-01 23:59:00 CEST (1585778340)
Finished import. 1439 raw records resulted in 1439 unique records being processed in 30.68 seconds.
Those records with a timestamp already in the archive will not have been
imported. Confirm successful import in the WeeWX log file.
But if I rerun the reports, the data is still missing in my plots. :(
And where can I find the logging-information?
Regards, Astrid
1439 records identified for import.
Proceeding will save all imported records in the WeeWX archive.
Are you sure you want to proceed (y/n)? y
Records processed: 1439; Unique records: 1439; Last timestamp: 2020-04-01 23:59:00 CEST (1585778340)
Finished import. 1439 raw records resulted in 1439 unique records being processed in 30.68 seconds.
Those records with a timestamp already in the archive will not have been
imported. Confirm successful import in the WeeWX log file.
But if I rerun the reports, the data is still missing in my plots. :(
And where can I find the logging-information?
Regards, Astrid
gjr80
Apr 3, 2020, 4:18:14 PM4/3/20
to weewx-user
Since you used the —log option you will find the log entries in the file log1.log. Since you did not provide an explicit path the file will likely be in the directory you were in when you ran the wee_import command. If you omit the —log option then wee_import log entries are placed in the same log file as used by WeeWX.
Gary
Gary
Astrid
Apr 4, 2020, 5:28:23 AM4/4/20
to weewx-user
Hello!
Thanx. But still not working. :(
wee_import --import-config=mycsv.conf --log=log.log
Thanx. But still not working. :(
wee_import --import-config=mycsv.conf --log=log.log
Starting wee_import...
A CSV import from source file '/var/tmp/data.csv' has been requested.
Using database binding 'wx_binding', which is bound to database 'weewx.sdb'
Destination table 'archive' unit system is '0x01' (US).
Missing derived observations will be calculated.
All WeeWX UV fields will be set to None.
All WeeWX radiation fields will be set to None.
Starting import ...
287 records identified for import.
287 records identified for import.
Proceeding will save all imported records in the WeeWX archive.
Are you sure you want to proceed (y/n)? y
Records processed: 287; Unique records: 287; Last timestamp: 2020-04-01 23:55:00 CEST (1585778100)
Finished import. 287 raw records resulted in 287 unique records being processed in 3.59 seconds.
Finished import. 287 raw records resulted in 287 unique records being processed in 3.59 seconds.
Those records with a timestamp already in the archive will not have been
imported. Confirm successful import in the WeeWX log file.
There is no file log.log on the whole harddisk.
And there is no additional data comming into my weewx database.
If I redo the plots after import, there is still the whole of
the missing values from 01.04.2020 inside. :(
How I can get the data inside?
Regards, Astrid
And there is no additional data comming into my weewx database.
If I redo the plots after import, there is still the whole of
the missing values from 01.04.2020 inside. :(
How I can get the data inside?
Regards, Astrid
gjr80
Apr 4, 2020, 5:38:07 AM4/4/20
to weewx-user
Just omit the —log option and the logging will be in whatever file WeeWX normally logs to.
Gary
Gary
gjr80
Apr 4, 2020, 5:39:23 AM4/4/20
to weewx-user
Or include a full path with the —log option, eg:
—log=/var/tmp/import.log
—log=/var/tmp/import.log
gjr80
Apr 4, 2020, 5:48:58 AM4/4/20
to weewx-user
Actually, disregard my last post. —log cannot be used to redirect logging to a file, it can only be used to disable wee_import logging by using —log=-. Running wee_import with —help or referring to the wee_import section of the Utilities Guide (http://weewx.com/docs/utilities.htm#Option_--log) makes this clear.
You will need to run wee_import without the —log option and find the we_import log entries in whatever file WeeWX logs to.
Gary
You will need to run wee_import without the —log option and find the we_import log entries in whatever file WeeWX logs to.
Gary
Astrid
Apr 4, 2020, 5:53:54 AM4/4/20
to weewx-user
Hello!
With sqlitebrowser I looked now into the database:
The datetime field is filled ok with the imported values from datetime.
But all other columns is standing NULL.
But I do not understand, why??
This is the beginning of my data.csv:
mydatetime,myouttemp,myouthumidity,mydewpoint,mybarometer,mywindspeed,mywindgust,mywinddir,myheatindex
1585692300,32.9,65,22.46,30.3337271116361,4.60312,4.60312,315,32.9
1585692600,32.9,65,22.46,30.3337271116361,4.60312,4.60312,315,32.9
1585692900,32.9,66,22.82,30.3337271116361,4.60312,5.7539,315,32.9
The 1585692300 is there, but all value columns NULL inside the database.
Regards, Astrid
With sqlitebrowser I looked now into the database:
The datetime field is filled ok with the imported values from datetime.
But all other columns is standing NULL.
But I do not understand, why??
This is the beginning of my data.csv:
mydatetime,myouttemp,myouthumidity,mydewpoint,mybarometer,mywindspeed,mywindgust,mywinddir,myheatindex
1585692300,32.9,65,22.46,30.3337271116361,4.60312,4.60312,315,32.9
1585692600,32.9,65,22.46,30.3337271116361,4.60312,4.60312,315,32.9
1585692900,32.9,66,22.82,30.3337271116361,4.60312,5.7539,315,32.9
The 1585692300 is there, but all value columns NULL inside the database.
Regards, Astrid
gjr80
Apr 4, 2020, 5:54:58 AM4/4/20
to weewx-user
Need the log please.
Gary
Astrid
Apr 4, 2020, 6:42:25 AM4/4/20
to weewx-user
Hello!
I removed the lines with NULL in datafield-values
and rerun the import for 01.04.2020 again and now
it adds the recored into database. :)
Apr 04 12:36:51 linux.lsg wee_import[3234]: manager: Added record 2020-04-01 00:05:00 CEST (1585692300) to database 'weewx.sdb'
Apr 04 12:36:51 linux.lsg wee_import[3234]: manager: Added record 2020-04-01 00:05:00 CEST (1585692300) to daily summary in 'weewx.sdb'
Apr 04 12:36:51 linux.lsg wee_import[3234]: manager: Added record 2020-04-01 00:10:00 CEST (1585692600) to database 'weewx.sdb'
Apr 04 12:36:51 linux.lsg wee_import[3234]: manager: Added record 2020-04-01 00:10:00 CEST (1585692600) to daily summary in 'weewx.sdb'
Apr 04 12:36:51 linux.lsg wee_import[3234]: manager: Added record 2020-04-01 00:15:00 CEST (1585692900) to database 'weewx.sdb'
Apr 04 12:36:51 linux.lsg wee_import[3234]: manager: Added record 2020-04-01 00:15:00 CEST (1585692900) to daily summary in 'weewx.sdb'
Thanx. :)
Regards, Astrid
I removed the lines with NULL in datafield-values
and rerun the import for 01.04.2020 again and now
it adds the recored into database. :)
Apr 04 12:36:51 linux.lsg wee_import[3234]: manager: Added record 2020-04-01 00:05:00 CEST (1585692300) to database 'weewx.sdb'
Apr 04 12:36:51 linux.lsg wee_import[3234]: manager: Added record 2020-04-01 00:05:00 CEST (1585692300) to daily summary in 'weewx.sdb'
Apr 04 12:36:51 linux.lsg wee_import[3234]: manager: Added record 2020-04-01 00:10:00 CEST (1585692600) to database 'weewx.sdb'
Apr 04 12:36:51 linux.lsg wee_import[3234]: manager: Added record 2020-04-01 00:10:00 CEST (1585692600) to daily summary in 'weewx.sdb'
Apr 04 12:36:51 linux.lsg wee_import[3234]: manager: Added record 2020-04-01 00:15:00 CEST (1585692900) to database 'weewx.sdb'
Apr 04 12:36:51 linux.lsg wee_import[3234]: manager: Added record 2020-04-01 00:15:00 CEST (1585692900) to daily summary in 'weewx.sdb'
Thanx. :)
Regards, Astrid
Reply all
Reply to author
Forward
0 new messages