Timestamp issue

[rae...@gmail.com]

So after many years of running weewx version 3.8 I decided to spin up another docker container with the latest version and start migrating everything over to it.  I use an MQTT server that my sensors publish their various data to and then read that using the wxMesh driver to populate in the weewx database.  Got that working fine, but I'm baffled by a time stamp issue that I cannot seem to find.  My MQTT server and the new weewx container all have the time zone set to EST and issuing a date command reveals they are in sync and correct.  

I also queried the time in the weewx container with a basic python script to make sure that python is also reading the correct time and it is.  Now the fun part...all my loop data is in UTC not EST but the archive records are correctly in EST which I think is buggering things up and I have no clue where it's getting that time stamp from.  I have my lat long set in weewx.conf so I assume it realizes I'm in the EST time zone.  Here is a snippet of debug log to show you:

Feb 16 18:48:08 f32ab2fbb6f9 user.debug weewxd: wxMesh: mqtt message received weather/windDir
Feb 16 18:48:08 f32ab2fbb6f9 user.debug weewxd: wxMesh: mqtt message payload 315
Feb 16 18:48:10 f32ab2fbb6f9 user.err weewxd: wxMesh: dateTime 1676573290
Feb 16 18:48:10 f32ab2fbb6f9 user.err weewxd: wxMesh: packet content: windSpeed =  6.48
Feb 16 18:48:10 f32ab2fbb6f9 user.err weewxd: wxMesh: packet content: windDir =  315
Feb 16 18:48:10 f32ab2fbb6f9 user.debug weewxd: wxMesh: mqtt message received weather/windSpeed
Feb 16 18:48:10 f32ab2fbb6f9 user.debug weewxd: wxMesh: mqtt message payload 7.14
Feb 16 18:48:10 f32ab2fbb6f9 user.debug weewxd: wxMesh: mqtt message received weather/windDir
Feb 16 18:48:10 f32ab2fbb6f9 user.debug weewxd: wxMesh: mqtt message payload 315
Feb 16 18:48:15 f32ab2fbb6f9 user.err weewxd: wxMesh: dateTime 1676573295
Feb 16 18:48:15 f32ab2fbb6f9 user.err weewxd: wxMesh: packet content: windSpeed =  7.14
Feb 16 18:48:15 f32ab2fbb6f9 user.err weewxd: wxMesh: packet content: windDir =  315
Feb 16 13:48:15 f32ab2fbb6f9 user.info weewx[1954] INFO weewx.manager: Added record 2023-02-16 18:48:00 UTC (1676573280) to database 'weewx.sdb'
Feb 16 13:48:15 f32ab2fbb6f9 user.info weewx[1954] INFO weewx.manager: Added record 2023-02-16 18:48:00 UTC (1676573280) to daily summary in 'weewx.sdb'
Feb 16 13:48:15 f32ab2fbb6f9 user.debug weewx[1954] DEBUG weewx.reportengine: Running reports for latest time in the database.
Feb 16 13:48:15 f32ab2fbb6f9 user.debug weewx[1954] DEBUG weewx.reportengine: Running report 'SeasonsReport'
Feb 16 13:48:15 f32ab2fbb6f9 user.debug weewx[1954] DEBUG weewx.reportengine: Found configuration file /home/weewx/skins/Seasons/skin.conf for report 'SeasonsReport'
Feb 16 13:48:15 f32ab2fbb6f9 user.debug weewx[1954] DEBUG weewx.cheetahgenerator: Using search list ['weewx.cheetahgenerator.Almanac', 'weewx.cheetahgenerator.Current', 'weewx.cheetahgenerator.DisplayOptions', 'weewx.cheetahgenerator.Extras', 'weewx.cheetahgenerator.Gettext', 'weewx.cheetahgenerator.JSONHelpers', 'weewx.cheetahgenerator.PlotInfo', 'weewx.cheetahgenerator.SkinInfo', 'weewx.cheetahgenerator.Station', 'weewx.cheetahgenerator.Stats', 'weewx.cheetahgenerator.UnitInfo']
Feb 16 13:48:15 f32ab2fbb6f9 user.debug weewx[1954] DEBUG weewx.manager: Daily summary version is 4.0
Feb 16 18:48:17 f32ab2fbb6f9 user.debug weewxd: wxMesh: mqtt message received weather/windSpeed
Feb 16 18:48:17 f32ab2fbb6f9 user.debug weewxd: wxMesh: mqtt message payload 5.84
Feb 16 18:48:17 f32ab2fbb6f9 user.debug weewxd: wxMesh: mqtt message received weather/windDir
Feb 16 18:48:17 f32ab2fbb6f9 user.debug weewxd: wxMesh: mqtt message payload 315
Feb 16 13:48:18 f32ab2fbb6f9 user.info weewx[1954] INFO weewx.cheetahgenerator: Generated 8 files for report SeasonsReport in 2.42 seconds
Feb 16 13:48:18 f32ab2fbb6f9 user.debug weewx[1954] DEBUG weewx.manager: Daily summary version is 4.0

Anyone have any clues as to where this is happening?

[michael.k...@gmx.at]

Strange log. Are there two deamons running? One (weewxd[1954]) is logging EST, the other one (weewxd, without pid?) is logging UTC? The archive records are not in EST, but in UTC (seconds since 01/01/1970 00:00 UTC). 

Setting lat/lon won't affect anything in this context.

What is the exact problem? Wrong data collection, wrong representation in the HTML or just the strange log?

[rae...@gmail.com]

This is in a container in my docker swarm so it's not 2 daemons running.  Really the only annoyance is the time in the html at the top left corner of the seasons skin is displaying UTC and I cannot find where to change that.

[rae...@gmail.com]

Also the logs on my old weewx container are consistent, loop data and archive data has the same timezone, and the new weewx container is subscribed to the same mqtt data.  So it's definitely something wrong with the new weewx config, but I cannot find it.  I have set the TZ environment variable for the container so it should have the right time zone as well.

[vince]

Just for grins, log into the container and see what it thinks the TZ is....

[gjr80]

A few observations:

• WeeWX obtains timezone info from the underlying operating system, it has nothing to do with WeeWX lat/long. I could setup my WeeWX system in Paris and if my OS is set to AEST WeeWX will display timestamps in Australian Eastern Standard Time.
• WeeWX records/uses unix epoch timestamps for loop packet/archive record timestamps so loop packets/archive records are not 'in any timezone' as such
• WeeWX logging changed when v4 was released. Previously the python syslog library was used, but from v4 onward the python logging library is used.
• Assuming the 'wxMesh driver' referred to is https://github.com/morrowwm/weewxMQTT. Looking at the wxMesh driver code, whilst it claims python3 and v4.x (or v4.1.x) compatibility, the wxMesh driver still uses the python syslog library for logging rather than the python logging library. This likely explains the different log formats in the OP. It does not (directly) explain the different timezones in the log entry timestamps.
• The WeeWX (not wxMesh) log entires where records are saved to archived uses UTC, implying WeeWX (and python) is obtaining UTC timezone information from the container or host OS.

Agree with Vince, what does the container say, would be good to see the command issued and the response received rather than a description. Would also be good to see a WeeWX log extract with debug =1 and showing a full WeeWX startup plus a couple of archive periods. You mention 'all my loop data is in UTC not EST but the archive records are correctly in EST' but I don't see any loop data displayed in UTC in your post and the only archive records I see in your post (the log entry saving an archive record) displays with EST. What happens when you run WeeWX directly, post a couple of archive periods of WeeWX output when running WeeWX directly.

Gary

[vince]

You need to configure your container instance's timezone to be what you want.

Docker host:

    vince@nuc2:~$ date
    Sat Feb 18 11:34:30 PST 2023

Debian10 image on that host:

    vince@nuc2:~$ docker run -it --rm debian:10 date
    Sat Feb 18 19:34:28 UTC 2023

Conclusion - the default debian:10 image uses UTC, just as one example.

[rae...@gmail.com]

Thanks guys, I already checked this, also already set the TZ env variable to EST for all my containers and issuing a date command yields all the correct timestamps for my time zone so I have no idea where the cheetah script that generates the images is getting that I'm in UTC.

[rae...@gmail.com]

Another oddity I just noticed in the Seasons skin it understands that I'm in the EST time zone but is reporting the sunset and sunrise (and moon) in UTC:

[rae...@gmail.com]

No ideas?  Anyone else run into this issue?

[vince]

I do not have this problem with Seasons in 4.9.1 using the Simulator driver under an Alpine base os.

docker-compose logs:

    weealpine | weewx[1] INFO weewx.manager: Added record 2023-02-27 14:05:00 PST (1677535500) to database 'weewx.sdb'
    weealpine | weewx[1] INFO weewx.manager: Added record 2023-02-27 14:05:00 PST (1677535500) to daily summary in 'weewx.sdb'

And the resulting Seasons skin index.html also shows a local timezone date as expected.

[vince]

I think I figured it out.

Make sure you are setting your lat/lon in weewx.conf

If you leave it 0.00 for both you'll get UTC sunrise/sunset on the equator.

[rae...@gmail.com]

I have my lat long set up in the weewx.conf.  It's very strange.  I don't really know where it's getting the UTC from or should say not picking up the TZ...all the involved containers have the correct TZ set up.  I'm using the SQL database that's in the same container as Weewx and obviously I want to get this all sorted out before I let it loose on my existing MariaDB with all my historical data.  I'm really stumped.