High CPU Usage by WeeWX Process Impacting Web Server Performance

[Marco Citossi]

I have observed that the WeeWX process has been consuming significant CPU resources for the past two weeks. This issue occurs approximately every minute and persists for some time (1/2 minutes). This excessive CPU usage is concerning as it adversely affects our web server's performance.

What could be the potential causes of this issue?

Thanks.

[Graham Eddy]

almost no info provided, but maybe you have archive interval at 1 min (generates all reports) and you have lots of reports (take ½ min to complete). archive interval of 5 mins is more usual, and maybe you don’t need all those reports

⊣GE⊢

On 29 Dec 2024, at 9:20 pm, Marco Citossi <mcit...@gmail.com> wrote:

[Marco Citossi]

Archive_interval is 300

[StdArchive]

    # If the station hardware supports data logging then the archive interval
    # will be downloaded from the station. Otherwise, specify it (in seconds).
    archive_interval = 300

    # If possible, new archive records are downloaded from the station
    # hardware. If the hardware does not support this, then new archive
    # records will be generated in software.
    # Set the following to "software" to force software record generation.
    record_generation = hardware

    # Whether to include LOOP data in hi/low statistics.
    loop_hilo = True

    # Uncomment and change to override logging for archive operations.
    # log_success = True
    # log_failure = True

    # The data binding used to save archive records.
    data_binding = wx_binding

Other information:

baudrate = 19200

    # TCP port (when using the WeatherLinkIP)
    tcp_port = 22222

    # TCP send delay (when using the WeatherLinkIP):
    tcp_send_delay = 0.5

    # The type of LOOP packet to request: 1 = LOOP1; 2 = LOOP2; 3 = both
    loop_request = 1

    # The id of your ISS station (usually 1). If you use a wind meter connected
    # to a anemometer transmitter kit, use its id
    iss_id = 1

    # How long to wait for a response from the station before giving up (in
    # seconds; must be greater than 2)
    timeout = 4

    # How long to wait before trying again (in seconds)
    wait_before_retry = 1.2

    # How many times to try before giving up:
    max_tries = 4

[vince]

You have provided no information we can use to try to help.  What kind of station ? How is the station connected to the host computer ? What version of weewx ? What kind of installation ? What skins and extensions have you added ?What hardware are you running on ?  What os and version ? Did this issue recently start ? Did you change anything recently ?

And as always we need to see debug=1 logs from weewx startup through two archive periods.

Lastly, please do not attach screenshots. They are almost impossible to read.

[Marco Citossi]

Sorry, here is some other information:

Station Vantage Pro2
WeeWX version: 5.1.0b6
Belchertown Skin Version: 1.3.1
(https://github.com/poblabs/weewx-belchertown)

   

OS: Oracle Linux Server 8.10

    type = ethernet
    port = /dev/ttyUSB0

    baudrate = 19200
    tcp_send_delay = 0.5

    loop_request = 1

    # The id of your ISS station (usually 1). If you use a wind meter connected
    # to a anemometer transmitter kit, use its id
    iss_id = 1

    # How long to wait for a response from the station before giving up (in
    # seconds; must be greater than 2)
    timeout = 4

    # How long to wait before trying again (in seconds)
    wait_before_retry = 1.2

    # How many times to try before giving up:
    max_tries = 4

    # Vantage model Type: 1 = Vantage Pro; 2 = Vantage Pro2
    model_type = 2

The issue started about 2 weeks ago.

Log between two reports:

Dec 29 19:11:31 web systemd[1]: Started WeeWX.
Dec 29 19:11:31 web weewxd[4674]: INFO __main__: Initializing weewxd version 5.1.0b6
Dec 29 19:11:31 web weewxd[4674]: INFO __main__: Command line: /usr/share/weewx/weewxd.py /etc/weewx/weewx.conf
Dec 29 19:11:31 web weewxd[4674]: INFO __main__: Using Python: 3.6.8 (default, Dec  4 2024, 01:35:34) #012[GCC 8.5.0 20210514 (Red Hat 8.5.0-22.0.1)]
Dec 29 19:11:31 web weewxd[4674]: INFO __main__: Located at:   /usr/bin/python3
Dec 29 19:11:31 web weewxd[4674]: INFO __main__: Platform:     Linux-5.4.17-2136.338.4.2.el8uek.x86_64-x86_64-with-oracle-8.10
Dec 29 19:11:31 web weewxd[4674]: INFO __main__: Locale:       'en_US.UTF-8'
Dec 29 19:11:31 web weewxd[4674]: INFO __main__: Entry path:   /usr/share/weewx/weewxd.py
Dec 29 19:11:31 web weewxd[4674]: INFO __main__: WEEWX_ROOT:   /etc/weewx
Dec 29 19:11:31 web weewxd[4674]: INFO __main__: Config file:  /etc/weewx/weewx.conf
Dec 29 19:11:31 web weewxd[4674]: INFO __main__: User module:  /etc/weewx/bin/user
Dec 29 19:11:31 web weewxd[4674]: INFO __main__: Debug:        1
Dec 29 19:11:31 web weewxd[4674]: INFO __main__: User:         weewx
Dec 29 19:11:31 web weewxd[4674]: INFO __main__: Group:        weewx
Dec 29 19:11:31 web weewxd[4674]: INFO __main__: Groups:       weewx
Dec 29 19:11:31 web weewxd[4674]: INFO weewx.engine: Loading station type Vantage (weewx.drivers.vantage)
Dec 29 19:11:37 web weewxd[4674]: INFO weewx.engine: StdConvert target unit is 0x1
Dec 29 19:11:37 web weewxd[4674]: INFO weewx.wxservices: StdWXCalculate will use data binding wx_binding
Dec 29 19:11:37 web weewxd[4674]: INFO weewx.engine: Archive will use data binding wx_binding
Dec 29 19:11:37 web weewxd[4674]: INFO weewx.engine: Record generation will be attempted in 'hardware'
Dec 29 19:11:37 web weewxd[4674]: INFO weewx.engine: The archive interval in the configuration file (900) does not match the station hardware interval (300).
Dec 29 19:11:37 web weewxd[4674]: INFO weewx.engine: Using archive interval of 300 seconds (specified by hardware)
Dec 29 19:11:37 web weewxd[4674]: INFO weewx.restx: StationRegistry: Registration not requested.
Dec 29 19:11:37 web weewxd[4674]: INFO weewx.restx: Wunderground: Posting not enabled.
Dec 29 19:11:37 web weewxd[4674]: INFO weewx.restx: PWSweather: Posting not enabled.
Dec 29 19:11:37 web weewxd[4674]: INFO weewx.restx: CWOP: Posting not enabled.
Dec 29 19:11:37 web weewxd[4674]: INFO weewx.restx: WOW: Posting not enabled.
Dec 29 19:11:37 web weewxd[4674]: INFO weewx.restx: AWEKAS: Posting not enabled.
Dec 29 19:11:37 web weewxd[4674]: INFO weewx.engine: 'pyephem' detected, extended almanac data is available
Dec 29 19:11:37 web weewxd[4674]: INFO __main__: Starting up weewx version 5.1.0b6
Dec 29 19:11:39 web weewxd[4674]: INFO weewx.engine: Clock error is -2.42 seconds (positive is fast)
Dec 29 19:11:39 web weewxd[4674]: INFO weewx.engine: Using binding 'wx_binding' to database 'weewx.sdb'
Dec 29 19:11:39 web weewxd[4674]: INFO weewx.manager: Starting backfill of daily summaries
Dec 29 19:11:39 web weewxd[4674]: INFO weewx.manager: Daily summaries up to date
Dec 29 19:11:42 web weewxd[4674]: INFO weewx.engine: Starting main packet loop.
Dec 29 19:15:18 web weewxd[4674]: INFO weewx.manager: Added record 2024-12-29 19:15:00 CET (1735496100) to database 'weewx.sdb'
Dec 29 19:15:18 web weewxd[4674]: INFO weewx.manager: Added record 2024-12-29 19:15:00 CET (1735496100) to daily summary in 'weewx.sdb'
Dec 29 19:15:18 web weewxd[4674]: INFO user.belchertown: version 1.3.1
Dec 29 19:16:47 web weewxd[4674]: INFO weewx.cheetahgenerator: Generated 12 files for report Belchertown in 88.94 seconds
Dec 29 19:16:47 web weewxd[4674]: INFO weewx.reportengine: Copied 40 files to /var/www/html/weewx/belchertown
Dec 29 19:20:18 web weewxd[4674]: INFO weewx.manager: Added record 2024-12-29 19:20:00 CET (1735496400) to database 'weewx.sdb'
Dec 29 19:20:18 web weewxd[4674]: INFO weewx.manager: Added record 2024-12-29 19:20:00 CET (1735496400) to daily summary in 'weewx.sdb'
Dec 29 19:21:45 web weewxd[4674]: INFO weewx.cheetahgenerator: Generated 12 files for report Belchertown in 87.56 seconds
Dec 29 19:21:45 web weewxd[4674]: INFO weewx.reportengine: Copied 3 files to /var/www/html/weewx/belchertown

Thanks.

[vince]

Belchertown taking very long to run is a known v5 issue related to some v5 weewx changes. Disable Belchertown and enable Seasons only for an hour and see if things improve. If so, it’s an easy fix.

Is your database very from v3 weewx perhaps?

[Marco Citossi]

Yes I've imported sdb from an old version of weewx (I think 3).

Thanks.

[Tom Keffer]

https://github.com/weewx/weewx/wiki/v5-performance-troubleshooting#belchertown-skin

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/weewx-user/2983a5c4-eacc-4fdc-9809-5dd9f8ccbf25n%40googlegroups.com.

[vince]

See the bottom of https://github.com/poblabs/weewx-belchertown/issues/924 for the usual quickest fix