macOS Sonoma 14.6.1 daemon launch under weewx 5.1.0 fails

[Gary S]

I'd been running 4.9.1 for too long and had errors now and again that required a manual restart, and I thought the Wunderground-caused update problem was actually me.

Anyway, I've migrated to 5.1.0 and I've run into a couple glitches.

1) Trying to run 

~weewx-data/scripts/setup-daemon.sh

fails because 'getent' is a Linux script (apparently), not a macOS one. I commented out that part and the remainder worked fine, however:

prompt> sudo launchctl load /Library/LaunchDaemons/com.weewx.weewxd.plist

Load failed: 5: Input/output error

Try running `launchctl bootstrap` as root for richer errors.

prompt>

I'm not sure how to proceed.

2) I also need to figure out how to merge weewx.sdb from 

/Users/shared/weewx/archive/weewx.sb (ends at 13:15 26 Aug)

with

 ~/weewx-data/archive/weewx.sdb (current and being updated)

Any advice is welcome on either!

[vince]

> "Try running `launchctl bootstrap` as root for richer errors."

> I'm not sure how to proceed.

I would try running 'launchctl bootstrap' as root for richer errors.

Sometimes you should try what they suggest.

[Gary S]

Fair enough. What is the "<domain-target">  argument that's relevant to weewx? Looking over the macOS launchctl man page and other docs hasn't helped.

prompt> sudo launchctl bootstrap 

Usage: launchctl bootstrap <domain-target> [service-path, service-path2, ...]

I also perused many of the available MacOS docs for weewx but many predate 5.0, so aren't relevant.

Thanks.

[vince]

The MacOS docs have nothing to do with weewx versions, so they should have been relevant but to me they were definitely clear as mud.  I figured it out experimentally.  Hope this helps....

The plist file the pip installer generates is incorrect.  This one worked for me.  I'll explain below.....

<?xml version="1.0" encoding="UTF-8"?>
<!-- property list file for weewx daemon -->
<!-- put this file in /Library/LaunchDaemons -->
<!-- to start weewx: -->
<!--   sudo launchctl load -w /Library/LaunchDaemons/com.weewx.weewxd.plist -->
<!-- to stop: -->
<!--   sudo launchctl unload /Library/LaunchDaemons/com.weewx.weewxd.plist -->
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.weewx.weewxd</string>
    <key>Disabled</key>
    <false/>
    <key>RunAtLoad</key>
    <true/>
    <key>ProgramArguments</key>
    <array>
        <string>/opt/homebrew/opt/pyt...@3.12/bin/python3.12</string>
        <string>/Users/vince/weewx-venv/lib/python3.12/site-packages/weewxd.py</string>
        <string>/Users/vince/weewx-data/weewx.conf</string>
    </array>
    <key>StandardErrorPath</key>
    <string>/var/log/weewx_err.log</string>
</dict>
</plist>

Basically the <string> lines are the path to python, to weewxd.py, and to the weewx.conf file you want to use.   Do a little light tweaking to match your setup's paths and copy it as indicated in the comments in the file.  Run the sudo command to start it up.  Check your logfile to verify it's running.

Note - "ps -ef | grep wee" will show the process, as will "ps aux | grep wee" to verify it's running.   Also for a Mac you need to append something to weewx.conf so that logging works.  I appended the following to my weewx.conf per the instructions in (I think) the wiki.

[Logging]
    [[root]]
        handlers = timed_rotate,
    [[handlers]]
        [[[timed_rotate]]]
            level = DEBUG
            formatter = verbose
            class = logging.handlers.TimedRotatingFileHandler
            filename = log/{process_name}.log
            when = midnight
            backupCount = 7

[vince]

Minor update just so I can find it myself years from now...

https://apple.stackexchange.com/questions/364094/how-to-view-status-of-service-e-g-whether-its-running-in-a-format-similar-to seems to be the best documentation I can quickly find. Once you load the plist file you can find it....

[vince@mini util]$ sudo launchctl print system | grep wee
   14780      - com.apple.periodic-weekly
   64515      - com.weewx.weewxd
"com.weewx.weewxd" => enabled

[vince@mini util]$ sudo launchctl print system/com.weewx.weewxd
system/com.weewx.weewxd = {
active count = 1
path = /Library/LaunchDaemons/com.weewx.weewxd.plist
type = LaunchDaemon
state = running

program = /opt/homebrew/opt/pyt...@3.12/bin/python3.12
arguments = {
/opt/homebrew/opt/pyt...@3.12/bin/python3.12
/Users/vince/weewx-venv/lib/python3.12/site-packages/weewxd.py
/Users/vince/weewx-data/weewx.conf
}

stderr path = /var/log/weewx_err.log
default environment = {
PATH => /usr/bin:/bin:/usr/sbin:/sbin
}

environment = {
XPC_SERVICE_NAME => com.weewx.weewxd
}

domain = system
minimum runtime = 10
exit timeout = 5
runs = 1
pid = 64515
immediate reason = speculative
forks = 2
execs = 2
initialized = 1
trampolined = 1
started suspended = 0
proxy started suspended = 0
last exit code = (never exited)

spawn type = daemon (3)
jetsam priority = 40
jetsam memory limit (active) = (unlimited)
jetsam memory limit (inactive) = (unlimited)
jetsamproperties category = daemon
submitted job. ignore execute allowed
jetsam thread limit = 32
cpumon = default
probabilistic guard malloc policy = {
activation rate = 1/1000
sample rate = 1/0
}

properties = runatload | inferred program
}

After unloading it, it's not visible in the commands above...

$ sudo launchctl unload /Library/LaunchDaemons/com.weewx.weewxd.plist

$ sudo launchctl print system/com.weewx.weewxd
Bad request.
Could not find service "com.weewx.weewxd" in domain for system

[matthew wall]

On Sunday, September 1, 2024 at 5:01:42 PM UTC-4 vince wrote:

Basically the <string> lines are the path to python, to weewxd.py, and to the weewx.conf file you want to use.   Do a little light tweaking to match your setup's paths and copy it as indicated in the comments in the file. 

the three 'string' program arguments are *supposed* to be modified when you create a station.  if those strings do not match the python, weewxd, and conf for the python/venv/pip installation from which you did a 'station create', then we have a bug in the pattern match in the function copy_util in station_actions.py

getent is available on linux and the BSDs, but apparently was removed in the more recent macos.  there is a fix for setup-daemon.sh that will be in the next weewx release (5.1.1?)

https://github.com/weewx/weewx/commit/df81fd83933c02895f573c31384c92d5c7f1f48b

[matthew wall]

On Sunday, September 1, 2024 at 2:36:35 PM UTC-4 w.gary...@gmail.com wrote:

prompt> sudo launchctl load /Library/LaunchDaemons/com.weewx.weewxd.plist

Load failed: 5: Input/output error

Try running `launchctl bootstrap` as root for richer errors.

prompt>

i am guessing that the launchctl invocation failed because the file /Library/LaunchDaemons/com.weewx.weewxd.plist does not exist.  since getent failed, the plist was probably never copied.

please check the contents of the plist in the weewx-data directory that was created when you did a 'weectl station create'.  are the python, weewxd, and weewx.conf paths correct in that plist file?

 

2) I also need to figure out how to merge weewx.sdb from 

/Users/shared/weewx/archive/weewx.sb (ends at 13:15 26 Aug)

with

 ~/weewx-data/archive/weewx.sdb (current and being updated)

do something like this (NOT TESTED!!!):

# stop weewx

sudo launchctl unload /Library/LaunchDaemons/com.weewx.plist

# make a copy of the old database and the active database

cp /Users/shared/weewx/archive/weewx.sdb weewx1.sdb

cp ~/weewx-data/archive/weewx.sdb weewx2.sdb

# move aside the active database

mv ~/weewx-data/archive/weewx.sdb ~/weewx-data/archive/weewx.sdb-YYmmdd

# copy new data into the copy of the old database

sqlite3 weewx1.sdb

sqlite> attach database 'weewx2.sdb' as 'weewx2';

sqlite> insert into weewx1.archive select * from weewx2.archive;

# rebuild the daily summaries

weectl database rebuild-daily

# move the newly modified database into place

cp weewx1.sdb ~/weewx-data/archive/weewx.sdb

# start weewx

sudo launchctl load /Library/LaunchDaemons/com.weewx.plist

# clean up

rm weewx1.sdb weewx2.sdb

[vince]

Matthew - just confirming your fix 'did' work great on sonoma 14.6.1 on M1 mini.

There's no bug in the string arguments in a clean installation so you're good there too.

Thanks !

[Gary S]

I now have a correct /Library/LaunchDaemons/com.weewx.weewxd.plist file pointing to the correct places, and stopping/restarting the daemon is working properly.

Thanks Vince and Matthew!

[Gary S]

On Sunday, September 1, 2024 at 5:51:40 PM UTC-6 matthew wall wrote:

[...]

do something like this (NOT TESTED!!!):

# stop weewx

sudo launchctl unload /Library/LaunchDaemons/com.weewx.plist

# make a copy of the old database and the active database

cp /Users/shared/weewx/archive/weewx.sdb weewx1.sdb

cp ~/weewx-data/archive/weewx.sdb weewx2.sdb

# move aside the active database

mv ~/weewx-data/archive/weewx.sdb ~/weewx-data/archive/weewx.sdb-YYmmdd

# copy new data into the copy of the old database

sqlite3 weewx1.sdb

sqlite> attach database 'weewx2.sdb' as 'weewx2';

sqlite> insert into weewx1.archive select * from weewx2.archive;

# rebuild the daily summaries

weectl database rebuild-daily

# move the newly modified database into place

cp weewx1.sdb ~/weewx-data/archive/weewx.sdb

# start weewx

sudo launchctl load /Library/LaunchDaemons/com.weewx.plist

# clean up

rm weewx1.sdb weewx2.sdb

Thanks! You've put me on the right path.

My remaining issue is that my old weewx.sdb and current weewx.sdb overlap in time, as expected, so insertion fails. Once I figure out the appropriate syntax for SQL I'll strip one or the other of the appropriate piece and reassemble.

If only these were netCDF files I'd be done in about two seconds...

 

[vince]

Gary you can do a sqlite3 backup of one db, and restore that into the other db (always working off copies of course).  Sqlite3 is smart enough to ignore any duplicate records.  Then do the rebuild-daily to rebuild the summary tables.

[Gary S]

Thanks for the tip - how would I do that exactly? I'm negligibly familiar with DBs and not hugely interested in figuring out something so trivial on my own. Thanks!

[vince]

On Saturday, September 7, 2024 at 9:26:25 AM UTC-7 Gary S wrote:

I'm negligibly familiar with DBs and not hugely interested in figuring out something so trivial on my own. 

That's not going to get you a lot of answers from folks not hugely interested in spending their limited spare time on this one....