Adding python try: and except: and finally: to extensions.py crashes weewx

[messyp...@gmail.com]

I read several add'l values in from static files. Every once in a great while, one of the files has extraneous character in it and it can't be read as a float() and extensions.py crashes weewx. So after reading about try/except/finally from several sources, I wanted to put the section where it reads the float in a try statement, and if it fails, the exception action is to assign a nominal float value for that weewx cycle, and write it to the database. However when I put in the try: it immediately causes weewx to exit, at all debug levels. 

If you are more knowledgeable on Python 3 than I am, please look at my extensions.py attachment and tell me how I need to restructure that code block so it will work. Note, I have tried a couple different positions for the initial try: block, no success. 
Further note please, that this only applies to the lines where I open '/home/pi/cputemp-opi5p.dat' and read the value into degf2 for it to store it in 'extraTemp7'. Note further that this code is working for all seven parameters being read and placed into the data, however, occasionally (about once a week), somehow the system reads in erroneous data (such as '94.8\x00' instead of '94.8')  and causes the python to exit. 

I have looked at places where this data is created and cannot find any evidence of that invalid value.

Here is the pertinent part of the log at failure time:

pi@PI4B1:~ $ systemctl status weewx
● weewx.service - WeeWX weather system
     Loaded: loaded (/etc/systemd/system/weewx.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Tue 2025-04-15 16:00:16 CDT; 4h 18min ago
       Docs: https://weewx.com/docs
    Process: 268425 ExecStart=/home/weewx/bin/weewxd /home/weewx/weewx.conf (code=exited, status=1/FAILURE)
   Main PID: 268425 (code=exited, status=1/FAILURE)
        CPU: 4h 37min 25.648s

Apr 15 16:00:16 PI4B1 weewxd[268425]: weewx[268425] CRITICAL __main__:     ****      self.engine.dispatchEvent(weewx.Event(weewx.NEW_ARCHIVE_RECORD,
Apr 15 16:00:16 PI4B1 weewxd[268425]: weewx[268425] CRITICAL __main__:     ****    File "/home/weewx/bin/weewx/engine.py", line 245, in dispatchEvent
Apr 15 16:00:16 PI4B1 weewxd[268425]: weewx[268425] CRITICAL __main__:     ****      callback(event)
Apr 15 16:00:16 PI4B1 weewxd[268425]: weewx[268425] CRITICAL __main__:     ****    File "/home/weewx/bin/user/extensions.py", line 98, in new_archive_packet
Apr 15 16:00:16 PI4B1 weewxd[268425]: weewx[268425] CRITICAL __main__:     ****      degf2 = float(float(f.read()))                #this is OPI5P
Apr 15 16:00:16 PI4B1 weewxd[268425]: weewx[268425] CRITICAL __main__:     ****  ValueError: could not convert string to float: '94.8\x00'
Apr 15 16:00:16 PI4B1 weewxd[268425]: weewx[268425] CRITICAL __main__:     ****  Exiting.
Apr 15 16:00:16 PI4B1 systemd[1]: weewx.service: Main process exited, code=exited, status=1/FAILURE
Apr 15 16:00:16 PI4B1 systemd[1]: weewx.service: Failed with result 'exit-code'.
Apr 15 16:00:16 PI4B1 systemd[1]: weewx.service: Consumed 4h 37min 25.648s CPU time.

[gjr80]

In the extension.py you posted the try..except..finally is not executed as each of the try..except..finally lines is commented out.

Gary

[Messy Potamia]

Of course they're commented out, when I uncomment [any] of them it exits / crashes. I need it to run until I can have someone tell me what's wrong / why the module crashes weewx. Duh.

--
You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/cM4T6W4XjdA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to weewx-user+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/weewx-user/62c17a5a-d508-4322-8458-c20941a0f45en%40googlegroups.com.

[gjr80]

Sorry, but I can't tell what code you are running other than to take at face value what you post. There is nothing wrong with the try..except..finally code as you posted it except for the comments. Comments aside, the code you posted cannot have caused the error trace you posted as the error trace shows the offending line as line 98 but the code you posted has the offending code at line 100. Can you post the exact error trace and the exact code that caused that error trace.

Gary

[Messy Potamia]

The error occurred before I added any try/except/finally. The error has to do with it trying to read something that has extraneous characters in it, and exiting because of it. So I wanted to add the error catching block...but when I do weewx refuses to run, and exits without any comment, even when weewx.conf has "debug = 2". As I wrote, the error happens about a couple time a month. I want to trap it so weewx will continue. 

To view this discussion visit https://groups.google.com/d/msgid/weewx-user/d9a3336a-129e-40df-b302-bff44a9eafdan%40googlegroups.com.

[p q]

The double float conversion looks wonky but wouldn't be the cause. You shouldn't need to do

degf2 = float(float(f.read()))

You should be able to do degf2 = float(f.read())

The error message in your email says what the problem is - you're getting 94.8\x00 as the string and it's failing to convert it to a float. Not sure why your try catch doesn't work. I'd like to see what Gary's asking for - put the try/catch in let's see the output.

You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/weewx-user/CAB7-S776S8ECPE_2wE%3D46%3DUpVR2-rfrkWqooL_pzhJjcnFEF7Q%40mail.gmail.com.

--

Peter Quinn
(415)794-2264

[Karen K]

messyp...@gmail.com schrieb am Mittwoch, 16. April 2025 um 06:31:01 UTC+2:

Apr 15 16:00:16 PI4B1 weewxd[268425]: weewx[268425] CRITICAL __main__:     ****  ValueError: could not convert string to float: '94.8\x00'

May be the nul character at the end of the file to read is the problem. In the error message above it is expressed as "\x00". 

[Susan Mackay]

Probably not the problem (as it this relates to code above the error line) but you have several places where you open a file, read every line and process the data in that line but only pass back the last value read. This seems to be a waste of a lot of processing
.

I agree with the comments above - the extraneous NULL in the file would seem to be the issue. Perhaps you need to look at that and see if you can fix the problem in the file. (Probably not as I'm guessing by the comments that the file is written by the OS.) Try 'rstrip("\0")' after the 'read()'

Susan

[messyp...@gmail.com]

When I put the try: except: finally: back in, weewx never runs, I get the syslog entry of

systemd[1]: weewx.service: Main process exited, code=exited, status=1/FAILURE

I have to comment out the try: except: finally:  out of my extensions.py and it runs normal.

I do not have a clue why, every once in a while (~once/week), a read statement picks up a text value from a temperature file and reads it in with extraneous characters. I've checked the source file (I have a history .log which has all the readings & their date/time in it) and that file at the point where that instance of the temperature is, does not have anything extra.

[messyp...@gmail.com]

I do not know where the  "\x00"  is coming from.  It only occurs about once a week, or less, but the python error causes weewx to stop.

[p q]

try running weewx from the command line with the try/catch in place and see if you get better error messages.

--

You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/weewx-user/ad167a14-82f9-4bf7-9fde-0b115c6ef102n%40googlegroups.com.

--

Peter Quinn
(415)794-2264

[messyp...@gmail.com]

cat cputemp-opi5p.dat

105.6

All of my cpu temps (I have 6 total) look just like the above (cputemp-opi5p.dat, -op5.dat, -pi4b1.dat, -pi3b2.dat, -pi5r.dat, and cputemp-ser8.dat)

The 6 .dat files are refreshed (rewritten) every 5 minutes, a minute before weewx runs, and scp'd to the server running weewx for the extensions.py to read; each read()

only goes to its own file.

I've attached my *latest* extensions.py --  you'll see the 6 temperature readings, 2 are from orange pi's, 1 is from an SER8, and 3 are from raspbery pi's.

You may see my python statements and you may find fault with them, but in order to get them working I have already done way more reading/experimenting/testing than I

wanted to in order to get them to run without crashing. So they're tested as - is.  What I'm still trying to figure out (hence the try: except: finally:) is if the read or cast fails, to catch it and

just plug in a ringer (nominal) value for that run. But something about the big weewx python engine doesn't like the try: added into extensions.py, it just give a systemd[1]: weewx.service: Main process exited, code=exited, status=1/FAILURE

in the syslogs, and I have to comment out the try: in extensions.py in order for it to run.

Hope this makes it clearer as to the situation.

Thanks  --

[Karen K]

I am quite sure there is more than just "Main process exited" in /var/log/syslog. The general procedure to find the reasons of errors is to set debug=1, then restart WeeWX, then wait for 2 archive cycles if possible, then inspect /var/log/syslog. 

If one of those files referred within the open() statements do not exist or the name is misspelled, then WeeWX will crash. May be you can start with one of those open() statements, commenting out all the others. If you find no more errors within that statement, you can uncomment the next etc. pp.

[messyp...@gmail.com]

There will be no cycles, because after I issue [sudo systemctl start weewx]  it   **immediately exits**. Therefore no debug=1 output.  The only thing in syslog is

Apr 15 21:05:48 PI4B1 systemd[1]: weewx.service: Main process exited, code=exited, status=1/FAILURE
Apr 15 21:05:48 PI4B1 systemd[1]: weewx.service: Failed with result 'exit-code'.

[Karen K]

messyp...@gmail.com schrieb am Freitag, 18. April 2025 um 14:36:13 UTC+2:

There will be no cycles, because after I issue [sudo systemctl start weewx]  it   **immediately exits**. Therefore no debug=1 output.  The only thing in syslog is

Apr 15 21:05:48 PI4B1 systemd[1]: weewx.service: Main process exited, code=exited, status=1/FAILURE
Apr 15 21:05:48 PI4B1 systemd[1]: weewx.service: Failed with result 'exit-code'.

I have difficulties to believe that there is really nothing before those messages.

As someone said before you could start WeeWX from the command line as described here.  

[Messy Potamia]

> As someone said before you could start WeeWX from the command line as described here.

This is 4.9.1, above is for weewx 5. I'm using systemd.
So after investigating further, here's what I ran:
pi@PI4B1:/etc/systemd/system $ sleep 3;sudo /home/weewx/bin/weewxd
(I sleep my command 3 secs so I have time to switch to my syslog window)
And it did not crash, after removing the comment try: except: finally:
lines in the python in extensions.py
So ... question is, why does starting it using systemd crash because
of extensions.py, but running it from the command line does not?
Not sure how much of this is OS, how much is weewx version, how much
is my diminished mental status. Just thought of one thing: I always
use the setup.py method to install weewx; this may be something that
Tom could weigh in on.
Repeatability: I'm going to CtrlC the command window, restart it using
systemctl start weewx, and verify it indeed exits:
And it restarts normally. I did some testing, in extensions.py I put
in some code, calling "IntroduceAnError()", and I get the expected
entries in syslog ("CRITICAL __main__: **** NameError: name
'IntruduceAnError' is not defined)
Removed my intentional python error out of extensions.py.
Then, stopped and restarted weewx, and it started normally.
So it's working as expected. I don't have a clue why it wouldn't start
weewx, then running it once manually, then running it systemd, it
suddenly works normally. I diff'd my two extensions.py and the only
diff was the comment out of the try: except: finally: lines, in case
I'd made a typo.
I'm clueless.
You're all up to date.
And thanks for all your help and suggestions. I wish I'd learned
something here, but it's frustrating that it just started working for
no noticeable reason, just like it failed to work for no apparent
reason.

--- Thanks again ---

> --
> You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/cM4T6W4XjdA/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to weewx-user+...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/weewx-user/ba2e6dcf-6fc0-432d-89c8-cc1651ab3785n%40googlegroups.com.

[Karen K]

Messy Potamia schrieb am Samstag, 19. April 2025 um 05:29:38 UTC+2:

> As someone said before you could start WeeWX from the command line as described here.
This is 4.9.1, above is for weewx 5. I'm using systemd.

If I may comment this: The link was to the WeeWX 5.1 documentation, not 4.9.1.

Runnung WeeWX from the command line is not an alternative to using systemd. It is for debugging purposes only. You may see more error messages, if you run WeeWX directly, than you see if you start WeeWX using systemd.