Installation of extension fails for pip installed WeeWX but works for apt installed WeeWX

[Karen K]

One user tried to install weewx-DWD into an WeeWX installation made by pip. The error messages refer to files of core WeeWX:

(weewx-venv) user@pc:~$ sudo weectl extension install weewx-dwd.zip
[sudo] Passwort für user:
Error parsing config file: Unable to find file 'weewx.conf'. Tried directories ['/root/weewx-data', '/etc/weewx', '/home/weewx']
Traceback (most recent call last):
File "/usr/share/weewx/weeutil/startup.py", line 86, in start_app
config_path, config_dict = weecfg.read_config(config_option, [config_arg])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/share/weewx/weecfg/init.py", line 147, in read_config
config_path = find_file(option_path, args, locations=locations, file_name=file_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/share/weewx/weecfg/init.py", line 104, in find_file
raise OSError(f"Unable to find file '{file_name}'. Tried directories {locations}")
OSError: Unable to find file 'weewx.conf'. Tried directories ['/root/weewx-data', '/etc/weewx', '/home/weewx']

I have no glue what happens there. The messages say, that weewx.conf could not be found. Looking for that file is not part of install.py. 

Unfortunately I cannot try the installation myself actually. And I have no experience with weewx-venv.

See install.py here.

There is a configure() method in this case, which is not widely used. But there is no reference to weewx.conf in it. I would have expected that the copying and linking in it may fail within a virtual environment, but the error messages say otherwise.

[Dominic Reich]

I doubt `sudo` should be used here.

Karen K <kk44...@gmail.com> wrote:

>One user tried to install [1]weewx-DWD into an WeeWX installation made by pip.

>The error messages refer to files of core WeeWX:
>
>
>
> (weewx-venv) user@pc:~$ sudo weectl extension install weewx-dwd.zip
> [sudo] Passwort für user:
> Error parsing config file: Unable to find file 'weewx.conf'. Tried
> directories ['/root/weewx-data', '/etc/weewx', '/home/weewx']
> Traceback (most recent call last):
> File "/usr/share/weewx/weeutil/startup.py", line 86, in start_app
> config_path, config_dict = weecfg.read_config(config_option, [config_arg])
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/usr/share/weewx/weecfg/init.py", line 147, in read_config
> config_path = find_file(option_path, args, locations=locations, file_name=
> file_name)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/usr/share/weewx/weecfg/init.py", line 104, in find_file
> raise OSError(f"Unable to find file '{file_name}'. Tried directories
> {locations}")
> OSError: Unable to find file 'weewx.conf'. Tried directories ['/root/
> weewx-data', '/etc/weewx', '/home/weewx']
>
>
>I have no glue what happens there. The messages say, that weewx.conf could not
>be found. Looking for that file is not part of install.py. 
>
>Unfortunately I cannot try the installation myself actually. And I have no
>experience with weewx-venv.
>

>See [2]install.py here.

>
>There is a configure() method in this case, which is not widely used. But there
>is no reference to weewx.conf in it. I would have expected that the copying and
>linking in it may fail within a virtual environment, but the error messages say
>otherwise.
>

--
Life is really simple, but we insist on making it complicated.
- Confucius

[vince]

Agree. Your install.py should 'not' be run under sudo, you also need to modify it to be able to copy and symlink into /usr/local/bin (which requires sudo).

Why aren't the /usr/local/bin items simply installed and running out of the bin/user directory ????

[Karen K]

vince schrieb am Sonntag, 14. Juli 2024 um 20:19:45 UTC+2:

Agree. Your install.py should 'not' be run under sudo, you also need to modify it to be able to copy and symlink into /usr/local/bin (which requires sudo).

I am afraid, the installation crashes before all those copying and linking. It looks for weewx.conf and does not find it. Why?

 

Why aren't the /usr/local/bin items simply installed and running out of the bin/user directory ????

I found no way to get the original location of the files within the configure() method. That's the reason.

[Tom Keffer]

If not specified, the utility weectl (and, for that matter, weewxd) looks in the "usual places," which are ~/weewx-data, /etc/weewx, and /home/weewx. If a config file is located someplace other than those three places, it must be specified explicitly using the --config option.

Because sudo is being used, ~/weewx-data resolves to /root/weewx-data. Is that what you intended?

In your install.py file, you do some path arithmetic around /usr/local/bin, which may or may not require root privileges. Any particular reason? Generally, pip installs do not use root privileges.

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/c7b9e991-67fd-4c24-861e-7b3ca8c766c1n%40googlegroups.com.

[Karen K]

Thank you all.

I read a little bit through the pip installation guide. There all calls of weectl are preceded by "source ~/weewx-venv/bin/activate". Could it be, that for installing extensions this is necessary, too? 

@Tom: When I wrote the first parts of this extension, I was new to WeeWX, and I used some separate scripts to create files that are later included into the skin using the #include directive. Later I was asked to publish those scripts. So they were included into this extension, and I had to deal with the installation of them. Not easy.

[Tom Keffer]

On Sun, Jul 14, 2024 at 12:13 PM Karen K <kk44...@gmail.com> wrote:

I read a little bit through the pip installation guide. There all calls of weectl are preceded by "source ~/weewx-venv/bin/activate". Could it be, that for installing extensions this is necessary, too? 

Absolutely. Sourcing the "activate" file nominates the Python interpreter, the Python runtime library, and the WeeWX code that is to be used. 

[Karen K]

So I will have to update all the installation instructions of my extensions to reflect this requirement. 

Would it be an idea to mention that there?

 

[vince]

It is in the FAQ at https://github.com/weewx/weewx/wiki/WeeWX-Frequently-Asked-Questions#why-do-i-get-command-not-found-when-i-type-weewx-commands but perhaps buried a bit.

If you put everything into bin/user and remove your shebang lines all the python scripts should probably work ok.    If you could rewrite your bash scripts into python that would probably help but I'm not 100% certain that is necessary.  I don't really understand your skin enough to know for certain.

You will need to pip3 install any additional modules into the venv as part of your instructions.  You seem to use quite a few added modules in your python scripts. 

[Tom Keffer]

Let's back up a bit. On closer inspection of your user's log, I see entries such as /usr/share/weewx. That implies a package install, so weewx-venv is not involved. Yet, weewx.conf was not found in /etc/weewx/weewx.conf, its normal location with a package install.

You'll have to get clarification from your user how s/he did the original install. It appears non-standard.

--

You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/c7eba711-9f44-41b1-a0b5-8d263099828bn%40googlegroups.com.

[Karen K]

Thank you. I will ask.

[Karen K]

The result is: If the virtual environment (venv) is used, weectl must not be invoked by sudo. Without sudo, the correct paths are found. Thank you, Tom, for pointing to this.

Additional question: Where can I place a script that can be invoked by the user from the command line in such an environment? 

[michael.k...@gmx.at]

bin/user?

[Graham Eddy]

venv/bin/ (where the python program uses that venv)

⊣GE⊢

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/7cc7ba3b-2bc7-48c4-b3bb-3f6373a4a245n%40googlegroups.com.

[Karen K]

michael.k...@gmx.at schrieb am Dienstag, 16. Juli 2024 um 09:39:11 UTC+2:

bin/user?

You need a directory for that purpose that is included in $PATH. Otherwise the shell does not find the script. And bin/user is not. 

[Tom Keffer]

The whole pip process carefully guards against putting things in PATH. The only way to do it is to declare it in the pyproject.toml file, after which pip builds a shim to call the appropriate script. Obviously, that's not going to work for an extension.

Question: do you expect the same script to be used for installations that have done a package install, as well as a pip install? 

If only the latter, then I can think of two possibilities:

1. Put it in venv/bin/user. It won't be in $PATH, so you'd have to instruct the user on where to find it.

2. Make it installable via pip. Write a pyproject.toml file, then use poetry or some other builder to create a wheel and upload it to pypi.org.

-tk

--

You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/43def605-b216-4969-9aaf-eb63507536e8n%40googlegroups.com.

[vince]

If you want to keep copying things into /usr/local/bin and chmod'ing them there, lines 128 and 142 of https://github.com/roe-dl/weewx-DWD/blob/master/install.py need to be changed to always use sudo

This assumes the user can use sudo, but a weewx pip user is already assumed to be able to do that so they can run the commands to install the system startup file and the like.   For a packaged installation it 'should' work the same but you'd need to test that of course.

Something like:

os.system("sudo some_command_here")

or the subprocess way to do it per https://docs.python.org/3/library/subprocess.html

[Karen K]

Tom Keffer schrieb am Dienstag, 16. Juli 2024 um 16:20:57 UTC+2:

Question: do you expect the same script to be used for installations that have done a package install, as well as a pip install? 

Yes. 

Primarily those scripts are invoked by crontab and create some files that are later used in skins by #include. Later on, when I added more modules to this extension, I wrote a WeeWX service that creates a thread, and that thread downloads data from some provider and processes them and then sleeps until new data are scheduled.

So I guess, I will have to set the migration of those old scripts into a WeeWX service to a higher priority.

 

[Tom Keffer]

That would be the "weewx way" of doing it!

--

You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/93e82688-7653-41d9-8553-3f994f6668a2n%40googlegroups.com.