getting /home/weewx/bin into $PATH

[Vince Skahan]

We continue to see 'many' posts where new unix/linux users simply don't understand $PATH and how to get the non-standard /home/weewx/bin location into their $PATH.

A quick fix would be to add a file to /etc/profile.d with the following contents.   I called mine /etc/profile.d/weewx-path.sh, but any name ending in .sh will work.

Tests ok on debian and raspian for me.

# if we find the daemon in the setup.py location

# then append the setup.py bindir to $PATH

if [ -e /home/weewx/bin/weewxd ] ; then

    PATH=$PATH:/home/weewx/bin

fi

Is this something we can add to weewx possibly ?    Add one file to the sources and one line to setup.py to install it.

I'd be happy to do a PR if that'll help.

Thoughts ?

[Tom Keffer]

I really don't like the idea of messing with profiles, especially system profiles. There is nothing standard about them. 

Understanding how paths work is fundamental to using command lines. We will just have to insist that users learn this minimum, especially if they are doing a setup.py install. If they can't handle that, they should do a package install.

--
You received this message because you are subscribed to the Google Groups "weewx-development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-developm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/2e6e6184-121b-42cc-b59f-159052f27953n%40googlegroups.com.

[Vince Skahan]

On Tuesday, April 20, 2021 at 8:20:35 AM UTC-7 Tom Keffer wrote:

I really don't like the idea of messing with profiles, especially system profiles. There is nothing standard about them. 

Understanding how paths work is fundamental to using command lines. We will just have to insist that users learn this minimum, especially if they are doing a setup.py install. If they can't handle that, they should do a package install.

Yes, I understand both points there....but history is proving that they can't handle that, and they are still doing setup installs, and they're still being very frustrated by the result.

This kind of thing is exactly what the profile.d hooks in the shell startup sequence was designed for.  Drop one file in there and 100% of those problem reports goes away, as do so many hours of people struggling os trivia they have no ability nor interest in learning.

Is there any actual downside of appending to $PATH ?

[gjr80]

I do think a bit of this is self-inflicted on our behalf. If you have a look at the various command strings in the docs, the wiki, readmes for extensions on GitHub etc you will see a lot that include no path with no mention that a path may be required. This is fine for package installs but setup.py install users take pot luck based on pwd. For some time I have tried  to remember to add a note in my readmes re the need for a path, makes the document less readable but hopefully points some setup.py novices in the right direction.

Gary

[Cameron D]

Downsides?  in reality, probably none as the likelihood  of name clashes are slim.

However, it would probably be against the general principals, because those are for all users on a system, while typically only a single user might have permission to access the system.

For that user the correct place would be in their own profile files.

If they rely on sudo, then I don't think it sources those extra /etc/profile.d files.

[Graham Eddy]

another argument in favour of a dedicated user for weewx software. all my weewx software runs as user ‘weewx’; i log in as weewx when i have a work session on it

(the default on RPi is that $HOME/bin is added to PATH if it exists, so /home/weewx/bin gets picked up automatically when log in as weewx)

it also saves me accidentally stomping on weewx files when i am poking around as my normal user

[Vince Skahan]

On Wednesday, April 21, 2021 at 6:41:27 PM UTC-7 Cameron D wrote:

Downsides?  in reality, probably none as the likelihood  of name clashes are slim.

The system dirs are ahead of weewx in $PATH anyway, so the risk is miniscule if any exists at all.

 

However, it would probably be against the general principals, because those are for all users on a system, while typically only a single user might have permission to access the system.

Actually it's exactly what the profile.d thing is for.   It's for making system-wide changes and not requiring each user to take action themselves.  Requiring users to take action themselves basically never works.  They're simply not going to do that as the general rule. 

My suggestion is to make it more newbie proof so we don't even have to ask them.

For that user the correct place would be in their own profile files.

Lean toward agree, 'if' the users would actually do so.  History proves the vast majority do not.  This is the underlying issue.  We can't make new users who are disinclined to care about learning anything to actually try to learn anything.

Also agree with Graham suggesting running weewx as a non-privileged user, but that's a bigger lift to make happen that I didn't want to suggest.   Getting into that would require the pieces of weewx that talk to the /dev device to have appropriate privs to do so, and there might be portability issues there. 

 

If they rely on sudo, then I don't think it sources those extra /etc/profile.d files.

It's already sourced in in the non-privileged shell you ran sudo from, so that's not an issue.   Tested on multiple variants of linux.

[Vince Skahan]

On Wednesday, April 21, 2021 at 6:50:03 PM UTC-7 Graham Eddy wrote:

(the default on RPi is that $HOME/bin is added to PATH if it exists, so /home/weewx/bin gets picked up automatically when log in as weewx)

Not true in any version of the pi os (Raspbian) from early 2016 to current that I have running on various pi here.   You probably added something to your setup.

Creating a 'weewx' user with homedir /home/weewx and its $PATH including /home/weewx/bin is an interesting alternative to my suggestion of a one-liner /etc/profile.d file, but it would be more complicated to make work.  I don't know quite how to address the user/group ownership on things and still keeping access to /var/log and /dev working.   Can you provide more details ?

[Graham Eddy]

maybe i did add the PATH mod myself - setting up systems so often for so long that no doubt i do some things without thought…

/var/log is no problem. rsyslogs(8) looks after perms, your rsyslog config tells it what they should be, weewx sends all msgs to rsyslog to write them. my weewx logs are owned by ‘weewx'

/dev problem has not changed. easy using udev(7) - instructions on how to find your usb serial port, which is required for most weewx configs, should be prefaced by a demand to configure udev (a simple script should pick up 99% of cases). alternatively, there is usually a simple tool in linux that just requires lateral thinking - maybe tmpfiles.d(5) to adjust perms after boot (haven’t tried it on char device). my char files are owned by ‘weewx’ using udev

/run/weewx/ needs to be read-write for ‘weewx' for pid files. i use tmpfiles.d(5)

--
You received this message because you are subscribed to the Google Groups "weewx-development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-developm...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/a19542d6-f985-43a8-9d07-bf25171d502fn%40googlegroups.com.

[Cameron D]

For that user the correct place would be in their own profile files.

Lean toward agree, 'if' the users would actually do so.  History proves the vast majority do not.  This is the underlying issue.  We can't make new users who are disinclined to care about learning anything to actually try to learn anything.

Also agree with Graham suggesting running weewx as a non-privileged user, but that's a bigger lift to make happen that I didn't want to suggest.   Getting into that would require the pieces of weewx that talk to the /dev device to have appropriate privs to do so, and there might be portability issues there. 

yes, I operate under a separate weewx user as well, but  I think this started about specific setup.py installs.

It would be relatively easy under specific packaging systems but I imagine it gets a bit of a mess covering all options.

Would the simplest option be to make more effort to discourage setup.py installs with a few warnings?

On the deb package installs there are wee_xxx wrapper scripts placed in /usr/bin so the added PATH complexity is not needed.

 

If they rely on sudo, then I don't think it sources those extra /etc/profile.d files.

It's already sourced in in the non-privileged shell you ran sudo from, so that's not an issue.   Tested on multiple variants of linux.

Debian 10 removed that security hole with the default securepath setting.

 

[Cameron D]

On Thursday, 22 April 2021 at 2:34:11 pm UTC+10 Graham Eddy wrote:

maybe i did add the PATH mod myself - setting up systems so often for so long that no doubt i do some things without thought…

Likewise - $HOME/bin is a standard setup on my systems - I do recall seeing it by default on a new install somewhere, but not sure how common it is.

 

/dev problem has not changed. easy using udev(7) - instructions on how to find your usb serial port, which is required for most weewx configs, should be prefaced by a demand to configure udev (a simple script should pick up 99% of cases). alternatively, there is usually a simple tool in linux that just requires lateral thinking - maybe tmpfiles.d(5) to adjust perms after boot (haven’t tried it on char device). my char files are owned by ‘weewx’ using udev

udev rules are be good enough for ownership and mode on raw usb and on ones that look like serial ports.

 

/run/weewx/ needs to be read-write for ‘weewx' for pid files. i use tmpfiles.d(5)

The preferred method of run-file ownership is specifying it in the unit-file - if you are using full systemd. 

These again illustrate the problems of trying to maintain support with so many underlying systems -  trying to explain to people who installed using setup.py but  can't even tell you how their system is configured seems to be doing things the hard way.

[Graham Eddy]

perhaps make user demonstrate sufficient knowledge by making a few changes (e.g. setting up PATH) before setup.py successfully executes? force users who cannot show enough knowledge (or initiative to learn) to package install for support reasons

[Glenn McKechnie]

On 22/04/2021, 'Cameron D' via weewx-development

<weewx-de...@googlegroups.com> wrote:
>
>
> On Thursday, 22 April 2021 at 2:34:11 pm UTC+10 Graham Eddy wrote:
>
>> maybe i did add the PATH mod myself - setting up systems so often for so
>> long that no doubt i do some things without thought…
>>
>
> Likewise - $HOME/bin is a standard setup on my systems - I do recall seeing
> it by default on a new install somewhere, but not sure how common it is.

It's installed as part of the Bash package.

I've got it on the Pi here - admittedly it's a tad old(oldstable)!


It's sourced from ~/.profile, available from /etc/skel/.profile.
On this Pi it's from ...
bash/oldoldstable 4.3-11+deb8u2 armhf [upgradable from: 4.3-11+deb8u1]
GNU Bourne Again SHell


# ~/.profile: executed by the command interpreter for login shells.
# This file is not read by bash(1), if ~/.bash_profile or ~/.bash_login
# exists.
# see /usr/share/doc/bash/examples/startup-files for examples.
# the files are located in the bash-doc package.

# the default umask is set in /etc/profile; for setting the umask
# for ssh logins, install and configure the libpam-umask package.
#umask 022

# if running bash
if [ -n "$BASH_VERSION" ]; then
# include .bashrc if it exists
if [ -f "$HOME/.bashrc" ]; then
. "$HOME/.bashrc"
fi
fi

# set PATH so it includes user's private bin if it exists
if [ -d "$HOME/bin" ] ; then
PATH="$HOME/bin:$PATH"
fi


--


Cheers
Glenn

rorpi - read only raspberry pi & various weewx addons
https://github.com/glennmckechnie

[Vince Skahan]

On Wednesday, April 21, 2021 at 10:20:37 PM UTC-7 Cameron D wrote:

Would the simplest option be to make more effort to discourage setup.py installs with a few warnings?

Think no.  It's already discouraged.  The problem is that the users ignore the docs saying 'you need some skills to install this way'.

 

On the deb package installs there are wee_xxx wrapper scripts placed in /usr/bin so the added PATH complexity is not needed.

That's a little contrary to the idea of setup.py putting everything weewx-related into /home/weewx other than the startup file hook of course.

  

If they rely on sudo, then I don't think it sources those extra /etc/profile.d files.

It's already sourced in in the non-privileged shell you ran sudo from, so that's not an issue.   Tested on multiple variants of linux.

Debian 10 removed that security hole with the default securepath setting.

Sigh.  Yes.  I updated my debian10 vagrant box and retested.  When I did 'sudo bash' the appended path indeed disappeared for root.   We'd need to append /home/weewx/bin to root's $PATH too.   Not as sure I like that concept.

This points to how hard this stuff is to do portably enough.

Still hoping somebody comes up with a simple solution here.  We get soooooooo many problem reports that all center around the same things.  Users trying to do privileged things without using sudo (permissions issues) and users who don't set their $PATH up correctly.

We can't make them read the docs, learn the docs, or follow the docs.  That's for sure.   Hoping there's a technical solution.

[mwall]

a few thoughts:

1) a setup.py install should not touch any part of the system other than the directory in which you install weewx.  if you want udev rules, logrotate, rsyslog, init.d/systemd, PATH, or anything else you do it yourself.  the weewx setup.py is not a typical python installation - it is a way to install weewx into a single directory.

2) a package install really should use an unprivileged user to run weewx, not root.  install as root, but run as unprivileged user.  (i've sat down to do this a few times, but inevitable stop because of testing all of the update/upgrade corner cases)

3) a package install should install all of these items, in a way that works out-of-the-box and degrades gracefully:

  - init.d/systemd

  - udev rules (but they must not conflict with anything else user might have)

  - rsyslog, logrotate, logwatch in /etc/weewx, ready to go but not enabled

  - weewx user+group so weewx is run as unprivileged user

ideally the package install would also probe the system for known hardware (if it is a first-time install), and automatically choose the right driver.

4) documentation should always use relative path, i.e., just the command, not the full path to command.  you can make a single comment about possible paths if you want, but that has already been done in the user guide and a dedicated wiki page.  do not pollute the rest of the documentation with that confusion.

5) in the docs, do not preface instructions with 'sudo'  unless you mean it, and be sure to preface with 'sudo' when you know that privilege escalation is required.

6) for any upgrade, i don't even like the fact that the weewx installer touches my config files (especially when it just mangles comments).  and the dependency on a version string in the weewx.conf file is a fail waiting to happen.

it is important to distinguish between documentation - something that shows the concepts - versus recipes - something that you might copy-paste to restore your system, or something that is bespoke for a specific configuration.  given the multiple platforms and experimental nature of weewx, there will never be one recipe to rule them all.

i'm pretty settled about (1) - like tom said, if a user does setup.py, that user must at least understand PATH.

if someone does not understand PATH, just refer him/her to the PATH wiki page.

sorry for expanding the scope of this thread, but this is a spaghetti problem, not a meatball problem.

m

[mwall]

a few more things:

- sorry the last posting was so blunt - i've been close to the (currently uncooperative) machine the past few days, so i'm more used to talking to a computer than to people.  i mean no disrespect, and i *really* appreciate all of the ideas and testing that vince, cameron, glenn, and graham are doing.  thank you!

- we will never *not* have noobs who do the wrong thing, or jump in over their head before they can even swim.  some might argue that's a good thing - that's how i learned a lot of what i do :)  so it might be a bit annoying when someone asks questions that have obvious answers that are posted in what we think are obvious places.  but maybe we just need to make them more obvious.  obvious != duplicated

[mbat...@gmail.com]

I agree with mwall.  Very astute and succinct.

[Vince Skahan]

On Friday, April 23, 2021 at 4:02:07 AM UTC-7 mwall wrote:

i'm pretty settled about (1) - like tom said, if a user does setup.py, that user must at least understand PATH.

if someone does not understand PATH, just refer him/her to the PATH wiki page.

I'd add use of sudo to that as well.   Perhaps a rethink of the docs for those two items or a minimal 'new to Linux?' page might be helpful...