Detect Linux Runlevel

[Wildman]

I there a way to detect what the Linux runlevel is from
within a Python program? I would like to be able to do
it without the use of an external program such as 'who'
or 'runlevel'.

--
<Wildman> GNU/Linux user #557453
The cow died so I don't need your bull!

[DFS]

On 12/05/2016 03:58 PM, Wildman wrote:
> I there a way to detect what the Linux runlevel is from
> within a Python program? I would like to be able to do
> it without the use of an external program such as 'who'
> or 'runlevel'.

Why not?

'>>> import os
'>>> os.system("systemctl get-default")
graphical.target



systemd 'graphical.target' corresponds to the old runlevel 5.

[Wildman]

Thanks but I knew about systemctl. As I already said my
goal is to do it without the use of an external program.

[Lew Pitcher]

On Monday December 5 2016 16:25, in comp.lang.python, "DFS" <nos...@dfs.com>
wrote:

> On 12/05/2016 03:58 PM, Wildman wrote:
>> I there a way to detect what the Linux runlevel is from
>> within a Python program? I would like to be able to do
>> it without the use of an external program such as 'who'
>> or 'runlevel'.
>
>
> Why not?
>
> '>>> import os
> '>>> os.system("systemctl get-default")
> graphical.target

Because

$ cat rlevel.py

import os
os.system("systemctl get-default")

16:36 $ python rlevel.py
sh: systemctl: command not found
16:36 $

> systemd 'graphical.target' corresponds to the old runlevel 5.

Yes? So?

The OP asked for the runlevel, not the systemd target.


--
Lew Pitcher
"In Skills, We Trust"
PGP public key available upon request

[Marko Rauhamaa]

Wildman <best...@yahoo.com>:

> Thanks but I knew about systemctl. As I already said my goal is to do
> it without the use of an external program.

Inspect:

<URL:
https://github.com/systemd/systemd/blob/master/src/systemctl/systemctl.c>

In particular:

get_state_one_unit()

Then, proceed to:

<URL: https://dbus.freedesktop.org/doc/dbus-python/doc/tutorial.html>


Marko

[Chris Angelico]

On Tue, Dec 6, 2016 at 8:38 AM, Lew Pitcher
<lew.p...@digitalfreehold.ca> wrote:
> The OP asked for the runlevel, not the systemd target.

Runlevels don't exist in systemd. And systemd targets don't exist in
Upstart. The question "what runlevel are we in" does not make sense
unless you're using an init system that works on the basis of
runlevels (eg sysvinit).

ChrisA

[Tim Chase]

On 2016-12-05 14:58, Wildman via Python-list wrote:
> I there a way to detect what the Linux runlevel is from
> within a Python program? I would like to be able to do
> it without the use of an external program such as 'who'
> or 'runlevel'.

You can use something like

https://gist.github.com/likexian/f9da722585036d372dca

to parse the /var/run/utmp contents. Based on some source-code
scrounging, it looks like you want the first field to be "1" for the
"runlevel" account. To extract the actual runlevel, you can take
the PID value from the second column ("53" in my example here) and
take it's integer value mod 256 (AKA "& 0xff") to get the character
value. So chr(int("53") & 0xff) returns "5" in my case, which is my
runlevel.

Additional links I found helpful while searching:

https://casper.berkeley.edu/svn/trunk/roach/sw/busybox-1.10.1/miscutils/runlevel.c
https://github.com/garabik/python-utmp

-tkc

[Marko Rauhamaa]

Chris Angelico <ros...@gmail.com>:

In fact, systemd is not an init system for Linux. Linux is the kernel of
the systemd operating system. Systemd is the

One Ring to rule them all, One Ring to find them,
One Ring to bring them all and in the darkness bind them


Marko

[Wildman]

On Mon, 05 Dec 2016 23:59:48 +0200, Marko Rauhamaa wrote:

> Wildman <best...@yahoo.com>:
>> Thanks but I knew about systemctl. As I already said my goal is to do
>> it without the use of an external program.
>
> Inspect:
>
> <URL:
> https://github.com/systemd/systemd/blob/master/src/systemctl/systemctl.c>
>
> In particular:
>
> get_state_one_unit()

Too bad I don't speak C. I am an amateur programmer and most or all
my experience has been with assembly and various flavors of BASIC,
including VB and PowerBASIC. I did look over the code but I guess
I'm just a rebel without a clue. <g>

> Then, proceed to:
>
> <URL: https://dbus.freedesktop.org/doc/dbus-python/doc/tutorial.html>
>
>
> Marko

Even if I could ‎figure this out, I can't depend on systemd being
installed. The way things are going that may change tho. I have
bookmarked the pages for possible future use.

Thank you, I do appreciate your post.

[Michael Torrie]

On 12/05/2016 03:34 PM, Wildman via Python-list wrote:
> Too bad I don't speak C. I am an amateur programmer and most or all
> my experience has been with assembly and various flavors of BASIC,
> including VB and PowerBASIC. I did look over the code but I guess
> I'm just a rebel without a clue. <g>

May I ask what you are trying to accomplish? Why does the runlevel matter?

[Michael Torrie]

Well I for one am glad of the systemd init system inside of my Linux
operating system. I would far rather deal with service ini files than
long arcane bash scripts that often re-implement (poorly) things like
pid files, and attempts to prevent more than one instance from running.
You were the one that posted earlier today about the many perils of
programming complicated scripts in bash. Init scripts pretty much hit on
all of those gotchas! It boggles my mind that people would actually
argue in favor of maintaining such things (and yes I have done it on
many servers over the years and it was always a pain to deal with custom
daemons).

I don't use very many of the systemd modules (most are not even
installed on my machine), nor will I ever need them. In fact I still
have rsyslog running.

[Lew Pitcher]

On Monday December 5 2016 17:24, in comp.lang.python, "Chris Angelico"

I repeat: The OP asked for the runlevel, not the systemd target.

That should tell you that an answer involving systemd "does not make sense".

To the OP: as others have said, the file
/var/run/utmp
contains various records, including the RUN_LVL (runlevel) record. You can
find some documentation in utmp(3), including a record layout, and an values
list.

I don't know that python includes a standard or builtin method to parse the
utmp file; to retrieve the runlevel, you may have to code your own access
routine as part of your python code, or resort to invoking an external
program like who(1).

HTH

[Wildman]

Of course you may. It is part of a program I'm working on
that reports different information about the Linux system it
is running on. A program similar to inxi. And I am trying
to write it without using external programs, where possible.

I am a hobby programmer and I've been trying to learn python
for a few months now. The program is 'worthlessware' but it
is a 'learning experience' for me. A friend wrote a similar
program in Bash script and I have been working on translating
it to Python.

[Marko Rauhamaa]

Michael Torrie <tor...@gmail.com>:

> On 12/05/2016 03:29 PM, Marko Rauhamaa wrote:
>> In fact, systemd is not an init system for Linux. Linux is the kernel of
>> the systemd operating system. Systemd is the
>>
>> One Ring to rule them all, One Ring to find them,
>> One Ring to bring them all and in the darkness bind them
>
> Well I for one am glad of the systemd init system inside of my Linux
> operating system. I would far rather deal with service ini files than
> long arcane bash scripts that often re-implement (poorly) things like
> pid files, and attempts to prevent more than one instance from
> running.

The situation before systemd *was* atrocious. Most of the protests
against systemd were misplaced. They advocated the ancient hacker
culture that placed the supreme authority on the System Administrator,
who fashioned the system into his own image.

I appreciate that finally there was a bold soul who took the point of
view of an Architect and laid down some higher-level principles for the
whole system to abide by.

Unfortunately, I am not wholly impressed by the end result. Mogadishu
has been replaced by Pyongyang. Some age-old Unix principles have been
abandoned without clear justification. For example, I was appalled to
find out that a systemd unit can be configured to react on the exit
status of a child process of a daemon.

Also, now D-Bus is a fixture for every daemon writer.

> You were the one that posted earlier today about the many perils of
> programming complicated scripts in bash. Init scripts pretty much hit on
> all of those gotchas!

Guess what -- the unit files come with their own gotchas. For example,
there is no simple way to convert an arbitrary pathname into an
ExecStart entry of a unit file.

The .ini file format was a lousy choice. Why not choose JSON in this day
and age?

Yes, the SysV init system had become a jungle. Still, I wish it had been
replaced with a rigorous protocol. The units should have been programs
that comply with given contracts.


Marko

[Michael Torrie]

On 12/05/2016 04:37 PM, Marko Rauhamaa wrote:
> Unfortunately, I am not wholly impressed by the end result. Mogadishu
> has been replaced by Pyongyang. Some age-old Unix principles have been
> abandoned without clear justification. For example, I was appalled to
> find out that a systemd unit can be configured to react on the exit
> status of a child process of a daemon.

I have yet to see any evidence of this Pyonguang situation.

What is wrong with reacting to the exit status of a daemon's child process?

> Also, now D-Bus is a fixture for every daemon writer.

But not directly. It need not be a dependency on the part of the daemon
writer. In fact the daemon writer may now leave out deamonizing code
entirely if he or she wishes. The systemd api is entirely optional for
the daemon to use.

> Guess what -- the unit files come with their own gotchas. For example,
> there is no simple way to convert an arbitrary pathname into an
> ExecStart entry of a unit file.
>
> The .ini file format was a lousy choice. Why not choose JSON in this day
> and age?

But why json? ini files are at least fairly human write-able and
read-able. Json is great, but not for this application. Thank goodness
they didn't choose xml. I never liked working with LaunchDaemon's plist
files or Solaris' service definition files.

> Yes, the SysV init system had become a jungle. Still, I wish it had been
> replaced with a rigorous protocol. The units should have been programs
> that comply with given contracts.

Yes there is certainly merit to this idea, if anyone could agree on a
protocol.

[Bernd Nawothnig]

On 2016-12-05, Wildman wrote:
> And I am trying to write it without using external programs, where
> possible.

That is not the Unix way.

> I am a hobby programmer and I've been trying to learn python
> for a few months now. The program is 'worthlessware' but it
> is a 'learning experience' for me.

It looks for me like a worthless learning experience.

> A friend wrote a similar program in Bash script and I have been
> working on translating it to Python.

Stay with shell script for such tasks. It is never a good idea to
choose the programming language before closer evaluating the problem.




Bernd

--
No time toulouse

[Wildman]

That is exactly the kind of thing I was looking for. Thank you.
Now all I have to do is get it to work with Python3.

[Michael Torrie]

> choose the programming language before closer evaluating the problem.I

I think Python is a good choice for such a utility, but I agree it is
much better to rely on these external utilities as children to do the
platform-dependent work, rather than try to re-implement everything in
Python. A long time ago I wrote a simple wrapper to Popen that would
run a command and return the standard out and standard error to me.

[Steve D'Aprano]

On Tue, 6 Dec 2016 11:08 am, Michael Torrie wrote about systemd:

> I have yet to see any evidence of this Pyonguang situation.

Let me guess... you're running a single-user Linux box?

Fortunately, I've managed to avoid needing to personally interact with
systemd at all. But over the last year or so, I've had to listen to a
continual chorus of complaints from the sys admins I work with as they
struggle to adapt our code to the Brave New World of systemd.

Let me put it this way: one of our techs took it upon himself to migrate our
Python code base from Python 2.6 to 3.4, some tens of thousands of lines.
It took him half of one afternoon.

In comparison, migrating to systemd has given us nothing but headaches, and
invariably when we try asking for help on the main systemd IRC channel
we're told that we're wrong for wanting to do what we want to do.

Not just "systemd can't do that", but "you shouldn't do that".

Why not? We used to do it, and it is necessary for our application.

"Because its wrong."




--
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

[Nathan Ernst]

OT, but I'm curious, do they explain *why* it's wrong and give an
alternative, or just outright deride it as "the wrong way". I ask because
I've read similar complaints about the community around systemd, but as it
rarely affects me personally, I've never bothered to care.

On Mon, Dec 5, 2016 at 8:48 PM, Steve D'Aprano <steve+...@pearwood.info>
wrote:

> --
> https://mail.python.org/mailman/listinfo/python-list
>

[Wildman]

On Mon, 05 Dec 2016 18:25:58 -0700, Michael Torrie wrote:

> I think Python is a good choice for such a utility, but I agree it is
> much better to rely on these external utilities as children to do the
> platform-dependent work, rather than try to re-implement everything in
> Python. A long time ago I wrote a simple wrapper to Popen that would
> run a command and return the standard out and standard error to me.

My rational is that all Linux distros are not created equal.
One comes with one certain set of utilities and another can
have different ones. I can't always depend on a given
utility being there. And there is not way to know the
names of same utility across all distros. This is especially
a problem when comparing .rpm with .deb based distros.

In cases where I have to use an external program, I mean in
cases where there is no apparent Python solution, I check
for the existence of that program and just skip over that
section of the code if it is not found.

BTW here is a link for the Bash script I mentioned in case
you would like to take a look at it. The guy who wrote
it had only been learning Bash for a few months. Not bad.
I have tried to tweak an interest in him for Python but
he sticks with Bash. He says that is the best language
for programming on Linux and he is not interested in GUI
programming.

https://github.com/marek-novotny/linfo

[Michael Torrie]

On 12/05/2016 07:48 PM, Steve D'Aprano wrote:
> On Tue, 6 Dec 2016 11:08 am, Michael Torrie wrote about systemd:
>
>> I have yet to see any evidence of this Pyonguang situation.
>
> Let me guess... you're running a single-user Linux box?

No I've done it on servers that weren't single-user (mail and web
servers in particular). Not sure what you're getting at there, or why
that's relevant. All Linux machines are set up and run as multi-user
boxes anyway. Even my laptop that only I use.

> Fortunately, I've managed to avoid needing to personally interact with
> systemd at all. But over the last year or so, I've had to listen to a
> continual chorus of complaints from the sys admins I work with as they
> struggle to adapt our code to the Brave New World of systemd.
>
> Let me put it this way: one of our techs took it upon himself to migrate our
> Python code base from Python 2.6 to 3.4, some tens of thousands of lines.
> It took him half of one afternoon.
>
> In comparison, migrating to systemd has given us nothing but headaches, and
> invariably when we try asking for help on the main systemd IRC channel
> we're told that we're wrong for wanting to do what we want to do.

Well since I have no idea what you were trying to do I can't comment.

> Not just "systemd can't do that", but "you shouldn't do that".
>
> Why not? We used to do it, and it is necessary for our application.

That sounds frustrating, but of course I've heard similar stories from
folks moving to Python 3. :)

> "Because its wrong."

I've yet to encounter any of those kind of problems your admins
apparently had. Also systemd in no way prevents you from using init
scripts or even inetd services if you choose, so there's always a
fallback position.

I'll have to specifically ask my friend who works for Bluehost about any
systemd troubles next time I speak with her.

[Tim Chase]

On 2016-12-05 18:26, Wildman via Python-list wrote:
> On Mon, 05 Dec 2016 16:08:57 -0600, Tim Chase wrote:
>
> > On 2016-12-05 14:58, Wildman via Python-list wrote:
> >> I there a way to detect what the Linux runlevel is from
> >> within a Python program? I would like to be able to do
> >> it without the use of an external program such as 'who'
> >> or 'runlevel'.
> >
> > You can use something like
> >
> > https://gist.github.com/likexian/f9da722585036d372dca
> >
> > to parse the /var/run/utmp contents. Based on some source-code
> > scrounging, it looks like you want the first field to be "1" for
> > the "runlevel" account. To extract the actual runlevel, you can
> > take the PID value from the second column ("53" in my example
> > here) and take it's integer value mod 256 (AKA "& 0xff") to get
> > the character value. So chr(int("53") & 0xff) returns "5" in my
> > case, which is my runlevel.
> >
> > Additional links I found helpful while searching:
> >
> > https://casper.berkeley.edu/svn/trunk/roach/sw/busybox-1.10.1/miscutils/runlevel.c
> > https://github.com/garabik/python-utmp
>

> That is exactly the kind of thing I was looking for. Thank you.
> Now all I have to do is get it to work with Python3.

This works based on my poking at it in both Py2 and Py3:

import struct
from collections import namedtuple

try:
basestring
except NameError:
basestring = str

UTMP = namedtuple("UTMP", [
"ut_type", # Type of record
"ut_pid", # PID of login process
"ut_line", # Device name of tty - "/dev/"
"ut_id", # Terminal name suffix, or inittab(5) ID
"ut_user", # Username
"ut_host", # Hostname for remote login, or kernel version for run-level messages
"e_termination", # Process termination status
"e_exit", # Process exit status
"ut_session", # Session ID (getsid(2)), used for windowing
"tv_sec", # Seconds
"tv_usec", # Microseconds
"ut_addr_v6a", # Internet address of remote host; IPv4 address uses just ut_addr_v6[0]
"ut_addr_v6b", # Internet address of remote host; IPv4 address uses just ut_addr_v6[0]
"ut_addr_v6c", # Internet address of remote host; IPv4 address uses just ut_addr_v6[0]
"ut_addr_v6d", # Internet address of remote host; IPv4 address uses just ut_addr_v6[0]
#"__unused", # Reserved for future use
])

XTMP_STRUCT = "hi32s4s32s256shhiiiiiii20x"
XTMP_STRUCT_SIZE = struct.calcsize(XTMP_STRUCT)

# ut_types
EMPTY = 0
RUN_LVL = 1
BOOT_TIME = 2
OLD_TIME = 3
NEW_TIME = 4
INIT_PROCESS = 5 # Process spawned by "init"
LOGIN_PROCESS = 6 # A "getty" process

DEFAULT_UTMP = "/var/run/utmp"

def parse_utmp(utmp_fname=DEFAULT_UTMP):
with open(utmp_fname, "rb") as f:
while True:
bytes = f.read(XTMP_STRUCT_SIZE)
if not bytes:
break
bits = struct.unpack(XTMP_STRUCT, bytes)
bits = [
bit.rstrip('\0') if isinstance(bit, basestring) else bit
for bit
in bits
]
yield UTMP(*bits)

def filter(ut_type, utmp_fname=DEFAULT_UTMP):
for utmp in parse_utmp(utmp_fname):
if utmp.ut_type == ut_type:
yield utmp

def get_runlevel(utmp_fname=DEFAULT_UTMP):
return chr(next(filter(RUN_LVL, utmp_fname)).ut_pid & 0xFF)

if __name__ == "__main__":
print("Runlevel: %s" % get_runlevel())


-tkc

[Michael Torrie]

On 12/05/2016 08:27 PM, Wildman via Python-list wrote:
> On Mon, 05 Dec 2016 18:25:58 -0700, Michael Torrie wrote:
>
>> I think Python is a good choice for such a utility, but I agree it is
>> much better to rely on these external utilities as children to do the
>> platform-dependent work, rather than try to re-implement everything in
>> Python. A long time ago I wrote a simple wrapper to Popen that would
>> run a command and return the standard out and standard error to me.
>
> My rational is that all Linux distros are not created equal.
> One comes with one certain set of utilities and another can
> have different ones. I can't always depend on a given
> utility being there. And there is not way to know the
> names of same utility across all distros. This is especially
> a problem when comparing .rpm with .deb based distros.

Well this is a problem regardless of which scripting language you choose
and the solutions will be the same.

> In cases where I have to use an external program, I mean in
> cases where there is no apparent Python solution, I check
> for the existence of that program and just skip over that
> section of the code if it is not found.

Sure. That's probably reasonable.

>
> BTW here is a link for the Bash script I mentioned in case
> you would like to take a look at it. The guy who wrote
> it had only been learning Bash for a few months. Not bad.
> I have tried to tweak an interest in him for Python but
> he sticks with Bash. He says that is the best language
> for programming on Linux and he is not interested in GUI
> programming.
>
> https://github.com/marek-novotny/linfo

That all depends on what he's programming. For anything a user interacts
with, Bash is a pretty poor tool. But Bash is really good at
shell-scripting system tasks. It treats external commands as
first-class entities, and process control and I/O piping is integrated
into the syntax of the language. On the other hand, Python is a good
language but it's not particularly well suited to shell scripting
(wasn't designed for that purpose), though it does have some facilities
like generators that make certain forms of system programming really
slick. In short they overlap in purpose, but they aren't good at the
same things. However when a bash script gets too long, I'll often
switch to Python, using my run wrapper and generator filters to process
the output of external commands.

Personally for a script of this type, I'd probably stick with Bash
myself. Which by the way is what inxi is written in.

[Wildman]

On Mon, 05 Dec 2016 20:46:22 -0700, Michael Torrie wrote:

> On 12/05/2016 08:27 PM, Wildman via Python-list wrote:
>> On Mon, 05 Dec 2016 18:25:58 -0700, Michael Torrie wrote:
>>
>>> I think Python is a good choice for such a utility, but I agree it is
>>> much better to rely on these external utilities as children to do the
>>> platform-dependent work, rather than try to re-implement everything in
>>> Python. A long time ago I wrote a simple wrapper to Popen that would
>>> run a command and return the standard out and standard error to me.
>>
>> My rational is that all Linux distros are not created equal.
>> One comes with one certain set of utilities and another can
>> have different ones. I can't always depend on a given
>> utility being there. And there is not way to know the
>> names of same utility across all distros. This is especially
>> a problem when comparing .rpm with .deb based distros.
>
> Well this is a problem regardless of which scripting language you choose
> and the solutions will be the same.

It is a problem only if you depend on the utility.

> Personally for a script of this type, I'd probably stick with Bash
> myself.

In most cases I would agree with that, but, in this case my
goal is learning Python. I did write a couple of programs
with Bash several months ago to learn a little about it.
One will take an image and convert it into an X-Face header
and the other will take an image and convert it into a Face
header. I later wrote GUI versions of the programs with
Python and Tkinter.

BTW, I don't depend on programming for a living. I would be
in bad shape if I did. It is a hobby that I greatly enjoy.
And, being in my later years, it keeps my mind sharp(er).

> Which by the way is what inxi is written in.

Yes, I was aware of that. It's over 12,000 lines!

[Wildman]

On Mon, 05 Dec 2016 21:42:52 -0600, Tim Chase wrote:

> On 2016-12-05 18:26, Wildman via Python-list wrote:
>> On Mon, 05 Dec 2016 16:08:57 -0600, Tim Chase wrote:
>>
>> > On 2016-12-05 14:58, Wildman via Python-list wrote:
>> >> I there a way to detect what the Linux runlevel is from
>> >> within a Python program? I would like to be able to do
>> >> it without the use of an external program such as 'who'
>> >> or 'runlevel'.
>> >
>> > You can use something like
>> >
>> > https://gist.github.com/likexian/f9da722585036d372dca
>> >
>> > to parse the /var/run/utmp contents. Based on some source-code
>> > scrounging, it looks like you want the first field to be "1" for
>> > the "runlevel" account. To extract the actual runlevel, you can
>> > take the PID value from the second column ("53" in my example
>> > here) and take it's integer value mod 256 (AKA "& 0xff") to get
>> > the character value. So chr(int("53") & 0xff) returns "5" in my
>> > case, which is my runlevel.
>> >
>> > Additional links I found helpful while searching:
>> >
>> > https://casper.berkeley.edu/svn/trunk/roach/sw/busybox-1.10.1/miscutils/runlevel.c
>> > https://github.com/garabik/python-utmp
>>
>> That is exactly the kind of thing I was looking for. Thank you.
>> Now all I have to do is get it to work with Python3.
>
> This works based on my poking at it in both Py2 and Py3:

That works perfectly. I owe you a big thanks. That was a
lot of work and time on your part. I really appreciate it.

[Marko Rauhamaa]

Michael Torrie <tor...@gmail.com>:

> On 12/05/2016 04:37 PM, Marko Rauhamaa wrote:
>> Unfortunately, I am not wholly impressed by the end result. Mogadishu
>> has been replaced by Pyongyang. Some age-old Unix principles have been
>> abandoned without clear justification. For example, I was appalled to
>> find out that a systemd unit can be configured to react on the exit
>> status of a child process of a daemon.
>
> I have yet to see any evidence of this Pyonguang situation.
>
> What is wrong with reacting to the exit status of a daemon's child
> process?

A basic black-box principle is violated. It is surprising, at least.

If I launch a child process, wait it out and ignore its exit status
code, I would think the exit status is meaningless. Not so, because the
Eye is watching.

>> Also, now D-Bus is a fixture for every daemon writer.
>
> But not directly. It need not be a dependency on the part of the
> daemon writer. In fact the daemon writer may now leave out deamonizing
> code entirely if he or she wishes. The systemd api is entirely
> optional for the daemon to use.

Systemd comes with dozens of legacy modes, and it is difficult to learn
what is systemd's "native" daemon type. However, it is evident that
"Type=notify" is it, meaning the daemon communicates with systemd
explicitly (<URL:
https://www.freedesktop.org/software/systemd/man/sd_notify.html>).

That is not necessarily a bad idea. Daemons have traditionally been
lousy at communicating their statuses appropriately. It's just that this
major requirement should be declared openly.

>> The .ini file format was a lousy choice. Why not choose JSON in this
>> day and age?
>
> But why json?

It comes with simple, rigorous, universal syntax. The .ini files don't.
That's why the unit files have brittle ad-hoc syntax.


Marko

[Tim Chase]

It was pretty straightforward to map it into a namedtuple using the
links I provided. The only killer for me was that the struct module
doesn't return an entry for "padding" bytes. Which makes sense, but
threw me off for a good while as I tried to figure why my
tuple-creation was complaining about a missing parameter (fixed by
commenting out the "__unused" member of the namedtuple; could also
have kept it while changing the "20x" to "20c" in the struct
format-string)

The rest was just basic text manipulation in vim to convert the C
structs into Python code.

And despite what Bernd Nawothnig wrote:

> It looks for me like a worthless learning experience.

it gave me the opportunity to learn about the internals of the utmp
format, something that has long been a curiosity ("it's a binary log,
not text, I wonder what all is in there? But I don't have real cause
to go poking around in there to learn the answer right now.") and this
gave me the push I needed to explore that.

Glad it helped.

-tkc

[Tim Chase]

On 2016-12-06 01:14, Bernd Nawothnig wrote:
> > I am a hobby programmer and I've been trying to learn python
> > for a few months now. The program is 'worthlessware' but it
> > is a 'learning experience' for me.
>
> It looks for me like a worthless learning experience.

Eh, one person's "worthless learning experience" is another person's
curiosity-satisfying education. I found it an exercise in learning
something new about the utmp log format (previously an opaque binary
blob)

> > A friend wrote a similar program in Bash script and I have been
> > working on translating it to Python.
>
> Stay with shell script for such tasks. It is never a good idea to
> choose the programming language before closer evaluating the
> problem.

Based on the OP's description, this is a small part of a much larger
program. And I would personally rather maintain a large Python
code-base than a large Bash code-base.

-tkc

[Wildman]

On Tue, 06 Dec 2016 01:14:35 +0100, Bernd Nawothnig wrote:

> On 2016-12-05, Wildman wrote:
>> And I am trying to write it without using external programs, where
>> possible.
>
> That is not the Unix way.

Yes, but it is my way.

>> I am a hobby programmer and I've been trying to learn python
>> for a few months now. The program is 'worthlessware' but it
>> is a 'learning experience' for me.
>
> It looks for me like a worthless learning experience.

It is sad that you consider learning something new to
be worthless. I used the term "worthlessware" in an
economical sense, meaning it has little or no commercial
value. However, from a learning standpoint I consider
it to be priceless.

>> A friend wrote a similar program in Bash script and I have been
>> working on translating it to Python.
>
> Stay with shell script for such tasks. It is never a good idea to
> choose the programming language before closer evaluating the problem.

You have a right to your opinion but I fail to see what
that has to do with the price of eggs. I picked Python
just because I wanted to learn it not because I had a
particular problem to solve. If your job was to advocate
Python, I would suggest you find another line of work.

[Michael Torrie]

On 12/06/2016 06:51 AM, Tim Chase wrote:
> Based on the OP's description, this is a small part of a much larger
> program. And I would personally rather maintain a large Python
> code-base than a large Bash code-base.

Absolutely. Especially when you consider inxi is 12,000 lines of bash
code in one file. Shudder. Though I'm sure bash made it very easy to
interact with dozens or even hundreds of different external programs on
dozens of operating systems and distributions to gather information.

[Michael Torrie]

On 12/06/2016 09:18 AM, Wildman via Python-list wrote:
> It is sad that you consider learning something new to
> be worthless. I used the term "worthlessware" in an
> economical sense, meaning it has little or no commercial
> value. However, from a learning standpoint I consider
> it to be priceless.

Well said. Some of us get hung up so much on the proper way to do
something that we end up not doing much at all, other than talk about
the proper way to do things. I tend to have this problem. While I
talked about the proper and theoretical ways of doing agricultural GPS
coverage mapping, another person with less formal programming training
than I started hacking and you know what? He has a functioning program
now that actually works. It's in C# (which I don't love), and it's got
rough spots in the code and it's a bit difficult to add certain features
to, but he simply went and did and learned as he went. Now he's at the
point where he could refactor (and do it quickly) to get the
architecture a bit more robust. But the point is I wasted all my time
thinking about how I might do it and he just did it. Was very
instructive to me.

>>> A friend wrote a similar program in Bash script and I have been
>>> working on translating it to Python.
>>
>> Stay with shell script for such tasks. It is never a good idea to
>> choose the programming language before closer evaluating the problem.
>
> You have a right to your opinion but I fail to see what
> that has to do with the price of eggs. I picked Python
> just because I wanted to learn it not because I had a
> particular problem to solve. If your job was to advocate
> Python, I would suggest you find another line of work.

I appreciate your measured response to what could be seen as an
inflammatory post.

Sometimes I think it all depends on the purpose for which you do
something. In this case it's for fun, so knock yourself out. If you were
instead writing this as part of a requirement for some enterprise server
process professionally, you'd probably want to stick with Bash rather
than shoe-horn Python into a systems programming language and
shell-scripting language, which it's not really that good at. I can say
this given my professional experience with server shell scripting.

[Wildman]

On Mon, 05 Dec 2016 16:08:57 -0600, Tim Chase wrote:

> On 2016-12-05 14:58, Wildman via Python-list wrote:
>> I there a way to detect what the Linux runlevel is from
>> within a Python program? I would like to be able to do
>> it without the use of an external program such as 'who'
>> or 'runlevel'.
>
> You can use something like
>
> https://gist.github.com/likexian/f9da722585036d372dca

I went back to the code from the above link to try to
get it to work with Python3, just to see if I could.
The problem was that the output was in bytes or partly
in bytes like this:

['1', '53', "b'~\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\
x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\ (...)

I was trying to convert the bytes to strings and that
is what I never got to work right. It didn't occur
to me that all I needed was the first two fields and
they were already strings.

The 'print output' part of the original code was this
which printed everything. Over kill for my use:

data = read_xtmp('/var/run/utmp')
for i in data:
print i

I changed it to this and it works:

data = read_xtmp('/var/run/utmp')
for i in data:
if i[0] == "1":
print("Runlevel: " + chr(int(i[1]) & 0xFF))
break

The output: Runlevel: 5

If I had tried this in the beginning, it would have
save you a lot of work.

Since both versions of the code works, which one do
you recommend? Or does it matter?

[Tim Chase]

On 2016-12-06 12:10, Wildman via Python-list wrote:

> If I had tried this in the beginning, it would have
> save you a lot of work.
>
> Since both versions of the code works, which one do
> you recommend? Or does it matter?

Heh, I'm not sure it matters much. The code I provided should be
expandable for tidily handling other entries in utmp, allowing you to
search for other system events that might interest you (check out the
invocation of the filter() function which filters by type).

Since I've already done the leg-work, I mildly advocate using my
version since it should be pretty clean and easy to expand. But if
you want the satisfaction of using your code, I won't take offense :-)

-tkc

[Wildman]

On Tue, 06 Dec 2016 13:06:35 -0600, Tim Chase wrote:

> On 2016-12-06 12:10, Wildman via Python-list wrote:
>> If I had tried this in the beginning, it would have
>> save you a lot of work.
>>
>> Since both versions of the code works, which one do
>> you recommend? Or does it matter?
>
> Heh, I'm not sure it matters much. The code I provided should be
> expandable for tidily handling other entries in utmp, allowing you to
> search for other system events that might interest you (check out the
> invocation of the filter() function which filters by type).

Yes, your code is very expandable and I will keep it
archived for that reason. In the future I might want
to delve further into utmp. But right now all I need
is the runlevel.

> Since I've already done the leg-work, I mildly advocate using my
> version since it should be pretty clean and easy to expand. But if
> you want the satisfaction of using your code, I won't take offense :-)
>
> -tkc

Yes, I think I will use my code. No offense intended. :-)
Again, I do appreciate all your work on my behalf. I hope
some day I can return the favor.

[Marko Rauhamaa]

Tim Chase <pytho...@tim.thechases.com>:

> This works based on my poking at it in both Py2 and Py3:

Great info, Tim.

A word a warning: your code doesn't lock /var/run/utmp before access,
which is a race condition. The file may be updated at any time, and
ordinary file reads may yield corrupted records.

The library functions getutline(), pututline() etc lock the file
internally (although that is not documented). You can see the (advisory
record) locking in action by running the command

strace who


Since the locking scheme hasn't been documented, the only safe way to
read /var/run/utmp is through the C API functions.

Another thing is that, as stated before, the runlevel business is
legacy. It is still supported by systemd-update-utmp, but for how long
is anybody's guess.


Marko

[Michael Torrie]

On 12/06/2016 03:29 PM, Marko Rauhamaa wrote:
> Another thing is that, as stated before, the runlevel business is
> legacy. It is still supported by systemd-update-utmp, but for how long
> is anybody's guess.

System V compatibility is still important to Linux, and as long as it
is, something resembling a runlevel has to be provided, even it's just
an abstraction of something else like a systemd target. So any system
that advertises System V compliance or compatibility will have a runlevel.

[Tim Chase]

On 2016-12-07 00:29, Marko Rauhamaa wrote:
> Tim Chase <pytho...@tim.thechases.com>:
>
> > This works based on my poking at it in both Py2 and Py3:
>
> Great info, Tim.
>
> A word a warning: your code doesn't lock /var/run/utmp before
> access, which is a race condition. The file may be updated at any
> time, and ordinary file reads may yield corrupted records.

Since the code is reading in record-sized blocks and never writing,
I'm not sure how much possibility there is for a race condition. At
worst, I imagine that it would result in reading the old data which
isn't a horrible condition.

For under a certain block-size (PIPE_BUF, which I think used to be
the minimum POSIX requirement of 512b, but is now 4096b on Linux),
*nix operating systems were atomic in their reading and writing. So
as long as the writer is writing a record at a time (or less) and not
issuing multiple writes to update disjoint parts of the record, I'm
pretty confident that atomicity won't be an issue for all intents and
purposes.

http://pubs.opengroup.org/onlinepubs/009695399/functions/write.html#tag_03_866_08

(though according to my "man 2 write" on my Linux box, before Linux
3.14, it wasn't atomic according to specs)

> The library functions getutline(), pututline() etc lock the file
> internally (although that is not documented). You can see the
> (advisory record) locking in action by running the command

C API functions which don't appear to be exposed in stock Python. ;-)

> Another thing is that, as stated before, the runlevel business is
> legacy. It is still supported by systemd-update-utmp, but for how
> long is anybody's guess.

Much like Unix itself, if the OP chooses to shoot off his own feet
with them, my aim is to do it efficiently as requested. ;-)

-tkc

[Wildman]

On Tue, 06 Dec 2016 09:45:05 -0700, Michael Torrie wrote:

> I appreciate your measured response to what could be seen as an
> inflammatory post.

It was inflammatory and I considered a different response but
after the knee jerking, I give it some thought and decided
otherwise. The simple fact is I'm an outsider here. Over
the last few months I have received some good advice and help
from some good folks and for that I am grateful. I do not
want to do anything that could jeopardize that. So I will
try my best to keep my posts civil.

And I thank you for your words.

[Wildman]

On Tue, 06 Dec 2016 13:06:35 -0600, Tim Chase wrote:

>

I forgot to mention that I want to include your name in the
final script as a contributor, if that is ok.

You will get a cut of the royalties. Lets see, how much is
20% of $0.00? Well, I'll let my account work that out as
soon as she gets home from the grocery.

--
<Wildman> GNU/Linux user #557453

The voices in my head may not be real
but they have some good ideas.

[Tim Chase]

On 2016-12-06 21:27, Wildman via Python-list wrote:
> On Tue, 06 Dec 2016 13:06:35 -0600, Tim Chase wrote:
> I forgot to mention that I want to include your name in the
> final script as a contributor, if that is ok.

No issues here.

> You will get a cut of the royalties. Lets see, how much is
> 20% of $0.00?

I'm not sure I'd settle for less than 25% of $0. ;-)

-tkc

[Marko Rauhamaa]

Tim Chase <pytho...@tim.thechases.com>:

> On 2016-12-07 00:29, Marko Rauhamaa wrote:
>> A word a warning: your code doesn't lock /var/run/utmp before
>> access, which is a race condition. The file may be updated at any
>> time, and ordinary file reads may yield corrupted records.
>
> Since the code is reading in record-sized blocks and never writing,
> I'm not sure how much possibility there is for a race condition. At
> worst, I imagine that it would result in reading the old data which
> isn't a horrible condition.

If you read a full record at an offset, you might be right. However,
your code uses Python's buffered I/O:

with open(utmp_fname, "rb") as f:
while True:
bytes = f.read(XTMP_STRUCT_SIZE)

instead of os.open() and os.read().

> For under a certain block-size (PIPE_BUF, which I think used to be
> the minimum POSIX requirement of 512b, but is now 4096b on Linux),
> *nix operating systems were atomic in their reading and writing.

That particular guarantee is true for pipes and FIFOs only.


Marko

[Marko Rauhamaa]

Michael Torrie <tor...@gmail.com>:

I wonder if Linux still suffers from this local DoS:

[ Now no-one else can log in ]

This is a problem with advisory locking. The fact that anyone can
create an exclusive lock on a file they can only read! Is this
behavior appropriate?

<URL: https://www.redhat.com/archives/linux-security/1996-Novembe
r/msg00026.html>

Didn't try it, but the utmp API seems hopeless in this regard.


Marko

[Steven D'Aprano]

On Wednesday 07 December 2016 18:08, Marko Rauhamaa wrote:

> This is a problem with advisory locking. The fact that anyone can
> create an exclusive lock on a file they can only read! Is this
> behavior appropriate?
>
> <URL: https://www.redhat.com/archives/linux-security/1996-Novembe
> r/msg00026.html>

Whereas if it were mandatory locking, enforced by the OS, it wouldn't be a
problem?

Here's that URL without the indent and word-wrapping:

https://www.redhat.com/archives/linux-security/1996-November/msg00026.html



--
Steven
"Ever since I learned about confirmation bias, I've been seeing
it everywhere." - Jon Ronson

[Marko Rauhamaa]

Steven D'Aprano <steve+comp....@pearwood.info>:

> Whereas if it were mandatory locking, enforced by the OS, it wouldn't
> be a problem?

The point is, the utmp scheme seems to be fundamentally broken. You
can't use a regular file for this kind of communication.


Marko

[Grant Edwards]

There are a few things in Unix that are fundamentally broken and
really just can't be used for the things they are intended for (serial
ports come to mind).

However, that doesn't seem to prevent them from having been used
sucessfully that way way for 40 years. ;)

--
Grant Edwards grant.b.edwards Yow! I KAISER ROLL?!
at What good is a Kaiser Roll
gmail.com without a little COLE SLAW
on the SIDE?

[Marko Rauhamaa]

Grant Edwards <grant.b...@gmail.com>:

> There are a few things in Unix that are fundamentally broken and
> really just can't be used for the things they are intended for (serial
> ports come to mind).
>
> However, that doesn't seem to prevent them from having been used
> sucessfully that way way for 40 years. ;)

Sigh. Those issues give me grief literally every day in the office.


Marko

[Tim Chase]

On 2016-12-07 07:30, Marko Rauhamaa wrote:
> Tim Chase <pytho...@tim.thechases.com>:
> > On 2016-12-07 00:29, Marko Rauhamaa wrote:
> >> A word a warning: your code doesn't lock /var/run/utmp before
> >> access, which is a race condition. The file may be updated at any
> >> time, and ordinary file reads may yield corrupted records.
> >
> > Since the code is reading in record-sized blocks and never
> > writing, I'm not sure how much possibility there is for a race
> > condition. At worst, I imagine that it would result in reading
> > the old data which isn't a horrible condition.
>
> If you read a full record at an offset, you might be right. However,
> your code uses Python's buffered I/O:
>
> with open(utmp_fname, "rb") as f:
> while True:
> bytes = f.read(XTMP_STRUCT_SIZE)
>
> instead of os.open() and os.read().

Interesting. I read up on os.open() and os.read()
https://docs.python.org/2/library/os.html#os.read
but didn't notice anything there clarifying that it was unbuffered
compared to the __builtins__.open() and fp.read() functions.

Could you point me to resources where I can learn more about the
distinctions?

> > For under a certain block-size (PIPE_BUF, which I think used to be
> > the minimum POSIX requirement of 512b, but is now 4096b on Linux),
> > *nix operating systems were atomic in their reading and writing.
>
> That particular guarantee is true for pipes and FIFOs only.

Ah, my error in misreading that as globally applicable. Thanks for
ensuring accuracy.

-tkc

[Marko Rauhamaa]

Tim Chase <pytho...@tim.thechases.com>:

> Interesting. I read up on os.open() and os.read()
> https://docs.python.org/2/library/os.html#os.read
> but didn't notice anything there clarifying that it was unbuffered
> compared to the __builtins__.open() and fp.read() functions.
>
> Could you point me to resources where I can learn more about the
> distinctions?

It is not explained very clearly in the documentation, but the os.*
functions are thin wrappers around the analogous C functions (system
calls or standard library functions).

There is just this allusion:

Note that using the file descriptor directly will bypass the file
object methods, ignoring aspects such as internal buffering of data.

<URL: https://docs.python.org/3/library/os.html?#file-descriptor-op
erations>

The os.* facilities are ultimately documented in the Linux man pages.


File object buffering can be turned off by adding buffering=0 to the
high-level open() builtin function:

When no buffering argument is given, the default buffering policy
works as follows:

* Binary files are buffered in fixed-size chunks; the size of the
buffer is chosen using a heuristic trying to determine the
underlying device’s “block size” and falling back on
io.DEFAULT_BUFFER_SIZE. On many systems, the buffer will typically
be 4096 or 8192 bytes long.

<URL: https://docs.python.org/3/library/functions.html#open>


In general, system programming is best done using system programming
facilities, ie, os.*.


Marko