File (terminal) is in /dev/pts, but nobody has it open???

[Kenny McCormack]

This is on a home Linux system, so there is only one user (me) and I have
root, so there is no issue with access.

I've noticed, on one particular machine, that sometimes there will be a
file, say /dev/pts/6, in /dev/pts, but there is no process running on that
device and nobody has it open. My understanding is that the file should be
purged when the last process that had it open closes it (or exits).

I verify that nobody has it open by doing: sudo fuser /dev/pts/6
and it reports nothing. My general guess as to the problem is that the
process that had it open exited, but whatever background process is
responsible for cleaning up /dev/pts didn't notice. Note also that I
cannot remove the file manually (even as root!)

The effect of this is that that pts is basically "lost", until the next
reboot. I want to know a) How/why this happens and b) If there is anything
I can do to get it back.

Note: This may sound like another, similar, problem, but it is not.
Specifically, I want to assure you that I am quite familiar with what you
might call the opposite problem to this one. That is, when there is no
file in /dev/pts for a given terminal, yet, when you allocate a new one,
you get some other number. This is caused by some process having the file
open, but somehow it gets deleted from /dev/pts even though somebody still
has it open. You can detect (and fix) this problem by doing:

# sudo ls -lsa /proc/*/fd | less

and searching for the word "deleted".

Note that, in this case, you can't use fuser to find the offending process,
because there is no file in /dev/pts to use as the argument to fuser.

--
Trump has normalized hate.

The media has normalized Trump.

[Scott Lurndal]

gaz...@shell.xmission.com (Kenny McCormack) writes:
>This is on a home Linux system, so there is only one user (me) and I have
>root, so there is no issue with access.
>
>I've noticed, on one particular machine, that sometimes there will be a
>file, say /dev/pts/6, in /dev/pts, but there is no process running on that
>device and nobody has it open. My understanding is that the file should be
>purged when the last process that had it open closes it (or exits).

It's not a file, per-se.

/dev/pts is managed by the 'ptsfs' filesystem.

$ mount | grep pts
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,seclabel,gid=5,mode=620,ptmxmode=000)

There is likely a reference to the pty somewhere; try using the 'w' command.

We use thousands of pty's every day with no leakage issues.

[Scott Lurndal]

gaz...@shell.xmission.com (Kenny McCormack) writes:
>This is on a home Linux system, so there is only one user (me) and I have
>root, so there is no issue with access.

More specifically, the "dangling" pts entry is like because
it remains a controlling TTY for some process somewhere. fuser
may or may not find it, but

$ sudo lsof | grep pts

should find it for you.

[Richard Kettlewell]

gaz...@shell.xmission.com (Kenny McCormack) writes:
> This is on a home Linux system, so there is only one user (me) and I have
> root, so there is no issue with access.
>
> I've noticed, on one particular machine, that sometimes there will be a
> file, say /dev/pts/6, in /dev/pts, but there is no process running on that
> device and nobody has it open. My understanding is that the file should be
> purged when the last process that had it open closes it (or exits).
>
> I verify that nobody has it open by doing: sudo fuser /dev/pts/6
> and it reports nothing. My general guess as to the problem is that the
> process that had it open exited, but whatever background process is
> responsible for cleaning up /dev/pts didn't notice. Note also that I
> cannot remove the file manually (even as root!)
>
> The effect of this is that that pts is basically "lost", until the next
> reboot. I want to know a) How/why this happens and b) If there is anything
> I can do to get it back.

The /dev/pts file is removed when all file descriptors on either end of
the pty are closed.

For example, if the pty belongs to an SSH session, then both the command
running inside the SSH session, _and_ the sshd process for that session,
must terminate before the pty is closed.

The sshd’s end of the pty will normally appear as /dev/ptmx rather than
/dev/pts/N, reflecting the pty creation process. I’ve reproduced the
behavior you see by sending SIGSTOP to an sshd and then terminating its
child.

To get the pty back, find the process holding the relevant copy of
/dev/ptmx open and (if it’s not serving some useful purpose) kill it.

There’s insufficient information to speculate why it happens in your
particular environment.

--
https://www.greenend.org.uk/rjk/

[Kaz Kylheku]

Linux is so awfully fragmented that you cannot say anything for sure,
though.

Most people use a vendor specific kernel with patches, and no two of
those are alike even if the upstream version numbers match.

Maybe someone tampered in the pty code in Kenny's kernel so there is a
genuine leak.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal

[Kenny McCormack]

In article <87fszm3...@LkoBDZeT.terraraq.uk>,
Richard Kettlewell <inv...@invalid.invalid> wrote:
...

>The /dev/pts file is removed when all file descriptors on either end of
>the pty are closed.

Yes, that is what is SUPPOSED TO HAPPEN.

But is isn't (always). fuser and lsof both say noone has it open.

I know this is hard for people responding in the newsgroup to accept, but
it doesn't always work the way it is supposed to.

Most of what has been posted so far in the resposes to this thread is stuff
I (obviously) already know.

--
Debating creationists on the topic of evolution is rather like trying to
play chess with a pigeon --- it knocks the pieces over, craps on the
board, and flies back to its flock to claim victory.

[Richard Kettlewell]

gaz...@shell.xmission.com (Kenny McCormack) writes:
> Richard Kettlewell <inv...@invalid.invalid> wrote:
> ...
>>The /dev/pts file is removed when all file descriptors on either end of
>>the pty are closed.
>
> Yes, that is what is SUPPOSED TO HAPPEN.
>
> But is isn't (always). fuser and lsof both say noone has it open.

You said you did ‘sudo fuser /dev/pts/6’, which only checks one end. If
you knew about checking for the other end as well then perhaps you
should have mentioned that, rather than expecting your readers to be
telepathic.

--
https://www.greenend.org.uk/rjk/

[Kenny McCormack]

In article <87a6pu3...@LkoBDZeT.terraraq.uk>,

Richard Kettlewell <inv...@invalid.invalid> wrote:
>gaz...@shell.xmission.com (Kenny McCormack) writes:
>> Richard Kettlewell <inv...@invalid.invalid> wrote:
>> ...
>>>The /dev/pts file is removed when all file descriptors on either end of
>>>the pty are closed.
>>

>> But is isn't (always). fuser and lsof both say noone has it open.

>checking for the other end as well

OK, I tried a few things, but could not figure out how to check for that.
Any ideas?

Lots of processes have /dev/ptmx open; how to know which one is connected
to /dev/pts/6?

--
"Every time Mitt opens his mouth, a swing state gets its wings."

(Should be on a bumper sticker)

[Richard Kettlewell]

gaz...@shell.xmission.com (Kenny McCormack) writes:
> Richard Kettlewell <inv...@invalid.invalid> wrote:
>>gaz...@shell.xmission.com (Kenny McCormack) writes:
>>> Richard Kettlewell <inv...@invalid.invalid> wrote:
>>> ...
>>>>The /dev/pts file is removed when all file descriptors on either end of
>>>>the pty are closed.
>>>
>>> But is isn't (always). fuser and lsof both say noone has it open.
>
>>checking for the other end as well
>
> OK, I tried a few things, but could not figure out how to check for that.
> Any ideas?
>
> Lots of processes have /dev/ptmx open; how to know which one is connected
> to /dev/pts/6?

I don’t know of a way to discover it directly; you will have to look
through all the processes with ptmx and figure out which is the relevant
one from contextual clues. e.g. sshd’s argv, or if there are no child
processes (or only child processes with no tty) that’s a good
candidate. I’d have thought someone who knows how it’s all supposed to
work could figure out this approach quite easily for themselves though.

--
https://www.greenend.org.uk/rjk/

[Zooe]

Le mardi 20 avril 2021, Kenny McCormack a écrit:
> Lots of processes have /dev/ptmx open; how to know which one is connected
> to /dev/pts/6?

For each fd in /proc/$pidmaster/fd/ that points to /dev/ptmx, check the
following (works only on recent kernels):

grep tty-index < /proc/$pidmaster/fdinfo/$fd

Another way to get the information, which is more universal but also is
more difficult to follow, is to make a TIOCGPTN system call on the file
descriptor (from a debugger attached to the process).

[Kenny McCormack]

In article <607eba59$0$32501$426a...@news.free.fr>,
Zooe <zo...@127.0.0.1> wrote:
>Le mardi 20 avril 2021, Kenny McCormack a crit:

>> Lots of processes have /dev/ptmx open; how to know which one is connected
>> to /dev/pts/6?
>
>For each fd in /proc/$pidmaster/fd/ that points to /dev/ptmx, check the
>following (works only on recent kernels):
>
> grep tty-index < /proc/$pidmaster/fdinfo/$fd

Ah, now we are getting somewhere.

Unfortunately, the system that has the problem is running kernel
4.something and does not have tty-index (in the fdinfo/X file). Other
systems here that are runnning kernel 5.something do have it, but "man
proc" doesn't mention it.

I'm guessing that it contains the number of the tty (/dev/pts/X) that this
instance of /dev/ptmx is connected to. Is that correct?

>Another way to get the information, which is more universal but also is
>more difficult to follow, is to make a TIOCGPTN system call on the file
>descriptor (from a debugger attached to the process).

Well, that sounds interesting. And very complicated...
(And time consuming...)

--
The coronavirus is the first thing, in his 74 pathetic years of existence,
that the orange menace has come into contact with, that he couldn't browbeat,
bully, bullshit, bribe, sue, legally harrass, get Daddy to fix, get his
siblings to bail him out of, or, if all else fails, simply wish it away.

[Zooe]

Le mardi 20 avril 2021, Kenny McCormack a écrit:
>> grep tty-index < /proc/$pidmaster/fdinfo/$fd
>
> Ah, now we are getting somewhere.
>
> Unfortunately, the system that has the problem is running kernel
> 4.something and does not have tty-index (in the fdinfo/X file). Other
> systems here that are runnning kernel 5.something do have it, but "man
> proc" doesn't mention it.
>
> I'm guessing that it contains the number of the tty (/dev/pts/X) that this
> instance of /dev/ptmx is connected to. Is that correct?

Answer is yes.

The patch that implements the feature was applied on July 2017.
It is documented here: https://lore.kernel.org/patchwork/patch/810490/

>>Another way to get the information, which is more universal but also is
>>more difficult to follow, is to make a TIOCGPTN system call on the file
>>descriptor (from a debugger attached to the process).
>
> Well, that sounds interesting. And very complicated...
> (And time consuming...)

I once saw a perl script automating this, but I cannot find it anywhere.

[Rainer Weikusat]

gaz...@shell.xmission.com (Kenny McCormack) writes:

[...]

>>Another way to get the information, which is more universal but also is
>>more difficult to follow, is to make a TIOCGPTN system call on the file
>>descriptor (from a debugger attached to the process).
>
> Well, that sounds interesting. And very complicated...
> (And time consuming...)

It's really quite simple. First, the numeric value of TIOCGPTN needs to
be known. The following program will print it:

------
#include <stdio.h>
#include <sys/ioctl.h>

int main(int argc, char **argv)
{
printf("%u\n", TIOCGPTN);
return 0;
}
------

On my system, it's 2147767344. Then, one needs to start gdb with the
program file as argument, eg

gdb /sbin/sshd

and use the attach command to attach it to the process one wants to
examinate:

(gdb) attach 9206

This will print a lengthy blurb. The ioctl needs an int * as argument,
hence, one needs to call malloc to get one ($p is a gdb "convenience
variable")

(gdb) set $p = malloc(4)

check that it succeeded

(gdb) p $p
$1 = (void *) 0x7f9f2a50f8e0

run the ioctl with the proper file descriptor number

(gdb) p (int)ioctl(10, 2147767344, $p)
$2 = 0

print the value

(gdb) p *(int *)$p
$3 = 3

finally, free the memory again:

(gdb) p (void)free($p)
$4 = void

and exit the debugger.

The sought-after pty number is thus 3.

[Kenny McCormack]

In article <874kg06...@doppelsaurus.mobileactivedefense.com>,

Rainer Weikusat <rwei...@talktalk.net> wrote:
>gaz...@shell.xmission.com (Kenny McCormack) writes:
>
>[...]
>
>>>Another way to get the information, which is more universal but also is
>>>more difficult to follow, is to make a TIOCGPTN system call on the file
>>>descriptor (from a debugger attached to the process).
>>
>> Well, that sounds interesting. And very complicated...
>> (And time consuming...)
>

>It's really quite simple. First, the numeric value of TIOCGPTN needs to ...

Thanks for posting that. I have done that sort of thing in the past, but
couldn't quite remember the jigs and jogs of it. FWIW, I find both gdb and
its documentation incomprehensible.

Anyway, it looks like you could also do it by using ptsname(3). I was
thinking of doing it along those lines - instead of using the ioctl().

Also, once one has the method down, I would write an Expect script to
automate the process (iterate through all the fds of all the processes that
have /dev/ptms open).

--
Joni Ernst (2014): Obama should be impeached because 2 people have died of Ebola.
Joni Ernst (2020): Trump is doing great things, because only 65,000 times as many people have died of COVID-19.

Josef Stalin (1947): When one person dies, it is a tragedy; when a million die, it is merely statistics.