upgrading past 8.2.2918 loses access to /dev/tty from athena gvim

[raf]

Hi,

macos-10.14.6
./configure --disable-darwin --with-x --enable-gui=athena --with-features=huge

I just upgraded vim from approximately 8.2.2550 to 8.2.3501
and something important has stopped working (from gvim).

I have an autocommand group for editing gpg1-encrypted files.
It executes gpg1 via a shell and that requires access to /dev/tty.

Before the upgrade, it worked in vim and (athena) gvim (i.e.
it would ask me for the passphrase). After the upgrade, it no
longer works in athena gvim. gpg reports:

gpg: cannot open tty `/dev/tty': Device not configured

The last version where this works is 8.2.2918.
The first version that doesn't work is:

8.2.2919: using ":!command" does not work if it uses posix_spawn()

That patch removed this:

diff --git a/src/os_unix.c b/src/os_unix.c
index 20c61106f..0a4f0e698 100644
--- a/src/os_unix.c
+++ b/src/os_unix.c
@@ -4775,11 +4775,6 @@ mch_call_shell_fork(
// push stream discipline modules
if (options & SHELL_COOKED)
setup_slavepty(pty_slave_fd);
-# ifdef TIOCSCTTY
- // Try to become controlling tty (probably doesn't work,
- // unless run by root)
- ioctl(pty_slave_fd, TIOCSCTTY, (char *)NULL);
-# endif
}
# endif
set_default_child_environment(FALSE);

If I run :!tty, it outputs /dev/ttys007, so I tried
replacing "!gpg1" in the commands with "!GPG_TTY=`tty` gpg1"
in the latest version, but that didn't work.

Any idea what I need to do to restore access to /dev/tty
from the athena gui? Or do I have to downgrade to 8.2.2918
and stay there?

cheers,
raf

[Christian Brabandt]

This may depend on the shell you use. So what shell are you using? See
also https://github.com/vim/vim/issues/8951

Best,
Christian
--
Was mich ein bischen tröstet ist, daß auch Politiker von BSE betroffen sind.
-- Heinz Rudolf Kunze

[Dominique Pellé]

Since you wrote:

> The last version where this works is 8.2.2918.
> The first version that doesn't work is:

Then the regression is introduced by:

commit 6a43b37b760347b9a1bedf12e41b458000922969 (tag: v8.2.2919)
Author: Bram Moolenaar <Br...@vim.org>
Date: Tue Jun 1 20:48:40 2021 +0200

patch 8.2.2919: using ":!command" does not work if it uses posix_spawn()

Problem: Using ":!command" does not work if the command uses
posix_spawn().
Solution: Do not call ioctl() with TIOCSCTTY. (Felipe Contreras)


Perhaps Felipe Contreras (added in copy) can check your patch.

Regards
Dominique

[Bram Moolenaar]

Since more than one person complained about this, and it still doesn't
fully work with zsh, I'll revert 8.2.2919. For zsh we need to find a
different solution.

--
hundred-and-one symptoms of being an internet addict:
140. You'd rather catch a score on the web than watch the game as
it is being played on tv.

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// \\\
\\\ sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

[raf]

Thanks. Yes, I'm using zsh as well. If I prefix the vim
command with "SHELL=/bin/sh", the latest version works.

cheers,
raf

[raf]

Thanks. Maybe blocking SIGHUP would fix the original problem with
background processes?

cheers,
raf

[Bram Moolenaar]

> > Since more than one person complained about this, and it still doesn't
> > fully work with zsh, I'll revert 8.2.2919. For zsh we need to find a
> > different solution.
>
> Thanks. Maybe blocking SIGHUP would fix the original problem with
> background processes?

Hopefully someone can find out what the actualy problem is and find a
solution that works and does not cause new problems.

--
hundred-and-one symptoms of being an internet addict:

150. You find yourself counting emoticons to get to sleep.

[raf]

On Thu, Oct 14, 2021 at 04:28:14PM +0100, Bram Moolenaar <Br...@moolenaar.net> wrote:

> > > Since more than one person complained about this, and it still doesn't
> > > fully work with zsh, I'll revert 8.2.2919. For zsh we need to find a
> > > different solution.
> >
> > Thanks. Maybe blocking SIGHUP would fix the original problem with
> > background processes?
>
> Hopefully someone can find out what the actualy problem is and find a
> solution that works and does not cause new problems.

Hmm, SIGHUP is already ignored in the gui.

My guess is that the original problem is that most
shells inherit gvim's ignoring of SIGHUP, but zsh
doesn't. So the command being executed doesn't ignore
SIGHUP either, and it dies when zsh terminates, rather
than when it's finished. But that is just a guess.

It's strange. The original bug report was for Arch Linux.
But it works fine on Debian Linux. It doesn't work on macOS.

I tried the following things on macOS:

- Append "; wait" to the command
- Prepend "setopt NO_HUP; " to the command
- Prepend "trap '' HUP; " to the command
- Prepend "setopt NO_HUP; trap '' HUP; " to the command
- set shellcmdflag=-o\ nohup\ -c

The only thing that "fixed" it was appending "; wait"
to the command. Below is a patch that does that,
only when the gui is running and the shell is zsh and
there's an ampersand in the command (so "&&" will be a
false positive).

diff --git a/src/misc2.c b/src/misc2.c
index 8e01434ea..20c56edde 100644
--- a/src/misc2.c
+++ b/src/misc2.c
@@ -1766,6 +1766,29 @@ call_shell(char_u *cmd, int opt)
proftime_T wait_time;
#endif

+#ifdef FEAT_GUI
+ // Workaround a problem when executing a backgrounded command
+ // with zsh from the gui, by telling zsh to wait for it to complete.
+ // Otherwise, the command fails to function. Theory: Other shells
+ // inherit gvim's ignoring of SIGHUP, but maybe zsh is not ignoring it.
+
+ char_u *modified_cmd = NULL;
+
+ if (cmd != NULL && gui.in_use && strstr((char *)gettail(p_sh), "zsh") != NULL && vim_strchr(cmd, (int)'&'))
+ {
+ const char *suffix = "; wait";
+ size_t suffix_len = strlen(suffix);
+ size_t cmd_len = strlen((const char *)cmd);
+
+ modified_cmd = vim_strnsave(cmd, cmd_len + suffix_len);
+ if (modified_cmd != NULL)
+ {
+ memcpy(modified_cmd + cmd_len, suffix, suffix_len + 1); // "+ 1" is not necessary
+ cmd = modified_cmd;
+ }
+ }
+#endif
+
if (p_verbose > 3)
{
verbose_enter();
@@ -1843,6 +1866,10 @@ call_shell(char_u *cmd, int opt)
if (do_profiling == PROF_YES)
prof_child_exit(&wait_time);
# endif
+#endif
+#ifdef FEAT_GUI
+ if (modified_cmd != NULL)
+ vim_free(modified_cmd);
#endif

return retval;

But it's awful and hacky. It "fixes" the problem by
effectively disabling the ability to background a
process in zsh. And since it isn't a problem on Debian
Linux, it would make things slightly worse on some
systems, but only for commands that actually take a
long time to execute.

Unfortunately, dtruss on macOS isn't working well
enough for me to trace what's happening. Perhaps
someone with Arch Linux can use strace to investigate.

Until then, the most practical solution is to just:

set shell=/bin/bash

That fixes it on macOS.

cheers,
raf

[Bram Moolenaar]

> On Thu, Oct 14, 2021 at 04:28:14PM +0100, Bram Moolenaar <Br...@moolenaar.net> wrote:
>
> > > > Since more than one person complained about this, and it still doesn't
> > > > fully work with zsh, I'll revert 8.2.2919. For zsh we need to find a
> > > > different solution.
> > >
> > > Thanks. Maybe blocking SIGHUP would fix the original problem with
> > > background processes?
> >
> > Hopefully someone can find out what the actualy problem is and find a
> > solution that works and does not cause new problems.
>
> Hmm, SIGHUP is already ignored in the gui.

Thanks for taking time to look into this.

> My guess is that the original problem is that most
> shells inherit gvim's ignoring of SIGHUP, but zsh
> doesn't. So the command being executed doesn't ignore
> SIGHUP either, and it dies when zsh terminates, rather
> than when it's finished. But that is just a guess.
>
> It's strange. The original bug report was for Arch Linux.
> But it works fine on Debian Linux. It doesn't work on macOS.

Thus we don't fully understand the cause of the problem. Perhaps it's a
specific zsh version or how it was configured?

Well, more a workaround than a fix. But indeed, just setting 'shell' to
something else than zsh should be a workable solution for most users.

I rather recommend using another shell than to include a hacky solution
with side effects.

--
If Microsoft would build a car...
... Occasionally your car would die on the freeway for no
reason. You would have to pull over to the side of the road,
close all of the car windows, shut it off, restart it, and
reopen the windows before you could continue. For some reason
you would simply accept this.

[raf]

On Sun, Oct 17, 2021 at 01:37:34PM +0100, Bram Moolenaar <Br...@moolenaar.net> wrote:

>
> > On Thu, Oct 14, 2021 at 04:28:14PM +0100, Bram Moolenaar <Br...@moolenaar.net> wrote:
> >
> > > > > Since more than one person complained about this, and it still doesn't
> > > > > fully work with zsh, I'll revert 8.2.2919. For zsh we need to find a
> > > > > different solution.
> > > >
> > > > Thanks. Maybe blocking SIGHUP would fix the original problem with
> > > > background processes?
> > >
> > > Hopefully someone can find out what the actualy problem is and find a
> > > solution that works and does not cause new problems.
> >
> > Hmm, SIGHUP is already ignored in the gui.
>
> Thanks for taking time to look into this.
>
> > My guess is that the original problem is that most
> > shells inherit gvim's ignoring of SIGHUP, but zsh
> > doesn't. So the command being executed doesn't ignore
> > SIGHUP either, and it dies when zsh terminates, rather
> > than when it's finished. But that is just a guess.
> >
> > It's strange. The original bug report was for Arch Linux.
> > But it works fine on Debian Linux. It doesn't work on macOS.
>
> Thus we don't fully understand the cause of the problem. Perhaps it's a
> specific zsh version or how it was configured?

I thought that too, i.e., maybe macOS (via macports)
and arch have a recent zsh that no longer works, but
debian stable might have an older version which still
works. But macports and debian stable both have zsh-5.8
which is the latest version. Arch should have the same.
Hmm, the system /bin/zsh on my macOS is zsh-5.3 and
that works fine. So it's still a mystery.

> > I tried the following things on macOS:
> >
> > - Append "; wait" to the command
> > - Prepend "setopt NO_HUP; " to the command
> > - Prepend "trap '' HUP; " to the command
> > - Prepend "setopt NO_HUP; trap '' HUP; " to the command
> > - set shellcmdflag=-o\ nohup\ -c
> >
> > The only thing that "fixed" it was appending "; wait"
> > to the command. Below is a patch that does that,
> > only when the gui is running and the shell is zsh and
> > there's an ampersand in the command (so "&&" will be a
> > false positive).
> >

> > [...]

> >
> > But it's awful and hacky. It "fixes" the problem by
> > effectively disabling the ability to background a
> > process in zsh. And since it isn't a problem on Debian
> > Linux, it would make things slightly worse on some
> > systems, but only for commands that actually take a
> > long time to execute.
> >
> > Unfortunately, dtruss on macOS isn't working well
> > enough for me to trace what's happening. Perhaps
> > someone with Arch Linux can use strace to investigate.
> >
> > Until then, the most practical solution is to just:
> >
> > set shell=/bin/bash
> >
> > That fixes it on macOS.
>
> Well, more a workaround than a fix. But indeed, just setting 'shell' to
> something else than zsh should be a workable solution for most users.
>
> I rather recommend using another shell than to include a hacky solution
> with side effects.

Indeed.

> --
> If Microsoft would build a car...
> ... Occasionally your car would die on the freeway for no
> reason. You would have to pull over to the side of the road,
> close all of the car windows, shut it off, restart it, and
> reopen the windows before you could continue. For some reason
> you would simply accept this.
>
> /// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
> /// \\\
> \\\ sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ ///
> \\\ help me help AIDS victims -- http://ICCF-Holland.org ///

cheers,
raf