Bug#908818: Zsh: [7] + 23074 suspended (tty output)

[TS]

Package: zsh
Version: 5.6.1-1
Severity: important

Hello,

sorry for the bad bug description, not sure how to tell it better.

running the function

zprofile() {
ZSH_PROFILE_RC=1 zsh "$@"
}

for profiling Zsh startup performance i now get:

% zprofile
[7] + 23074 suspended (tty output)

then have to fg to see the actual output from profiling.
Today "zprofile" has worked once as usual but now i get this
"[7] + 23074 suspended (tty output)" instead.

This leads me to the impression that this is a race condition.
Further i have the impression, that this related to #908033.

If i can help you with any further tests / informations just let me know.


Thanks and kind regards,


Thilo


-- Package-specific info:

Packages which provide vendor completions:

Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture
Description
+++-================================-=====================-=====================-=====================================================================
ii curl 7.61.0-1 amd64
command line tool for transferring data with URL syntax
ii git-extras 4.6.0-1 all
Extra commands for git
ii git-flow 1.11.0-1 all
Git extension to provide a high-level branching model
ii pdfgrep 2.0.1-1 amd64
search in pdf files for strings matching a regular expression
ii pulseaudio 12.2-1 amd64
PulseAudio sound server
ii systemd 239-9 amd64
system and service manager
ii udev 239-9 amd64
/dev/ and hotplug management daemon
ii vlc-bin 3.0.4-1+b1 amd64
binaries from VLC
ii youtube-dl 2018.09.08-dmo1 all
downloader of videos from YouTube and other sites

dpkg-query: no path found matching pattern /usr/share/zsh/vendor-functions/


-- System Information:
Debian Release: buster/sid
APT prefers unstable
APT policy: (500, 'unstable'), (500, 'testing')
Architecture: amd64 (x86_64)

Kernel: Linux 4.18.0-1-amd64 (SMP w/2 CPU cores)
Locale: LANG=C, LC_CTYPE=C (charmap=UTF-8) (ignored: LC_ALL set to C.UTF-8),
LANGUAGE=C (charmap=UTF-8) (ignored: LC_ALL set to C.UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages zsh depends on:
ii libc6 2.27-6
ii libcap2 1:2.25-1.2
ii libtinfo6 6.1+20180714-1
ii zsh-common 5.6.1-1

Versions of packages zsh recommends:
ii libc6 2.27-6
ii libncursesw6 6.1+20180714-1
ii libpcre3 2:8.39-11

Versions of packages zsh suggests:
ii zsh-doc 5.6.1-1

-- no debconf information

[Daniel Shahaf]

TS wrote on Fri, 14 Sep 2018 14:35 +0200:
> This leads me to the impression that this is a race condition.
> Further i have the impression, that this related to #908033.
>
> If i can help you with any further tests / informations just let me know.

Upstream has committed another patch for #908033; that patch postdates
the 5.6.1-1 package:

[[[
## Branch: master

43446: More entersubsh() / addproc() wiring.

Fix additional races by passing back use of list_pipe_job
from subshell.

By Peter Stephenson on 09/12/2018 08:22
[**View Changes**](https://sourceforge.net/p/zsh/code/ci/7c5241edf37fbafc9e1f833aede76c524ce62bcb/)
]]]

It will be included in upstream 5.6.2 which is due to be released soon.
It would be great if you could test the issue with this patch and
report back.

Cheers,

Daniel

[TS]

Hi Daniel,

Daniel Shahaf schrieb/wrote:

-- <snip> --

> It will be included in upstream 5.6.2 which is due to be released soon.
> It would be great if you could test the issue with this patch and
> report back.
>
> Cheers,
>
> Daniel

I am not familar with building from source. Need to dig through haystacks of
documentation first. Not sure i will make it right even then.

So, do have pre-built .debs somewhere with this patch i can test?
Otherwise will jump through the pre-said loops if needed.

kind regards,

Thilo

[Daniel Shahaf]

TS wrote on Fri, 14 Sep 2018 16:26 +0200:
> I am not familar with building from source. Need to dig through haystacks of
> documentation first. Not sure i will make it right even then.
>
> So, do have pre-built .debs somewhere with this patch i can test?
> Otherwise will jump through the pre-said loops if needed.

Upstream just released 5.6.2, which contains this patch. You could just
wait for it to be packaged in sid.

If you'd like to build it yourself, then first of all thank you for the
willingness. The procedure is documented on
http://ircbots.debian.net/factoids/factoid.php?key=package+recompile ;
between step 4 and step 5 you should apply the patch, such as by running
'patch -p1 < /path/to/patch' and edit the first line of debian/changelog
to read, say, 5.6.1-1+~foo1 instead of 5.6.1-1. The plus-tilde
construct ensures that after you install the package you'll still
upgrade automatically to any future version.

The patch is 43446:
https://github.com/zsh-users/zsh/commit/7c5241edf37fbafc9e1f833aede76c524ce62bcb.patch
(and http://www.zsh.org/mla/zsh-workers/43446)

Cheers,

Daniel

[TS]

Hello,

Daniel Shahaf schrieb/wrote:

-- <snip> --

> If you'd like to build it yourself, then first of all thank you for the
> willingness. The procedure is documented on
> http://ircbots.debian.net/factoids/factoid.php?key=package+recompile ;
> between step 4 and step 5 you should apply the patch, such as by running
> 'patch -p1 < /path/to/patch' and edit the first line of debian/changelog
> to read, say, 5.6.1-1+~foo1 instead of 5.6.1-1. The plus-tilde
> construct ensures that after you install the package you'll still
> upgrade automatically to any future version.
>
> The patch is 43446:
> https://github.com/zsh-users/zsh/commit/7c5241edf37fbafc9e1f833aede76c524ce62bcb.patch
> (and http://www.zsh.org/mla/zsh-workers/43446)

For the record build does not finish for me:

Running test: with message two tabs will start menu completion
Test successful.
Running test: with message and _oldlist, two tabs will start menu completion
Test successful.
Running test: partial completion of option with an argument
Test successful.
Running test: with -W options mixed with arguments
Test successful.
../../Test/Y03arguments.ztst: all tests successful.
**************************************
49 successful test scripts, 1 failure, 1 skipped
**************************************
make[3]: *** [Makefile:188: check] Error 1
make[3]: Leaving directory '/home/luke/temp/zsh-build/zsh-5.6.1/obj/Test'
make[2]: *** [Makefile:263: test] Error 2
make[2]: Leaving directory '/home/luke/temp/zsh-build/zsh-5.6.1/obj'
dh_auto_test: cd obj && make -j2 test VERBOSE=1 returned exit code 2
make[1]: *** [debian/rules:63: override_dh_auto_test-arch] Error 2
make[1]: Leaving directory '/home/luke/temp/zsh-build/zsh-5.6.1'
make: *** [debian/rules:51: binary] Error 2
dpkg-buildpackage: error: debian/rules binary subprocess returned exit status 2

[TS]

Hello Axel,

thanks for uploading 5.6.2, but no luck.

I still get reliable:

% zprofile
[7] + 10408 suspended (tty output)

% pZSH_VERSION
scalar-readonly ZSH_VERSION='5.6.2'


Debian Bug Tracking System schrieb/wrote:
> This is an automatic notification regarding your Bug report
> which was filed against the zsh package:
>
> #908818: Zsh: [7] + 23074 suspended (tty output)
>
> It has been closed by Axel Beckert <a...@debian.org>.

[TS]

TS schrieb/wrote:

> Hello Axel,
>
> thanks for uploading 5.6.2, but no luck.
>
> I still get reliable:
>
> % zprofile
> [7] + 10408 suspended (tty output)
>
> % pZSH_VERSION
> scalar-readonly ZSH_VERSION='5.6.2'

After downgrading Zsh to 5.4.2-3...

% asi zsh
i zsh | 5.4.2-3 | 5.6.2-1
i zsh-common | 5.4.2-3 | 5.6.2-1
i zsh-doc | 5.4.2-3 | 5.6.2-1

% policy zsh
zsh:
Installed: 5.4.2-3
Candidate: 5.6.2-1
Version table:
5.6.2-1 500
500 http://ftp2.de.debian.org/debian unstable/main amd64 Packages
5.6.1-1 500
500 http://ftp.de.debian.org/debian buster/main amd64 Packages
*** 5.4.2-3 500
500 file:/home/archive-local/local local/private amd64 Packages
100 /var/lib/dpkg/status


...i am not able to reproduce the "[7] + 10408 suspended (tty output)" thing.


HTH



kind regards,

Thilo

[TS]

-- <snip> --

> After downgrading Zsh to 5.4.2-3...

-- <snip> --

Tested with 5.5.1-1

% asi zsh
i zsh | 5.5.1-1 | 5.6.2-1
i zsh-common | 5.5.1-1 | 5.6.2-1
i zsh-doc | 5.5.1-1 | 5.6.2-1

>
>
> ...i am not able to reproduce the "[7] + 10408 suspended (tty output)" thing.

up to 5.5.1-1 seem fine, problem starts with 5.6


>
>
> HTH
>
>
>
> kind regards,
>
> Thilo
>

[TS]

-- <snip> --

> up to 5.5.1-1 seem fine, problem starts with 5.6

for the record what zprofile does...

zprofile() {
ZSH_PROFILE_RC=1 zsh "$@"
}

...is it activates this code snipped in my .zshrc:

#
# zprofile
#
(( ${+ZSH_PROFILE_RC} )) && builtin zmodload zsh/zprof


in case you want test yourself.




kind regards,

Thilo

[Daniel Shahaf]

Thanks for testing 5.6.2.

I can't reproduce the bug with a zshrc that contains nothing but an
(unconditional) zmodload zsh/zprof. I'm using a self-compiled
static build from upstream git, not a package build.

Could you please send a bug report upstream (zsh-w...@zsh.org)?
Ideally, the bug report would include a reproduction script that sets up
a new temporary directory, creates a minimal zshrc in it, and runs
'ZDOTDIR=/path/to/dir zsh -d' and provokes the bug.

Thanks!

Daniel

[TS]

-- <snip> --

> I can't reproduce the bug with a zshrc that contains nothing but an
> (unconditional) zmodload zsh/zprof. I'm using a self-compiled
> static build from upstream git, not a package build.
>
> Could you please send a bug report upstream (zsh-w...@zsh.org)?
> Ideally, the bug report would include a reproduction script that sets up
> a new temporary directory, creates a minimal zshrc in it, and runs
> 'ZDOTDIR=/path/to/dir zsh -d' and provokes the bug.
>
> Thanks!
>
> Daniel
>

Hello shortest testcase i can come with atteched. Will send it upstream, too.

with 5.6.2:

% su -l heinb
Password:
tosh% exec zsh
zsh: suspended (tty output)
tosh% fg
[2] + continued
num calls time self name
-----------------------------------------------------------------------------------
tosh%



with 5.5.1:

% su -l heinb
Password:
tosh% exec zsh
num calls time self name
-----------------------------------------------------------------------------------
tosh%



kind regards,

Thilo

[TS]

-- <snip> --

> Could you please send a bug report upstream (zsh-w...@zsh.org)?
> Ideally, the bug report would include a reproduction script that sets up
> a new temporary directory, creates a minimal zshrc in it, and runs
> 'ZDOTDIR=/path/to/dir zsh -d' and provokes the bug.
>
> Thanks!
>
> Daniel

done.

zsh-workers 43468
http://www.zsh.org/mla/workers/2018/msg01272.html


kind regards,

Thilo

[Daniel Shahaf]

Upstream has committed 551ff842721d6ca83727dbe6cd40178f46cc8201 (43464:
Another attachtty() fix). I haven't been able to reproduce the issue so
I can't confirm that commit fixes it.

[TS]

Hello,

Daniel Shahaf schrieb/wrote:

> Upstream has committed 551ff842721d6ca83727dbe6cd40178f46cc8201 (43464:
> Another attachtty() fix). I haven't been able to reproduce the issue so
> I can't confirm that commit fixes it.

Thank your for your support. Really i mean this.

But

There are FAQs out there on "How to write good bugreports".
AFAICK i have followed them TTOMK.

Can we now start to write FAQs on "How to apply fixes you reported correctly".
That would help novices like me mutch.

Jumped through the dpkg/quilt/git loops as mutch as i can, still it does not
built for me:

Test successful.
Running test: with message two tabs will start menu completion
Test successful.
Running test: with message and _oldlist, two tabs will start menu completion
Test successful.
Running test: partial completion of option with an argument
Test successful.
Running test: with -W options mixed with arguments
Test successful.
../../Test/Y03arguments.ztst: all tests successful.
**************************************
49 successful test scripts, 1 failure, 1 skipped
**************************************
make[3]: *** [Makefile:188: check] Error 1

make[3]: Leaving directory '/home/luke/temp/zsh-build/zsh-5.6.2/obj/Test'

make[2]: *** [Makefile:263: test] Error 2

make[2]: Leaving directory '/home/luke/temp/zsh-build/zsh-5.6.2/obj'

dh_auto_test: cd obj && make -j2 test VERBOSE=1 returned exit code 2
make[1]: *** [debian/rules:63: override_dh_auto_test-arch] Error 2

make[1]: Leaving directory '/home/luke/temp/zsh-build/zsh-5.6.2'

make: *** [debian/rules:51: binary] Error 2
dpkg-buildpackage: error: debian/rules binary subprocess returned exit status 2

To make it short.
@Axel please upload a .deb to experimental with said patch applied (cherypicked
as you say).
I will happily test it. Otherwise we are stuck here.
Sorry for that.


kind regards,

Thilo

[Axel Beckert]

Hi,

TS wrote:
> Can we now start to write FAQs on "How to apply fixes you reported correctly".
> That would help novices like me mutch.

Already in my head, yes.

> @Axel please upload a .deb to experimental with said patch applied (cherypicked
> as you say).

That was in my head, too, but I planned to try another variant inbetween:

As soon as I've cherry-picked that commit in git, I wanted to ask you
with very precise instruction to build the package off a packaging git
checkout. That should be more reliable from my point of view.

To give you an idea what the idea looks like the commands as they are
currently (untested) in my head:

As root if not already done:

# apt build-dep zsh

As user:

$ git clone https://salsa.debian.org/debian/zsh.git
$ cd zsh
$ dpkg-buildpackage -b

Then install those resulting packages you already have installed as
root. If you want to install all build packages, you can do that with
"dpkg -i …/zsh*_5.6.2-2_*.deb" or so.

But as said, I need to cherry-pick that commit first. Might be able to
do this later this evening.

Regards, Axel
--
,''`. | Axel Beckert <a...@debian.org>, https://people.debian.org/~abe/
: :' : | Debian Developer, ftp.ch.debian.org Admin
`. `' | 4096R: 2517 B724 C5F6 CA99 5329 6E61 2FF9 CD59 6126 16B5
`- | 1024D: F067 EA27 26B9 C3FC 1486 202E C09E 1D89 9593 0EDE

[Daniel Shahaf]

Axel Beckert wrote on Mon, 17 Sep 2018 20:25 +0200:
> $ dpkg-buildpackage -b
>

Thilo may need to pass --no-sign here.

> Then install those resulting packages you already have installed as
> root. If you want to install all build packages, you can do that with
> "dpkg -i …/zsh*_5.6.2-2_*.deb" or so.
>
> But as said, I need to cherry-pick that commit first. Might be able to
> do this later this evening.

For the time being I did a quick build with that commit backported and
uploaded it to https://home.apache.org/~danielsh/zsh/debian/ . It's
signed with my key which is in /usr/share/keyrings/debian-maintainer.gpg.

Thilo, if you'd like to further debug your build issues you may find it
easier to do that on IRC. The zsh package's channel is #pkg-zsh on
freenode, but it has few subscribers; if it's silent, #debian and possibly
#debian-mentors on oftc would be my next ports of call.

Daniel

[TS]

Daniel Shahaf schrieb/wrote:

-- <snip> --

> For the time being I did a quick build with that commit backported and
> uploaded it to https://home.apache.org/~danielsh/zsh/debian/ .

Hello,

Just to be sure i again tried stock debian zsh 5.6.2 which still fails for me.
The version you uploaded fixes it for me.

Thanks!

kind regards,

Thilo

[Daniel Shahaf]

Control: tags -1 fixed-upstream

TS wrote on Mon, 17 Sep 2018 21:54 +0200:
> Just to be sure i again tried stock debian zsh 5.6.2 which still fails for me.
> The version you uploaded fixes it for me.

Thanks for testing both versions!

Daniel

[Axel Beckert]

Hi,

Daniel Shahaf wrote:
> TS wrote on Mon, 17 Sep 2018 21:54 +0200:
> > Just to be sure i again tried stock debian zsh 5.6.2 which still fails for me.
> > The version you uploaded fixes it for me.
>
> Thanks for testing both versions!

A thanks from me, too.

Will soon upload 5.6.2-2 with that patch and some more changes I have
planned for the next upload.

Daniel: If you have the patch already in git for your test build, feel
free to push it to the debian branch on Salsa.

[Daniel Shahaf]

Axel Beckert wrote on Mon, 17 Sep 2018 23:17 +0200:
> Daniel: If you have the patch already in git for your test build, feel
> free to push it to the debian branch on Salsa.

Actually I just did 'apt-get source zsh/sid' earlier, but I ported that
to git and pushed. The commit2quilt script generated a different
debian/patches/* filename than the one you created last weekend, but
I left that as is.

Thanks,

Daniel

[Axel Beckert]

Hi Daniel,

I did that manually for years. I had forgotten that we even have a
script for that, despite I added it. Probably because I didn't need
it for quite a while.