Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Bug#923423: texlive-fonts-extra installation hangs
452 views
Skip to first unread message
Vincent Lefevre
Feb 27, 2019, 7:10:03 PM2/27/19
to
Package: texlive-fonts-extra
Version: 2018.20190227-1
Severity: serious
Installation of texlive-fonts-extra 2018.20190227-1 hangs.
Here's px information:
# px 5553
/usr/bin/dpkg
--status-fd
80
--no-triggers
--unpack
--auto-deconfigure
--recursive
/tmp/apt-dpkg-install-mos7uq
kernel(0) root
init(1) root
lightdm(915) root
lightdm(1525) root
zsh(1547) vinc17
fvwm2(1629) vinc17
sh(1745) vinc17
xterm(1746) vinc17
zsh(1750) vinc17
su(1839) root
bash(1841) root
aptitude(2012) root
----------------------> dpkg(5553) root
17m01s ago dpkg was started, at 2019-02-28T00:48:45+01:00.
1.5% has been its average CPU usage since then, or 15s/17m01s
Other processes started close to dpkg(5553):
[kworker/0:1-events](5351) was started 2m57s before dpkg(5553)
[kworker/u17:0-kcryptd](5540) was started 1.0s before dpkg(5553)
[kworker/4:2](5771) was started 2m19s after dpkg(5553)
[kworker/u16:1-events_unbound](5928) was started 4m21s after dpkg(5553)
[kworker/5:2-mm_percpu_wq](6068) was started 5m06s after dpkg(5553)
Users logged in when dpkg(5553) started:
vinc17 from 155.133.131.76
vinc17 from :0
2019-02-28T01:05:46.585799: Now invoking lsof, this can take over a minute on a big system...
2019-02-28T01:05:47.098503: lsof done, proceeding.
File descriptors:
stdin : [CHR] /dev/pts/16
stdout: [CHR] /dev/pts/16
stderr: [CHR] /dev/pts/16
Network connections:
Inter Process Communication:
aptitude(2012): [FIFO] pipe
For a list of all open files, do "sudo lsof -p 5553", or "sudo watch lsof -p 5553" for a live view.
-- Package-specific info:
IMPORTANT INFORMATION: We will only consider bug reports concerning
the packaging of TeX Live as relevant. If you have problems with
combination of packages in a LaTeX document, please consult your
local TeX User Group, the comp.text.tex user group, the author of
the original .sty file, or any other help resource.
In particular, bugs that are related to up-upstream, i.e., neither
Debian nor TeX Live (upstream), but the original package authors,
will be closed immediately.
*** The Debian TeX Team is *not* a LaTeX Help Desk ***
If you report an error when running one of the TeX-related binaries
(latex, pdftex, metafont,...), or if the bug is related to bad or wrong
output, please include a MINIMAL example input file that produces the
error in your report.
Please run your example with
(pdf)latex -recorder ...
(or any other program that supports -recorder) and send us the generated
file with the extension .fls, it lists all the files loaded during
the run and can easily explain problems induced by outdated files in
your home directory.
Don't forget to also include minimal examples of other files that are
needed, e.g. bibtex databases. Often it also helps
to include the logfile. Please, never send included pictures!
If your example file isn't short or produces more than one page of
output (except when multiple pages are needed to show the problem),
you can probably minimize it further. Instructions on how to do that
can be found at
http://www.minimalbeispiel.de/mini-en.html (english)
or
http://www.minimalbeispiel.de/mini.html (german)
##################################
minimal input file
##################################
other files
######################################
List of ls-R files
-rw-r--r-- 1 root root 2880 2019-02-27 03:14:07 /var/lib/texmf/ls-R
lrwxrwxrwx 1 root root 29 2018-09-02 14:32:33 /usr/share/texmf/ls-R -> /var/lib/texmf/ls-R-TEXMFMAIN
lrwxrwxrwx 1 root root 31 2019-02-27 01:08:59 /usr/share/texlive/texmf-dist/ls-R -> /var/lib/texmf/ls-R-TEXLIVEDIST
lrwxrwxrwx 1 root root 31 2019-02-27 01:08:59 /usr/share/texlive/texmf-dist/ls-R -> /var/lib/texmf/ls-R-TEXLIVEDIST
######################################
Config files
-rw-r--r-- 1 root root 475 2018-09-02 20:20:53 /etc/texmf/web2c/texmf.cnf
lrwxrwxrwx 1 root root 33 2019-02-27 01:08:59 /usr/share/texmf/web2c/fmtutil.cnf -> /var/lib/texmf/fmtutil.cnf-DEBIAN
lrwxrwxrwx 1 root root 32 2019-02-27 01:08:59 /usr/share/texmf/web2c/updmap.cfg -> /var/lib/texmf/updmap.cfg-DEBIAN
-rw-r--r-- 1 root root 5089 2019-02-27 03:13:40 /var/lib/texmf/tex/generic/config/language.dat
######################################
Files in /etc/texmf/web2c/
total 8
-rw-r--r-- 1 root root 283 2014-10-21 02:46:09 mktex.cnf
-rw-r--r-- 1 root root 475 2018-09-02 20:20:53 texmf.cnf
######################################
md5sums of texmf.d
ca40c66f144b4bafc3e59a2dd32ecb9c /etc/texmf/texmf.d/00debian.cnf
-- System Information:
Debian Release: buster/sid
APT prefers unstable-debug
APT policy: (500, 'unstable-debug'), (500, 'stable-updates'), (500, 'unstable'), (500, 'testing'), (500, 'stable'), (1, 'experimental')
Architecture: amd64 (x86_64)
Kernel: Linux 4.19.0-3-amd64 (SMP w/8 CPU cores)
Kernel taint flags: TAINT_PROPRIETARY_MODULE, TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=POSIX, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE=POSIX (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
Versions of packages texlive-fonts-extra depends on:
it tex-common 6.10
ii texlive-base 2018.20190207-1
Versions of packages texlive-fonts-extra recommends:
ii fonts-adf-accanthis 0.20110505-3
ii fonts-adf-berenis 0.20110505-3
ii fonts-adf-gillius 0.20110505-3
ii fonts-adf-universalis 0.20110505-3
ii fonts-cabin 1.5-2
ii fonts-comfortaa 3.001-2
ii fonts-croscore 20181227-1
ii fonts-crosextra-caladea 20130214-2
ii fonts-crosextra-carlito 20130920-1
ii fonts-dejavu-core 2.37-1
ii fonts-dejavu-extra 2.37-1
ii fonts-ebgaramond 0.016-1
ii fonts-ebgaramond-extra 0.016-1
ii fonts-font-awesome 4.7.0~dfsg-3
ii fonts-freefont-otf 20120503-9
ii fonts-freefont-ttf 20120503-9
ii fonts-gfs-artemisia 1.1-5
ii fonts-gfs-complutum 1.1-6
ii fonts-gfs-didot 1.1-6
ii fonts-gfs-neohellenic 1.1-6
ii fonts-gfs-olga 1.1-5
ii fonts-gfs-solomos 1.1-5
ii fonts-go 0~20170330-1
ii fonts-junicode 1.001-2
ii fonts-lato 2.0-2
ii fonts-linuxlibertine 5.3.0-4
ii fonts-lobstertwo 2.0-2
ii fonts-noto-hinted 20181227-1
ii fonts-noto-mono 20181227-1
ii fonts-oflb-asana-math 000.907-6
ii fonts-open-sans 1.11-1
pn fonts-roboto-hinted <none>
ii fonts-roboto-unhinted 2:0~20170802-2
ii fonts-sil-gentium 20081126:1.03-2
ii fonts-sil-gentium-basic 1.102-1
ii fonts-sil-gentiumplus 5.000-2
ii fonts-sil-gentiumplus-compact 5.000-2
ii fonts-stix 1.1.1-4
iu texlive-fonts-extra-links 2018.20190227-1
pn texlive-latex-extra <none>
Versions of packages texlive-fonts-extra suggests:
ii cm-super 0.3.4-13
iu texlive-fonts-extra-doc 2018.20190227-1
Versions of packages tex-common depends on:
ii dpkg 1.19.5
ii ucf 3.0038+nmu1
Versions of packages tex-common suggests:
ii debhelper 12.1.1
Versions of packages texlive-fonts-extra is related to:
it tex-common 6.10
ii texlive-binaries 2018.20181218.49446-1
-- debconf information:
tex-common/check_texmf_wrong:
tex-common/check_texmf_missing:
Version: 2018.20190227-1
Severity: serious
Installation of texlive-fonts-extra 2018.20190227-1 hangs.
Here's px information:
# px 5553
/usr/bin/dpkg
--status-fd
80
--no-triggers
--unpack
--auto-deconfigure
--recursive
/tmp/apt-dpkg-install-mos7uq
kernel(0) root
init(1) root
lightdm(915) root
lightdm(1525) root
zsh(1547) vinc17
fvwm2(1629) vinc17
sh(1745) vinc17
xterm(1746) vinc17
zsh(1750) vinc17
su(1839) root
bash(1841) root
aptitude(2012) root
----------------------> dpkg(5553) root
17m01s ago dpkg was started, at 2019-02-28T00:48:45+01:00.
1.5% has been its average CPU usage since then, or 15s/17m01s
Other processes started close to dpkg(5553):
[kworker/0:1-events](5351) was started 2m57s before dpkg(5553)
[kworker/u17:0-kcryptd](5540) was started 1.0s before dpkg(5553)
[kworker/4:2](5771) was started 2m19s after dpkg(5553)
[kworker/u16:1-events_unbound](5928) was started 4m21s after dpkg(5553)
[kworker/5:2-mm_percpu_wq](6068) was started 5m06s after dpkg(5553)
Users logged in when dpkg(5553) started:
vinc17 from 155.133.131.76
vinc17 from :0
2019-02-28T01:05:46.585799: Now invoking lsof, this can take over a minute on a big system...
2019-02-28T01:05:47.098503: lsof done, proceeding.
File descriptors:
stdin : [CHR] /dev/pts/16
stdout: [CHR] /dev/pts/16
stderr: [CHR] /dev/pts/16
Network connections:
Inter Process Communication:
aptitude(2012): [FIFO] pipe
For a list of all open files, do "sudo lsof -p 5553", or "sudo watch lsof -p 5553" for a live view.
-- Package-specific info:
IMPORTANT INFORMATION: We will only consider bug reports concerning
the packaging of TeX Live as relevant. If you have problems with
combination of packages in a LaTeX document, please consult your
local TeX User Group, the comp.text.tex user group, the author of
the original .sty file, or any other help resource.
In particular, bugs that are related to up-upstream, i.e., neither
Debian nor TeX Live (upstream), but the original package authors,
will be closed immediately.
*** The Debian TeX Team is *not* a LaTeX Help Desk ***
If you report an error when running one of the TeX-related binaries
(latex, pdftex, metafont,...), or if the bug is related to bad or wrong
output, please include a MINIMAL example input file that produces the
error in your report.
Please run your example with
(pdf)latex -recorder ...
(or any other program that supports -recorder) and send us the generated
file with the extension .fls, it lists all the files loaded during
the run and can easily explain problems induced by outdated files in
your home directory.
Don't forget to also include minimal examples of other files that are
needed, e.g. bibtex databases. Often it also helps
to include the logfile. Please, never send included pictures!
If your example file isn't short or produces more than one page of
output (except when multiple pages are needed to show the problem),
you can probably minimize it further. Instructions on how to do that
can be found at
http://www.minimalbeispiel.de/mini-en.html (english)
or
http://www.minimalbeispiel.de/mini.html (german)
##################################
minimal input file
##################################
other files
######################################
List of ls-R files
-rw-r--r-- 1 root root 2880 2019-02-27 03:14:07 /var/lib/texmf/ls-R
lrwxrwxrwx 1 root root 29 2018-09-02 14:32:33 /usr/share/texmf/ls-R -> /var/lib/texmf/ls-R-TEXMFMAIN
lrwxrwxrwx 1 root root 31 2019-02-27 01:08:59 /usr/share/texlive/texmf-dist/ls-R -> /var/lib/texmf/ls-R-TEXLIVEDIST
lrwxrwxrwx 1 root root 31 2019-02-27 01:08:59 /usr/share/texlive/texmf-dist/ls-R -> /var/lib/texmf/ls-R-TEXLIVEDIST
######################################
Config files
-rw-r--r-- 1 root root 475 2018-09-02 20:20:53 /etc/texmf/web2c/texmf.cnf
lrwxrwxrwx 1 root root 33 2019-02-27 01:08:59 /usr/share/texmf/web2c/fmtutil.cnf -> /var/lib/texmf/fmtutil.cnf-DEBIAN
lrwxrwxrwx 1 root root 32 2019-02-27 01:08:59 /usr/share/texmf/web2c/updmap.cfg -> /var/lib/texmf/updmap.cfg-DEBIAN
-rw-r--r-- 1 root root 5089 2019-02-27 03:13:40 /var/lib/texmf/tex/generic/config/language.dat
######################################
Files in /etc/texmf/web2c/
total 8
-rw-r--r-- 1 root root 283 2014-10-21 02:46:09 mktex.cnf
-rw-r--r-- 1 root root 475 2018-09-02 20:20:53 texmf.cnf
######################################
md5sums of texmf.d
ca40c66f144b4bafc3e59a2dd32ecb9c /etc/texmf/texmf.d/00debian.cnf
-- System Information:
Debian Release: buster/sid
APT prefers unstable-debug
APT policy: (500, 'unstable-debug'), (500, 'stable-updates'), (500, 'unstable'), (500, 'testing'), (500, 'stable'), (1, 'experimental')
Architecture: amd64 (x86_64)
Kernel: Linux 4.19.0-3-amd64 (SMP w/8 CPU cores)
Kernel taint flags: TAINT_PROPRIETARY_MODULE, TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=POSIX, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE=POSIX (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
Versions of packages texlive-fonts-extra depends on:
it tex-common 6.10
ii texlive-base 2018.20190207-1
Versions of packages texlive-fonts-extra recommends:
ii fonts-adf-accanthis 0.20110505-3
ii fonts-adf-berenis 0.20110505-3
ii fonts-adf-gillius 0.20110505-3
ii fonts-adf-universalis 0.20110505-3
ii fonts-cabin 1.5-2
ii fonts-comfortaa 3.001-2
ii fonts-croscore 20181227-1
ii fonts-crosextra-caladea 20130214-2
ii fonts-crosextra-carlito 20130920-1
ii fonts-dejavu-core 2.37-1
ii fonts-dejavu-extra 2.37-1
ii fonts-ebgaramond 0.016-1
ii fonts-ebgaramond-extra 0.016-1
ii fonts-font-awesome 4.7.0~dfsg-3
ii fonts-freefont-otf 20120503-9
ii fonts-freefont-ttf 20120503-9
ii fonts-gfs-artemisia 1.1-5
ii fonts-gfs-complutum 1.1-6
ii fonts-gfs-didot 1.1-6
ii fonts-gfs-neohellenic 1.1-6
ii fonts-gfs-olga 1.1-5
ii fonts-gfs-solomos 1.1-5
ii fonts-go 0~20170330-1
ii fonts-junicode 1.001-2
ii fonts-lato 2.0-2
ii fonts-linuxlibertine 5.3.0-4
ii fonts-lobstertwo 2.0-2
ii fonts-noto-hinted 20181227-1
ii fonts-noto-mono 20181227-1
ii fonts-oflb-asana-math 000.907-6
ii fonts-open-sans 1.11-1
pn fonts-roboto-hinted <none>
ii fonts-roboto-unhinted 2:0~20170802-2
ii fonts-sil-gentium 20081126:1.03-2
ii fonts-sil-gentium-basic 1.102-1
ii fonts-sil-gentiumplus 5.000-2
ii fonts-sil-gentiumplus-compact 5.000-2
ii fonts-stix 1.1.1-4
iu texlive-fonts-extra-links 2018.20190227-1
pn texlive-latex-extra <none>
Versions of packages texlive-fonts-extra suggests:
ii cm-super 0.3.4-13
iu texlive-fonts-extra-doc 2018.20190227-1
Versions of packages tex-common depends on:
ii dpkg 1.19.5
ii ucf 3.0038+nmu1
Versions of packages tex-common suggests:
ii debhelper 12.1.1
Versions of packages texlive-fonts-extra is related to:
it tex-common 6.10
ii texlive-binaries 2018.20181218.49446-1
-- debconf information:
tex-common/check_texmf_wrong:
tex-common/check_texmf_missing:
Vincent Lefevre
Feb 27, 2019, 7:30:02 PM2/27/19
to
Control: severity -1 important
Control: retitle -1 texlive-fonts-extra installation hangs for 19 minutes
On 2019-02-28 01:06:28 +0100, Vincent Lefevre wrote:
> Package: texlive-fonts-extra
> Version: 2018.20190227-1
> Severity: serious
>
> Installation of texlive-fonts-extra 2018.20190227-1 hangs.
[...]
It eventually resumed. In the /var/log/dpkg.log file:
[...]
2019-02-28 00:49:41 install texlive-bibtex-extra:all 2018.20190207-1 2018.20190227-1
2019-02-28 00:49:41 status half-installed texlive-bibtex-extra:all 2018.20190207-1
2019-02-28 00:50:27 status unpacked texlive-bibtex-extra:all 2018.20190227-1
2019-02-28 00:50:27 install texlive-extra-utils:all 2018.20190207-1 2018.20190227-1
2019-02-28 00:50:27 status half-installed texlive-extra-utils:all 2018.20190207-1
2019-02-28 00:50:55 status unpacked texlive-extra-utils:all 2018.20190227-1
2019-02-28 00:50:55 install texlive-font-utils:all 2018.20190207-1 2018.20190227-1
2019-02-28 00:50:55 status half-installed texlive-font-utils:all 2018.20190207-1
2019-02-28 00:51:00 status unpacked texlive-font-utils:all 2018.20190227-1
2019-02-28 00:51:01 install texlive-fonts-extra:all 2018.20190207-1 2018.20190227-1
2019-02-28 00:51:01 status half-installed texlive-fonts-extra:all 2018.20190207-1
2019-02-28 01:10:50 status unpacked texlive-fonts-extra:all 2018.20190227-1
2019-02-28 01:10:50 install texlive-fonts-recommended:all 2018.20190207-1 2018.20190227-1
2019-02-28 01:10:50 status half-installed texlive-fonts-recommended:all 2018.20190207-1
2019-02-28 01:11:49 status unpacked texlive-fonts-recommended:all 2018.20190227-1
[...]
So that's about 19 minutes with almost no CPU/disk/network activity.
--
Vincent Lefèvre <vin...@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)
Control: retitle -1 texlive-fonts-extra installation hangs for 19 minutes
On 2019-02-28 01:06:28 +0100, Vincent Lefevre wrote:
> Package: texlive-fonts-extra
> Version: 2018.20190227-1
> Severity: serious
>
> Installation of texlive-fonts-extra 2018.20190227-1 hangs.
It eventually resumed. In the /var/log/dpkg.log file:
[...]
2019-02-28 00:49:41 install texlive-bibtex-extra:all 2018.20190207-1 2018.20190227-1
2019-02-28 00:49:41 status half-installed texlive-bibtex-extra:all 2018.20190207-1
2019-02-28 00:50:27 status unpacked texlive-bibtex-extra:all 2018.20190227-1
2019-02-28 00:50:27 install texlive-extra-utils:all 2018.20190207-1 2018.20190227-1
2019-02-28 00:50:27 status half-installed texlive-extra-utils:all 2018.20190207-1
2019-02-28 00:50:55 status unpacked texlive-extra-utils:all 2018.20190227-1
2019-02-28 00:50:55 install texlive-font-utils:all 2018.20190207-1 2018.20190227-1
2019-02-28 00:50:55 status half-installed texlive-font-utils:all 2018.20190207-1
2019-02-28 00:51:00 status unpacked texlive-font-utils:all 2018.20190227-1
2019-02-28 00:51:01 install texlive-fonts-extra:all 2018.20190207-1 2018.20190227-1
2019-02-28 00:51:01 status half-installed texlive-fonts-extra:all 2018.20190207-1
2019-02-28 01:10:50 status unpacked texlive-fonts-extra:all 2018.20190227-1
2019-02-28 01:10:50 install texlive-fonts-recommended:all 2018.20190207-1 2018.20190227-1
2019-02-28 01:10:50 status half-installed texlive-fonts-recommended:all 2018.20190207-1
2019-02-28 01:11:49 status unpacked texlive-fonts-recommended:all 2018.20190227-1
[...]
So that's about 19 minutes with almost no CPU/disk/network activity.
--
Vincent Lefèvre <vin...@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)
Norbert Preining
Feb 27, 2019, 10:10:03 PM2/27/19
to
reassign 923423 dpkg
thanks
On Thu, 28 Feb 2019, Vincent Lefevre wrote:
> It eventually resumed. In the /var/log/dpkg.log file:
>
> [...]
> 2019-02-28 00:49:41 install texlive-bibtex-extra:all 2018.20190207-1 2018.20190227-1
> 2019-02-28 00:49:41 status half-installed texlive-bibtex-extra:all 2018.20190207-1
> 2019-02-28 00:50:27 status unpacked texlive-bibtex-extra:all 2018.20190227-1
Nothing I can do about it, please ask dpkg maintainers what is going on.
Norbert
--
PREINING Norbert http://www.preining.info
Accelia Inc. + JAIST + TeX Live + Debian Developer
GPG: 0x860CDC13 fp: F7D8 A928 26E3 16A1 9FA0 ACF0 6CAC A448 860C DC13
thanks
On Thu, 28 Feb 2019, Vincent Lefevre wrote:
> It eventually resumed. In the /var/log/dpkg.log file:
>
> [...]
> 2019-02-28 00:49:41 install texlive-bibtex-extra:all 2018.20190207-1 2018.20190227-1
> 2019-02-28 00:49:41 status half-installed texlive-bibtex-extra:all 2018.20190207-1
> 2019-02-28 00:50:27 status unpacked texlive-bibtex-extra:all 2018.20190227-1
Norbert
--
PREINING Norbert http://www.preining.info
Accelia Inc. + JAIST + TeX Live + Debian Developer
GPG: 0x860CDC13 fp: F7D8 A928 26E3 16A1 9FA0 ACF0 6CAC A448 860C DC13
Vincent Lefevre
Feb 28, 2019, 4:10:03 AM2/28/19
to
In short:
[...]
And ditto in the past:
[...]
2019-02-02 16:21:55 status half-installed texlive-fonts-extra:all 2018.20190126-1
2019-02-02 16:41:32 status unpacked texlive-fonts-extra:all 2018.20190131-1
[...]
[...]
2019-02-26 01:35:08 status half-installed texlive-fonts-extra:all 2018.20190131-1
2019-02-26 01:55:12 status unpacked texlive-fonts-extra:all 2018.20190131-2
[...]
[...]
2019-02-27 02:33:14 status half-installed texlive-fonts-extra:all 2018.20190131-2
2019-02-27 02:53:18 status unpacked texlive-fonts-extra:all 2018.20190207-1
[...]
[...]
2019-02-28 00:51:01 status half-installed texlive-fonts-extra:all 2018.20190207-1
2019-02-28 01:10:50 status unpacked texlive-fonts-extra:all 2018.20190227-1
[...]
2019-02-28 01:10:50 status unpacked texlive-fonts-extra:all 2018.20190227-1
And ditto in the past:
[...]
2019-02-02 16:21:55 status half-installed texlive-fonts-extra:all 2018.20190126-1
2019-02-02 16:41:32 status unpacked texlive-fonts-extra:all 2018.20190131-1
[...]
[...]
2019-02-26 01:35:08 status half-installed texlive-fonts-extra:all 2018.20190131-1
2019-02-26 01:55:12 status unpacked texlive-fonts-extra:all 2018.20190131-2
[...]
[...]
2019-02-27 02:33:14 status half-installed texlive-fonts-extra:all 2018.20190131-2
2019-02-27 02:53:18 status unpacked texlive-fonts-extra:all 2018.20190207-1
[...]
Guillem Jover
Mar 2, 2019, 6:30:02 PM3/2/19
to
Hi!
On Thu, 2019-02-28 at 01:06:28 +0100, Vincent Lefevre wrote:
> Package: texlive-fonts-extra
> Version: 2018.20190227-1
> Severity: serious
> Installation of texlive-fonts-extra 2018.20190227-1 hangs.
problem to me, though.
Did this start happening due to the new kernel version? What
filesystem are you using (say btrfs or similar?), etc, can you
reproduce on another system? I certainly can't. :)
I guess attaching strace to the running dpkg might also give some more
info on what's going on, or where it's stalling. Also if you can
simply reproduce via «dpkg -i», then enabling all dpkg debug flags
might also help.
> -- System Information:
> Debian Release: buster/sid
> APT prefers unstable-debug
> APT policy: (500, 'unstable-debug'), (500, 'stable-updates'), (500, 'unstable'), (500, 'testing'), (500, 'stable'), (1, 'experimental')
> Architecture: amd64 (x86_64)
>
> Kernel: Linux 4.19.0-3-amd64 (SMP w/8 CPU cores)
> Kernel taint flags: TAINT_PROPRIETARY_MODULE, TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
> Locale: LANG=POSIX, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE=POSIX (charmap=UTF-8)
> Shell: /bin/sh linked to /bin/dash
> Init: systemd (via /run/systemd/system)
> LSM: AppArmor: enabled
I see the kernel is tainted. Perhaps try to unload those and see? :)
> Versions of packages tex-common depends on:
> ii dpkg 1.19.5
Thanks,
Guillem
On Thu, 2019-02-28 at 01:06:28 +0100, Vincent Lefevre wrote:
> Package: texlive-fonts-extra
> Version: 2018.20190227-1
> Severity: serious
> Installation of texlive-fonts-extra 2018.20190227-1 hangs.
> 17m01s ago dpkg was started, at 2019-02-28T00:48:45+01:00.
> 1.5% has been its average CPU usage since then, or 15s/17m01s
>
> Other processes started close to dpkg(5553):
> [kworker/0:1-events](5351) was started 2m57s before dpkg(5553)
> [kworker/u17:0-kcryptd](5540) was started 1.0s before dpkg(5553)
> [kworker/4:2](5771) was started 2m19s after dpkg(5553)
> [kworker/u16:1-events_unbound](5928) was started 4m21s after dpkg(5553)
> [kworker/5:2-mm_percpu_wq](6068) was started 5m06s after dpkg(5553)
Thanks for all the info. This does look like a hardware or kernel
> 1.5% has been its average CPU usage since then, or 15s/17m01s
>
> Other processes started close to dpkg(5553):
> [kworker/0:1-events](5351) was started 2m57s before dpkg(5553)
> [kworker/u17:0-kcryptd](5540) was started 1.0s before dpkg(5553)
> [kworker/4:2](5771) was started 2m19s after dpkg(5553)
> [kworker/u16:1-events_unbound](5928) was started 4m21s after dpkg(5553)
> [kworker/5:2-mm_percpu_wq](6068) was started 5m06s after dpkg(5553)
problem to me, though.
Did this start happening due to the new kernel version? What
filesystem are you using (say btrfs or similar?), etc, can you
reproduce on another system? I certainly can't. :)
I guess attaching strace to the running dpkg might also give some more
info on what's going on, or where it's stalling. Also if you can
simply reproduce via «dpkg -i», then enabling all dpkg debug flags
might also help.
> -- System Information:
> Debian Release: buster/sid
> APT prefers unstable-debug
> APT policy: (500, 'unstable-debug'), (500, 'stable-updates'), (500, 'unstable'), (500, 'testing'), (500, 'stable'), (1, 'experimental')
> Architecture: amd64 (x86_64)
>
> Kernel: Linux 4.19.0-3-amd64 (SMP w/8 CPU cores)
> Kernel taint flags: TAINT_PROPRIETARY_MODULE, TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
> Locale: LANG=POSIX, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE=POSIX (charmap=UTF-8)
> Shell: /bin/sh linked to /bin/dash
> Init: systemd (via /run/systemd/system)
> LSM: AppArmor: enabled
> Versions of packages tex-common depends on:
> ii dpkg 1.19.5
Guillem
Vincent Lefevre
Mar 2, 2019, 8:50:03 PM3/2/19
to
On 2019-03-03 00:15:13 +0100, Guillem Jover wrote:
> Did this start happening due to the new kernel version?
In March 2018, this was already slow, but much less:
> Did this start happening due to the new kernel version?
[...]
2018-03-08 00:55:55 status unpacked texlive-fonts-extra:all 2017.20180225-1
2018-03-08 00:55:55 status half-installed texlive-fonts-extra:all 2017.20180225-1
2018-03-08 01:02:55 status half-installed texlive-fonts-extra:all 2017.20180225-1
2018-03-08 01:02:56 status unpacked texlive-fonts-extra:all 2017.20180305-2
2018-03-08 01:02:56 status unpacked texlive-fonts-extra:all 2017.20180305-2
[...]
I don't have older information.
> What filesystem are you using (say btrfs or similar?),
/dev/mapper/zira--vg-root on / type ext4 (rw,relatime,errors=remount-ro)
> etc, can you reproduce on another system? I certainly can't. :)
(both for old and new versions). I assume that this is what
is expected.
> I guess attaching strace to the running dpkg might also give some more
> info on what's going on, or where it's stalling. Also if you can
> simply reproduce via «dpkg -i», then enabling all dpkg debug flags
> might also help.
dpkg -i /var/cache/apt/archives/texlive-fonts-extra_2018.20190227-1_all.deb
GKrellM shows full disk activity for a bit less than 2 minutes, then
no activity, but dpkg is still running. Attaching strace against it
during this time shows lots of rename's:
[...]
rename("/usr/share/texlive/texmf-dist/fonts/tfm/arkandis/berenisadf/ybdr2cijw8y.tfm.dpkg-new", "/usr/share/texlive/texmf-dist/fonts/tfm/arkandis/berenisadf/ybdr2cijw8y.tfm") = 0
openat(AT_FDCWD, "/usr/share/texlive/texmf-dist/fonts/tfm/arkandis/berenisadf/ybdr2ciw8y.tfm.dpkg-new", O_WRONLY) = 11
fsync(11) = 0
close(11) = 0
rename("/usr/share/texlive/texmf-dist/fonts/tfm/arkandis/berenisadf/ybdr2ciw8y.tfm.dpkg-new", "/usr/share/texlive/texmf-dist/fonts/tfm/arkandis/berenisadf/ybdr2ciw8y.tfm") = 0
openat(AT_FDCWD, "/usr/share/texlive/texmf-dist/fonts/tfm/arkandis/berenisadf/ybdr2cj8c.tfm.dpkg-new", O_WRONLY) = 11
fsync(11) = 0
close(11) = 0
rename("/usr/share/texlive/texmf-dist/fonts/tfm/arkandis/berenisadf/ybdr2cj8c.tfm.dpkg-new", "/usr/share/texlive/texmf-dist/fonts/tfm/arkandis/berenisadf/ybdr2cj8c.tfm") = 0
openat(AT_FDCWD, "/usr/share/texlive/texmf-dist/fonts/tfm/arkandis/berenisadf/ybdr2cj8t.tfm.dpkg-new", O_WRONLY) = 11
fsync(11) = 0
close(11) = 0
[...]
> > -- System Information:
> > Debian Release: buster/sid
> > APT prefers unstable-debug
> > APT policy: (500, 'unstable-debug'), (500, 'stable-updates'), (500, 'unstable'), (500, 'testing'), (500, 'stable'), (1, 'experimental')
> > Architecture: amd64 (x86_64)
> >
> > Kernel: Linux 4.19.0-3-amd64 (SMP w/8 CPU cores)
> > Kernel taint flags: TAINT_PROPRIETARY_MODULE, TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
> > Locale: LANG=POSIX, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE=POSIX (charmap=UTF-8)
> > Shell: /bin/sh linked to /bin/dash
> > Init: systemd (via /run/systemd/system)
> > LSM: AppArmor: enabled
>
> I see the kernel is tainted. Perhaps try to unload those and see? :)
Guillem Jover
Mar 5, 2019, 8:00:03 AM3/5/19
to
Hi!
On Sun, 2019-03-03 at 02:38:10 +0100, Vincent Lefevre wrote:
> On 2019-03-03 00:15:13 +0100, Guillem Jover wrote:
> > Did this start happening due to the new kernel version?
>
> In March 2018, this was already slow, but much less:
>
> [...]
> 2018-03-08 00:55:55 status unpacked texlive-fonts-extra:all 2017.20180225-1
> 2018-03-08 00:55:55 status half-installed texlive-fonts-extra:all 2017.20180225-1
> 2018-03-08 01:02:55 status half-installed texlive-fonts-extra:all 2017.20180225-1
> 2018-03-08 01:02:56 status unpacked texlive-fonts-extra:all 2017.20180305-2
> 2018-03-08 01:02:56 status unpacked texlive-fonts-extra:all 2017.20180305-2
> [...]
>
> I don't have older information.
>
> > What filesystem are you using (say btrfs or similar?),
>
> mount information:
>
> /dev/mapper/zira--vg-root on / type ext4 (rw,relatime,errors=remount-ro)
>
> > etc, can you reproduce on another system? I certainly can't. :)
>
> No, on two of my other machines, it takes around 1 minute
> (both for old and new versions). I assume that this is what
> is expected.
Yes. This does really look like a kernel or hardware issue TBH.
Could you try downgrading the kernel to earlier versions to see
whether that improves, that'd give a definitive answer.
I'd say try also older dpkg versions, but I seriously doubt that'd
change much, as I don't remember any major changes in this area
recently.
> > I guess attaching strace to the running dpkg might also give some more
> > info on what's going on, or where it's stalling. Also if you can
> > simply reproduce via «dpkg -i», then enabling all dpkg debug flags
> > might also help.
>
> With
>
> dpkg -i /var/cache/apt/archives/texlive-fonts-extra_2018.20190227-1_all.deb
>
> GKrellM shows full disk activity for a bit less than 2 minutes, then
> no activity, but dpkg is still running. Attaching strace against it
> during this time shows lots of rename's:
>
> [...]
> rename("/usr/share/texlive/texmf-dist/fonts/tfm/arkandis/berenisadf/ybdr2cijw8y.tfm.dpkg-new", "/usr/share/texlive/texmf-dist/fonts/tfm/arkandis/berenisadf/ybdr2cijw8y.tfm") = 0
> openat(AT_FDCWD, "/usr/share/texlive/texmf-dist/fonts/tfm/arkandis/berenisadf/ybdr2ciw8y.tfm.dpkg-new", O_WRONLY) = 11
> fsync(11) = 0
> close(11) = 0
> rename("/usr/share/texlive/texmf-dist/fonts/tfm/arkandis/berenisadf/ybdr2ciw8y.tfm.dpkg-new", "/usr/share/texlive/texmf-dist/fonts/tfm/arkandis/berenisadf/ybdr2ciw8y.tfm") = 0
> openat(AT_FDCWD, "/usr/share/texlive/texmf-dist/fonts/tfm/arkandis/berenisadf/ybdr2cj8c.tfm.dpkg-new", O_WRONLY) = 11
> fsync(11) = 0
> close(11) = 0
> rename("/usr/share/texlive/texmf-dist/fonts/tfm/arkandis/berenisadf/ybdr2cj8c.tfm.dpkg-new", "/usr/share/texlive/texmf-dist/fonts/tfm/arkandis/berenisadf/ybdr2cj8c.tfm") = 0
> openat(AT_FDCWD, "/usr/share/texlive/texmf-dist/fonts/tfm/arkandis/berenisadf/ybdr2cj8t.tfm.dpkg-new", O_WRONLY) = 11
> fsync(11) = 0
> close(11) = 0
> [...]
Thanks. Yes this would be the code doing the delayed fsync(3 ) with
atomic renames(3), which has been the solution to earlier problems
with ext4.
Thanks,
Guillem
On Sun, 2019-03-03 at 02:38:10 +0100, Vincent Lefevre wrote:
> On 2019-03-03 00:15:13 +0100, Guillem Jover wrote:
> > Did this start happening due to the new kernel version?
>
> In March 2018, this was already slow, but much less:
>
> [...]
> 2018-03-08 00:55:55 status unpacked texlive-fonts-extra:all 2017.20180225-1
> 2018-03-08 00:55:55 status half-installed texlive-fonts-extra:all 2017.20180225-1
> 2018-03-08 01:02:55 status half-installed texlive-fonts-extra:all 2017.20180225-1
> 2018-03-08 01:02:56 status unpacked texlive-fonts-extra:all 2017.20180305-2
> 2018-03-08 01:02:56 status unpacked texlive-fonts-extra:all 2017.20180305-2
> [...]
>
> I don't have older information.
>
> > What filesystem are you using (say btrfs or similar?),
>
> mount information:
>
> /dev/mapper/zira--vg-root on / type ext4 (rw,relatime,errors=remount-ro)
>
> > etc, can you reproduce on another system? I certainly can't. :)
>
> No, on two of my other machines, it takes around 1 minute
> (both for old and new versions). I assume that this is what
> is expected.
Could you try downgrading the kernel to earlier versions to see
whether that improves, that'd give a definitive answer.
I'd say try also older dpkg versions, but I seriously doubt that'd
change much, as I don't remember any major changes in this area
recently.
> > I guess attaching strace to the running dpkg might also give some more
> > info on what's going on, or where it's stalling. Also if you can
> > simply reproduce via «dpkg -i», then enabling all dpkg debug flags
> > might also help.
>
> With
>
> dpkg -i /var/cache/apt/archives/texlive-fonts-extra_2018.20190227-1_all.deb
>
> GKrellM shows full disk activity for a bit less than 2 minutes, then
> no activity, but dpkg is still running. Attaching strace against it
> during this time shows lots of rename's:
>
> [...]
> rename("/usr/share/texlive/texmf-dist/fonts/tfm/arkandis/berenisadf/ybdr2cijw8y.tfm.dpkg-new", "/usr/share/texlive/texmf-dist/fonts/tfm/arkandis/berenisadf/ybdr2cijw8y.tfm") = 0
> openat(AT_FDCWD, "/usr/share/texlive/texmf-dist/fonts/tfm/arkandis/berenisadf/ybdr2ciw8y.tfm.dpkg-new", O_WRONLY) = 11
> fsync(11) = 0
> close(11) = 0
> rename("/usr/share/texlive/texmf-dist/fonts/tfm/arkandis/berenisadf/ybdr2ciw8y.tfm.dpkg-new", "/usr/share/texlive/texmf-dist/fonts/tfm/arkandis/berenisadf/ybdr2ciw8y.tfm") = 0
> openat(AT_FDCWD, "/usr/share/texlive/texmf-dist/fonts/tfm/arkandis/berenisadf/ybdr2cj8c.tfm.dpkg-new", O_WRONLY) = 11
> fsync(11) = 0
> close(11) = 0
> rename("/usr/share/texlive/texmf-dist/fonts/tfm/arkandis/berenisadf/ybdr2cj8c.tfm.dpkg-new", "/usr/share/texlive/texmf-dist/fonts/tfm/arkandis/berenisadf/ybdr2cj8c.tfm") = 0
> openat(AT_FDCWD, "/usr/share/texlive/texmf-dist/fonts/tfm/arkandis/berenisadf/ybdr2cj8t.tfm.dpkg-new", O_WRONLY) = 11
> fsync(11) = 0
> close(11) = 0
> [...]
atomic renames(3), which has been the solution to earlier problems
with ext4.
Thanks,
Guillem
Vincent Lefevre
Mar 5, 2019, 9:00:03 AM3/5/19
to
On 2019-03-05 13:52:27 +0100, Guillem Jover wrote:
> Hi!
>
> On Sun, 2019-03-03 at 02:38:10 +0100, Vincent Lefevre wrote:
> > On 2019-03-03 00:15:13 +0100, Guillem Jover wrote:
> > > Did this start happening due to the new kernel version?
> >
> > In March 2018, this was already slow, but much less:
> >
> > [...]
> > 2018-03-08 00:55:55 status unpacked texlive-fonts-extra:all 2017.20180225-1
> > 2018-03-08 00:55:55 status half-installed texlive-fonts-extra:all 2017.20180225-1
> > 2018-03-08 01:02:55 status half-installed texlive-fonts-extra:all 2017.20180225-1
> > 2018-03-08 01:02:56 status unpacked texlive-fonts-extra:all 2017.20180305-2
> > 2018-03-08 01:02:56 status unpacked texlive-fonts-extra:all 2017.20180305-2
> > [...]
> >
> > I don't have older information.
> >
> > > What filesystem are you using (say btrfs or similar?),
> >
> > mount information:
> >
> > /dev/mapper/zira--vg-root on / type ext4 (rw,relatime,errors=remount-ro)
> >
> > > etc, can you reproduce on another system? I certainly can't. :)
> >
> > No, on two of my other machines, it takes around 1 minute
> > (both for old and new versions). I assume that this is what
> > is expected.
>
> Yes. This does really look like a kernel or hardware issue TBH.
> Could you try downgrading the kernel to earlier versions to see
> whether that improves, that'd give a definitive answer.
Same issue (also around 20 minutes) with the kernel from stable:
> Hi!
>
> On Sun, 2019-03-03 at 02:38:10 +0100, Vincent Lefevre wrote:
> > On 2019-03-03 00:15:13 +0100, Guillem Jover wrote:
> > > Did this start happening due to the new kernel version?
> >
> > In March 2018, this was already slow, but much less:
> >
> > [...]
> > 2018-03-08 00:55:55 status unpacked texlive-fonts-extra:all 2017.20180225-1
> > 2018-03-08 00:55:55 status half-installed texlive-fonts-extra:all 2017.20180225-1
> > 2018-03-08 01:02:55 status half-installed texlive-fonts-extra:all 2017.20180225-1
> > 2018-03-08 01:02:56 status unpacked texlive-fonts-extra:all 2017.20180305-2
> > 2018-03-08 01:02:56 status unpacked texlive-fonts-extra:all 2017.20180305-2
> > [...]
> >
> > I don't have older information.
> >
> > > What filesystem are you using (say btrfs or similar?),
> >
> > mount information:
> >
> > /dev/mapper/zira--vg-root on / type ext4 (rw,relatime,errors=remount-ro)
> >
> > > etc, can you reproduce on another system? I certainly can't. :)
> >
> > No, on two of my other machines, it takes around 1 minute
> > (both for old and new versions). I assume that this is what
> > is expected.
>
> Yes. This does really look like a kernel or hardware issue TBH.
> Could you try downgrading the kernel to earlier versions to see
> whether that improves, that'd give a definitive answer.
Linux zira 4.9.0-8-amd64 #1 SMP Debian 4.9.144-3.1 (2019-02-19) x86_64 GNU/Linux
If it's a hardware issue, I'm wondering why I don't have messages
in journalctl output.
Vincent Lefevre
Mar 5, 2019, 9:50:03 AM3/5/19
to
On 2019-03-05 15:43:05 +0100, Vincent Lefevre wrote:
> I've done a test with "iozone -a -e", and with this -e option
> (to include fsync), the operations are much slower than on the
> other machine.
>
> zira:~> iozone -a -e
> [...]
> random random bkwd record stride
> kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread
> 64 4 1821 1987 1922 3703 3657 1770 1621 1628 1697 1707 1773 1879725 2892445
> 64 8 1606 1684 1713 3493 3476 1700 1765 1730 1812 2135 1852 2278628 3406295
> 64 16 1807 1823 1986 3898 3798 1977 1813 1988 1871 2049 3320 2561267 4018152
> 64 32 1702 1714 1755 3494 3683 1741 1841 1796 1737 1633 1674 1879725 4018152
> 64 64 1808 1790 1632 3314 3387 2254 1995 1843 1984 1789 1820 2801873 3203069
> [...]
I meant, except fread and freread.
> I've done a test with "iozone -a -e", and with this -e option
> (to include fsync), the operations are much slower than on the
> other machine.
>
> zira:~> iozone -a -e
> [...]
> random random bkwd record stride
> kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread
> 64 4 1821 1987 1922 3703 3657 1770 1621 1628 1697 1707 1773 1879725 2892445
> 64 8 1606 1684 1713 3493 3476 1700 1765 1730 1812 2135 1852 2278628 3406295
> 64 16 1807 1823 1986 3898 3798 1977 1813 1988 1871 2049 3320 2561267 4018152
> 64 32 1702 1714 1755 3494 3683 1741 1841 1796 1737 1633 1674 1879725 4018152
> 64 64 1808 1790 1632 3314 3387 2254 1995 1843 1984 1789 1820 2801873 3203069
> [...]
I meant, except fread and freread.
Vincent Lefevre
Mar 5, 2019, 9:50:03 AM3/5/19
to
I've done a test with "iozone -a -e", and with this -e option
(to include fsync), the operations are much slower than on the
other machine.
zira:~> iozone -a -e
[...]
random random bkwd record stride
kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread
64 4 1821 1987 1922 3703 3657 1770 1621 1628 1697 1707 1773 1879725 2892445
64 8 1606 1684 1713 3493 3476 1700 1765 1730 1812 2135 1852 2278628 3406295
64 16 1807 1823 1986 3898 3798 1977 1813 1988 1871 2049 3320 2561267 4018152
64 32 1702 1714 1755 3494 3683 1741 1841 1796 1737 1633 1674 1879725 4018152
64 64 1808 1790 1632 3314 3387 2254 1995 1843 1984 1789 1820 2801873 3203069
[...]
(to include fsync), the operations are much slower than on the
other machine.
zira:~> iozone -a -e
[...]
random random bkwd record stride
kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread
64 4 1821 1987 1922 3703 3657 1770 1621 1628 1697 1707 1773 1879725 2892445
64 8 1606 1684 1713 3493 3476 1700 1765 1730 1812 2135 1852 2278628 3406295
64 16 1807 1823 1986 3898 3798 1977 1813 1988 1871 2049 3320 2561267 4018152
64 32 1702 1714 1755 3494 3683 1741 1841 1796 1737 1633 1674 1879725 4018152
64 64 1808 1790 1632 3314 3387 2254 1995 1843 1984 1789 1820 2801873 3203069
[...]
Guillem Jover
Apr 8, 2019, 6:40:02 PM4/8/19
to
Hi!
On Tue, 2019-03-05 at 15:43:05 +0100, Vincent Lefevre wrote:
> I've done a test with "iozone -a -e", and with this -e option
> (to include fsync), the operations are much slower than on the
> other machine.
>
> zira:~> iozone -a -e
> [...]
> random random bkwd record stride
> kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread
> 64 4 1821 1987 1922 3703 3657 1770 1621 1628 1697 1707 1773 1879725 2892445
> 64 8 1606 1684 1713 3493 3476 1700 1765 1730 1812 2135 1852 2278628 3406295
> 64 16 1807 1823 1986 3898 3798 1977 1813 1988 1871 2049 3320 2561267 4018152
> 64 32 1702 1714 1755 3494 3683 1741 1841 1796 1737 1633 1674 1879725 4018152
> 64 64 1808 1790 1632 3314 3387 2254 1995 1843 1984 1789 1820 2801873 3203069
> [...]
Ok, can we conclude this is not a problem in dpkg then? :) It seems to
me either hardware, filesystem or kernel related as mentioned before.
Could you reassign where you think it would be more appropriate?
Thanks,
Guillem
On Tue, 2019-03-05 at 15:43:05 +0100, Vincent Lefevre wrote:
> I've done a test with "iozone -a -e", and with this -e option
> (to include fsync), the operations are much slower than on the
> other machine.
>
> zira:~> iozone -a -e
> [...]
> random random bkwd record stride
> kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread
> 64 4 1821 1987 1922 3703 3657 1770 1621 1628 1697 1707 1773 1879725 2892445
> 64 8 1606 1684 1713 3493 3476 1700 1765 1730 1812 2135 1852 2278628 3406295
> 64 16 1807 1823 1986 3898 3798 1977 1813 1988 1871 2049 3320 2561267 4018152
> 64 32 1702 1714 1755 3494 3683 1741 1841 1796 1737 1633 1674 1879725 4018152
> 64 64 1808 1790 1632 3314 3387 2254 1995 1843 1984 1789 1820 2801873 3203069
> [...]
me either hardware, filesystem or kernel related as mentioned before.
Could you reassign where you think it would be more appropriate?
Thanks,
Guillem
Vincent Lefevre
Apr 9, 2019, 5:40:03 AM4/9/19
to
Control: retitle -1 dpkg: package install can be very slow with some disks due to too frequent fsync
Hi,
On 2019-04-09 00:33:27 +0200, Guillem Jover wrote:
> On Tue, 2019-03-05 at 15:43:05 +0100, Vincent Lefevre wrote:
> > zira:~> iozone -a -e
> > [...]
> > random random bkwd record stride
> > kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread
> > 64 4 1821 1987 1922 3703 3657 1770 1621 1628 1697 1707 1773 1879725 2892445
> > 64 8 1606 1684 1713 3493 3476 1700 1765 1730 1812 2135 1852 2278628 3406295
> > 64 16 1807 1823 1986 3898 3798 1977 1813 1988 1871 2049 3320 2561267 4018152
> > 64 32 1702 1714 1755 3494 3683 1741 1841 1796 1737 1633 1674 1879725 4018152
> > 64 64 1808 1790 1632 3314 3387 2254 1995 1843 1984 1789 1820 2801873 3203069
> > [...]
>
> Ok, can we conclude this is not a problem in dpkg then? :)
I'm not sure. This may just be a slow disk after all, and in practice
(I mean except benchmarking), the problem seems specific to dpkg.
I assume that programs normally don't do a fsync at every fraction
of microsecond! I don't know what dpkg is trying to achieve with such
a frequency of fsync, but this doesn't seem to make sense to me. You
had said "which has been the solution to earlier problems with ext4",
but the real solution would be to fix ext4 (does "earlier" mean this
has eventually been fixed?).
> It seems to me either hardware, filesystem or kernel related as
> mentioned before.
I've seen https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1785020
"fsync is slow on later kernels with ext4 filesystms" but the comments
in this bug page and 4.19.28-1 kernel logs show that this has been
fixed.
FYI, I get very similar timings on the same machine, but on /boot,
which is:
/dev/sda1 on /boot type ext2 (rw,relatime,block_validity,barrier,user_xattr,acl,stripe=4)
The fact that the main, ext4 partition is encrypted but not this
one makes another difference, but the timings are similar, so that
this doesn't seem to be related to the FS system itself and appears
to be at a lower level.
Note also that my machine is a laptop, and I couldn't do a comparison
with other laptops, just in case.
> Could you reassign where you think it would be more appropriate?
I think that this should still be regarded as a dpkg bug, based on
my first paragraph above (the fsync's occur much too often in dpkg
compared to other programs, and this doesn't seem to be useful).
Hi,
On 2019-04-09 00:33:27 +0200, Guillem Jover wrote:
> On Tue, 2019-03-05 at 15:43:05 +0100, Vincent Lefevre wrote:
> > zira:~> iozone -a -e
> > [...]
> > random random bkwd record stride
> > kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread
> > 64 4 1821 1987 1922 3703 3657 1770 1621 1628 1697 1707 1773 1879725 2892445
> > 64 8 1606 1684 1713 3493 3476 1700 1765 1730 1812 2135 1852 2278628 3406295
> > 64 16 1807 1823 1986 3898 3798 1977 1813 1988 1871 2049 3320 2561267 4018152
> > 64 32 1702 1714 1755 3494 3683 1741 1841 1796 1737 1633 1674 1879725 4018152
> > 64 64 1808 1790 1632 3314 3387 2254 1995 1843 1984 1789 1820 2801873 3203069
> > [...]
>
> Ok, can we conclude this is not a problem in dpkg then? :)
(I mean except benchmarking), the problem seems specific to dpkg.
I assume that programs normally don't do a fsync at every fraction
of microsecond! I don't know what dpkg is trying to achieve with such
a frequency of fsync, but this doesn't seem to make sense to me. You
had said "which has been the solution to earlier problems with ext4",
but the real solution would be to fix ext4 (does "earlier" mean this
has eventually been fixed?).
> It seems to me either hardware, filesystem or kernel related as
> mentioned before.
"fsync is slow on later kernels with ext4 filesystms" but the comments
in this bug page and 4.19.28-1 kernel logs show that this has been
fixed.
FYI, I get very similar timings on the same machine, but on /boot,
which is:
/dev/sda1 on /boot type ext2 (rw,relatime,block_validity,barrier,user_xattr,acl,stripe=4)
The fact that the main, ext4 partition is encrypted but not this
one makes another difference, but the timings are similar, so that
this doesn't seem to be related to the FS system itself and appears
to be at a lower level.
Note also that my machine is a laptop, and I couldn't do a comparison
with other laptops, just in case.
> Could you reassign where you think it would be more appropriate?
my first paragraph above (the fsync's occur much too often in dpkg
compared to other programs, and this doesn't seem to be useful).
Joachim Wuttke
Apr 1, 2021, 6:00:05 AM4/1/21
to
To confirm the issue:
Unpacking texlive-fonts-extra (2020.20210202-3) over (2020.20210202-1) ...
is incredibly slow.
I'm running Debian/bullseye; the SSD is ext4 and far from full.
What is the status of this issue?
What are the conclusions after all the above correspondence?
Is there any hope?
- Joachim
Unpacking texlive-fonts-extra (2020.20210202-3) over (2020.20210202-1) ...
is incredibly slow.
I'm running Debian/bullseye; the SSD is ext4 and far from full.
What is the status of this issue?
What are the conclusions after all the above correspondence?
Is there any hope?
- Joachim
Vincent Lefevre
Oct 15, 2022, 7:50:04 AM10/15/22
to
Control: reopen -1
On 2022-10-14 02:26:20 +0200, Guillem Jover wrote:
> All these fsync()s you see in rapid succession are used as a
> synchronization points, way after the data has been requested to be
> synced to disk asynchronously via sync_file_range().
I don't know why, but what strace shows is fsync(), not
sync_file_range(). See strace output at
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=923423#44
So this is not "asynchronously".
> What this is trying to achieve is durability, so that dpkg can know
> the data is on the disk, so that it can mark the package as installed.
I agree that there should be a sync at the end (at least). But here,
there sems to be one for *every* of the 92000 files!
> This is explained on the dpkg FAQ:
>
> <https://wiki.debian.org/Teams/Dpkg/FAQ#Q:_Why_is_dpkg_so_slow_when_using_new_filesystems_such_as_btrfs_or_ext4.3F>
> [...]
> Most programs do not seem concerned about making sure the data is
> stored safely on disk, I'm afraid.
>
> In any case, I don't think there's anything else for dpkg to do here.
> Please see the FAQ entry. I'm thus closing this now.
According to the discussion at bug 1021750, and in particular
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1021750#45
this FAQ is wrong (concerning both the performance and data safety).
If you disagree, please comment there.
Since this is based on incorrect assumptions (wrong FAQ, and sync's
are not asynchronous) and synchronizations could be less frequent
without making data store less safe[*], I'm reopening the bug.
[*] On the opposite, I would tend to think that such frequent
synchronizations tend to yield more writes on the disk, thus more
stress on the hardware, which could make data store less safe.
On 2022-10-14 02:26:20 +0200, Guillem Jover wrote:
> All these fsync()s you see in rapid succession are used as a
> synchronization points, way after the data has been requested to be
> synced to disk asynchronously via sync_file_range().
I don't know why, but what strace shows is fsync(), not
sync_file_range(). See strace output at
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=923423#44
So this is not "asynchronously".
> What this is trying to achieve is durability, so that dpkg can know
> the data is on the disk, so that it can mark the package as installed.
I agree that there should be a sync at the end (at least). But here,
there sems to be one for *every* of the 92000 files!
> This is explained on the dpkg FAQ:
>
> <https://wiki.debian.org/Teams/Dpkg/FAQ#Q:_Why_is_dpkg_so_slow_when_using_new_filesystems_such_as_btrfs_or_ext4.3F>
> [...]
> Most programs do not seem concerned about making sure the data is
> stored safely on disk, I'm afraid.
>
> In any case, I don't think there's anything else for dpkg to do here.
> Please see the FAQ entry. I'm thus closing this now.
According to the discussion at bug 1021750, and in particular
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1021750#45
this FAQ is wrong (concerning both the performance and data safety).
If you disagree, please comment there.
Since this is based on incorrect assumptions (wrong FAQ, and sync's
are not asynchronous) and synchronizations could be less frequent
without making data store less safe[*], I'm reopening the bug.
[*] On the opposite, I would tend to think that such frequent
synchronizations tend to yield more writes on the disk, thus more
stress on the hardware, which could make data store less safe.
0 new messages