Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Bug#994789: rsync: Errors out with "deflate on token returned 0 (8864 bytes left)" when tranferring 3GB file
2,213 views
Skip to first unread message
Faheem Mitha
Sep 20, 2021, 5:20:03 PM9/20/21
to
Package: rsync
Version: 3.2.3-7
Severity: normal
Dear Maintainer,
After upgrading to Debian 11 (bullseye), I started seeing a problem
when running the following backup command
rsync -abvz --partial -e ssh /var/spool/mail/faheem faheem@servername:Mail/faheem
With this command `rsync` returns
sending incremental file list
faheem
deflate on token returned 0 (8864 bytes left)
rsync error: error in rsync protocol data stream (code 12) at token.c(476) [sender=3.2.3]
The file `/var/spool/mail/faheem` is approximately 3GB.
ls -lah /var/spool/mail/faheem
-rw-rw---- 1 faheem mail 3.1G Sep 21 02:15 /var/spool/mail/faheem
I can confirm that the file is not correctly updated.
I didn't see this issue with the `rsync` version in the previous
release, Debian 10 (buster), namely `3.1.3-6`. I also don't see this
issue with smaller files. Any pointers or suggestions for debugging
appreciated.
Regards, Faheem Mitha
-- System Information:
Debian Release: 11.0
APT prefers stable-updates
APT policy: (500, 'stable-updates'), (500, 'stable-security'), (500, 'oldstable-updates'), (500, 'oldstable-debug'), (500, 'stable'), (500, 'oldstable'), (50, 'unstable'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386
Kernel: Linux 5.10.0-8-amd64 (SMP w/6 CPU threads)
Kernel taint flags: TAINT_PROPRIETARY_MODULE, TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE=en_US:en
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
Versions of packages rsync depends on:
ii init-system-helpers 1.60
ii libacl1 2.2.53-10
ii libc6 2.31-13
ii liblz4-1 1.9.3-2
ii libpopt0 1.18-2
ii libssl1.1 1.1.1k-1+deb11u1
ii libxxhash0 0.8.0-2
ii libzstd1 1.4.8+dfsg-2.1
ii lsb-base 11.1.0
ii zlib1g 1:1.2.11.dfsg-2
rsync recommends no packages.
Versions of packages rsync suggests:
ii openssh-client 1:8.4p1-5
ii openssh-server 1:8.4p1-5
ii python3 3.9.2-3
-- debconf-show failed
Version: 3.2.3-7
Severity: normal
Dear Maintainer,
After upgrading to Debian 11 (bullseye), I started seeing a problem
when running the following backup command
rsync -abvz --partial -e ssh /var/spool/mail/faheem faheem@servername:Mail/faheem
With this command `rsync` returns
sending incremental file list
faheem
deflate on token returned 0 (8864 bytes left)
rsync error: error in rsync protocol data stream (code 12) at token.c(476) [sender=3.2.3]
The file `/var/spool/mail/faheem` is approximately 3GB.
ls -lah /var/spool/mail/faheem
-rw-rw---- 1 faheem mail 3.1G Sep 21 02:15 /var/spool/mail/faheem
I can confirm that the file is not correctly updated.
I didn't see this issue with the `rsync` version in the previous
release, Debian 10 (buster), namely `3.1.3-6`. I also don't see this
issue with smaller files. Any pointers or suggestions for debugging
appreciated.
Regards, Faheem Mitha
-- System Information:
Debian Release: 11.0
APT prefers stable-updates
APT policy: (500, 'stable-updates'), (500, 'stable-security'), (500, 'oldstable-updates'), (500, 'oldstable-debug'), (500, 'stable'), (500, 'oldstable'), (50, 'unstable'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386
Kernel: Linux 5.10.0-8-amd64 (SMP w/6 CPU threads)
Kernel taint flags: TAINT_PROPRIETARY_MODULE, TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE=en_US:en
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
Versions of packages rsync depends on:
ii init-system-helpers 1.60
ii libacl1 2.2.53-10
ii libc6 2.31-13
ii liblz4-1 1.9.3-2
ii libpopt0 1.18-2
ii libssl1.1 1.1.1k-1+deb11u1
ii libxxhash0 0.8.0-2
ii libzstd1 1.4.8+dfsg-2.1
ii lsb-base 11.1.0
ii zlib1g 1:1.2.11.dfsg-2
rsync recommends no packages.
Versions of packages rsync suggests:
ii openssh-client 1:8.4p1-5
ii openssh-server 1:8.4p1-5
ii python3 3.9.2-3
-- debconf-show failed
Samuel Henrique
Sep 24, 2021, 4:10:03 PM9/24/21
to
Hello Faheem, thank you for the bug report :)
I can see you're transferring to another host, this issue could be
related to the target's rsync version/distribution.
Can you share some details around what you have running on the other
end? I can see that the sender has 3.2.3.
It would also be a good idea to try to reproduce the issue with the
parameters "-vvv", which will provide a verbose output. The root cause
might be showing up there.
I think this is a good starting point in the investigation.
Thank you,
--
Samuel Henrique <samueloph>
I can see you're transferring to another host, this issue could be
related to the target's rsync version/distribution.
Can you share some details around what you have running on the other
end? I can see that the sender has 3.2.3.
It would also be a good idea to try to reproduce the issue with the
parameters "-vvv", which will provide a verbose output. The root cause
might be showing up there.
I think this is a good starting point in the investigation.
Thank you,
--
Samuel Henrique <samueloph>
Faheem Mitha
Sep 24, 2021, 4:40:03 PM9/24/21
to
Hi Samuel,
priors, I was not expecting to get a reply.
The problem is that I'm connecting to a remote VPS, which is an OpenVZ
VM, and currently runs a badly outdated version of Debian, Debian 8
(jessie). It was running the default version of rsync for that release
(3.1.1-3+deb8u2), which appears to be too old to do the handshake that
rsync expects for compression. I'm going by what is in the manual,
i.e.
Rsync supports multiple compression methods and will choose one
for you unless you force the choice using the --compress-choice
(--zc) option.
Run rsync --version to see the default compress list compiled
into your version.
When both sides of the transfer are at least 3.2.0, rsync chooses
the first algorithm in the client's list of choices that is also
in the server's list of choices. If no common compress choice is
found, rsync exits with an error. If the remote rsync is too old
to support checksum negotiation, its list is assumed to be
"zlib".
However, as an error message, this really sucked. Unless the rsync
maintainers consider compatibility with older rsync versions not worth
bothering with, it would be good to have a more intelligible error
message. But I don't know if it is worth trying to follow up with the
rsync project.
I managed to backport the current version of rsync to Debian 8 on the
OpenVZ VPS, and the error went away. Other workarounds I tried (like
--compress-choice) all failed, because the rsync version seemed to be
too old.
The output of `apt-cache policy rsync` on that server now looks like
rsync:
Installed: 3.2.3-7
Candidate: 3.1.1-3+deb8u2
Package pin: (not found)
Version table:
3.2.3-7 1001
50 http://ftp.us.debian.org/debian/ testing/main amd64 Packages
50 http://ftp.us.debian.org/debian/ unstable/main amd64 Packages
*** 3.2.3-7 1001
100 /var/lib/dpkg/status
3.1.1-3+deb8u2 1001
500 http://security.debian.org/ jessie/updates/main amd64 Packages
3.1.1-3+deb8u1 1001
500 http://ftp.us.debian.org/debian/ jessie/main amd64 Packages
If you don't think it's worth following up upstream regarding the
quality of their error messages, you can close this bug report.
Regards, Faheem Mitha
Samuel Henrique
Sep 25, 2021, 10:30:03 AM9/25/21
to
Hello Faheem,
> I did some investigation, but forgot to update the bug report. Based on
> priors, I was not expecting to get a reply.
Yeah, sometimes it's hard to keep up with the bug reports, thanks for
not giving up!
> The problem is that I'm connecting to a remote VPS, which is an OpenVZ
> VM, and currently runs a badly outdated version of Debian, Debian 8
> (jessie). It was running the default version of rsync for that release
> (3.1.1-3+deb8u2), which appears to be too old to do the handshake that
> rsync expects for compression.
Wow, that is quite outdated yes, at this point the only support there
is for Jessie is the paid one by the Debian LTS project[0][1].
Since you're a customer, I would kindly suggest cutting a ticket to
them, asking for an updated release.
> However, as an error message, this really sucked. Unless the rsync
> maintainers consider compatibility with older rsync versions not worth
> bothering with, it would be good to have a more intelligible error
> message. But I don't know if it is worth trying to follow up with the
> rsync project.
My experience with rsync upstream is that they would improve the error
message if possible, so it might be worth reporting it.
> I managed to backport the current version of rsync to Debian 8 on the
> OpenVZ VPS, and the error went away. Other workarounds I tried (like
> --compress-choice) all failed, because the rsync version seemed to be
> too old.
Happy to know your workaround fixed the issue, I always try to upload
an updated rsync release to the backports repository as well, but that
didn't happen with Jessie
> The output of `apt-cache policy rsync` on that server now looks like
>
> rsync:
> Installed: 3.2.3-7
> Candidate: 3.1.1-3+deb8u2
> Package pin: (not found)
> Version table:
> 3.2.3-7 1001
> 50 http://ftp.us.debian.org/debian/ testing/main amd64 Packages
> 50 http://ftp.us.debian.org/debian/ unstable/main amd64 Packages
> *** 3.2.3-7 1001
> 100 /var/lib/dpkg/status
> 3.1.1-3+deb8u2 1001
> 500 http://security.debian.org/ jessie/updates/main amd64 Packages
> 3.1.1-3+deb8u1 1001
> 500 http://ftp.us.debian.org/debian/ jessie/main amd64 Packages
By looking at these, it seems like your VPS provider is not using ELTS
so it's possible they're not updating their images for more than a
year now. However, this is just a guess, I suggest contacting them and
explaining your concerns with outdated images.
> If you don't think it's worth following up upstream regarding the
> quality of their error messages, you can close this bug report.
I think it's worth it, but I will let this up for you, so I'm
resolving this bug report.
[0] https://wiki.debian.org/LTS/Extended
[1] https://deb.freexian.com/extended-lts/
> I did some investigation, but forgot to update the bug report. Based on
> priors, I was not expecting to get a reply.
not giving up!
> The problem is that I'm connecting to a remote VPS, which is an OpenVZ
> VM, and currently runs a badly outdated version of Debian, Debian 8
> (jessie). It was running the default version of rsync for that release
> (3.1.1-3+deb8u2), which appears to be too old to do the handshake that
> rsync expects for compression.
is for Jessie is the paid one by the Debian LTS project[0][1].
Since you're a customer, I would kindly suggest cutting a ticket to
them, asking for an updated release.
> However, as an error message, this really sucked. Unless the rsync
> maintainers consider compatibility with older rsync versions not worth
> bothering with, it would be good to have a more intelligible error
> message. But I don't know if it is worth trying to follow up with the
> rsync project.
message if possible, so it might be worth reporting it.
> I managed to backport the current version of rsync to Debian 8 on the
> OpenVZ VPS, and the error went away. Other workarounds I tried (like
> --compress-choice) all failed, because the rsync version seemed to be
> too old.
an updated rsync release to the backports repository as well, but that
didn't happen with Jessie
> The output of `apt-cache policy rsync` on that server now looks like
>
> rsync:
> Installed: 3.2.3-7
> Candidate: 3.1.1-3+deb8u2
> Package pin: (not found)
> Version table:
> 3.2.3-7 1001
> 50 http://ftp.us.debian.org/debian/ testing/main amd64 Packages
> 50 http://ftp.us.debian.org/debian/ unstable/main amd64 Packages
> *** 3.2.3-7 1001
> 100 /var/lib/dpkg/status
> 3.1.1-3+deb8u2 1001
> 500 http://security.debian.org/ jessie/updates/main amd64 Packages
> 3.1.1-3+deb8u1 1001
> 500 http://ftp.us.debian.org/debian/ jessie/main amd64 Packages
so it's possible they're not updating their images for more than a
year now. However, this is just a guess, I suggest contacting them and
explaining your concerns with outdated images.
> If you don't think it's worth following up upstream regarding the
> quality of their error messages, you can close this bug report.
resolving this bug report.
[0] https://wiki.debian.org/LTS/Extended
[1] https://deb.freexian.com/extended-lts/
0 new messages