Getting error "rsync error: error in rsync protocol data stream (code 12)" when using rsnapshot

[Michael Mell]

I am trying to setup my CH3SNAS (hardware identical to the DNS-323-B1) to run at my step-parents house and perform off-site backups for my NAS, which I have at home.
The device is runnig Alt-F 1.0.
I am setting up rsnapshot to automatically connect and copy the changes from my NAS.
The data is quite large amounting to roughly 2.5TB, which it needs to scan for changes and copy the changes.

Unfortunately I am getting the following error during synchronization:

2017/10/31 20:43:27 [2199] rsync: connection unexpectedly closed (13261078 bytes received so far) [generator]
2017/10/31 20:43:27 [2199] rsync error: error in rsync protocol data stream (code 12) at io.c(226) [generator=3.1.2]

when running this command:

/usr/bin/rsync -avn --delete  --log-file=/mnt/vg_bkp_disks-lv_bkp_data/log/rsync.log --numeric-ids --relative --delete-excluded \
    --rsh="/usr/bin/ssh -i /etc/ssh_keys/id_rsa" \
    ro...@192.168.13.50:/media/data_volume \
    /mnt/vg_bkp_disks-lv_bkp_data/backups/daily.0/origin/ \
    > /mnt/vg_bkp_disks-lv_bkp_data/log/rsync_synced_files.log

This is essentially rsync command as created by rsnapshot, which I extended with -avn (for a verbose dry-run) and logging.

Furthermore I appear to be running into memory issues which might be the cause for rsync being killed by the kernel (see attached dmesg-dump).

SSH is definitely working as the output lists new files and it appeared to be copying data correctly in past runs - even though I got the error (obviously not performing a dry-run).
However it always produces this error.

So I have two questions:

1. Has anyone ever tried something similar using the this hardware (the CH3SNAS or DNS-323-B1))? I know how anemic this hardware is and looking at the dmesg output, it seems that my woes are being caused by the low amount of RAM in this device -- which I believe is only 64MB. If this were the case there is really no point in continuing my endeavor.
2. Is there anything else I can try to debug and better understand this problem?

I would be thankful for any suggestions or reports on experiences with regards to trying something similar.

Thanks and best regards,

Michael

[Joao Cardoso]

On Tuesday, October 31, 2017 at 9:10:43 PM UTC, Michael Mell wrote:

I am trying to setup my CH3SNAS (hardware identical to the DNS-323-B1) to run at my step-parents house and perform off-site backups for my NAS, which I have at home.
The device is runnig Alt-F 1.0.
I am setting up rsnapshot to automatically connect and copy the changes from my NAS.
The data is quite large amounting to roughly 2.5TB, which it needs to scan for changes and copy the changes.

Unfortunately I am getting the following error during synchronization:

2017/10/31 20:43:27 [2199] rsync: connection unexpectedly closed (13261078 bytes received so far) [generator]
2017/10/31 20:43:27 [2199] rsync error: error in rsync protocol data stream (code 12) at io.c(226) [generator=3.1.2]

when running this command:

/usr/bin/rsync -avn --delete  --log-file=/mnt/vg_bkp_disks-lv_bkp_data/log/rsync.log --numeric-ids --relative --delete-excluded \
    --rsh="/usr/bin/ssh -i /etc/ssh_keys/id_rsa" \
    ro...@192.168.13.50:/media/data_volume \
    /mnt/vg_bkp_disks-lv_bkp_data/backups/daily.0/origin/ \
    > /mnt/vg_bkp_disks-lv_bkp_data/log/rsync_synced_files.log

This is essentially rsync command as created by rsnapshot, which I extended with -avn (for a verbose dry-run) and logging.

Furthermore I appear to be running into memory issues which might be the cause for rsync being killed by the kernel (see attached dmesg-dump).

Yes, rsync is being killed by the kernel because of lack of memory. And rsync is not even using many memory (~15MB)

You don't seem to have swap active (on internal disks), that could alleviate the problem.

You can do several things:

-add and activate a swap partition on an internal disk (rsync might not be killed but it will be slower),

-split the copy using subfolders,

-create a split-rsync wrapper, see https://groups.google.com/d/msg/alt-f/m78yQhGIxlI/a4HYgmC4AQAJ

[Michael Mell]

Hi Joao,

after writing to you I also realized that I did not have an active swap partition - I thought I had set it up that way.
After activating it I re-ran that rsync-process last night and it did not get killed and ran through to the end (using the dry-run option).
I am presently rerunning it without the dry-run option to see, if everything works as expected now - I will update, once I have more information.

Thank you for your always quick replies (and Alt-F) and best regards,
Michael

[Michael Mell]

I can confirm that it works correctly now. So the swap partition did the trick - as one would expect with issues due to limited memory issues.
Somehow I thought I had setup a swap during initial setup of Alt-F.
Oh. Well a couple of hours wasted ...

Thanks again for all your great work on Alf-F!!!

Best regards,

Michael

On Tuesday, October 31, 2017 at 10:10:43 PM UTC+1, Michael Mell wrote: