rsync speeds

[Nicolas Richard (Neko)]

i have a few dns323 running alt f , 

on a version C drive to drive rsync is avg 5~ 8MB/s

receiving drive is a 500mb cache 14tb WD 7200RPM

sending drive is a 64mb cache 3tb WD 7200RPM

 

if i rsync across my network ( 1000 MB wired , but 1500MTU not jumboframe) i get worser

speed  500KB/s  rsync to a 14 tb from an identical 3tb...

now if i ftp paintext and without encryption i get at least a 15MB/s  .

anyways i can crank the rsync speeds like id get in ftp otherwise instead of taking 36hours to transfer 3tb i takes 8 days.  drives SMART table show no relocated or whatsoever bad tresh at all.

using these curselines ;D

using rsync -avW --numeric-ids --info=progress2 /mnt/ /mnt/

using rsync -avW --numeric-ids --info=progress2 credential@othernasip:/mnt/ /mnt/

cheers !

neko

 

[Nicolas Richard (Neko)]

sorry for the typo further down the post : CAN I , CAN I GET BETTER SPEED ( not i can.. sorry bout that)

thank you

Cheers !

Neko

[Nicolas Richard (Neko)]

 rsync -avW --numeric-ids --info=progress2 /mnt/sdb2/  /mnt/sda2/

is faster than

rsync -avW --numeric-ids --info=progress2 credential@othernasip:/mnt/sdb2 /mnt/sda2/

i cant seem to run

rsync -avW --numeric-ids --info=progress2 rsync://credential@othernasip:/mnt/sdb2/ /mnt/sda2/

or any other plaintext transfer other than ftp

[João Cardoso]

rsync uses encryption when syncing through the network, and no encryption for local syncs.

So you have to specify a light enough cipher for dropbear or ssh (if you have installed openssl).
I remember using "arc"? (or even "none"?) for making scp faster, which it did, but it is now considered insecure and disabled.

[João Cardoso]

Instead of using ssh, which uses encryption and is thus slow, rsh could be used instead.

Alternatively to ssh, you can use a rsync daemon at the other network end. As it doesn't use encryption it is faster than ssh.

If using ssh, a "light" and supported cipher for dropbear ssh is chacha20:

   [root@DNS-327L]# dbclient  -c help
   dbclient: Available ciphers: chacha20...@openssh.com,aes128-ctr,aes256-ctr

for openssh ssh chacha20 is also available:

   [root@DNS-327L]# ssh -Q cipher

   ...
   chacha20...@openssh.com

chacha20 is up to three times faster than aes256 on processors without aes instructions, having an equivalent encryption strength.

Setting up CESA hardware accelerators for use with ssh with a aes cipher is another possibility, but it is very complex to do.

[neko]

I dont understand where to add this when imho its should be in the rsync commandline ...

Since this is running a 15y old discontinued debian , mind explaining where exactly to add this in all this old acheology ?

Cheers

--
You received this message because you are subscribed to the Google Groups "Alt-F" group.
To unsubscribe from this group and stop receiving emails from it, send an email to alt-f+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/alt-f/dcaa8e8a-a68e-4bd8-a02a-194f0f2c465cn%40googlegroups.com.

[João Cardoso]

On Sunday, January 14, 2024 at 6:18:35 PM UTC cutte...@gmail.com wrote:

I dont understand where to add this when imho its should be in the rsync commandline ...

use the '-e ssh -c <cipher>' rsync option, where <cipher> can be obtained

For openssh "ssh":

[root@DNS-327L]# ssh -Q ciphers
3des-cbc
aes128-cbc
aes192-cbc
aes256-cbc
rijnda...@lysator.liu.se
aes128-ctr
aes192-ctr
aes256-ctr
aes12...@openssh.com
aes25...@openssh.com
chacha20...@openssh.com

For dropbear "ssh":

[root@DNS-327L]# dbclient -c help
dbclient: Available ciphers: chacha20...@openssh.com,aes128-ctr,aes256-ctr

Don't understand why you say that you couldn't use a rsync daemon, I have no issues. Have you update packages? Go to Packages->Alt-F and "update all"

Using a resync daemon server (that you have to set up) is the faster way, as it by default uses no encryption.

Here are some figures I get:

--Asking for rsync modules on a dns-323 from a linux PC  -- a rsync daemon should be running on the dns-323, see Services->Network, rsyncs, Configure

> rsync dns-323::
Public-RO       Public Area for reading only
Public-RW       Public Area for reading and writing
Backup          Backup area
Root            For development only

--A transfer reading a file from a rsync daemon module on a DNS-323 to a linux PC: 12,3 MBps
> rsync -av dns-323::Public-RW/The.Martian.mp4 tmp
receiving incremental file list
The.Martian.mp4

sent 43 bytes  received 239,854,862 bytes  12,300,251.54 bytes/sec
total size is 239,796,224  speedup is 1.00


--The same, but using the default ssh transfer mode: 1,8MBps:

> rsync -av dns-323:/Public/RW/The.Martian.mp4 tmp
receiving incremental file list
The.Martian.mp4

sent 43 bytes  received 239,854,876 bytes  1,796,666.06 bytes/sec
total size is 239,796,224  speedup is 1.00

-- The same, using a aes128-ctr cipher: 1,8MBps
> rsync -av -e "ssh -c aes128-ctr" dns-323:/Public/RW/The.Martian.mp4 tmp
receiving incremental file list
The.Martian.mp4

sent 43 bytes  received 239,854,876 bytes  1,810,225.80 bytes/sec
total size is 239,796,224  speedup is 1.00

-- the same using chacha20 ciper: 3,2MBs:
> rsync -av -e "ssh -c chacha20...@openssh.com" dns-323:/Public/RW/The.Marti
an.mp4 tmp
receiving incremental file list
The.Martian.mp4

sent 43 bytes  received 239,854,876 bytes  3,219,529.11 bytes/sec
total size is 239,796,224  speedup is 1.00

 

Since this is running a 15y old discontinued debian , mind explaining where exactly to add this in all this old acheology ?

Can't understand. Where is the Debian  install running? On a linux PC? On a DNS-323? Do you want to rsync from or to the the Debiand install on a PC? Aren't you using Alt-F rsync, ssh amd dropbear?

[neko]

Altf rubs an old debian right ?  Not all the tools of altf linux offers the same as what manuals says today. Hense me calling this old archeology. 

You seems to confuse you but this couldnt be simpler.

I log to any of my dns 323 ,  using putty. They all run alt F.  

Box one and two rev C.  Box tree is rev B. Dns 323.    They are wired 1000mb.  1500mtu.

I normally use ftp but since im consolidating all 14tb into a 14 tb drive , using rsync because i want to see and skip errors,  not wing it blindly using cp.    

Now rsync speeds sucks at less than 6mb/s.  While i get at least double.while ftp plaintext.

All this needs no security as its running in a bunker under a lake, not even connected to the internet. 

I ask for tricks to speedup rsync as would ftp run without encryption, as alt.f default are the oposite of what im looking for.  I just want to knock out encryption from rsync service , but manuals dont cover alt F.  Hense me saying its palezoitic archeology to get the info.  

So im asking the 2 folks that know how to solve this in the world. 

Im 18 days down the copy so im almost done.  Id love to simply have resolve the salt/encryption slowdown as it wouldve been done copy like 6 day ago

N

To view this discussion on the web visit https://groups.google.com/d/msgid/alt-f/e55c4b40-608d-4c9b-b60d-27fa5d6c2cdan%40googlegroups.com.