SFTP mount is slow

[divv...@gmail.com]

Hi,

I discovered that file access/copying via the new SFTP mount mechanism is extremely slow. Here are some comparative numbers for the same file on the same wifi:

File size: 37.6 MB

RTT to server: 90ms

Native Chrome OS file copying:

Secure Shell mount: 1:22 (0.46MB/s)

Secure Shell via SSH tunnel and HTTP download in Chrome: 1:01 (0.62MB/s)

SFTP File System: 1:07 (0.56MB/s)

Via Android apps:

Total Commander SFTP: 4:18 (0.15MB/s)

Total Commander SCP: 0:24 (1.6MB/s)

GNURoot Debian SFTP: 0:29 (1.3MB/s)

GNURoot Debian SCP: 0:29 (1.3MB/s)

Note that I observe similar issues with most Android SFTP/SCP apps on a Pixel C - for some reason anything except for GNURoot and Total Commander SCP seems to be extremely slow. The Total Commander case makes me question the protocol support - could it be that they are using just a too small block size for SFTP read requests (this is a non-issue for SCP as SCP only ever can read the full file)? The sftp inside Debian defaults to a 32k buffer, which sounds like this limits performance to "32k once every 90ms", i.e. 0.35MB/s - a value that gets exceeded though. Also, SFTP on Debian needs to be told to use a buffer as small as 1024 bytes in order to drop to 0.54MB/s.

Are there any tunables for SFTP/SSH performance? What else can be done about this?

Thank you,

Rudolf

[Mike Frysinger]

we haven't done any real performance tests on the code base.  focus was on correctness/stability, and making sure things were "good enough".  at this point, i would say we're hitting those marks :).

i wouldn't be surprised if there was room for improvement here.  we know we are somewhat limited by the FSP integration as sometimes the Files app requests are not as efficient as they could be.  but for bulk transfers, i don't think that should apply.

there aren't any tunables available today other than whatever the ssh command line and ssh_config options provide.  and i don't think it provides any in this regard.

Secure Shell itself isn't `sftp` ... it's a JS implementation of the SFTP protocol that uses `ssh` for transport.  so we use the buffers that `ssh` uses which, iirc, tend to be tuned for interactive rather than bulk.

so the current status: help wanted ;)

-mike

--
You received this message because you are subscribed to the Google Groups "chromium-hterm" group.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-hterm/24f4d9b4-6edb-48af-945a-c38fa5d13656%40chromium.org.

[Rudolf Polzer]

On May 2, 2017 4:32 PM, "Mike Frysinger" <vap...@chromium.org> wrote:

we haven't done any real performance tests on the code base.  focus was on correctness/stability, and making sure things were "good enough".  at this point, i would say we're hitting those marks :).

i wouldn't be surprised if there was room for improvement here.  we know we are somewhat limited by the FSP integration as sometimes the Files app requests are not as efficient as they could be.  but for bulk transfers, i don't think that should apply.

there aren't any tunables available today other than whatever the ssh command line and ssh_config options provide.  and i don't think it provides any in this regard.

Secure Shell itself isn't `sftp` ... it's a JS implementation of the SFTP protocol that uses `ssh` for transport.  so we use the buffers that `ssh` uses which, iirc, tend to be tuned for interactive rather than bulk.

Well, that sounds good - the part that I mainly suspect is the request size of the read requests, not the SSH-level transport buffers (my HTTP example shows that these aren't quite optimal either, but that may be something for me to research regarding the SSH command line).

Do you happen to know where the read request size is coming from - is the SFTP mount always using whatever request size the underlying read request from the OS was using (e.g. the Files app), or are request sizes changed/reduced to fit some internal buffers/limits?

If not, I'll do my own digging.

If the SFTP protocol implementation does something with the request sizes, that may be possible to change - if it doesn't but just forwards the OS-provided sizes, changes to the Files app (to copy files using a larger block size) as well as some other apps (e.g. the media player) may be advisable. This, or a caching layer could be put inside the SFTP mount support.

[divv...@gmail.com]

On Tuesday, May 2, 2017 at 6:17:37 PM UTC-4, Rudolf Polzer wrote:

On May 2, 2017 4:32 PM, "Mike Frysinger" wrote:

we haven't done any real performance tests on the code base.  focus was on correctness/stability, and making sure things were "good enough".  at this point, i would say we're hitting those marks :).

i wouldn't be surprised if there was room for improvement here.  we know we are somewhat limited by the FSP integration as sometimes the Files app requests are not as efficient as they could be.  but for bulk transfers, i don't think that should apply.

there aren't any tunables available today other than whatever the ssh command line and ssh_config options provide.  and i don't think it provides any in this regard.

Secure Shell itself isn't `sftp` ... it's a JS implementation of the SFTP protocol that uses `ssh` for transport.  so we use the buffers that `ssh` uses which, iirc, tend to be tuned for interactive rather than bulk.

Well, that sounds good - the part that I mainly suspect is the request size of the read requests, not the SSH-level transport buffers (my HTTP example shows that these aren't quite optimal either, but that may be something for me to research regarding the SSH command line).

Do you happen to know where the read request size is coming from - is the SFTP mount always using whatever request size the underlying read request from the OS was using (e.g. the Files app), or are request sizes changed/reduced to fit some internal buffers/limits?

If not, I'll do my own digging.

If the SFTP protocol implementation does something with the request sizes, that may be possible to change - if it doesn't but just forwards the OS-provided sizes, changes to the Files app (to copy files using a larger block size) as well as some other apps (e.g. the media player) may be advisable. This, or a caching layer could be put inside the SFTP mount support.

I did find one thing: read requests are split into nassh.sftp.fsp.DATA_LIMIT (default 32k) sized chunks and resolved in parallel as asynchronous promises. This parameter can be changed at runtime from the JS console and applies directly during the read requests. Still this is potentially relevant, as the request size used by ChromeOS is in the multi-MB range (good).

This splitting probably doesn't improve performance, as the SFTP protocol itself does not really support parallel requests (unless you open multiple channels, which I doubt we're doing) - it may rather be intended to improve interactivity so a copy operation doesn't make everything wait for whole megabytes to be transferred. Nevertheless I'll try playing with this parameter and seeing what happens.

[Bajza Ferenc]

I wrote an article about this, is it helpful for you?
https://feriman.com/configure-scp-sftp-with-total-commander-step-by-step-guide/