Finder browsing is slow with sshfs

[Jej]

Hi,

I am quite newbie to Mac Os and the first thing I installed was
macfuse! (too nostalgic of the kio kde layer...) In fact I was very
disappointed to see that the finder browsing get really slow on a
sshfs drive. As this does not append when browsing in non apple native
apps (jEdit or shell), I suspected some finder issues and not the
sshfs or bandwidth themself... I did find any thread on that, but
maybe that's because I miss some keywords in this new apple world ;)

Here are the options I used to get it working like a charm. Probably
the lonely needed is noappledouble but I noticed some improvements
with the others.

sshfs -o no_readahead,noappledouble,nolocalcaches user@server:. /mount/
point

Just add "-o no_readahead,noappledouble,nolocalcaches" in the extra
options field if you use MacFusion. Maybe this should be default
options.

Hope it helps,
Jej

[shadow...@gmail.com]

HOLY CRAP.
Godsend. Jej- thank you. This works beautifully - back to being nice
and speedy like it was on tiger. I was sure it was a Leopard finder
issue!
Thanks again!

ShaBot

[shai]

THANK YOU,, it worked 100%%%%

[shadow...@gmail.com]

After using this for awhile I had to remove the most beneficial of
these options the nodoubleapple ):
It throws up a pair of errors every time I try to copy a file over
complaining about permissions and other things.
It's really friggin annoying ):<
If anyone has a solution I'd be very grateful.

[Graham Perrin]

Re <http://code.google.com/p/macfuse/wiki/OPTIONS>

Whilst noapplexattr and nodoubleapple options exist, requirements such
as

# disallow Finder information

are fairly _extraordinary_.

The effect of options on _ordinary_ applications such as Finder should
be considere -- carefully -- before applying the options.

----

On 19 Apr 2008, at 00:09, shadow...@gmail.com wrote:

> After using this for awhile I had to remove the most beneficial of
> these options the nodoubleapple ):
>
> It throws up a pair of errors every time I try to copy a file over
> complaining about permissions and other things.

<http://code.google.com/p/macfuse/wiki/CHANGELOG> for MacFUSE 0.2.2 and
<http://groups.google.com/group/macfuse-devel/msg/64887b62fc9327bd>
caution:

### 'noapplespecial' … in case somebody wants to experiment …
### Use at your own peril.

## using 'noapplexattr' or 'noappledouble' is likely to
## make Apple applications *very* unhappy.

Highlights from <http://code.google.com/p/macfuse/wiki/FAQ>:

# MacFUSE itself isn't a distributed remote file system!

# nontrivial user-space file systems can still be complex, and
# their performance/behavior can depend upon numerous factors
# besides MacFUSE itself.

# The quality and behavior of individual programs--of
# any kind--can vary wildly

The qualities of Finder, and of numerous other Apple and third party
applications for Mac OS X, require (or benefit from) a file system
that supports extended attributes.

Mac OS Extended Format (HFS+) is the prime example of a suitable file
system.

Apple Filing Protocol (AFP)
<http://developer.apple.com/documentation/Networking/Conceptual/AFP/>
is the prime example of a protocol through which users of Mac OS X
work with a remote file system that _does_ support extended attributes.

If with Mac OS X we choose to use a file system that does _not_
support extended attributes, then:

# the kernel [creates ._ files]. It's the kernel that determines
# whether a volume doesn't support extended attributes, and if so,
# it falls back to an alternate code path that
# creates/deletes/reads/writes ._ files.

- <http://groups.google.com/group/macfuse-devel/msg/fb52cc6dba0c08b8>

Hint: FTP and SFTP are not file systems, CurlFtpFS and SSHFS are not
distributed remote file systems ;)

CurlFtpFS <http://curlftpfs.sourceforge.net/> and
SSHFS <http://fuse.sourceforge.net/sshfs.html> are user space file
systems and when we use (for example) SSHFS, it's guaranteed that we
will encounter limitations of SFTP, for example

# doesn't support reporting free space properly

# not possible to display proper usage

# Why does SVN (etc...) fail to rename files?

- <http://fuse.sourceforge.net/wiki/index.php/SshfsFaq>

NOW: if we stretch things even further -- if we disallow operations
that might be required by Finder or by the kernel -- then in my
opinion, we're asking for trouble.

Realistically: we can not _disallow_ things that are fundamental to an
application *and then* continue to expect good behaviour (or
meaningful error messages) from that application.

----

Returning to the source of this thread:

>> I was very disappointed to see that the finder browsing get really
>> slow on a sshfs drive.

> If anyone has a solution I'd be very grateful.

Focus on other factors that may contributory to the performance issue.
In no particular order, and not comprehensively:

* network/Internet bandwidth
* performance of the SFTP server
* SFTP service implementation.

Regarding the latter, it's not unknown for servers to behave poorly or
crash in response to certain use cases.

* Optimistically: client-side debugging may present a service
description and version number, and we may *imagine* something simple
behind the scenes, and so performance issues may be too easily blamed
on the most up-front parts of the whole caboodle: MacFusion, Finder
and so on.

* However: server set-ups may be more complex/tricky than we
imagine. See for example
<http://code.google.com/p/macfuse/issues/detail?id=159#c28>.

I do understand users' frustrations but the big picture, as I see it,
is:

* Mac OS X, Finder etc. offer very rich experiences

* for working with remote files: I can't imagine FTP, CurlFtpFS,
SFTP or SSHFS coming close to Apple's AFP levels of support for that
richness

* we can't have our cake and eat it ;)

Or, can we? …

Best regards
Graham

[Michael Gorbach]

Yes, finder does not handle being cut off in terms of xattrs very well.
The real solution to this problem is to cache cache xattrs in a
database, which has already been implemented in expandrive and they
get great performance. This can be done as layer _on_top_ of arbitrary
FUSE filesystems, and this will make a world of difference for any
network FS. It is a significant project, but much needed for Mac FUSE
community.

~ Michael Gorbach

[shadow...@gmail.com]

So... it's understood it's much needed. I get that. I'm sure that
you'll accept help from users, but are the lead devs (you guys) going
to be targeting it at all? I mean, do you have any plans currently to
build this layer?
Thanks for the replies, very helpful.

ShaBot

On Apr 19, 11:38 am, Michael Gorbach <mgorb...@gmail.com> wrote:
> Yes, finder does not handle being cut off in terms of xattrs very well.
> The real solution to this problem is to cache cache xattrs in a  
> database, which has already been implemented in expandrive and they  
> get great performance. This can be done as layer _on_top_ of arbitrary  
> FUSE filesystems, and this will make a world of difference for any  
> network FS. It is a significant project, but much needed for Mac FUSE  
> community.
>
> ~ Michael Gorbach
>
> On Apr 19, 2008, at 5:08 AM, Graham Perrin wrote:
>
>
>
> > Re <http://code.google.com/p/macfuse/wiki/OPTIONS>
>
> > Whilst noapplexattr and nodoubleapple options exist, requirements such
> > as
>
> > # disallow Finder information
>
> > are fairly _extraordinary_.
>
> > The effect of options on _ordinary_ applications such as Finder should
> > be considere -- carefully -- before applying the options.
>
> > ----
>

[Michael Gorbach]

ShaBot,
In terms of MacFusion, outside of the area of plugins, I am currently
really the only developer. Right now, my priority is to get Macfusion2
released. It is almost done, but I'm bogged down until mid-May due to
school. After Macfusion2 is released, I would be interested in working
on a such a solution. This is the key to getting good performance out
of any network filesystem. It is, however, quite a significant project
and a different beast from macfusion itself. Of course, I would love
help from other developers.

Sincerely,
~ Michael Gorbach