OSXFUSE openFileAtPath and rsync

[Alessandro Diaferia]

Hi, I'm developing a backup facility that takes advantage of rsync behind the scenes. In addition to this, not every file you see in the volume is actually there, locally, but might be hosted remotely only. When the user wants to download a remote file an rsync process is fired to start the download. The drawback for this is that actually the whole rsync download occurs in the openFileAtPath method. This way there's no visual feedback to the user regarding the current progress. In addition to this it may occur, especially with slow connections, that OSXFUSE stops to work while the download is in progress. Is this an OSXFUSE-related issue or is it something lower? In either cases, do you have any hint on how to address this?

Thank you in advance for your time.

Kind regards.

[Benjamin Fleischer]

Hi Alessandro,

are you running into the 60 seconds timeout after which your filesystem is forcibly ejected? If this is the case, you could take a look at other network file systems like YouTubeFS, sshfs or curlftpfs. Most likely the have to deal with the same problem. Maybe you could solve this problem by running downloads in a separate threads. Your main thread would stay responsive.

It should be possible to display the progress of the download in a window similar to Finder's "Copy" progress window. In the end your file system is a regular Mac OS X application. In addition to that you could overlay the the icon of the file, that is being downloaded, with a progress bar like Safari does for downloads. This of cause won't help with an application, that is unresponsive until the file is downloaded and available locally.

Regards,
Benjamin

[Alessandro Diaferia]

Thank you for your hint.

I would like to go with a different thread for download too but my concern is about the sequence open->read->close. If I launch the download thread in the openFileAtPath method and return immediately, wouldn't it go with a readFileAtPath subsequently? By that time the file could still be downloading and not ready for read. Am I misunderstanding the whole thing?

I'd go through YouTubeFS and sshfs code in the meantime :)

[Benjamin Fleischer]

Scratch my comment about YourTubeFS. The version in OSXFUSE's filesystem repository just creates "links" to the videos. The videos are not downloaded. Some time ago I came across a file system that actually downloaded videos, but I can't seem to remember its name.

[Alessandro Diaferia]

Yup, and sshfs takes advantage of sftp to read files remotely without actually download them. I need to download files instead.

[Alessandro Diaferia]

Would it be a "correct" approach to fire a detached thread to download the file and return EAGAIN error in subsequent read as long as the file is still downloading? I'd like to prevent Finder error messages from appearing while the file is downloading.

[Sam Moffatt]

If it is opened in a non-blocking mode then EAGAIN would be reasonable
however if it is marked as blocking then block until you have the data
requested. I'm not entirely familiar with the Obj-C interface however
only the C one so there may be an implementation difference.

Cheers,

Sam Moffatt
http://pasamio.id.au

[ramesh]

I am developing a FUSE file system and even I have come across the same problem now. Could you resolve this issue in any way without affecting the user experience?

[free.d...@gmail.com]

Hi Guys,

So, did EAGAIN works for anyone?

I used EAGAIN in OpenFileAtPath and ReadFileAtPath - i don't see any difference. 

вторник, 15 апреля 2014 г., 15:12:14 UTC+3 пользователь ramesh написал:

[Sam Moffatt]

You mean when you return EAGAIN in your FS, the original caller doesn’t get the EAGAIN? What error does the caller get?

Sam Moffatt
http://pasamio.id.au

--
You received this message because you are subscribed to the Google Groups "OSXFUSE" group.
To unsubscribe from this group and stop receiving emails from it, send an email to osxfuse-group+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Roman Sidorakin]

Hi, guys.

I faced with the similar problem. I need to stop (pause) read calls, until file will be downloaded from cloud.

So, currently i'm pausing thread inside readFileAtPath call. But it causes freezing UI of Finder, in case if i am trying to open context menu of this file. Also, if i have an image opened in Preview.app, and downloading of other image in progress (read call paused), then UI of Preview.app also freezes until readFileAtPath calls will be resumed.

Can you please explain a little more about EAGAIN and O_NONBLOCK?

Currently i have tried to return EAGAIN from readFileAtPath method if file not downloaded, but from the logs i'm seeing, that readFileAtPath returns EAGAIN about 2-3 times, and then i see error, that file can't be opened.

I will be happy for any answer, even related to C-based api.

Thanks.

пятница, 31 марта 2017 г., 7:05:23 UTC+3 пользователь Samuel Moffatt написал:

Sam Moffatt
http://pasamio.id.au

To unsubscribe from this group and stop receiving emails from it, send an email to osxfuse-grou...@googlegroups.com.