Ultra.cc Rclone

[Jason]

Irecently dipped my toes into the world of seedboxes and have moved onto Ultraseedbox in the last week. With all the normal app setup out of the way after testing and good success. I felt ready for the next task of getting my rclone to work with my google drive.

ATM I am stuck at pretty much the same place as this post = Rclone VFS and MergerFS Setup on Ultraseedbox. Reading that post it became clear that the --allow--other was causing the issue.

I've searched and searched this forum, Google, etc. and tried for many hours to get this to work but definately outside my depth, hence why I need your guys help. @Animosity022 I know you are the beast in this area and would like if you could have a look.

systemctl --user enable --now rclone-vfs && systemctl --user enable --now mergerfs

Job for rclone-vfs.service failed because the control process exited with error code.

See "systemctl --user status rclone-vfs.service" and "journalctl --user -xe" for details.

What I know from the previous post ( Rclone VFS and MergerFS Setup on Ultraseedbox) is that the root, seems to cause the issue in my rclone-vfs.service file. From what I could understand is that --allow--other needs to be amended.

You'd want to remove that. Allow Other is used for a single server solution and that means anyone else on the server can see your files, which is why it's off on the seedbox by default and commented out of /etc/fuse.conf.

So my simple test, I copy a file, see it is in my local disk, one rclone to move it and validate it's on my rclone. Takes 1 minute for polling to pick it up by default or I have mine set to 15 seconds.

Just make sure you change the paths to what you have. Also to test it out, in addition to what animosity said, if you want to test out the systemd timer you can just copy from the ExecStart= to second to the last flag (which I belive is --drive-stop-on-upload-limit) and paste it in your terminal. Below's what you'd want to paste to test out (replacing paths to whatever you have.

@Animosity022, thanks for the heads up for the --allow-other flag which was on our service files. On our setup, clients cannot access other users home slots other than their own and it is commented out in fuse.conf as you mentioned. But just to be safe, I've removed it from the service files, in addition to adding the new --drive-stop-on-upload-limit introduced from 1.51 for the upload scripts.

Just to check, I assume if I just take the standard upload script it will run it once a day, right? If thats the case I don't think I see a reason to have this quicker than that (unless any more insights can be shared).

Beyond that it seems the files move from MergerFS into the Stuff/Local when copied to MergerFS. Ie only the ones not yet uploaded to Google Drive. Assume the Script will do the rest tonight and move this to the cloud, assuming the standard script does the upload without any timing tweaks.

Yep. If you use the systemd uploader/timer by default, it'll execute daily +-30 minutes at CET/CEST. You can also change it too if you'd like. The Arch Linux wiki has some nifty examples on how to do so:

I'd suggest sticking to whatever defaults we have. You're free to change any of the ones on the systemd files if you'd like but I suggest not to. The ones on the docs works great in terms of streaming, uploading files within the Google Drive Upload limits and such. It's gonna be down to your peering basically.

A WebDAV mount with rclone. I have a 100mbit/s internet, and I'm trying to mount a remote directory to smoothly playback high bitrate files. The files average 60mbit/s but can peak at 120mbit/s. My wish is to use vfs cache read ahead to maximise my internet use so that I have a smoother playback experience during the bumpy parts.

My mergerfs folder appears blank every few days and the only way to resolve this is by restarting the rclone service and mergerfs service. Im not sure why this is happening and if someone could kindly assist me. I will provide further information when requested.

Paths are specified as remote:path. If the path does not begin witha / it is relative to the home directory of the user. An empty pathremote: refers to the user's home directory. For example, rclone lsd remote:would list the home directory of the user configured in the rclone remote config(i.e /home/sftpuser). However, rclone lsd remote:/ would list the rootdirectory for remote machine (i.e. /)

The key file can be specified in either an external file (key_file) or contained within therclone config file (key_pem). If using key_pem in the config file, the entry should be on asingle line with new line ('\n' or '\r\n') separating lines. i.e.

If you don't specify pass, key_file, or key_pem or ask_password thenrclone will attempt to contact an ssh-agent. You can also specify key_use_agentto force the usage of an ssh-agent. In this case key_file or key_pem canalso be specified to force the usage of a specific key in the ssh-agent.

If you have a certificate you may use it to sign your public key, creating aseparate SSH user certificate that should be used instead of the plain public keyextracted from the private key. Then you must provide the path to theuser certificate public key file in pubkey_file.

By default rclone will not check the server's host key for validation. Thiscan allow an attacker to replace a server with their own and if you usepassword authentication then this can lead to that password being exposed.

Some functionality of the SFTP backend relies on remote shell access,and the possibility to execute commands. This includes checksum,and in some cases also about. The shell commands thatmust be executed may be different on different type of shells, and alsoquoting/escaping of file path arguments containing special characters maybe different. Rclone therefore needs to know what type of shell it is,and if shell access is available at all.

Most servers run on some version of Unix, and then a basic Unix shell canbe assumed, without further distinction. Windows 10, Server 2019, and latercan also run a SSH server, which is a port of OpenSSH (see officialinstallation guide). On a Windows server the shell handling is different: Although it can alsobe set up to use a Unix type shell, e.g. Cygwin bash, the default is touse Windows Command Prompt (cmd.exe), and PowerShell is a recommendedalternative. All of these have behave differently, which rclone must handle.

Rclone tries to auto-detect what type of shell is used on the server,first time you access the SFTP remote. If a remote shell session issuccessfully created, it will look for indications that it is CMD orPowerShell, with fall-back to Unix if not something else is detected.If unable to even create a remote shell session, then shell commandexecution will be disabled entirely. The result is stored in the SFTPremote configuration, in option shell_type, so that the auto-detectiononly have to be performed once. If you manually set a value for thisoption before first run, the auto-detection will be skipped, and ifyou set a different value later this will override any existing.Value none can be set to avoid any attempts at executing shellcommands, e.g. if this is not allowed on the server.If you have shell_type = none in the configuration thenthe ssh must not be set.

When the server is rclone serve sftp,the rclone SFTP remote will detect this as a Unix type shell - evenif it is running on Windows. This server does not actually have a shell,but it accepts input commands matching the specific ones that theSFTP backend relies on for Unix shells, e.g. md5sum and df. Alsoit handles the string escape rules used for Unix shell. Treating itas a Unix type shell from a SFTP remote will therefore always becorrect, and support all features.

The shell type auto-detection logic, described above, means thatby default rclone will try to run a shell command the first timea new sftp remote is accessed. If you configure a sftp remotewithout a config file, e.g. an on the flyremote, rclone will have nowhere to store the result, and itwill re-run the command on every access. To avoid this you shouldexplicitly set the shell_type option to the correct value,or to none if you want to prevent rclone from executing anyremote shell commands.

It is also important to note that, since the shell type decideshow quoting and escaping of file paths used as command-line argumentsare performed, configuring the wrong shell type may leave you exposedto command injection exploits. Make sure to confirm the auto-detectedshell type, or explicitly set the shell type you know is correct,or disable shell access until you know.

SFTP does not natively support checksums (file hash), but rcloneis able to use checksumming if the same login has shell access,and can execute remote commands. If there is a command that cancalculate compatible checksums on the remote system, Rclone canthen be configured to execute this whenever a checksum is needed,and read back the results. Currently MD5 and SHA-1 are supported.

Normally this requires an external utility being available onthe server. By default rclone will try commands md5sum, md5and rclone md5sum for MD5 checksums, and the first one found usablewill be picked. Same with sha1sum, sha1 and rclone sha1sumcommands for SHA-1 checksums. These utilities normally need tobe in the remote's PATH to be found.

In some cases the shell itself is capable of calculating checksums.PowerShell is an example of such a shell. If rclone detects that theremote shell is PowerShell, which means it most probably is aWindows OpenSSH server, rclone will use a predefined script blockto produce the checksums when no external checksum commands are found(see shell access). This assumes PowerShell version4.0 or newer.

3a8082e126