Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Rclone Download Directory

5 views
Skip to first unread message

Brandy Lauro

unread,
Jan 25, 2024, 6:09:59 PM1/25/24
to
<div>Hello, I'm new to rclone but I've looked around and could not find an answer to this question.</div><div></div><div>I am trying to copy information from my local drive to Box. The directory I want to copy is "testrclone", which has two subdirectories and each directory (including testrclone) has one text file. The full path for testrclone is C:\Users\username\Documents\testrclone.</div><div></div><div>I am in the Documents directory when executing the rclone commands I've tried.</div><div></div><div>My goal is to get the testrclone directory included in the copy I am performing, but what I end up getting is only its file and all of the subdirectories and their files.</div><div></div><div>I've tried:</div><div></div><div>rclone copy -Pv testrclone\ boxconfig:MyTargetFiles</div><div></div><div>rclone copy -Pv .testrclone\ boxconfig:MyTargetFiles</div><div></div><div>rclone copy -Pv .testrclone\ boxconfig:MyTargetFiles</div><div></div><div>rclone copy -Pv testrclone\ boxconfig:MyTargetFiles</div><div></div><div>rclone copy -Pv \testrclone\ boxconfig:MyTargetFiles</div><div></div><div>rclone copy -Pv .\testrclone\ boxconfig:MyTargetFiles</div><div></div><div>And, honestly, a couple other permutations that I cannot remember.</div><div></div><div>What I am trying to achieve is to, after the copy, have MyTargetFiles/testrclone on Box. What I am ending up with (at best) is MyTargetFiles/ and MyTargetFiles/</div><div></div><div></div><div>I have noticed that some of my notes were dropped from my original post.</div><div></div><div>RE: what I am ending up with:</div><div></div><div>MyTargetFiles/TextFile.txt and MyTargetFiles/SubDirectoryUnder testrclone with all text files</div><div></div><div></div><div></div><div></div><div></div><div>rclone download directory</div><div></div><div>Download File: https://t.co/ndmscyGu7q </div><div></div><div></div><div>_copy/</div><div></div><div>Note that it is always the contents of the directory that is synced, not the directory so when source:path is a directory, it's the contents of source:path that are copied, not the directory name and contents.</div><div></div><div></div><div>First, you'll need to configure rclone. As the object storage systemshave quite complicated authentication these are kept in a config file.(See the --config entry for how to find the configfile and choose its location.)</div><div></div><div></div><div>rclone normally syncs or copies directories. However, if the sourceremote points to a file, rclone will just copy that file. Thedestination remote must point to a directory - rclone will give theerror Failed to create file system for "remote:file": is a file not a directory if it isn't.</div><div></div><div></div><div>On most backends this is refers to the same directory asremote:path/to/dir and that format should be preferred. On a verysmall number of remotes (FTP, SFTP, Dropbox for business) this willrefer to a different directory. On these, paths without a leading /will refer to your "home" directory and paths with a leading / willrefer to the root.</div><div></div><div></div><div>If you supply extra configuration to a backend by command line flag,environment variable or connection string then rclone will add asuffix based on the hash of the config to the name of the remote, eg</div><div></div><div></div><div>Starting with rclone version 1.61, any Unicode numbers and letters are allowed,while in older versions it was limited to plain ASCII (0-9, A-Z, a-z). If you usethe same rclone configuration from different shells, which may be configured withdifferent character encoding, you must be cautious to use characters that arepossible to write in all of them. This is mostly a problem on Windows, wherethe console traditionally uses a non-Unicode character set - definedby the so-called "code page".</div><div></div><div></div><div>rclone uses : to mark a remote name. This is, however, a validfilename component in non-Windows OSes. The remote name parser willonly search for a : up to the first / so if you need to act on afile or directory like this then use the full path starting with a/, or use ./ as a current directory prefix.</div><div></div><div></div><div>Rclone only supports a one-time sync of metadata. This means thatmetadata will be synced from the source object to the destinationobject only when the source object has changed and needs to bere-uploaded. If the metadata subsequently changes on the source objectwithout changing the object itself then it won't be synced to thedestination object. This is in line with the way rclone syncsContent-Type without the --metadata flag.</div><div></div><div></div><div></div><div></div><div></div><div></div><div>Options that take parameters can have the values passed in two ways,--option=value or --option value. However boolean (true/false)options behave slightly differently to the other options in that--boolean sets the option to true and the absence of the flag setsit to false. It is also possible to specify --boolean=false or--boolean=true. Note that --boolean false is not valid - this isparsed as --boolean and the false is parsed as an extra commandline argument for rclone.</div><div></div><div></div><div>The remote in use must support server-side move or copy and you mustuse the same remote as the destination of the sync. The backupdirectory must not overlap the destination directory without it beingexcluded by a filter rule.</div><div></div><div></div><div>Note that the units are Byte/s, not bit/s. Typicallyconnections are measured in bit/s - to convert divide by 8. Forexample, let's say you have a 10 Mbit/s connection and you wish rcloneto use half of it - 5 Mbit/s. This is 5/8 = 0.625 MiB/s so you woulduse a --bwlimit 0.625M parameter for rclone.</div><div></div><div></div><div>If this flag is set then in a sync, copy or move, rclone will doall the checks to see whether files need to be transferred beforedoing any of the transfers. Normally rclone would start runningtransfers as soon as possible.</div><div></div><div></div><div>If both --check-first and --order-by are set when doing rclone movethen rclone will use the transfer thread to delete source files whichdon't need transferring. This will enable perfect ordering of thetransfers and deletes but will cause the transfer stats to have moreitems in than expected.</div><div></div><div></div><div>The fact that an existing file rclone.conf in the same directoryas the rclone executable is always preferred, means that it is easyto run in "portable" mode by downloading rclone executable to awritable directory and then create an empty file rclone.conf in thesame directory.</div><div></div><div></div><div>If the location is set to empty string "" or path to a filewith name notfound, or the os null device represented by value NUL onWindows and /dev/null on Unix systems, then rclone will keep theconfig file in memory only.</div><div></div><div></div><div>The file format is basic INI:Sections of text, led by a [section] header and followed bykey=value entries on separate lines. In rclone each remote isrepresented by its own section, where the section name defines thename of the remote. Options are specified as the key=value entries,where the key is the option name without the --backend- prefix,in lowercase and with _ instead of -. E.g. option --mega-hard-deletecorresponds to key hard_delete. Only backend options can be specified.A special, and required, key type identifies the storage system,where the value is the internal lowercase name as returned by commandrclone help backends. Comments are indicated by ; or # at thebeginning of a line.</div><div></div><div></div><div>Note that passwords are in obscuredform. Also, many storage systems uses token-based authentication insteadof passwords, and this requires additional steps. It is easier, and safer,to use the interactive command rclone config instead of manuallyediting the configuration file.</div><div></div><div></div><div>To reduce risk of corrupting an existing configuration file, rclonewill not write directly to it when saving changes. Instead it willfirst write to a new, temporary, file. If a configuration file alreadyexisted, it will (on Unix systems) try to mirror its permissions tothe new file. Then it will rename the existing file to a temporaryname as backup. Next, rclone will rename the new file to the correct name,before finally cleaning up by deleting the backup file.</div><div></div><div></div><div>If the configuration file path used by rclone is a symbolic link, thenthis will be evaluated and rclone will write to the resolved path, insteadof overwriting the symbolic link. Temporary files used in the process(described above) will be written to the same parent directory as thatof the resolved configuration file, but if this directory is also asymbolic link it will not be resolved and the temporary files will bewritten to the location of the directory symbolic link.</div><div></div><div></div><div>This option allows rclone to return exit code 9 if no files were transferredbetween the source and destination. This allows using rclone in scripts, andtriggering follow-on actions if data was copied, or skipping if not.</div><div></div><div></div><div>When using rclone via the API rclone caches created remotes for 5minutes by default in the "fs cache". This means that if you dorepeated actions on the same remote then rclone won't have to build itagain from scratch, which makes it more efficient.</div><div></div><div></div><div>Note that if you use some flags, eg --backup-dir and if this is setto 0 rclone may build two remotes (one for the source or destinationand one for the --backup-dir where it may have only built onebefore.</div><div></div><div></div><div>Normally rclone will look at modification time and size of files tosee if they are equal. If you set this flag then rclone will checkonly the modification time. If --checksum is set then it onlychecks the checksum.</div><div></div><div></div><div>With this option set, files will be created and deleted as requested,but existing files will never be updated. If an existing file doesnot match between the source and destination, rclone will give the errorSource and destination exist but do not match: immutable file modified.</div><div></div><div></div><div>Without --inplace (the default) rclone will first upload to atemporary file with an extension like this, where XXXXXX represents arandom string and .partial is --partial-suffix value(.partial by default).</div><div></div><div></div><div>This means that an incomplete file will be visible in the directorylistings while the upload is in progress and any existing files willbe overwritten as soon as the upload starts. If the transfer failsthen the file will be deleted. This can cause data loss of theexisting file if the transfer fails.</div><div></div><div></div><div>Comma separated list of log format options. Accepted options are date,time, microseconds, pid, longfile, shortfile, UTC. Any otherkeywords will be silently ignored. pid will tag log messages with processidentifier which useful with rclone mount --daemon. Other acceptedoptions are explained in the go documentation.The default log format is "date,time".</div><div></div><div></div><div>This shouldn't need to be changed from the default in normal operations.However, if you get a lot of low level retries you may wishto reduce the value so rclone moves on to a high level retry (see the--retries flag) quicker.</div><div></div><div> df19127ead</div>
0 new messages