Copying local data to local Drobo
41 views
Skip to first unread message
Hylton Conacher (ZR1HPC)
Jul 11, 2014, 9:37:03 AM7/11/14
to Drobo -talk
Hi,
As a fairly long time subscriber to this list and running OpenSuSe 11.2.
I had initially hoped that installing the Drobo software would give me
the ability to use the Drobo software to initiate and maintain a
backup. I could then update the machine to a newer release and using
newer software restore the backup.
Dreams will be dreams and whilst the Drobo certainly did give the
attached storage space on which to make a backup, the software side as
such, would need to take the line of something via the cli for the time
being i.e. rsync especially as cp was FAR too time intensive.
I would like to make a bare metal backup but
I want to make a backup, at least of the home directory as well as a few
others i.e. /etc, to the Drobo via USB as there is no Firewire on the
computer nor an Ethernet connection on the Drobo.
The Drobo is owned by root and whilst I know the root password, I prefer
to be logged on as a regular user, using sudo if necessary. but it is
the syntax of the rsync command that is stopping me from executing as
rsync can apparently be a BEAST if used incorrectly.
I have read and re-read the man pages as well as doing some Google
searches and reading Suse Linux Toolbox by Chris Negus. Many of
references show copying from a remote host or using a remote daemon.
Since I am doing it all locally I hope the list can help i.e. I am at
the terminal on the machine that holds the information to be backed up
and that also has the Drobo connected.
My reading has given me the command so far as:
$> rsync -avh --progress /home root@Umalusi:/drobo/backup-'date +%D'/
This would, I believe, recursively copy all items in the /home
directory, archiving files and folders, being verbose, and provide a
human readable progress indicator, and place them in a dated(yyyymmdd)
folder on the backup medium with full path being under
/Drobo/backup-yyyymmdd/?
Since it is copying onto a device owned by root and who only root can
create files on, I assume the root password would be needed on execution.
Can you answer any of the queries, correct my command syntax, or
redirect my thinking?
Help towards using the cli property is the main aim, but of course the
backup is equally as important.
Your thoughts?
Regards
Hylton
P.S: I need to do a cli backup before upgrading to latest stable release
of Os
As a fairly long time subscriber to this list and running OpenSuSe 11.2.
I had initially hoped that installing the Drobo software would give me
the ability to use the Drobo software to initiate and maintain a
backup. I could then update the machine to a newer release and using
newer software restore the backup.
Dreams will be dreams and whilst the Drobo certainly did give the
attached storage space on which to make a backup, the software side as
such, would need to take the line of something via the cli for the time
being i.e. rsync especially as cp was FAR too time intensive.
I would like to make a bare metal backup but
I want to make a backup, at least of the home directory as well as a few
others i.e. /etc, to the Drobo via USB as there is no Firewire on the
computer nor an Ethernet connection on the Drobo.
The Drobo is owned by root and whilst I know the root password, I prefer
to be logged on as a regular user, using sudo if necessary. but it is
the syntax of the rsync command that is stopping me from executing as
rsync can apparently be a BEAST if used incorrectly.
I have read and re-read the man pages as well as doing some Google
searches and reading Suse Linux Toolbox by Chris Negus. Many of
references show copying from a remote host or using a remote daemon.
Since I am doing it all locally I hope the list can help i.e. I am at
the terminal on the machine that holds the information to be backed up
and that also has the Drobo connected.
My reading has given me the command so far as:
$> rsync -avh --progress /home root@Umalusi:/drobo/backup-'date +%D'/
This would, I believe, recursively copy all items in the /home
directory, archiving files and folders, being verbose, and provide a
human readable progress indicator, and place them in a dated(yyyymmdd)
folder on the backup medium with full path being under
/Drobo/backup-yyyymmdd/?
Since it is copying onto a device owned by root and who only root can
create files on, I assume the root password would be needed on execution.
Can you answer any of the queries, correct my command syntax, or
redirect my thinking?
Help towards using the cli property is the main aim, but of course the
backup is equally as important.
Your thoughts?
Regards
Hylton
P.S: I need to do a cli backup before upgrading to latest stable release
of Os
Darryl
Jul 12, 2014, 1:29:22 AM7/12/14
to drobo...@googlegroups.com, hyl...@conacher.co.za
On Friday, July 11, 2014 6:37:03 AM UTC-7, Hylton wrote:
> The Drobo is owned by root and whilst I know the root password, I
> prefer to be logged on as a regular user, using sudo if necessary.
> but it is the syntax of the rsync command that is stopping me from
> executing as rsync can apparently be a BEAST if used incorrectly.
> The Drobo is owned by root and whilst I know the root password, I
> prefer to be logged on as a regular user, using sudo if necessary.
> but it is the syntax of the rsync command that is stopping me from
> executing as rsync can apparently be a BEAST if used incorrectly.
I'm not sure what you mean by "beast", but rsync is generally no worse
or better than any other backup program.
Unfortunately, you have a lot of questions, and answering them requires
some basic unix knowledge, and I don't know how much you know:
1. You need to know the basics of unix file and directory permissions.
2. (Maybe) You need to know how to setup and use ssh, preferably with
ssh certificates (because "recent" versions of rsync use ssh by
default).
You should also have rsync version 3.0.9 or later (because of bugfixes).
Hopefully, that isn't hard for you, as that dates back to 2011. The
latest version is from June 22, 2014 (version 3.1.1), but having that
isn't necessary.
To answer your questions:
* rsync typically writes files as the user that you use. If you happen
to use (for example) the account, "johndoe", rsync will write files
owned by "johndoe", and will require that the destination directory be
writable by "johndoe".
If your destination directory is owned by root and does not allow
writing by "johndoe", rsync is going to fail.
If your destination is owned by root (or any other account), and
allows writing by "johndoe", rsync will work.
If your destination is owned by "johndoe", and you have not done
anything to disallow writes by "johndoe", rsync will work.
HOWEVER, WRITING AS "root" IS SPECIAL, UNLIKE OTHER USERS (like
"johndoe"). That's because, when you write as "root", rsync goes into
a special mode that also preserves file ownerships. When you write as
a regular user (like, "johhdoe") all destination files become owned by
"johndoe", and do not preserve the original file owners.
In your example:
or better than any other backup program.
Unfortunately, you have a lot of questions, and answering them requires
some basic unix knowledge, and I don't know how much you know:
1. You need to know the basics of unix file and directory permissions.
2. (Maybe) You need to know how to setup and use ssh, preferably with
ssh certificates (because "recent" versions of rsync use ssh by
default).
You should also have rsync version 3.0.9 or later (because of bugfixes).
Hopefully, that isn't hard for you, as that dates back to 2011. The
latest version is from June 22, 2014 (version 3.1.1), but having that
isn't necessary.
To answer your questions:
* rsync typically writes files as the user that you use. If you happen
to use (for example) the account, "johndoe", rsync will write files
owned by "johndoe", and will require that the destination directory be
writable by "johndoe".
If your destination directory is owned by root and does not allow
writing by "johndoe", rsync is going to fail.
If your destination is owned by root (or any other account), and
allows writing by "johndoe", rsync will work.
If your destination is owned by "johndoe", and you have not done
anything to disallow writes by "johndoe", rsync will work.
HOWEVER, WRITING AS "root" IS SPECIAL, UNLIKE OTHER USERS (like
"johndoe"). That's because, when you write as "root", rsync goes into
a special mode that also preserves file ownerships. When you write as
a regular user (like, "johhdoe") all destination files become owned by
"johndoe", and do not preserve the original file owners.
In your example:
rsync -avh --progress /home root@Umalusi:/drobo/backup-`date +%D`/
You are writing as "root", on the system, "Umalusi". In this case,
rsync will preserve the owners on the destination files.
However, please note that "preserving ownerships" actually means
"preserving numeric user ID ownerships", as unix really does
everything under-the-hood via numeric user IDs. User-friendly names
like "root" and "johndoe" are, well, simply user-friendly names that
you use. Deep down, generally hidden from you, unix handles
everything using numeric user IDs. This is important because, if you
rsync as root, the numeric user IDs should be synchronized between the
two machines. If they're not synchronized, you generally end up with
one or more of:
a. When viewing file ownerships of the backup files (on the
destination system), the files/directories will appear to be owned
by the wrong users.
b. When viewing file ownerships of the backup files (on the
destination system), the files/directories will appear to be owned
by numeric ids (the user names are replaced by numbers).
* You can use a non-root user on the remote system (but see below, as
your example command is likely not what you want):
rsync -avh --progress /home johndoe@Umalusi:/drobo/backup-`date +%D`/
This will cause all files on the destination system (Unalusi) to be
owned by "johndoe". If you're only backing up your own user files
(assuming that your login is "johndoe"), this isn't a problem. If
you're backing up files for multiple users, or for system users, this
is a problem.
* Your example command does not need an rsync server/daemon (you can set
this up if you want, but it's overkill for what you're trying to do).
However, you do need to be able to login to the destination system
using ssh (logins via ssh need to be enabled on the destination
system).
* Since "recent" versions of rsync uses ssh, setting up ssh certificate
authentication means that you don't need to enter a password. (On the
other hand, maybe you want to enter some password.)
Well, you could use rlogin instead of ssh, but that's a bad practice,
as rlogin is insecure. (Now, the network that you're using might be
"secure" enough to use rlogin, but using rlogin is still a bad habit.)
* Your command isn't quite right:
rsync will preserve the owners on the destination files.
However, please note that "preserving ownerships" actually means
"preserving numeric user ID ownerships", as unix really does
everything under-the-hood via numeric user IDs. User-friendly names
like "root" and "johndoe" are, well, simply user-friendly names that
you use. Deep down, generally hidden from you, unix handles
everything using numeric user IDs. This is important because, if you
rsync as root, the numeric user IDs should be synchronized between the
two machines. If they're not synchronized, you generally end up with
one or more of:
a. When viewing file ownerships of the backup files (on the
destination system), the files/directories will appear to be owned
by the wrong users.
b. When viewing file ownerships of the backup files (on the
destination system), the files/directories will appear to be owned
by numeric ids (the user names are replaced by numbers).
* You can use a non-root user on the remote system (but see below, as
your example command is likely not what you want):
rsync -avh --progress /home johndoe@Umalusi:/drobo/backup-`date +%D`/
This will cause all files on the destination system (Unalusi) to be
owned by "johndoe". If you're only backing up your own user files
(assuming that your login is "johndoe"), this isn't a problem. If
you're backing up files for multiple users, or for system users, this
is a problem.
* Your example command does not need an rsync server/daemon (you can set
this up if you want, but it's overkill for what you're trying to do).
However, you do need to be able to login to the destination system
using ssh (logins via ssh need to be enabled on the destination
system).
* Since "recent" versions of rsync uses ssh, setting up ssh certificate
authentication means that you don't need to enter a password. (On the
other hand, maybe you want to enter some password.)
Well, you could use rlogin instead of ssh, but that's a bad practice,
as rlogin is insecure. (Now, the network that you're using might be
"secure" enough to use rlogin, but using rlogin is still a bad habit.)
* Your command isn't quite right:
rsync -avh --progress /home root@Umalusi:/drobo/backup-'date +%D'/
It should probably be:
rsync -avh --progress /home root@Umalusi:/drobo/backup-`date +%D`/
(You need to use single BACKquotes instead of single FORWARD quotes.)
However, this will still not do what you want:
However, this will still not do what you want:
> This would, I believe, recursively copy all items in the /home
> directory, archiving files and folders, being verbose, and provide a
> human readable progress indicator, and place them in a
> dated(yyyymmdd) folder on the backup medium with full path being
> under /Drobo/backup-yyyymmdd/?
This will attempt to write to directories below (NOTE: "/drobo", not
"/Drobo" -- unix filesystems are typically case sensitive):
/drobo/backup-mm/dd/yy/
Which, since there are additional directory separators, will likely
cause rsync to fail unless you create that directory BEFORE running
rsync. For today, "date +%D" returns "07/11/14", and so this
directory would be used:
/drobo/backup-07/11/14/
Please read the man page for "date" -- that explains that "%D" means
"mm/dd/yy" (which might be affected by your locale, but I'm not sure).
You probably want to use something like "date +%Y%m%d" (for
"20140711") or maybe "date +%Y-%m-%d" ("2014-07-11"). Don't use
directory separators in the specification, unless you want to create
that directory before running rsync.
So, your command would be something like:
rsync -avh --progress /home root@Umalusi:/drobo/backup-`date +%Y%m%d`/
or:
rsync -avh --progress /home root@Umalusi:/drobo/backup-`date +%Y-%m-%d`/
(Again, backward single quotes -- not forward single quotes.)
"/Drobo" -- unix filesystems are typically case sensitive):
/drobo/backup-mm/dd/yy/
Which, since there are additional directory separators, will likely
cause rsync to fail unless you create that directory BEFORE running
rsync. For today, "date +%D" returns "07/11/14", and so this
directory would be used:
/drobo/backup-07/11/14/
Please read the man page for "date" -- that explains that "%D" means
"mm/dd/yy" (which might be affected by your locale, but I'm not sure).
You probably want to use something like "date +%Y%m%d" (for
"20140711") or maybe "date +%Y-%m-%d" ("2014-07-11"). Don't use
directory separators in the specification, unless you want to create
that directory before running rsync.
So, your command would be something like:
rsync -avh --progress /home root@Umalusi:/drobo/backup-`date +%Y%m%d`/
or:
rsync -avh --progress /home root@Umalusi:/drobo/backup-`date +%Y-%m-%d`/
(Again, backward single quotes -- not forward single quotes.)
Hylton Conacher (ZR1HPC)
Aug 20, 2014, 3:11:27 PM8/20/14
to drobo...@googlegroups.com, dar...@gmail.com
Hi Darryl,
Please excuse the delay in replying. Unfortunately all my work has
ground to a halt with the addition to the family of an 8week old puppy.
After almost 3 weeks, I do not even like any domesticated canine any more :)
On 12/07/14 07:29, Darryl wrote:
> On Friday, July 11, 2014 6:37:03 AM UTC-7, Hylton wrote:
>
> > The Drobo is owned by root and whilst I know the root password, I
> > prefer to be logged on as a regular user, using sudo if necessary.
> > but it is the syntax of the rsync command that is stopping me from
> > executing as rsync can apparently be a BEAST if used incorrectly.
>
> I'm not sure what you mean by "beast", but rsync is generally no worse
> or better than any other backup program.
I just figured, as a newbie to the cli its options might rattle me a
bit, however I have moved on and away from rsync for the moment.
> Unfortunately, you have a lot of questions, and answering them requires
> some basic unix knowledge, and I don't know how much you know:
Short story: Ex Windows user, moved to openSuSe KDE abt 2010, just now
starting to dip into cli.
to me. This will only improve with practice, unfortunately.
mirrored.
> You should also have rsync version 3.0.9 or later (because of bugfixes).
> Hopefully, that isn't hard for you, as that dates back to 2011. The
> latest version is from June 22, 2014 (version 3.1.1), but having that
> isn't necessary.
Man page reading and $rsync --version indicates I have version 3.0.6 so
again system needs mirror, then upgrade.
> To answer your questions:
<snip>
Thanks for answering my questions and providing valuable learning
material. Much valid stuff there and appreciated.
Seeing as there is already some outdated software on the machine I would
prefer to merely mirror the entire system onto a Drobo.
Perhaps `dd` would do it.
I'll start a new thread for that question.
Regards
Hylton
Please excuse the delay in replying. Unfortunately all my work has
ground to a halt with the addition to the family of an 8week old puppy.
After almost 3 weeks, I do not even like any domesticated canine any more :)
On 12/07/14 07:29, Darryl wrote:
> On Friday, July 11, 2014 6:37:03 AM UTC-7, Hylton wrote:
>
> > The Drobo is owned by root and whilst I know the root password, I
> > prefer to be logged on as a regular user, using sudo if necessary.
> > but it is the syntax of the rsync command that is stopping me from
> > executing as rsync can apparently be a BEAST if used incorrectly.
>
> I'm not sure what you mean by "beast", but rsync is generally no worse
> or better than any other backup program.
bit, however I have moved on and away from rsync for the moment.
> Unfortunately, you have a lot of questions, and answering them requires
> some basic unix knowledge, and I don't know how much you know:
starting to dip into cli.
>
> 1. You need to know the basics of unix file and directory permissions.
Basic basics :) But can use a man page, just sometimes the syntax gets
> 1. You need to know the basics of unix file and directory permissions.
to me. This will only improve with practice, unfortunately.
>
> 2. (Maybe) You need to know how to setup and use ssh, preferably with
> ssh certificates (because "recent" versions of rsync use ssh by
> default).
Not currently possible as machine needs upgrade first, after system is
> 2. (Maybe) You need to know how to setup and use ssh, preferably with
> ssh certificates (because "recent" versions of rsync use ssh by
> default).
mirrored.
> You should also have rsync version 3.0.9 or later (because of bugfixes).
> Hopefully, that isn't hard for you, as that dates back to 2011. The
> latest version is from June 22, 2014 (version 3.1.1), but having that
> isn't necessary.
again system needs mirror, then upgrade.
> To answer your questions:
<snip>
Thanks for answering my questions and providing valuable learning
material. Much valid stuff there and appreciated.
Seeing as there is already some outdated software on the machine I would
prefer to merely mirror the entire system onto a Drobo.
Perhaps `dd` would do it.
I'll start a new thread for that question.
Regards
Hylton
Reply all
Reply to author
Forward
0 new messages