dd or cp over network: should I use scp?
Dotan Cohen
that I'd rather not get into I cannot remove the drive from the
laptop.
Should I just use scp to copy over the LAN? Something like this?
scp -r / ro...@178.63.65.136:/
Can I actually use dd over the network, maybe by piping to scp
somehow? What is the canonical way of doing this?
Thanks!
--
Dotan Cohen
http://gibberish.co.il
http://what-is-what.com
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/AANLkTinXBXVhoP-rBPWc0...@mail.gmail.com
Sjoerd Hardeman
> I need to dd or cp my laptop's harddrive over the LAN. For a reason
> that I'd rather not get into I cannot remove the drive from the
> laptop.
>
> Should I just use scp to copy over the LAN? Something like this?
> scp -r / ro...@178.63.65.136:/
rsync -aAXxPz / ro...@178.63.65.136:/
where -a preserves permissions, modification times etc
-A preserves acls
-X preserves xattrs
-x keeps you in one filesystem (so you won't copy /dev etc.)
-P keeps partial copied files for faster stop-and-proceed, and shows a
progress bar
-z compresses the data stream
This should give you an identical copy. Good luck!
Sjoerd
Steven Ayre
dd if=/dev/sda1 | ssh user@host "dd of=/path/to/sda1.img"
You might also want to enable SSH compression, or pipe it via gzip+gunzip.
Which is better (dd+ssh/scp) depends on your setup. dd copies the
entire disk bit for bit, even unused space. On the plus side you get
all data, permissions, layout. On the downside you're copying far more
than you need to.
Personally, I'd use scp unless I need an exact copy of the disk.
-Steve
Archive: http://lists.debian.org/AANLkTimpjGXcPYNDz40pG...@mail.gmail.com
BOYPT
dd if=/dev/sdX | nc -l 4321
On the other box:
nc ip.of.the.serverbox 4321 | dd of=/dev/sdY
--
Sent from my mobile device
Linuxer using Arch/Ubuntu, Pythoner, Geek
--> Blog: http://apt-blog.net
--> FB: http://www.facebook.com/boypt
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/AANLkTinprxT-OqCeb5UFQ...@mail.gmail.com
Ron Johnson
If this is a trusted LAN, then no need for the encryption overhead.
On 02/24/2011 06:44 AM, Steven Ayre wrote:
> You can use dd via ssh if you wish. I think the command would be:
> dd if=/dev/sda1 | ssh user@host "dd of=/path/to/sda1.img"
>
> You might also want to enable SSH compression, or pipe it via gzip+gunzip.
>
> Which is better (dd+ssh/scp) depends on your setup. dd copies the
> entire disk bit for bit, even unused space. On the plus side you get
> all data, permissions, layout. On the downside you're copying far more
> than you need to.
>
> Personally, I'd use scp unless I need an exact copy of the disk.
>
> -Steve
>
>
> On 24 February 2011 12:32, Dotan Cohen<dotan...@gmail.com> wrote:
>> I need to dd or cp my laptop's harddrive over the LAN. For a reason
>> that I'd rather not get into I cannot remove the drive from the
>> laptop.
>>
>> Should I just use scp to copy over the LAN? Something like this?
>> scp -r / ro...@178.63.65.136:/
>>
>> Can I actually use dd over the network, maybe by piping to scp
>> somehow? What is the canonical way of doing this?
>> Thanks!
>>
--
"The normal condition of mankind is tyranny and misery."
Milton Friedman
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Eduardo M KALINOWSKI
> What about hardlinks? I've always stuck to dd because I'm not confident
> rsync will give me an exact enough copy to completely restore the
> original from.
rsync can preserve hard links, but that's not enabled by default by
-a. IIRC the option is -H (or maybe -h ?), but check the man page.
--
If a guru falls in the forest with no one to hear him, was he really a
guru at all?
-- Strange de Jim, "The Metasexuals"
Eduardo M KALINOWSKI
edu...@kalinowski.com.br
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/20110224101329....@mail.kalinowski.com.br
Tixy
What about hardlinks? I've always stuck to dd because I'm not confident
rsync will give me an exact enough copy to completely restore the
original from.
--
Tixy () The ASCII Ribbon Campaign (www.asciiribbon.org)
/\ Against HTML e-mail and proprietary attachments
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/1298553068.2328.6.camel@ubuntu
shawn wilson
Forgot to mention that netcat IS NOT secure. This is an idea to be expanded upon not used verbatim.
> I didn't know you could separate dd like Steven suggested - that's pretty
> cool.
>
> I'll add two ideas:
> 1. I think cp can make a block copy, if I'm remembering correctly with that,
> you might want to see if scp can do the same.
> 2. You could do
> ... of=/dev/tcp/ip/port
> (IIRC) on >2.4 kernels and get nc to catch and pipe the output. Don't know
> how you'd deal with simultaneous uploads there and check for the new
> connection to figure out where the start and stop of the dump was, but...
shawn wilson
Celejar
BOYPT <pen...@gmail.com> wrote:
> netcat may help:
>
> dd if=/dev/sdX | nc -l 4321
>
> On the other box:
>
> nc ip.of.the.serverbox 4321 | dd of=/dev/sdY
Or perhaps cryptcat, with built-in twofish encryption (I haven't played
much with these tools, so this is just speculation - I have no idea,
for example, how using cryptcat would compare to tunneling netcat over
ssh).
Celejar
--
foffl.sourceforge.net - Feeds OFFLine, an offline RSS/Atom aggregator
mailmin.sourceforge.net - remote access via secure (OpenPGP) email
ssuds.sourceforge.net - A Simple Sudoku Solver and Generator
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/20110224092558....@gmail.com
Karl E. Jorgensen
On Thu, Feb 24, 2011 at 02:32:43PM +0200, Dotan Cohen wrote:
> I need to dd or cp my laptop's harddrive over the LAN. For a reason
> that I'd rather not get into I cannot remove the drive from the
> laptop.
>
> Should I just use scp to copy over the LAN? Something like this?
> scp -r / ro...@178.63.65.136:/
Well... SSH has some overhead because it does encryption - you simply cannot
have encryption without _some_ overhead.
I would recommend rsync instead - simply because if things go wrong during the
copy, it can pick up from where it left, rather than restarting from scratch:
rsync -av --numeric-ids / root@otherhost:/
--numeric-ids is to ensure it uses the _numeric_ user/group IDS on the other
side. You may find that some users have differnt IDs between machines - e.g.
IDs that are added by packages (e.g. mysql and stuff).
> Can I actually use dd over the network, maybe by piping to scp
> somehow? What is the canonical way of doing this?
> Thanks!
Yes - if the disks are the same size, it's a no-brainer. My personal preference
for this is to use "nc" - which is lightweight, but does not do encryption.
On the "source" machine, send it out over the network
sourcebox:~# dd if=/dev/sda | nc -l -p 9999
and on the destination box - which should be running a "live CD" or similar, as
blasting over a disk which is in use leads to kernel confusion and human
madness:
destbox:~# nc $IP_OF_OTHER_BOX 9999 | dd of=/dev/sda
You may want to use gzip to trade off CPU usage for better network bandwith:
sourcebox:~# dd if=/dev/sda | gzip | nc -l -p 9999
destbox:~# nc $IP_OF_OTHER_BOX 9999 | gunzip | dd of=/dev/sda
Hope this helps
--
Karl E. Jorgensen
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/20110224125...@fizzback.net
Jochen Schulz
>
> I need to dd or cp my laptop's harddrive over the LAN. For a reason
> that I'd rather not get into I cannot remove the drive from the
> laptop.
What do you want to achieve? Cloning a filesystem that is in use always
has some drawbacks, depending on the method.
If you use LVM: use snapshots. You'll get a dirty filesystem that needs
a fsck, but the result should be consistent.
Otherwise, I would probably use rsync because it might save time if the
copy is interrupted for some reason. Dd will probably yield very
inconsistent results when you use it to copy a filesystem that is in
use.
J.
--
I like my Toyota RAV4 because of the commanding view of the traffic
jams.
[Agree] [Disagree]
<http://www.slowlydownward.com/NODATA/data_enter2.html>
shawn wilson
On Feb 24, 2011 9:23 AM, "Karl E. Jorgensen" <ka...@fizzback.net> wrote:
>
> Hi!
>
> On Thu, Feb 24, 2011 at 02:32:43PM +0200, Dotan Cohen wrote:
> > I need to dd or cp my laptop's harddrive over the LAN. For a reason
> > that I'd rather not get into I cannot remove the drive from the
> > laptop.
> >
> > Should I just use scp to copy over the LAN? Something like this?
> > scp -r / ro...@178.63.65.136:/
>
> Well... SSH has some overhead because it does encryption - you simply cannot
> have encryption without _some_ overhead.
>
> I would recommend rsync instead - simply because if things go wrong during the
> copy, it can pick up from where it left, rather than restarting from scratch:
>
> rsync -av --numeric-ids / root@otherhost:/
>
IIRC, current rsync uses ssh by default anyway. You have to disable it to avoid that overhead.
However, rsync is the only option stated that allows resume. If you're using a boot cd, your only option might be to pipe in the data from another command because I seriously doubt you've got 40+ gigs of RAM on that laptop.
Boyd Stephen Smith Jr.
> No, it tries to talk to the rsync daemon by default, but you can run
> it over ssh using:
No, it only tries to connect to an rsync daemon if the remote "URL" uses two
colons like host::path. Otherwise it defaults to ssh. You can, of course,
force it to rsh if you like.
--
Boyd Stephen Smith Jr. ,= ,-_-. =.
b...@iguanasuicide.net ((_/)o o(\_))
ICQ: 514984 YM/AIM: DaTwinkDaddy `-'(. .)`-'
http://iguanasuicide.net/ \_/
Steven Ayre
it over ssh using:
rsync '-e ssh'
-Steve
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/AANLkTinMYf4453p9XPxz0...@mail.gmail.com
Boyd Stephen Smith Jr.
> On Qui, 24 Fev 2011, Tixy wrote:
> > What about hardlinks? I've always stuck to dd because I'm not confident
> > rsync will give me an exact enough copy to completely restore the
> > original from.
>
> rsync can preserve hard links, but that's not enabled by default by
> -a. IIRC the option is -H (or maybe -h ?), but check the man page.
I use -HaAxX (mnemonic: "hacks") when archiving over rsync. That preserves
everything (hard/symlinks, ACLs, permissions, extended attributes, etc.).
Steven Ayre
-Steve
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/AANLkTik7Xac_MuC1NqCxw...@mail.gmail.com
Curt Howland
Hash: SHA1
> I need to dd or cp my laptop's harddrive over the LAN.
Don't use dd if its a mounted file system.
scp will work, but you have to be careful about thing like symlinks
which scp WILL FOLLOW, and which can increase disk space use.
> scp -r / ro...@178.63.65.136:/
Two reasons this won't work.
1) most of the time, "root" login through ssh is blocked.
2) copying to "/" will overwrite the existing, running, system.
Much better, if you have the disk space on the receiving machine, is
to make a directory under your normal user,
like "/home/dotan/Laptop", and do something like this, AS ROOT:
# scp -r / do...@178.63.65.136:Laptop
and then the entire laptop file system will be available to recover at
will.
But in general, it's necessary to back up only /etc and /home. These
are where settings and user data are stored, and rebuilding the whole
system it can be better to build the "system" anew, then just recover
the user data and any needed custom settings from /etc.
> What is the canonical way of doing this?
Canonical runs Ubuntu, you need to ask that question in the Ubuntu
forums. This is the Debian user list, and while Ubuntu gets their
software packages mostly from Debian, they have their own issues with
versions and custom packages.
But this is a more generic kind of question, so don't worry about
that.
> Thanks!
You're very welcome, and I hope your problems work out successfully.
Curt-
- --
Those who torment us for our own good will torment us without end,
for they do so with the approval of their consciences.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
iQEVAwUBTWZ30C9Y35yItIgBAQKxzwf/W7IsbYXftoiD9nh0OhO6UM6cyVPe/DBL
6bbQzLCyOsxNgc/Qrz993MIjWIWffpUSPnEL4T1JAI8j26whxvgmAV6N4X4kxgq4
hiP/vY9TWx3dBLYnvnlVeOqIAGG1/fsEe7wEGb2Ti2fUtpBDWXI6EyWfRrzGUsrx
jxBef+QO831vKMcrFvhJgkhSP01nLV5qcQjHhB/tSWRfz0IkBi4gAJP6uhonJM0x
pD03PAlGClhirDqejdPR8u4bjTBhzQRKBsIKDNtAmLx/rkLxwjbZx2sgkpH8eb8M
zjHocmNe1+5w0pZZ63PEnwhOET/1Q6eaUGy3Cu9xnPwRMEb4e6Rj0Q==
=FGQz
-----END PGP SIGNATURE-----
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Darac Marjal
>
> > What is the canonical way of doing this?
>
> Canonical runs Ubuntu, you need to ask that question in the Ubuntu
> forums. This is the Debian user list, and while Ubuntu gets their
> software packages mostly from Debian, they have their own issues with
> versions and custom packages.
>
> But this is a more generic kind of question, so don't worry about
> that.
That's Canonical with a capital C. This is canonical with a lower case
C. It basically means "what's the accepted/correct way of doing this?".
While there may be more than one way to achieve a task, there's often
one best practice; that's the canonical method.
Boyd Stephen Smith Jr.
> > What is the canonical way of doing this?
>
> Canonical runs Ubuntu, you need to ask that question in the Ubuntu
> forums.
Whoa there!
He used the word "canonical", not the trademark "Canonical". From wiktionary:
Adjective
canonical (comparative more canonical, superlative most canonical)
1. Present in a canon, religious or otherwise.
The Gospel of Luke is a canonical New Testament book.
2. According to recognised or orthodox rules.
The men played golf in the most canonical way, with no local rules.
3. Stated or used in the most basic and straightforwardly applicable manner.
This definition would be more useful if it were canonical.
4. Prototypical.
5. (religion) In conformity with canon law.
6. (music) In the form of a canon.
7. (religion) Of or pertaining to an ecclesiastical chapter
8. (mathematics, computing) In canonical form.
(More at: <http://en.wiktionary.org/wiki/canonical>)
I don't appreciate Ubnutu questions on this mailing list (they have their
own), but the grandparent post had nothing to do with Ubuntu.
Curt Howland
Hash: SHA1
On Thursday 24 February 2011, Darac Marjal <maili...@darac.org.uk>
> That's Canonical with a capital C. This is canonical with a lower
> case C.
Having never heard the word used in a computer context except in
reference to Canonical/Ubuntu, I plead simple error.
Good thing he didn't say concatenate, I likely would have associated
it with cat food.
Curt-
- --
Those who torment us for our own good will torment us without end,
for they do so with the approval of their consciences.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
iQEVAwUBTWaccC9Y35yItIgBAQL1NAf+LQ5jRUDpjpOYLwmX0fRbCwMxsIoEsQcp
t1GSx46kHRxhHAc5HYhowK/3k/OCa6D+GVvKEWSAcQSE6Oua06WtbQo9myRRgxZn
L0un4ttexsVJ/GJtICGKL4HmHCTIuVV5wA1RzxpdSRED05ZHRPvnc4YzJmpfMg31
e6sL1RMVpWdHJA/fkGeOtLfMBhTfpjX2DoR7CJwoVbXd2sESU1bWLf4Bz2uDPvoC
KM8/6+q8zhBek8r3vg/dAU1F5TXNeDwbgI400THBeHIgrLWJVxDxnzdKM2gvPKKm
74wiUdC2AmIo0GhzDMs3TOuS7W1K+oHXG7X7G/MPm6nHBJwQFXTlqw==
=gnw5
-----END PGP SIGNATURE-----
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
shawn wilson
On Feb 24, 2011 10:31 AM, "Steven Ayre" <stev...@gmail.com> wrote:
>
> Hmm, interesting. Didn't realise that was the case.
>
> -Steve
>
>
> On 24 February 2011 15:06, Boyd Stephen Smith Jr. <b...@iguanasuicide.net> wrote:
> > On Thursday 24 February 2011 09:00:33 Steven Ayre wrote:
> >> No, it tries to talk to the rsync daemon by default, but you can run
> >> it over ssh using:
> >
> > No, it only tries to connect to an rsync daemon if the remote "URL" uses two
> > colons like host::path. Otherwise it defaults to ssh. You can, of course,
> > force it to rsh if you like.
> > --
IIRC, this is a recent (<2 years old) convention though and is just for gnu rsync.
Boyd Stephen Smith Jr.
On Thursday 24 February 2011 14:20:57 shawn wilson wrote:
> On Feb 24, 2011 10:31 AM, "Steven Ayre" <stev...@gmail.com> wrote:
> > On 24 February 2011 15:06, Boyd Stephen Smith Jr. <b...@iguanasuicide.net>
> > wrote:
> > > On Thursday 24 February 2011 09:00:33 Steven Ayre wrote:
> > >> No, it tries to talk to the rsync daemon by default, but you can run
> > >> it over ssh using:
> > > No, it only tries to connect to an rsync daemon if the remote "URL"
> > > uses
> > > two
> > > colons like host::path. Otherwise it defaults to ssh. You can, of
> > > course,
> > > force it to rsh if you like.
> > Hmm, interesting. Didn't realise that was the case.
>
> IIRC, this is a recent (<2 years old) convention though and is just for gnu
> rsync.
Hrm, this (rsync defaults to ssh) was true when I was installing Gentoo in
late 2004.
I don't have a lot of experience with non-GNU rsync though; rsync is not part
of any standard I could find, so I simply assume it doesn't exist on the AIX
and HP-UX systems that I use.
shawn wilson
On Feb 24, 2011 3:40 PM, "Celejar" <cel...@gmail.com> wrote:
>
> On Thu, 24 Feb 2011 10:22:56 -0500
> Curt Howland <How...@priss.com> wrote:
>
> ...
>
> > But in general, it's necessary to back up only /etc and /home. These
> > are where settings and user data are stored, and rebuilding the whole
> > system it can be better to build the "system" anew, then just recover
> > the user data and any needed custom settings from /etc.
>
> I'd strongly recommend backing up at least parts of /var (what if you
> have mail in the mail spool when the system dies?).
>
I have a development vm that I backup with a simple svn commit from my home directory.
In other words, your backup scheme should depend on what the system is used for.
Celejar
Curt Howland <How...@priss.com> wrote:
...
> But in general, it's necessary to back up only /etc and /home. These
> are where settings and user data are stored, and rebuilding the whole
> system it can be better to build the "system" anew, then just recover
> the user data and any needed custom settings from /etc.
I'd strongly recommend backing up at least parts of /var (what if you
have mail in the mail spool when the system dies?).
Celejar
--
foffl.sourceforge.net - Feeds OFFLine, an offline RSS/Atom aggregator
mailmin.sourceforge.net - remote access via secure (OpenPGP) email
ssuds.sourceforge.net - A Simple Sudoku Solver and Generator
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/20110224154014....@gmail.com
Celejar
Certainly - but my advice stands for typical, general purpose desktops,
laptops and servers.
Celejar
--
foffl.sourceforge.net - Feeds OFFLine, an offline RSS/Atom aggregator
mailmin.sourceforge.net - remote access via secure (OpenPGP) email
ssuds.sourceforge.net - A Simple Sudoku Solver and Generator
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/20110224163309....@gmail.com
Andrei Popescu
>
> Certainly - but my advice stands for typical, general purpose desktops,
> laptops and servers.
I doubt typical, general purpose desktops and laptops (yes, I omitted
servers) are using /var/spool/mail or /var/spool/cron/crontabs
Regards,
Andrei
--
Offtopic discussions among Debian users and developers:
http://lists.alioth.debian.org/mailman/listinfo/d-community-offtopic
Boyd Stephen Smith Jr.
> On Jo, 24 feb 11, 16:33:09, Celejar wrote:
> > Certainly - but my advice stands for typical, general purpose desktops,
> > laptops and servers.
>
> I doubt typical, general purpose desktops and laptops (yes, I omitted
> servers) are using /var/spool/mail or /var/spool/cron/crontabs
Having a couple of user cronjobs around is pretty normal for me. And while
remote mail is optional, I consider a system that can't deliver local mail to
be incomplete.
There's also other stuff under /var I use. PostgreSQL databases for one.
NB: I don't keep backups. I know I should, but I don't. I gave up after
calculating the size of the pile of DVDs I'd need to back up my 4TB file
system.
Celejar
"Boyd Stephen Smith Jr." <b...@iguanasuicide.net> wrote:
> On Thursday 24 February 2011 16:10:09 Andrei Popescu wrote:
> > On Jo, 24 feb 11, 16:33:09, Celejar wrote:
> > > Certainly - but my advice stands for typical, general purpose desktops,
> > > laptops and servers.
> >
> > I doubt typical, general purpose desktops and laptops (yes, I omitted
> > servers) are using /var/spool/mail or /var/spool/cron/crontabs
>
> Having a couple of user cronjobs around is pretty normal for me. And while
> remote mail is optional, I consider a system that can't deliver local mail to
> be incomplete.
>
> There's also other stuff under /var I use. PostgreSQL databases for one.
And various software keeps stuff under /var/lib - kismet, bluetooth.
> NB: I don't keep backups. I know I should, but I don't. I gave up after
> calculating the size of the pile of DVDs I'd need to back up my 4TB file
Why not use HDDs?
Celejar
--
foffl.sourceforge.net - Feeds OFFLine, an offline RSS/Atom aggregator
mailmin.sourceforge.net - remote access via secure (OpenPGP) email
ssuds.sourceforge.net - A Simple Sudoku Solver and Generator
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/20110224174930....@gmail.com
Celejar
Andrei Popescu <andreim...@gmail.com> wrote:
> On Jo, 24 feb 11, 16:33:09, Celejar wrote:
> >
> > Certainly - but my advice stands for typical, general purpose desktops,
> > laptops and servers.
>
> I doubt typical, general purpose desktops and laptops (yes, I omitted
> servers) are using /var/spool/mail or /var/spool/cron/crontabs
I always thought my getmail POP setup was pretty typical - I have it
configured to use /var/spool/mail/username as the mail spool. Is that
atypical, or a bad idea for some reason?
And my crontab does seem to be stored
under /var/spool/cron/crontabs. Is that not typical?
Celejar
--
foffl.sourceforge.net - Feeds OFFLine, an offline RSS/Atom aggregator
mailmin.sourceforge.net - remote access via secure (OpenPGP) email
ssuds.sourceforge.net - A Simple Sudoku Solver and Generator
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/20110224174741....@gmail.com
Boyd Stephen Smith Jr.
> On Thu, 24 Feb 2011 16:27:20 -0600
> "Boyd Stephen Smith Jr." <b...@iguanasuicide.net> wrote:
> > NB: I don't keep backups. I know I should, but I don't. I gave up after
> > calculating the size of the pile of DVDs I'd need to back up my 4TB file
>
> Why not use HDDs?
$$$
The data store itself costs $$$ which I can barely justify, to keep one backup
I'd have to pay 2x$$$ total and that's not justifiable. I've already had to
stymie the normal growth rate of my data store. :(
shawn wilson
On Thu, 24 Feb 2011 15:48:00 -0500
Certainly - but my advice stands for typical, general purpose desktops,> On Feb 24, 2011 3:40 PM, "Celejar" <cel...@gmail.com> wrote:
> >
> > On Thu, 24 Feb 2011 10:22:56 -0500
> > Curt Howland <How...@priss.com> wrote:
> >
> > ...
> >
> > > But in general, it's necessary to back up only /etc and /home. These
> > > are where settings and user data are stored, and rebuilding the whole
> > > system it can be better to build the "system" anew, then just recover
> > > the user data and any needed custom settings from /etc.
> >
> > I'd strongly recommend backing up at least parts of /var (what if you
> > have mail in the mail spool when the system dies?).
> >
>
> I have a development vm that I backup with a simple svn commit from my home
> directory.
>
> In other words, your backup scheme should depend on what the system is used
> for.
laptops and servers.
funny, i don't have a "general purpose" anything that needs all that backed up. the closest thing to 'general purpose' i have is graphic terminals - a macbook pro and a kubuntu box. however, for those machines, the only thing i really care about is my firefox profile. other than that, i have single purpose machines / vms - a mac mini media server (has one directory i care about), a dev box (backed up with svn), a mysql server (mysqldump does it there), drupal test (i'm not backing that up at all :) ), i backup the mini and sql server between each other and the sql server holds backups for everything else.
so, what everyone is saying is that, on any and all linux computers, one should keep a backup of:
/etc - for configurations - why? this is pointless if i don't change anything / much. i generally end up shutting down unneeded services on new installs, but unless there's apache or slight changes to my.cnf (which i always have other machines with edited configs of) i don't really change much.
/home - my dev box has a /home/cpan which i rsync daily, do i really want to back that up? that's pointless imo. and, there's only one real user there - me!
/var - WTF? why? tons of useless old logs to delete? a mail spool full of cron telling me that things have been done? my databases that were backed up elsewhere? did i leave anything out or is this absolute stupidity?
/root - oh this is great, here is my crap. yes, full of wget tests or partial log files when i'm looking for something but i'm not exactly sure what it is. or, c programs that have already been installed (that i've probably documented and maybe even made an svn trunk for), i see a file called ./fail, ./grep, and ./generate.txt in there too. yeah, i'm sure i'll want to go through that pile of crap after rma'ing a hdd.
/usr/share/perl5 - now i have thought of backing this up (i don't). however, if shit hits the fan and i have to spend a day recompiling everything that i use (or other modules might use), i'm going to want to kill lots of things - probably every things. but, my other thought is that i don't do an update until there's a feature or something that i want, so it'd probably be good. i'd still want to kill though.
Dotan Cohen
> netcat may help:
>
> dd if=/dev/sdX | nc -l 4321
>
> On the other box:
>
> nc ip.of.the.serverbox 4321 | dd of=/dev/sdY
>
Somehow I've never heard of netcat before. That is one handy feline.
--
Dotan Cohen
http://gibberish.co.il
http://what-is-what.com
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/AANLkTim621q6YwV5zvfzJ...@mail.gmail.com
Dotan Cohen
> You can use dd via ssh if you wish. I think the command would be:
> dd if=/dev/sda1 | ssh user@host "dd of=/path/to/sda1.img"
>
Wow, that is really cool. I did not know that dd could be piped as such. Thanks!
--
Dotan Cohen
http://gibberish.co.il
http://what-is-what.com
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/AANLkTingbW2DQ0CEphZUq...@mail.gmail.com
Dotan Cohen
<sjo...@lorentz.leidenuniv.nl> wrote:
> I would use rsync, as it allows compression
>
> rsync -aAXxPz / ro...@178.63.65.136:/
> where -a preserves permissions, modification times etc
> -A preserves acls
> -X preserves xattrs
> -x keeps you in one filesystem (so you won't copy /dev etc.)
> -P keeps partial copied files for faster stop-and-proceed, and shows a
> progress bar
> -z compresses the data stream
>
> This should give you an identical copy. Good luck!
>
Thanks, Sjoerd, I do use rsync when I need to copy only changes. I
thought that scp would be faster here. But I suppose you're right.
Thanks for the tips on which options to use. I didn't know that -x
keeps one out of /dev!
--
Dotan Cohen
http://gibberish.co.il
http://what-is-what.com
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/AANLkTikfo=i8=FPnQbD2koFRnzidvV2gzcPnEBf=5w...@mail.gmail.com
Dotan Cohen
> I would recommend rsync instead - simply because if things go wrong during the
> copy, it can pick up from where it left, rather than restarting from scratch:
>
> rsync -av --numeric-ids / root@otherhost:/
>
> --numeric-ids is to ensure it uses the _numeric_ user/group IDS on the other
> side. You may find that some users have differnt IDs between machines - e.g.
> IDs that are added by packages (e.g. mysql and stuff).
>
>
Thanks, I actually did once run into the problem of uids on a Fedora
box being different from those on a Debian box.
>> Can I actually use dd over the network, maybe by piping to scp
>> somehow? What is the canonical way of doing this?
>> Thanks!
>
> Yes - if the disks are the same size, it's a no-brainer. My personal preference
> for this is to use "nc" - which is lightweight, but does not do encryption.
>
> On the "source" machine, send it out over the network
> sourcebox:~# dd if=/dev/sda | nc -l -p 9999
>
> and on the destination box - which should be running a "live CD" or similar, as
> blasting over a disk which is in use leads to kernel confusion and human
> madness:
> destbox:~# nc $IP_OF_OTHER_BOX 9999 | dd of=/dev/sda
>
> You may want to use gzip to trade off CPU usage for better network bandwith:
>
> sourcebox:~# dd if=/dev/sda | gzip | nc -l -p 9999
> destbox:~# nc $IP_OF_OTHER_BOX 9999 | gunzip | dd of=/dev/sda
>
> Hope this helps
Thanks Karl!
--
Dotan Cohen
http://gibberish.co.il
http://what-is-what.com
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/AANLkTinvzX50bVQAi5tD...@mail.gmail.com
Dotan Cohen
>> I need to dd or cp my laptop's harddrive over the LAN.
>
> Don't use dd if its a mounted file system.
>
Thanks!
> scp will work, but you have to be careful about thing like symlinks
> which scp WILL FOLLOW, and which can increase disk space use.
>
>> scp -r / ro...@178.63.65.136:/
>
> Two reasons this won't work.
>
> 1) most of the time, "root" login through ssh is blocked.
>
> 2) copying to "/" will overwrite the existing, running, system.
>
> Much better, if you have the disk space on the receiving machine, is
> to make a directory under your normal user,
> like "/home/dotan/Laptop", and do something like this, AS ROOT:
>
> # scp -r / do...@178.63.65.136:Laptop
>
> and then the entire laptop file system will be available to recover at
> will.
>
Of course, it was only a syntax example.
> But in general, it's necessary to back up only /etc and /home. These
> are where settings and user data are stored, and rebuilding the whole
> system it can be better to build the "system" anew, then just recover
> the user data and any needed custom settings from /etc.
>
>> What is the canonical way of doing this?
>
> Canonical runs Ubuntu, you need to ask that question in the Ubuntu
> forums. This is the Debian user list, and while Ubuntu gets their
> software packages mostly from Debian, they have their own issues with
> versions and custom packages.
>
Nice! I'll use that line next time someone asks me for a canonical matrix!
--
Dotan Cohen
http://gibberish.co.il
http://what-is-what.com
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/AANLkTi=1dNnxNV+FnTTpi2r1Dca8F5RUeFxL=105...@mail.gmail.com
Dotan Cohen
> I'd strongly recommend backing up at least parts of /var (what if you
> have mail in the mail spool when the system dies?).
>
Also, /var/www might be a bit important. Aren't the MySQL files in /var as well.
I also do /opt if it exists.
--
Dotan Cohen
http://gibberish.co.il
http://what-is-what.com
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/AANLkTim3W2N1CGWzVyhfd...@mail.gmail.com
Dotan Cohen
> I always thought my getmail POP setup was pretty typical - I have it
> configured to use /var/spool/mail/username as the mail spool. Is that
> atypical, or a bad idea for some reason?
>
It is atypical as it is not webmail! Very few people today are using
something they access via POP3 or IMAP, and even most of those aren't
running on their desktop but rather at the company.
I personally use the POP3 / SMTP of my website host, as I hate webmail
(but gmail is great for mailing lists). But people are surprised when
they discover that I can't just check and send mail from any computer
with a web browser. Most people don't even know that there is an
alternative to webmail.
Oh, and Exchange is pretty popular too, but that is typically
associated with business mail, somehow that is a different paradigm
from "personal mail" for Exchange users.
--
Dotan Cohen
http://gibberish.co.il
http://what-is-what.com
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/AANLkTinpLk-d-RB4SNrGD...@mail.gmail.com
Celejar
On Fri, 25 Feb 2011 15:57:19 +0200
Dotan Cohen <dotan...@gmail.com> wrote:
> On Fri, Feb 25, 2011 at 00:47, Celejar <cel...@gmail.com> wrote:
> > I always thought my getmail POP setup was pretty typical - I have it
> > configured to use /var/spool/mail/username as the mail spool. Is that
> > atypical, or a bad idea for some reason?
> >
>
> It is atypical as it is not webmail! Very few people today are using
> something they access via POP3 or IMAP, and even most of those aren't
> running on their desktop but rather at the company.
Okay, I have no idea of the numbers, but POP3/IMAP is certainly the (or
a) classic way of doing mail.
> I personally use the POP3 / SMTP of my website host, as I hate webmail
> (but gmail is great for mailing lists). But people are surprised when
> they discover that I can't just check and send mail from any computer
> with a web browser. Most people don't even know that there is an
> alternative to webmail.
>
> Oh, and Exchange is pretty popular too, but that is typically
> associated with business mail, somehow that is a different paradigm
> from "personal mail" for Exchange users.
Celejar
--
foffl.sourceforge.net - Feeds OFFLine, an offline RSS/Atom aggregator
mailmin.sourceforge.net - remote access via secure (OpenPGP) email
ssuds.sourceforge.net - A Simple Sudoku Solver and Generator
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/20110225092249....@gmail.com
Celejar
"Boyd Stephen Smith Jr." <b...@iguanasuicide.net> wrote:
> On Thursday 24 February 2011 16:49:30 Celejar wrote:
> > On Thu, 24 Feb 2011 16:27:20 -0600
> > "Boyd Stephen Smith Jr." <b...@iguanasuicide.net> wrote:
> > > NB: I don't keep backups. I know I should, but I don't. I gave up after
> > > calculating the size of the pile of DVDs I'd need to back up my 4TB file
> >
> > Why not use HDDs?
>
> $$$
>
> The data store itself costs $$$ which I can barely justify, to keep one backup
> I'd have to pay 2x$$$ total and that's not justifiable. I've already had to
> stymie the normal growth rate of my data store. :(
Ah, okay. That certainly makes sense.
Celejar
--
foffl.sourceforge.net - Feeds OFFLine, an offline RSS/Atom aggregator
mailmin.sourceforge.net - remote access via secure (OpenPGP) email
ssuds.sourceforge.net - A Simple Sudoku Solver and Generator
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/20110225092431....@gmail.com
Celejar
You may not, but "general purpose" desktop / laptop linux machines are
quite common. My statements stand.
> up. the closest thing to 'general purpose' i have is graphic terminals - a
> macbook pro and a kubuntu box. however, for those machines, the only thing i
> really care about is my firefox profile. other than that, i have single
> purpose machines / vms - a mac mini media server (has one directory i care
> about), a dev box (backed up with svn), a mysql server (mysqldump does it
> there), drupal test (i'm not backing that up at all :) ), i backup the mini
> and sql server between each other and the sql server holds backups for
> everything else.
>
> so, what everyone is saying is that, on any and all linux computers, one
> should keep a backup of:
> /etc - for configurations - why? this is pointless if i don't change
> anything / much. i generally end up shutting down unneeded services on new
> installs, but unless there's apache or slight changes to my.cnf (which i
> always have other machines with edited configs of) i don't really change
> much.
You're missing the point. Of course those who don't make any config
changes under /etc needn't bother with it, but many of us do. Just
because linux gives us the flexibility to run very different sorts of
systems doesn't mean that it's bad to give general advice (noting it as
such) that is widely applicable.
> /home - my dev box has a /home/cpan which i rsync daily, do i really want to
> back that up? that's pointless imo. and, there's only one real user there -
> me!
Most /home s on desktop / laptop boxes contain more interesting
things. See above.
> /var
- WTF? why? tons of useless old logs to delete? a mail spool full of
I said parts of /var. If you don't get *any* important mail *at all*
in your spool, than by all means don't back it up. I'm under the
impression that many of us do get real mail there.
> cron telling me that things have been done? my databases that were backed up
> elsewhere? did i leave anything out or is this absolute stupidity?
> /root - oh this is great, here is my crap. yes, full of wget tests or
> partial log files when i'm looking for something but i'm not exactly sure
> what it is. or, c programs that have already been installed (that i've
> probably documented and maybe even made an svn trunk for), i see a file
> called ./fail, ./grep, and ./generate.txt in there too. yeah, i'm sure i'll
> want to go through that pile of crap after rma'ing a hdd.
> /usr/share/perl5 - now i have thought of backing this up (i don't). however,
> if shit hits the fan and i have to spend a day recompiling everything that i
> use (or other modules might use), i'm going to want to kill lots of things -
> probably every things. but, my other thought is that i don't do an update
> until there's a feature or something that i want, so it'd probably be good.
> i'd still want to kill though.
Celejar
--
foffl.sourceforge.net - Feeds OFFLine, an offline RSS/Atom aggregator
mailmin.sourceforge.net - remote access via secure (OpenPGP) email
ssuds.sourceforge.net - A Simple Sudoku Solver and Generator
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/20110225093244....@gmail.com
Andrei Popescu
> On Thursday 24 February 2011 16:10:09 Andrei Popescu wrote:
> > On Jo, 24 feb 11, 16:33:09, Celejar wrote:
> > > Certainly - but my advice stands for typical, general purpose desktops,
> > > laptops and servers.
> >
> > I doubt typical, general purpose desktops and laptops (yes, I omitted
> > servers) are using /var/spool/mail or /var/spool/cron/crontabs
>
> Having a couple of user cronjobs around is pretty normal for me. And while
> remote mail is optional, I consider a system that can't deliver local mail to
> be incomplete.
>
> There's also other stuff under /var I use. PostgreSQL databases for one.
>
> NB: I don't keep backups. I know I should, but I don't. I gave up after
> calculating the size of the pile of DVDs I'd need to back up my 4TB file
> system.
I don't consider a debian-user subscriber a "typical desktop / laptop
user" :)
Regards
David Jardine
> I don't consider a debian-user subscriber a "typical desktop / laptop
> user" :)
A typical debian-user subscriber may not be a typical desktop/laptop
user, but a typical Debian desktop/laptop user may well be a
debian-user subscriber, I think.
Cheers,
David
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/2011022522...@gennes.augarten
Dotan Cohen
Sorry. The Open Office list is just the opposite (we _must_ cc as one
need not be subscribed to post, and lots of new users don't
subscribe). I'm on 40+ lists so I loose track.
>> It is atypical as it is not webmail! Very few people today are using
>> something they access via POP3 or IMAP, and even most of those aren't
>> running on their desktop but rather at the company.
>
> Okay, I have no idea of the numbers, but POP3/IMAP is certainly the (or
> a) classic way of doing mail.
>
I don't have numbers either, but POP3/IMAP/SMTP is certainly rare in
the under-30 crowd. Rare, as in they don't even know that options
exists, and don't understand how one could have email but not have
access to it from any web browser.
--
Dotan Cohen
http://gibberish.co.il
http://what-is-what.com
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/AANLkTim-fQ3OUtsqk8WtQ...@mail.gmail.com
shawn wilson
On Fri, Feb 25, 2011 at 16:22, Celejar <cel...@gmail.com> wrote:I don't have numbers either, but POP3/IMAP/SMTP is certainly rare in
>> It is atypical as it is not webmail! Very few people today are using
>> something they access via POP3 or IMAP, and even most of those aren't
>> running on their desktop but rather at the company.
>
> Okay, I have no idea of the numbers, but POP3/IMAP is certainly the (or
> a) classic way of doing mail.
>
the under-30 crowd. Rare, as in they don't even know that options
exists, and don't understand how one could have email but not have
access to it from any web browser.
that said, i use the gmail web interface from my computer. and imap+idle to my phone pushes mail faster than their webui... go figure.
Andrei Popescu
>
> I don't have numbers either, but POP3/IMAP/SMTP is certainly rare in
> the under-30 crowd. Rare, as in they don't even know that options
> exists, and don't understand how one could have email but not have
> access to it from any web browser.
I agree with the first part, but you're making it sound like
POP3/IMAP4/SMTP is excluding webmail ;)
Celejar
David Jardine <da...@jardine.de> wrote:
> On Fri, Feb 25, 2011 at 10:40:55PM +0200, Andrei Popescu wrote:
>
> > I don't consider a debian-user subscriber a "typical desktop / laptop
> > user" :)
>
> A typical debian-user subscriber may not be a typical desktop/laptop
> user, but a typical Debian desktop/laptop user may well be a
> debian-user subscriber, I think.
FWIW, I was the one who originally used the phrase in question, and
that's pretty much what I meant.
Celejar
--
foffl.sourceforge.net - Feeds OFFLine, an offline RSS/Atom aggregator
mailmin.sourceforge.net - remote access via secure (OpenPGP) email
ssuds.sourceforge.net - A Simple Sudoku Solver and Generator
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/20110227143331....@gmail.com
Celejar
Dotan Cohen <dotan...@gmail.com> wrote:
> On Fri, Feb 25, 2011 at 16:22, Celejar <cel...@gmail.com> wrote:
> > [Please don't cc me on replies.]
> >
>
> Sorry. The Open Office list is just the opposite (we _must_ cc as one
> need not be subscribed to post, and lots of new users don't
> subscribe). I'm on 40+ lists so I loose track.
No problem - just reminding you ;)
Celejar
--
foffl.sourceforge.net - Feeds OFFLine, an offline RSS/Atom aggregator
mailmin.sourceforge.net - remote access via secure (OpenPGP) email
ssuds.sourceforge.net - A Simple Sudoku Solver and Generator
--
To UNSUBSCRIBE, email to debian-us...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/20110227165629....@gmail.com