Help - how to clone disk image on Linux?
L
I have a server running RH Linux 9 with two HDD slots. What is the easiest
way to clone a disk, i.e. copy the whole and exact image of the disk in slot
1 to an identical disk in slot 2?
Thanks,
T.
--
Dances With Crows
said:
> I have a server running RH Linux 9 with two HDD slots. What is the
> easiest way to clone a disk, i.e. copy the whole and exact image of
> the disk in slot 1 to an identical disk in slot 2?
cp -a , or tar , or partimage ( http://partimage.com/ ). If any
filesystem on disk 0 is mounted and is writeable, copying that
filesystem to disk 1 will be difficult because files on the mounted
filesystem may change while the copy/tar/partimage is running.
--
Matt G|There is no Darkness in Eternity/But only Light too dim for us to see
Brainbench MVP for Linux Admin / mail: TRAP + SPAN don't belong
http://www.brainbench.com / Hire me!
-----------------------------/ http://crow202.dyndns.org/~mhgraham/resume
matthieu imbert
> On Thu, 2 Sep 2004 11:23:19 -0500, L staggered into the Black Sun and
> said:
>
>>I have a server running RH Linux 9 with two HDD slots. What is the
>>easiest way to clone a disk, i.e. copy the whole and exact image of
>>the disk in slot 1 to an identical disk in slot 2?
>
>
> cp -a , or tar , or partimage ( http://partimage.com/ ). If any
> filesystem on disk 0 is mounted and is writeable, copying that
> filesystem to disk 1 will be difficult because files on the mounted
> filesystem may change while the copy/tar/partimage is running.
>
ownerships) with great success to transfer the whole system
of a linux box to a newer one. Don't forget though to do
this for every partition (for example /boot and / ). You'll
also have to install the boot sector manually with lilo or grub.
to avoid the problem of files changing during the copy, i
booted with a live linux cd (i used SystemRescueCd).
Also, if the two disks are exactly the same, dd could also
be used i guess.
L
"matthieu imbert" <nos...@nospam.com> wrote in message
news:41375558$0$26992$626a...@news.free.fr...
Can I use /dev/sda (the source disk) and /dev/sdb (the dest disk) to copy
the whole and exact image from the src disk to the dest disk some how?
Thanks,
T.
--
L
"L" <l_x...@yahoo.com> wrote in message
news:2pp3q4F...@uni-berlin.de...
Once boot up Linux from the source disk, can I use "dd if=/dev/sda
of=/dev/sdb" to copy the whole and exact image from the source disk
(/dev/sda) to the dest disk (/dev/sdb)?
Thanks,
T.
--
Dances With Crows
said:
Why did you post this twice?
> "L" <l_x...@yahoo.com> wrote in message
> news:2pp3q4F...@uni-berlin.de...
>> "matthieu imbert" <nos...@nospam.com> wrote in message
>> news:41375558$0$26992$626a...@news.free.fr...
>> > Dances With Crows wrote:
>> > > On Thu, 2 Sep 2004 11:23:19 -0500, L staggered into the Black Sun
>> > > and said:
>> > >>I have a server running RH Linux 9 with two HDD slots. What is
>> > >>the easiest way to clone a disk, i.e. copy the whole and exact
>> > >>image of the disk in slot 1 to an identical disk in slot 2?
>> > > cp -a , or tar , or partimage ( http://partimage.com/ ). If any
>> > i have used cp -a (the -a preserves attributes and
>> > ownerships) with great success to transfer the whole system
>> > of a linux box to a newer one. Don't forget though to do
>> Can I use /dev/sda (the source disk) and /dev/sdb (the dest disk) to
>> copy the whole and exact image from the src disk to the dest disk
>> some how?
> Once boot up Linux from the source disk, can I use "dd if=/dev/sda
> of=/dev/sdb" to copy the whole and exact image from the source disk
> (/dev/sda) to the dest disk (/dev/sdb)?
You can, but it'd be a really dumb idea. dd copies *everything*, including
all the unused space on the disk, so it's slower than using tar or cp -a
or partimage. If the disks have different sizes, the partition table on
the destination disk will also be incorrect. If you decide to use dd
anyway, don't say we didn't warn you, and make sure to pass the bs=32k
parameter, since dd uses a default blocksize of 512 bytes, and that will
make things take even longer.
Use tar or cp -a or partimage. Really. That's what they are designed
for.
spi...@freenet.co.uk
> Can I use /dev/sda (the source disk) and /dev/sdb (the dest disk) to copy
> the whole and exact image from the src disk to the dest disk some how?
If the two disks have identical sizes and geometries
(sectors/tracks/cylinders), then yes.
dd if=/dev/sda of=/dev/sdb bs=512
If the second disk is bigger, you may have problems with that method, cos
your partitions will be the same sizes and you might not be able to make
more... I've never tried dd on different sized disks.
--
______________________________________________________________________________
| spi...@freenet.co.uk | "I'm alive!!! I can touch! I can taste! |
|Andrew Halliwell BSc(hons)| I can SMELL!!! KRYTEN!!! Unpack Rachel and |
| in | get out the puncture repair kit!" |
| Computer Science | Arnold Judas Rimmer- Red Dwarf |
------------------------------------------------------------------------------
Alexander Skwar
> If the two disks have identical sizes and geometries
> (sectors/tracks/cylinders), then yes.
>
> dd if=/dev/sda of=/dev/sdb bs=512
I'm curious: Why's everyone using dd for this and not the IMO simpler
cat (as in "cat /dev/sda > /dev/sdb"). Is there any advantage in
using dd?
Alexander Skwar
--
You can't start worrying about what's going to happen. You get spastic
enough worrying about what's happening now.
-- Lauren Bacall
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
Paul Lutus
/ ...
>> Can I use /dev/sda (the source disk) and /dev/sdb (the dest disk) to copy
>> the whole and exact image from the src disk to the dest disk some how?
/ ...
> Once boot up Linux from the source disk, can I use "dd if=/dev/sda
> of=/dev/sdb" to copy the whole and exact image from the source disk
> (/dev/sda) to the dest disk (/dev/sdb)?
Between posts, you added a condition that poses a problem. If you are using
the source disk as your active OS while copying, the destination copy will
not be identical to the source. The result can range between no noticeable
problems to a total and complete failure of the filesystem.
It is much better to boot from a bootable CD or other medium, and copy only
unmounted drives.
--
Paul Lutus
http://www.arachnoid.com
Jim Scott
correctly, you should probably do a checksum. I've copied 1 MB
blocks and done the checksum using MD5 on each block transferred.
Look at the man page for MD5 to see how to do it. The man page for dd
will show you how to skip to the area you want to copy. Look for the
options for skip and seek (for the input and output) as well as set
the block size (bs=?).
Or you could just use Ghost (from Symantec) to do it. It does
essentially the same thing.
Jim
Robert Heller
In a message on Thu, 02 Sep 2004 20:20:24 +0200, wrote :
AS> spi...@freenet.co.uk wrote:
AS>
AS> > If the two disks have identical sizes and geometries
AS> > (sectors/tracks/cylinders), then yes.
AS> >
AS> > dd if=/dev/sda of=/dev/sdb bs=512
AS>
AS> I'm curious: Why's everyone using dd for this and not the IMO simpler
AS> cat (as in "cat /dev/sda > /dev/sdb"). Is there any advantage in
AS> using dd?
Dd will work properly at the device level. Cat is really only meant for
concatenating normal files. DD can also perform buffering and
re-blocking (see 'man dd' for more details), which is sometimes needed
when moving data at this level.
Note that *any* I/O below the file system level is dangerous can extreme
care needs to be taken.
spike1's proviso 'If the two disks have identical sizes and geometries'
is a very important condition. While it is possible to copy from a
smaller to a larger disk and 'get away with it', the reverse will be bad
news. There can also be problems if one (or both) of the disks have bad
blocks that are mapped out at the FS level.
AS>
AS> Alexander Skwar
AS> --
AS> You can't start worrying about what's going to happen. You get spastic
AS> enough worrying about what's happening now.
AS> -- Lauren Bacall
AS> ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
AS>
\/
Robert Heller ||InterNet: hel...@cs.umass.edu
http://vis-www.cs.umass.edu/~heller || hel...@deepsoft.com
http://www.deepsoft.com /\FidoNet: 1:321/153
Robert Heller
In a message on Thu, 02 Sep 2004 11:24:08 -0700, wrote :
PL> L wrote:
PL>
PL> / ...
PL>
PL> >> Can I use /dev/sda (the source disk) and /dev/sdb (the dest disk) to copy
PL> >> the whole and exact image from the src disk to the dest disk some how?
PL>
PL> / ...
PL>
PL> > Once boot up Linux from the source disk, can I use "dd if=/dev/sda
PL> > of=/dev/sdb" to copy the whole and exact image from the source disk
PL> > (/dev/sda) to the dest disk (/dev/sdb)?
PL>
PL> Between posts, you added a condition that poses a problem. If you are using
PL> the source disk as your active OS while copying, the destination copy will
PL> not be identical to the source. The result can range between no noticeable
PL> problems to a total and complete failure of the filesystem.
PL>
PL> It is much better to boot from a bootable CD or other medium, and copy only
PL> unmounted drives.
It is quite possible to successfully 'clone' a running system, using
a disk-to-disk dump (eg: 'dump 0f - / | (cd /newdisk;restore rf -)',
repeat for /var, /usr, /home, etc.). You do need some kind of boot
media (rescue floppy/CD-ROM) to re-install the boot loader, if needed.
Using dd to copy a 'live' system disk is likely to be bad news. It is
not a problem using dump/cpio/tar/cp/etc. since these are work at the FS
level and won't be affected by (normal) low-level changes to the
filesystem -- all that would be lost would be randomly updated system
files (eg log files), which won't affect the runablity of the cloned
system disk.
PL>
PL> --
PL> Paul Lutus
PL> http://www.arachnoid.com
PL>
PL>
Alexander Skwar
> Dd will work properly at the device level. Cat is really only meant for
> concatenating normal files.
Okay, maybe I don't have enough experience, but did you ever have
problems with cat, that were not present with dd?
I haven't.
Alexander Skwar
--
I must Create a System, or be enslav'd by another Man's;
I will not Reason and Compare; my business is to Create.
-- William Blake, "Jerusalem"
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
Alexander Skwar
> Alexander Skwar <fr...@alexander.skwar.name>,
> In a message on Thu, 02 Sep 2004 20:20:24 +0200, wrote :
>
> AS> spi...@freenet.co.uk wrote:
> AS>
> AS> > If the two disks have identical sizes and geometries
> AS> > (sectors/tracks/cylinders), then yes.
> AS> >
> AS> > dd if=/dev/sda of=/dev/sdb bs=512
> AS>
> AS> I'm curious: Why's everyone using dd for this and not the IMO simpler
> AS> cat (as in "cat /dev/sda > /dev/sdb"). Is there any advantage in
> AS> using dd?
>
> Dd will work properly at the device level. Cat is really only meant for
> concatenating normal files.
So is dd, according to the man page.
dd copies a file (from standard input to standard output, by default)
using specific input and output blocksizes, while optionally performing
conversions on it.
> DD can also perform buffering and
> re-blocking
How?
> (see 'man dd' for more details),
Where? Are we talking about the same dd? I've got the man page of
"GNU fileutils 4.0" and "dd (coreutils) 5.2.1"; it's --help says:
Usage: dd [OPTION]...
Copy a file, converting and formatting according to the options.
bs=BYTES force ibs=BYTES and obs=BYTES
cbs=BYTES convert BYTES bytes at a time
conv=KEYWORDS convert the file as per the comma separated keyword list
count=BLOCKS copy only BLOCKS input blocks
ibs=BYTES read BYTES bytes at a time
if=FILE read from FILE instead of stdin
obs=BYTES write BYTES bytes at a time
of=FILE write to FILE instead of stdout
seek=BLOCKS skip BLOCKS obs-sized blocks at start of output
skip=BLOCKS skip BLOCKS ibs-sized blocks at start of input
--help display this help and exit
--version output version information and exit
Alexander Skwar
--
It is well known that *things* from undesirable universes are always seeking
an entrance into this one, which is the psychic equivalent of handy for the
buses and closer to the shops.
-- Terry Pratchett, "The Light Fantastic"
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
John Reiser
> cat (as in "cat /dev/sda > /dev/sdb"). Is there any advantage in
> using dd?
dd allows control of the blocksize (the number of bytes transfered
per system call), and using a blocksize such as bs=32k, 64k, or 128k
can make the transfer take less clock time. Modern harddrives
have builtin RAM buffer+cache with readahead+writebehind, so the
time savings is not as much as years ago, but taking only
1.5 hours by specifying a blocksize instead of 2 hours using
the defaults is nothing to ignore.
--
Dances With Crows
Black Sun and said:
> spi...@freenet.co.uk wrote:
>> If the two disks have identical sizes and geometries
>> (sectors/tracks/cylinders), then yes. dd if=/dev/sda of=/dev/sdb
>> bs=512
> I'm curious: Why's everyone using dd for this and not the IMO simpler
> cat (as in "cat /dev/sda > /dev/sdb"). Is there any advantage in using
> dd?
Block size. You can specify a block size to dd. It's faster to do 1
read() of 32k than it is to do 64 read()s of 512 bytes. cat probably
reads 4k at a time, and that's not optimal when you're copying a large
chunk of data.
And if you didn't know, dd is a *terrible* way to back up and/or mirror
something. It copies everything, even unused space, so it's slower than
tar or cp -a or partimage if your filesystem isn't 100% full. If the
second disk has a different size or a different geometry, the partition
table you copied from the first disk will be incorrect. The only
advantage dd has over other backup/mirror methods is that you don't have
to reinstall GRUB or LILO. Reinstalling GRUB takes 4 commands, which
you can put in a shell script like so:
#!/bin/bash
/sbin/grub --batch << EOF > /dev/null
root (hdX,Y)
setup (hdX)
quit
EOF
# modify X and Y for your particular setup.
...I forget what the proper commands are for LILO as it's been several
years since I've used it.
Robert Heller
In a message on Thu, 02 Sep 2004 22:09:55 +0200, wrote :
AS> Robert Heller wrote:
AS>
AS> > Dd will work properly at the device level. Cat is really only meant for
AS> > concatenating normal files.
AS>
AS> Okay, maybe I don't have enough experience, but did you ever have
AS> problems with cat, that were not present with dd?
Cat lacks some useful options:
dd if=boot.img of=/dev/fd0 bs=9216
Copies a floppy disk image and does the floppy I/O in track sized
buffers. This is a bit faster than cat boot.img >/dev/fd0, which does
it byte-by-byte (yes, there is buffering inside the glibc code, but not
specifically floppy track sized). Also dd's parameter methodology is
more convenient than cat when used with sudo. Compare:
sudo dd if=boot.img of=/dev/fd0
vs.
sudo sh -c 'cat boot.img >/dev/fd0'
Cat is quite useful in many situations (I use it all of the time), but
dd is a more powerful command and has many advantages in *other*
situations. Compare using a hammer to 'pound in' a screw. This can be
done, but a screwdriver works better (to drive a screw).
AS>
AS> I haven't.
AS>
AS>
AS> Alexander Skwar
AS> --
AS> I must Create a System, or be enslav'd by another Man's;
AS> I will not Reason and Compare; my business is to Create.
AS> -- William Blake, "Jerusalem"
Robert Heller
In a message on Thu, 02 Sep 2004 22:17:26 +0200, wrote :
AS> Robert Heller wrote:
AS> > Alexander Skwar <fr...@alexander.skwar.name>,
AS> > In a message on Thu, 02 Sep 2004 20:20:24 +0200, wrote :
AS> >
AS> > AS> spi...@freenet.co.uk wrote:
AS> > AS>
AS> > AS> > If the two disks have identical sizes and geometries
AS> > AS> > (sectors/tracks/cylinders), then yes.
AS> > AS> >
AS> > AS> > dd if=/dev/sda of=/dev/sdb bs=512
AS> > AS>
AS> > AS> I'm curious: Why's everyone using dd for this and not the IMO simpler
AS> > AS> cat (as in "cat /dev/sda > /dev/sdb"). Is there any advantage in
AS> > AS> using dd?
AS> >
AS> > Dd will work properly at the device level. Cat is really only meant for
AS> > concatenating normal files.
AS>
AS> So is dd, according to the man page.
AS>
AS> dd copies a file (from standard input to standard output, by default)
AS> using specific input and output blocksizes, while optionally performing
AS> conversions on it.
AS>
AS>
AS> > DD can also perform buffering and
AS> > re-blocking
AS>
AS> How?
Using different values for ibs and obs. For example:
dd ibs=40b obs=20b if=/dev/st0 | tar xvf -
Reads a tape recorded with a blocking factor of 40x512 bytes (20k) and
converts it to tar's standard 20x512 blocks (10k). Using a larger
block size can improve tape streaming and thus improve tape I/O
performance.
AS>
AS> > (see 'man dd' for more details),
AS>
AS> Where? Are we talking about the same dd? I've got the man page of
AS> "GNU fileutils 4.0" and "dd (coreutils) 5.2.1"; it's --help says:
'dd --help' is not the same as 'man dd'
Also, there is even more documentation at 'info dd', including some
examples.
AS>
AS> Usage: dd [OPTION]...
AS> Copy a file, converting and formatting according to the options.
AS>
AS> bs=BYTES force ibs=BYTES and obs=BYTES
AS> cbs=BYTES convert BYTES bytes at a time
AS> conv=KEYWORDS convert the file as per the comma separated keyword list
AS> count=BLOCKS copy only BLOCKS input blocks
AS> ibs=BYTES read BYTES bytes at a time
AS> if=FILE read from FILE instead of stdin
AS> obs=BYTES write BYTES bytes at a time
AS> of=FILE write to FILE instead of stdout
AS> seek=BLOCKS skip BLOCKS obs-sized blocks at start of output
AS> skip=BLOCKS skip BLOCKS ibs-sized blocks at start of input
AS> --help display this help and exit
AS> --version output version information and exit
AS>
AS>
AS>
AS> Alexander Skwar
AS> --
AS> It is well known that *things* from undesirable universes are always seeking
AS> an entrance into this one, which is the psychic equivalent of handy for the
AS> buses and closer to the shops.
AS> -- Terry Pratchett, "The Light Fantastic"
Robert Heller
In a message on 2 Sep 2004 20:21:19 GMT, wrote :
DWC> On Thu, 02 Sep 2004 20:20:24 +0200, Alexander Skwar staggered into the
DWC> Black Sun and said:
DWC> > spi...@freenet.co.uk wrote:
DWC> >> If the two disks have identical sizes and geometries
DWC> >> (sectors/tracks/cylinders), then yes. dd if=/dev/sda of=/dev/sdb
DWC> >> bs=512
DWC> > I'm curious: Why's everyone using dd for this and not the IMO simpler
DWC> > cat (as in "cat /dev/sda > /dev/sdb"). Is there any advantage in using
DWC> > dd?
DWC>
DWC> Block size. You can specify a block size to dd. It's faster to do 1
DWC> read() of 32k than it is to do 64 read()s of 512 bytes. cat probably
DWC> reads 4k at a time, and that's not optimal when you're copying a large
DWC> chunk of data.
DWC>
DWC> And if you didn't know, dd is a *terrible* way to back up and/or mirror
DWC> something. It copies everything, even unused space, so it's slower than
DWC> tar or cp -a or partimage if your filesystem isn't 100% full. If the
DWC> second disk has a different size or a different geometry, the partition
DWC> table you copied from the first disk will be incorrect. The only
DWC> advantage dd has over other backup/mirror methods is that you don't have
DWC> to reinstall GRUB or LILO. Reinstalling GRUB takes 4 commands, which
DWC> you can put in a shell script like so:
DWC>
DWC> #!/bin/bash
DWC> /sbin/grub --batch << EOF > /dev/null
DWC> root (hdX,Y)
DWC> setup (hdX)
DWC> quit
DWC> EOF
DWC> # modify X and Y for your particular setup.
DWC>
DWC> ...I forget what the proper commands are for LILO as it's been several
DWC> years since I've used it.
Assuming that /etc/lilo.conf exists (it would be copied with
tar/cp/cpio/dump/etc.), then it is just:
/sbin/lilo -r /newdisk
Otherwise lilo takes ALL of the config options directly on the command
line. So, lilo always takes one command, which may be simple or
complex, depending.
DWC>
DWC> --
DWC> Matt G|There is no Darkness in Eternity/But only Light too dim for us to see
DWC> Brainbench MVP for Linux Admin / mail: TRAP + SPAN don't belong
DWC> http://www.brainbench.com / Hire me!
DWC> -----------------------------/ http://crow202.dyndns.org/~mhgraham/resume
DWC>
Alexander Skwar
> On Thu, 02 Sep 2004 20:20:24 +0200, Alexander Skwar staggered into the
> Black Sun and said:
>
>>spi...@freenet.co.uk wrote:
>>
>>>If the two disks have identical sizes and geometries
>>>(sectors/tracks/cylinders), then yes. dd if=/dev/sda of=/dev/sdb
>>>bs=512
>>
>>I'm curious: Why's everyone using dd for this and not the IMO simpler
>>cat (as in "cat /dev/sda > /dev/sdb"). Is there any advantage in using
>>dd?
>
>
> Block size. You can specify a block size to dd.
Okay.
> reads 4k at a time, and that's not optimal when you're copying a large
> chunk of data.
Is it? Some "benchmarks". Those are all filesystems on a LVM.
The fs were NOT mounted (neither nor anytime at all after boot).
So the blocks should not be cached.
The filesystems are of the same size and should not be
"interleaved".
server read-test # time cat /dev/system_vg/ext2 > cat ; sync ; time dd if=/dev/system_vg/ext3 of=dd-32k bs=32k
real 1m55.575s
user 0m0.136s
sys 0m9.025s
real 2m21.200s
user 0m0.087s
sys 0m8.795s
I did more tests with other filesystems of the same size
and always get about the same times - dd with bs=32k is
slower than cat.
Am I doing something wrong?
> And if you didn't know, dd is a *terrible* way to back up and/or mirror
> something. It copies everything, even unused space, so it's slower than
> tar or cp -a
Uhm, how do you make an *IMAGE* with tar or cp? I use those only
for making backups, but not for IMAGES. The good thing about dd/cat is,
that it reads the whole partition/harddisk - INCLUDING the
free space. This way, you get an EXACT copy. Especially useful, if
you make an image of the whole hard drive, because you then
also have a copy of the MBR and partitioning. Thus, you'll only
need to do a "dd if=backup-file of=/dev/hda" to get a fully
working systme.
With tar/cp, you'll also need to have some documentation about
how the system was partitioned and how to install the boot loader.
> partimage
Yes, since partimage (*VERY* roughly) does a dd of the hard drive
and thus cannot be compared at all to a backup of the files on
the filesystem, which is what tar and cp do.
Alexander Skwar
--
Do Miami a favor. When you leave, take someone with you.
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
John Reiser
> something. It copies everything, even unused space, so it's slower than
> tar or cp -a or partimage if your filesystem isn't 100% full. If the
> second disk has a different size or a different geometry, the partition
> table you copied from the first disk will be incorrect. The only
> advantage dd has over other backup/mirror methods is that you don't have
> to reinstall GRUB or LILO.
It depends on the number of files, their total size, the distribution of
their sizes, and the existing fragmentation. I have seen ext2/ext3 file-
systems that are only 85% full but which have over 40% non-contiguous files
(used for software development), and dd is faster than tar, cp, or partimage
because of fewer seeks: one seek per cylinder for dd, zillions of seeks for
anything that deals with all the files. If the filesystem is nearly full,
then dd is attractive for shortest clock time. On the other hand, a file-
by-file backup-then-restore gives "free" defragmentation if you endure
the seeks during backup.
--
John Reiser
> problems with cat, that were not present with dd?
Yes: 'cat' frequently takes too long in contrast to 'dd' with
an appropriate blocksize.
--
Robert Heller
In a message on Thu, 02 Sep 2004 23:13:30 +0200, wrote :
AS> Dances With Crows wrote:
AS> > On Thu, 02 Sep 2004 20:20:24 +0200, Alexander Skwar staggered into the
AS> > Black Sun and said:
AS> >
AS> >>spi...@freenet.co.uk wrote:
AS> >>
AS> >>>If the two disks have identical sizes and geometries
AS> >>>(sectors/tracks/cylinders), then yes. dd if=/dev/sda of=/dev/sdb
AS> >>>bs=512
AS> >>
AS> >>I'm curious: Why's everyone using dd for this and not the IMO simpler
AS> >>cat (as in "cat /dev/sda > /dev/sdb"). Is there any advantage in using
AS> >>dd?
AS> >
AS> >
AS> > Block size. You can specify a block size to dd.
AS>
AS> Okay.
AS>
AS> > reads 4k at a time, and that's not optimal when you're copying a large
AS> > chunk of data.
AS>
AS> Is it? Some "benchmarks". Those are all filesystems on a LVM.
AS> The fs were NOT mounted (neither nor anytime at all after boot).
AS> So the blocks should not be cached.
AS>
AS> The filesystems are of the same size and should not be
AS> "interleaved".
AS>
AS> server read-test # time cat /dev/system_vg/ext2 > cat ; sync ; time dd if=/dev/system_vg/ext3 of=dd-32k bs=32k
AS>
AS> real 1m55.575s
AS> user 0m0.136s
AS> sys 0m9.025s
AS>
AS> real 2m21.200s
AS> user 0m0.087s
AS> sys 0m8.795s
AS>
AS> I did more tests with other filesystems of the same size
AS> and always get about the same times - dd with bs=32k is
AS> slower than cat.
AS>
AS> Am I doing something wrong?
AS>
AS> > And if you didn't know, dd is a *terrible* way to back up and/or mirror
AS> > something. It copies everything, even unused space, so it's slower than
AS> > tar or cp -a
AS>
AS> Uhm, how do you make an *IMAGE* with tar or cp? I use those only
AS> for making backups, but not for IMAGES. The good thing about dd/cat is,
AS> that it reads the whole partition/harddisk - INCLUDING the
AS> free space. This way, you get an EXACT copy. Especially useful, if
AS> you make an image of the whole hard drive, because you then
AS> also have a copy of the MBR and partitioning. Thus, you'll only
AS> need to do a "dd if=backup-file of=/dev/hda" to get a fully
AS> working systme.
Image backups can be seriously bad news, since they are not 'portable'.
If I make a tar backup of my Linux box (of some *random* version of the
O/S with some random version of ext2 or ext3 or whatever file system), I
can view/extract/whatever elements of that tar file on some other
system, including not only *any* other UNIX flavor, but ALSO MS-Windows
or MacOS (classic OR OSX) -- WinZIP understands tar files, so does
StuffitExpander. If I have an image backup, I can only make use of it
by getting and *exactly* twin of the original system (same O/S, same fs
version, and importantly, the *same* disk).
AS>
AS> With tar/cp, you'll also need to have some documentation about
AS> how the system was partitioned and how to install the boot loader.
AS>
/sbin/fdisk -l | lpr -
lpr /etc/lilo.conf
lpr /etc/grub.conf (guessing -- I don't use grub myself).
AS> > partimage
AS>
AS> Yes, since partimage (*VERY* roughly) does a dd of the hard drive
AS> and thus cannot be compared at all to a backup of the files on
AS> the filesystem, which is what tar and cp do.
AS>
AS> Alexander Skwar
AS> --
AS> Do Miami a favor. When you leave, take someone with you.
AS> ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
AS>
Nick Landsberg
> Alexander Skwar <fr...@alexander.skwar.name>,
> In a message on Thu, 02 Sep 2004 22:09:55 +0200, wrote :
>
> AS> Robert Heller wrote:
> AS>
> AS> > Dd will work properly at the device level. Cat is really only meant for
> AS> > concatenating normal files.
> AS>
> AS> Okay, maybe I don't have enough experience, but did you ever have
> AS> problems with cat, that were not present with dd?
>
> Cat lacks some useful options:
>
> dd if=boot.img of=/dev/fd0 bs=9216
>
> Copies a floppy disk image and does the floppy I/O in track sized
> buffers. This is a bit faster than cat boot.img >/dev/fd0, which does
> it byte-by-byte (yes, there is buffering inside the glibc code, but not
> specifically floppy track sized). Also dd's parameter methodology is
> more convenient than cat when used with sudo. Compare:
>
> sudo dd if=boot.img of=/dev/fd0
>
> vs.
>
> sudo sh -c 'cat boot.img >/dev/fd0'
>
> Cat is quite useful in many situations (I use it all of the time), but
> dd is a more powerful command and has many advantages in *other*
> situations. Compare using a hammer to 'pound in' a screw. This can be
> done, but a screwdriver works better (to drive a screw).
Hmmm... actually, all the options to dd make it look more
like a Swiss Army Knife :)
Where I found dd the most useful, in the dim dark past
where computers exchanged data mostly via tapes,
was its capability to convert formats.
Consider reading an EBCDIC 9-track tape created on an IBM
mainframe:
dd if=/dev/rmt0n of=tape_header conv=ascii,lcase cbs=80 count=1
# the obligatory tape header read in through the
# "no rewind" minor-device of the driver
dd if=/dev/rmt0n of=/dev/null ibs=128 skip=10000
# skip first 10000 records of 128 bytes each, no rewind
dd if=/dev/rmt0 of=tape_file conv=ascii,lcase cbs=128
(Obviously, all of the above was in a script, because
I wasn't about to type that at the console every time.
If I had been able to "cat" those steenking tapes, I would
have :)
Nowadays, the need for this has almost disappeared,
I'd guess, but dd is still around.
It all depends on the problem you're trying to solve.
If your problem is strictly copying one physical
device to another identical physical device, then,
by all means, use "cat", since it doesn't carry all
that baggage in it's code of converting and variable
block sizes and such.
If you're going to back up to a device which is not
identical in all respects, tarballs and/or cpio
(whichever you feel most comfortable with) are
probably more appropriate.
If you have something out of the ordinary, like
creating a floppy with some odd-ball block size,
then the "swiss army knife" is probably the way to
go.
Use the right tool for the right job.
NPL
>
> AS>
> AS> I haven't.
> AS>
> AS>
> AS> Alexander Skwar
> AS> --
> AS> I must Create a System, or be enslav'd by another Man's;
> AS> I will not Reason and Compare; my business is to Create.
> AS> -- William Blake, "Jerusalem"
> AS> ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
> AS>
>
> \/
> Robert Heller ||InterNet: hel...@cs.umass.edu
> http://vis-www.cs.umass.edu/~heller || hel...@deepsoft.com
> http://www.deepsoft.com /\FidoNet: 1:321/153
>
>
>
>
>
>
>
--
"It is impossible to make anything foolproof
because fools are so ingenious"
- A. Bloch
Alexander Skwar
Well... Seems not.
Alexander Skwar
--
The right half of the brain controls the left half of the body. This
means that only left handed people are in their right mind.
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
Alexander Skwar
> Image backups can be seriously bad news, since they are not 'portable'.
Uhm? They are very much so:
mount -o loop -t <whatever> /backup/image /mnt
Alexander Skwar
--
It seems that more and more mathematicians are using a new, high level
language named "research student".
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
Alexander Skwar
> Hmmm... actually, all the options to dd make it look more
> like a Swiss Army Knife :)
Yes, that's right. For reading "strange" tapes or something
like this, all the options dd has, will be very handy.
However, we weren't talking about somethings like this, but
about making a mere image of a partition.
> If you're going to back up to a device which is not
> identical in all respects, tarballs and/or cpio
> (whichever you feel most comfortable with) are
> probably more appropriate.
Yes.
> If you have something out of the ordinary, like
> creating a floppy with some odd-ball block size,
> then the "swiss army knife" is probably the way to
> go.
You're right.
Robert Heller
In a message on Fri, 03 Sep 2004 06:33:53 +0200, wrote :
AS> Robert Heller wrote:
AS>
AS> > Image backups can be seriously bad news, since they are not 'portable'.
AS>
AS> Uhm? They are very much so:
AS>
AS> mount -o loop -t <whatever> /backup/image /mnt
You can't do this under MS-Windows or MacOS (Classic or OSX) or Solaris.
I wrote a version of tar for my old CP/M-68K box and there was a version
of tar for the Exploror Lisp Machines. A tar backup can be read on all
manner of systems now AND IN THE FUTURE.
AS>
AS> Alexander Skwar
AS> --
AS> It seems that more and more mathematicians are using a new, high level
AS> language named "research student".
Alexander W. Skwar
> Alexander Skwar <fr...@alexander.skwar.name>,
> AS> mount -o loop -t <whatever> /backup/image /mnt
>
> You can't do this under MS-Windows or MacOS (Classic or OSX) or Solaris.
Yes, that's somewhat correct. ext2 can be read in Windows, IIRC.
Alexander Skwar
--
panic("aha1740.c"); /* Goodbye */
2.2.16 /usr/src/linux/drivers/scsi/aha1740.c
ŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻ
Dances With Crows
the Black Sun and said:
>> You can't do this under MS-Windows or MacOS (Classic or OSX) or
>> Solaris.
> Yes, that's somewhat correct. ext2 can be read in Windows, IIRC.
I believe what Robert was referring to was "mount -o loop", not "mount
-t ext2". Windows does not provide a good method for mounting a file
and treating it as if it were a filesystem. OS X has "disk images", but
those aren't filesystems, they're filesystems with a wrapper of some
sort around them.
Also, you previously wrote:
>partimage (*VERY* roughly) does a dd of the hard drive and thus cannot
>be compared at all to a backup of the files on the filesystem,
This is incorrect. partimage understands enough about ext2, ReiserFS,
FAT, and even NTFS to look in the superblock and determine which blocks
are in use and which blocks are not. partimage will only back up blocks
that are in use.
Alexander Skwar
> On Fri, 03 Sep 2004 14:46:04 +0200, Alexander W. Skwar staggered into
> the Black Sun and said:
>
>>Robert Heller wrote:
>>
>>> Alexander Skwar <fr...@alexander.skwar.name>,
>>>
>>>>mount -o loop -t <whatever> /backup/image /mnt
>>>
>>>You can't do this under MS-Windows or MacOS (Classic or OSX) or
>>>Solaris.
>>
>>Yes, that's somewhat correct. ext2 can be read in Windows, IIRC.
>
>
> I believe what Robert was referring to was "mount -o loop", not "mount
> -t ext2".
Yes, I think so too. However, I *think* that there's a way to look
into ext2 files. Don't know it for sure.
> Windows does not provide a good method for mounting a file
> and treating it as if it were a filesystem.
That's absolutely right.
>>partimage (*VERY* roughly) does a dd of the hard drive and thus cannot
>>be compared at all to a backup of the files on the filesystem,
>
>
> This is incorrect.
Well, when compared with tar/cpio/cp and dd/cat, partimage is much
more in the dd/cat field, as it makes an image of the partition and
not a backup of the files.
> are in use and which blocks are not. partimage will only back up blocks
> that are in use.
Yep.
Alexander Skwar
--
This login session: $13.99
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
Alexander Skwar
> Yes, I think so too. However, I *think* that there's a way to look
> into ext2 files.
There is, with the tool "Explore2fs". But nonetheless I totally
agree that Linux' loop-mount is still WAY more flexible.
Alexander Skwar
--
"We are on the verge: Today our program proved Fermat's next-to-last theorem."
-- Epigrams in Programming, ACM SIGPLAN Sept. 1982
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
Chris F.A. Johnson
> Alexander Skwar <fr...@alexander.skwar.name>,
> In a message on Fri, 03 Sep 2004 06:33:53 +0200, wrote :
>
> AS> Robert Heller wrote:
> AS>
> AS> > Image backups can be seriously bad news, since they are not 'portable'.
> AS>
> AS> Uhm? They are very much so:
> AS>
> AS> mount -o loop -t <whatever> /backup/image /mnt
>
> You can't do this under MS-Windows or MacOS (Classic or OSX) or Solaris.
>
> I wrote a version of tar for my old CP/M-68K box and there was a version
> of tar for the Exploror Lisp Machines. A tar backup can be read on all
> manner of systems now AND IN THE FUTURE.
Unfortuntely, there are incompatibilities among the various
versions of tar.
--
Chris F.A. Johnson http://cfaj.freeshell.org/shell
===================================================================
My code (if any) in this post is copyright 2004, Chris F.A. Johnson
and may be copied under the terms of the GNU General Public License
Alexander Skwar
> Unfortuntely, there are incompatibilities among the various
> versions of tar.
Yes. However, tar is standardized (POSIX). So, if you don't use
GNU tar, you can be very certain that every compliant tar can
handle the tar archive.
Alexander Skwar
--
Hey, if pi == 3, and three == 0, does that make pi == 0? :-)
-- Larry Wall in <1997110119...@wall.org>
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
Jose Maria Lopez Hernandez
> Robert Heller wrote:
>
>> Alexander Skwar <fr...@alexander.skwar.name>,
>
>
>>AS> mount -o loop -t <whatever> /backup/image /mnt
>>
>>You can't do this under MS-Windows or MacOS (Classic or OSX) or Solaris.
>
>
> Yes, that's somewhat correct. ext2 can be read in Windows, IIRC.
>
> Alexander Skwar
It can be done (both things commented) with some tools, but at least
for mounting an image you need a commercial tool like nero, and to
read a ext2 image from windows you have to needed badly, because it's
deadly slow and you can't have write support.
--
Jose Maria Lopez Hernandez
Director Tecnico de bgSEC
jker...@bgsec.com
bgSEC Seguridad y Consultoria de Sistemas Informaticos
http://www.bgsec.com
ESPAÑA
The only people for me are the mad ones -- the ones who are mad to live,
mad to talk, mad to be saved, desirous of everything at the same time,
the ones who never yawn or say a commonplace thing, but burn, burn, burn
like fabulous yellow Roman candles.
-- Jack Kerouac, "On the Road"
Robert Heller
In a message on Sat, 04 Sep 2004 00:21:36 GMT, wrote :
H> In article <6fa2e$4137b771$cb248f0$25...@nf1.news-service-com>,
H> Robert Heller <hel...@deepsoft.com> wrote:
H> >
H> > Image backups can be seriously bad news, since they are not 'portable'.
H> > If I make a tar backup of my Linux box (of some *random* version of the
H> > O/S with some random version of ext2 or ext3 or whatever file system), I
H> > can view/extract/whatever elements of that tar file on some other
H> > system, including not only *any* other UNIX flavor, but ALSO MS-Windows
H> > or MacOS (classic OR OSX) -- WinZIP understands tar files, so does
H> > StuffitExpander. If I have an image backup, I can only make use of it
H> > by getting and *exactly* twin of the original system (same O/S, same fs
H> > version, and importantly, the *same* disk).
H>
H> No:
H>
H> mount -o loop,offset=whatever /path/to/hda-backup /mnt/temp
H>
H> The offset may be difficult to figure out, but once you know it for one
H> partition, given the partition table it's easy to figure out for the rest.
H> You can also extract the filesystems with dd.
H>
H> You're right that you can't do this in Windows or MacOS. That may or may
H> not be a factor.
Murrfy's Law: your Linux box crashes the day before there is some
random little text file you absolutely need. Fortunately, you have a
tar backup on a FAT16 formatted Zip cart and your signifiant other has
this junkly little MS-Windows box with a USB Zip drive -- you would
never normally use the silly box -- so you pop in your backup Zip Cart,
fire up WinZip, click you way past the silly 'Register Now?' dialog box
and extract the file you need. Fire up WordPad (which happens to gronk
UNIX-style newlines), select Print... from the File menu, grab your
hard copy and there you are. MS-Windows *comes* with WinZip installed
these days.
ALL of the compressed tar files I've ever created have worked fine on
MS-Windows boxes using WinZip. Ditto for StuffitExpander under MacOS.
I no longer bother to create .ZIP or .sea files of my code anymore (even
though there is a version of Stuffit for Linux and all current Linux
distros come with InfoZip.
H>
H> --
H> -eben ebQ...@EtaRmpTabYayU.rIr.OcoPm home.tampabay.rr.com/hactar
H> LIBRA: A big promotion is just around the corner for someone
H> much more talented than you. Laughter is the very best medicine,
H> remember that when your appendix bursts next week. -- Weird Al
H>
Alexander Skwar
> Murrfy's Law: your Linux box crashes the day before there is some
> random little text file you absolutely need.
Knoppix can be booted almost everywhere.
And if you apply Murphys Law to your tar file, the ZIP drive's broken,
or the tar file's defective.
> MS-Windows *comes* with WinZip installed
> these days.
Wrong.
Alexander Skwar
--
A mushroom cloud has no silver lining.
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯