For yucks, and for backup, I wrote a simple script to clone my harddisk. The disk has several partitions, all ext2 (except for swap). A slightly simplified version of the script follows: (live disk is /dev/hda and backup is /dev/hdb - hdb has partition structure identical to hda's - fstab mounts hda2 on /usr)
#---- start of script ---- mount /dev/hdb1 /mnt rsync -a --exclude "proc/" --exclude "usr/" --exclude "mnt/" /* /mnt mkdir /mnt/proc mkdir /mnt/usr mkdir /mnt/mnt umount /mnt
mount /dev/hdb2 /mnt rsync -a /usr/* /mnt umount /mnt
#...other partitions backed up here
dd if=/dev/hda of=/dev/hdb bs=512 count=1 #---- end of script ----
Well, much to my astonishment, this basically works - X, network, samba, VMware, the works. Two questions, though...
1. I thought that the "dd" would copy the mbr such that the clone (hdb) could be booted (rejumpered as hda, of course). It apparently did not, since I got "LI" and had to boot from a floppy and run lilo - then, I could boot directly from the clone hard disk. IS THERE SOMETHING I CAN PUT IN THE SCRIPT TO BUILD A BOOTABLE MBR?
2. Could my technique have caused some lurking problem that I may not see for a while? I can see no obviously suspect boot messages and, as I mentioned, everything seems ok. ANY OTHER "HEALTH CHECKS" THAT I CAN DO?
> For yucks, and for backup, I wrote a simple script to clone my harddisk. > The disk has several partitions, all ext2 (except for swap). A slightly > simplified version of the script follows: (live disk is /dev/hda and backup > is /dev/hdb - hdb has partition structure identical to hda's - fstab mounts > hda2 on /usr)
> mount /dev/hdb2 /mnt > rsync -a /usr/* /mnt > umount /mnt
> #...other partitions backed up here
> dd if=/dev/hda of=/dev/hdb bs=512 count=1 > #---- end of script ----
> Well, much to my astonishment, this basically works - X, network, samba, > VMware, the works. Two questions, though...
> 1. I thought that the "dd" would copy the mbr such that the clone (hdb) > could be booted (rejumpered as hda, of course). It apparently did not, > since I got "LI" and had to boot from a floppy and run lilo - then, I could > boot directly from the clone hard disk. IS THERE SOMETHING I CAN PUT IN THE > SCRIPT TO BUILD A BOOTABLE MBR?
> 2. Could my technique have caused some lurking problem that I may not see > for a while? I can see no obviously suspect boot messages and, as I > mentioned, everything seems ok. ANY OTHER "HEALTH CHECKS" THAT I CAN DO?
> Debian/potato, BTW - not that that should matter.
> -jeff
Well, a couple of comments. There was no point in making partitions and directories on hdb, because your dd command overwrote them. Also, bs=512 is way small and I would expect that to cause the copy to take a long time. If using dd on disks, I typically use bs=65536.
Are you using identical disks? If so, here is what I do: # cp /dev/hda /dev/hdb Does the same thing as dd. cp understands that I am referencing raw disk devices rather than file systems, and behaves correctly. I did not partition the disk prior to the copy. This copied my MBR, extended partitions, Linux partitions, swap partition, and Windoze partitions, and the resulting disk boots and runs just like the original one. As I recall, it took me 35 minutes to copy a 20G disk.
The fact that your copy did not initially boot makes me wonder whether they really were identical, because if they were, the dd command should have correctly copied the MBR, and the disk should have booted.
-- My real email is akamail.com@dclark (or something like that).
On Wed, 23 Aug 2000 16:21:38 -0700, Duane <junkm...@junkmail.com> wrote: >jeff wrote:
>> For yucks, and for backup, I wrote a simple script to clone my harddisk. >> The disk has several partitions, all ext2 (except for swap). A slightly >> simplified version of the script follows: (live disk is /dev/hda and backup >> is /dev/hdb - hdb has partition structure identical to hda's - fstab mounts >> hda2 on /usr)
>> mount /dev/hdb2 /mnt >> rsync -a /usr/* /mnt >> umount /mnt
>> #...other partitions backed up here
>> dd if=/dev/hda of=/dev/hdb bs=512 count=1 >> #---- end of script ----
>> Well, much to my astonishment, this basically works - X, network, samba, >> VMware, the works. Two questions, though...
>> 1. I thought that the "dd" would copy the mbr such that the clone (hdb) >> could be booted (rejumpered as hda, of course). It apparently did not, >> since I got "LI" and had to boot from a floppy and run lilo - then, I could >> boot directly from the clone hard disk. IS THERE SOMETHING I CAN PUT IN THE >> SCRIPT TO BUILD A BOOTABLE MBR?
>> 2. Could my technique have caused some lurking problem that I may not see >> for a while? I can see no obviously suspect boot messages and, as I >> mentioned, everything seems ok. ANY OTHER "HEALTH CHECKS" THAT I CAN DO?
>> Debian/potato, BTW - not that that should matter.
>> -jeff
>Well, a couple of comments. There was no point in making partitions and >directories on hdb, because your dd command overwrote them. Also, bs=512 >is way small and I would expect that to cause the copy to take a long >time. If using dd on disks, I typically use bs=65536.
Hmmm... the dd has a count=1 parm. My intent was to copy ONLY the MBR with the dd.
>Are you using identical disks? If so, here is what I do: ># cp /dev/hda /dev/hdb
I'll may give this a shot, but rsync has a major advantage. It can detect and copy only changes in the group of files being processed, so - after the initial copy - I can run a daily quick cron job to keep the clone in sync.
>Does the same thing as dd. cp understands that I am referencing raw disk >devices rather than file systems, and behaves correctly. I did not >partition the disk prior to the copy. This copied my MBR, extended >partitions, Linux partitions, swap partition, and Windoze partitions, >and the resulting disk boots and runs just like the original one. As I >recall, it took me 35 minutes to copy a 20G disk.
>The fact that your copy did not initially boot makes me wonder whether >they really were identical, because if they were, the dd command should >have correctly copied the MBR, and the disk should have booted.
>-- >My real email is akamail.com@dclark (or something like that).
> Well, a couple of comments. There was no point in making partitions and > directories on hdb, because your dd command overwrote them. Also, bs=512 > is way small and I would expect that to cause the copy to take a long > time. If using dd on disks, I typically use bs=65536.
> Are you using identical disks? If so, here is what I do: > # cp /dev/hda /dev/hdb > Does the same thing as dd. cp understands that I am referencing raw disk > devices rather than file systems, and behaves correctly. I did not > partition the disk prior to the copy. This copied my MBR, extended > partitions, Linux partitions, swap partition, and Windoze partitions, > and the resulting disk boots and runs just like the original one. As I > recall, it took me 35 minutes to copy a 20G disk.
> The fact that your copy did not initially boot makes me wonder whether > they really were identical, because if they were, the dd command should > have correctly copied the MBR, and the disk should have booted.
> -- > My real email is akamail.com@dclark (or something like that).
I agree that things should have been identical. I'm looking into doing that same thing at work for a FTP server. I set aside a disk so that I can clone the root drive and have it bootable in the event of a failure. (I thought this would be easier than building root-raid.)
I was just planning on doing
dd if=/dev/sda of=/dev/sdb bs=65536
and hoping things worked. Didn't realize that 'cp' did this. But, dd should do the same thing, right?
Well, I put 2 and 2 together, and finally got 4! I think that my problem was this... An "LI," according to the lilo doc, could indicate that /boot/boot.d has moved. Since this is almost certainly true in the case of my rsync'd cloned disk, it explains the message, and the inability to boot.
How to fix this? One thought is that I could create a small /boot partition and use dd to copy it. The big advantage of using rsync over dd and cp, for copying the bulk of the system, is that rsync can copy only changes, so keeping the clone in sync can be accomplished with a quickie nightly cron job.
On 24 Aug 2000 00:57:05 GMT, jeff <j...@myhost.mydomain> wrote:
>>> For yucks, and for backup, I wrote a simple script to clone my harddisk. >>> The disk has several partitions, all ext2 (except for swap). A slightly >>> simplified version of the script follows: (live disk is /dev/hda and backup >>> is /dev/hdb - hdb has partition structure identical to hda's - fstab mounts >>> hda2 on /usr)
>>> mount /dev/hdb2 /mnt >>> rsync -a /usr/* /mnt >>> umount /mnt
>>> #...other partitions backed up here
>>> dd if=/dev/hda of=/dev/hdb bs=512 count=1 >>> #---- end of script ----
>>> Well, much to my astonishment, this basically works - X, network, samba, >>> VMware, the works. Two questions, though...
>>> 1. I thought that the "dd" would copy the mbr such that the clone (hdb) >>> could be booted (rejumpered as hda, of course). It apparently did not, >>> since I got "LI" and had to boot from a floppy and run lilo - then, I could >>> boot directly from the clone hard disk. IS THERE SOMETHING I CAN PUT IN THE >>> SCRIPT TO BUILD A BOOTABLE MBR?
>>> 2. Could my technique have caused some lurking problem that I may not see >>> for a while? I can see no obviously suspect boot messages and, as I >>> mentioned, everything seems ok. ANY OTHER "HEALTH CHECKS" THAT I CAN DO?
>>> Debian/potato, BTW - not that that should matter.
>>> -jeff
>>Well, a couple of comments. There was no point in making partitions and >>directories on hdb, because your dd command overwrote them. Also, bs=512 >>is way small and I would expect that to cause the copy to take a long >>time. If using dd on disks, I typically use bs=65536.
>Hmmm... the dd has a count=1 parm. My intent was to copy ONLY the MBR with >the dd.
>>Are you using identical disks? If so, here is what I do: >># cp /dev/hda /dev/hdb
>I'll may give this a shot, but rsync has a major advantage. It can detect >and copy only changes in the group of files being processed, so - after the >initial copy - I can run a daily quick cron job to keep the clone in sync.
>>Does the same thing as dd. cp understands that I am referencing raw disk >>devices rather than file systems, and behaves correctly. I did not >>partition the disk prior to the copy. This copied my MBR, extended >>partitions, Linux partitions, swap partition, and Windoze partitions, >>and the resulting disk boots and runs just like the original one. As I >>recall, it took me 35 minutes to copy a 20G disk.
>>The fact that your copy did not initially boot makes me wonder whether >>they really were identical, because if they were, the dd command should >>have correctly copied the MBR, and the disk should have booted.
>>-- >>My real email is akamail.com@dclark (or something like that).
> Well, I put 2 and 2 together, and finally got 4! I think that my problem > was this... An "LI," according to the lilo doc, could indicate that > /boot/boot.d has moved. Since this is almost certainly true in the case of > my rsync'd cloned disk, it explains the message, and the inability to boot.
> How to fix this? One thought is that I could create a small /boot partition > and use dd to copy it. The big advantage of using rsync over dd and cp, for > copying the bulk of the system, is that rsync can copy only changes, so > keeping the clone in sync can be accomplished with a quickie nightly cron > job.
I'm sorry, Jeff - NO, you're still getting 5 by adding 2 and 2... You still mix up the copying of files from one filesystem to another ( as is e.g. cp's, rsync's or tar's behaviour ), with dd's way of operation - that is, transferring each and every bit from one place to another without care of any overlaying structures like partitions, filesystems and such !!! Let me repeat that - "dd" is a typical UNIX tool - simple, but effective and very mighty not DESPITE, but JUST BECAUSE of its simplicity ! dd basically does no more than to read a stream of bytes - the input"file", "if=..." - and to write that stream back to another location - outputfile, "of=...". What is the clue to the enormous flexibility, is any UNIX's concept of "device files", saying (in short) that you can access devices like floppy or hard disks, CD-ROMs, whatever, by using a special file such as /dev/fd0, dev/hda, dev/hdb and so on. So, one of the mistakes you wrote in one of your postings was the notion that you only talk about the MBR when using /dev/hda - in fact, you talk about the WHOLE DISK instead... Thus, if you think again, the command 'dd if=/dev/hda of=/dev/hdb' does the job you want - i.e. transferring EVERY sector from HD 1 to EVERY sector on HD 2 !! BUT, maybe now you see this can only work if both disks are IDENTICAL in size, i.e. share the same number of sectors - they don't even need to be by the same manufacturer, or one can be IDE and the other SCSI, as long as they don't differ even by one single sector ! Along with that sector-by-sector transfer, you do transfer the MBR including the partition table, starting and ending of each partition, and every file on each partition's file system. So, according to the subject, this would be a perfect clone : if the 2 disks are 2 items of the same model, every operating system and every human could *only* discern between both by the drive's serial number - which really is irrelevant otherwise... So, if you can't boot from the clone, you either didn't issue the above command correctly (what to me seems hard to do...), or you don't meet the OBLIGATORY condition for this - your disks aren't identical in size !!! Maybe now you also see that your 'rsync' magic previous to the 'dd' is totally useless - as you overwrite the copied files afterwards again! Think about that, please, Jeff, and read dd's manual page thoroughly, for instance how to copy only one or some sectors/bytes, and please do test some examples, for instance with floppies, like "dd if=/dev/fd0 of=/tmp/floppy-image". You can then try to mount the *disk file* /tmp/floppy-image as "loop device" and compare it's contents including file system to the floppy disk itself; or, simply clone your floppy by exchanging "if" and "of" in the above command - then we're ready for the next lesson...
Thanks for the post, Juergen - but what we're having here is, I believe, a failure to communicate. I _AM_ using dd to copy _ONLY_ THE MBR. dd is NOT copying anyhing in hda1 or any other partition. The COUNT=1 AND BS=512 AND IF=HDA on the dd specify that only the _FIRST_ 512-BYTE BLOCK ON THE PHYSICAL DISK is to be copied. The rsyncs are copying the data within the partitions, the dd is copying ONLY the mbr (count=1 bs=512).
AM I WRONG ABOUT ANY OF THAT?
It all makes perfect sense AND explains my LI message problem, as I posted previously. 2 + 2 still seems to = 4! But if I'm still being dense and missing your point, please try again :)
-jeff
(BTW - to repeat a pervious point - rsync is a much better long term for my purpose than dd because - once the clone is created and synced the first time - I can resync it every night it mere minutes - since rsync will copy only changes.) ----------
>I'm sorry, Jeff - NO, you're still getting 5 by adding 2 and 2... >You still mix up the copying of files from one filesystem to another >( as is e.g. cp's, rsync's or tar's behaviour ), with dd's way of >operation - that is, transferring each and every bit from one place >to another without care of any overlaying structures like partitions, >filesystems and such !!! >Let me repeat that - "dd" is a typical UNIX tool - simple, but effective >and very mighty not DESPITE, but JUST BECAUSE of its simplicity ! >dd basically does no more than to read a stream of bytes - the >input"file", >"if=..." - and to write that stream back to another location - >outputfile, >"of=...". What is the clue to the enormous flexibility, is any UNIX's >concept of "device files", saying (in short) that you can access devices >like floppy or hard disks, CD-ROMs, whatever, by using a special file >such as /dev/fd0, dev/hda, dev/hdb and so on. >So, one of the mistakes you wrote in one of your postings was the notion >that you only talk about the MBR when using /dev/hda - in fact, you talk >about the WHOLE DISK instead... >Thus, if you think again, the command 'dd if=/dev/hda of=/dev/hdb' does >the job you want - i.e. transferring EVERY sector from HD 1 to EVERY >sector on HD 2 !! BUT, maybe now you see this can only work if both >disks are IDENTICAL in size, i.e. share the same number of sectors - >they don't even need to be by the same manufacturer, or one can be IDE >and the other SCSI, as long as they don't differ even by one single >sector ! >Along with that sector-by-sector transfer, you do transfer the MBR >including the partition table, starting and ending of each partition, >and every file on each partition's file system. So, according to the >subject, this would be a perfect clone : if the 2 disks are 2 items of >the same model, every operating system and every human could *only* >discern between both by the drive's serial number - which really is >irrelevant otherwise... >So, if you can't boot from the clone, you either didn't issue the >above command correctly (what to me seems hard to do...), or you >don't meet the OBLIGATORY condition for this - your disks aren't >identical in size !!! >Maybe now you also see that your 'rsync' magic previous to the >'dd' is totally useless - as you overwrite the copied files >afterwards again! >Think about that, please, Jeff, and read dd's manual page thoroughly, >for instance how to copy only one or some sectors/bytes, and please >do test some examples, for instance with floppies, like >"dd if=/dev/fd0 of=/tmp/floppy-image". You can then try to mount >the *disk file* /tmp/floppy-image as "loop device" and compare it's >contents including file system to the floppy disk itself; or, simply >clone your floppy by exchanging "if" and "of" in the above command - >then we're ready for the next lesson...
> Thanks for the post, Juergen - but what we're having here is, I believe, a > failure to communicate. I _AM_ using dd to copy _ONLY_ THE MBR. dd is NOT > copying anyhing in hda1 or any other partition. The COUNT=1 AND BS=512 AND > IF=HDA on the dd specify that only the _FIRST_ 512-BYTE BLOCK ON THE > PHYSICAL DISK is to be copied. The rsyncs are copying the data within the > partitions, the dd is copying ONLY the mbr (count=1 bs=512).
> AM I WRONG ABOUT ANY OF THAT?
No, perhaps I read over that... Of course, the bs=512 count=1 parameters actually specify to transfer only the very first sector, generally known as MBR, between the disks - including the disk's *partition table* (which is in byte 446-509 of the MBR, counting from 0). To me, that seems not too wise, though : if the disks ARE identical in size, and your goal IS to make a clone disk, you had better copy ALL sectors in order to achieve this - and if your disks are NOT identical, perhaps partitioned differently, the transferred partition table would be *wrong* - i.e. not fitting to the actual starting and ending sectors. You are free to do it anyway you like, but firstly I would call that procedure only copying, not cloneing - regardless if any user will notice any difference or not, the difference IS there. To put it clear, by your rsync method you might have the same directory and file (system) structure on both HDs, but there's no guarantee that file A from disk 1 uses the same disk sectors as file A on disk B. That might be an academic issue, but the subject of this thread IS "cloneing", and your procedure is not an example for that. We're not talking about the time "rsync" vs "dd" takes, either - why would one want to compare timings of _not_ comaparative methods?
> It all makes perfect sense AND explains my LI message problem, as I posted > previously. 2 + 2 still seems to = 4! But if I'm still being dense and > missing your point, please try again :)
Well, what's not perfect with your method is that MBR transfer - in fact, it's totally senseless and useless !!! Please think about it, Jeff : if you insist in only rsync'ing your files between your disks, you'd have to admit that the target disk already has to be partitioned properly - and that is, REGARDLESS if both disk share the same sector count or not. Thus, there's absolutely NO point in overwriting your working partition table on the target with one potentially different and wrong. Furthermore, as the first 446 bytes of a LILO MBR contain machine code that (a. o.) jumps to the ABSOLUTE CHS location of some files in /boot, e.g. /boot/map, this code will *almost certain* fail because the absolute location of these files will *almost certain* differ between your two copies-not-clones. Thus, in order to boot from your copy-not-clone, you will HAVE TO reinstall LILO anyway - so why transfer a LILO machine code first you KNOW won't work ???? As a resume, you'd better omit that MBR transfer - keep it as a backup file on your HD, if some Micros~1 OS goes berserk...
> Well, a couple of comments. There was no point in making partitions and > directories on hdb, because your dd command overwrote them. Also, bs=512 > is way small and I would expect that to cause the copy to take a long > time. If using dd on disks, I typically use bs=65536.
> Are you using identical disks? If so, here is what I do: > # cp /dev/hda /dev/hdb
That is awesome! I've been trying to figure out a way to have a mirror copy of my system disk without using RAID. I've read numerous posts regarding backup techiques--tar, cpio, cp -a, etc. But these all seemed cumbersome, and none of them seemed to provide a bootable backup (I wasn't sure about dd).
Low and behold, little cp does the job! I tested it just to make sure, since I've often gotten incorrect information off NGs. It works just as you said. I now can run a simple cron job once a day and have a complete bootable backup just by switching a jumper on my HDD. How sweet it is.
-- "The worst form of inequality is to try to make unequal things equal."
