06 July 2019

Using dd instead of disk and partition cloners like CloneZilla or PartImage

The problem with applications like CloneZilla is that although they are excellent for disk imaging, they have very un-intuitive interfaces, the number of steps they make you go through is huge and they suffer from problems with UEFI when creating a USB version.

Linux's dd command on the other hand, is projected as being very dangerous and people say it's better to use something like CloneZilla just to be safe. But from what I've seen, as long as you type the command properly, dd is actually far more convenient than the other applications. If you want a safer GUI version, there's GDiskDump and KinDD.

Here are some useful commands:

To compress and clone entire sda onto a folder on sdb or sdc:

*** Caution: Verify all the commands below before executing them ***
*** Caution: The folder paths and partitions will be different for you *** 

When you plugin the pen drive or external hard disk, it'll get mounted as sdb or sdc. Find out where it got mounted. Mine was in /media/ubuntu/Nav/ and it was in sdc, since sdb was the Ubuntu startup USB. Created a folder named "bkp" in Nav and:
sudo dd bs=1M if=/dev/sda | gzip > /media/ubuntu/Nav/bkp/backup.img.gz
Don't worry. Even though sdc and sdb are mounted on /media, which is the root folder (and hence you'd think it's part of sda), Linux knows that these are mounted filesystems, so it won't consider it part of sda.
Instead of using bs=1M, you could speed it up by checking what your disk buffer size is using sudo hdparm -i /dev/sda, and set that as the bs value. So if the buffer size is 8MB, set bs=8M.

If the disk was not automatically mounted, you can mount it yourself using:
cd mnt/
sudo mkdir bkp
mount /dev/sdc1 /mnt/bkp

Now take a backup
sudo dd bs=8M if=/dev/sda | gzip > /mnt/bkp/backup.img.gz

Restoring sda is done via:
sudo gunzip -kv /mnt/bkp/backup.img.gz | dd of=/dev/sda
 
Taking a backup of sda means that even unallocated space gets backed up, and that can take very long (3.6 hours for a 500GB HDD). If you don't need a full disk backup and can manage with just the few numbered sda partitions, then just do a backup and restore of sda1, sda2 etc. To do that you'll have to also backup and restore a copy of the partition table as shown below.

You can take a copy of the partition table info using:
sudo sfdisk -d /dev/sda > my_partitions

Now create backups (these commands are a bit risky because you have to take care to not mess up the "if" and "of". Safer commands are shown below):
sudo dd if=/dev/sda1 of=/media/ubuntu/Nav/bkp/sda1.img bs=8M
sudo dd if=/dev/sda2 of=/media/ubuntu/Nav/bkp/sda2.img bs=8M

Restoring  the partition table info is done via:
sudo sfdisk --force /dev/sda < my_partitions

Restoring backups:
sudo dd if=/media/ubuntu/Nav/bkp/sda1.img of=/dev/sda1 bs=8M
sudo dd if=/media/ubuntu/Nav/bkp/sda2.img of=/dev/sda2 bs=8M

In Ubuntu, you can make use of a default sound file to tell you when the operation completes:
alias beep='paplay /usr/share/sounds/freedesktop/stereo/complete.oga'
 

Okay, so the safer commands are:
sudo dd bs=8M if=/dev/sda1 | gzip > /mnt/bkp/sda1.img.gz; beep
sudo dd bs=8M if=/dev/sda2 | gzip > /mnt/bkp/sda2.img.gz; beep

Restoring sda is done via:
gunzip -c -k /mnt/bkp/sda1.img.gz | sudo dd of=/dev/sda1; beep
gunzip -c -k /mnt/bkp/sda2.img.gz | sudo dd of=/dev/sda2; beep

The "-k" helps avoid deleting the original gz file after it is extracted. The "-c" performs the extraction in memory and writes to the output disk, instead of temporarily extracting to the input disk. The good part about these safer commands is that they also compress the backup, which saves you a good amount of space. I recommend taking backups of each partition separately instead of taking a full disk backup or directly backing up the entire sda, because having separate partition backups helps restore them separately too.


To wipe the first 1MB of any disk:

sudo dd if=/dev/zero of=/dev/sdb bs=1M count=1

Here, the "b" highlighted in bold red should be changed to "a" or "c" etc., based on which disk's first 1MB you want to wipe. The partition information will disappear, so this command is useful when trying to re-purpose any disk. Once the command is run, just use GParted to create the partitions.


To write zeros onto the entire disk:

sudo dd if=/dev/zero of=/dev/sda bs=4096

This is helpful when you want to experiment with whether dd works well for backups. I used this command to fill the disk with zeros, then installed Windows and tried backing up. The 500GB disk with around 20GB of data on the Windows partition got gzip backed up into 12GB. The high compression was because the rest of the disk was full of zeros.
 

To view the progress of dd:
 
pv is the short form of "pipe viewer". It's a handy little program that helps monitor the progress of data through a pipe. When used with dd, it'll show you the progress of the operation in terms of the amount of time remaining for completion and the progress in terms of the number of MB or GB.

sudo apt-get install -y pv
sudo dd if=/dev/sdb | pv | dd of=/thePath/theFilename


No comments: