...making Linux just a little more fun!
By Alan Keates
|
...making Linux just a little more fun! |
By Alan Keates |
Until recently the extent of my backup efforts were to take the occasional CD copy of my home directory and keep copies of important files somewhere else, usually on another disk partition, or a floppy disk.
All this changed with the need to run some Windows legacy applications. The only machine really suitable for this work was my main workstation, a 1.2 GHz Athlon machine, multiboot with four distributions. I decided to free up the 1st primary partition, which held Mandrake 9.0, and set up a Windows partition.
I freed up the 1st primary partition by transferring the contents of that to the 7th partition, overwriting an expendable Vector Linux 3.0 Distribution. To be totally safe I booted into Debian 3.0, mounted both partitions to individual mount points in /mnt and as root used tar and a pipe to copy everything including all links and permissions from the source partition to the target partition. A few minutes later, after changing my grub boot menu, I was able to boot into Mandrake 9.0 Linux in the 7th partition and verify that everything worked as expected.
At this point one would normally just DOS format the now free first partition and install Windows. However I began to feel a little uneasy. Windows could just format the whole darn drive, or some other similar screwup could happen, in which case I would be placed in the position of fdisk'ing the partitions and reinstalling everything from scratch. The original disks would, of course have all the applications except for those extra packages installed by me, but any custom configurations would all be lost.
The machine was now running Mandrake 9.0, Debian 3.0 and Slackware 8.1. Of these, only losing my Slackware install would cause me grief. This has been running like a top, boots to KDE 3.0 in less than 30 seconds, including my sign on, and is absolutely rock solid stable. It also has the CUPS print system set up perfectly for all my printers on the LAN. So I must retain this setup at all costs. The solution of course is to fully back up everything from the Slackware install.
At that point the desire to have a simple, easy and foolproof backup and recovery method took hold.
If we are a home or SOHO Linux user I would suggest the following, it should:
A quick review of past Gazette articles and a search of the web will turn up hundreds of backup solutions. Many are specifically aimed at the backup function, many at the repair and system recovery part of the overall effort to get back to some predefined state. Virtually none are customized to your system, or your specific requirements, so why not roll your own solution? That is what we do here.
Most home or SOHO users do not have a tape drive system and are unlikely to purchase one for the sole purpose of backup, given that the cost of the tape system and software most probably exceeds that of the computer itself. This essentially leaves just backup to removable disk, backup to the same or another hard drive, backup to CD and backup over a network to some other hard drive. This last is essentially just a more complicated backup to local hard drive except there is zero chance of it being lost when your system goes down. So let us look at these options.
Floppy - Good for incremental backups on a daily basis and perhaps the best solution for saving work as it progresses, but useless for system wide restoration. The LS120 Disk and the Zip disk are not large enough or common enough to be considered for the sort of simple but complete backup considered here.
Hard Drive - One can back up to a separate partition on the same drive, which of course is of little use if that drive fails, or one can backup to another hard drive in the same computer. This is good except there is a fair chance that a power supply failure or nearby lightning strike could fry both drives (or somebody could steal the computer), leaving nothing to restore.
Network File System Transfer - This is a good solution to backup and restore of the files, for one interested enough to correctly install it, however it does nothing for the process of getting the system up again to the point where one can restore the files. Too complicated for most to institute.
CD-ROM - This is where it begins to look interesting. These days most Linux users have installed a CD burner and the availability of cheap CD-RW disks means that the cost of maintaining something akin to the traditional rotating backup system is definitely on. This is the one for us.
The most essential requirement is to have a working and reliable CD burner. Any current Linux distribution will have the tools required, and to minimize media costs, about $4 will supply two good quality CD-RW disks. For daily backups these will last for about five and a half years and used weekly a machine eternity!
The scheme proposed here is to use the two CD-RW disks to take backups in rotation; in my actual implementation I have color coded the spine of the disk covers Red and Green respectively, to aid in the correct rotation.
We also require the backup disk to self boot into a minimal working Linux system. This is to ensure that we can re-establish the Master Boot Record (MBR) and the rest of the original partition information if required. This rules out using a boot disk image as commonly supplied with the majority of distributions. These supply just a boot method and a Linux kernel, and usually boot straight to the partition they are customized to boot.
After a quick perusal of the small Linux on self boot CDs I settled on using the classic and well tried TomsRtBt disk in 2.88 MB image format. This is not an ISO image, but is suitable for being the boot image of an ISO we will burn. It is also to be found at various other sources on the web. I have used this in the floppy format and it is very good and quite complete. Note that it also includes a Toms FAQ.
In order to restore our working Linux system to a given state we will require records of all of the current directory contents which are changing on a day to day basis or have changed as customizations since initial install. This can be done laboriously by inspection and detailed lists, which will minimize what must be restored, or accomplished very easily by backing up the entire contents of these directories.
In my case I have decided to back up the entire contents of /home /etc /usr/local /opt /var /root /boot of the Slackware 8.1 partition.
In addition to the contents of each of the identified directories above there are some more very important pieces of information one wouldn't want to be without if a sudden failure to boot occured. These are a binary copy of the MBR, a text list of the Partition Table, a copy of the fstab file in case you have forgotten which partitions correspond to what filesystem, and optionally a copy of the current XF86Config file and/or the text output of commands like lsdev and lspci for full system information.
Also how are we going to structure all of this information to ensure it gets onto the CD in such a way as to be completely self contained and usable for the task at hand?
Here is what I did. Firstly create a directory to hold all of the information to backup. As root: mkdir /tmp/backup. Note here that I am using /tmp as repository for the constant part of the backup CD. This is safe in Slackware, but might not be in other distributions, choose a safe location and one not itself backup up by the tar file.
Put into the backup directory a copy of TomsRtBt Img file : cp ./tomsrtbt288.img /tmp/backup/tomsrtbt288.img, here the img file is in my home directory.
Put into the backup directory a copy of the Master Boot Record: dd if=/dev/hda bs=512 count=1 > /tmp/backup/MBR.bin. The MBR holds the first stage of the boot mechanism you employ, in my case stage1 of Grub, the GRand Unified Boot Loader, or LILO, and also the partition information for the Primary Partitions. The Extended Partition information is held elsewhere on the disk and can if required be restored with the information you will store from the fdisk command detailed next.
Put into the backup directory a list of the Partition Information : fdisk -l > /tmp/backup/Partition_Table, this will be used to compare with a Toms listing of the partition table before any restoration takes place.
Put into the backup directory a copy of fstab which defines the file system mount points, any errors here and the files and devices will not be accessible. cp /etc/fstab /tmp/backup/fstab.bak
Optionally copy any other information you wish available to you before you are able to boot into your newly restored Linux system. For easy accessability I keep a copy of XF86Config on the disk to ensure that I can always set up X the way I like even if installing a new system upgrade, and a copy of menu.lst as I use Grub as my boot loader of choice. cp /etc/X11/XF86Config /tmp/backup/XF86Config.bak ... cp /boot/grub/menu.lst /tmp/backup/menu.lst.bak
These files will be added to every copy of the backup disk that is burned, and need only be changed if one of them changes, when of course it should be copied over.
The script to accomplish this is shown below, for a text copy see backup. Be sure to rename the file without the .sh.txt part and to make it executable - chmod 755 ./backup - and copy to somewhere in roots PATH, /usr/local/bin is a good place, and do the same for the next script.
#!/bin/bash
# backup
#------------------------------------------------------------------------------
# Script to enable easy backup of all important Linux files
# and also creates a customized system repair disk.
# Uses two CD-RW Disks labled "RED" and "GREEN to rotate backups
#------------------------------------------------------------------------------
# The backup directory already contains files for boot and recovery.
# One can add more - my Slackware 8.1 system backup is < 580MB.
Backup_Dirs="/home /etc /usr/local /opt /var /root /boot"
Backup_Dest_Dir=/tmp/backup
Backup_Date=`date +%b%d%Y`
Image_File=/tmp/backup.iso
declare -i Size
# Create tar file with todays Month Day Year prepended for easy identification
tar cvzf $Backup_Dest_Dir/$Backup_Date.tar.gz $Backup_Dirs &> /dev/null
# Start backup process to local CD-RW drive
echo "Backing up $Backup_Dest_Dir to CD-RW Drive - $Backup_Date"
echo "Creating ISO9660 file system image ($Image_File)."
mkisofs -b toms288.img -c boot.cat -r \
-o $Image_File $Backup_Dest_Dir &> /dev/nul
# Check size of directory to burn in MB
Size=`du -m $Image_File | cut -c 1-3`
if [ $Size -lt 650 ]
then
echo "Size of ISO Image $Size MB, OK to Burn"
else
echo "Size of ISO Backup Image too Large to burn"
rm $Backup_Dest_Dir/$Backup_Date.tar.gz # Remove dated tar file
rm $Image_File # ISO is overwritten next backup but cleanup anyway
exit 1
fi
# Burn the CD-RW
Speed=4 # Use best speed for CD-RW disks on YOUR system
echo "Burning the disk."
# Set dev=x,x,x from cdrecord -scanbus
cdrecord -v speed=$Speed blank=fast dev=1,0,0 $Image_File &> /dev/null
Md5sum_Iso=`md5sum $Image_File`
echo "The md5sum of the created ISO is $Md5sum_Iso"
# Could TEST here using Md5sum_Iso to verify md5sums but problem is tricky.
echo "To verify use script md5scd, this will produce the burned CD's md5sum"
echo "run this as User with backup CD in drive to be used for recovery."
echo "This verifies not only the md5sum but that disk will read OK when needed."
# Remove image file and tar file
echo "Removing $Image_File"
rm $Image_File
echo "Removing : $Backup_Dest_Dir/$Backup_Date.tar.gz"
rm $Backup_Dest_Dir/$Backup_Date.tar.gz
echo "END BACKUP $Backup_Date"
echo "Be sure to place this backup in the RED CD case and previous CD in GREEN"
echo "------------------------------------------------------------------------"
exit 0
In use the process is simple, I usually backup every day, if not doing much on the system then every week. At the start of every backup I place the cdrom from the Green marked case into the CD burner. In an xterm I su to root and issue the command nohup backup &> /tmp/backup.log &, close the xterm and go to bed. The backup only takes about 15 minutes and so can also be done at any convenient time in a work day. When next at the computer I cat /tmp/backup.log and check that backup went well.
If I also want to verify the backup ISO I note down the first and last four or five letters of the listed ISO md5sum. As my burner will not reliably read back the CD just written I transfer the CD to my cdrom and verify that the md5sums are identical using the script md5scd, see below for a listing. If they are, I put that newly burned CD into the red case and the last burned CD into the green case ready for the next backup cycle. If any possibility of confusion exists one can always check the date on the tar file. Note that because of the burner not reliably reading the backup CD, that I have not included an automatic check of the md5sums, as failure to validate does not mean the CD is in error, just the read from the burner was. In fact, I have never experienced a md5sum compare failure when using my cdrom. I consider the MD5 checksum essential as even a single bit error could conceivably corrupt the whole compressed archive.
#!/bin/bash
#------------------------------------------------------------------------
# md5scd ---- Data CD md5sum Verifier
# Script to automate determining Md5sum for ISO9660 CDs
# NOTE - This script assumes that correct md5sum is known and one wishes
# to verify that a particular CD copy has been burnt correctly.
# If working from a downloaded ISO image use "md5sum ISO" at command line
#------------------------------------------------------------------------
# Requires standard tools found in all Linux Distributions
# If script invoked as user, check all permissions, groups, etc.
# Missing arguments?
if [ $# -ne 2 ]
then
echo "Usage - md5scd mountpoint device, ex - md5scd /mnt/cdrom /dev/hdc"
exit 1
else
: OK have arguments
fi
# Loop over md5sum determination ...100 good copies for install-fest?
again=yes
while [ "$again" = yes ]
do
echo "Please insert CD at $1 and press ENTER when ready"
read #Wait for insertion of disk
mount $1
Block_Count=`df -k $1 | grep $1 | cut -c 25-30`
#700Mb disk cannot exceed this ^^^^^ column limit in 1k blocks
umount $1
Md5sum_Cd=`dd if=$2 count=$Block_Count bs=1024 | md5sum`
echo "The md5sum of the CD at $1 is " $Md5sum_Cd
echo
echo -n "Verify another CD? [yes/no]"
read again # Wait for "yes" -> repeat, anything else -> drop through'
done
exit 0
Before that eventuality one should make sure the backup disk will boot, make sure one understands the limitations of tomsrtbt and as the only editor available is VI, practice reading the various files placed on the backup disk. The disk will have to be mounted first mount -t iso9660 /dev/xxx /mnt. It is possible to unzip and untar the tarred file using tomsrtbt by first using gzip and then tar.
However it is probably better to first check that the partition table is correct by fdisk -l and reference to the stored table, and if not to restore the MBR dd if=/mnt/MBR.bin of=/dev/hda bs=1 count=512. This will restore the primary partitions and the bootloader. Then use fdisk and the partition table file to manually restore the extended partition and the logical partitions within. This all requires nerve and practice! However any changes can be abandoned if not sure or only practicing.
Next do a clean install to the now proper partitions. Reboot to the point where one has a root console, mount the backup CD and execute tar xvzf /Mount_Dir/Backup_Tar_Filename. This will restore all of the previous directories into their correct places, and should leave you with an almost fully restored system.
Note that if the problem is only lost or corrupted files, one can also restore any of the saved directories at any time by extracting with tar xvzf /Mount_Dir/Backup_Tar_Filename /home, for example.
The test of any system is, "Does it work?" I initially verified that the backup CD does boot into Toms wonderful Linux system, that all of the text material was readable, of course fdisk -l did correspond to the stored version. I did not reinstate the MBR from the binary image file, however it is there if I ever need it.
The final test was to restore my Slackware 8.1 system in place of the original Mandrake 9.0 system, before installing Windows and perhaps needing to restore.
In brief,
Backup is easy to do and easy to keep doing. In use there are a number of small improvements that could be made. The manual backup and verification commands could be made shell variables and invoked with a single word. Also if the total file size becomes a factor one could use the --exclude flag of tar to not include large sections of invariant code in the tar file, or use bzip2 compression. As it is now, complete directories from root are saved.
The urgent need for the Windows applications turned out not to be so urgent, but provided the prod to actually backup regularly. Perhaps my next project will be to install Wine and try to get those pesky applications to run within Linux, without the need to always reboot.
I would be interested in any comments for improvement, indeed any feedback would be welcome, particularly if glaring flaws or omissions are evident. I can be reached at this address. This scheme has been in use for only a short time but so far I'm very satisfied, I encourage you to also institute regular backups. If you want a quick ready made approach try this one,a few changes to the scripts should have you backing up today and everyday after that .
![[BIO]](../gx/2002/note.png)
Retired Control Systems Engineer, spent much of career designing and
implementing Computerized Control and Shutdown Systems for Canada's CANDU
Nuclear Reactors. A programmer for over 40 yrs and a Linux enthusiast since
1994, first log entry shows 7.83 Bogomips on a 386 DX33 machine still running.
Linux and Golf are in first and second place among many other hobbies.