Another technical question: back ups and restores

Those who do a lot of work with a computer may know the old phrase, "Jesus saves, and so should you."

We have probably all had the experience not just of losing minutes to hours of work on a document or project, but perhaps losing, and disastrously, everything because of a hard-drive crash or some other problem.

If you haven’t… you will.

There are offsite back-up companies.  There onsite solutions.

What do you do?

I am particularly interested in hearing about

Programs and hardware for backing up a computer so that you could do a "restore", even to the "bank" disk of a different computer if necessary.  Or even for migrating everything you need to a new computer.

Do you use the built in functions in, say, Windows?

Do you use specialized software?

About Fr. John Zuhlsdorf

Fr. Z is the guy who runs this blog. o{]:¬)
This entry was posted in "How To..." - Practical Notes. Bookmark the permalink.

29 Comments

  1. JohnMa says:

    I use Acronis True Image to back up everything once a month. I make a clone of my hard drive onto an external hard drive.

    I also have all my important files for school backed up on via my website.

  2. Jerry says:

    An on-site backup will protect against loss resulting from a system or component failure; to protect against events affecting the facility (fire flood, tornado, etc.) the backup media must be rotated off site.

    Whatever your backup strategy (except using a cloud based backup service), always have a least two sets of rotating backup media. If backing up to the cloud, consider alternating between two different providers.

    Periodically attempt to restore your data just to make sure it can be done successfully.

    The easiest hardware backup solutions are USB thumb drives or external hard drives, depending on how much data you have to save. There are several companies that sell small disk servers based on the Microsoft Windows Home Server software that will perform automatic backups of your system using incremental snapshots. You configure how many of the incremental backups to save. These servers also provide the capability to share files between multiple computers at home, and even to access the data from the Internet.

  3. Andrew says:

    For software I rely on the original disks that came with the purchased software. For all other things I use external hard drives.

  4. zapman449 says:

    To restore a system image from computer x to computer y in windows[2] is VERY challenging to do. There are companies that make a lot of money for tools that do this. Unfortunately, most of them have gone out of business, or have just gone into limbo[1].

    However, what you can do is build all of your important systems as ‘Virtual Machines’, and then back up their ‘virtual disks’. You could then install the Virtual Machine software on a different computer, and run the old Virtual Machines on the new system. You’ll sacrifice some performance but from a backup/restore perspective it’s far easier.

    Backups are a really messy problem.

    One strong suggestion I have is to first categorize what your needs are for various bits of data. Software installed? Probably not terribly important, assuming you still have the media. The data produced? Probably very important. Once you have some broad categories, you can then make provisions to protect the data as appropriate[3]. You also know roughly how much data you’re actually talking about[5].

    One useful suggestion might be to build/obtain a NAS[4]. Then you have to get disciplined about making sure that all your ‘hard to replace’ data is saved to the NAS. The good news is that once it’s there, you can get to it from any machine in your network. Finally, after building this piece of infrastructure, ensuring that it’s backed up regularly is a good idea (whether to Amazon’s S3, Mozy, etc).

    –Jason

    [1] Racemi doesn’t sell theirs anymore… I think you can still buy Platespin’s tool through Novell, but it’s mostly working in the Virtualization space now.
    [2] With Linux, you can plug the old hard drive into a new machine, and it’ll probably ‘just work’ so long as you’re playing with a mainstream distro. MacOSX has tools that can do this as well (TimeMachine, etc).
    [3] Companies pay good salaries to the people who handle this task… usually sysadmins.
    [4] Network Attached Storage. Basically it’s some number of disks, and an network jack. There are a million ways to build one these days with various pros/cons.
    [5] which helps in budgeting.

  5. markomalley says:

    The simplest solution is to buy a USB hard disk and use either Windows or your security software to manage the backup (I know Symantec and ESET products do so automatically, I believe that MacAfee does as well)

    The more elegant solution is to configure your hard drives as a RAID (redundant array of independent disks). This is typically set up through the BIOS (if the motherboard has a built-in RAID controller). The nice part about RAID is that, dependent upon the RAID configuration you select, it automatically writes the data to multiple independent physical drives at the same time (that way if one drive fails, the data is still maintained)

    My problem with off-site storage for individual users is that you have to “TRUST” the storage provider. You have to trust that he will maintain your data in a physically secure manner and that the data will be stored in a manner that will prevent malefactors from gaining access to that data. While a reputable provider that has as its primary market corporate or government entities (like Iron Mountain or the like) will have the necessary means in place, I am concerned about providers who market toward home users…a/k/a you get what you pay for.

  6. chcrix says:

    How is the machine configured?

    When I was still running windows I always kept 2 hard drives in the box. I would install windows independently on each drive stand alone. Then I would use windows backup to

    1. back up my data volume
    2. back up my system volume
    3. back up the system state (crucial in windows)

    Those backups would be copied to the second drive. That way if the drive failed the system could be rebooted on the second drive and everything could be restored. It sounds complicated and it is, but it is not too bad for the tech saavy. It doesn’t take long to back up a data volume.

    Obviously this does not protect one from a catastrophic electric strike that fries the entire machine.

    I don’t backup to offsite providers – free or otherwise. As Herod Agrippa said in “I, Claudius”: “Trust no one Marmoset.”

    I no longer do these things since I went Linux. I just copy the contents of my /home directory on my first hard drive to the corresponding directory on the second drive.

  7. ericrun says:

    I use Linux, and have a simple script to copy any new or modified files in certain folders update to the backup (internal) hard drive once a day. On Windows, this would be a batch job, run from the Scheduled Tasks screen in the Control Panel.

    Then, periodically, copy the files to an SD card or spare hard drive, and carry offsite. One way to do the offsite is use a safe deposit box at the bank.

    If you want to secure the data, you should probably encrypt it before you let it out of your control.

  8. Fr Matthew says:

    I use Time Machine on my MacBook, but I also back up on line using the provider “Diino”, which has a nice incremental backup client that runs in the background and is a very affordable service.

    On Windows, I played around a little with network storage devices which come with their own backup/restore software, as well as backing up manually to external USB drives.

  9. papaefidelis says:

    I also use Acronis True Image 2010; I have a second internal HD for daily incremental backups and an external HD for full weekly backups. There are free alternatives as well but I haven’t used them nor have I used any of the offsite backup services, of which Dropbox is very popular service. I also burn certain files to DVD’s (like the collection of Acta Apostolicae/Sanctae Sedis and my electronic Oxford Latin Dictionary), but I trust that method least.

    Acronis True Image allows one to mount the image as a disk and browse just as if it were a drive.

  10. kallman says:

    For WindowsI use a backup to hard drive program that came with the computer. The internal HDD is partitioned and can restore Windows OS from the second partition.
    For Mac I use Time Machine as well as back up data to another HDD. Also can use retrospect archival backup and Silver Keeper finder-level backup. Separate back up of all photos, email, music,etc on a third drive also

  11. rakesvines says:

    Ditto on external hard drives or USB. Rationale: If the system comes down, the data is not held hostage.

    Virus detection programs e.g. Webroot also has a back-up function – even to a remote site. You can probably automate.

  12. bittergeek says:

    I second the recommendation of a RAID setup. (Which actually stands for Redundant Array of _Inexpensive_ Disks, as distinct from SLED, the Single Large Expensive Disk. It’s a very old term.) Two identical (or close) disks, ideally run on separate controllers on your motherboard if possible, to allow for function even in the event the problem is actually in the controller not the disk. You lose the capacity of the second drive entirely, since it’s merely a live copy of the primary drive, but you can work through a blown drive, controller chip, or loose cable without necessarily even noticing. Disks are cheap, your time in recovery has value, and data that can’t be gotten back is priceless.

    Pro:
    Live copy, so no data lost no matter how recent.
    No new software to learn or buy – just about any possible computer will support this in BIOS or through the operating system with a minimum of effort.
    Faster reads – depending on your setup, your OS can read from both disks at the same time, for a performance boost.
    No third party has access to your offsite data.

    Con:
    Lose the entire capacity of a new drive.
    Live copy, so if you screw up you can’t go back to the older version. Corrupting a file will affect both disks.
    Requires opening the case and then mucking about with the BIOS. This is where things can go horribly wrong for the novice.
    All data is onsite, which is no protection at all from physical disaster.

    In addition to this, I would recommend some form of versioning for important documents. If you aren’t authoring enough data for a full content management system, at least use “Save As” often, appending the time and date to the end of the file name. At the end of the day when you make changes, copy all of these versions offsite, even if only to a USB drive which you then *get out of your house* – the trunk of your car may be enough. The true value to offsite backup is that it’s really unlikely that both sites will be hit by disaster at the same time. If you trust google or whoever with your files, great. If not, at least make it really hard for all copies of your important files to be wiped out in the same fire. Or from the same whupsy by that most dangerous of mechanical failings, a loose nut behind the keyboard. Nearly every human alive will have more “what was I thinking?” moments than fires. Like my sister, who only recovered her Master’s thesis, a week before submission, by scanning in an earlier draft after accidentally deleting the current copy in an effort to conserve space. Whups. That’s an advantage to backup via software: you can go back in time, but you can only recover data as it was at the time the backup was run. If you’re doing a lot of writing, make a lot more copies. Space is cheaper now.

  13. wmeyer says:

    Remember that your exposure is greatest on data; programs can be reinstalled from CD or DVD. Yes, it is tedious, but it’s no disaster. Lost data, on the other hand, whether it is tax returns, family tree, work records, or your own literary creations, is a different matter.

    Start by creating a base folder on your C: drive, or even better, have a different drive, where you organize data. Keeping the data in a different folder tree than the apps makes it very easy to manage backups.

    I take snapshots of development work using WinZip. For more serious protection, a source control system provides a well ordered solution. Off-site storage is also a good idea. A casual off-site approach might be as simple as mailing a file to yourself in gmail. Google takes care of guarding against data loss, so you don’t have to.

  14. laurazim says:

    I use Carbonite offsite backup system for my entire hard drive, as recommended by Rush. I do not have the brain for all of the technical rigamarole of trying to do it any other way, but I know that the things on my computer are valuable to me, and should something happen to my computer, I know that I could go to the Carbonite website and have the entire contents uploaded onto a new computer at any time. It updates itself silently at least daily, letting me know if there are files needing to be backed up by the color of the little padlock icon changing from green (“everything backed up”) to yellow (“some files not backed up”). It switches back to green when everything is done.

  15. Jim of Bowie says:

    I back up my data only (documents, pics and music) every two weeks to flash drives. I did’t see where windows allows you to overwrite only changed files like DOS used to, so I bought software called Backup Magic which does it. It takes about 5 minutes every two weeks.

  16. NeoCarlist says:

    I don’t bother backing up application software or the operating system, since a disaster would give me an excuse to do a clean reinstall and retweak of the settings to my ever evolving preferences. My data files are segregated into their own folder structure, so my onsite backups are simply copies to a USB stick and every other time or so to a CD which are stored in a fireproof file cabinet at home. I periodically (monthly) also burn copies to CDs that are stored in my safety deposit box and a fireproof box at my office. I also use Mozy for remote backup, using their option to create your own encryption key. Is this a little extreme? Perhaps, but my profession is Database Administration and IT Security so I’m paid to be paranoid.

  17. cregduff says:

    Father,

    A couple of options that have not yet been mentioned:

    There is a program called ghost, which enables one to take a snapshot of your system and store it away on either a dvd or external hard-disk. This is great for developing a standard image, which is the OS + applications + settings you want on a given machine, and makes it easy to either replicate that image on multiple computers, or restore an image to a computer.

    With local storage, you must have RAID. Given your areas apparent issues with power, this is a must. I would recommend a product called Drobo. It’s basically a chassis in which you can add different size hard disks, prices of which are getting very cheap. They have a proprietary kind of raid that has been very successful. The big advantage here is that as hard drive prices plummet and their sizes increase, you can swap out smaller disks for larger ones and the system automatically adjusts. It’s pretty brilliant, and there are alot of creative professionals who are storing their work on these. Their site has the details and a great storage calculator.

    I once wrote to you, maybe a couple of years ago, that I thought that your work, especially the blog, but I am sure your research and digital library, should be stored on redundant hard disks off site, maybe even outside the continental US. Terabyte drives have dropped tremendously in price lately, and it would not be difficult I assume to get a single copy of most if not all your media onto one of these small-sized, large storage drives. Amazon wish list addition?

    I would like to steer you specifically away from Tape backups systems, most places are moving away from this media backup strategy, especially since disk is so cheap now. For off line storage it is still used but largely in the enterprise for retention and discovery purposes. In that case, the best strategy is disk to disk to tape.

    I happen to be responsible for architecting backup systems, in the enterprise. One system I designed, for a creative group, and deployed three years ago now has amassed almost 2000 DLT3 tapes with a total storage of 1.6petabytes. We used Atempo Time Navigator. It’s amazing. But it is maintenance intensive and costly for consumables.

    Figure how long you can be down if you suffer a loss in deciding on solutions. Recovery time is a key component in the design of a backup strategy.

    Ed Casey

  18. czemike says:

    I use Windows Home Server (WHS). Nightly backups, can restore the computer image in about 30 minutes, even to a different hardware platform (but expect to spend an hour on the phone with MSFT getting them to give you a new Product Key… you can’t move a Windows license from one hardware profile to another but if you’re doing it as part of a “restore for system crash” with WHS they will hassle you for a while and then give you a new Product Key. Just know that up front and have a coffee, book to read, and notepad ready).

    WHS hardware: I use an HP MediaSmart server which has four SATA HDD bays and is smaller than my two-volume Haydock Bible. Backing up the WHS: you can use one of the on-board HDDs as a system backup and configure the device to copy all contents to two of the other HDDs so no one HDD failure brings the thing down. As for backing up the WHS: you can attach a drive to WHS and perform a system backup to it. That backup drive can be onsite, offsite, etc.

    Did I mention you can use the WHS as network storage to be shared with multiple computers on your home network? Very handy!

    Two success stories personally: I have saved one machine by doing a complete “brain transplant” from old hardware to new hardware — AMD/VIA to Intel/Intel no less! — without loosing any data or settings, and have full access to the last daily backup status of my main Vista Pro workstation when the motherboard failed which I can mount as a hard drive — the “Z” drive, if you can believe it — on my new Windows 7 Ultimate workstation.

    With WHS you only have to worry about loosing multiple HDDs on the WHS device at the same time. That said, you can back up the WHS to a HDD and then back up that backup HDD to the cloud with Carbonite or Mozy if you’re really paranoid, but whatever you decide, WHS is worth giving a long look.

  19. papaefidelis says:

    I would REALLY advise against backups to your main HD (e.g. kallman). If the HD crashes or become nonfunctional, you’ve lost not only your data but your backup as well.

  20. dahveed says:

    Hi Father,
    I use Time Machine on my Macbook Pro (work machine). I also have a general backup disk image, for most content. I also periodically made data dvds of my collection of old time radio. ;) I work in the IT field supporting a large number of people, primarily on Macintoshs, although perhaps 8-10 percent are on two flavors of Windows. Those machines are backed up (desktop and My Documents folders) via the server used by those people. Some of the higher food chain individuals on that platform do period self backups on dvds as well. I hope that helps.

  21. For documents I use regularly and need shared between multiple machines (or available from other computer), I use DropBox…it’s free up to 2GB of storage and syncs automatically, and there’s a web interface to grab files remotely. I’m even currently using it at one site to store BackupExec disaster recovery information automatically (system info and configuration information that simplify a full system restore, along with a recovery CD image that has system-specific drivers.) It’s really useful for me, since I’m working on the same documents from multiple computers (desktop at home, notebook on the road), and it’s nice to know I’m working with the latest files.

    For critical business data, tape with offsite storage. Expensive, but solid and proven technology.

    For a critical system that just can’t go down, RAID-1 for the system disk (the speed difference makes up for the space wasted), RAID-5 for data. The truly paranoid can opt for RAID-6 or even something more crazy like RAID-5+1 (mirrored stripes). Include at least one hot-spare. This must be checked regularly for drive failure, or else it’s useless…ideally there would be included device management tools that can be configured to monitor drive status and issue an alert (beeping, email, text message to the phone, something) in case of failure.

    Macs have Time Machine, which is a nice feature that I use on one of my Macs. For Windows PCs, Comodo has free software that does the same thing, but I haven’t tried it yet. They also have a backup program that will store data on an FTP server automatically, to give that additional security (they have a paid service to provide the offsite storage). There are others out there doing similar things, like Carbonite. But I’d rather not pay a monthly fee, personally.

    My current ideal home solution (which I may set up some day) for my Windows desktop: RAID-1 internal drives, 2TB USB hard drive, Comodo Time Machine configured to use the 2TB drive and an offsite FTP site. I’d only lose the data if a tornado ripped through town and took out both my house and my parents’ house (which is where I have access to offsite storage for free…my server, their basement :D)

    For an absolute basic no-frills backup solution, use an external USB drive and Robocopy with the /MIR option (among other switches, like /R:0 to not retry locked files, and /W:0 to not wait the default 30 seconds on a locked file) launched from the scheduler in the middle of the night. The drive won’t be bootable, but it will be a duplicate of all of the data, and won’t require the use of any special software to retrieve the data on another system. I’m actually using robocopy at one customer site in a domain logoff script to mirror their user files to a server.

    One important question I would ask: how many computers need to be backed up? Most of the above really only applies for a single system. Multiple systems can share a single NAS for local-ish storage, which then can also be backed up to other media or locations.

  22. Markus says:

    Father,
    For graphics programs I have an extra internal drive that automatically backs up all work. Most pro programs have this feature. For my Vista and 7 machines I have an USB external hard drive that Windows backs up twice a week. It has been working OK for the last 3 years on 3 machines. Offsites back ups have to slow your connections, one would think.

    Regards, Markus

  23. I too use Carbonite. With all my photography, not all of which I upload to Smugmug, I needed a way to protect it readily.

    One thing I considered is the need for something offsite in the event of something like a fire. I had been backing things up onto another drive.

    One word of caution is that Carbonite, which use to backup videos, has since backed off on that. They can still be backed up manually. There are other files they will not back up so read the print if you go with it.

  24. Kavi says:

    As the risk of repeating what people have already said, here’s my tuppence (or 3 cents).

    If your question is simply “offline or online backup?” the answer is: both. If you’re asking which is better…that could get complicated. It depends largely on how good your local backup is.

    The ONE BEST SOLUTION is to have multiple hard drives, and to backup any essential non-replaceable files – Documents and Pictures, mainly – just by clicking and dragging. You can get programs (or use Windows Backup) that do this automatically for you, but they’re not incredibly useful unless you feel like you’re not going to have time to make the backup manually. I myself like to choose which files go where in my backups.

    The vast majority of file loss is caused by hard drive faliure, so having multiple hard drives will prevent most file loss. But I do use online storage also, for two reasons: firstly, I often need to share files with people on the other side of the globe, and using online storage means that I can allow others to access it over the internet if I so desire. Secondly, I myself might want to access it from the other side of the globe. But if you primarily use one computer and don’t need those files when travelling, online storage isn’t incredibly useful.

    There is an exception to this – if you have a very fast internet connection and don’t want to shell out the $60-100 for an extra hard drive, you could use online storage as a extra hd alternative. But a backup in the hand is worth two on the internet!

    One final thought – I get a lot more complaints about external hard drives vs internal ones. External hard drives can be used, as Markus noted, for managing files between several computers, but a more durable option is preferable if you use external storage a lot, such as USB memory, a Solid State Disk, or online storage. An external hard drive is just a regular hard drive with a case which provides a USB interface. The USB interface tends to provide slower access speeds, and simply the fact that it isn’t in a PC case exposes it to a lot more potentially harmful circumstances, like my cat.

  25. cfhuber says:

    As no one has yet mentioned it, I would recommend CrashPlan (crashplan.com). Once set-up (fairly easy) it performs incremental backups at times of your configuring. Restores are simple. Backups are encrypted and compressed before they leave your machine providing security and efficiency.

    The developers of Crash Plan are very keen on the benefit of off-site backups and have designed their products to work across the ‘Net. Non-commercial use of Crash Plan is free. They provide a fee-based service if you don’t have your own off-site computer for a backup target.

    Crash Plan is multi-platform. It achieves this by using the platform’s Java Engine. I like the multi-platform support because it means it runs on Mac as natively as it runs on anything else.

    Probably the best thing about Crash Plan is that once configured, it just keeps working. Most back-up plans fail due to lack of discipline in following backup procedures. Crash Plan will even send email notices warning about the failure of a configured machine to reach its backup destination. (It also sends mail confirming that machines are backed-up.)

  26. moon1234 says:

    Father,
    A backup solution is hard to recommend without knowing what your requirements are. Do you have a single computer that is used for simple tasks such as writting letters, etc? Do you have a server at your home that is prforming critical tasks?

    For the former, backing up data is the best solution. This can be onsite or offsite. The location is really dictated more by the amount of data than anything else. Once you get beyond 10-20GB of data offsite solutions become cumbersome in the event of a restore. This means onsite storage will become necessary. This could be several external hard drives, a smaller tape backup, etc. If you are backing up more than one computer then tape will be cumbersome. External storage, such as hard drives, will be the easiest.

    Restoring whole physical computers is not really practical for a home user unless you have identical or very similar equipment. You can use programs such as Symantec Ghost or Acronis True Image, but these programs are geared more towards enterprises who need to copy the same computer image onto many computers with the same hardware. They are very good at taking snapshots of your entire computer, these can then be restored in the future if a hard drive were to crash.

    Virtual machines for a home user are not really practical. They do however make backups very easy. The entire operating system is contained within a single or very few files. These files are then copied to tape, external or enterprise storage. This techonology is used more for servers that home computers. Because the virtual hardware stays the same, the underlying real computer hardware can change. This makes restoration to dissimilar hardware possible.

    Until we know what your requirements are in a little more detail, it will be hard to make a recommendation.

  27. RichardT says:

    Cheap solution – on throwing out an old desktop, I removed the hard drive (it just needed a screwdriver; even I could do it) and for a few pounds bought a gizmo that enabled it to be connected to a USB port. Instant, almost free back-up device.

    I back up documents etc. from my laptop to that – I just “drag and drop” the “My Documents” folder and anything else I want.

    Because the hard drive is just storing documents rather than programme files, it fits lots of dated back-ups. Because it’s completely physically separate, it’s more secure.

    Not ideal though, because it relies on me remembering to do a back-up. And it only backs up what I tell it to, not programmes, settings or anything like that.

    When I replaced my laptop the same method worked perfectly well for transferring data. I also transferred e-mails, e-mail addresses and the diary in the same way. Programmes I installed from the original source (disc or download), which seemed easier than messing about with copying, although did mean having to re-do settings.

  28. RichardT says:

    P.S. Cheap solutions means more money left for good food & wine.

  29. aquinasdad says:

    I am glad cregduff chimed in and mentioned Atempo, the best enterprise backup solution out there.
    For large businesses.
    I design backup solutions for enterprise and SMB (small to medium businesses) and you are in the worst place – lots of OK options.
    If you simply want it taken care of, Carbonite is a good solution, as is Asigra. You almost certainly have a local company that will run Asigra backups for you and do the major restores, as well. If you want to do it yourself, BackupExec is good and you can get a non-profit discount for use by clergy. BE can backup your data to disk, tape, or both.
    Let me know if you want to talk prices, time, etc.

Comments are closed.