Hard drive failure


If there was one word which captures the actionable advice behind the principle of embracing failure it would be…”backup”.  Not embracing failure means assuming that everything will go more or less to plan.  Embracing failure will be assuming and building into your plan – whether it be your business plan or your life plan – that things will go wrong.  Then you need a backup. A “Plan B”. If you have embraced failure with backup, then most failures in life will be minor bumps. Unfortunately, those bumps get a bit harsher when the backups themselves fail. Getting a flat tire is annoying. Getting a flat tire and finding that the spare in your boot is flat…is a big problem.

This tenet has been hammered home to me this past month with the failure that every digital citizen dreads – the hard disk failure.  In this day and age, computers have gotten so much more reliable; one could be seduced into thinking that they never completely fall apart.  The last hard-drive failure I had was in the 90s (though a reminder of hard-drive mortality hit me last year when an old machine I kept for testing had its drive fail). I do have a bit of a complicated computer environment.  I run a “virtual computer” on Windows (7.0) inside my MacBook Pro computer which doubles the operating environments each of which have their own backup protocols and tools.

Fortunately, practicing what I preach here, I was well backed up and now more or less back to normal.  I’m not sharing the story in some sort of sanctimonious ‘I told you so’, but rather as a wakeup call for just how down the downside can be of problems.  It turns out that not only did my hard disk fail, but a whole series of failures took place one after the other.  It was only the rigour of my backup practices (and a bit of extra money for some recovery assistance) that saved me. 

The litany of failures were…

  • Hard Disk Fail – My internal hard drive failed on my computer when I dropped it (okay, duh, but sh*t happens).
  • Disk Recovery Fail – I took the failed drive to a disk recovery specialist and the data on the drive was unrecoverable even with their clever clean room tools and tricks (these guys can sometimes do wonders…but not every time).
  • Mac Remote Drive Fail – The remote drive I used for Apple’s Time Machine back up program stopped working (fortunately just after I used it to restore my MacBook Pro…close call!)
  • Mac Backup Software Fail.  Apple’s backup program, Time Machine, actually worked impressively well and fast to restore my Mac environment to the clean replacement drive I bought…BUT it turns out that it doesn’t backup virtual machines (ie. my Windows environment) so that major part of my data was completely omitted. Fortunately, I had manually backed up the VM, but the backup was a bit older than my Time Machine one that had recently run.
  • Windows Remote Drive Fail – When I went to restore my Windows environment (under Parallels virtualization software), my remote drive chose to pack up and fail as well! As it turned out, it was just and chassis failure and the disk mechanism itself was fine. A superb service outfit in Las Vegas (Century 23) swapped the drive into a new chassis and it was fine.
  • Windows Backup Software Fail – After everything had been sorted out and restored recently, I resumed my backup of the Windows environment only to get a number of reported problems. The free backup process in Windows is convenient and value-priced, but it is not the greatest backup program (as I learned troubleshooting my way through various problems it threw at me).
  • Lori Backup Drive Fail – I used the occasion of all of the backup work to do a long overdue backup for my wife Lori’s computer. Lo and behold, I plugged in the Maxtor USB drive…and it too had failed.

If backups were spare tires, then my experience was the equivalent of having 7 different spares in the boot/trunk all flat!

In the final tally, I restored my machine and paid the deep price of the effort doing do, the stress of not really knowing what would and wouldn’t be restored, a productivity hit for a few days and about 4 hours of work lost from the time of my most recent backup before the crash. Not a disaster.

But what is more intriguing for here are the upsides to the whole mess. The incident and the aftermath actually and literally shook up my whole computing set up.  In the aftermath, I perversely feel that my computer is working better than ever and better than it would have been had I not suffered the blow.  I think that the productivity gains have quickly compensated for the hours lost in the lost work and recovery process.

Learnings – The first dividends of any failure are the lessons learned.

  • Hard drives are more likely to fail (in more ways) than you might think.
  • Apple Time Machine does not back up virtual machines.

Improving Backup – This bonus area is a combination of Learning and a more thorough embrace of failure. I thought that I was pretty well set on all of my backup protocols, but the “this is not a drill” crisis and the knock-on failures that followed (see above) pointed the way to a number of enhancements to my backing up now…

  • New Backup Devices – I’ve invested in two new Western Digital terabyte drives (one for the Windows environment and one for the Mac environment).
  • Wireless Backup – Part of my loss (4 hours of work lost) was from the frequency of my backing up. I thought that a weekly back up would be adequate. And it is for averting catastrophe. But not for averting the loss of hours of work you have done that week. I do more than 4 hours of work a week, but much of it is captured online. Since the incident I thought of doing of nightly backup setting it off before bed each night, but that will be easy to get slack on. The current state-of-the-art is wireless ongoing backup. A number of products on the market connect to your wireless router and their backup programs continually back up your work over the air (so you don’t have to think about instigating it).
  • VM Backup – Now that I understand better how Apple Time Machine deals with VMs, I am changing how I approach backing up the VM.
  • Organising Active from Archive Files – One of the things getting in the way of regular backups is the sheer volume. A good backup program should be able to do incremental backups on only the stuff you’ve changed, but in practice it needs to at least look at every file. And after 30 years of computing, I have a lot of files. And some programs (eg. DeepZoom Composer) produce not just hundreds or thousands, but tens of thousands as artefacts. Going through this pile slows the whole process down. The answer is to segregate your “Archive” material (files that are not like to change like archived email, photos, videos, old documents) and “Active” material (your current documents). When I did this, I found that only 4 GB of my 90 GB were in any way “active”. Once I segregated them, now even a brute force backup of my 4 GB active files just takes minutes (and I could put all of them on an SD card to have access to on any machine) compared to hours before.

Cleaning House – The new hard drive in my MacBook Pro meant I could set it up properly from scratch. I always remember in the early days of the PC that when there was a Windows upgrade, it was often a good idea (given the awkward complexities of the platform at the time) to simply back up all of the data and format the hard drive for a completely fresh start. You would have to go through the rigmarole of re-installing all your apps and restoring all your data, but it was a worthwhile investment to get everything running the most smoothly. Now upgrade processes are so refined that this approach rarely makes sense. But that situation does mean that people now go years with lots of mess slowly accruing on their machines that becomes hard to decipher and untangle.

  • Reconfigured New Environment – When I first got my MacBook Pro 3 years ago, I was a relative novice to the new environment (after my 15 year career at Microsoft). As such, I made a number of missteps along the way of setting up and building my environment over time (eg. I set up a partition I didn’t need, I got a clutter of Identities, I set up two user profiles when I could have done one). I learned the error of my ways, but in a number of cases it was easier to kludge a work-around than roll up my sleeves for a comprehensive fix.
  • Cleaned up MacBook Pro – The Genius Bar of the Apple Stores is about the best conceived and best delivered service concept that there is. With my machine checked in to install the new drive, they also took the opportunity to replace a missing screw and pad and clean out the dust that had accumulated inside with a specialist vacuum. I even go my cracked iPhone back fixed for a modest fee (didn’t know you could do that at the Apple store).
  • Enhanced Backup Setup – I set up an entirely new Windows backup using a Western Digital MyCloud wifi system. I also changed my Mac Backup to a new Western Digital My Passport USB drive for Mac.

The moral of the story: Embrace the failure of computer hard drives. A cautionary tale on the too often taken for granted world of backing up your computer. Belt, braces and duct tape. Backup, backup, backup.  And when the catastrophic failures do happen, you can look forward to the stronger situation you will get to in the aftermath.