I'm a Loser... (of Data on Hard Drives!)

I started getting good at building computers in 9th grade, and throughout high school, I had lots of fun constantly upgrading and improving my "big rig" to have more hard drive space than anyone else in the neighborhood.  By 2005, I was well on my way to 2 terabytes of space, and I now have approximately 20 hard drives ranging from 100GB to 2TB in size, and with both SATA & PATA (IDE) interfaces.  (This doesn't count all the really old <5GB drives I used to play with while I was busting my chops.)

Anyone who's taken a probability class with a focus on electrical engineering knows of a little statistic that works against me and anyone else who owns a large number of hard drives: this statistic is called "Mean Time Between Failure" (MTBF).  In short, the more hard drives you own, the more likely any one of them will fail on a given day.  Well, over the last 16 years, I've had a fair share of failures:

• 1997 - Laptop hard drive
• 2006 - Power surge on my "server", prompting me to get real Web hosting - $2200 recovery
• 2008 - External hard drive failure
• 2011 - 1TB hard drive partition table became corrupt after computer showed "CPU Over-temperature Error", which was ironic as it was only 6 degrees Fahrenheit outside that day :-P
• 2013 - Really old 250GB SATA drive going slower, showing numerous SMART errors increasing daily

This doesn't count the times in which user error has been the primary reason for a failure, like the time I borked a disk using PartitionMagic and lost some really cool videos, or the time recently when I was using dd to make a disk clone, but had just rebooted and didn't realize all the hard drive sda_ assignments had just changed, so I was writing to the wrong disk!  Turns out I was making a disk clone of the failing 250 onto my real live game show hard drive instead of my blank backup drive!  Good thing I was maintaining a clone of that drive -- originally it was part of a RAID 1 group, but that's a whole different story.  So let's say I wasn't extremely pissed off that day, which was probably a first for a data loss event. :-P

I learned about ddrescue in Linux from a coworker after mentioning the story about my 1TB drive, but after it made a disk image that seemed to have absolutely nothing on it, I abandoned that and haven't looked back.  I had great luck with the testdisk utility which works in Windows & Linux, and have gotten most of the stuff I'm interested in off that drive already.

The only problem with my clone-keeping of the game show drives was sometimes I did stuff on one hard drive and didn't do it to the other, such as extracting VOBs from ISOs.  This caused me to become interested in the differences prior to wiping my partially-overwritten drive to make it part of a NAS box.  Well, due to the way I corrupted the hard drive with dd, testdisk told me the filesystem was damaged and couldn't read anything for me like it had so handily done with my 1TB drive.  Looking for another solution, I stumbled upon GetDataBack for NTFS.  I haven't had too much luck with Windows data recovery solutions in the past, so I was willing to give something besides R-Studio and EASEUS Data Recovery a shot.  One of the reviews noted how GetDataBack would even give you a preview of the data it's recovering by showing you real files open in real applications, instead of just showing you the names of the files it recovered and having you pay $79 only to realize whatever you're trying to recover is actually empty.  That'd be a bummer.  I feel sorry for anyone who has paid for software on false promises it'd recover your data.

GetDataBack scanned my hard drive for files overnight, and it found everything I wanted!  I painstakingly scanned and compared the contents with my good game show drive and found around 100 files on the corrupted drive that weren't on the good drive.  Conveniently, you can save your hard drive's layout in 2 or 3 different steps so you can quit the program and reboot without losing any of your progress; I took advantage of that feature several times.  Now it was time to pony up $79 for a full license to GetDataBack in order to recover my files... OR NOT!  One of the first things I did after installation was to adjust the application settings.  One such setting was the location of the temporary folder the application used (for what reason it needed one was unbeknownst to me at the time), and since I was curious what kind of data it would be getting on my hard drive, I changed it to something easily accessible to me.  Originally I was poking through GetDataBack's data files to find out where it had detected all these files started and ended on my hard drive, and was planning to use dd for Windows in order to just clone the data myself.  After seeing how that didn't exactly work on the first try for a JPEG image, I decided to use a hex editor to find out if I was even close.  I opened up what I saved in dd in Hexplorer, then picked Hexplorer to open up the original file from within GetDataBack.  There, Hexplorer uncovered the weakness of GetDataBack: it actually saves the files to that temporary directory before you preview them, then it opens the file or asks you which application you want to open it!

Well, now I knew what to do.  I needed to recover files of up to 4.5GB in size, and I really didn't need another application to try and "open" them on the spot.  I used GetDataBack to "Open with..." all these files, and then hit Esc to cancel out of the popup menu once it came up (which happened after quite a while).  Then I would simply rename the file it produced into the temporary folder, and after doing several files, I'd shuff them off into a completely different directory so GetDataBack wouldn't wipe them away when it cleared my temporary directory.  However, there was one pesky file that wouldn't move because it was being used by "System" for Heaven knows what reason -- you know how Windows works, and the way to cure most any problem is a reboot.  I don't care much for rebooting because while my Windows 8 boots up in about 2 seconds from the SSD, the rest of my motherboard takes about a minute to come up & POST, initialize all the hard drive controllers, and do all the rest of the stuff an old, slow BIOS does.  (Yes, for someone who has written uEFI executables that can run in the uEFI Shell, and worked with testing various facets of it for four years, I still don't have a uEFI-enabled computer myself!)  Nevertheless, after my reboot, the one remaining file I couldn't copy was still in its place.  GetDataBack doesn't even delete the contents of its temp folder when you close the application.  If anyone from that company reads this blog, I would implore that you add this feature so people's hard disks don't become cluttered with files they'll never know exist.

Now, the rest of the story is that I'm trying to populate my NAS box with all sorts of data because I'm sick of having hard drives scattered throughout various computers or sitting idle in a bin, and tired of having to set up all sorts of slow network shares to access data from wherever else I am besides at that computer's keyboard.  That's been a struggle so far too, since I purchased a cheap NAS box (on a closeout, no less) that has mixed reviews at best.  I got a terrible transfer rate (1.25-1.75 MB/s) using Windows with Wi-Fi.  Once I had my installer come in and finish the patch panel, I was able to wire the NAS box straight into my gigabit switch; then I was able to see a sustained data rate of almost 23 MB/s over about 8 hours.  Not all good things are meant to last, though; the transfer was interrupted once again for some unknown reason, and now I have to figure out which file it was trying to write to and what directories haven't made it in yet.  Not only that, but the dying 1TB drive's warranty expires next month and I need to make sure I recover everything good from it before I RMA it.  And I'm still working on delivering products to my Kickstarter backers... lots of stuff going on!