top of page
Aug 9, 20127 min read

Long-term Data Storage Considerations


If you value your photos, videos, and documents – and I know you do – you may occasionally back up your digital files, without any real consideration for long-term storage. But we all know hard drives (even external ones you rarely use) will fail on occasion, and plastic storage media like DVDs and CDs degrade over time. So what’s one to do?


First and foremost, the best practice to battle the digital reaper is to copy/back up the same data to various types of media. I am currently in the process of backing up everything on my two computers to an external hard drive and to a pair of DVDs. When a project or a set of files is finished copying to the backup hard drive, I also burn that project to a primary DVD and then make a backup DVD.


In 2007, Google wrote a rather long-winded report on the typical failure rate for hard disk drives (linked at bottom). In summary, the biggest killers of hard drives are heat and shock. Time usage can also certainly be a contributing factor, though Google’s results came from hard drives that “remain powered on for most of their life time.” Such drives, on average, had the biggest jump in failure rate between years 1 and 2. Additionally, it’s been noted in the tech world that traditional, magnetic hard drives (aka Hard Disk Drives or HDD) lose their magnetism, and thus their ability to store information, over time.


On the other hand, solid state hard drives (SSD) are quickly gaining favor because they have no moving parts that might fail, and they can retrieve data much quicker than magnetic drives. Browsing through various forums and articles, however, it seems there is consensus that long-term storage on a SSD is inadvisable. Such drives can begin losing information within a matter of months without frequent use.


The positive, though, is that the previously mentioned Google study proclaims the SSD failure rate to be lower than that of those HDDs with all the moving parts.


Drobo storage:

To help solve that problem of disk failure and subsequent data loss, some companies have developed technologies that copy data to multiple locations. Here in the office, we use the Drobo S hard drive storage system, to back up video that continually costs millions of dollars to produce. The unit is an external box capable of holding a total of five hard drives, for maximum storage space of 15 TB. The drives in the Drobo all work in unison, essentially making the whole unit one big-azz external hard drive. When you use the “dual disk redundancy” setting, though, the 15 TB storage then drops down to around 7 TB total. In that mode, the information on the drives is duplicated and spread around on all the drives in the Drobo unit. Our tech consultant purports that all the data would be safe even if two of the five drives in the Drobo were to crash simultaneously. Woah!


G-RAID Storage:

A lot of tech types in the entertainment business gravitate towards G-Technology’s G-RAID drives. They look sleek and cool and seem super durable. I worked on a TV show a couple years ago, wrangling and backing up all the footage shot everyday. Prepping for the show, the Producers came to me to find out what we should be storing all the footage on. I suggested G-RAIDs. “Nooo, noo, nooooooo! We can’t afford that,” they responded. So they bought a bunch of regular G-DRIVES instead. If you aren’t familiar with the conventions, I will briefly describe them in the short paragraph below. Otherwise, feel free to skip ahead.


– G-RAID drives are actually comprised of two hard drives, connected together inside an nice, aluminum, external hard drive housing. In the standard operating mode, when you connect the drive to the computer and drag files onto it, it copies the information to both drives inside the unit. If one of the drives were to become corrupt, your data should be retrievable from the mirrored, non-corrupt drive contained in the housing. As opposed to G-RAID drives, regular ol’ G-Drives are made by the same company and only have one drive inside the similar-looking housing. –


Backing up all the data for the aforementioned show onto the regular G-Drives ended up working great. I replicated the “dual disk redundancy” of a G-RAID by connecting the two G-Drives together and copying all the files to the main drive I designated, and then copying the same information to the backup drive.


Fast forwarding to the show I work on now, the Drobo unit was purchased as a replacement for those G-RAID drives that I had always previously thought were so indestructible…. it turns out they aren’t quite so reliable after all. Remember, I said before that the footage we back up costs millions of dollars to produce? Well, that means we also have to be able to access that footage at any time for flashback sequences and such. Over the years, we amassed eight G-RAIDS, and two of them ended up failing. Two of eight – that’s 25%! Our computer guy had to send them in for data retrieval, which meant they were out of service for a couple months. That’s right, when we plugged in drives to get the data we needed off them, two of the G-RAIDS failed. Done. Kaput. Stemming from actual experience, my preference these days is to backup files to two separate drives, rather than using a single “raided” enclosure that shares one circuit board.


Summary so Far:

G-RAID drives aren’t as safe as they might seem. If possible, backup your data to multiple hard drives and then to other storage media, like DVDs. The up-and-coming solid state hard drives have a limited life span, just as magnetic drives do, and it’s also worth noting that those SSD drives don’t hold information so well without having frequent use.  ( –> http://www.tomshardware.com/reviews/ssd-reliability-failure-rate,2923-9.html).

Obsolescence:

Aside from the question of which storage media to use for long-term backup, another consideration for the future is file format compatibility. Codecs change and many programs eventually lose backward-compatibility with previous versions of the software. Unless you plan on keeping old computers around, consider archiving your most important documents as basic files.


For DOCUMENTS, consider saving in Real Text File (RTF) or basic Text (TXT). These formats lack in the ability to save things like flashy banners, text boxes, and other things that make your document look more like a fancy webpage or magazine cover, but they will continue to save your basic text for the long-term.


For PHOTOS, JPEG files will probably be the best bet for the future. The majority of cameras these days save in that standard format, and JPEG is the predominant format to display images on the internet.


For MUSIC backup, consider MP3 files. The proliferation of that file format really came about with online file sharing in the late ’90s and the subsequent development of MP3 players that so many people take to the gym, hook up in the car, and pack on vacations. The MP3 format will likely continue to be around for a long time. For audio enthusiasts, WAV is another great format to use. It’s the format encoded on standard music CDs and provides a higher fidelity sound than that of the MP3 format. For many people, the difference in sound in negligible.


Off-site Data Storage:

Off-site data storage is an important consideration if, God forbid, disaster strikes your area. Here in California, a sizable earthquake could break down this infrastructure for a long while, and such destruction could destroy those hard drives and other storage media. Aside from earthquakes, some parts of the world have to contend with floods, tornadoes, hurricanes, wildfires, and tsunamis. Sometimes houses and apartment buildings burn down. These aren’t pleasant thoughts, but it’s true. For these reasons, it’s wise to keep a backup copy of your data off-site. When finished with my current project of backing up ALL my photos, music, and documents, I plan on keeping a small book of DVDs at a relative’s house, which is in a different town. You could also keep your backups with a friend, at the office, in a safe deposit box, etc. Also, when keeping such things in the possession of others, be sure to use your best judgment, based on what the content of your backups is. If some of the information is sensitive and potentially embarrassing, take the safe deposit approach or just leave it off your off-site backups.

Photo Prints:

For photos that are really important to you, print ’em out! My family has some tintype photos from the 1800s, portrait paintings from the early 1900s, and a collection of film negatives and slides. In other words, physical portraits can last an extremely long time, especially under proper care. For long-term storage, again, the best bet is to keep a copy at home and another off-site. Be sure your printer paper is acid-free, as well as any envelopes you may be storing them. If at all possible, keep the photos stored away so exposure to air is minimal, and don’t touch the photos if you don’t have to. That way you will keep oils from your skin off the photos.

Second (and Final) Summary

Ultimately, there is no single, bullet-proof data storage solution. Your best best to ward off data loss is to back up files to two or more storage mediums, which can include a combination of hard drives, DVDs, online/cloud storage, tape drive, and even CDs. Periodic backup is also crucial, since hard drives will inevitable fail and plastic storage media will degrade. Even online storage shouldn’t be used as a single source of data backup.

If possible, transfer your backup files to new media every 3-5 years to keep your storage devices fresh. Keep copies of your data off-site and, lastly, put your most important files in a physical medium (like photos) when applicable. Happy data preservation!



 


Comments

bottom of page