Best way to archive data? (External HD vs Optical media)
 

I've been rethinking how I archive my data and would like some input. I've got a lot of music, movies, tv shows, etc that I've downloaded, and, rather than delete them after watching, I prefer to archive the files so that I can return to them later, perhaps with a friend, or if I just feel like watching it again. I've been burning these files onto DVD for many years and now have a few hundred discs worth of data. I don't mind having the data on DVDs since I can just put them in large binders for storage, but it's not the most convenient. So, I'm debating whether or not to ditch the DVDs and start using external hard drives for archived data instead. Here's what I've come up with so far:

DVD Media:

• About 6.4 cents per gigabyte
• Takes up more space
• Less convenient: Harder to search for the disc I want; takes time to burn each disc
• Subject to potential disc rot
• Permanent: if I decide I no longer want something, I can't reuse the disc for something else

External HD:

• More expensive: about 10 cents per gigabyte
• A bad external HD means losing a TB of data, but a bad DVD means losing just 4.7GB
• Takes up less space: one 1TB drive fits over 200 discs
• More convenient: Multiple series can fit in one place; easier to store
• Not permanent: If I want, I can just delete things I don't want, and I can reorganize at will

So, anything I'm missing? What are your thoughts? The external hard drive option is more expensive, but competitively priced, and I'm thinking it may be worth it for the convenience. I'm a little concerned about longevity though. Disc rot is fairly uncommon - I have yet to encounter it - and there are no moving parts to be concerned about with DVD media. The idea of an external hard drive going bad and causing me to lose a TB worth of data is pretty horrible. I'd rather go with the less convenient option if data integrity will be better.

So, fellow downloaders, what do you use to store your TB's worth of data? Do you think I should continue burning DVDs, or should I switch to buying a new external HD every so often?

Bonus question: If I go the external hard drive route, what's the difference between the Western Digital Element and Western Digital My Book Essential? Do you think one is significantly better than the other? Why? If nothing else, it seems the Element would be easier to stack and store. Maybe you have an entirely different external hard drive recommendation?

I don't know Western Digital HD externals, but I'm saving my $$$ to afford one of these:

HP MediaSmart Server

I think the choice is clear. Purchase as many external HDs as needed and double it. Instant access to your files without the annoyance of flipping through DVDs. More expensive but well worth not having to put up with the DVD fiasco. Plus - optical media eventually goes bad and won't read anymore. Do you really want to have to reburn hundreds of DVDs every 5 years? Time to make the migration to hard drive storage.

I use a 2nd HD personally, I keep it unplugged (power and SATA) until I'm doing a backup. Power down, open case, plug in both connections, power up, backup, power down, unplug both connections.

Crucial data I also copy over to my wife's computer into a folder just for that. Same with her system, 2nd HD and crucial data is mirrored onto my system.
As to the Western Digital HD externals as far as I can tell one is constant on and the other powers on and off with your computer otherwise the specs seem to identical (other then physical size that is).

I've seen disk rot (I didn't know that was what it was called) on CDs before, but those were ones I burned with my 1x SCSI burner I got from a friend after he upgraded to his ultra fast 4x back about 13 years ago. I still have some CDs I've burned from that time period that work, most don't. I think it has something to do with the quality of the CD/DVD, I used cheap CDs back in the day.

I wonder about external hard drives ever sense I had a girl come in with her external HD that had died on her. She was a senior in college that had kept all of her final projects on this HD (some of which where huge flash projects) and when she plugged it into the PC, it smoked a bit, then died. I ripped it out of the casing and tried plugging it directly into a pc. Nothing. Nothing under linux either. I watched a bit of smoke come off of the circuit board as power ran through it. I am kind of leary of external HDs after that.

Now correct me if I'm wrong, but arn't most external HDs just casing around a normal HD or have they advanced more than that? Couldn't you just buy a few 1TB HDs and back up everything then store the drives in a safe place? Best of both worlds?

I like Lucifer's idea the best. If I was to want to back up TBs of data (which I'm getting to that stage now), I would buy (or trade, haggle, scrape together) a motherboard with RAID capability and build me a server. Buy a few TB hds and make RAID array (which RAID type you want to use is up to you and what the MB is capable of) then use it as a media server. Surge protection + UPS and you have instant access to all of your data. If you run out of room, add more HDs. Getto, but it would work.

I have a IDE/SATA USB plug
Newegg.com - VANTEC CB-ISATAU2 SATA/IDE to USB 2.0 Adapter - Adapters & Gender Changers

I use this because I keep all the drives I have owned save a handful that I didn't think of keeping (back in the 386/486 days they were only 120Mb tops)

All the other drives I have access via this cable system if needed. Drives are cheap. I make multiple backups. I also have web space that I FTP backup to.

I use Syncback to handle it. It has come in handy for clients, as I can remotely restore things for them.

2BrightSparks | Downloads

as I get more SATA drives, I'm inclined to get one of these docks.
http://www.newegg.com/Product/Produc...82E16817707175

Don't forget another major reason not to use removable media for archiving: obsolescence.

With all the progress in removable media technology, current mediums are no longer cutting-edge after a very short time. Remember floppy discs? 8", 5.25", 3.5"... then ZIP drives were the in thing, and now you barely see those. CD-Rs and CD-RWs became the in thing, and now those aren't used for data much any more (still for audio use obviously, and perhaps for people with older MP3-CD stereos etc). DVD-Rs and DVD-RWs are standard now (although the whole multiple-layer/side/+/- compatibility thing still confuses me), but they will soon be replaced by higher capacity mediums. Blu-ray looks like the way it is going to go, but who knows how long that will last.

At least with hard drives, the interfaces remain standard long enough that you could easily and quickly transfer data to a newer hard drive as the storage cost reduces. Transferring 1TB of hard drive data could be done in a matter of hours... but the equivalent time it would take to copy the same amount of data off DVDs would be ridiculous, not to mention the amount of space 100-200 DVDs would take up.

I have accumulated a pile of hard drives as I have upgraded my computers... I think I have 10-15 hard drives sitting around (not actually in computers). I usually buy a hard drive with the lowest cost-per-megabyte about once a year, and copy and organise data over as appropriate over a weekend. Data I kept from 10 years ago is still occasionally useful to me, and worth the trouble and cost of keeping it.

By the way cyn, that USB plug is GENIUS.

I have two of those plugs at my work place, I can not tell you how much I love those things. You can even buy a little adapter for them to use with old IDE laptop HDs. For anyone that has old HDs sitting around, that little thing is great.

I had this exact same query a few months back, and it brough great insight to me about the benefits and disadvantages of "physical" storage versus digital backup.

To keep it short, external HDs were overwhelmingly recommended over writable DVDs.

Whatever method is chosen, keep in mind you need to keep it in a different location than the source. HD crashes are one thing, fire or flood are another.

I am by no means any kind of expert here but I had a thought while reading this string. What about SD cards or flash media??

I have no idea what the longevity of these are for storing data but a 32gig flash card takes up a lot less space than 7 DVD's, they are reusable and would allow for a reasonable amount of common media on a single card.

Of course ignore this if they aren't considered a long term storage medium, just thought you might be over looking an option.

The question is really one of how important that data is to you.

Regardless, the per MB price isn't sufficiently higher to justify removable media over an external hard drive. If you're really that concerned about crashes, just add redundancy -- get two.

Flash media is not reliable enough to be a viable option for data backup.

get a couple drives and set up a raid ..

mirror that bitch.

---------- Post added at 01:39 PM ---------- Previous post was at 01:38 PM ----------

oh.. these are pretty awesome too..

used one for some media not too long ago

CalDigit HDOne

more than you want to pay I'm sure. I'm sure you can do this a lot cheaper as well..

RAID arrarys are more about uptime than data security. It's not necessary. One drive in a USB enclosure will do, plus a second in another box (elsewhere on the local network, offsite, or even living in a closet) if you're paranoid of if the data you're backing up is very important.

I'd go with the HD, either internal or external. DVD's are a pain in the ass. They take up space, get unorganized, damaged, etc.

I used to copy my Netflix movies to DVDs. Now I rip the movie to an mkv file. It's so much nicer.

Alright, I was leaning pretty heavily toward switching to hard drives instead of DVDs, but I needed something to push me over the edge. This thread has succeeded!That's just awesome. I'll have to get one.Not really an option. I'm talking a few TB of data here.Yes, you understand exactly where I'm coming from then.That's the only difference I could see, and maybe I'm missing something but I don't see what the point is. If I'm not using it, it's not plugged in, and therefore not on. Do I really need a separate power button?

I have thousands of photos I've taken over the past few years. I back them up to archive quality DVD such as these. Gold DVD. I write the DVD and store it in a jewel case in cardboard boxes under my desk at home. I don't consider any form of a hard drive to be reliable long term storage since the hard drive itself is a relatively fragile piece of electro-mechanical equipment that is sensitive to being dropped, misplugged, etc. Also, I'm not so sure about the long time reliability of the mechanical or electronic components. Electrolytic capacitors, for instance, used to leak and corrode. Maybe that's changed.

CDROM and DVD, if properly cared for and quality media are, as far as I am concerned, more reliable media.

As far as technology shifts, I bought my first PC in 1992, which had IDE drives. More recent drives are either SATA or USB, with IDE being less popular. I got my first CDROM drive around 1994, and CDROMs had been around for a few years before that. Current DVD drives can still read CDROMS, so I think the risk of technology change rendering DVDs (and CDROMs) useless is small. There's going to be a lot of angry people if CDs and DVDs are dead in the next 5-10 years.

A couple notes.

I realize I should be keeping my media somewhere else than home, but I don't have alternate storage space.

Also, do not rely on commodity grade CDROM or maybe DVD for long lifetimes. I have a fairly large number of audio CDs that I burned 10-12 years ago that have been gradually failing over the last 3-4 years. Some are still good. I thought I got a deal one time on some cheap commodity media. That started going bad within a year.

Maybe the external HD is the best option. I was previously thinking up building some setup that could store multiple HDs after I run out of space but in the end it is going to cost a lot more than just buying external HDs and it will also was electricity. External is also safer for your data. You can keep it in a fire/flood proof case.

Does your motherboard have an eSATA port? eSATA is a lot faster than USB 2.0.

It does, but it's in the back which is frustrating to get to. After dealing with burning and reading DVDs, I'm sure I can manage with USB 2.0 ;)

There is also the consideration of magnetic damage, in addition to flood / fire. Mission-critical data should be stored in a safe, with a backup in a different location. If you don't have a storage space, perhaps at the house of a friend you trust or a family member.

While this is in general good practice, I doubt your True Blood torrents are really that vital. Do it if you want to, but I'm inclined to call a simple external back up good enough for this application. No need to make this too complicated.

Yeah, while I care about data integrity, it's more because of the time involved in acquiring the data and the difficulty of replacing it should it no longer be readily available later on down the road. Actual personal data that needs serious backing up? That probably amounts to 100GB at most.

I use my friends as a fail-safe backup. Multiple people have full mirrors of my media drives (total content: about 1.5tb), and they get updated every couple months with whatever my 'new' content is, on the condition they aren't allowed to delete significant portions without telling me. New stuff is backed up on the internal drive of the torrenting computer, as I like to seed for a few weeks as a bare minimum, and isn't deleted until I get a chance to copy it to one or two of my 'backups'.

If my in-house external drive ever takes a crap, I have 100% data recovery, since either my friends have a copy, or it hasn't been deleted from my 'in box' yet, so my maximum exposure is if my torrent boxes' hard drive dies, at which point I'd be out a couple weeks work at most.

Actually, depending on what you do and what hobbies etc you do, you CAN have lots of personal data. As a musician, I do music production, photography (I can easily fill up an 8gb card if I'm away on a photo-taking holiday for a few days), and have videos and recordings of rehearsals and gigs. I keep them all and still refer to and use them... I probably have 3-4tb of data from the last 5-6 years. Filmmakers probably have many times that!

Now obviously some people might argue about whether my data needs serious backing up or not, but the same question could be asked of anyone. :)

I should have clarified: I have maybe 100GB at most of personal data. I can definitely see how others might have more though. Didn't mean to come across as if I were generalizing.

Buy some of the large cheap internals and get a single SATA to USB adaptor. That should save you some money.

heh.. we go through about 36 TB's a month at the office.. constantly logged and rotated out of main storage. So yeah.. video advertising/television/film does take up a bit more than normal. ;)