digital photo storage backup-- what are you doing?

greenspun.com : LUSENET : Large format photography : One Thread

Hey guys,

I've got 52 gigabytes of 4x5" drum scans on my Mac's hard drive, which represents eleven years of work; i couldn't even estimate the personal cost i've incurred to get the images, the scans, and do the Photoshop work for printing. Suffice it to say that hard drive is by FAR the most valuable possession in my world.

My backup system is to have two additional 120 GB external hard drives, each of which contains a mirror image of the first. Every week or so I do a backup, so in case any of the three drives fails, i have the whole 52 gigabytes on the other two.

But, all three drives are in the same place, so if there's a catastrophe such as an earthquake, all three could go at once. Plus, i've heard that no hard drive can be expected to last more than a few years, so this definitely is not an "archival" solution.

So, what are other people doing for archiving of large quantities of important image files? burning CD's is a possibility but very time consuming for 52 GB; i'm thinking about burning DVD's (it'd take about 12 of them to back up my whole library). have other people tried tape drives or other stuff? any suggestions would be appreciated,

~cj (Seattle)

-- chris jordan (cjordan@yarmuth.com), April 24, 2002

Answers

Hi Chris

I work with the 700MB CDs these days and from all important staff I have 2 CDs but all my CDs are in the same house but not at the same place.

But I feel its save!

-- Armin Seeholzer (armin.seeholzer@smile.ch), April 24, 2002.


currently I back-up on c/d but I realize that they are not really archival so I plan on moving all my digital info to the next new thing and the next new thing and so on and so on. My work doesn't involve much manipulation so I still consider my film to be my true archival storage medium.

-- doug (doug@ajenda.com), April 24, 2002.

I keep on separate external (and portable) 80gb firewire disk drive. Keep one in a separate place and periodically update. I think you got the general idea right. Life's too short for the DVD/CD solution.

-- Donald Brewster (dpbrewster@prodigy.net), April 24, 2002.

Check this out for alot of data....

http://www.archive.org/about/storage.html#Storage

fwiw, here's a link to the access project we're getting drawn into...in the working guidelines to this, you can find some more links.

http://www.ncecho.org/

Here's the working site for the Library of Congress's American Memory site, there are tons of links to tech info here...

http://memory.loc.gov/ammem/ftpfiles.html

Not much of an answer I guess...we do use digital files in our access records, I hesitate to say just how they're backed up...not my field.But you'll find alot of info on large amounts of data & preservation of digital files & images in these links...just about every archive in this country is involved in a digitization project now, and agencies generate electronic records that have to be retained for set periods of time....maybe you'll find some good tips in these links beyond what you may get as a typical answer....good luck--p.s....you probably need to set up some off-site storage though. Since I'm talking records & such (note: I am not an archivist):

Opinions expressed in this message may not represent the policy of my agency.

-- dk thompson (kthompson@moh.dcr.state.nc.us), April 24, 2002.


Hi Chris...

I try them all.. and I agree that's CD is time consuming (but it's safe)... backup on another hard drive (external) is one solution... recently I went for DLT tape... The first time you set it up it's will be long...that's what they do in corporate envirenment... in my opinion (for a small scale operation).. just bite the bullet, get a spindle of good quality 700Mb CD and do it fréquently... DVD is OK too....

-- dan n. (dan@egmail.com), April 24, 2002.



Iomega now makes a 20GB portable drive called Peerless. It accepts both 10GB and 20GB disks. You just buy more disks as your need grows. You have a choice between a USB or FireWire connection.

The Iomega drives are much faster than burning CDs/DVDs. The 10/20GB disks are also pretty compact so you could easily store them somewhere offsite.

This is not a permanent solution. Then again, I'm not sure there is a permanent "digital" solution yet. CDs/DVDs have a limited lifetime and obsolesence is always a problem with any digital technology.

http://www.iomega.com

-- E Rothman (erothman@hotmail.com), April 24, 2002.


I'd second the Iomega suggestion ... an innovative company with high quality products, plus you'd need only three or four of the portable drives.

The CD/DVD solution is inexpensive, and probably not too time consuming once you've brought things up to date. I'm using CD's now, but I'll soon be looking at the portable drive solution.

-- Michael Mahoney (mike.mahoney@nf.sympatico.ca), April 24, 2002.


I forgot to add this link from Luminous Landscape

http://luminous-landscape.com/hd-back.htm

-- Michael Mahoney (mike.mahoney@nf.sympatico.ca), April 24, 2002.


I think what you're doing now is a good solution if you just add off-site storage for one copy. That would seem to cover all the bases. I've been using CDs for the past 6-7 years and I've had a couple of failures out of the 300 or so that I've written data on. Thats not a bad track record, but it is a problem if the photo or file you want is on the one that is corrupted! None of our digital storage media is permanent, your data will have to be migrated to newer media over the years to come, probably many times in your lifetime. Given the relatively low prices for hard drive storage and for your time spent archiving you are mostly doing the right thing now.

-- Henry Ambrose (henry@henryambrose.com), April 24, 2002.

Chris,

I am a bit paranoid when it comes to back up, and use a quadruple back-up. Most of my backup effort has do with my writings, so these are pretty small files.

1. I keep and identical copy of my hard-drive on an external drive (there are quite a few programs which will scan all folders and update (I use Synchronize! for mac, which works fine)

2. i back up everything monthly onto CD (you would need DVD due to size) using Retrospect. I keep these in a safe-deposit box at a bank.

3. I have two cheap 80gb Firewire drives, which with Synchronize! I keep all media files on. One stays at work, another is parked at home.

4. I use an small FTP internet back-up for current work (80mb max)

If you can borrow a friends tape drive, this may be the best bet. The good ones are very expensive, and hopefully you would never need to use it. I wouldn't bother with DVD, it would simply take too long. Perhaps your best bet is to buy two or three 80gb firewire drives (NOT Maxstor) and keep them parked in different places, work, home, friends place.

-- jason (sanford@temple.edu), April 24, 2002.



A caution about the Iomega solutions:

Iomega makes nice stuff, but are often (always?) the only company producing it. In my opinion this makes it completely useless as a backup medium. Just like any other non-standard prorietary solution.

Even if you go for standard solutions you might get into trouble: Raise hands everyone who can still read 5.25" floppy disks :)

Backing up to hard disks, external firewire or internal disks, is a relatively cheap and quick solution. You need be careful though when you upgrade your computer or operating system. You may not be able to read them afterwards.

The same goes for tapes of course: My unix machine can't read my Retrospect MacOS 9 tapes, and I have no idea if tapes made on Windows 2000 can be read on Windows 2005 when it comes out.

CDs are relatively safe because the ISO 9660 format is pretty standard across different platforms. They are a bit slow and cumbersome though. And *a lot* of work when finally the day comes that CDs are obsolete and your data needs to be moved to a different medium.

DVDs I am not so sure of. Is it finally clear which of the many formats of DVD-R, DVD-RW, etc will become *THE* standard? i haven't paid attention, so I'm not sure.

I try to steer clear from proprietary solutions like Iomega's, and usually backup to CD and/or hard disks attached to different computers in different locations. (isn't the internet great?)

-- Alex Le Heux (alexlh@funk.org), April 25, 2002.


Keep doing the hard drive backups just as you're doing now. Burn a cd (a pair if it makes you feel better), make sure the cd is a gold or platinum composition, and have a relative keep them (or store them in a safe deposit box).

-- Wayne DeWitt (wdewitt@snip.net), April 25, 2002.

I used to be sanguine about my ability to back up everything with minimal effort, but recent technology changes in the way hard disks work mean that they are growing in capacity much faster than Moore's law, and removable media cannot keep up.

I still use DAT tape for my regular incremental backups, but things are reaching the point it will be simpler to just buy a few 80 Gb firewire drives and pretend that they're tapes. Tape drives are so expensive, and hard disks so cheap, that tapes really only win if you want to make lots of copies of your data, or if you want to send someone a disposable copy. For personal backup neither is really necessary, although popping a tape in the post to your parents/children/friends for off-site safety is a useful option.

I wouldn't recommend any magnetic storage for long term archiving. I personally use DVD-RAM for my archive of last resort. 5.2 Gb on a two-sided disk is convenient enough for a one or two-time copy, and the technology is such that I can leave a disk on the shelf for years without worrying about it deteriorating. Pure magnetic storage, or CDRs are not reliable enough for me. I used to use MO for my write-and-forget backups, but these days that's a Fujitsu-only solution and the drives are hard to find. DVD formats are not standardised yet, but I needed something *now* and DVD-RAM can be read by most of the other format drives.

I also burn CDRs for distribution and working copies, since burners and drives are ubiquitous now. But I've had too many CDRs spontaneously turn unreadable (even the 'good' ones) to trust them for my main archive.

So I would say, if you want something to supplement your external drives *now*, get a DVD format based on phase-change media. If you can wait a while, see which format becomes the norm and get that, but if not, bite the bullet and get something before it's too late. It may seem expensive, but compared to re-scanning all your negs and slides it's a drop in the bucket.

-- Struan Gray (struan.gray@sljus.lu.se), April 25, 2002.


Chris -

Forget any type of magnetic storage for archiving purposes - that includes DLT, any Iomega drives - period. One small jolt of static electricity can destroy it all in the blink of an eye.

Second - make at least 2 copies of everything - and store one copy with someone you truse (after 9/11 - even bank vaults aren't safe).

Your only real choice, then, is optical - CD, DVD, and MO (magneto- optical). In the MO drives, the M (magneto) refers to the encoding of the data - not the storage. So, they're safe.

The only issue will be how long do you want to spend doing the actual copying. Personally, I might go with a combination of MO and DVD - and keep a separate drive for each one in event of failure. The reason? CD size is limited to 650-700 MB (some can double the capacity with special hardware and software) - so it'll require a lot to store it all.

Make sure you make some sort of gallery/contact sheet for each disc you burn - even if it's only an html browser disc. If you're on a Mac, there are numerous options available - same as on windows - but the html version may be the only one to withstand the test of time.

Ciao.

-- Alan Agardi (alanagardi02@sprintmail.com), April 25, 2002.


Greetings,

IRT the comments by Alan, I'm not sure his statement about MO drives are entirely accurate. I haven't kept up with the technology recently, but some years ago a MO drive was one that used optical sensors to accurately position the read/write head and magnetic technology to record the data. IMHO, any device that uses magnetism as its means to store data is far from archival.

IRT to optical storage... Do we really have archival data on CD's or DVD's? They're made from plastic. Is the plastic archival, or will it break down over time?

Regards, Pete

-- Pete Caluori (pcaluori@hotmail.com), April 25, 2002.



Some of my CD-R's are failing after less than a year, probably a bad brand. Do not trust any CD brand blindly. Find two tested brands, make one copy on each brand. /Åke

-- Åke Vinberg (ake@vinberg.nu), April 25, 2002.

You all can probably find more info on lifespans of media on CoOL--conservation online--if you look under electronic media, etc.

http://palimpsest.stanford.edu/

Back in '97, our museum sent us up to the Smithsonian to a conference on managing photo collections & digital projects....the message was pretty much, "don't do it....", but anyways, we got alot of good info out of it, and I have a whole binder chock full of aging test papers on magnetic media, cd's etc. I don't know if this link is in the CoOL documents, but here's one from a study on magnetic tapes, optical discs etc. From my notes at the time, the recomendation for the lifespan of magnetic tape & video was 5-25 years in average conditions...the higher numbers being for an archive/controlled condtion. Apparently the plastic binder of the tape over time eventually delaminates...they called it "shedding" or something like that....I remember them telling us stories of salvaging tapes having "sticky tape syndrome" by heating them up in ovens to get the tape to play one last time....they then copied it over onto new media & trashed the original...

http://www.nta.org/Publications/TechnicalReports/TechnologyAssessments /TechAssessmentFinalReport1994/11_arch_stability.html

If you don't want to dig through all that stuff, this page has some good info on CD-R longevity & testing:

http://www.mscience.com/faq.html

last link...just for reading,

http://www.longnow.org/10klibrary/TimeBitsDisc/

Just about all these reports sum up that the worry is not in how long the media will last, but in obsolesence...I know one of the large institutions up there had just stopped a major project back in 96 or so..the year before we were there. In fact my boss had toured this project a couple of years earlier... it had become obsolete in about 4 years, despite alot of backup hardware & migrating across updated platforms....they had been using a large laser disc media that became obsolete very fast....the digital images we create, well we view them as disposable almost, but they live on a hard drive. The actual longterm file is film. I know from a records point of view, the standard is still microfilm...in fact some computer files are backed up on computer output microfilm....polysulfide toned microfilm is rated at 500 years....and that's in a typical archive setting...i.e. not in a cold storage vault...

If you live near a cold storage rental facility? Instead of a safe deposit box, a bona-fide cold vault, like Hollywood Vaults or someplace similar would probably be tailored to storage better...of course, as an individual that might be sorta expensive....

Opinions expressed in this message may not represent the policy of my agency

-- DK thompson (kthompson@moh.dcr.state.nc.us), April 25, 2002.


Have you considered sending your files to film using a film recorder? You could output to transparency or negative film as the image requires and then you'd have a compact, fairly long lasting record of your photos to store off-site. It might get expensive, but this could be another method to conserve your work. Another idea might be to print all of your collection -now and as you make new pictures- and store those prints for an additional archive.

I still think that proper upkeep and migration to new media/formats is the simple/good/cheap answer to keeping your digital files. A new 500GB hard drive may only cost $100.00 three years from now and it surely would be easy to transfer your files to a new drive (or several drives) every few years. This would never have been a possibility when a 1GB drive cost $800.00 - but that has sure changed!

Another wonder. Will the current format that you've saved the files in be viable for printing 10 years from now? 20 years from now? Will there be a machine that uses those puny little 300MB TIFFs to make great prints?

hmmmmmmm… there's gonna be some work involved in keeping your archive current, useable and safe.

-- Henry Ambrose (henry@henryambrose.com), April 26, 2002.


This has been a concern for me too. I have used CDs to store my images for many years. Then the DVD-Ram came up and I have bought successively two generations of drives, but the media prices never came down to what I consider a reasonable price for storage, and the drives are desperately slow.

I am now considering DVD-R. I made a first approach to buy a Pioneer in january, but saw that a new version was planned for april, so waited. Now the DVD-R A04 is available, but there is a new 4X write (current is 2X) version planned for July... so I will wait. But I just heard of the new Blue laser standard and maybe it should be worth the wait... ;-) no I think the DVD-R A05 will be a good choice for some time. Just think: writing 4,7 GB in 12 min. on a $7 media!

The obsolescence of this media should not be imminent for this is a very popular standard, just like VHS. The media are insensitive to magnetic fields and are said to last 100 years in good storage conditions, which probably exceeds by far the readers availability anyway. I have now installed a 160 GB drive that I use for backup, and just pray that no incident will occur before I am able to transfer all that stuff to DVD's!

-- Paul Schilliger (pschilliger@smile.ch), April 29, 2002.


Moderation questions? read the FAQ