$.02 on topic. I was team "long term" storage for while. Since early Apple II days I always made at least one disk backup. Moving forward, it was tapes, then ZIP Disks, CDs, and finally DVDs. Early on I was more than happy leaving data for long periods of time on old media. I always had concerns however with old disk drives and of course the need for some platform to access them. The equipment concerns started to weigh in as floppy was phased out from motherboards, then later 40 pin IDE. I never let data sit too long. I would move to newer devices as budget and time permitted. Later I had some issues with CDs. Nothing major, but it got me thinking about a 'real' long term solution. I also found it uncomfortable relying on old tech as the years progressed. About 2010 I went local attached storage and moved all data to it - live and backup. RAID 1 is mirrored HDDs. It's 2 HHDs which yields only half the storage space due to mirroring, but more is much more redundant due to the unlikelihood both drives would fail at once. And this means the data is local and accessible. Of course I have a good motherboard and use a good line filter, power strip, and UPS. Last thing I need is a power spike taking out my computer or risking both HDDs. For 'backups' I can attach an external device and copy to it, be it HDD or Flash. I can also just remove a HDD, insert new HDD, and the RAID Array auto rebuilds to the new drive. And the removed drive is still accessible outside the RAID when connected by itself. So this medium/devices can be kept offsite for a second level of protection. My third level is cloud based backup which also offers search and versioning. CrashPlan is $10 a month and offers unlimited space. I currently have 4TBs there. All you need is basic cable modem access and it's fine. I have 2mbit upload for example. A decent used RAID Controller (ASR-6805T) is like $40. No need for new. 4TB HDDs can be had used for $45. So for under $150 (assuming you have PCIe Slot open and available channels) you can add 4TB of RAID 1 to your system. And for $120 a year, backup online too. Or just take that money and buy to more HDDs and swap out of the Array every month and keep the 'backup' drives in your storage space. Barring a major natural disaster you'll always be fine. Most years my backup costs are just that of CrashPlan - under $.40 a day. Semi-related to topic - Each year I like to crate a new folder, 2020 in this case, and resave projects being worked on and new things there. So I have access to past years, and know that my projects/data was current to that point. If/when I ever suspect an issue I can quickly and simply go back and open a file to check. When keeping large amounts of data however it's imperative to be well organized. Personal, Business, sub folders for Vendors, Media, etc. I'm a backup guy and don't mind duplicating things from year to year, like Media folders from 2019 to 2020, or even go quarterly if I feel needed. It's a lot of work compared to just a few folders. But I can find things from the past very simply when needed. You may be happy with just one media folder with subfolders of years for example. But all that's a different discussion. TL;DR - Deep storage is tricky at best. Local attached RAID might do exactly what you want and cheaply too with much less risk in the long term. Henry S. Courbis Office Toll Free: (800) REACTIVE (732-2848) Office/Mobile Direct: (856) 779-1900 www.ReActiveMicro.com <http://www.ReactiveMicro.com> - Sales, Support, and News, Our Headquarters on the Internet ReActiveMicro.com/wiki - Support, Software, Manuals, and History. Create your own page today! Facebook.com/reactivemicrousa - Our Social Media Outlet and Support On Thu, Jan 2, 2020 at 6:36 PM Michael Brutman via vcf-midatlantic < vcf-midatlantic@lists.vcfed.org> wrote:
I used to write enterprise class hard drive firmware for HGST before they got bought by Western Digital. Our code was also shard with the consumer SATA drives.
Hard drives have *very* robust error detection and correction. They will generally go to extraordinary lengths to recover and relocate data on marginal sectors. The ECC algorithms implemented in the hardware are very robust. Drives do need to be powered up once in a while to do background scanning for weak sectors, so just writing data and putting the drive on a shelf is not the best idea.
As with many other things, the best approach is defense in depth (layers of defense). Store several copies of the data, and not on the same brand of media or hard drive model. Verify the data using a hash that was created at the time the data was written to detect corruption, and use another copy (that hopefully isn't also corrupted) to mitigate the problem. Filesystems like ZFS are awesome.
All of the big cloud data storage providers have your data stored in multiple places; two copies isn't considered safe enough. And of course they spread them out geographically to mitigate damage from external factors.
Mike