My Optical Media Digital Archiving Workflow (V2 02/24)

My onsite optical media archive (box 1!) based mostly on the M-Disc. Photo: Daniel Rosehill.

Should I Use M-Discs Or “regular” Blu Rays?Target:

Create archival copies of my ongoing personal data pool, including (but not limited to):

Creative outputs like:

• Originals of YouTube videos
• Photographs
• Podcast episodes
• Blogs

Also video specific stuff like:

• B-roll that may later be useful
• Raw shoot archives
• VFX, SFX outputs

Personal things like:

• Documents

Physical Medium:

I currently use a mixture of high-quality inorganic (HTL) Blu Rays (25GB-128GB per disc) as well as Verbatim M-Discs.

I currently source Blu-Rays from Amazon Japan.

I like and trust these products from Sony and Panasonic respectively:

For lower quality media, here’s a bulk pack of Verbatim BD-R:

Digital Archiving — A Basic Approach

Per the 3–2–1 rule, you’ll need to create two optical media archives at geographically independent sites.

This is a base requirement and many people will create additional duplicates for more important data.

My approach is based on physical periodic offsiting of physical media. More regular offsiting would be more ideal — ideally as soon after disc creation as possible.

I store my creative outputs on a Synology NAS using a simple file structure to attempt to capture as much of the data that I care about as possible:

Disc Authoring Process

Step 1 — Download Data

Download data from NAS/computer/intermediate/working storage.

Step 2 — Generate ISO

Create a burning image (.iso).

Step 3- Generate ECC or parity data

Use dvdisaster to generate error correction code (ECC). I use the RS03 methodology to store ECC both on the disc being created and on an independent file system.

Parity data can also be used for this purpose.

Step 4 — Burn 2 x copies

After generating the new image containing the ECC, author 2 x copies of the archival data.

One copy should be retained locally. Another should be stored offsite.

I write the disc identifier on the transparent inset of the discs and also write on the surface using a CD safe marker. Finally, I write the disc identifier on the jewel case.

Step 5 — Catalog media

Finally I catalog the media that I have created using a digital cataloging software. I use VVV primarily because I use Ubuntu Linux. For Windows users, WinCatalog adds some nice additional features:

Virtual Volumes View

GitHub - speed47/dvdisaster: A tool providing additional ECC protection for optical media…

Other Bells and Whistles You Can Add

• Calculate checksums and record them in your cataloging software.
• Disc encryption for security. This can be complemented by physical security measures such as storing the media in physically secured locations, including vaults and safes.

Common FAQs

• Why not just store ‘stuff’ on the cloud?

This approach is intended for people who want to create physical archives of their digital data. There are a number of reasons why some people feel uncomfortable committing important personal data to the cloud. The cloud is … simply somebody else’s computer.

• But … isn’t optical media dead?

Optical media’s popularity has drastically declined but it remains a relatively popular medium for data archiving thanks to the relatively longevity of the medium in cold/offline storage environments.

• Why not use LTO?

LTO is great for cheap archival storage. But it’s not actually a great medium for cold/offline archival. Tape can experience data rot in as little as 30 years. Many forms of optical media are rated for stability in excess of 100 years.

• But … hard drives are so much cheaper!

The advantage of optical media for archival use-cases is data permanence not cost savings. In cold storage environments, archival-grade optical media can be expected to outlast HDDs and SDDs by many decades.

• Is it time consuming?

Refining the workflow and writing about it has been far more time-consuming than the actual process of maintaining the archive (but this has been a labor of love and a source of periodic joy for me. I get a somewhat regular stream of emails from other optical media users around the world!)

I create about 2 x 25GB BD-Rs per month. I could reduce my frequency further by using bigger discs but … I quite enjoy the monthly ritual and creating physical products from my creative efforts. Burning those two discs requires about 10 minutes of hands-on effort.

• Don’t you need to store lots and lots of discs?

If you’re a photographer/videographer (etc) storing relatively modest quantities of personal data …. this doesn’t require as many discs or as much room as people seem to imagine.

It took me the bulk of 2 years to fill up half a small box…. and many of those discs were test burns. If you use the largest capacity media on the market currently (128GB BD-R) you can use even less space.

Imagine you’re archiving episodes of your podcast and each episode is about 100Mb. You can fit more than 1,200 episodes on a single 128GB BDXL. You get the idea.

• Should I Use M-Discs Or “regular” Blu Rays?

People have different opinions on this, often strident ones.

I have come around to the opinion that “regular” inorganic HTL Blu-Rays with MABL are perfectly good enough for archival use.

I would still focus on buying media from reputable manufacturers and avoiding ‘bargains’. Shop for quality.

I still use lots of M-Discs but no longer regard them as ‘essential’.

I do not believe that M-Disc are selling regular HTL media as BD-R Mdiscs (see: the “fake M-Disc charge”). However I do believe that they have done a woefully inadequate job at explaining how their BD-R M-Discs are superior to their regular product line. I would like to see head to head comparisons and transparency about the differences between the products. However I also think that it’s a somewhat moot point.

• What about using CDs or DVDs for data storage?

I don’t recommend using CD-Rs and DVD-R for data storage, including products marketed as “archival grade.”

The gold CDs look impressive (visually) but still contain organic dyes in the recording layer.

I would take a bet on an MABL BD-R disc over these products any day of the week.

• What burner should I buy for the M-Disc?

Assuming you’re looking at BD-R M-Disc and not the DVDs … just about any Blu Ray burner on the market should be able to read and write them. For BDXL .. it’s worth double-checking the spec and remembering that tri-layer (BD-TL) and quad-layer (BD-QL) are usually listed separately.

I’ve heard that Pioneer make good products. A stronger belief is that products with an independent AC-DC supply should be favored over those that can only draw bus power from USB connections. I would favor internal SATA drives in an enclosure over external slimline ones.

Some More Photos Of The (Growing!) Archive!