Monday, 7 September 2015

Archiving your DAW sessions (media)

It's Dave Grohl's fault! Again! ;-)  I was watching an episode of 'Sonic Highways' the other day, and loved the way he and a producer went through reels of tapes of old recordings. You also see it in the Beatles Anthology, where George Martin sits and goes through old reels of magnetic tape with Paul, Ringo and George. Combine this with the question ....

"How and where should I actually save my old projects?"

I'm not talking 'system backup' ... or even your daily 'data backup'. I'm talking about being able to to to a shelve, and pull out a session from 5 years ago. As to what format, which tracks etc .... that is an entirely different topic of its own, which I may go into later. Let's just concentrate on the 'where' for now.

As of 2015, we are lucky in the sense that we have many options. And it doesn't have to cost much either. The main problem is though "Will this technology where I'm saving my sessions to, still be accessible in 10-15 years?" We all know how quickly technology changes, and most of the people reading this can still remember floppy disks, 'stiffys', SCSI drives, Mini-Disc, ZIP Drives, cassettes, 8-Tracks, all different sizes of vinyl records .... running at 78, 45, 33 speeds, magnetic tape reels, videos tapes. Video2000, Betamax, VHS, .... CD-R's ?  Which of these are still relevant today? Right! Vinyl isn't going under without a fight ..... but seriously?

A valid question may be "Why do you want to keep all that old crap anyway?" The chances are good you'll never access them again, but what if you do? Do you REALLY want to throw it all away just yet? Probably not. So, let's see what we have as of 2015. This is from the standpoint of a private, hobby musician with maybe a small home studio. Big businesses and professional recording studios have way more cash to look for a solution, but let's keep it ..... on 'our' level ;-)

Internal Hard Disc Drive
That's right. Many of us probably have more than one physical (don't be fooled by partitions!) hard drives in our systems. We could just save our projects to nice little folders there, can#t we? Fast, easy, always accessible. Always? Ehhh .... no. The MAJOR problem and utter fact is that hard drives WILL die! Yeap, maybe not today, maybe not tomorrow ..... but probably exactly one day after you've forgotten about it! And it's a royal PAIN if you're not 'prepared'. So, no .... no long term solution.

External 3,5" Hard Drive (USB)
One of those fancy cases, connecting a hard drive to the PC via USB. Needs power. Well, it's the same thing in blue really. OK, so you may not have it running ALL the time and you're thinking it'll last longer then, right? Wrong! The crazy thing is, if a Hard Drive is NOT running for a long time, the insides get all sticky and dried out. Chances are that such a drive will even die on you before his 'internal' cousin. No good either

Solid State Drives (SSD) [SATA]
Now we're talking! No moving parts, quiet, fast due to it's cable connection. Still in their early stages and the capacity isn't too much. These drives are pricey if you consider buck/gb. And they are mainly 'internal'. So you're not gonna open your PC all the time and swap between 3-4 of these really.
Good for working PC drives ... and maybe even backups!!, but archiving our DAW sessions for later access? Meh ....

Magnetic Tape (DAT, AIT, DLT)
Wait! Didn't we say 2015? Oh, Magnetic tape is still around. Forget the little DAT tapes, there are some really modern, fancy and high capacity tape solutions out there. AIT, DLT ... look it up! The problem? Cost! And anyone's guess if you're going to hook up these things to your PC in 10 years time. So .... nope!

CD-R, DVD, Blu-ray
Maybe the standard CD-R with the normal 700mb capacity isn't doable anymore, but backing up sessions on a DVD-R is what I personally did for years now. You get a 500 spindle for next to no cost. The quality certainly improved from way back in the mid 90s ...... so the chances that you'll be able to read them in 10 years is pretty good, I think. The MAIN concern out there is that we may not HAVE the standard CD/DVD drive in our PC's anymore in 10 years, but I don't really see that to be such a short term problem. Besides, my idea is to get a external CD/DVD/Blu-ray drive that hooks up via USB. This should be usable for quite a while still.
The reason I still like this option is because you can even stack all those DVD's on top of each other in that spindle holder again if you're not too rough. They don't scratch THAT easily, and what space do you need to store THAT? You could of course, also use these zipper cases which holds a could of disks.
A DVD lets you store 4,5GB worth of data. That should be enough for a few tracks. Even if you need 3-4 disks to cover a whole EP or Album project ...... cheap!
Blu-ray's are even cooler! up to 25GB (or 50gb with you get the double layer ones). Blanks cost a bit more, but still cheapo. I don't know how long they are gonna last as a 'avalable media' .... but if you have a USB drive ... access is no problem. So ... DVD or Blu Ray ..... possible!

2,5" External USB Drive
These little wonders are a real blessing. Small, cheap, lots of space, easily connected via USB and quite durable. If you choose the ones with a simple mini-USB connector, you don't even need a cable for each. I already have 4-5 of these flat, little space wonders laying next to each other. With USB3 the transfer is a breeze too, but even USB2 .... no real issue.
The cost is going down as well, so you could very easily get 3-4 smaller ones. After all, we don't need 1-2TB every time.
My initial idea/wish was to have a 'changable media' on a shelve per Project ..... but maybe I should ask myself if that is really sensible. After all ..... didn't we complain about just that the other day? No more dust collectors than absolutely necessary!!
So is it a contender? Definitely.

Cloud (Drive, OneDrive, DropBox, ...)
For the sake of covering all popular 'space' options these days, we have to consider storing data in a cloud. No matter if it's Google's Drive, Microsoft's OneDrive (formerly SkyDrive), DropBox or any other commercial variation thereof.
The times are coming where access and speed to a cloud is the same as to some local drive. And you'll have the tools on your system that automatically sync any data you need.
I'm really in two minds about cloud storage. Sure it's 'easy' and comfortable. And I guess most of these place have huge Data Centres with high-end, in-time backups. I make extensive use of DropBox for our band, exchanging audio files, tracks and even whole sessions. Do I store personal stuff on a cloud? No. Are there any guarantees that your whole cloud-archive might not just be hacked tomorrow? No. You make up your own minds, but for me personally ..... I will use cloud storage for temporary, unimportant, non-personal exchange. It's great for that. For a 10-year archive solution? Hmmmm..... maybe not

Conclusion ....
Well, thats quite a few options we have. As I have mentioned, businesses and pro studios may spend more money on high-end gear. They also work on a 3-2-1 storage principle. 2 copies, 2 different technologies, 1 copy off-site. But for us 'mere mortals' .... that might be overkill. Which leads us to the REAL answer to our question. The word is REDUNDANCY !!!

Instead of debating whether we should save our session on DVD OR a little external drive, it should be 'AND'. It seems the best solution here is to either use a "DVD/Blu-ray and 2,5" USB drive" combination, or just using "TWO separate 2,5" USB drives" solution.

Lastly, one needs to add that next to 'redundancy' ..... there also is 'migration' to think of. So, as SOON as you see that one of your options isn't accessible any more, get a new one QUICK, and migrate your stuff on there. Any media can die on you tomorrow, but the chance of two separate ones dying on you on the same day is fairly small :-)

There you go. I'm still not sure if I'm going to get myself a 'Multi-Drive' or just go and get another 2,5" drive ...... but at least I have SOME idea where to keep my stuff for the next few years :-)



  1. It is a tricky problem to solve. The stories of Jimmy Page trying to salvage video recordings of shows from various concerts is interesting. It was hard to find the right machines for some of them.

    I would hope that USB storage will be supported for a long time to come. You would only know a drive had failed when you tried to use it.

    Another option is self-hosted cloud storage. There are apps that let you back up over the internet to a drive on another PC. You could arrange to do this for each other with some of your musical collaborators. The limitation tends to be upload speed.

    I'm mainly using Google Drive for backup these days as it's just convenient and cheap. I do have some external drives I could use in combination with this. Need to keep one off-site, e.g. at work. Multi-terabyte drives are relatively cheap now. I need to automate things as much as possible to make life easier.

    What about version control? Do you need to keep multiple versions of a project? Does the DAW provide this? I know you also mentioned keeping stem files so you don't rely on having the original app.

  2. Well, I was more talking about archiving sessions over a long time, AFTER the project is done. A self-hosted solution would not be much use as one STILL has normal drives to cope with. All that is perfectly suitable for daily, and short term backups though.
    Versioning is included in most DAWs ... plus I like to add my own 'Save As...' versioning, as my band members will tell you :-) Has worked very well till now, but again ..... thats just till the project is finished.

    'Stem files', consolidation, 'flat-multifiles' etc .... are all issues in the HOW a session is archived. I just wanted to deal with the 'where' first