Way way back in the day my friend’s dad was quite familiar with how computers worked. He showed me how to open his beige case up and point out the individual components. I had no idea what he was talking about. But two things stood out to me, and since he said it I never forgot those words: One, if you can get more space, take it. And two, you will always run out. Stay organized and back it all up.
Over time I was introduced to a variety of computer systems and their operating systems. From Atari to Gateway, from Apples to Dells, desktops, laptops, netbooks, powerbooks, iPads, iMacs, .. and in 2017 I was able to do finally upgrade my old 24″ iMac from 2009 with a 27″ iMac from 2017.
Not only did this upgrade come with a much improved Intel CPU, and other faster internals. The screen went from an old fashioned 24″ to a 27″ retina 5k, and the internal drive from 640gb slow spinnable parts hard disk to a new Nvme 1TB SSD that’s significantly faster. Of course, the ports on the back went from USB2 to USB-C/USB3, which really helps as well. And the latter, that’s what this blog article is basically about.
Because from way way back in the day all the way up to this new SSD drive, I’ve had many drives. And because I’ve been quite picky with the ones I bought: They actually lasted! On top of that, my policy on how to deal with data apparently has helped as well. Which is: Each time I need a new drive, try to either buy one that’s dramatically faster, or if it’s for storage, then dramatically bigger. And get the data off of the oldest, smallest, slowest drive and then destroy that one (after it has served it’s purpose of being an ‘extra’ backup drive for a while. Any drive of course that shows any sign of breaking down is getting destroyed without remorse.
Okay, that brings us to today. The new iMac is here, and I’ve familiarized myself with it. And in the last six months I’ve been saving up to buy a few SSD drives (on the cheap) and a few usb-c and usb-3 cables (to sata3 adapter) to help me quickly connect these old(er) drives to the system. Excellent! It really works well, and I am very happy about that.
One extra thing that I had to do is split the dozen of drives in three piles: These are big and fast, these are new and faster, and these one or two are just terrible and have bad sectors and are falling apart.
The last pile I went through, got the data off of them that I wanted to keep, checked the data’s integrity and backed it all up. Then those drives were destroyed. They were too small to be used for backups, and too unreliable to be used for projects. To name a few things.
The pile of drives that were new and fast I’ve tested. The best performing ones I’ve put on the usb-c connection, and the bigger and a bit slower ones I’ve put on USB-3. The usb-c connected drives I basically consider internal secondary drives and I don’t remove those. They are quiet, all SSD as well, and I consider them part of the system. The other ones on USB-3 I consider media. This also means I can unplug them if needed, or at least unmount them. Sidenote: The usb-3 drives are pretty quiet, so it’s fine to have them on the whole time. Unlike the 4bay that I am mentioning next.
And that left me with 1 free usb-3 port, and a pile of drives that are older, slower, and of different sizes. And they certainly do not need to be connected all the time, or mounted all the time. In fact, they might contain data I don’t wish to have linked to a system that’s always on the Internet. My solution was easy: A four drive bay external usb-3 enclosure. And nothing fancy, no need for hardware RAID, etc. I really just wanted to put those four drives in there and have them mount, so I could access them. And it worked. It’s not too expensive (around 90 euro), and all my old 3.5″ drives mounted just fine. They sure are noisy though, geesh. But that’s okay. It’s a solution to a problem, and I only incidentally need access to those drives.
Assume everything is connected at the same time. All the ports in the back are occupied by external hard drives and SSD drives. What do we have?
Well, an internal 1TB drive, and on the usb-c ports I have both SSD, they’re 240gb and 525gb (my first time machine). Then the 3 usb-c ports have my LaCIE 500gb drive, my extra backup drive of 320gb that’s basically a clone of the LaCIE drive. And a 2 TB media drive that’s being used for projects, clips, photos, and what not. Yep, that’s already the first 5 TB of data.
The last usb-3 port has the four bay enclosure with inside it three 1 TB drives, and a 750gb drive. There you have it, the last 3, making it a total of 8 TB on this system.
Performance note: The internal SSD is the fastest, obviously. The usb-c ports are 10gbit, giving the really fast external SSD their best performance possible at 6gbit max. The usb-3 ports are 5gbit, but have older hard disk drives, at 6gbit, but of course at 5gbit max. Not that they can reach those speeds, but I thought it was worth mentioning it.
Why do I have it like this? Because this is what I can afford. We’re talking 150 bucks in enclosures and cables, versus a hardware RAID nas on Thunderbolt 3 with 2TB ssd drives etc. Something that’s too expensive, obviously.
Fine! What kind of speeds are we talking about? Well, the internal drive has read speeds of around 2500, the external ssd around 500, and the older drives around 100. The point is 50% performance, and 50% access to my data. Before this setup on the old system, I had a couple of usb ports with only 2 external loud enclosures, on usb2. And it was a terrible experience to make backups, and to do anything with the data. Now I have every drive connected to the system, and I can tidy things up. And whatever I don’t use daily I can just unplug. And the drives that do need performance are performing really well. Not only that, if I somehow get a 2TB sata3 drive, I just unplug the 240 and there we go, done.
Anyway, let’s move on to the tidying up of all this data.
My desktop finally has all those drives mounted, so first thing I did was change all the passwords on the encrypted drives to something new (so it was no longer a guessing game anymore since I never write these passwords down). And I made sure every drive that wasn’t enjoying full disk encryption is certainly encrypted now. At the same time I was also renaming them, giving them some tags, and simply making sure I know what’s on which drive predominantly.
I wrote down what type of data I might have, media, documents, music, you know, the works. And I’ve decided where to store what, how, and why. And just took it from there. No more ‘organize this folder’ directories spread all over these drives, but a central point where I can dump that and go through it. No more img_41.jpg files all over the place, but moved to a central point, organized by year and type (personal, private,.. etc) and if it was part of a project, properly organized and moved to the projects. All that sort of stuff.
But you can do that for days, weeks, probably years. Because, eight terabyte really isn’t a joke. It’s years of data, from 1995 onwards, even if it are just 3 files. Or a backup .zip from 2006’s old powerbook. A server .tarball.gz dump with databases and web server files, that’s encrypted and needs to be moved to the right place (and maybe there are 17 backups of it that need to be figured out). Not to mention checking file integrity, corruption (and avoiding it). While dealing with older and newer file system types.
At the same time the current system is trying to index all of this data, and keeping track of it. As well as backing it up on Time Machine, or at the least, I have to also back it up manually because I can’t depend on just that automated backup system from Apple.
I know, it doesn’t have to be finished tonight. And throughout the years I’ve attempted various times to address this problem. Now with this current solution of a faster system, more i/o with faster transfer speeds, and better ways of handling bigger data or massive amount of .txt files, etc. It’s a lot easier. And with all these drives mounted, there is overview as well. It’s no longer on a per disk basis.
That’s actually the other reason I write this blog article. To kind of show the importance of knowing what data you have, and to let you think about storing it, backing it up, organizing it. Letting you think along with me if you might have 3000+ pictures that need to be deleted, or.. moved to a project folder. What’s public? Personal or private?
Oh, let’s talk about that. Since a year or so I’ve moved more towards a system where I consider public data on my system stuff that I can just grab and share on social media, with friends or strangers. It’s already been vetted by me basically. Then there’s personal data, maybe that’s not really public, like a picture of my family dinner. But I can still share that with certain people. Like my family members or close friends. But maybe I would have to think twice about sharing it on the Internet or whatever. And there is private data. That’s just for me, my eyes only. By default this is where everything goes. Unvetted, so to speak. These can be screenshots of things I buy online, to personal conversations that are backed up (like an email thread or staff forum discussions) – things that aren’t meant to be shared with others.
This is a system that works pretty well. Everything that goes onto the computer one way or the other is by default private, until reviewed and considered public or personal. An example: You and I have a chat session over lunch about your new website. I give some pointers, we draw a couple of things on the iPad, and we take examples of the web for inspiration. This ‘concept’ data goes into the projects section, is considered private. There’s no big secret here, but it’s obviously not meant to be shared online. It’s your site in development, that’s nobody’s business. A project can always have an _export/ folder with _public/ content that eventually made it to social media after a private chat: “let’s post a sneak peak on instagram? sure!”.
Okay, so I have years of data, spread over various systems, loads of duplicates, loads of backups, and some of it is private, some of it is public. It needs to be tidied up. Now I can. At least, I can make a start, and I can do it a lot better.
And who knows, at some point in the future all these drives get replaced with a 10 TB external drive, giving me all those external ports back again. But at least I know what I will be putting on it. Oh, and if you want to help me out get to that point: Feel free to give me a personal gift of some money to save up for it, or perhaps the drive in question. I am always open to improving my tech stuff.
Back to tidying up, I’ve distracted myself long enough writing this blog article. And the backup has completed transferring between a couple of drives. It’s time for me figure out what to do with this archive of backups before I move on to the next thing.
Oh, if you have some computers, some hard drive, and no idea what to do. Hopefully this post can help you get started. If not.. contact me. I am sure we can talk about your personal situation and I’d love to give you some personal advice on what hardware to get (if any) or on how to get your data organized. And please, on macOS, just right click that external drive, select ‘encrypt’ and give it a password you don’t forget (and don’t have to share with others). It’s the first step to show that you actually care about your personal data.