So I’ve been running a home media server with the arr suite on it, but it only has about 2 tb storage. The storage has been full for a couple months now which has rendered my server useless. Now I’ve bought a used nas with 32 tb on 7 disks. I’ve been advised to set it up in RAID. I’ve learned how to do that in OMV and it’s setting that up now.
It takes a while though, so that leaves me some room to think. Why do we do this? I understand that it gives me more unified storage, but I also want to run other things on this machine. Like k8s to deploy my own applications and play around with devops stuff. Or even just regular file storage and photobackup next to the films and series. I thonk I would like to keep those things seperate though, is that still possible with RAID? Or is it better to put 5 disks in RAID and keep the other two for other stuff? I think my question is, how do you use RAID “practically”? How do you store different things on there? Do you compartmentalize the disks again, or do you just make different folders?
RAID comes down to reducing the risk of any single disk failure making you lose data. But you can also approach that with backups.
There’s also the secondary benefit of avoiding downtime but that’s questionable in a self-hosted setup, unless you’ve come to depend on services that can’t continue when the server is offline. In my case that’s not true because email is not self-hosted, things like calendar and contacts can function offline on the phone, just don’t have server sync for the duration, and things like photo archive aren’t essential (important yes, but not “can’t function daily without them”-essential).
Speaking of which, I think the first thing you should do is categorize your data by degrees of gravity along these two criteria: (1) how over would my life be if I lost this data and (2) how essential it is to my daily function.
It’s important to realize that these two criteria overlap but aren’t the same, first of all. Secondly, I think you’ll find that data criticality in regard to your life is maybe not what you expected. Truly essential things will probably end up being smaller than you think, and that makes them easy to back-up, making RAID maybe not such a must-have.
IMO it’s more important to set up a 3-2-1 backup strategy and consider your methods for achieving that than to have a RAID setup. Some cheap and effective methods are: backups to “cold” HDDs (which you keep in the closet, bring out once a month for a backup, and put back); backups to optical disc (Blu Ray); backups to online storage.
Please be aware that backups tools and copy/sync tools are different beasts. Rsync or rclone for example is not backup tools, they’re fancy copy tools. A true backup tool (borg, restic) will give you integrity checks (independent on the supposedly-still-pristine-source), deduplication, incremental backups, encryption (essential for online storage), recovery, adaptive compression etc. They also work transparently on top of regular filesystems and/or SSH, which lets you use generic online storage for backup. For example you can sync a local borg repo to a free Dropbox account, or use borg over ssh with a Hetzner storage box.
Personally I stay away from complicated array setups. I use pairs of HDDs in RAID1 and that’s all the RAID I need. RAID5 is a data loss waiting to happen because the long and intensive rebuild puts too much stress on the rest of the array members in case of one disk loss, so basically RAID6 is a must, but that makes things even longer and more complicated. ZFS is a very complex beast, it needs a lot of RAM to function efficiently, and the misconfiguration scenarios and subtle failure scenarios are terrifying. It’s also not going to solve any problem that you can solve with backups, which you should prioritize anyway.
Another thing that you must realize and commit to, with any approach that uses disks, is that any disk can fail at any time. Any of the backup drives, any of the live drives. You MUST always have at least one known good spare on hand. And I mean literally in hand, because if you have to go shopping in today’s day and age that means weeks away. Always have the spare local.
Also, don’t think this doesn’t include SSD’s, particularly system disks. Losing the M.2 that the server runs off can hurt like a bitch. Losing any of the fancy SSD’s in your desktop/gaming computer/laptop, even more so. But people often overlook those and hope for the best, when same logic applies: backup, and have a spare in hand.