I’m planning on setting up a nas/home server (primarily storage with some jellyfin and nextcloud and such mixed in) and since it is primarily for data storage I’d like to follow the data preservation rules of 3-2-1 backups. 3 copies on 2 mediums with 1 offsite - well actually I’m more trying to go for a 2-1 with 2 copies and one offsite, but that’s besides the point. Now I’m wondering how to do the offsite backup properly.

My main goal would be to have an automatic system that does full system backups at a reasonable rate (I assume daily would be a bit much considering it’s gonna be a few TB worth of HDDs which aren’t exactly fast, but maybe weekly?) and then have 2-3 of those backups offsite at once as a sort of version control, if possible.

This has two components, the local upload system and the offsite storage provider. First the local system:

What is good software to encrypt the data before/while it’s uploaded?

While I’d preferably upload the data to a provider I trust, accidents happen, and since they don’t need to access the data, I’d prefer them not being able to, maliciously or not, so what is a good way to encrypt the data before it leaves my system?

What is a good way to upload the data?

After it has been encrypted, it needs to be sent. Is there any good software that can upload backups automatically on regular intervals? Maybe something that also handles the encryption part on the way?

Then there’s the offsite storage provider. Personally I’d appreciate as many suggestions as possible, as there is of course no one size fits all, so if you’ve got good experiences with any, please do send their names. I’m basically just looking for network attached drives. I send my data to them, I leave it there and trust it stays there, and in case too many drives in my system fail for RAID-Z to handle, so 2, I’d like to be able to get the data off there after I’ve replaced my drives. That’s all I really need from them.

For reference, this is gonna be my first NAS/Server/Anything of this sort. I realize it’s mostly a regular computer and am familiar enough with Linux, so I can handle that basic stuff, but for the things you wouldn’t do with a normal computer I am quite unfamiliar, so if any questions here seem dumb, I apologize. Thank you in advance for any information!

  • @7rokhym@lemmy.ca
    link
    fedilink
    English
    05 months ago

    I use Linux, so encryption is easy with LUKS, and Free File Sync to drives that rotate to a safety deposit box at the bank for catastrophic event, such as a house fire. Usually anything from the last few months are still on my mobile devices.

  • fmstrat
    link
    fedilink
    English
    05 months ago

    If you use ZFS this becomes easy, because you can do incremental backups at the block level.

    I have my home lab server and do snapshots and sends to a server at my fathers house. Then I also have an external drive that I snapshot to as well.

  • @thecoffeehobbit@sopuli.xyz
    link
    fedilink
    English
    05 months ago

    I have an external storage unit a couple kilometers away and two 8TB hard drives with luks+btrfs. One of them is always in the box and after taking backups, when I feel like it, I detach the drive and bike to the box to switch. I’m currently researching btrbk for updating the backup drive on my pc automatically, it’s pretty manual atm. For most scenarios the automatic btrfs snapshots on my main disks are going to be enough anyway.

  • @glizzyguzzler@lemmy.blahaj.zone
    link
    fedilink
    English
    0
    edit-2
    5 months ago

    I got my parents to get a NAS box, stuck it in their basement. They need to back up their stuff anyway. I put in 2 18 TB drives (mirrored BTRFS raid1) from server part deals (peeps have said that site has jacked their prices, look for alts). They only need like 4 TB at most. I made a backup samba share for myself. It’s the cheapest symbology box possible, their software to make a samba share with a quota.

    I then set up a wireguard connection on an RPi, taped that to the NAS, and wireguard to the local network with a batch script. Mount the samba share and then use restic to back up my data. It works great. Restic is encrypted, I don’t have to pay for storage monthly, their electricity is cheap af, they have backups, I keep tabs on it, everyone wins.

    Next step is to go the opposite way for them, but no rush on that goal, I don’t think their basement would get totaled in a fire and I don’t think their house (other than the basement) would get totaled in a flood.

    If you don’t have a friend or relative to do a box-at-their-house (peeps might be enticed with reciprocal backups), restic still fits the bill. Destination is encrypted, has simple commands to check data for validity.

    Rclone crypt is not good enough. Too many issues (path length limits, password “obscured” but otherwise there, file structure preserved even if names are encrypted). On a VPS I use rclone to be a pass-through for restic to backup a small amount of data to a goog drive. Works great. Just don’t fuck with the rclone crypt for major stuff.

    Lastly I do use rclone crypt to upload a copy of the restic binary to the destination, as the crypt means the binary can’t be fucked with and the binary there means that is all you need to recover the data (in addition to the restic password you stored safely!).

      • @huquad@lemmy.ml
        link
        fedilink
        English
        0
        edit-2
        5 months ago

        My most critical data is only ~2-3TB, including backups of all my documents and family photos, so I have a 4TB ssd attached which the pi also boots from. I have ~40TB of other Linux isos that have 2-drive redundancy, but no backups. If I lose those, i can always redownload.

      • @thejml@lemm.ee
        link
        fedilink
        English
        05 months ago

        Have it sync the backup files from the -2- part. You can then copy them out of the syncthing folder to a local one with a cron to rotate them. That way you get the sync offsite and you can keep them out of the rotation as long as you want.

      • @huquad@lemmy.ml
        link
        fedilink
        English
        05 months ago

        Agreed. I have it configured on a delay and with multiple file versions. I also have another pi running rsnapshot (rsync tool).

      • @Appoxo@lemmy.dbzer0.com
        link
        fedilink
        English
        05 months ago

        In theory you could setup a cron with a docker compose to fire up a container, sync and once all endpoint jobs are synced to shut down.
        As it seemingly has an API it should be possible.

    • @dave@lemmy.wtf
      link
      fedilink
      English
      05 months ago

      using a meshVPN like tailscale or netbird would another option as well. it would allow you to use proper backup software like restic or whatever, and with tailscale on both devices, it would allow restic to be able to find the pi device even if the other person moved to a new house. (although a pi with ethernet would be preferable so all they have to do is plug it in to their new network and everything would be good. if it was a pi zero then someone would have to update the wifi password)

      • @huquad@lemmy.ml
        link
        fedilink
        English
        05 months ago

        Funny you mention it. This is exactly what I do. Don’t use the relay servers for syncthing, just my tailnet for device to device networking.

  • TrumpetX
    link
    fedilink
    English
    05 months ago

    Look into storj and tardigrade. It’s a crypto thing, but don’t get scared. You back up to S3 compatible endpoints and it’s super cheap (and pay with USD credit card)

  • drkt
    link
    fedilink
    English
    05 months ago

    I rsync a copy of it to a friends house every night. It’s straight forward, simple and free.

    • @qjkxbmwvz@startrek.website
      link
      fedilink
      English
      05 months ago

      Same — rsync to a pi 3 with a (single) ZFS drive at family’s house. Retain some daily/weekly/monthly snapshots.

      I have a (free) VPS with static IPv4 which is how I connect everything.

      Both the VPS and the remote site have limited network speed (I think 50Mbps for VPS), so the initial sync was done sneakernet (well…“airplane net”). Nightly rsync is no problem bandwidth-wise, and is mostly just any new videos I’ve uploaded to my local Immich instance.

  • Matt The Horwood
    link
    fedilink
    English
    05 months ago

    There’s some really good options in this thread, just remember that whatever you pick. Unless you test your backups, they are as good as not existing.

    • @Showroom7561@lemmy.ca
      link
      fedilink
      English
      05 months ago

      How does one realistically test their backups, if they are doing the 3-2-1 backup plan?

      I validate (or whatever the term used is) my backups, once a month, and trust that it means something 😰

      • @Appoxo@lemmy.dbzer0.com
        link
        fedilink
        English
        05 months ago

        Deploy the backup (or some part of it) to a test system. If it can boot or you can get the files back, they work.

      • Matt The Horwood
        link
        fedilink
        English
        05 months ago

        Untill you test a backup it’s not complete, how you test it is up to you.

        If you upload to a remote location, pull it down and unpack it. Check that you can open import files, if you can’t open it then the backup is not worth the dick space

    • dave@hal9000
      link
      fedilink
      English
      05 months ago

      Is there some good automated way of doing that? What would it look like, something that compares hashes?

      • Matt The Horwood
        link
        fedilink
        English
        05 months ago

        That very much depends on your backup of choice, that’s also the point. How do you recover your backup?

        Start with a manual recover a backup and unpack it, check import files open. Write down all the steps you did, how do you automate them.

      • @sugar_in_your_tea@sh.itjust.works
        link
        fedilink
        English
        0
        edit-2
        5 months ago

        I don’t trust automation for restoring from backup, so I keep the restoration process extremely simple:

        1. automate recreating services - have my podman files in a repository
        2. manually download and extract data to a standard location
        3. restart everything and verify that each service works properly

        Do that once/year in a VM or something and you should be good. If things are simple enough, it shouldn’t take long (well under an hour).

  • @LandedGentry@lemmy.zip
    link
    fedilink
    English
    05 months ago

    Cloud is kind of the default these days but given you’re on this community, I’m guessing you want to keep third parties out of it.

    Traditionally, at least in the video editing world, we would keep LTO or some other format offsite and pay for housing it or if you have multiple locations available to you just have those drives shipped back-and-forth as they are updated at regular intervals.

    I don’t know what you really have access to or what you’re willing to compromise on so it’s kind of hard to answer the question to be honest. Lots of ways to do it

  • @SirMaple__@lemmy.ca
    link
    fedilink
    English
    0
    edit-2
    5 months ago

    I use Proxmox PBS for all my backups. Datastore is on my file server at home. I sync the datastore daily to a little NAS at a family members house and to a super cheap storage VPS on the other side of the country. I also do a manual sync to an external drive that keep offline at home.

    Any super important documents such as tax records, health related files, backup of the data volume from vaultwarden, or anything related to wills & estates get backed up as well to 2 USB thumb drives that are LUKS encrypted. I keep 1 in my go bag and another is hidden somewhere… Thumb drives get updated once a month, or sooner if anything major changes.

  • d00phy
    link
    fedilink
    English
    05 months ago

    My dad and I each have Synology NAS. We do a hyper sync backup from one to the other. I back up to his and vice versa. I also use syncthing to backup my plex media so he can mount it locally on his plex server.

  • @hperrin@lemmy.ca
    link
    fedilink
    English
    05 months ago

    I just rsync it once in a while to a home server running my dad’s house. I want it done manually in a “pull” direction rather than a “push” in case I ever get hit with ransomware.

  • @bandwidthcrisis@lemmy.world
    link
    fedilink
    English
    05 months ago

    I use rsync.net

    It’s not the lowest price, but I like the flexibility of access.

    For instance, I was able to run rclone on their servers to do a direct copy from OneDrive to rsync.net, 400Gb without having to go through my connection.

    I can mount backups with sshfs if I want to, including the daily zfs snapshots.