It's not rare that I find myself looking for something several years old, whether it be for nostalgia (like a Minecraft world from 2013) or for serious reasons (I implemented something in the past that I would really like to not have to implement again) – only to find out that past-me was awful at backing stuff up and it's lost forever. What a shame; I'll never see that house made of diamond blocks again.
Situations like that happen often enough that I swore to myself that I would never be like past-TJ, so I abandoned the backup-less lifestyle and created my own personal backup solution loosely based off of the tried-and-tested 3-2-1 backup strategy:
- 3 copies of your data…
- …on at least 2 different media types…
- …with at least 1 offsite copy
I'll share exactly how I achieved this, and how you can get a setup like this as well so you never lose a file ever again. In my situation, the backups are located on:
- The source machine (laptop, etc.)
- A NAS I set up in my apartment
- Backblaze/B2 (depending on the type of machine)
- Special Case: MacBook backs up to a portable SSD as well
Not all my machines back up to all the locations (based on the importance of data on each), so it ends up looking like this:
To break this mess down:
- Personal MacBook Pro backs up via Time Machine to my NAS and portable SSD, and to Backblaze via their app.
- Windows Laptop backs up to Backblaze via their app.
- Windows Desktop backs up to Backblaze via their app, and to my NAS via occasionally moving important files over.
- Files on the NAS (minus the Time Machine backup) are backed up to Backblaze via an rclone cron job.
Let's go over how to set each of these up.
Setting Up Your Backups
Skip to a section:
This is probably the easiest to set up and very cheap (starting at $6/mo) for cloud backups. The Backblaze app is ridiculously easy to set up, and is very set-and-forget. If you are on a Windows or macOS machine, you can grab the Backblaze app here. The instructions they provide are pretty good.
For Everything Else
If you are on a Linux machine or want to back up something else to Backblaze, they offer a service called B2, which is pretty much just AWS S3 or Google Cloud Storage, but it's hosted on Backblaze. Much like their personal backup solution, B2 is also very cheap. You can take a look at the pricing and sign up here.
For backing up to B2, I use a piece of software called rclone (which is incredibly useful for many things outside of backing up as well). Basically, it's like rsync but supports any source/destination imaginable. Want to copy files from your hard drive directly to Google Drive with a single command? What about from Dropbox to OneDrive? rclone can do that and more – it is very versatile (in fact, you could follow this guide substituting B2 with your cloud storage provider of choice if you wanted). What makes it especially useful for backups is the incremental sync support. This means that we won't be uploading every single thing during every backup session – only the stuff that has been modified or added. This saves bandwidth, processing power, and most importantly time. So let's get started.
- Grab rclone from your favorite package manager or via the project's website.
- Set up a Backblaze B2 account.
Once you're all set up, we need to create a bucket for our backup to live in. Head over to the B2 Buckets page and hit the "Create a Bucket" button. Name the bucket whatever you like (note that it must be unique across all of B2), but know that you can't change its name later. Make sure it's set to Private as well, unless you want everyone to be able to access your backups for some reason!
Once your bucket has been created, we need to make rclone aware of it. Enter
rclone config into your terminal of choice to get to the interactive rclone configuration tool. You will likely see this:
No remotes found - make a new one n) New remote q) Quit config n/q> n
n to create a new remote. You'll be prompted for a name. You can set this to whatever you like, but I recommend
b2. It will be used as a reference to this remote in rclone commands/configuration.
You'll then be prompted for the remote type. We want B2, so type
b2 and confirm.
Type of storage to configure. Choose a number from below, or type in your own value [snip] XX / Backblaze B2 \ "b2" [snip] Storage> b2
Next, you will be prompted for an Application Key ID. You can grab this value from the B2 App Keys page. On this page, you will find the master key – you can use this key but I highly discourage this. It has full access to your buckets and if it ever gets compromised you are likely to be SOL. To prevent this from ever being possible, we will create an application key specifically for rclone. This way you can easily track access to your B2 buckets and quickly revoke any keys in case of compromise.
Press the "Add a New Application Key" button. Here are my recommended values for the key:
- Name of Key:
rclone(Can be anything, just make it memorable)
- Allow access to Bucket(s): (Set this to the bucket you created earlier)
- Type of Access: Read and Write (Needed for rclone to work)
Once you create the key, you will be given the key ID and the key itself. Enter these when rclone asks for them.
Account ID or Application Key ID account> 123456789abc Application Key key> 0123456789abcdef0123456789abcdef0123456789
It will ask for an endpoint – you can leave it blank.
Endpoint for the service - leave blank normally. endpoint>
Do one more check to make sure everything is correct, then confirm the remote config.
Remote config -------------------- [b2] account = 123456789abc key = 0123456789abcdef0123456789abcdef0123456789 endpoint = -------------------- y) Yes this is OK e) Edit this remote d) Delete this remote y/e/d> y
You're done with config! To test that rclone can successfully access your bucket, use this command, replacing
b2 with the remote name you set earlier, and
bucket with the name of your bucket.
rclone ls b2:bucket
For more information on configuring B2 with rclone, check out their docs. There are tons of options that you may be interested in.
Now that rclone is aware of our B2 account, we can set up a simple cron job to sync stuff from any directory you want over to your bucket. For example, if you wanted to back up
/mnt/drive to your bucket
my-cool-backup hourly, you could add this to your crontab (with
# Note how we are not using `rclone sync` -- this will delete files in the destination! `rclone copy` only creates/modifies. 0 * * * * rclone copy /mnt/drive b2:my-cool-backup
Check out the rclone docs for all the possible options you can use to customize your backup: https://rclone.org/flags/ (there's a lot you can change!)
Something to also note is how B2 will keep all file versions by default, so you don't need to worry about losing a file you recently overwrote. You can change the file lifecycle rules on the Buckets page under "Lifecycle Settings". They are very customizable – you can choose a different lifecycle rule for each directory if you wanted.
Local NAS (via SMB)
If you have a Raspberry Pi and a hard drive, you have a NAS! Of course, you could use any hardware with a Linux distro on it, but a Raspberry Pi 4 is perfect for the job. We are going to install samba on our host machine, which will expose any directory we want under a SMB file share (and in our case, we will be setting up a Time Machine backup on it). Let's get started.
- Grab samba via apt (
sudo apt update && sudo apt install samba)
- Plug in your drive and mount it somewhere
Once samba's installed and your drive is mounted (we will be using the path
/mnt/drive for the rest of the post), we need to configure samba to point to it. Open up
/etc/samba/smb.conf with your favorite text editor (as root):
sudo $EDITOR /etc/samba/smb.conf
This is where we will define the shares we want samba to expose to the rest of our network. In my case, I have two separate shares – one to throw whatever I want on, and the other for the Time Machine backup. Both of these shares are on the same drive, but the Time Machine one is special and lives in a subdirectory on my NAS drive. If this is something you want, copy both configs, but if not you only need to copy the first one.
# "stuff" is the share name -- you can set it to whatever you want [stuff] path = /mnt/drive writeable = Yes create mask = 0777 directory mask = 0777 public = no # The special Time Machine share. It has additional config flags to tell it to be supported as a Time Machine volume, or else macOS won't want to backup to it. [timemachine] comment = Time Machine path = /mnt/drive/timemachine browseable = yes writeable = yes create mask = 0600 directory mask = 0700 spotlight = yes vfs objects = catia fruit streams_xattr fruit:aapl = yes fruit:time machine = yes
Once that's all done, we need to set up your user's samba password. In a terminal, run the
smbpasswd command. This will set the password you will use to authenticate with your shares. (You probably shouldn't be exposing this over the internet, so it doesn't need to be terribly secure.)
sudo systemctl enable smbd && sudo systemctl start smbd. This will enable the samba daemon at startup (and start it right now).
From a different machine on your network, see if you can access your new network share. On macOS, go to Finder > Go > Connect to Server... and type
smb://hostname.of.your.samba.server. If all is well, it should ask you for your username and password (use the one you set earlier), and then present you with a list of shares you can mount (if you are setting up a Time Machine share, you want to mount that share right now so we can use it later).
On Windows, go to Explorer > Computer (the ribbon at the top) > Map network drive. Choose a letter for your share, and enter
\\hostname.of.your.samba.server\sharename. Check "Connect using different credentials" if you need to. Sign in with your samba username and password, and it should be mounted like a normal Windows drive.
With this setup, you can back up anything important to your NAS and combine the above B2 setup to automatically back up anything on your NAS to B2!
So now let's set up Time Machine to back up to our share whenever it's on the same network. Once your Time Machine share is mounted, you can go to System Preferences > Time Machine, and add your share from there. After that, it should Just Work. It might take a long time for the initial backup, though, so be patient.
Testing Your Backups
Backups aren't any good if you don't know they work. That's why testing your backups (hopefully often) is a good practice. Since I don't know exactly how you set up your backups, this might look different for you. But generally, here is how to access your backups on the services I listed above:
If you are using the personal backup service (e.g. not B2), Backblaze provides a few ways of getting your files back. In the case you need to recover only a few files, you can download the files directly through their web UI by going to View/Restore Files. If your entire system made an oopsie (or you need to recover significantly larger files), you can request that Backblaze physically send you a flash drive or external hard drive with the data you need on it. These options are $99 and $189 respectively, but you will be refunded if you return the drive once you are done with it.
If you are using B2, there is unfortunately no option to have a drive physically sent to you, but you can still make a snapshot of and download the entire bucket. Head over to the Buckets list and grab the files you need there.