Update 2/6/2022: This blog post was featured in a video by Linus Tech Tips. If you want to watch him explain this stuff (and why it's a bad idea) instead:
Recently I posted a tweet that made the rounds on various social media like Reddit, Discord, and probably Hacker News:
Except it wasn't really the tweet, it was actually a screenshot of the thread and the first reply which showed the disk mounted, but not me actually activating the swap. Kinda annoying. But whatever.
Lots of people were wondering how it worked. Had I really been able to actually download more RAM from my Google Drive? And I'm here to tell you today that, yes, the rumors are true. I've been able to break the barriers of what's possible with computing. When one jests "just download more RAM," no longer will it be a joke, but an actual recommendation. I have figured out how to (kinda-sorta) download more RAM. And you can too. Here's how.
Your computer doesn't have unlimited memory. Yet, when applications on your machine exceed memory usage, what is there to do? Well, on most systems there is what's called "swap space" (also sometimes called a pagefile), which is a region on your hard drive dedicated to acting as what is essentially more RAM if your physical RAM fills up.
We are going to use this to our advantage and make Google Drive act as our hard drive in this scenario.
The reason why computers don't just do this all the time is because RAM needs to be really fast, and your hard drive is just, well, not. Even with an incredibly fast SSD, the hard drive speed pale in comparison to good-quality RAM.
As you might imagine, Google Drive is a lot slower than your RAM. But who cares; it gives you more time to reflect as you load your Blender file or train your ML model.
Step 1. Obtain Rclone
The software we are going to use to mount Google Drive is called
rclone. Aside from assisting in the bastardization of what scientists thought computers were capable of, it has many other uses. But that is not what we are focusing on today.
You can download rclone here. Or using your package manager or whatever.
Once you have it, you will want to configure Google Drive (or another backend of your choice) to be used with it.
Step 2. Mount Google Drive
First, create a folder called
swap in your Drive. We will use this as our cursed ground so we can burn down this affront to god all at once when we are done.
Now make a directory wherever you want to mount this abomination.
Once you've done that, you must run this cursed command (replacing
gdrive with whatever you named yours):
rclone mount gdrive:swap /your/mount/path --vfs-cache-mode full --allow-other --uid 0 --gid 0 --file-perms 0600
You might be noticing something at this point: "
vfs-cache-mode? what's that?"
Yes, reader, I have been lying to you this entire time, but only kind of. The swap space we are mounting is actually VFS-cached. So you're actually writing to your own disk. But more on that later.
Step 3. Make the Swap File
Now that we have Google Drive mounted, we can make the file we are going to enable the swap space on. Inside the mounted directory, run this command:
dd if=/dev/null bs=1 seek=10T of=swap
This will make a file called
swap in the directory. It will be ten terabytes big.
Now we also need to mount this file as a loop device because rclone doesn't support the operations necessary for true file swapping:
sudo losetup -f /your/mount/path/swap
Now we need to actually use
mkswap to set up the file structure necessary for swap:
sudo mkswap /dev/loop0
Make sure the correct loop device is selected. I am not responsible for accidentally turning your backup disk images into swap space.
Step 4. Enable Swap
The moment of truth.
sudo swapon /dev/loop0
If it works, great! You have just committed a crime against computers. If it doesn't, also great. Because this is an awful idea and should have never formed itself in my puny brain.
Stop Lying To Me
Ok, fine. It's not really downloading more RAM, for a couple of reasons:
- We are using vfs caching, so it is first caching to disk
- Google Drive does not support random reads/writes, so it can't actually work as swap space
- If rclone ever swapped to Google Drive, it would probably run into an infinite loop
And probably many more. But this was fun to do.