New Self-Hosting Adventures

In these first cooler weeks of autumn, I dedicated some of my spare time configuring a Raspberry Pi. I’ve owned one since 2017 or so and used it mainly as an ad blocker. Now, after moving into the new home, I wanted to host some services on a tiny personal server.

Resiliency Is Mandatory

Working in data protection, I know how easy it is to lose data; so, backups and redundancy were mandatory for this project. I considered buying a refurbished server, mostly for the RAID controllers of which they are provided, but the energy consumption is very high, and overall it would be excessive — more than anything, my girlfriend would hate it and she’d hide it somewhere. So, I did some research and found several solutions to implement RAID on a Raspberry Pi.

I admit I asked ChatGPT the pros and cons of several setup, and I’m happy with the answers. Eventually, the choice fell on a HAT adapter for two SATA drives. It does not support RAID, but I chose to use ZFS as filesystem: it surely uses more CPU than a RAID controller, but the consumption on a Pi 5 is irrelevant.

So, according to these instructions, this is the approach I’d recommend to install ZFS on Raspberry Pi OS:

echo 'deb http://deb.debian.org/debian bookworm-backports main contrib non-free non-free-firmware' > /etc/apt/sources.list.d/backports.list
$ sudo apt update
$ sudo apt install --no-install-recommends dkms
$ sudo apt install --no-install-recommends zfsutils-linux/bookworm-backports zfs-dkms/bookworm-backports
$ sudo modprobe zfs
$ sudo systemctl enable zfs-import-cache
$ sudo systemctl enable zfs-mount

Then, to create a ZFS mirror with the name zdata:

$ sudo zpool create -o ashift=12 -o autoexpand=on -O compression=lz4 -O recordsize=1M -O atime=off -O xattr=sa zdata mirror wwn-0x5000c500fab2c696 wwn-0x5000c500fab296a6

Note: the last two parameters are the names of the drives attached to the controller found with /dev/disk/by-id/.

About the storage, I chose two 3.5" 2TB 7200rpm disks — they should have a longer life span than 2.5" ones. One huge downside, though, is that these drives needed 12V input, while the Pi 5 wants 5V. Simplest solution: use a different power supply attached directly to the SATA HAT. Easy to say, but there was a further issue: I couldn’t find in any documentation the size of the 12V input barrel, so I had to wait for the part to arrive and then try a bunch of them (a very kind thank you to Layla, who provided the part!). Eventually, I found it after a couple of attempts.

A photo of several barrel adapters for 12V power supplies, each with a different diameter. — I think we need a new standard, as for USB-C

So I had the Pi 5, the HAT, two hard drives, and two power adapters. Ready to boot! About that… What OS to choose?
After pondering a bit, I stuck with Raspberry Pi OS Bookworm: since it’s based on Debian, I will probably find the best compatibility with any Linux software I want to install.

ZFS and Backups

As mentioned above, I immediately set up a ZFS mirror (a.k.a. a software RAID), and I was impressed by the swiftness with which the software manages its internal checks. I had never used it before, and it’s always fun to implement new things when they work well! I hope I’ll never need to check whether the error detection works flawlessly or whether it’s easy to recover a whole disk of the pool, but it should be, and it’s enough for now.¹

So, about the local redundancy, for a home server with not-so-critical data, it should be fine. Then, it was time to think about the total loss of the Raspberry. I needed a remote backup. Where then? With my VPS, I use an Hetzner Storage Box, so I opted for the same solution. Since 1TB is a lot of space, but optimization is better, I chose restic as a backup tool: it deduplicates and encrypts data, and it’s quite fast! Restoring files is surely slower than a simple copy and paste, but one just needs to learn a couple of commands.

Therefore, after scheduling² the scripts to back up the several applications (we’ll talk about them in a minute), I’m quite confident that the system is well-protected against failure. So… What is this system for?

Always DeGoogling

The main reason I bought all this stuff was to create a simple media server. First, I tried OpenMediaVault, but then I realized that Jellyfin was all I needed: fast to configure, simple, and efficient. I exclusively use it locally, so I didn’t set up a reverse proxy, and it works very well on all our devices — except for the Samsung smart TV, may the devil catch the creators of Tizen OS.

Then, I decided to try an alternative to Google Photos. I already wrote about Ente, but after the issues while importing Google Takeouts, I decided to try the other main player: Immich. Well… The problems were the same, but I understand that it’s not the developers’ fault. It’s Google that slightly compresses the photos, so the ones on the phone seem different to softwares than the ones uploaded in the cloud. So, when importing the Takeout first, and then the ones on the local device… it duplicates everything.

Luckily, I discovered that doing the opposite doesn’t cause the issue, and it works quite well (so it should do the same for Ente): first the phone, then the Takeout. It worked! After deleting a few duplicates easily found with the proper Immich utility, I was good to go: all my ~80GB of photos were imported! It took a few hours for the machine learning container to process all the photos and videos, but it did a great job recognizing the faces (this function always creeps me out) and organizing the “map” of the pictures.

Finally, I quickly set up Caddy as a reverse proxy (I’ve always used NGINX, but I wanted to try something else) and Authentik as local OAuth for 2FA, and I finally got the full package! To be honest, configuring Authentik was maybe the hardest part because I had no idea how it worked. The documentation is well-written, but the software is so huge that it’s not easy to just set up a TOTP as I wanted. Eventually I got it, and I felt like Hackerman.

Chat Control Mania

After writing another similar script to copy the Immich and Authentik backups and data to the cloud, I was ready for the latest big project: an XMPP server for personal use.

Those were the days when Chat Control was in vogue again on Mastodon and some YouTube channels. I thought this could be the occasion to test XMPP, which has always been a niche product from my point of view. I came across Snikket, which seemed to be the simplest solution. On Hacker News, a couple of months ago, I had read a very positive experience, but unfortunately I spent a day trying to solve an issue I could barely find on the internet. I could have asked the community, sure, but the comments on HN made me curious about Prosody.

It took me a while to understand the different modules I could (or should) activate, and a few more hours to understand the TURN and STUN server. It was a lot of information 😵‍💫, but eventually… now you can reach me also through a brand-new XMPP JID.

🎮 Nothing, but I download the S.T.A.L.K.E.R. trilogy on PS4

🎧 I just discovered a super power rock single from Norway

📖 An horror anthology that started with a bang!

In my experience, even with RAID controllers, it’s always very delicate to recover a configuration or rebuild the array when it’s not done automatically. ↩︎
Instead of the simple crontab, I recently started using the systemd timers, and they’re much handier because they write in the journal/log — managing the output is much easier! ↩︎

Reply by email ✉️