I’ve been a Linux user for many, many years. Going all the way back to Red Hat 5.2, which I picked up to install on an ancient Packard Bell 486 in the late 90s. Since then there’s always been at least one Linux machine in my dorm, apartment or house somewhere. At various times I’ve even run it for my desktop OS, although these days I use macOS for that.
For much of that time, Linux was the choice of hackers, but was definitely not a choice for everyday users and required a significant amount of technical knowledge to run. That’s not true so much anymore, but growing in that environment I learned a lot about how computers and operating systems work.
But every now and then, Linux does something that positively delights me.
I have a large fileserver in my closet that runs Plex digital media library among other things. This machine has an SSD for it’s primary drive, and a 5-disk Linux software RAID array. Because I am a part hoarder, while cleaning out my parts closet one day, I realized I had all the parts to build a near duplicate of this machine.
So I assembled the backup machine. On this ancient Athlon motherboard, the SATA
ports weren’t labeled, so I just plugged the drives in what I thought was the
right order, with /dev/sda
being the OS drive, and /dev/sd(b-f)
being the
RAID drives. When installing Linux on it (Ubuntu 16.04), it detected them in a
seemingly random order, so the OS drive was /dev/sdf
and the rest of the
drives were the array. Guess I got them reversed. Meh, whatever, I’ll just
configure it that way.
So I took it to work to allow it to be an offsite backup in case tornadoes destroy my house. I set up rsync to replicate nightly between the two, and everything was good!
… except that ancient motherboard had some issues. Anytime it lost power for more than a few minutes, it would throw CMOS Checksum Errors on the BIOS. I figured this was a bad CMOS battery, so I replaced the battery, but no dice. And because this board is 10 years old, good luck getting support on it.
So over Black Friday, I scored a good deal on a new motherboard. Added in some memory and a power supply that doesn’t sound like a jet engine (how did I deal with this in 2006?) I had a totally new machine. I intended to just use the existing hard drives with the new machine.
So I took the old motherboard and power supply out, leaving just the drives, and installed the new one. On the newer motherboard, the SATA ports were actually marked, so I installed the SSD on port 0, then the drives in top-top-bottom order in the remaining ports.
I was fully prepared to reinstall, but I decided, what the heck, let’s boot this thing up and see what happens. So I turned it on.
And it booted Linux. With almost no problems. All the way to the login screen.
And when I logged in, even the RAID array was there, in a clean state,
completely ready. Literally the only thing that didn’t work was the network
card, and once I ran dpkg-reconfigure
on the linux-image and fixed the
/etc/network/interfaces
even the network card came up.
Mind = blown. I literally rebuilt this entire machine, and it came back up like nothing happened.
The disks coming up was the coolest part of all. What was /dev/sdf
on the old
machine was now /dev/sda
, but both the system and the RAID array were aware of
this change. And, after a bit of research, the reason I concluded was because
both don’t use node numbers anymore, but UUIDs or another type of ID. A quick
glace at /etc/fstab
will show something like this:
# / was on /dev/sdf3 during installation
UUID=baa30826-d0dd-49e0-b3eb-21751accc1bf / ext4 errors=remount-ro 0 1
# /boot was on /dev/sdf1 during installation
UUID=078a7b64-0b84-4747-a07f-7bb74470385d /boot ext4 defaults 0 2
# /mnt/storage was on /dev/md0 during installation
UUID=a988ff4c-1489-44d2-ac4d-bd24cdc70829 /mnt/storage ext4 defaults 0 2
# swap was on /dev/sdf2 during installation
UUID=e22b8588-b9fb-422f-a479-d79e3ae50cba none swap sw 0 0
And some research seems to show that Linux mdraid uses metadata and not node numbers to assemble the array, so you can reassemble it in any order. Brilliant.