In the year 2021 there are a lot of things that you just take for granted. Remember when you used to have to use jumpers to set things on your computer? Or worrying about IRQ conflicts? Or whether you could get the the drivers you needed to work?
These are all parts of the “bad old days” of computers that I don’t miss very much. These days if I plug things into my computer - any of them - I expect them to “just work.” And very often, surprisingly, this is the case. Especially common, well supported things like network cards.
So it is notable when I encounter something where that isn’t the case. But first, let’s back up a little bit.
How I Got Here
I have been using pfSense now for many years, since I migrated from DD-WRT. And to say that I have been pleased with it would be an understatement. It’s astounding that I can get professional-grade router support for free.
But in January, I woke up one morning to a broken network. I could not ping even the server sitting in the closet next to me. And after a round of troubleshooting, in frustration, I unplugged the router from the network. And everything started working again. Clearly, this meant the router - a mini ITX computer in a small form factor case, was to blame. Likely, the network adapter had failed.
But, the board was a dual-NIC motherboard, and because of the small form factor case, I could not simply slam a new NIC in it. So, off to build a new router I went.
But common equipment like network cards are just not something you think about needing to check the hardware compatibility lists for. They Just Work™. So I merrily ordered a new motherboard and a network card and built a new router, though this time I built it in a case so if this happened again could just slam a new NIC in it and move on with my life.
Then The Problems Start
The motherboard, an Asus PRIME H410M-E, had one built-in network card. I ordered another. Both ended up being based on the RealTek 8168 chipset. So I installed pfSense and racked the new machine and all was good…
… except it wasn’t. Every so often the machine would just totally die. Like stop responding at all and you had to hard-cycle it. This was totally out of character for pfSense, whose uptime on my old machine was often measured in months and only interrupted by upgrades. pfSense was the definition of stability.
The first time it happened I was curious. The second time it happened I was annoyed. All I could gleam from the logs was these cryptic line:
kernel: arpresolve: can't allocate llinfo for x.x.x.x on re1
kernel: re1: watchdog timeout
kernel: re1: link state changed to DOWN
kernel: re1: link state changed to UP
The frustrating part about this was how random it was. It would be up for weeks at a time, then suddenly die. Usually at the worst possible time, like when my daughter was trying to do her daily virtual meeting. Sometimes it would come back after a few minutes. Sometimes nothing but a hard reboot would fix it.
Tracing The Cause
My first thought was that this might be a problem with my cable modem, as re1 was the WAN port. But it was just too coincidental to be the cable modem. I changed the cable - I have seen cables go bad before - but that wasn’t it either. And the messages are so frustratingly vague that there could be a hundred different causes.
After one particularly frustrating day where I had to reset the machine multiple times, I was ready to take the machine out in the back yard and go Office Space on it. But, finally I decided to start thinking. What if there is a driver issue here?
So I started going through every piece of hardware attached to the pfsense machine and checking Google for issue reports. And when I reached the Realtek network cards, I hit jackpot.
As it turns out, a stock pfSense installation emphatically does not like Realtek network cards. And this is because the default Realtek drivers in FreeBSD (which pfSense is based on) are old and have many issues. As it turns out, this is exactly what was reported in this forum post.
Solving The Problem
So, finally I had a solution to why this machine was causing me such frustration. And knowing that, there are a couple of different solutions.
First, I could buy a new NIC. This may end up being the best possible solution. But I need a solution now, and getting a network card locally is basically impossible. I would have to order it to be delivered.
Second, from the address above, I could install updated Realtek drivers that are better than the default ones. This is what I ended up doing.
-
Upgrade to pfSense 2.5.0 or newer, if you haven’t already. You can do this on older versions, but it is far easier to do on 2.5.0+ because it’s packaged.
-
Shell to your pfSense machine and run
pkg install realtek-re-kmod. -
Follow the messages on the screen.
-
Reboot
That’s it. Working Realtek network cards. Now, it’s only been a few days but I haven’t had a network issue since I upgraded the Realtek drivers.
But Why?
This whole experience has left me with questions. First, why did I never encounter this before? Well, my previous pfSense installation was on an Intel D2500CCE motherboard, which included (you might have guessed) Intel network chips. Those are super supported by FreeBSD and pfSense and work out of the box with no issues.
But the bigger question is “why would you intentionally ship broken software?” I mean, this is not some random off-the-wall component. It’s a network card. And, this issue has been around long enough that the community has made a package for it, and the package is in the ports tree! But FreeBSD and pfSense still, by default will use the old and busted drivers and leave users frustrated.
As far as I can tell there aren’t even any licensing issues. Linux machines, which are under a much more restrictive license, have no problems with Realtek network cards. Several of my Linux machines have them, in fact. It’s just seems that FreeBSD does not want to fix this problem for whatever reason.
And that just sucks. The whole point of computers is to serve the users, not the other way around. And making things this hostile and obtuse for something as simple as a network card does nothing to further that cause. Either fix drivers, or don’t ship them at all - don’t knowingly ship broken software. Fix the bugs, make the machines work, make the users happy.
But, for the time being, I am happy to finally have a working router.
